
How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition
Chapter 4 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Clarifying the Measurement Problem. In this chapter Hubbard focuses on two questions. The first is getting the reader to answer what is the decision that measurement is supposed to support. The second is what is the definition of the thing being measured in terms of observable consequences. These questions sound very basic; however, I found myself asking variations of those questions more than once recently when reviewing a relatively mature measurement program. Answering these questions are often at the heart of defining the real mission of any measurement or measurement program and whether a measure will have value.
Chapter 4 begins the second section of the book titled Before you Measure, in which Hubbard begins a deeper dive into his measurement approach initially identified in Chapter One. The framework is:
- Define the decision. This step includes defining not only the dilemma we are attempting to solve with measurement, but all of the relevant variables in unambiguous terms. Chapter 4 focuses on this step.
- Determine what you know. This step is about determining the amount of uncertainty in the decision and measures defined in step 1
- Compute the value of additional information. If the value of additional information is low then you are done (go to step 5).
- Measure where the information value is high. After collecting new data, repeat steps 2 and 3 until further measurement does not provide enough additional value.
- Make a decision; act upon it.
All measurement begins by defining the decision that will be made. The question you need to be ask and answer is what is the problem or dilemma for which a decision needs to be made. In order to truly answer the question of what decision will be made, you need to clearly articulate the specific action the measurement will inform in a clear and unambiguous fashion. Failure to correctly identify the purpose will lead to debates later when the ambiguities are exposed. For example, I recently listened to a debate on whether an organization’s productivity (defined as delivered functionality per staff month of effort) had increased. The debate had broken down into a fierce argument of what delivered functionality meant and whose effort would be included in the definition of a staff month. All of these ambiguities stemmed from a lack of finality on what decision the organization was trying to make with the measure, and therefore what needed to be measured.
Part of the definitional problem is a failure to understand the requirements needed to make a decision. Hubbard provides a set of criteria that need to be met in decision-making scenario:
- A decision requires two or more realistic alternatives.
- A decision has uncertainty.
- The decision has a risk. Risk is the potential and negative consequences if the wrong alternative is taken.
- A decision has a decision maker.
Failure to meet any one of these criteria means you are not facing a decision-making scenario. For example, if you were deciding whether to go out for lunch or not and there were no restaurants, food truck or hot dog carts, you would not really have a decision to make. If there were one restaurant there again would be no uncertainty, therefore, no decision to be made.
Modeling a decision is a mechanism to lay bare any remaining ambiguity. Any decision can be modeled. The concept of weighted shortest job first (WSJF), a tool to prioritize (a form of decision making) is a tool often used to model which piece of work should be done first. Model can be simple such as a cost-benefit analysis or complex such as a computer model or Monti-Carlo analysis. Hubbard suggests that every decision is modeled whether even if modeling is expressed by intuition.
Decisions require risk and uncertainty. As with many other measurement concepts, understanding what risk and uncertainty means is critical to being able to measure anything. The chapter concludes with a discussion of the definition of uncertainty and risk.
Uncertainty is the lack of complete certainty of an outcome of a decision. Uncertainty reflects that there exists more than one possible outcomes of a decision.
Measurement of uncertainty a set of probabilities assigned to the possible outcomes. For example, there are two possibilities for the weather tomorrow, precipitation or no precipitation. The measurement of certainty might be expressed as a 60% chance of rain (40% chance of no rain can be inferred).
Risk is the uncertainty that a loss or some other bad thing will occur.
Measurement of risk is a quantification of the set of possibilities that combines the probability of occurrence with the quantified impact of an outcome. For example, the risk of a decision to spend money on a picnic that would require an expenditure of thirty dollars on perishable food could be expressed as a 60% chance of rain tomorrow with a potential loss of $30 for the picnic lunch.
Clarifying the measurement problem requires defining what we mean. Definition begins with unambiguously defining the decision to be made. Once we know the decision that needs to be made we can define and measure uncertainty and risk for each of possible outcomes.
Previous Installments in Re-read Saturday, How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition
How To Measure Anything, Third Edition, Introduction
January 10, 2016 at 6:02 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
January 10, 2016 at 6:06 pm
Thomas your summary of Chapter 4 is very thorough!
Did you know there is a companion HTMA workbook for the HTMA 3rd edition book we are re-reading?
One question from the workbook I missed is the concept of “false dichotomy” (see p. 75 in the HTMA book) when exploring a decision to measure.
Make sure the decision you are supporting through a measurement is not a false dichotomy. That is, not a feasible alternative. Typically, as Hubbard explains “Yes/no choice between two extremes” or as I think about it, a “Mom and apple pie” decision statement.
My example: suppose your decision is whether you should exercise or not. Drill down on that decision statement and define what type of exercise program you should engage in instead (e.g., gym workouts, running, bike riding, hiking) and then figure-out the measurement to support that decision.
Hubbard’s examples were (1) clean drinking water (book) and (2) worker safety (workbook).
March 31, 2016 at 11:00 pm
I glossed over the concept of a false dichotomy excellent point.
I think you have convinced me to buy the workbook.
January 16, 2016 at 11:56 pm
[…] in Business” Third Edition, is titled: Calibrated Estimates: How Much Do You Know Now? Chapter 4 described how to define the decision that needs to be made and the data that will be needed to make […]
January 23, 2016 at 11:57 pm
[…] Edition, is titled: Quantifying Risk Through Modeling. Chapter 6 builds on the basics described in Chapter 4 (define the decision and data that will be needed) and Chapter 5 (determine what is known). Hubbard […]
January 31, 2016 at 12:05 am
[…] Chapter 4: Clarifying the Measurement Problem […]
February 6, 2016 at 11:56 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
February 13, 2016 at 11:57 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
February 27, 2016 at 11:56 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
March 5, 2016 at 11:56 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
March 12, 2016 at 11:58 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
March 19, 2016 at 11:55 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
March 26, 2016 at 11:56 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
April 2, 2016 at 11:57 pm
[…] Chapter 4: Clarifying the Measurement Problem […]
March 9, 2017 at 11:55 pm
[…] Chapter 4: Clarifying the Measurement Problem […]