HTMA

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 4 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Clarifying the Measurement Problem. In this chapter Hubbard focuses on two questions.  The first is getting the reader to answer what is the decision that measurement is supposed to support. The second is what is the definition of the thing being measured in terms of observable consequences.  These questions sound very basic; however, I found myself asking variations of those questions more than once recently when reviewing a relatively mature measurement program.  Answering these questions are often at the heart of defining the real mission of any measurement or measurement program and whether a measure will have value.

Chapter 4 begins the second section of the book titled Before you Measure, in which Hubbard begins a deeper dive into his measurement approach initially identified in Chapter One.  The framework is:

  1. Define the decision. This step includes defining not only the dilemma we are attempting to solve with measurement, but all of the relevant variables in unambiguous terms. Chapter 4 focuses on this step.
  2. Determine what you know. This step is about determining the amount of uncertainty in the decision and measures defined in step 1
  3. Compute the value of additional information. If the value of additional information is low then you are done (go to step 5).
  4. Measure where the information value is high. After collecting new data, repeat steps 2 and 3 until further measurement does not provide enough additional value.
  5. Make a decision; act upon it.

All measurement begins by defining the decision that will be made. The question you need to be ask and answer is what is the problem or dilemma for which a decision needs to be made. In order to truly answer the question of what decision will be made, you need to clearly articulate the specific action the measurement will inform in a clear and unambiguous fashion. Failure to correctly identify the purpose will lead to debates later when the ambiguities are exposed. For example, I recently listened to a debate on whether an organization’s productivity (defined as delivered functionality per staff month of effort) had increased. The debate had broken down into a fierce argument of what delivered functionality meant and whose effort would be included in the definition of a staff month. All of these ambiguities stemmed from a lack of finality on what decision the organization was trying to make with the measure, and therefore what needed to be measured.

Part of the definitional problem is a failure to understand the requirements needed to make a decision. Hubbard provides a set of criteria that need to be met in decision-making scenario:

  • A decision requires two or more realistic alternatives.
  • A decision has uncertainty.
  • The decision has a risk.  Risk is the potential and negative consequences if the wrong alternative is taken.
  • A decision has a decision maker.

Failure to meet any one of these criteria means you are not facing a decision-making scenario.  For example, if you were deciding whether to go out for lunch or not and there were no restaurants, food truck or hot dog carts, you would not really have a decision to make.  If there were one restaurant there again would be no uncertainty, therefore, no decision to be made.

Modeling a decision is a mechanism to lay bare any remaining ambiguity.  Any decision can be modeled.  The concept of weighted shortest job first (WSJF), a tool to prioritize (a form of decision making) is a tool often used to model which piece of work should be done first.  Model can be simple such as a cost-benefit analysis or complex such as a computer model or Monti-Carlo analysis. Hubbard suggests that every decision is modeled whether even if modeling is expressed by intuition.

Decisions require risk and uncertainty. As with many other measurement concepts, understanding what risk and uncertainty means is critical to being able to measure anything. The chapter concludes with a discussion of the definition of uncertainty and risk.

Uncertainty is the lack of complete certainty of an outcome of a decision. Uncertainty reflects that there exists more than one possible outcomes of a decision.

Measurement of uncertainty a set of probabilities assigned to the possible outcomes.  For example, there are two possibilities for the weather tomorrow, precipitation or no precipitation.  The measurement of certainty might be expressed as a 60% chance of rain (40% chance of no rain can be inferred).

Risk is the uncertainty that a loss or some other bad thing will occur.

Measurement of risk is a quantification of the set of possibilities that combines the probability of occurrence with the quantified impact of an outcome. For example, the risk of a decision to spend money on a picnic that would require an expenditure of thirty dollars on perishable food could be expressed as a 60% chance of rain tomorrow with a potential loss of $30 for the picnic lunch.

Clarifying the measurement problem requires defining what we mean.  Definition begins with unambiguously defining the decision to be made.  Once we know the decision that needs to be made we can define and measure uncertainty and risk for each of possible outcomes.

Previous Installments in Re-read Saturday, How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

How To Measure Anything, Third Edition, Introduction

HTMA, Chapter 1

HTMA, Chapter 2

HTMA, Chapter 3