HTMA

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 6 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Quantifying Risk Through Modeling. Chapter 6 builds on the basics described in Chapter 4 (define the decision and data that will be needed) and Chapter 5 (determine what is known). Hubbard addresses the process of quantifying risk in two overarching themes. The first theme is the quantification of risk and the second is using the Monte Carlo analysis to model outcomes.

Risk is the possibility that a loss or some other bad thing will occur. The possibility that something equates to uncertainty. Risk is often expressed in qualitative terms such low, medium, high and the ever popular really high rather than in quantified terms. However, that qualitative approach generates ambiguity. Qualitative designations provide little measurement value outside of a sort of measurement placebo effect. Even though an absolute value for risk might not be knowable, risk can be quantified as a range of values. The qualification of risk is important, both in terms of understanding risk and in terms of usefulness for defining overall outcomes. Defining a range of risks makes understanding the amount of risk in any decision less ambiguous. Quantifying risk is also the foundation of further measurement needed for decision-making. Hubbard hammers home the point that measurement is done to inform some decision that is uncertain and has negative consequences if it turns out wrong.

The goal of Monte Carlo analysis is to provide input for better decision making under uncertainty. When you allow yourself to use ranges and probabilities, you really don’t really have to assume anything you don’t know for a fact (Chapter 5 showed us how to estimate based on what we know). All risks can be expressed by the range of uncertainty on the costs and benefits and the probabilities of events that may affect them. Turning a range of estimates into a set of predicted outcomes requires math. Monte Carlo analysis is a mathematical technique that uses the estimated uncertainty in decisions to furnish a range of possible outcomes and the probabilities they will occur.

Monte Carlo analysis can incorporate a wide range of scenarios and variables.  Hubbard points out that it is easy to get carried away with the detail of the model. Models should only be as sophisticated as needed to add value to a decision. Remember, as Gene Hughson of Form Follows Function says, “all models are wrong.”  Models are abstractions of real life, there is always detail you leave out, no matter how sophisticated the model becomes. Hubbard suggests that model users always ask whether a new more complex model is an improvement on any alternative model. Quantification provides a platform for making consistent choices in order to clearly state how risk-averse or risk tolerant any organization really is.

Hubbard closes this chapter by stating a risk paradox.

“If an organization uses quantitative risk analysis at all, it is usually for routine operation decisions.  The largest, most risky decisions get the least amount of risk analysis.”

The combination of estimation (Chapter 5), quantifying risk and Monte Carlo analysis may seem complex keeps many decision makers from using the technique, this is especially true in software development, hence the paradox.  For example, every software development estimation problem, whether Agile, lean or plan based, has a large degree of uncertainty embedded in the process and therefore, is a perfect candidate to use Monte Carlo analysis. However, very few estimator understand or use the technique. Learning Monte Carlo analysis (and using the one of the many tools to do the mathematical heavy lifting) or alternately hiring some to perform risk analysis are both paths to addressing adding quantitative data to decision making.  When making decisions under conditions of uncertainty,  Monte Carlo analysis is a necessity to do the math needed. 

Previous Installments in Re-read Saturday, How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

How To Measure Anything, Third Edition, Introduction

Chapter 1: The Challenge of Intangibles

Chapter 2: An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily

Chapter 3: The Illusions of Intangibles: Why Immeasurables Aren’t

Chapter 4: Clarifying the Measurement Problem

Chapter 5: Calibrated Estimates: How Much Do You Know Now?