Chapter 6 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Quantifying Risk Through Modeling. Chapter 6 builds on the basics described in Chapter 4 (define the decision and data that will be needed) and Chapter 5 (determine what is known). Hubbard addresses the process of quantifying risk in two overarching themes. The first theme is the quantification of risk and the second is using the Monte Carlo analysis to model outcomes.
The goal of Monte Carlo analysis is to provide input for better decision making under uncertainty. When you allow yourself to use ranges and probabilities, you really don’t really have to assume anything you don’t know for a fact (Chapter 5 showed us how to estimate based on what we know). All risks can be expressed by the range of uncertainty on the costs and benefits and the probabilities of events that may affect them. Turning a range of estimates into a set of predicted outcomes requires math. Monte Carlo analysis is a mathematical technique that uses the estimated uncertainty in decisions to furnish a range of possible outcomes and the probabilities they will occur.
Monte Carlo analysis can incorporate a wide range of scenarios and variables. Hubbard points out that it is easy to get carried away with the detail of the model. Models should only be as sophisticated as needed to add value to a decision. Remember, as Gene Hughson of Form Follows Function says, “all models are wrong.” Models are abstractions of real life, there is always detail you leave out, no matter how sophisticated the model becomes. Hubbard suggests that model users always ask whether a new more complex model is an improvement on any alternative model. Quantification provides a platform for making consistent choices in order to clearly state how risk-averse or risk tolerant any organization really is.
Hubbard closes this chapter by stating a risk paradox.
“If an organization uses quantitative risk analysis at all, it is usually for routine operation decisions. The largest, most risky decisions get the least amount of risk analysis.”
The combination of estimation (Chapter 5), quantifying risk and Monte Carlo analysis may seem complex keeps many decision makers from using the technique, this is especially true in software development, hence the paradox. For example, every software development estimation problem, whether Agile, lean or plan based, has a large degree of uncertainty embedded in the process and therefore, is a perfect candidate to use Monte Carlo analysis. However, very few estimator understand or use the technique. Learning Monte Carlo analysis (and using the one of the many tools to do the mathematical heavy lifting) or alternately hiring some to perform risk analysis are both paths to addressing adding quantitative data to decision making. When making decisions under conditions of uncertainty, Monte Carlo analysis is a necessity to do the math needed.
Previous Installments in Re-read Saturday, How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition
Chapter 1: The Challenge of Intangibles
Chapter 2: An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily
Chapter 3: The Illusions of Intangibles: Why Immeasurables Aren’t