
How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition
Chapter 14 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition is the last chapter in the book. Next week I will spend a few moments reflecting on the value I have gotten from this re-read; HOWEVER, the last chapter continues to deliver content, so let’s not get ahead of ourselves. This chapter shows us:
- A process for applying Applied Information Economics (which I used recently), and that
- AIE is applicable in nearly every scenario.
Hubbard introduced Applied Information Economics (AIE) in Chapter One (page 9 to be exact). The methodology includes five steps:
- Define the decision.
- Determine what you know.
- Compute the value of additional information.
- Measure where the information value is high.
- Make a decision; act upon it.
AIE is the centerpiece of How To Measure Anything. Chapter 14 brings all pieces together into an overall process populated with procedures and techniques. Hubbard lays out the application of AIE in four phases (0 – 3).
Phase 0 is a preparation phase which includes identifying workshop participants, developing the first cut of the measurement questions and then assigning the workshop participants pre-reading (homework) based on those initial questions. Maximizing the value of the workshops requires priming the participants with homework. The homework makes sure everyone is prepared for the workshops so that time is note wasted having coming up to speed. This also helps to reset any organizational anchor bias.
Phase 1: Hold workshop(s) for problem definition, building a decision model, and developing initially calibrated estimates. Calibration exercises aid participants so they can quantify the initial variables as a range at a 90% confidence interval or as a probability distribution, rather than a single number.
Phase 2: This phase focuses on analyzing the value of information, the first cut at the measurement methods, refining the measurement methods, updating the decision model and then re-running the value of information analysis to make sure we don’t have to change the measurement approach . Hubbard points out (and my experience attests) that during this step, you often determine that most variables have sufficient certainty, so the organization needs to do no further measurement beyond the calibrated estimate. This step ensures that the variables that move forward in the measurement process add value.
Phase 3: Use the data to make the decision(s) to run a Monte Carlo analysis to refine any of the metrics procedures needed, use the data to make the decisions identified and generate a final report and presentation (even Hubbard is a consultant, thus, a presentation).
The basic flow espoused by Hubbard is meant to cut through the standard rationalization to find the real questions. Then to determine how to answer those questions, using measurement with an emphasis on making sure the organization does not already have the data needed to answer the questions, and then getting the data that make economic sense. The process sounds simple; however as a practitioner, the problem I have observed is often that generating the initial involvement is difficult and that participants often have pet theories that are difficult to disarm. For example, I once ran across an executive that was firmly convinced that having his software development teams work longer hours would increase productivity (he forgot that productivity equals output divided by input). Therefore, he wanted to measure which monitoring applications would make his developers work more hours. It took several examples to retrain him to recognize that to increase productivity, he had to increase output (functionality) more than he increased input (effort). The process described by Hubbard is extremely useful, but remember that making it work requires both math and facilitation skills.
The remainder of the chapter focuses on providing examples that show the concepts in the book in action. The cases cover a wide range of scenarios, from improving logistics (forecasting fuel needs for the Marine Corps) to measuring the value of a department. Each case provides a lesson for the reader; however three messages make my bottom line:
- While some say that the data is too hard to get, it usually isn’t.
- Reducing uncertainty often requires only one or few measures.
- Framing the question as a reduction in uncertainty means that almost anything is measurable.
These three bottom line lessons summarize the philosophy of How To Measure Anything. But like the process to apply this philosophy, the devil is in the details.
Past installments of the Re-read Saturday of How To Measure Anything, Third Edition
Chapter 1: The Challenge of Intangibles
Chapter 2: An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily
Chapter 3: The Illusions of Intangibles: Why Immeasurables Aren’t
Chapter 4: Clarifying the Measurement Problem
Chapter 5: Calibrated Estimates: How Much Do You Know Now?
Chapter 6: Quantifying Risk Through Modeling
Chapter 7: Quantifying The Value of Information
Chapter 8 The Transition: From What to Measure to How to Measure
Chapter 9: Sampling Reality: How Observing Some Things Tells Us about All Things
Chapter 10: Bayes: Adding To What You Know Now
Chapter 11: Preferences and Attitudes: The Softer Side of Measurement
Chapter 12: The Ultimate Measurement Instrument: Human Judges
Chapter 13: New Measurement Instruments for Management
We continue with the selection process for the next’ish book for the Re-Read Saturday. We will read Commitment – Novel About Managing Project Risk by Olav Maassen and Chris Matts next. Buy your copy today and start reading (use the link to support the podcast). Mr. Adams has suggested that we will blow through the read of this book, therefore, doing the poll now will save time in a few weeks! As in past polls please vote twice or suggest a write-in candidate in the comments. We will run the poll for two more weeks.
March 28, 2016 at 1:19 am
I like the summary of this book which comes from question #23 (Chapter 14) in the HTMA Workbook, and I am quoting both the question and answer …
Summarize six points the author makes about the AIE philosophy.
If it’s really that important, it’s something you can define. If it’s something you think exists at all, it’s something you’ve already observed somehow.
If it’s something important and something uncertain, you have a cost of being wrong and a chance of being wrong.
You can quantify your current uncertainty with calibrated estimates.
You can compute the value of information by knowing the “threshold” of the measurement where it begins to make a difference compared to you existing uncertainty.
Once you know what it is worth to measure something, you can put the measurement effort in context and decide on the effort it should take.
Knowing just a few methods for random sampling, controlled experiments, Bayesian methods, or even merely improving on the judgements of experts can lead to a significant reduction in uncertainty.
Hubbard last paragraph in the HTMA book tells to how to start applying this knowledge (p. 385) “… and the practical cases described make you a little more skeptical about claims that something critical to your business cannot be measured”.
March 28, 2016 at 9:19 pm
Thanks! What are your thoughts on the workbook? I have nearly bought it twice over the past few months.
April 3, 2016 at 12:41 am
Workbook is a great study guide for this material. It is short and helps focuses the reader on what the author thought was important.
By the way, I found the above quote in the main book (today), on page 385.
April 5, 2016 at 11:04 am
I purchased the workbook this morning!
March 28, 2016 at 1:20 am
[…] Chapter 14: A Universal Measurement Method: Applied Information Economics […]
April 2, 2016 at 11:58 pm
[…] Chapter 14: A Universal Measurement Method: Applied Information Economics […]
March 9, 2017 at 11:56 pm
[…] Chapter 14: A Universal Measurement Method: Applied Information Economics […]
October 17, 2017 at 11:56 pm
[…] is a method for measuring and determining whether measuring would be valuable. (See our re-read of How to Measure Anything). The goal of spending the time and effort to measure is both intangible value (and costs), so […]