HTMA

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 14 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition is the last chapter in the book.  Next week I will spend a few moments reflecting on the value I have gotten from this re-read; HOWEVER, the last chapter continues to deliver content, so let’s not get ahead of ourselves.  This chapter shows us:

  • A process for applying Applied Information Economics (which I used recently), and that
  • AIE is applicable in nearly every scenario.

Hubbard introduced Applied Information Economics (AIE) in Chapter One (page 9 to be exact).  The methodology includes five steps:

  1. Define the decision. 
  2. Determine what you know. 
  3. Compute the value of additional information. 
  4. Measure where the information value is high. 
  5. Make a decision; act upon it.

AIE is the centerpiece of How To Measure Anything. Chapter 14 brings all pieces together into an overall process populated with procedures and techniques. Hubbard lays out the application of AIE in four phases (0 – 3).

Phase 0 is a preparation phase which includes identifying workshop participants, developing the first cut of the measurement questions and then assigning the workshop participants pre-reading (homework) based on those initial questions.  Maximizing the value of the workshops requires priming the participants with homework.  The homework makes sure everyone is prepared for the workshops so that time is note wasted having coming up to speed. This also helps to reset any organizational anchor bias.

Phase 1:  Hold workshop(s) for problem definition, building a decision model, and developing initially calibrated estimates. Calibration exercises aid participants so they can quantify the initial variables as a range at a 90% confidence interval or as a probability distribution, rather than a single number.

Phase 2: This phase focuses on analyzing the value of information, the first cut at the measurement methods, refining the measurement methods, updating the decision model and then re-running the value of information analysis to make sure we don’t have  to change the measurement approach . Hubbard points out (and my experience attests) that during this step, you often determine that most variables have sufficient certainty, so the organization needs to do no further measurement beyond the calibrated estimate. This step ensures that the variables that move forward in the measurement process add value.

Phase 3: Use the data to make the decision(s) to run a Monte Carlo analysis to refine any of the metrics procedures needed, use the data to make the decisions identified and generate a final report and presentation (even Hubbard is a consultant, thus, a presentation).

The basic flow espoused by Hubbard is meant to cut through the standard rationalization to find the real questions.  Then to determine how to answer those questions, using measurement with an emphasis on making sure the organization does not already have the data needed to answer the questions, and then getting the data that make economic sense. The process sounds simple; however as a practitioner, the problem I have observed is often that generating the initial involvement is difficult and that participants often have pet theories that are difficult to disarm.  For example, I once ran across an executive that was firmly convinced that having his software development teams work longer hours would increase productivity (he forgot that productivity equals output divided by input). Therefore, he wanted to measure which monitoring applications would make his developers work more hours.  It took several examples to retrain him to recognize that to increase productivity, he had to increase output (functionality) more than he increased input (effort). The process described by Hubbard is extremely useful, but remember that making it work requires both math and facilitation skills.

The remainder of the chapter focuses on providing examples that show the concepts in the book in action.  The cases cover a wide range of scenarios, from improving logistics (forecasting fuel needs for the Marine Corps) to measuring the value of a department.  Each case provides a lesson for the reader; however three messages make my bottom line:

  • While some say that the data is too hard to get, it usually isn’t.
  • Reducing uncertainty often requires only one or few measures.
  • Framing the question as a reduction in uncertainty means that almost anything is measurable.

These three bottom line lessons summarize the philosophy of How To Measure Anything. But like the process to apply this philosophy, the devil is in the details.

Past installments of the Re-read Saturday of  How To Measure Anything, Third Edition

Chapter 1: The Challenge of Intangibles

Chapter 2: An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily

Chapter 3: The Illusions of Intangibles: Why Immeasurables Aren’t

Chapter 4: Clarifying the Measurement Problem

Chapter 5: Calibrated Estimates: How Much Do You Know Now?

Chapter 6: Quantifying Risk Through Modeling

Chapter 7: Quantifying The Value of Information

Chapter 8 The Transition: From What to Measure to How to Measure

Chapter 9: Sampling Reality: How Observing Some Things Tells Us about All Things

Chapter 10: Bayes: Adding To What You Know Now

Chapter 11: Preferences and Attitudes: The Softer Side of Measurement

Chapter 12: The Ultimate Measurement Instrument: Human Judges

Chapter 13: New Measurement Instruments for Management

We continue with the selection process for the next’ish book for the Re-Read Saturday.  We will read Commitment – Novel About Managing Project Risk by Olav Maassen and Chris Matts next.  Buy your copy today and start reading (use the link to support the podcast).  Mr. Adams has suggested that we will blow through the read of this book, therefore, doing the poll now will save time in a few weeks!  As in past polls please vote twice or suggest a write-in candidate in the comments.  We will run the poll for two more weeks.