Chapter 3 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled The Illusion of Intangibles: Why Immeasurables Aren’t. In this chapter Hubbard explores three misconceptions of measurement that lead people to believe they can measure something, three reasons why people think something should not be measured and four useful measurement assumptions.
Hubbard begins the chapter with a discussion of the reasons people most commonly suggest that something can’t be measured. The misconceptions are summarized into three categories:
- The concept of measurement. – The first misconception stems from not understanding the definition of measurement. Hubbard defines measurement as an observation that quantifiably reduce uncertainty. If we considered that most of the business decisions are faced with are based on imperfect information, therefore, made under uncertainty. The measurement is an activity that adds information that improves on prior knowledge. Like the bias that causes people to avoid tackling risks that can’t be reduced to zero, some people will avoid measurement if they can’t reduce uncertainty to zero. Rarely if ever does measurement eliminate all uncertainty but rather reduces uncertainty, however; all measures that reduce enough uncertainty is often valuable. The concept of the need for measurement data that reduce uncertainty can be seen in portfolio management questions which decide which projects will be funded even before all of the requirements are known.
All types of attributes can be measured regardless of whether they are qualitative (for example team capabilities) or quantitative (for example project cost). Another consideration when an understanding measurement is that there are numerous measurement scales. Different measurement scales include nominal, ordinal, interval and ratio scales. Each scale allows different statistical operations and can present different conceptual challenges. It is imperative to understand the how can be used and the mathematical operations that can be leveraged for each (we will explore these on the blog in the near future).
Hubbard concludes this section with a discussion of two of his basic assumptions. The first is the assumption is that there is a prior state of uncertainty that be quantified or estimated. And that uncertainty is a feature of the observer not necessarily that of the thing being observed. This is a basic argument of Bayesian statistic. In Bayesian statistics both the initial and change in uncertainty will be quantified.
Bottom line, it is imperative to understand the definition of measurement, measurement scales and Bayesian statistics so that you can apply the concepts of measurement to reducing uncertainty.
- The object of measurement. – The second misconception stems from the use of sloppy language or a failure to define what is being measured. In order to measure something, we must unambiguously state what the object of measurement means. For example, many organizations wish to understand the productivity of development and maintenance teams but don’t construct a precise definition of the concept AND why we care / why the measure is valuable.
Bottom Line: Decomposing what is going to be measured from vague to explicit should always be the first step in the measurement process.
- Methods of measurement. – The third misconception is a reflection of not understanding what constitutes measurement. The process and procedures are often constrained to direct measurement such as counting the daily receipts at a retail store. Most of the difficult measurement in the business (or a variety of other) environments must be done using indirect measurement. For example, in Chapter 2 Hubbard used the example of Eratosthenes measurement of the circumference of the earth. Eratosthenes used observations of shadows and the angle of the sun to indirectly determine the circumference. A direct measure would have been if he had used a really long tape measure (pretty close to impossible).
A second topic related to this misconception is thought that valuable measurement requires either measuring the whole population or a large sample. Studying populations is often impractical. Hubbard shares the rule of five (proper random samples of five observations) or the single sample majority rule yield can dramatically significantly narrow the range of uncertainty.
Bottom line: Don’t rely on your intuition about sample size. The natural tendency is to believe a large sample is needed to reduce uncertainty, therefore, many managers will either decide that measurement is not possible managers because they are uncomfortable with sampling.
Even when it possible to measure the argument often turns to why you shouldn’t. Hubbard summarizes the “shouldn’t” into three categories.
- Cost too much / economic objections. – Hubbard suggests that most variables that are measured yield little or no informational value, they do not reduce uncertainty in decisions. The value delivered by measurement must outweigh the cost (this is one of the reasons you should “why” you want any measure) of collecting and analysis.
Bottom Line: Calculate the value of information based on the reducing uncertainty. Variables that have enough value justify deliberate measurement is justified. Hubbard suggests (and I concur) when someone says something is too expensive or too hard to measure the question in return should be compared to what.
- Measures lack usefulness or meaningfulness. – It is often said that you can prove anything with statistics as a reason to point out that measurement is not very meaningful. Hubbard suggests you “you can prove anything” the statement is patently unprovable. What is really meant is that numbers can be used to confuse people especially those without skills in statistics.
Bottom Line: Investing in statistical knowledge is important for anyone that needs to make decisions and wants to outperform expert judgment.
- Ethical objection – Measurement can be construed as dehumanizing. Reducing everything can be thought of as taking all human factors out of a decision, however, measurement does not suggest there are only black and white answers. Measurement increase information while reducing uncertainty. Hubbard provides a great quote, ” the preference for ignorance is over even marginal reduction ignorance is never moral.”
Bottom Line: Information and reduction of uncertainty are neither moral or immoral.
The chapter is capped by four useful measurement assumptions that provide a summary for the chapter.
- Everything has been measured before. Do your research before you start!
- You have data than you think. – Consider both direct and indirect data.
- You need far fewer data points than you think. – Remember the rule of five.
- New observations are more accessible than you think. – Sample and use simple measures.
More than occasionally I have been told the measuring is meaningless since the project or piece work being measured is unique and that the past does not predict the future. Interesting these same people yell the loudest when I suggest that if the past does not count that team members can be considered fungible assets and trade in and out of a project. Measurement and knowledge of the past reduce almost always reduces uncertainty.
Previous Installments in Re-read Saturday, How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition