HTMA

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 11 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition begins the fourth and final section of the book.  This section addresses topics that are beyond the basics. Hubbard titled Chapter 11 Preferences and Attitudes: The Softer Side of Measurement.  The softer side is a euphemism for attitudes and opinions.  Here is a summary of the chapter in a few bullet points:

·         Measure opinions and feelings.

·         Design out bias in surveys and questions.

·         Observe opinions and feelings through trade-offs.

·         Use trade-offs to describe risk tolerance. (more…)

 How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 10 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition is titled, Bayes: Adding to What You Know Now.  Here is a summary of the chapter in a few bullet points:

  • Prior knowledge influences how we process new information.
  • Bayesian statistics help us move from what we know to what we don’t know.
  • We are all Bayesian to some extent, but maybe not enough.
  • Many myths about using information are just plain wrong, and Bayes proves it.

(more…)

His portfolio doesn't look so healthy.

His portfolio doesn’t look so healthy.

As we have noted in past entries, effective measurement is a balance. The five Agile Portfolio Metrics Categories as whole represent a pallet from which an organization can craft a balanced set of Agile metrics. The goal of any measure is to reduce the uncertainty the organization is facing.  Value and portfolio health metrics shift the discussion from the high-level management of the distribution of work (who is doing what for whom) to the value of work (value metrics) and how the activities across a portfolio of work are perceived (portfolio health metrics). Each category is targeted on its own set of specific questions, but each category can and often does influence another category. We break down the second two categories below. (more…)

532154894_d077e58fbf_z

Portfolio metrics provide direction

Agile portfolio metrics are integral to prioritization and validating the flow of work. The term portfolio has many uses in a software development organizations, ranging from product portfolios to application portfolios.  I have even seen scenarios where organizations put all their development, enhancement and maintenance efforts into a single bucket for prioritization and queuing until there was the capacity to work on them.  We will use an inclusive definition of the portfolio to include, at a high level, all the software development and enhancement work in an organization. Products, lines or business, and even applications are often used to define portfolios.  I break Agile portfolio metrics into five high-level categories. (more…)

HTMA

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 8 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, begins the third section of the book.  Part III is focused on Measurement Methods.  Chapter 8 is titled, The Transition: From What to Measure to How to Measure. This is where you roll up your sleeves, crack your knuckles and get to work. Whenever you are beginning something new, the question of where to start emerges.  If I were to summarize the chapter in three sentences I would say: (more…)

HTMA

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 4 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Clarifying the Measurement Problem. In this chapter Hubbard focuses on two questions.  The first is getting the reader to answer what is the decision that measurement is supposed to support. The second is what is the definition of the thing being measured in terms of observable consequences.  These questions sound very basic; however, I found myself asking variations of those questions more than once recently when reviewing a relatively mature measurement program.  Answering these questions are often at the heart of defining the real mission of any measurement or measurement program and whether a measure will have value. (more…)

 How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 3 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled The Illusion of Intangibles: Why Immeasurables Aren’t.  In this chapter Hubbard explores three misconceptions of measurement that lead people to believe they can measure something, three reasons why people think something should not be measured and four useful measurement assumptions.

Hubbard begins the chapter with a discussion of the  reasons people most commonly suggest that something can’t be measured. The misconceptions are summarized into three categories:

  1. The concept of measurement. – The first misconception stems from not understanding the definition of measurement. Hubbard defines measurement as an observation that quantifiably reduce uncertainty. If we considered that most of the business decisions are faced with are based on imperfect information, therefore, made under uncertainty. The   measurement is an activity that adds information that improves on prior knowledge. Like the bias that causes people to avoid tackling risks that can’t be reduced to zero, some people will avoid measurement if they can’t reduce uncertainty to zero. Rarely if ever does measurement eliminate all uncertainty but rather  reduces uncertainty, however; all measures that reduce enough uncertainty is often valuable. The concept of the need for measurement data that reduce uncertainty can be seen in portfolio management questions which decide which projects will be funded even before all of the requirements are known.

    All types of attributes can be measured regardless of whether they are qualitative (for example team capabilities) or quantitative (for example project cost). Another consideration when an understanding measurement is that there are numerous measurement scales. Different measurement scales include nominal, ordinal, interval and  ratio scales.  Each scale allows different statistical operations and can present different conceptual challenges. It is imperative to understand the how can be used and the mathematical operations that can be leveraged for each (we will explore these on the blog in the near future).

    Hubbard concludes this section with a discussion of two of his basic assumptions. The first is the assumption is that there is a prior state of uncertainty that be quantified or estimated. And that uncertainty is a feature of the observer not necessarily that of the thing being observed.  This is a basic argument of Bayesian statistic. In Bayesian statistics both the initial and change in uncertainty will be quantified.

    Bottom line, it is imperative to understand the definition of measurement, measurement scales and Bayesian statistics so that you can apply the concepts of measurement to reducing uncertainty.

  1. The object of measurement. – The second misconception stems from the use of sloppy language or a failure to  define what is being measured. In order to measure something, we must unambiguously state what the object of measurement means. For example, many organizations wish to understand the productivity of development and maintenance teams but don’t construct a precise  definition  of the concept AND why we care / why the measure is valuable.

    Bottom Line: Decomposing what is going to be measured from vague to explicit should always be the first step in the measurement process.

  1. Methods of measurement. – The third misconception is a reflection of not understanding what constitutes measurement. The process and procedures are often constrained to direct measurement such as counting the daily receipts at a retail store. Most of the difficult measurement in the business (or a variety of other) environments must be done using indirect measurement. For example, in Chapter 2 Hubbard used the example of  Eratosthenes measurement of the circumference of the earth. Eratosthenes used observations of shadows and the angle of the sun to indirectly determine the circumference. A direct measure would have been if he had used a really long tape measure (pretty close to impossible).

    A second topic related to this misconception is thought that valuable measurement requires either measuring the whole population or a large sample. Studying populations is often impractical.  Hubbard shares the rule of five (proper random samples of five observations) or the single sample majority rule yield can dramatically significantly narrow the range of uncertainty.

    Bottom line: Don’t rely on your intuition about sample size.  The natural tendency is to believe a large sample is needed to reduce uncertainty, therefore, many managers will either decide that measurement is not possible managers because they are uncomfortable with sampling.

Even when it possible to measure the argument often turns to why you shouldn’t. Hubbard summarizes the “shouldn’t” into three categories.

  1. Cost too much / economic objections. – Hubbard suggests that most variables that are measured yield little or no informational value, they do not reduce uncertainty in decisions. The value delivered by measurement must outweigh the cost (this is one of the reasons you should “why” you want any measure) of collecting and analysis.

    Bottom Line:  Calculate the value of information based on the reducing uncertainty. Variables that have enough value justify deliberate measurement is justified. Hubbard suggests (and I concur) when someone says something is too expensive or too hard to measure the question in return should be compared to what.

  1. Measures lack usefulness or meaningfulness. – It is often said that you can prove anything with statistics as a reason to point out that measurement is not very meaningful. Hubbard suggests you “you can prove anything” the statement is patently unprovable. What is really meant is that numbers can be used to confuse people especially those without skills in statistics.

    Bottom Line: Investing in statistical knowledge is important for anyone that needs to make decisions and wants to outperform expert judgment.

  1. Ethical objection – Measurement can be construed as dehumanizing. Reducing everything can be thought of as taking all human factors out of a decision, however, measurement does not suggest there are only black and white answers. Measurement increase information while reducing uncertainty.  Hubbard provides a great quote, ”   the preference for ignorance is over even marginal reduction ignorance is never moral.”

    Bottom Line: Information and reduction of uncertainty are  neither moral or immoral.

The chapter is capped by four useful measurement assumptions that provide a summary for the chapter.

  1. Everything has been measured before. Do your research before you start!
  2. You have data than you think. – Consider both direct and indirect data.
  3. You need far fewer data points than you think. – Remember the rule of five.
  4. New observations are more accessible than you think. – Sample and use simple measures.

More than occasionally I have been told the measuring is meaningless since the project or piece work being measured is unique and that the past does not predict the future.  Interesting these same people yell the loudest when I suggest that if the past does not count that team members can be considered fungible assets and trade in and out of a project.  Measurement and knowledge of the past reduce almost always reduces uncertainty.

Previous Installments in Re-read Saturday, How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

How To Measure Anything, Third Edition, Introduction

HTMA, Chapter 1

HTMA, Chapter 2

 

 

Follow

Get every new post delivered to your Inbox.

Join 5,111 other followers