Dr. Deming

Dr. Deming

The Seven Deadly Sins of metrics programs are:

  1. Pride – Believing that a single number/metric is more important than any other factor.
  2. Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  3. Wrath – Using measures to create friction between groups or teams.
  4. Sloth – Unwillingness to act on or care about the measures you create.
  5. Greed – Allowing metrics to be used as a tool to game the system.
  6. Gluttony – Application of an excess of metrics.
  7. Lust – Pursuit of the number rather than the business goal.

In the end, these sins are a reflection of the organization’s culture. Bad metrics can generate bad behavior and reinforce an organizational culture issues. Adopting good measures is a step in the right direction however culture can’t be changed by good metrics alone. Shifting the focus on an organizations business goals, fostering transparency to reduce gaming and then using measures as tools rather than weapons can support changing the culture. Measurement can generate behavior that leads towards a healthier environment.  As leaders, measurement and process improvement professionals, we should push to shape their environment so that everyone can work effectively for the company.

The Shewhart PDCA Cycle (or Deming Wheel), set outs of model where measurement becomes a means to an end rather than an end in their own right. The Deming wheel popularized the Plan, Do Check, Act (PDCA) cycle which is focused on delivering business value. Using the PDCA cycle, organizational changes are first planned, executed, checked by measurement and then refined based on a positive feedback model. In his book The New Economics Deming wrote “Reward for good performance may be the same as reward to the weather man for a pleasant day.” Organizations that fall prey to the Seven Deadly Sins of metrics programs are apt to incent the wrong behavior.

(Thank you Dr. Deming).

Can't see forest for the trees

Can’t see forest for the trees

The first deadly sin is pride. In the cannon of deadly sins, pride is the sin from which all other spring. In the world of metrics programs, the sin of pride is when a metrics program settles on a single metric that is used to reflect the value or well-being of a project, a group or organization. Examples of abound of metrics programs that fixated on cost or productivity to the exclusion of a broader palette of metrics and work attributes.  Most metrics professionals quickly learn that one metric cannot be used for all projects.  If you can’t easily answer the question, “Does this relate?” each time you use a metric and for each metric you use, the information generated through measurement and analysis will provide little or no value.  The goal is to understand the differences between groups of work so that when comparisons are made, you can discern what is driving the difference (or even if there is a difference).  Comparing package implementations, hardware intensive projects or custom development is rational only if you understand that there will be differences and what those differences mean.   The bottom line is that rarely does a single metric deliver that deep level of understanding that generates value from measurement.

Another example of the single metric syndrome generated by the sin of pride occurs when an organization uses a single metric to value performance in a contractual arrangement.  While entire contracts are rarely stipulated on a single metric, it easy for a single metric to be given disproportional weight due to the framers’ lack of understanding or a disconnect between the framers and the people that administer the contract.  Poor understanding of the relationship between the numbers and the concepts they represent is akin to failure in punctuation when writing.  The resulting meaning can be garbled as the contract is negotiated, implemented and managed.  We won’t get into an existential argument over whether something is a sin if it is inadvertent; the result is the same. Garbled concepts can lead to a single metric focus which once discovered will beg to be taken advantage of.  This usually causes an overemphasis on a specific portion of the value chain, such as productivity being emphasized over time-to-market, quality or cost.

3216110254_15d4a00bf2_b

In Christianity, the seven deadly sins are the root of all other sins. This concept has been used as an analogy for the ills or risks for many professions.  The analogy fits as well for software metrics; focusing attention on the behaviors that could sap your program’s integrity, effectiveness and lifespan. Here we will look at the deadly sins from the point of view of a person or group that is creating or managing a metrics program. As with many things in life, forewarned is forearmed, and knowledge is a step towards avoidance.

Here are the seven deadly sins of metrics programs:

  • Pride – Believing that a single number/metric is more important than any other factor.
  • Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  • Wrath – Using measures to create friction between groups or teams.
  • Sloth – Unwillingness to act on or care about the measures you create.
  • Greed – Allowing metrics to be used as a tool to game the system for gain.
  • Gluttony – Application of an excess of metrics.
  • Lust – Pursuit of the number rather than the business goal.

All of the deadly sins have an impact on the value a metrics program can deliver.  Whether anyone sin is more detrimental than another is often a reflection of where a metrics program is in it’s life cycle. For instance, pride, the belief that one number is more important than all other factors, is more detrimental than sloth or a lack of motivation as a program begins whereas sloth becomes more of an issue as a program matures.  These are two very different issues with two very different impacts, however neither should be sneezed at if you value the long-term health of a metrics program. Pride can lead to overestimating your capabilities and sloth can lead to not using those you have in the end self-knowledge is the greatest antidote.

Over the next few days we will visit the seven deadly sins of metrics!