The Seven Deadly Sins popularized by Dante have been used as an analogy for a fallen man’s tendency to sin and as analogy for the ills or risks of many professions. The analogy fits as well for software metrics; focusing attention on the behaviors that could sap your program’s integrity, effectiveness and life span. This paper looks at a set of deadly sins from the point of view of a person or group that is creating or managing a metrics program. The paper will be delivered as several essays on the Software Process and Measurement podcast (www.spamcast.net).

The Seven Deadly Sins of Metrics Programs:

  • Pride – Believing that a single number / metric is more important than any other factor.
  • Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  • Wrath – Using measures to create friction between groups or teams.
  • Sloth – Unwillingness to act or care about the measures you create.
  • Greed – Allowing metrics to be used as a tool to game the system for gain.
  • Gluttony – Collecting data for data’s sake.
  • Lust (Extravagance) – Pursuit of the number rather than the business goal.

The impact each sin on a metrics program depends on many factors ranging from management style to organizaional culture. An example can be seen when comparing the belief that one number is more important than all other factors (always) which illustrates the sin of pride compared to the lack of motivation illustrated by sloth. These are two very different issues with two very different impacts however neither should be sneezed at if you value the long term health of a metrics program.

Pride:
The first deadly sin is pride. In the cannon of deadly sins, pride is the sin from which all others spring. In the world of metrics programs, the sin of pride is exhibited when a metrics program settles on a single metric that is used to reflect the value or well being of a project, a group or organization. Examples of abound of metrics programs that fixated on cost or productivity to the exclusion of a broader palette of metrics and work attributes. Most metrics professionals quickly learn that one metric can not be used for all projects. If you can’t easily answer the question, “Why is this true?” each time you use a metric and for each metric you use, the information generated through measurement and analysis will provide little or no value. The goal is to understand the differences between groups of work so that when comparisons are made, you can discern what is driving the difference (or even if there is a difference). Comparing package implementations, hardware intensive projects or custom development is rational only if you understand that there will be differences and what those differences mean. The bottom line is that rarely does a single point of view, a single metric, deliver that deep level of understanding that generates value from measurement.

Another example of the single metric syndrome generated by the sin of pride occurs when an organization uses a single metric to value performance in a contractual arrangement. While entire contracts are rarely stipulated on a single metric it easy for a single metric to be given disproportional weight due to a lack of understanding by the framers or a disconnect between the framers and the people that administer the contract. Poor understanding of the relationship between the numbers and the concepts they represent is akin to failure in punctuation when writing. The resulting meaning can be garbled as the contract is negotiated, implemented and managed. We won’t get into an existential argument over whether something is a sin if inadvertent, the result is the same. Garbled concepts can lead to a single metric focus which once discovered will beg to be taken advantage of. The type of advantage usually causes an overemphasis on a specific portion of the value chain such as productivity being emphasized over time-to-market, quality or cost.

 

Advertisements