Dr. Deming

Dr. Deming

The Seven Deadly Sins of metrics programs are:

  1. Pride – Believing that a single number/metric is more important than any other factor.
  2. Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  3. Wrath – Using measures to create friction between groups or teams.
  4. Sloth – Unwillingness to act on or care about the measures you create.
  5. Greed – Allowing metrics to be used as a tool to game the system.
  6. Gluttony – Application of an excess of metrics.
  7. Lust – Pursuit of the number rather than the business goal.

In the end, these sins are a reflection of the organization’s culture. Bad metrics can generate bad behavior and reinforce an organizational culture issues. Adopting good measures is a step in the right direction however culture can’t be changed by good metrics alone. Shifting the focus on an organizations business goals, fostering transparency to reduce gaming and then using measures as tools rather than weapons can support changing the culture. Measurement can generate behavior that leads towards a healthier environment.  As leaders, measurement and process improvement professionals, we should push to shape their environment so that everyone can work effectively for the company.

The Shewhart PDCA Cycle (or Deming Wheel), set outs of model where measurement becomes a means to an end rather than an end in their own right. The Deming wheel popularized the Plan, Do Check, Act (PDCA) cycle which is focused on delivering business value. Using the PDCA cycle, organizational changes are first planned, executed, checked by measurement and then refined based on a positive feedback model. In his book The New Economics Deming wrote “Reward for good performance may be the same as reward to the weather man for a pleasant day.” Organizations that fall prey to the Seven Deadly Sins of metrics programs are apt to incent the wrong behavior.

(Thank you Dr. Deming).

1123084506_c69acb0424_b

The measurement/performance feedback loop causes an addiction to a single metric. The addict will exclude what is really important.

There is a famous adage: you get what you measure. When an organization measures a specific activity or process, people tend to execute so they maximize their performance against that measure. Managers and change agents often create measures to incentivize teams or individuals to perform work in a specific then to generate a feedback loop. The measurement/performance feedback loop causes an addiction to a single metric. The addict will exclude what is really important. Chasing the endorphins that the feedback will generate is the sin of lust in the measurement world. Lust, like wrath, is a loss of control which affects your ability to think clearly. Balanced goals and medium to long-term focus are tools to defeat the worst side effects of measurement lust. The ultimate solution is a focus on the long-term goals of the organization.

How does this type of unbalanced behavior occur?  Usually measurement lust is generated by either an unbalanced measurement programs or performance compensation programs.   Both cases can generate the same types of unintended consequences. I call this the “one number syndrome”. An example of the “one number syndrome” is when outsourcing contracts include penalty and bonus clauses based on a single measure, such as productivity improvements.  Productivity is a simple metric that can be affected by a wide range of project and organizational attributes. Therefore just focusing on measuring just productivity can have all sorts of outcomes as teams tweak the attributes affecting productivity and then review performance based on feedback.  For example, one common tactic used to influence productivity is by changing the level of quality that a project is targeting; generally higher quality generates lower productivity and vice versa. Another typical example of organizations or teams maximize productivity is to throttle the work entering the organization. Reducing the work entering an organization or team generally increases productivity. In our examples the feedback loop created by fixating on improving productivity may have the unintended consequence.

A critical shortcoming caused by measurement lust is a shift toward short-term thinking as teams attempt to maximize the factors that will use to just their performance. We have all seen the type of short-term thinking that occurs when a manager (or an organization) does everything in their power to make some monthly goal. At the time the choices are made they seem to be perfectly rational. Short-term thinking has the ability to convert the choices made today into the boat anchors of the next quarter. For example, right after I left university I worked for a now defunct garment manufacturer. On occasion salespeople would rush a client into an order at the end of a sales cycle to make their quota. All sorts of shenanigans typically ensued including returns, sale rebates but the behavior always caught up one or two sales periods later. In a cycle of chasing short-term goals with short-term thinking, a major failure is merely a matter of time. I’m convinced from reading the accounts of the Enron debacle that the cycle of short-term thinking generated by the lust to meet their numbers made it less and less likely that anyone could perceive just how irrational their decisions were becoming.

The fix is easy (at least conceptually). You need to recognize that measurement is a behavioral tool and create a balanced set of measures (frameworks like the Balanced Scorecard are very helpful) that therefore encourage balanced behavior.  I strongly suggest that as you are defining measures and metrics, take the time to forecast the behaviors each measure could generate.  Ask yourself whether these are the behaviors you want and whether other measures will be needed to avoid negative excesses.

Lust rarely occurs without a negative feedback loop that enables the behavior. Measures like productivity or velocity when used for purely process improvement or planning rather than to judge performance (or for bonuses) don’t create measurement lust. Balanced goals, balanced metrics, balanced feedback and balanced compensation are all a part of plan to generate balanced behavior. Imbalances of any of these layers will generate imbalances in behavior. Rebalancing can change behavior but just make sure it is the behavior you anticipate and it doesn’t cause unintended consequences by shifting measurement lust to another target.

Greed is taking all the food and not leaving some for everyone else.

Greed is taking all the food and not leaving some for everyone else.

Greed, in metrics programs, means allowing metrics to be used as a tool to game the system to gain more resources than one needs or deserves.  At that point measurement programs start down the path to abandonment. The literature shows that greed, like envy, is affected by a combination of personal and organizational attributes.   Whether the root of the problem is nature or nurture, organizational culture can make the incidence of greed worse and that is something we can do something about.

One of the critical cultural drivers that create a platform for greed is fear.  W. Edward Deming in his famous 14 Principles addressed fear: “Drive out fear, so that everyone may work effectively for the company.” Fear is its own disease, however combined with an extremely competitive culture that stresses win/lose transactions, it creates an atmosphere that causes greed to become an economically rational behavior.  Accumulating and hoarding resources reduces your internal competitors’ ability to compete and reduces the possibility of losing because of lack of resources.  Fear-driven greed creates its own insidious cycle of ever increasing fear as the person infected with greed fears that their resource horde is at risk and requires defense (attributed to Sun Tzu in the Art of War). An example of the negative behaviors caused by fear that I recently heard about was a company that had announced that they cull the lower ten percent of the organization annually at the beginning of last year.  Their thought was that completion would help them identify the best and the brightest.  In a recent management meeting the person telling the story indicated that the CIO had expressed exasperation with projects that hadn’t shared resources and that there were cases in which personnel managers had actively redirected resources to less critical projects.

Creating an atmosphere that fosters greed can generate a whole host of bad behaviors including:

  1. Disloyalty
  2. Betrayal
  3. Hoarding
  4. Cliques/silos
  5. Manipulation of authority

Coupling goals, objectives and bonuses to measures in your metrics program can induce greed and have a dramatic effect on many of the other Seven Deadly Sins. For example, programs that have wrestled with implementing a common measure of project size and focused on measuring effectiveness and efficiency will be able to highlight how resources are used.  Organizations that then set goals that based on comparing team effectiveness and efficiency will create an environment in which hoarding resources generate a higher economic burden on the hoarder, because it reduces the efficiency of other teams.  That potential places a burden on a measurement program to create an environment where greed is less likely to occur.

Measurement programs can help create an atmosphere that defuses greed by providing transparency and accountability for results. Alternately as we have seen in earlier parts of this essay, poor measurement programs can and do foster a wide range of poor behaviors.

Wrath

Wrath

Wrath is the inordinate and uncontrolled feelings of hatred and anger.  I suspect that you conjure a picture of someone striking out with potentially catastrophic results.  When applied to measurement, wrath is the use of data in a negative or self-destructive manner (rather than an act of wrathful measurement). Very few people are moved to measure by wrath, rather they are moved by wrath to use measurement badly. Wrath causes people to act in a manner that might not be in their or in the organization’s best interest. Both scenarios are bad.  Data and the information (good or bad) derived from that data can used as a weapon in a manner that destroys the credibility of the program and the measurement practitioners.  

Anger impairs one’s ability to process information and to exert cognitive control over their behavior. An angry person may lose his/her objectivity, empathy, prudence or thoughtfulness and may cause harm to others. Actions driven by extreme anger is easily recognized by observers, but rarely by those perpetrating the behavior. This is an example of being blind with rage.  There is no room in the workplace for rage. Protect your measurement program and your career by staying in control. When confronted with scenarios that induce rage you need to learn how to step back and see the whole situation. Being mad or angry is fine if those emotions do not cloud your judgment. Teaching yourself to always see things more calmly will help your realize the truth of the harm that you are causing to yourself and others through rage. I once saw a CIO fly off the hook when are project shared it’s measurement dashboard, the project reporting that they were behind schedule, defects were above projections and the number of potential risks were rising. The uncontrolled rant was awe inspiring however the CIO lost the support of his senior leaders and within a month he was gone. Control puts you in a position to react in a more rational manner.

Measurement data and the information derived from that data deliver the ability to understand why things happen: why a project is late, why a project costs what it does or even why a specific level of quality was achieved.  Measurement is a tool to take action to improve how work is done.  What it should not be is a weapon of indiscriminate destruction. Acting in a rage changes all of that. When you strike out in an uncontrolled manner you have transformed that data into a weapon with very little guidance. Think of the difference between the indiscriminate nature of a land mine and the precision of phasers of the Star Ship Enterprise. Wrath turns a potentially valuable tool into something far less reliable. For example, a purposeful misrepresentation of the meaning of data can lead to team or organization making wrong decisions. Other examples include errors of omissions (leaving out salient facts) or inclusion (including irrelevant data that changes the conclusions drawn from the data).  Whether omission or inclusion, poor use of data erodes the value of the measurement program though politicization or placing doubt about the value of measurement into people’s minds. Remember that all analysis requires interpretation, however the interpretations are generally based on an assumption that people will act logically and consistently. That includes your behavior. Analysis based on an obviously false assumptions just to make a point does no one any good in the long run.  For example, assuming productivity is constant across all sized of projects so that you can show that a project under-performed to get back at someone will destroy your credibility even if you win the argument. Be true to the data or be the point of a failure in trust.  

Do not confuse passion and rage; they are not the same. You must have passion to be effective but what you can’t do, is to lose control of your emotions to the point that you stop thinking before you act. The deadly sin of wrath is a sin that reflection of bad behavior, if you let wrath affect your behavior you will begin a spiral that ends with a failure of trust.

Definition of done

Don’t trip the runner next to you just to win.

The results of software measurement can be held up as badge of honor. It is not uncommon for a CIO, department manager, project manager or even technical lead to hold up the performance on their projects in front others, engendering envy from other projects. Envy is a feeling of discontent and resentment aroused by and in conjunction with desire for the possessions or qualities of another. Measurement is a spotlight that can focus other’s envy if the situation is right. That can occur when  bonuses are tied to measurement and when the assignment and staffing of projects is driven by unknown factors. There are two major types of metrics-based envy: one must be addressed at the personnel level and the second must be addressed organizationally.

Envy can be caused when the metrics of projects managed by others in your peer group (real or perceived) are held up as examples to be emulated.  The active component of envy at this level is triggered by a social comparison that threatens a person’s self image, and can be exacerbated when the attributes that impact performance are outside of the team’s control. The type or complexity of the work coming to a team is generally negotiable. Teams that get the really tough problem will generally not have the highest productivity even though they may solved an intractable business problem. Envy generated by this type of problem translates into a variety of harmful behaviors. In benign cases, we might just pass it off as office politics (which everybody loves, not), or in a worst case scenario could generate a self destructive spiral of negative behavior which is not helpful to anyone.  Typical envy-driven behaviors to watch for include the loss of will, poor communication, withdrawal and hiding.  While the amateur psychologist in me would be happy to pontificate on the personal side of envy, I am self aware enough to know that I shouldn’t.  If you have fallen into the trap of envy, get professional help. If you are a manager of a person that is falling into this hole, get them help or get them out of the organization.

The other category of triggers are organizational.  These are the triggers that as managers, we have more control over and have the obligation to address.  As leaders we have a chance to mold the organizational culture to be supportive of efficiency and effectiveness.  Cultures and environments can facilitate and foster both good and bad behaviors.  Cultures that support an atmosphere of individual competition above collaboration can create an atmosphere where envy will flourish. This will act as a feedback loop to further deepen silos and the possibility of envy. For example, Sid may feel that Joe always gets the best recruits and he is powerless to change the equation (for whatever reason), therefore he can’t compete.  Envy may cause him to focus on stealing Joe’s recruits rather than coaching his own. Thisculture can disrupt communication and collaboration and create silos. In this type of environment positive behaviors, such as displaying measurement data, can act as feedback loop to deepen the competitive culture rather than generating collaboration and communication.  Typical behaviors generated by envy triggered by organizational issues include those noted earlier and outright sabotage of projects and careers (tripping the runner next to you so you can win), and just as bad, the pursuit of individual goals at the expense of the overall business goals.

Measurement programs can take the lead in developing a culture where teams can perform, be recognized for that performance and then share the lessons that delivered that performance when it is truly special. An important way to understand what type of performance really should be held up and emulated is based on the work of W. Edward Deming. In Deming’s seminal work Out of the Crisis, he suggested that only variation caused by by special causes should be specifically reviewed rather than normal or common cause performance. Understanding and using the concepts of common and special cause of variation as tools in your analysis will help ground your message in a reality that focuses on where specific performance is different enough to be studied. Common cause variation is generated by outcomes that are within the capability of the system.  Whereas special cause outcomes represent performance outside the normal capacity of the system. In every case, performance outside of the norm, should be studied and where positive, held up for others to emulate. By focusing your spotlight on these outcomes you have the opportunity to identify new cutting edge ideas and ideas that should be avoided.  Another technique for fostering collaboration (an environment where envy is less likely to take root) is to invite all parties to participate in the analysis of measurement data using tools such as a WIKI. The measurement group should provide the first wave of analysis, then let the stakeholders participate in shaping the final analysis, using the crowd sourcing techniques made famous by Jimmy Wales and Wikipedia.  Getting everyone involved creates a learning environment that uses measurement not only as tool to generate information, but also as a tool to shape the environment and channel the corporate culture.

Measurement and measurement programs don’t cause the sin of envy.  People and organizational cultures foster this sin in equal measure. Done correctly, measurement programs can act as a tool to tame the excess that lead to this sin. However the corollary is also true.  Done incorrectly or poorly, measurement ceases to be a positive tool and becomes part of the problem.  Measurement that fosters transparency and collaboration will help an organization communicate, grow and improve.

3216110254_15d4a00bf2_b

In Christianity, the seven deadly sins are the root of all other sins. This concept has been used as an analogy for the ills or risks for many professions.  The analogy fits as well for software metrics; focusing attention on the behaviors that could sap your program’s integrity, effectiveness and lifespan. Here we will look at the deadly sins from the point of view of a person or group that is creating or managing a metrics program. As with many things in life, forewarned is forearmed, and knowledge is a step towards avoidance.

Here are the seven deadly sins of metrics programs:

  • Pride – Believing that a single number/metric is more important than any other factor.
  • Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  • Wrath – Using measures to create friction between groups or teams.
  • Sloth – Unwillingness to act on or care about the measures you create.
  • Greed – Allowing metrics to be used as a tool to game the system for gain.
  • Gluttony – Application of an excess of metrics.
  • Lust – Pursuit of the number rather than the business goal.

All of the deadly sins have an impact on the value a metrics program can deliver.  Whether anyone sin is more detrimental than another is often a reflection of where a metrics program is in it’s life cycle. For instance, pride, the belief that one number is more important than all other factors, is more detrimental than sloth or a lack of motivation as a program begins whereas sloth becomes more of an issue as a program matures.  These are two very different issues with two very different impacts, however neither should be sneezed at if you value the long-term health of a metrics program. Pride can lead to overestimating your capabilities and sloth can lead to not using those you have in the end self-knowledge is the greatest antidote.

Over the next few days we will visit the seven deadly sins of metrics!

The seventh deadly sin of measurement programs is greed.  Greed in this perspective means allowing metrics to be used as a tool to game the system to gain resources; this is generally a reflection of fear.  When metrics become a tool to manipulate the system based on an inordinate desire to acquire more resources than one needs or deserves we see greed.  At that point measurement programs start down the path to abandonment.

There is an active debate whether greed is a basic human instinct or whether it is a behavior generated by environmental / cultural conditions. I suggest that the nature or nurture discussion is a red herring when it comes to the health and well being of a measurement program. I believe the literature shows that greed like envy, at the very least is affected by a combination of personal and organizational failings and in most cases psychoanalysis is outside the mandate of most measurement programs.  Whether the root of the problem is nature or nurture, organizational culture can make the incidence of greed worse and that is something we can do something about.

One of the critical cultural drivers that create a platform for greed is fear.  W. Edward Deming in his famous 14 Principles addressed fear.  Principle Three is “Drive out fear, so that everyone may work effectively for the company.[i]”  Fear is its own disease however combined with an extremely competitive culture that stresses win / lose transactions creates an atmosphere that causes greed to become an economically rational behavior.  Accumulating and hoarding resources reduces your internal competitors’ ability to compete and reduces the possibility of losing because of lack of resources.  Fear driven greed creates its own insidious cycle of ever increasing fear as the person infected with greed fears that their resource horde is at risk and requires defense (Sun Tzu).   An example of the negative behaviors caused by fear that I recently heard about was a company that had announced that they cull the lower ten percent of the organization annually at the beginning of last year.  Their thought was that completion would help them identify the best and the brightest.  In a recent management meeting the person telling the story indicated that the CIO had expressed exasperation with projects that hadn’t shared resources and that there were cases in which personnel managers had actively redirected resources to less critical projects.  Creating an atmosphere that fosters greed can generate a whole host of bad behaviors including:

  1. Disloyalty
  2. Betrayal
  3. Hoarding
  4. Cliques / silos
  5. Manipulation of authority

The measures and metrics in your metrics program can have a dramatic effect on many of the Seven Deadly Sins (or are a reflection of just how entrenched those sins have become).  Programs that have wrestled with a common measure of project size and focused on measuring effectiveness and efficiency will be able to highlight how resources are used.  Organizations that have set goals that are supported by effectiveness and efficiency will create an environment in which hoarding resources generates a higher economic burden on the hoarder because it reduces efficiency.  That burden will therefore create an environment where greed is less likely to occur.

Measurement programs can help create an atmosphere that defuses greed by providing transparency and accountability of results. Alternately as we have seen in earlier parts of this essay, poor measurement programs can and do foster a wide range of poor behaviors.

Summary

We began this journey through the Seven Deadly Sins of Measurement Programs nearly four years ago only to go on hiatus for nearly three years and then we began again 14 weeks ago.  The Seven Deadly Sins we have covered are:

  1. Pride – Believing that a single number / metric is more important than any other factor.
  2. Wrath – Using measures to create friction between groups or teams.
  3. Sloth – Unwillingness to act or care about the measures you create.
  4. Gluttony – Collecting data for data’s sake.
  5. Lust (Extravagance) – Pursuit of the number rather than the business goal.
  6. Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  7. Greed – Allowing metrics to be used as a tool to game the system to gain resources.

In many of the cases the behaviors that the Sins generated are a reflection of the organization’s culture.  A culture that can’t be changed by good metrics alone but in every case measures that focus on the business goals, foster transparency and are used as tools rather than weapons can provide a healthier environment.  As leaders, measurement and process improvement professionals should push to shape their environment so that everyone may work effectively for the company (Thank you Dr. Deming).


[i] Out of the Crisis by W. Edward Deming.  In my opinion, if you read one book on quality and process improvement, this MUST be it.

There is a famous adage: you get what you measure.  The point is if you focus on a specific activity or process, people will perform.Unfortunately the tendency to please and feedback can create an addiction or a fixation with a single metric or attribute. Fixation is an extreme of behavior that can cause the addict to exclude what is really important. Fixating and chasing a single measure to exclusion of everything else is the sin of lust in the measurement world. Lust, like wrath, is a loss of control which affects your ability to think clearly. Balanced goals and medium to long-term focus are tools to defeat the worst side effects of measurement lust. The ultimate solution is a focus on the long-term goals of the organization.

How does this type of unbalanced behavior occur?  Usually measurement lust is generated by either an unbalanced measurement programs or performance compensation programs.   Both cases can generate the same types of unintended consequences. I call this the “one number syndrome”. An example of the “one number syndrome” is when outsourcing contracts include penalty and bonus clauses based on a single measure, such as productivity improvements.  Productivity is a simple metric that can be affected by a wide range of attributes. Therefore just focusing on productivity can have all sorts of outcomes.  For example productivity can be impacted by changing the levels of quality, the throughput of project through the organization can be varied by throttling the work entering the organization generally increasing productivity or by changing staffing levels or location.  None of the common tactics for improving productivity may have the consequence you intended if you are fixated on just the measure.

A critical shortcoming caused by measurement lust is a shift toward short-term thinking. We have all seen the type of short-term thinking that occurs when the manager (or an organization) does everything in their power to make some monthly goal. At the time the choices seem to be perfectly rational, but short-term thinking has the ability to convert the choices made today into the boat anchors of the next quarter.  In a cycle of chasing short-term goals with short-term thinking, a major failure is merely a matter of time. I’m convinced from reading the accounts of the Enron debacle that the cycle of short-term thinking generated by the lust to meet their numbers made it less and less likely that anyone could perceive just how irrational their decisions were becoming.

The fix is easy (at least conceptually). You need to recognize that measurement is a behavioral tool and create a balanced set of measures (frameworks like the Balanced Scorecard are very helpful) that therefore encourage balanced behavior.  I strongly suggest that as you are defining measures and metrics, take the time to forecast the behaviors each measure could generate.  Ask yourself whether these are the behaviors you want and whether other measures will be needed to avoid negative excesses.

Lust rarely occurs without a feedback loop that enables the behavior. Balanced goals, balanced metrics, balanced feedback and balanced compensation are all a part of plan to generate balanced behavior. Imbalances of any of these layers will generate imbalances in behavior. Rebalancing can change behavior but just make sure it is the behavior you anticipate and it doesn’t cause the cascade failure of measurement lust.

This was broadcast on the Software Process and Measurement Cast 108.

Envy is a feeling of discontent and resentment aroused by and in conjunction with desire for the  possessions or qualities of another. Measurement is a spotlight that can generate envy if the environment and people involved combine to create the perfect storm.  Two major factors develop envy in metrics programs; one is a defect of person and the second a defect of the organization.

Envy can be caused personal problems usually caused by low self esteem.  The active component of envy is triggered by a social comparison that threatens a person’s self image. This translates into a variety of harmful behaviors. In benign cases, we might just pass it off as office politics (which everybody loves . . .not), or in a worst case scenario could generate a self destructive spiral of negative behavior which is not helpful to anyone.  Typical envy-driven behaviors to watch for include the loss of will, poor communication and withdrawal and hiding.  While the amateur psychologist in me would be happy to pontificate on the personal side of envy, I am self aware enough to know that I shouldn’t.  If you have fallen into the trap of envy, get professional help; if you are a manager of a person that is falling into this hole, get them help or get them out of the organization.

The other category of triggers are organizational.  These are the triggers that as managers we have more control over and have the obligation to address.  As leaders we have a chance to mold the organizational culture to be supportive of efficiency and effectiveness.  Cultures and environments can facilitate and foster both good and bad behaviors.  Cultures that support an atmosphere of individual completion above collaboration can create an atmosphere where envy can flourish. This will act as a feedback loop to further deepen silos and the possibility of envy. For example, Sid may feel that Joe always gets the best recruits and he is powerless to change the equation (for whatever reason), therefore he can’t compete.  Envy may cause him to focus on stealing Joe’s recruits rather than coaching his own.

Another organizational climate that fosters envious feelings are those where a climate of extreme competition has developed between individuals and teams.  This type of culture can disrupt communication and collaboration and foster silos. In this type of environment positive behaviors, such as displaying measurement data, can act as feedback loop to deepen the competitive culture rather than generating collaboration and communication.  Typical behaviors generated by envy triggered by organizational issues include those noted earlier and outright sabotage of projects and careers (tripping the runner next to you so you can win), and just as bad, the pursuit of individual goals at the expense of the overall business goals.
Measurement programs can take the lead in developing a culture where teams can perform, be recognized for that performance and then share the lessons that delivered that performance.  Understanding and using the concepts of common and special cause of variation as tools in your analysis will help ground your message in a reality that focuses on where specific performance is different enough to be studied. Common cause variation is generated by outcomes that are within the capability of the system.  Whereas special cause outcomes represent performance outside the norm. In every case, performance outside of the norm, should be studied. By focusing your spotlight on these outcomes you have the opportunity to identify new cutting edge ideas and ideas that should be avoided.  Another technique for fostering collaboration (an environment where envy is less likely to take root) is to invite all parties to participate in the analysis of measurement data using tools such as a WIKI. The measurement group should provide the first wave of analysis, then let the stakeholders participate in shaping the final analysis, using the crowd sourcing techniques made famous by Jimmy Wales and Wikipedia.  Getting everyone involved creates a learning environment and peer pressure moderates behavior that could stray outside the norm. Use measurement not only as tool to generate information, but also as a tool to shape the environment and channel the corporate culture.
Measurement and measurement programs don’t cause the sin of envy.  People and organizational cultures foster this sin in equal measure. Done correctly, measurement programs can act as a tool to tame the excess that lead to this sin. However the corollary is also true.  Done incorrectly or poorly, measurement ceases to be a positive tool and becomes part of the problem.  Measurement that fosters transparency and collaboration will help an organization communicate, grow and improve.

The Seven Deadly Sins popularized by Dante have been used as an analogy for a fallen man’s tendency to sin and as analogy for the ills or risks of many professions. The analogy fits as well for software metrics; focusing attention on the behaviors that could sap your program’s integrity, effectiveness and life span. This paper looks at a set of deadly sins from the point of view of a person or group that is creating or managing a metrics program. The paper will be delivered as several essays on the Software Process and Measurement podcast (www.spamcast.net).

The Seven Deadly Sins of Metrics Programs:

  • Pride – Believing that a single number / metric is more important than any other factor.
  • Envy – Instituting measures that facilitate the insatiable desire for another team’s people, tools or applications.
  • Wrath – Using measures to create friction between groups or teams.
  • Sloth – Unwillingness to act or care about the measures you create.
  • Greed – Allowing metrics to be used as a tool to game the system for gain.
  • Gluttony – Collecting data for data’s sake.
  • Lust (Extravagance) – Pursuit of the number rather than the business goal.

The impact each sin on a metrics program depends on many factors ranging from management style to organizaional culture. An example can be seen when comparing the belief that one number is more important than all other factors (always) which illustrates the sin of pride compared to the lack of motivation illustrated by sloth. These are two very different issues with two very different impacts however neither should be sneezed at if you value the long term health of a metrics program.

Pride:
The first deadly sin is pride. In the cannon of deadly sins, pride is the sin from which all others spring. In the world of metrics programs, the sin of pride is exhibited when a metrics program settles on a single metric that is used to reflect the value or well being of a project, a group or organization. Examples of abound of metrics programs that fixated on cost or productivity to the exclusion of a broader palette of metrics and work attributes. Most metrics professionals quickly learn that one metric can not be used for all projects. If you can’t easily answer the question, “Why is this true?” each time you use a metric and for each metric you use, the information generated through measurement and analysis will provide little or no value. The goal is to understand the differences between groups of work so that when comparisons are made, you can discern what is driving the difference (or even if there is a difference). Comparing package implementations, hardware intensive projects or custom development is rational only if you understand that there will be differences and what those differences mean. The bottom line is that rarely does a single point of view, a single metric, deliver that deep level of understanding that generates value from measurement.

Another example of the single metric syndrome generated by the sin of pride occurs when an organization uses a single metric to value performance in a contractual arrangement. While entire contracts are rarely stipulated on a single metric it easy for a single metric to be given disproportional weight due to a lack of understanding by the framers or a disconnect between the framers and the people that administer the contract. Poor understanding of the relationship between the numbers and the concepts they represent is akin to failure in punctuation when writing. The resulting meaning can be garbled as the contract is negotiated, implemented and managed. We won’t get into an existential argument over whether something is a sin if inadvertent, the result is the same. Garbled concepts can lead to a single metric focus which once discovered will beg to be taken advantage of. The type of advantage usually causes an overemphasis on a specific portion of the value chain such as productivity being emphasized over time-to-market, quality or cost.