Vanity metrics are not merely just inconvenience; they can be harmful to teams and organizations. Vanity metrics can elicit three major categories of poor behavior.
- Distraction. You get what you measure. Vanity metrics can lead teams or organizations into putting time and effort into practices or work products that don’t improve the delivery of value.
- Trick teams or organizations into believing they have answers when they don’t. A close cousin to distraction is the belief that the numbers are providing an insight into how to improve value delivery when what is being measured isn’t connected to the flow of value. For example, the organization that measures the raw number of stories delivered across the department should not draw many inferences in the velocity of stories delivered on a month-to-month basis.
- Make teams or organizations feel good without providing guidance. Another kissing cousin to distraction are metrics that don’t provide guidance. Metrics that don’t provide guidance steal time from work that can provide real value becasue they require time to collect, analyze and discuss. On Twitter, Gregor Wikstrand recently pointed out:
@TCagley actually, injuries and sick days are very good inverse indicators of general management ability
While I agree with Greger’s statement, his assessment is premised on someone using the metric to affect how work is done. All too often, metrics such as injuries and sick days are used to communicate with the outside world rather than to provide guidance on how work is delivered.
Vanity metrics can distract teams and organizations by sapping time and energy from delivering value. Teams and organizations should invest their energy in collecting metrics that help them make decisions. A simple test for every measure or metric is to ask: Based on the number or trend, do you know what you need to do? If the answer is ‘no’, you have the wrong metric.