Unlike vanity license plates, calling a measure or metric a ‘vanity metric’ is not meant as a compliment. The real answer is never as cut and dry as when someone jumps up in the middle of a presentation and yells, “that is a vanity metric, you are suggesting we go back to the middle ages.” Before you brand a metric with the pejorative of “vanity metric,” consider:
- Not all vanity metrics are useless.
- Your perception might not be same as someone else.
- Just because you call something a vanity metric does not make it true.
I recently toured several organizations that had posted metrics. Several charts caught my eye. Three examples included:
- Number of workdays injury-free;
- Number of function points billed in the current quarter, and
- A daily total of user calls.
Using our four criteria (gamability, linked to business outcomes, provides process knowledge and actionable) I could classify each of the metrics above as a vanity metric but that might just be my perception based on the part of the process I understand.
The number of workday injury-free is a simple metric I have seen at construction job sites, manufacturing plants and warehouses since I entered the workplace. The number tends to increment over time until it suddenly shifts to zero. By all definitions of a vanity metric, the number shown has little to do with the output of the process and nor is it really actionable. That said, the metric provides workers with the some assurance that management is “interested” in the well-being of their employees or at least want to avoid the fines for not posting the chart. Clearly some vanity metrics are useful.
IFPUG function points are a measure of software functionality delivered. Function points were introduced in the late 1980’s and have evolved over the years. Function points are sometimes perceived as a vanity metric when not used as a system metric. While this might be true in some scenarios, if we consider the common purchasing practice of paying per function point used in several countries (including the US, Brazil, Australia, Korea and Japan to name a few), the metric clearly is linked to a business outcome and is actionable, and therefore is not a vanity metric. The perception who is using the metric clearly impacts how a metric is classified.
I have worked for call centers, help desks and voice credit card authorization organizations during my career. One of the most ubiquitous metric collected and displayed is the total number of calls answered daily (versions of this chart are limitless and include calls per hour and calls during peak hours). During a recent tour of a warehouse call center, one of the people on the tour suggested the metric was purely for the the vanity of the organization and for showing visitors. The tour leader pointed out the metric was used for staffing the call center properly so employees would not burn out and so that customers got their questions answered in a timely manner. I made sure I was not standing next to him for the rest of tour in case retribution for the snide question was required. Calling something a vanity metric is can be related perception; however in some cases it is a knee jerk negative reaction to any form of measurement. Clearly just because someone calls a measure or metric a vanity metric does not mean the epithet is true.
The concept of measurement and metrics in software development is always an interesting discussion. Metrics and measures are a useful tool support empirical processes, such as Scrum used in software development. Measures and metrics provide transparency so that we can inspect and adapt. That does not mean that vanity metrics don’t exist. For example, on my tours i saw a chart that represented the number of user stories completed this year by month . . . for the whole department. I have no clue how the metric could be used and nor did my hosts when I asked what decisions were driven by the data shown. Clearly vanity a metric based on any criteria you might propose.