398137720_d2b6404e4a_b

When driving to work or watching a baseball game, we can observe Newton’s Third Law of Motion, “for every action there is an equal and opposite reaction.” The engine explodes gasoline the engine and transmission makes the wheels turn generating motion.  A baseball player swings his bat and the ball flies from the stadium. Measurement has a similar set of effects. In Agile Metrics: Philosophy we discussed the impact of the effort required for measurement on teams (every hour spent measuring is one less hour building functionality – equal and opposite). Equally as important is the impact of what is being measured on behavior. What is measured signals what is important to the project teams and the organization, therefore influencing both what work is done and how the work is done. All measurement programs must have balance so as not to let one measure create runaway behavior that generates too much of a good think and overwhelms other desired behaviors. To paraphrase the Third Law of Motion: for every measure there must be an opposite measure. A simple four quadrant metrics framework can be used to shape a balanced approach to measurement.  The framework includes four measurement quadrants that push and pull against each other to minimize runaway behavior.

  1. Labor Productivity – Labor productivity measures the efficiency of the transformation of labor into something of higher value. For example, the productivity of a software project would be measured as the amount of software that is produced per hour (or more typically a person-month). The metric is generally represented as an average value for team over a period of time. Productivity is very similar to common Agile metric, velocity which measures how many stories or story points a sprint a team averages. A singular focus on productivity and velocity can push teams toward working more efficiently or can cause teams to cut corners and undervalue quality and value.
  2. Quality – Quality measures the compliance of the function delivered against a standard. This typically assessed by some form of comparison. Testing, for example, is a comparison of functionality against the standard of predicted behavior. Standards can include technical coding standards, security standards, and measures of technical debt (corners cut), customer satisfaction or defects delivered.
  3. Predictability – Predictability measures a team’s (or organization’s) ability to deliver on its commitments. Teams commit to the number of stories (or story points) they will delivery in a sprint and similarly IT departments and even organizations commit to delivering levels of value and functionality their stakeholders. Measures such time-to-market (of which there are many variants), the number incomplete stories or program-level burn-up charts are often used to measure and highlight predictability.
  4. Value – Value measures the worth or usefulness of the functionality (or other deliverable) delivered. Value can be a difficult concept to quantify, especially when the functionality (or any deliverable) does not interface with the ultimate customer. However most projects have gone through a qualification process with some form of cost/benefit analysis. While this process can be very informal in some organizations, someone has gone through a process to rationalize the benefits that are expected from the project or feature they are requesting. A typical measure for value is return on investment (ROI).

A balanced metrics model will help ensure that one measure does not generate unanticipated negative results. For example, an over-focus on quality can cause organizations to spend more time and effort on testing than needed; causing costs to rise and value to diminish. There is no single formula for a balanced set of metrics, each organization will need to generate a balance based on their business context.

Advertisements