Is value limited?

Organization or team decides to embrace agile and lean techniques for many reasons. Arguably there may be a disconnect between the reasons why people say they are addressing a method and the real reason.  How people are incented (paid or bonused) and how people are measured might be more a accurate representation. For example, if product cost in important to an organization, measures will have a systems perspective; however, if cost is important without a systems perspective the focus may be on the cost of components or individual components. The nine of the most commonly mentioned reasons for adopting agile are all can be measured however, some attributes are more or less difficult to measure. Common metrics include (note we will come back to how useful each is in the next entry in this series):

Reasons for Adopting Agile That Are Easier to Measure 

  • Lower Cost – The reduction of the cost of work is often sighted as a reason for transforming to agile or lean.  Cost is typically a function of labor, overhead and other inputs (inputs are a function of what you are building).  In a software product, such as SAP or Microsoft Word,  labor tends to the largest cost component. Measuring the impact on cost will focus on the part of the life cycle the organization is interested in.  The best cost metrics will focus on the entirety of the life cycle but will be able to drill down to a more granular level. Common metrics used to track the change in cost for work include:
  1.      Cost per unit of product delivered 
  2.      Return on Investment (most of the R’s are appropriate)
  3.      Cost per function point
  4.      Cost burn rate per hour (day)
  5.      Budget burn rate 
  • Faster Completions – Speed is a critical attribute in many organizations.  Agile is often sold with the promise that projects will get done faster due to higher levels of focus and timeboxes (to name a few reasons).  We won’t debate the truth of this promise just the need to have data to prove or disprove the goal.  It is usually true that the faster a product, enhancement or defect can be address, the better. Faster completions are almost always a measured using calendar time. Common metrics used to track the change in speed of completion for work include:
  1.      Cycle time 
  2.      Concept to cash
  • More Frequent Deliveries – Speed often comes in two flavors.  When will the whole piece of work be done (above) and when will pieces be available to use in production.  One of the major premises of agile is that the business will get pieces of functionality faster even if the whole piece of work takes the same amount of time. A common metric for determining if the frequency of delivery is changing is:
  1.      Time between production implementations.
  • Predictability – Timeboxes and breaking work into smaller chunks ( for example breaking work down from feature to epic and finally user stories) are agile techniques that are useful for increasing predictability. Measures of predictability indicate whether the promises made by teams or organizations are apt to be met.  Regardless of method product owners and other business personnel continue to ask: “when will the work be done and what will it cost?” Common metrics for determining if teams are predictable or becoming more predictability include:
  1.      Planned to actual user stories
  2.      Story escape Rate
  3.      Cycle time
  • Increased Quality – Increased user involvement, collaboration, test driven development and continual grooming of user stories are some of the techniques in agile that can improve quality. Quality impacts how fast a team can go and whether customers are satisfied because they get what they want and does not need repair. Metrics that are used to monitor quality include:
  1.      Delivered Defects 
  2.      Satisfaction survey
  3.      Cycle time
  4.      Cost per function point
  5.      Rework

Reasons for Adopting Agile That Are Harder to Measure 

Attributes are generally harder to measure if their performance is not directly tied to a tangible goal.  For example, transparency is often difficult to measure because the definition is nebulous (not tied to behaviors) and not tied to tangible goals. The measures for these attributes are typically more of a proxy and which metrics are useful in any specific situation will depend on why you are doing agile.

  • Transparency – One of the three pillars of Scrum is transparency. Agile works best when everyone has the same understanding of agendas, policies, conditions, and decisions. available to everyone involved in delivering value (one definition of transparency).  It is generally thought that transparency impacts quality, time to market, retention rates, and satisfaction.  The measures used to assess transparency include:
  1.      Satisfaction Surveys 
  2.      Productivity
  3.      Rework
  4.      Turnover rates
  5.      Cost per unit of work
  • Business and Customer Focus – Agile principles call for the intimate involvement of business and customer personnel. Scrum includes the role of the product owner to ensure daily interaction with the business.  Measuring business and customer focus reflects whether the team is delivering what the business is asking for.  This metric can be a double-edged sword; for example, if the team only does customer facing work it is possible that the critical non-functional or architectural work may be ignored.  When interpreting measures of focus, the balance the organization wants needs to be kept in mind. Metrics providing feedback on business and customer focus include:
  1.      Involvement 
  2.      Comparison of business priorities to what is being delivered
  3.      Satisfaction surveys
  • Engagement – Agile breaks down the boundaries between teams and the business so that everyone feels like they are pursuing the same goal which is reflected in engagement. Measures of engagement need to account for proper involvement between teams, stakeholders, and customers.  Working collaboratively is expected to increase satisfaction, productivity while lowering cost, cycle time and turnover-rate. Metrics to track engagement include:
  1.      Involvement
  2.      Satisfaction Surveys
  3.      Productivity
  4.      Rework
  5.      Turnover rates
  • Flexibility – Agile touts the flexibility to address changes in business needs late in the process.  Several studies have linked flexibility to team autonomy and diversity. Flexibility improves market success, speed to market (cycle time) and the functionality delivered.  Stated differently, how well can a team(s) change direction to deliver what is required. 
  1.      Productivity
  2.      Sales or ROI
  3.      Cycle time
  4.      Satisfaction surveys

Each of the common reasons that teams and organizations can be measured either directly or by proxy.  Some of the metrics that are better than other metrics in providing “proof” and others for delivering information to the teams and organization.

Metrics and whether they are useful from the team’s perspective next.