As we have noted in past entries, effective measurement is a balance. The five Agile Portfolio Metrics Categories as whole represent a pallet from which an organization can craft a balanced set of Agile metrics. The goal of any measure is to reduce the uncertainty the organization is facing. Value and portfolio health metrics shift the discussion from the high-level management of the distribution of work (who is doing what for whom) to the value of work (value metrics) and how the activities across a portfolio of work are perceived (portfolio health metrics). Each category is targeted on its own set of specific questions, but each category can and often does influence another category. We break down the second two categories below.
Value metrics provide portfolio managers and the organization with feedback on the economic value of the work forecasted or delivered from the portfolio.
Basic Focus/Question: How much value for money is an organization getting from a specific portfolio? Alternate versions of the question include: Is the amount of value delivered projected to change? Is organizational spending focused on delivering value?
- Value for Money is a high-level metric that accounts for cost, benefits, and the mix of resource utilization, quality, satisfaction, and timeliness. This metric is also called E3 (economy, efficiency, and effectiveness).
- Time to Market measures the speed work moves from an idea to production. This measure is always denominated by calendar time, but the numerator can either be value or size depending on the question the organization needs to answer.
- Concept to Cash is a variant of the time to the market metric used in SAFe and Lean circles that measures flow across the life cycle from ideation to delivery of value. The focus of this metrics is to maximize the flow rather than individual steps.
- The R’s (ROI, ROA, and IRR) are all financial metrics that show the return on the output of a project. Each metric provides a variation on a common theme with the difference depending on whether investments (ROI), assets (ROA) or the internal hurdle rate (IRR) is chosen.
Note: Value is easily the most talked about measure in Agile, but one of the least measured. There are a variety of reasons (or rationalizations) for why it is not measured. The focus of most of the reasons is on the difficulty of measuring return or value. All the reasons are rationalizations (except for the person that told me once, “I just don’t want to measure value”) that we will tackle in the future. In the interim call if you would like to discuss the topic.
Portfolio Health metrics provide feedback on how satisfied stakeholders and team members are with the work performed. This category is often a leading indicator of future delivery problems. Satisfaction often declines and risk tends to increase before the delivery problems.
Basic Focus/Question: Are the stakeholder’s satisfied with the service and value they are receiving? Are team members satisfied with the work and work environment? Alternate versions of the question include: Would our stakeholders or client recommend us? Is the portfolio value at risk?
- Satisfaction is a measure of how products, services, behaviors or work environment meet expectations. Product or customer satisfaction is typically measured after a release or on a periodic basis. Many teams measure team satisfaction using Agile techniques (for example: Fist-to-Five) on a daily basis.
- The Net Promoter metric is an advanced form of a customer satisfaction measure. The metric is calculated as the percentage of detractors subtracted from the percentage of promoters. The larger the promoter value remaining the higher the chance your work will be recommended.
- Value at Risk represents the potential impact of risk (if were to become an issue) on the value of a project or portfolio of projects.
- Team Engagement is the measure of the attributes that drive the relationship between an organization and the team. A highly engaged team will take more positive steps to achieve the team’s goals than a disengaged team.
Measuring a portfolio requires balance. Part of getting the balance right is to commit to the asking the right questions and then measuring the right things so you can gather the information you need. Value and portfolio health are part of a balanced perspective of your portfolio.