Metrics Minute: Return on Investment (ROI)
Thomas M. Cagley Jr.
Audio Version: SPaMCAST 163
ROI is a standard measure of project profitability that is the discounted profits over the life of the project expressed as a percentage of initial investment. ROI is a classic financial metric that when applied to projects, is typically used as a technique for project acceptance or, in retrospect, as a tool to evaluate overall performance. When applied in agile projects, ROI can be used to determine priority of epics, themes or even stories.
ROI = (Average Net Benefits) ÷ (Initial Costs)
Initial Costs includes any expense related to the initial development of the project.
The primary use of ROI is to compare a potential project to other possible projects in order to determine which makes the most sense to pursue (like Return on Assets – ROA). Projects with a higher ROI are more likely to be included in the portfolio of projects. When applying ROI to agile projects I suggest at the very least using ROI as a tool to prioritize epics and features into a release plan.
Alternately ROI can be used as a tool to evaluate continued investment in a product or a portfolio of products. In this case the metric would be periodically recalculated to determine whether an overall project portfolio represents an efficient use of assets or if segments of the portfolio offer high a higher return.
There are several issues with the calculation and use of ROI as a decision tool including poor math, failing to make an apples-to-apples comparison or in extreme cases the combination of poor math and bad comparisons. The most significant issues are:
Return on Investment is a financial tool designed to help build a business case to support decision making by evaluating the forecasted impact to your organization’s bottom line. ROI or any other financial analyses should not be your only evaluation criteria. High ROI’s may be an artifact of wishful thinking or creative accounting for benefits. Rigorous follow-up helps stop this type of behavior.
A second issue is failing to account for the time value of money. The time value of money suggests that a benefit or cost today has a higher absolute value than the same value or cost in the future. Techniques like Net Present Value normalize benefits and costs in the future based on a discount rate (such as the presumed interest rate over the period or the organizations internal rate of return).
Comparisons of factors like ROI for two projects must be across similar timeframes so that an apples-to-apples comparison can be made. In environments that are rapidly evolving shorten the comparison period to favor projects that deliver benefits earlier rather than later. The comparison problem can be exacerbated when used for the smaller work packages usually seen in agile projects. Gross value may be more easily determined for work packages like stories.
A final issue is that intangible benefits may affect the rational for project selection which forces organizations to address valuation of intangible benefits. This issue is a relative of the “numbers don’t tell the whole story” argument. As in the discussion of comparing work items above, as the work packages become more granular, it may become more difficult to accurately evaluate intangible or non-business value in order to calculate ROI. Create rules for how you will allocate or value intangible benefits and costs. Valuation of intangible benefits requires a judgment call and unless that judgment is based on rules or guidelines those judgment calls will be open to question or manipulation.
- Return on Equity (ROE)
- Return on Assets (ROA)
- Payback Period
- Internal Rate of Return (IRR)
- Net Present Value
- Total cost of ownership
Criticisms of ROI, like the issues noted earlier, revolve around how the metric is used.
This first criticism is that comparisons across different types of projects are difficult due differing cost and benefit structures that effect ROI. An example of problems that occur when comparing differing types of projects would be making an investment decision about an infrastructure project versus a project that generates revenue. It is usually more difficult to identify and monetize the benefits of an infrastructure project which makes these types of projects more difficult to fund even though they may be required to enable projects with a more classic benefit stream. This criticism is true; however, I would suggest this is a failure to enforce a linkage between infrastructure projects to the impact on revenue streams.
A second criticism: Valuing intangible costs and benefits is at best an inexact science and is open to internal political games. This criticism is the most troublesome. There isn’t general agreement on how to value the intangible. I would suggest not using a one-size-fits-all solution; rather I would suggest ensuring that valuations of intangible cost or benefits are consistently evaluated based on a risk adjusted discount rate. The higher the risk, the higher the discount rate would be set. A formal approach to measuring or estimating risk will be required to avoid manipulation of the analysis.
A third criticism of ROI centers on using a single point in time to qualify projects. The calculation of ROI generally reflects a snapshot of at particular moment in time. Unless you live under a rock it is difficult to forget that the business environment is – - – dynamic. The idea that an ROI calculation can be done and evaluated once to ensure that the organizations portfolio of projects continually maximizes value is ludicrous. To deal with the issue I suggest periodic revaluations of the portfolio to ensure that the business environment does not change enough to require changing the composition of the project portfolio. Note: I would suggest that there is a need re-evaluate projects more often if they in the gray area between acceptance and rejection.