In a recent discussion of the concepts of Cost of Delay (CoD) and Weighted Short Job First (WSJF) the used of tables and mathematical formulae led to a discussion of accuracy and precision.  The use of  math and formulae suggest high levels of accuracy and precision even though the CoD and project sizes used were relative estimates. One of the questions posed in the class was whether the application of WSJF, or for that matter ANY prioritization or estimation technique is actuate enough to use in making decisions when the input is imprecise estimated data. Techniques like CoD and WSJF when used to prioritize work are generally based on estimates, rather than ultra-precise measures. Remember the of techniques like WSJF is to generate accurate priorities so that the most important work gets into the pipeline. Prioritization based on frameworks like WSJF require an understanding of the concepts of accuracy and precision and what is required in terms of accuracy or precision when prioritizing work. Often the two concepts are conflated however the two concepts of accuracy and precision are different but interrelated.

Accuracy is defined as how close a measured value is to the true value (www.mathisfun.com) or as freedom from mistake or error (www.merriam-webster.com/dictionary/accuracy). In some instances, accuracy is easy to judge. For example, (Pi) to 10 decimal points is 3.1415926535 (if you are a geek, PI to a million). Whereas Pi calculated to two decimal points is 3.14. Both are accurate. Judging accuracy gets messier when the outcome is unknown, for instance generating an estimate for project prioritization. In order to judge accuracy we generally apply a decision making rule or standard which is typically expressed as a confidence interval or a degree of risk. Another popular alternative is to use a comparison to a relative measure, such an analogy or story points. In order for anything to be perceived as accurate it has to be within the standards set so that the organization can have confidence as they prioritize work.

Precision is a reflection of exactness. In the example of Pi, the 3.14 is accurate but not precise. Calculating Pi to a million decimal points increases the level of precision but doesn’t make the calculation precise. PI is an irrational number we will never be able to calculate PI precisely. In software development, most activities required to deliver business value have some level of variability in performance. Variability makes precise prediction difficult or impossible. Estimates of software development are imprecise by nature. Imprecision driven by the variability in the processes and by the inability to know all the factors that will be encountered makes precision difficult at best. Precision is typically generated by padding the estimates, cutting or adding scope so that the outcome matches the estimate, or in some cases fibbing about the results.

Given effort and consternation that often are often taken to generate false precision, a better avenue is to define an acceptable level of precision so that accuracy is an artifact of doing the work rather than an artifact of controlling how the work is measured. For example, estimating that a piece of work will be done at 2:33.06 PM EST takes far more effort than estimating it will be completed this afternoon.  Both estimates could be equally accurate although the later has more chance at accuracy. Techniques like CoD and WSJF when used to prioritize work are generally based on estimates, rather than ultra-precise measures. The goal is to generate accurate priorities so that the most important work gets into the pipeline. Like many things in Agile and other iterative frameworks, just because a piece of work is not at the top of the list today does not mean its priority can’t increase when the list is reassessed. Organizations have to decide what level of precision is really needed and whether they are willing to trade precision for accuracy.

This weekend I am going to run the 35th Annual St. Malachi Church Run in a time of roughly fifty minutes. The race is five miles long and the forecast is for a very cold rain. In this case, my estimate is probably accurate (time will tell), but not precise. Last year, I actually ran the race in 49.06.36 minutes. The measured results are very precise and are accurate within the tolerance of the measurement system used (chips in the race bib). While I feel faster this year, but I am a bit heavier and a bit older which increases the possible variability in my speed. In terms of setting expectations, I think I would rather be accurate than precise so that my support team can meet me with the umbrella without have to stand in the rain too long themselves.

PS March 14th is Pi Day – have an irrationally good day!


Aging is good for scotch, not for software development.

Aging is good for scotch, not for software development.

Lean is the systematic removal of waste within a process, known as muda in Japanese.  Delay is often one of the predominant forms of waste in any process. Cost of delay (CoD) is a method of placing a value on the waste inherent in delay. In manufacturing or other industries, the cost of delay is often obvious. Simple examples of the CoD might include the cost of warehousing incomplete items or holding stock in that room of a store that is waiting where it can’t be sold. In software development the concept CoD is not as easy to see, but no less real. In software development (including development, enhancements and maintenance) CoD is generally conceived as a measure of the reduction in value caused as work sits when it is not completed. Delay is the time a unit of work is not being actively being worked on. For example, if it takes a day to write and unit test a piece of code and then that piece of code has to wait a month to be tested then delay is measured as one month. The total delay can be determined by summing total time the unit of work spends sitting around until it gets to a point where the final user of the software can actually use the it! Conceptually CoD is similar to financial techniques like Return on Investment and Net Present Value (NPV). NPV normalizes benefits and costs in the future based on a discount rate (such as the presumed interest rate over the period or the organization’s internal rate of return). Simply put, a dollar invested at ten percent interest today will be worth more in a year than the dollar that sat waiting to be approved to be invested for 364 days. The difference in value is the cost of delay. Delay can generate cost many ways including:

Delay to gaining value: Until a unit of work is delivered, it can’t be used. For example, when we discussed Test Driven Development we define the following user story:

As a beer logo glass collector, I want to be able to log a logo glass into my collection so I do not recollect the same glass!

If this story was completed, but implementation of the story was delayed a week and I bought a duplicate glass for$5.35 (including sales tax) during the delay, the tangible cost of delay would be $5.35.

Opportunity costs: In the example above, if I had the $5.35 in my pocket I could have done something else with it. While this might be considered an intangible cost, if the $5.35 had been invested in another value generating asset or feature, the cost of delay can be quite real.

Value of a feature decays over time: In most organizations being first to market with an app or feature is viewed as valuable. The value is often couched in terms of market share or the ability to charge a premium. Being late to market can often be difference between success and failure.

The concepts of delay and the cost of delay generates a focus on queues (where work sits while waiting to be worked on) and speed to delivery. In general, the faster a story or feature can be delivered, the faster the organization can begin to accrue the inherent value. Understanding the CoD provides data to understand the trade-offs between starting one thing and then waiting to deliver and starting and completing something else instead. If we assume that the work we are doing has value if it is completed then then letting it sit around and age generally doesn’t make sense unless we are making cheese or wine.