Before we dive into Chapter 15 (two chapters left, and chapter 17 is a case study!), I want to alert you to the poll at the end of this entry. It is time to choose the next book in the re-read series. Feel free to start a write-in category.

Chapter 15 is titled, *Monte Carlo Methods Introduction* in Daniel S. Vacanti’s Actionable Agile Metrics for Predictability: An Introduction (buy a copy today). FYI, I interviewed Daniel Vacanti and will post the interview at the conclusion of the re-read (it was wicked cool).

Monte Carlo simulations are a mechanism for solving complex statistical problems. In a Monte Carlo simulation or analysis, a problem is solved many times (hundreds or thousands of times). Vacanti points out that the process can be thought of as a set of experiments with random numbers. The range of the random numbers is bounded by how variable your process is (low variability = predictable). A large number of observations (experiments) provides the person running a simulation with the range of possibilities and the probability of outcomes.

Monte Carlo simulations use mathematical modeling techniques to account for uncertainty (risk) to derive the range of possible outcomes and the probability of each outcome. This injects the idea of a range of possible outcomes into the conversation. The output of a Monte Carlo simulation shows the range of outcomes and the probability of each possible outcome. Monte Carlo simulation is a powerful tool to provide an understanding of the impact of the flow principles and metrics and how they make Monte Carlo simulations more actionable.

Vacanti provides suggests two pieces of advice:

- If you have data, use your own data. If you don’t have data use someone else’s data and/or spend the time to measure your own process. (My addition to Mr. Vacanti’s advice: if you use someone else’s data make sure you start collecting your own and make sure the results you are getting make sense for your process and culture) If you do not have your own data you will not be sure whether your process is stable, which will make it difficult to trust and use the results of the analysis. Simply put, if your process is not stable (predictable), Monte Carlo might not be a great tool for your situation.
- Be keenly aware of the assumptions that the analysis is built on. For example, the analysis might assume that if there is available WIP that work will begin on the next item immediately. The real life pull policy might be different. For example, a recent process I observed required specific lab equipment to restored to an initial state before another round of tests could begin. The person that does the restores lives several time zones away from the rest of the team and does not check the restore requests when sleeping (note – this is a huge process improvement possibility, but there are others that are even more urgent). The vagaries of the process was not built into the simulation making forecast the movement of work problematic.

The validity of the output of the Monte Carlo analysis will be in direct proportion to the stability of the process and how well you are meeting the assumptions (this is true for ANY model). Vacanti states,

“The model must mirror the real world policies and procedures that you’re using to drive your work.”

Monte Carlo simulations are a forecasting tool. The analysis presents users with a range of possible outcomes and the probability of each possible outcomes — otherwise known as forecasts! Forecasts are predictions of the future. All predictions of the future are uncertain. The cycle time metric coupled by a scatter plot is a good mechanism to support making predictions, adding Monte Carlo analyses to your toolkit will make your data more actionable.

**The poll for the next re-read! **

**Previous Installments**

Week 2: Flow, Flow Metrics, and Predictability

Week 3: The Basics of Flow Metrics

Week 4: An Introduction to Little’s Law

Week 6: Workflow Metrics and CFDs

Week 8: Conservation of Flow, Part I

Week 9: Conservation of Flow, Part II

Week 11: Introduction to Cycle Time Scatterplots

Week 12: Cycle Time Histograms

Week13: Interpreting Cycle Time Scatterplots

Week 14: Service Level Agreements

Week 16: Introduction to Forecasting

Support the author (and the blog), buy a copy of Actionable Agile Metrics for Predictability: An Introduction by Daniel S. Vacanti

Get your copy and begin reading (or re-reading)!

February 18, 2018 at 3:53 am

Vacanti makes a strong case for using Monte Carlo simulation to predict the future for software development teams; software estimates typically are guesses made under pressure that are not very close to what ends up actually happening.

“Given the uncertainty in knowledge work it seems strange that our industry has been rather late to the Monte Carlo game.” (p. 249)

If we can create a stable process and collect the flow metrics, as Vacanti has explained in the preceding chapters, then we can use that data to perform Monte Carlo simulations that will accurately forecast the future in a probabilistic way.

—

Note: stable means NOT violating the 5 assumptions behind Little’s Law very often.

February 18, 2018 at 10:10 pm

[…] Week 17: Monte Carlo Method Introduction […]

February 25, 2018 at 10:11 pm

[…] Week 17: Monte Carlo Method Introduction […]

March 3, 2018 at 11:56 pm

[…] Week 17: Monte Carlo Method Introduction […]

March 4, 2018 at 10:12 pm

[…] Week 17: Monte Carlo Method Introduction […]

March 10, 2018 at 11:57 pm

[…] Week 17: Monte Carlo Method Introduction […]

March 11, 2018 at 9:08 pm

[…] Week 17: Monte Carlo Method Introduction […]

March 20, 2018 at 8:55 pm

[…] Week 17: Monte Carlo Method Introduction […]

May 7, 2019 at 5:47 am

[…] Week 14: Service Level Agreements Week 15: Pull Policies Week 16: Introduction to Forecasting Week 17: Monte Carlo Method Introduction Week 18: Getting Started Dead Tree Book […]