Monte Carlo analysis provides a way to handle answering questions with significant uncertainty in the inputs that influence the outcome of the work so you can have the difficult “when, what, and how much” type conversations with sponsors, stakeholders, and marketing people. That definition is an explicit admission that almost ALL of the hard questions asked about projects cannot be answered using simple a + b = c formulas or arguments. This leads us to use tools like Monte Carlo analysis. There are four common assumptions often overlooked or misunderstood when using Monte Carlo methods. (more…)

August 23, 2018

## Monte Carlo: The Assumptions

Posted by tcagley under Agile, Management | Tags: Decision Making, Monte Carlo, Statistics |1 Comment

August 21, 2018

## Monte Carlo Analysis – A Brief Rationale For Use In Agile

Posted by tcagley under Agile | Tags: Agile, planning, Statistics |[2] Comments

One of the most often used saying in agile is that yesterday’s weather is a good predictor of tomorrow’s performance. I have lived in Louisiana where if you blink the weather will change. I currently live near Cleveland (and I like it), and in 2014 the temperature went from 39 F to -11 F in less than 24 hours. I went running on both days; they were very different. Even if I grant that yesterday can be an important indicator of performance tomorrow, a sample size of one does not capture the degree of variability that might be present in the environment. Why does anyone care about variability in performance? As suggested in Agile Metrics: An Interlude, leaders have not stopped asking what they can get, when they can get it and how much it will cost type questions. Even if they don’t ask all of those questions there will be questions about budgets. These are not evil, unagile people; they are business people that need to plan things like cash flows and making payroll. Answering when they can deliver product to the market or when a change to the HR portal will be made are important questions. Just relying on yesterday’s weather is not always sufficient and there is no Oracle of Delphi that provides a single, unambiguous answer to any of those questions. The answers are always a range. In most circumstances, each possible outcome is more or less probable than the next. Uncertainty makes it difficult to have a conversation about when, what and how much. Monte Carlo analysis provides a way to handle answering questions with significant uncertainty in the inputs that influence the outcome of the work so you can have the difficult when, what and how much type conversations. (more…)

January 13, 2018

## Actionable Agile Metrics for Predictability by Daniel S. Vacanti, Re-read Saturday Week 12: Cycle Time Histograms

Posted by tcagley under Re-read Saturday | Tags: Agile, Histrogram, Re-read, Statistics, Vacanti |[20] Comments

Chapter 10A of Daniel S. Vacanti’s Actionable Agile Metrics for Predictability: An Introduction (buy a copy today) is a very short chapter (I should have planned to include this in chapter 10B, but hindsight is 20/20) covering a discussion of histograms but given the time and space we can spend additional time with an example. The chapter is titled *Cycle Time Histograms*. (more…)

November 29, 2016

## Are Vanity Metrics Statistically Valid?

Posted by tcagley under Measurement | Tags: Statistics, Twitter, Vanity Metrics |Leave a Comment

Over the last two weeks, we published three articles on vanity metrics:

Vanity Metrics in Software Organizations

Vanity Metrics Have a Downside!

One of those articles elicited an interesting discussion on Twitter. The discussion was predominately between @EtharUK (Ethar Alali) and @GregerWikstrand. The discussion started by using the blog entry from 4 Big Myths of Profile Pictures from OKCupid to illustrate vanity metrics, and then shifted to a discussion of whether vanity metrics can be recognized based on statistical validity. The nice thing about Twitter is that responses are forced to be succinct. See the whole conversation here. (more…)

February 27, 2016

## How To Measure Anything, Chapter 10: Bayes: Adding To What You Know Now

Posted by tcagley under Re-read Saturday | Tags: Bayes, Measurement, Re-read, Re-read Saturday, Statistics |[10] Comments

Chapter 10 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition is titled, Bayes: Adding to What You Know Now. Here is a summary of the chapter in a few bullet points:

- Prior knowledge influences how we process new information.
- Bayesian statistics help us move from what we know to what we don’t know.
- We are all Bayesian to some extent, but maybe not enough.
- Many myths about using information are just plain wrong, and Bayes proves it.

June 30, 2014

## Measures of Central Tendency

Posted by tcagley under Uncategorized | Tags: Mean, Median, Mode, Statistics |Leave a Comment

Measures of central tendency attempt to define the center of a set of data. Measures of central tendency are important to for interpreting benchmarks and when single rates are used in contracts. There are many ways to measure however the three most popular are mean, mode and median. Each of the measures of central tendency provides interesting information and each is more or less useful in different circumstances. Let’s explore the following simple data set.

**Mean**

The mean is the most popular and well known measure of central tendency. The mean is calculated by summing the values in the sample (or population) and dividing by the total number of observations. In the example the mean is calculated as 231.43 / 13 or 17.80. The mean is most useful when the data points are disturbed evenly around the mean or are normally distributed. A mean is highly influenced by outliers.

Advantages include:

- Most common measure of central tendency used and therefore most quickly understood.
- The answer is unique.

Disadvantages include

- Influenced by extremes (skewed data and outliers).

**Median**

Median is the middle observation in a set of data. Median is affected less by outliers or skewed data. In order to find the median (by hand) you need to arrange the data in numerical order. Using the same data set:

The median is 18.64 (six observations above and six observations below. Since the median is positional, it is less affected by extreme values. Therefore the median is a better reflection of central tendency for data that has outliers or is skewed. Most project metrics include outliers and tend to be skewed therefore the median is very valuable when evaluating software measures.

Advantages

- Extreme values (outliers) do not affect the median as strongly as they do the mean.
- The answer is unique.

Disadvantages

- Not as popular as the mean.

**Mode**

The mode is the most frequent observation in the set of data. Modes may not be the best measure of central tendency and may not be unique. Worse the set may not have a mode. The mode is most useful when the data is non-numeric or when you are attempting to the most popular item in a data set. Determine the mode by counting the number of each unique observations. In our example data set:

The mode in this data set is 26.43; it has two observations.

Advantages:

- Extreme values (outliers) do not affect the mode.

Disadvantages:

- May be more than one answer.
- If every value is unique the mode is useless (every value is the mode).
- May be difficult to interpret.

Based on our test data set the three measures of central tendency return the following values:

- Mean: 17.8
- Median: 18.64
- Mode: 26.43

Each statistic returns different values. The mean and median provide relatively similar values therefore it would be important to understand whether the data set represents a sample or whether the data set represents the population. If the data is from a sample or could become more skewed by extreme values, the median is probably a better representation of the central tendency in this case. If the population is evenly distributed about the mean (or is normally distributed) the mean is a better representation of central tendency. In the sample data set the mode provides little explanative power. Understanding which measure of central tendency allows change agents to better target changes and if your contract uses metrics to determine performance, which measure of central measure you can have an impact. Changing or arguing over which to use smacks of poor contracting or gaming the measure.