How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 6 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Quantifying Risk Through Modeling. Chapter 6 builds on the basics described in Chapter 4 (define the decision and data that will be needed) and Chapter 5 (determine what is known). Hubbard addresses the process of quantifying risk in two overarching themes. The first theme is the quantification of risk and the second is using the Monte Carlo analysis to model outcomes. (more…)


How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition

Chapter 4 of How to Measure Anything, Finding the Value of “Intangibles in Business” Third Edition, is titled: Clarifying the Measurement Problem. In this chapter Hubbard focuses on two questions.  The first is getting the reader to answer what is the decision that measurement is supposed to support. The second is what is the definition of the thing being measured in terms of observable consequences.  These questions sound very basic; however, I found myself asking variations of those questions more than once recently when reviewing a relatively mature measurement program.  Answering these questions are often at the heart of defining the real mission of any measurement or measurement program and whether a measure will have value. (more…)

Listen Now

Subscribe on iTunes

Software Process and Measurement Cast 344 features our conversation with Susan Parente.  We talked about Agile risk management. Risk is not always discussed in polite Agile circles however Susan suggests that if you do not have a plan to address risk you are asking for pain for yourself and everyone around you.

Susan’s Bio

Susan Parente is a Principal Consultant at S3 Technologies, LLC and an Associate Professor at Post University. She is an author, mentor and teacher focused on project and risk management. Her experience is augmented by her Masters in Engineering Management with a focus in Marketing of Technology from George Washington University, DC, along with a number of professional certifications. Ms. Parente has 16+ years’ experience leading software and business development projects in the private and public sectors, including a decade of experience implementing IT projects for the DoD.

Contact Data:

Email: parente@s3-tec.com

Phone: 203-307-5246

LinkedIn: https://www.linkedin.com/in/susanparente

Risk Management Resources: www.techriskmanager.com

Company website: www.s3-tec.com

Agile Risk Management LinkedIn Group:


Call to action!

Reviews of the Podcast help to attract new listeners.  Can you write a review of the Software Process and Measurement Cast and post it on the podcatcher of your choice?  Whether you listen on ITunes or any other podcatcher, a review will help to grow the podcast!  Thank you in advance!

Re-Read Saturday News

The Re-Read Saturday focus on Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement began on February 21nd. The Goal has been hugely influential because it introduced the Theory of Constraints, which is central to lean thinking. The book is written as a business novel. Visit the Software Process and Measurement Blog and catch up on the re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Next . . . The Mythical Man-Month Get a copy now and start reading! We will start in four weeks!

Upcoming Events

June 9 – 12
San Diego, California
I will be speaking on June 10.  My presentation is titled “Agile Estimation Using Functional Metrics.”

Let me know if you are attending!

Also upcoming conferences I will be involved in include and SQTM in September. More on these great conferences next week.

Next SPaMCast

The next Software Process and Measurement Cast will feature our will essay on Cognitive Bias.  The core of software development, enhancements and maintenance is people. Knowledge of cognitive biases can help us understand and predict team behaviors. Will will also have the first installment Jeremy Berriault’s QA Corner.  QA Corner is all about testing.


Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Risk management is like sorting out the cords.

Risk management is like sorting out the cords.

As we have noted, the difference between the classic and Agile approaches to risk management boils down to a few serious dichotomies. The first is that classic methods tend to be project-manager driven, while Agile processes involve and hold the entire team responsible. Second, Agile risk identification and management is built on the continuous re-planning process that is intrinsic to Agile rather than being event/review driven (e.g. phase gates or defined review cycles) which is more the norm in classic project management. All projects need to expend time and effort on dealing with regardless of the processes used to identify, monitor and manage risks. That time and effort means there will be less time to address the functionality requested by the product owner and product management. This leads project personnel to try to balance the effort they spend on the risk management process AND to only focus on the risks that really matter.

Lean risk management processes, such as the one we have described, focus on minimizing the effort needed to identify and manage risks by integrating risk management into other processes. Examples include using the definition of done to mitigate risk and the product backlog to document risks as user stories. Risk management does require specialized meetings and deliverables that increase the perceived overhead. Building risk management into day-to-day activities has secondary benefit of creating team level involvement which is useful for reinforcing the risk management process.

One of the common features of mature risk management processes (whether they reflect waterfall or Agile methods) is risk prioritization. We have explored several mechanisms to evaluate the probability the an risk will become an issue and the potential impact of the issue (Agile and Risk Management: Prioritization Techniques, Part 1 and Agile and Risk Management: Prioritization and Measurement Techniques, Part 2). In all cases the goal of the processes is to consistently prioritize risks so that teams and managers spend their time on the risks that really matter. I recently chatted with a project “risk manager” while waiting for a table at a restaurant. He suggested that he often sees projects without formal prioritization techniques spending precious time and effort on worrying about risks that they can’t influence or have a nearly zero chance of happening, but sound scary. Every erg of energy and every minute spent on risks that are not relevant is waste and provides fodder for those that see risk management as a waste of space.

One common complaint about risk management is that we can never anticipate everything; there are unknown unknowns. The conclusion some practitioners make is to abandon planning and to just stay vigilant. The argument is the effort for risk management is not worth the return. This approach might work, however I have not seen it work on any sizable project. Coupling a lean risk management process with Agile risk management maximizes the value from risk management. This morning while listening to the Gist podcast, Mike Pesca (the host) stated that worrying about the future is an important survival mechanism.  True but that survival mechanism doesn’t require a 100-page risk register that no one will ever look at to be effective.

You can't capture risk with your camera. You need to have a conversation with a diverse group of stakeholders.

You can’t capture risk with your camera. You need to have a conversation with a diverse group of stakeholders.

At a recent Q&A session I was asked: where could a person get their project risks? I stifled a smart-alecky answer that would have included driving to the grocery store, and decided that the question that was being asked was really: how do I go about recognizing and capturing risks? Perhaps a more boring question, but far more important. If I answered the first question the answer would have been that risks are generated by the interaction of the project with other projects, applications, the business, technology and world (risk categories) – pretty much the existence of a project could be considered a risk magnet. The answer to the second question is that once you have a risk magnet (a project) you will need to ask as many different people as is feasible to recognize the possible risks. The discussion of risk always appropriate, however the typical meeting/events and the types of people to consider in the conversation need to be planned. The discovery process typically follows the requirements/user story discovery process outlined below.

  1. Carve out time when you are developing the backlog and ask as diverse a group as possible to identify the potential problems that could get in the away of delivering the value promised by the project. Prompt the group to consider business, technical, operational and organizational factors. Diversity is incredibly important to inject different perspectives, so that the team does not fall prey to only seeing the risks they expect (a form of cognitive bias).
  2. Form a small team (consider the Three Amigos) to interview stakeholders that were not part of the planning exercise. Explain the project and use the same category prompts to generate a risk discussion.
  3. Gather risk data though surveys when the program stakeholders are geographically diverse. (Note: I have only seen this used well in very large programs with professional market research staffs)
  4. Interview customers or potential customers. Customer interviews are not generally used as a standalone risk discovery tool, but rather as a tool to gather requirements/user stories. However, piggybacking a few questions to solicit potential risks is useful to add a diversity of thought to risk identification.
  5. Periodically ask about risks either as an agenda item or as a follow-on to standard meetings. For example, I have seen teams that have successfully added a five-minute follow on to the last daily stand-up of the week in order to consider risks. A quick risk recognition session can easily be added to other standard meeting many projects have. Other standard scrum meetings that can be used to identify risks include: demonstrations, retrospectives and sprint planning. Each of these meetings will provide a different perspective on the project and the team therefore could expose other potential risks.

The baseline answer to the question of how can I recognize and capture risks is by involving all of the projects stakeholders in a discussion of potential risks. The process of collaborative discussion will help increase diversity of thought, reducing (but NOT eliminating) the potential number of unknowns – unknowns that could impact the projects ability to deliver value.

Lava is not a risk category that is commonly encountered in software development.

Lava is not a risk category that is commonly encountered in software development.

I have recently been asking software development practitioners (including enhancements and maintenance) why they think that risk management is not consistently being practiced or practiced well. The group includes methodologists, developers, project and program managers, testers and test managers, operations personnel and executives. Very few have indicated that they thought that risk management was top of mind for any project, other than on an occasional event basis, or as one executive stated “when a project is asked publicly about risks.” The most common reasons why risk management is a second-class citizen include:

  1. There are too many other things to do directly related to delivering functionality. When asked why some project managers are perceived not to see the value in risk management, one respondent answered “They can’t when they’re working 12 hour days to keep the critical path moving.” This is a fairly common point of view. First, when you’re trying to do a twelve-hour job in an eight-hour day you will cut corners (add in multitasking and a disaster looms). Building risk management into the flow of work so that it isn’t a separate process with a separate risk plan and risk register should reduce the overhead generated when risk is an add-on to day-to-day project activities. A lean risk-management process will incorporate risk activities into team activities, but what needs to be addressed to mitigate the problem is the growing insistence on 50 – 60 hour workweeks.
  2. Risk management processes are driven by a need for an external certification. Many if not most of the people I talked with had a risk process, and if pressed could find a list of risks. However those processes and deliverables were developed for auditors and appraisers. In many cases the linkage between external frameworks, an organization’s expectations and policies (specifically for risk) have not been clearly linked to project outcomes. This is a common problem when teams are early in the adoption of Agile. Coaching is generally required to help teams develop the understanding that they will be able to deliver more value when then spend some time considering risks that could impact their ability to deliver. Coaches need to spend the time needed to help the team, organization or project manager to see the linkage between avoiding real issues and delivering value.
  3. Common risks are continually identified and nothing is done about those risks. Continually identifying environmental or long-term organization cultural issue that can’t be solved by team or the IT department is debilitating. In many cases what is being identified are issues that need to be mitigated or planned around. The example used above of project managers working 12-hour days to keep projects on track is not a risk, but an issue. Everyone involved needs to help address the problem the behavior will cause, unless the organizational culture can be changed and that is not something totally up to a team. Techniques such as sharing roles (team level self-organization) can be helpful to mitigate these types of issues. This is similar to issues seen in retrospectives that continually focus on issues that are not solvable (for example, a team that wants to work from home, but corporate policy forbids it due to security reasons). Over time a team will begin to feel helpless and will reject process. This, very simply, is a training and coaching problem.

One person noted that since risk management was not directly mentioned in the original book on Scrum, it must not be very important. I believe the comment was meant sarcastically, however it does make the point that risk management is only an issue in large projects or when project or program managers are involved. Risks and issues plague every human endeavor large or small, and must be addressed for effective and efficient delivery. Overburdened, multitasking teams need to be addressed as a rule. Incorporate a lean risk process into standard Agile or waterfall processes. Risks are just a different kind of user story or requirement that can be addressed in the common flow of work.

Risks coming in many categories!

Risks coming in many categories!

Some risks are fairly common. They pop up either as user stories, on risk logs or in conversations with team members. All risks need to be recognized, but not all risks are within the team’s ability to control or even influence. What a team can influence is how it reacts if a risk becomes an issue. Understanding common categories can help a team consider the risks they may face and whether they have considered everything they need to consider. A common taxonomy of risk categories is:

  1. Business Risk – Markets change due to changing tastes, regulations, and new discoveries, to name a few reasons. This is typically a difficult category of risk for technical teams to recognize, and is often recognized as changes and additions to scope. Inclusion of business product owner is useful to help a team to understand why change is needed and to provide leadership through prioritization. The groomed, managed product backlog is an active mitigation tool for this type of risk, HOWEVER business risk will exist for all projects.
  2. Technical Risk – Technology, like the business environment, evolves to generate risk. New projects that will require architectures and environments to be defined and implemented are fraught with risk (which is a reason many people buy commercial solutions rather developing their own). Maintenance and enhancement projects typically have the lowest amount of technical risk. Technical teams are generally adept at recognizing technical risks but are less adept at seeing when technical risks becomes an issue due to cultural biases. Agile techniques, such as short development cycles (sprints and program increments), test-driven development and continuous builds and integration, help to reduce technical risk.
  3. DevOps/Operational Risk – Many development teams and methods only play lip service or wait too late to consider how their software will fit within the organization’s operational environment. The organization’s operational ecosystem often includes operations, help desk personnel, IT personnel and business support personnel, such as training. Developing and testing a solution that no one can run or support is nearly meaningless. Barriers between organizational groups complicate recognizing and mitigating this category of risk in hierarchal organizations. Including DevOps personnel in project teams, in planning sessions (SAFe Release Planning, for example) and as active participants in demonstrations are mechanisms for reducing the possibility of this type of risk becoming an issue.
  4. Process Risk – Teams adopt and adapt techniques for delivering value based on organizational standards, environmental constraints and the problem they are trying to solve. Given that all projects have at least some unique characteristics, it is possible that the team’s common technique or the technique chosen may not be a match, causing risk. Consistent and honest retrospectives are useful for mitigating this risk and for minimizing the impact of process risks that become issues.
  5. Organizational/People Risk – Projects operate in an environment populated by people with all the possible caveats and footnotes that entails. The organizational/people risks can include personnel changes, dysfunctional office politics, conflicts for staff, multi-tasking and many others. Teams typically become blind to many of these risks believing that they can deal with these issues. Embracing stable teams and the Agile principles of self-organizing and self-managing teams are mechanisms for mitigating these types of risk.

Each of these categories can be further broken down to add focus on specific areas. Lists of risk categories and specific risks become less valuable when they are used as checklists that require each item on the list to be addressed. Having a set of risk categories is useful when they are used to direct thought and discussion about risk.

Did you anticipate the jellyfish or not?

Did you anticipate the jellyfish or not?


On February 12 at 5:44 AM a sink hole in Bowling Green, Kentucky opened up below the National Corvette Museum Skydome exhibit area swallowing eight cars. The risk of the sinkhole had not been foreseen; therefore no one had monitored the risk. The problem was discovered when a motion detector was triggered. The motion detector was in place to monitor and mitigate a more commonplace risk. I am convinced that the risk managers at the museum were convinced they had anticipated all reasonable risks. Every time methodologists, developers and project managers (of any stripe) decide that they have found the perfect process or method that risk-proofs their project that disaster is right around the corner. We have described how Agile mitigates some types of risks (Agile and Risk Management), however risk in general still needs to be managed and controlled in any size project. Software risk management is crucial to the success of all software development, enhancement and maintenance projects by avoiding problems that can be avoided and recognizing those that can’t be avoided. A lean process to manage risk in Agile projects is shown below:

  1. Identify knowable risks. – Identify the knowable risks when generating the initial backlog. Remember that risk identification, like the identification of users stories, will be an iterative process. Teams that have trouble identifying risks could leverage a checklist or a list of common risks.
  2. Build mitigation for common risks into the definition of done. – The definition of done is the requirements that the software must meet to be considered complete. Common risks such as the failure of the software to integrate properly can be built into the definition of done, which provides mitigation for the risk.
  3. Generate stories for less common risks and add them to the projects backlog. – Risks that can’t be mitigated and monitored through the definition of done can be treated as a specialized form of a user story and added directly to the product backlog.
  4. Review risks when grooming stories. – Just identifying and listing risks is useful, but not sufficient. It is necessary to review the risks placed on the product backlog.  The backlog grooming process provides a good platform for periodic risk review to ensure that if a risk needs to be mitigated steps can be added to the next sprint or that if risks are becoming issues that steps can be taken immediately.
  5. Carve out time during planning to identify emerging risks. – Project environments are dynamic. New risks may emerge or current risks evolve as the project progresses. Carving out time to reflect on risk as part of planning will help the team avoid being surprised.

Why is the discussion of risk important? Projects need understand what can reasonably happen and be prepared. While very few projects would have to plan and prepare to deal with a sink hole opening up and eating their data center or the project team (unless they were in Bowling Green) potential risks that are reasonable should be anticipated. In a world where most projects are integrated into an organizations value chain, potential disruptions or failures need to be explored and monitored.



Hand Drawn Chart Saturday

I once asked the question, “Has the adoption of Agile techniques magically erased risk from software projects?” The obvious answer is no, however Agile has provided a framework and tools to reduce the overall level of performance variance. Even if we can reduce risk by damping the variance driven by complexity, the size of the work, process discipline and people, we still need to “ROAM” the remaining risks. ROAM is a model that helps teams identify risks. Applying the model, a team would review each risk and classify then as:

  • Resolved, the risk has been answered and avoided or eliminated.
  • Owned, someone has accepted the responsibility for doing something about the risk.
  • Accepted, the risk has been understood and the team has agreed that nothing will be done about it.
  • Mitigated, something has been done so that the probability or potential impact is reduced.

When we consider any risk we need to recognize the two attributes: impact and probability.  Impact is what will happen if the risk becomes something tangible. Impacts are typically monetized or stated as the amount of effort needed to correct the problem if it occurs. The size of the impact can vary depending on when the risk occurs. For example, if we suddenly decide that the system architecture will not scale to the level required during sprint 19, the cost in rework would be higher than if that fact were discovered in sprint 2. Probability is the likelihood a risk will become an issue. In a similar manner to impact, probability varies over time.

We defined risk as “any uncertain event that can have an impact on the success of a project.” Does using Agile change our need to recognize and mitigate risk?  No, but instead of a classic risk management plan and a risk log, a more Agile approach to risk management might be generating additional user stories. While Agile techniques reduce some forms of risk we still need to be vigilant. Adding risks to the project or program backlog will help ensure there is less chance of variability and surprises.

The CFO here wants to move away from vague generic invoices because he feels (rightly so) that the agency interprets the relationship as having carte blanche...

The CFO here wants to move away from vague generic invoices because he feels (rightly so) that the agency interprets the relationship as having carte blanche…

There are many factors that cause variability in the performance of projects and releases, including complexity, the size of the work, people and process discipline. Consistency and predictability are difficult when the process is being made up on the spot. Agile has come to reflect (at least in practice) a wide range of values ranging from faster delivery to more structured frameworks such as Scrum, Extreme Programing and Scale Agile Framework Enterprise. Lack of at least some structure nearly always increases the variability in delivery and therefore the risk to the organization.

I recently received the following note from a reader (and listener to the podcast) who will remain nameless (all names redacted at the request of the reader).

“All of the development is outsourced to a company with many off-shore and a few on-site resources.

The development agency has, somehow, sold the business on the idea that because they are “Agile”, their ability to dynamically/quickly react and implement requires a lack of formal “accounting.”  The CFO here wants to move away from vague generic invoices because he feels (rightly so) that the agency interprets the relationship as having carte blanche to work on anything and everything ever scratched out on a cocktail napkin without proper project charters, buy-in, and SOW.”

This observation reflects a risk to the organization of an ill-defined process in terms the value that get delivered to the business, financial risk and from the risk to customer satisfaction. Repeatability and consistency of process are not a dirty words.

Scrum and other Agile frameworks are light-weight empirical models. At their most basic levels they summarized as:

  1. Agree upon what your are going to do (build a backlog),
  2. Plan work directly ahead (sprint/iteration planning),
  3. Build a little bit while interacting with the customer (short time box development),
  4. Review what has been done with the stakeholders (demonstration),
  5. Make corrections to the process (retrospective),
  6. Repeat as needed until the goals of the work are met.

Deming would have recognized the embedded plan-do-check-act cycle. There is nothing ad-hoc about the frame even though it is not overly prescriptive.

I recently toured a research facility for a major snack manufacturer. The people in the labs were busy dreaming up the next big snack food. Personnel were involved in both “pure” and applied research, both highly creative endeavors. When I asked about the process they were using what was described was something similar to Scrum. Creatively being pursued within a framework to reduce risk.

Ad-hoc software development and maintenance was never in style. In today’s business environment where software in an integral the delivery of value, just winging the process of development increases risk of an already somewhat risky proposition.

« Previous PageNext Page »