Audio Version:  SPaMCAST 171

ImageWhen the Software Process and Measurement Cast began five years ago it was the culmination of an idea that had begun percolating for many years. It did not spring into existence fully formed. The cast began as a bi-weekly show combining both an essay and an interview. We have evolved to a weekly cast. The metrics minutes essays have come a regular feature. 

 The cast has been a path to give something back to the industry that has been so good to me while at the same time do something that would satiate my need to continually learn while being exposed to new ideas. The audience for the first show (and more than a few after that) could be counted on on one hand or on some multiple of hands and feet. After a few breaks and a lot hard work the audience, measured in downloads, has reached 10k per month.  I am happy to say that we are still growing. There are so many people to thank for the shows success that it is a deeply humbling experience.  Interviewees, sponsors, great contributors and my family have made all of this possible. 

 Does this sound a bit like a swan song?  Not on your life.  Where do we (we because I am including you) go from here? While I am happy with the general format I think the intro and outros need to be freshened up.  Another change I would like to affect is to engage more of you to provide content to build the essay segment into more of a journal or magazine. If you have a review or essay and want to be an internet star, I have the platform for you! If you have a submission let me know, I will be happy to review it and help produce it.  Note all submission will be reviewed to make sure they fit the show’s  format and level of quality. If you don’t want to record your review or essay, I can have it narrated.  A recent example of audience content can be found on SPaMCAST 169 where Elizabeth Harrin wrote a great book review and Barb Cagley provided the narration.

 Other plans and goals for 2012 include growing the audience by 50%, a new website that consolidates the podcast and blog, re-institution the SPaMCAST newsletter and adding at least one video cast (Pat Ferdinandi has rubbed off on me). Want to help or have ideas on how to grow the audience? Contact me and we can discuss how we can make something happen. Anyone that helps will get exposure to the whole SPaMCAST audience.

 Finally, I am actively searching for additional sponsors like LeanKit Kanban. Chris Hefley and the crew at LeanKit have been and continue to be great sponsors.  Sponsors help with upgrading equipment, offsetting the hosting and other costs. 

 Year Six will be a great time for all of us. I hope the cast will continual to be useful, informative and exciting!  Thank all of you for making this possible!

 

Year Five Press Release (PDF)

For Immediate Release
January 23, 2012

Avon Lake, OH – The Software Process and Measurement Podcast (SPaMCAST) is celebrating its 170th episode after five years of interviewing many of the leaders in the software development world. The anniversary edition of SPaMCAST features an interview with Hillel Glazer, speaker, process guru and author of High Performance Operations.

SPaMCAST feature interviews have included:

  • Chris Hefley, Chief Executive Officer, Leankit Kanban, Bandit Software, LLC
  • Dean Leffingwell author of Scaling Software Agility and others
  • Peter Taylor  author of many books including The Lazy Project Manager
  • Elizabeth Harrin author, award winning blog, The Girl’s Guide to Project Management
  • Tim Lister, co-author of  Adrenaline Junkies and Template Zombies
  • David Anderson the author of  Agile Management for Software Engineering
  • Kent Beck, pioneer in Agile Methods
  • Scott Ambler, though leader in Test Driven Development
  • Ivar Jacobson, developer of Use Cases
  • Grady Booch, discussing Life, the Universe and Development

The Cast covers topics that deal with the challenges of how work is done in information technology organizations as they grow and evolve.  The show combines commentaries, interviews and feedback to serve up ideas, opinions, advice and facts.  In a nutshell, the Cast has provided and will continue to provide advice for and from practitioners, methodologists, pundits and consultants. The editor, Tom Cagley, is a leading consultant in software development process improvement, the Vice President of Consulting for the David Consulting Group, Past President of the International Function Point Users Group and co-author of Mastering Software Project Management.

The Software Process and Measurement Cast can be found at www.spamcast.net. It is also available on all major podcast services including iTunes and the Zune Marketplace. All previous episodes are available download.  The Cast currently enjoys 10,000 downloads a month, up 20% in the past year It is delivered as a free public service to the information technology community and has listeners across the globe.

Contact:
Thomas M. Cagley Jr.
Editor

Email: Spamcastinfo@gmail.com
Mobile: 440.668.5717
Skype ID: Thomas,Cagley.Jr

Serial Mono-tasking In Action
Part Two of The Case Against Multitaking.
By Thomas M. Cagley Jr.

Audio Version of Part One
Audio Version of Part Two

Recap of Part One: True multitasking in the work place is rare at best.  Even when we think of fast-switching as a form of multitasking, multitasking lacks efficiency due to switching costs. If you decide to accept the cost of multitasking, the act of multitasking is — complicated. Complexity causes errors and makes more work which reduces effectiveness even more.

Serial mono-tasking with planning is a mechanism that can help minimize switching costs and complexity.  As noted in Part One of the essay, mono-tasking makes the most sense if efficiency and productivity are your goals. How we implement mono-tasking in the face of an interrupt-driven world is an open question.

Serial Mono-tasking In Action

I would suggest that any approach to dealing with tasks whether for a program, project or a personal to-do list must be based on a process that includes specific coping mechanisms.

Prioritization:   As noted in “The Case Against Multitasking”, having a few tasks queued up reduces switching time. Switching time is the time required for the person doing work to prepare to do the next task based on a new set of rules. As an example of these different rules, consider the rules and constraints you have in mind when writing an email to your significant other; now consider the constraints and rules you would have to keep in mind if writing an email to your boss (unless they are the same). Another example of a task with different rules would be coding a user-facing website as compared to loading a data warehouse table. Prioritizing tasks can serve three purposes:  The first is to minimize the differences between rule sets when switching (I will do all of my work email before updating Facebook) which will reduce the impact of switching. Secondly, prioritization at the team level provides the ability for the team to group (self-organization) the work so as to reduce rules and constraint differences between tasks. Thirdly, prioritization lets team members mentally prepare before the switch (another coping mechanism). Bottom line, prioritization is about tackling what is important first.

Isolation: Team environments tend to be noisy and interrupt-driven because IT personnel are nothing if not individualistic. Each person will have their own coping mechanisms. Find your own mechanism to block distractions; wear headphones, learn to focus, turn off IM for periods of time or consider working from home if possible. In other words, adapt to the environment without withdrawing or rejecting those around you. Remember that as noted in the first part of the essay, background noise does not have a substantive scientific basis so blasting “En da Godadiva” might not be a great idea.

Focus:  Filtering or blocking things out and focus are related but they are also different. Focus is about narrowing your consciousness to contemplate a specific topic. To aid in developing focus, avoid distractions; turn off IM, don’t check emails and in radical cases hide from team members (just for a little while).

Adjust:  When things don’t work out as you plan (and how many plans work perfectly?) make changes. Adjust both the process you are using and the environment to maximize your effectiveness (note, I did not say efficiency or productivity). Any process with human involvement is naturally chaotic requiring adjustments. One of the observations I have heard from studies of the Toyota Production System is that if a process is not being changed and adapted then it is not being used properly.

Process:  A system or a process is required to control the flow of work.  It would be difficult to prioritize and focus on a set of tasks and therefore to be productive if there was no control on how a task could enter or be accepted to be worked on.  Processes like Kanban, SCRUM and even the venerable waterfall methods all include mechanisms to control the flow of work and to decide on which items should be addressed and when.

A side note in the discussion of process, is that fully committing all resources on specific tasks in a project leaves no time for probems therefore is tantamount to enforcing overtime or to falling behind schedule. All processes must allow time for both the interruptions a corporate environment always has and the overhead of the workday (email, time accounting, non-project meetings, coffee, bathroom breaks . . .to name a few). One method suggested by many time management gurus is compartmentalization.  For example, blocking a period of time for heads down working followed by a window of time to read and return emails or phone messages.  The goal is to have a process that allows you to work as simply as possible. This means the process needs the ability to capture to-dos, categorize, prioritize and then track work to completion (which is exactly what backlog is for an agile project or a work queue in Kanban).

 

At a personal level have a process, prioritize your work, filter out distractions and focus on what is important.  Learn to postpone interruptions rather than switching. When interruptions can’t be avoided, shut down rather than just stopping what you doing to react.  Shutting down will minimize retooling when you restart. When the urge strikes to check email or jump back into Twitter, just say no, take a deep breath and try to refocus. At a personal level I am a fan of David Allen’s Getting Things Done (GTD) and personal Kanban.

When working on projects, the multitasking problem can be addressed by adopting techniques that support serial mono-tasking with planning at a tactical level. Agile and Lean techniques are tailor-made for addressing this issue. Agile and Lean techniques have recognized the power of doing one thing at a time. Scrum and Kanban both feature isolation of tasks so that they are selected and disposed of in a serial manner.

Serial mono-tasking requires discipline to have a process and for you or your team to focus.  It does not mean slowing down, but it does mean making choices.  In many cases, rushing off to deal with interruptions gives the impression of importance but in the long run it probably makes your project late or reduces the quality of the product you deliver.

Audio Version:  SPaMCAST 167

To multitask or not to multitask, that is the question.  Whether ’tis nobler in the mind to suffer the slings and arrows of mono-tasking or to take arms against a sea of trouble and by opposing do more…at least appear to do more.  Is the discussion of mono-tasking versus multitasking a tempest in a teapot, a true productivity killer or perhaps are we really discussing how we segment work?  Depending on how you define the word, I believe it is the later.  The problem is that like so many other words we have conflated a number of concepts into a broader idea.

In my opinion, there are three common scenarios that get conflated into the term multitasking:

  1. Actively doing two or more things at once (breathing and talking).
  2. Actively doing one thing while passively doing another (writing this essay while listening to the radio).
  3. Switching between tasks related, unrelated or loosely coupled (rotating between reading a book and updating Facebook).

The formal definition from the Merriam Webster dictionary defines multitasking[1] as “the performance of multiple tasks at one time.”  This fits scenario one and two (to a less extent) but definitely not three.

In the work place, true multitasking is rare.  It is not that we humans can’t multitask because we can multitask even using the strictest application of the definition of the word.   We are good at multitasking when it is a combination that includes an autonomic task (like breathing, heart beating or sweating) and something more active such as chewing gum or when it includes accidents such as the combination of talking on a cell phone while driving and running into the back of my car. The data shows that generally humans are not really very good at true multitasking in the workplace. Linda Stone noted in the Huffington Post[2] that people tend to stop breathing while they answer email. She even named the malady, email apnea. If you need more examples just reflect on the data concerning cell phone usage and driving or if data doesn’t work for you then try rubbing your stomach and patting your head at the same time. Computers, on the other hand, are really good at multitasking and no matter the number of processors we have on our desktop we have not crossed that chasm to become full cyborgs yet.

The second scenario termed multitasking is a bit of a nuance: actively performing a task while passively performing another.  A classic example of this form of multitasking is reading a book while the radio is playing in the background.  In many cases this form of multitasking is an attempt to manipulate the work environment to aid focus.  The question of whether background noise affects concentration has been often studied.  In the January 2010 edition of the Scientific American, the magazine noted that background or low-level noise often disrupts people’s concentration[3]. Whether background noise helps or hurts concentration is probably one of brain wiring.   During the Christmas holiday I observed my son-in-law who can concentrate for hours but zones everything else out and my youngest daughter who requires multiple simultaneous inputs to get into flow state.  One for background noise and one against, maybe everyone is different.  At any rate, the data suggests that even on as basic a level as the having the radio on while reading, multitasking generally does not improve focus and efficiency.  By the way, if you are one of those that require background noise to concentrate, I recommend good headphones.

 

The final scenario generally conflated with multitasking is switching between multiple tasks.  This scenario is also known as fast-switching or serial mono-tasking.  Switching is in reality the juggling of resources to accomplish a set of tasks; at a macro-level you might be multitasking; however, on a micro-level you are mono-tasking.  The issue with this type of behavior is that juggling is not always easy even if you are good at it.  Inefficiencies are caused by both the queuing/scheduling of tasks and then the retooling that occurs when switching between tasks. During my research for this essay I found that in the brain, juggling multiple tasks is performed by mental executive processes that manage the individual tasks and determine how, when, and with what priorities they get performed[4]. The executive process coordinates activities so that the right outcomes occur, an analogy for what is going on inside of our minds is the air traffic control system.  The air traffic control system makes sure planes get where they are going with a minimum of delay and without two planes trying to use the same spot in the sky at the same time (bad). The coordination of tasks requires a level of overhead, just think about coordinating schedules for shared project resources if you need proof that overhead is required. However, more significant inefficiencies occur when a person switches between tasks.  Task switching experiments have shown a need for the person switching tasks to take time and mental resources to reorient. The reorientation tax (the amount of effort you need to expend to switch tasks) goes up with task complexity, lack of familiarity of the next task and the relative differences between the tasks.  Research has shown that task queuing (lining tasks up in order of precedence) so that the person doing the work can know what is coming and /or can influence the order they are to be done, can be used to reduce the impact of switching[5].  Reduction in the impact of switching can be mediated by separable executive control processes that prepare systemically for transitions between successive tasks[6].  The issue with the fast switching brand of multitasking is that in many cases the queuing of tasks is not as seamless as it should be which creates wait-states or multiple re-tooling situations because work does not flow as cleanly as it is diagramed on the Microsoft Project schedule (this is one of the reasons Reinertsen indicates that full allocation reduces efficiency).  Please note I am not comparing this type of multitasking to taking breaks between tasks to clear the “buffers” which has been shown to be valuable.

The data suggests that a mono-tasking environment that reduces interruptions is the most efficient work scenario[7]; however, the work environment is rich in interruptions.  Further according to Capers Jones, the information technology field has more named specialties than any other profession which means that individuals are spread across more project teams so they can practice their specialty.  Switching between projects leads to the switching tax we mentioned earlier.  Switching between tasks and projects is firmly etched into the classic project management body of knowledge based on 19th century manufacturing thinking (it’s now the 21st century).  We have even gone as far as to building the scheduling of shared resources into our project management tools which suggests that getting rid of the problem will not happen in the near future.  Today’s working environment leaves us with few options as methodologists.  Our goal must be to avoid switching when possible, minimize the impact when we can’t and then to decide to live with what we can’t change.

 

Next . . . A plan to address at least part of the problem

 

 


[2] http://www.huffingtonpost.com/linda-stone/just-breathe-building-the_b_85651.html , January 1, 2012

[4] Choices, Choices: Limits of the Brain, Anthrostrategist  Blog, August 28, 2011 (referenced December 17, 2011)

[5] Rubinstein, J. S., Meyer, D. E., & Evans, J. E. (2001). Executive Control of Cognitive Processes in Task Switching. Journal of Experimental Psychology: Human Perception and Performance, 27(4),763-797.

 

[6] Rubinstein, J. S., Meyer, D. E., & Evans, J. E. (2001). Executive Control of Cognitive Processes in Task Switching. Journal of Experimental Psychology: Human Perception and Performance, 27(4),763-797.

 

[7] Multitasking and Monotasking: The Effects of Mental Workload on Deferred Task Interruptions, Dario D. Salvucci and Peter Bogunovich, PDF, https://www.cs.drexel.edu/~salvucci/publications/Salvucci-CHI10.pdf, December 12, 2011, p1

The envolope please . . . It is time for the Top Ten Interviews.  2011 had some fantastic interviews with some fantastic people.  If you missed any of the interviews or want to listen to them again (I did over the last couple of weeks).

SPaMCAST 136 – Ginger Levin and LeRoy Ward, Program Management Complexity
SPaMCAST 156 – Linda Rising, Agile, Patterns for Fearless Change
SPaMCAST 112 – Israel Gat, Technical Debt
SPaMCAST 138 – Jo Ann Sweeney, Communication
SPaMCAST 118 - Elizabeth Harrin, Social Media for Project Managers
SPaMCAST 116 – Naomi Karten, Presentation Skills for Technical Professionals
SPaMCAST 150 – Yuval Yeret, Kanban, Agile
SPaMCAST 144 – Mary Lynn Penn, CMMI and Six Sigma
SPaMCAST 140 – Raja Bavani, The Ten Best Influences On Software Product Engineering
SPaMCAST 158 – Peter Taylor, The Lazy Project Manager

The thread, if one exists, is change, dealing with change and the future.  Which interview was your favorate?

21012 will be as good or even better (I have some ideas  . . .).  If you have ideas for interviewees, please let me know!

Note:  I will be counting down the top ten on twitter with # 1 coinciding with LSU v Alabama football game.

 

It is that time of year!  Time to celebrate what worked and what didn’t (yes I said celebrate what did not – without experimentation there is no growth). I have compiled the top ten interviews and and the top ten essays from the last year.  

Top Ten Essays 

SPaMCAST 137 – Abstractions, Tool Review Agile Board, Joseph Hurtado
SPaMCAST 135 – Metrics Minute – Value at Risk
SPaMCAST 141 – Ready For Agile, A Quiz
SPaMCAST 131 – Agile is form Venus PMOs from Mars, Part Two
SPaMCAST 149 – CMMI Readiness Checklist
SPaMCAST 127 – Agile is from Venus and PMOs from Mars Part 1, MAIN News
SPaMCAST 147 – Selfishness and Process Improvement
SPaMCAST 155 – Systems and Systems Thinking, Part 1
SPaMCAST 145 – Metrics Minute: IFPUG Function Points
SPaMCAST 143 – Do You Have Management Support?

To listen again just click on the links!

It is interesting that SPaMCAST 137 the top essay (and nearly the top podcast).  I believe it was popular due to Joseph’s great review.  I would like to include more reviews in 2012.  Another item that caught my eye was the meteoric rise of SPaMCAST 155 on Systems and Systems Thinking.  I need to back that up with an interview or three.  Do you have ideas on whom I should talk to?

Which of these or any of the other essays was your favorite and why?  Are there other observations you would make by looking at the list?  

Next the top ten interviews! 

IT-CMF:  A Framework, A Certification and More
Thomas M. Cagley Jr

Audio Version:  SPaMCAST 165

On Thursday morning I checked into a testing center to test the knowledge and skills that are central to understanding and leveraging the IT Capability Maturity Framework (IT-CMF) and also to certify.  When I completed the exam I tweeted my results to the world and got resounding response of  . . .”so what exactly is IT-CMF?” and “why do we need another model?”

The IT-CMF is a framework that provides a management roadmap to optimize business value derived from IT investments. The framework is stewarded by the Innovation Value Institute (IVI) based at the National University of Ireland in Maynooth.  The Innovation Value Institute was co-founded by NUI Maynooth and Intel in 2006 (http://ivi.nuim.ie/about/index.shtml).

The IT-CMF consists of a five-stage maturity model that acts as structure for tracking IT improvement efforts. The use of five stages to describe frameworks seems to have become ubiquitous, someday a bit of creativity will be applied and we will see framework described as a Möbius strip!  Regardless of the lack of creativity in structure, the model is nothing if not compendious.   The framework draws its power from a set of 33 critical processes that describe a set of interlocking best practices. The specific recognition of some of my pet peeves like architecture helped sway my opinion of the framework.  The IT-CMF is organized into four foundational macro processes:

Managing IT like a Business

Managing the IT Budget

Managing the IT Capability

Managing IT for Business Value

While focused on IT, the IT-CMF approaches the definition of value on a broader plane which includes the business from product to finance.  The breadth is reflective of other modern frameworks such as the CMMI or agile albeit the IT-CMF takes a broader approach.  The IF-CMF will challenge the user to step away from development life cycles and think about IT from an Entrepreneur’s point-of-view.  I will admit that this understanding did not immediately leap off the page.  The stated goal to more firmly align IT investments with bottom line business results connotes a structure that could be oppressive.  The ability to apply each critical practice independently (ish) provides a good deal of flexibility that more structured frameworks can lack.

The question you might ask is why another model is needed in an already crowded world? Based on the potential overlap in the myriad of frameworks a second question must be broached, can the IT-CMF be effectively integrated into a multi-modal process improvement framework?

An assessment using the IT-CMF framework provides a view into an IT organization’s maturity across the core processes.  The IT-CMF supports four types of assessments; Executive, Critical Process, Cluster and Industry Benchmark.  Each type of assessment delivers different levels of results   Assessment results taken together with comparisons to industry benchmarks and best practices, will highlight a company’s key maturity gaps and value-creation opportunities.

ComputerWeekly[1] noted that “CIOs spend 80% of their budgets “keeping the lights on”. But their most intractable problem is to get the best return possible from the other 20% of their budget.”  The IT-CMF is a framework that seeks to more firmly align IT investments with bottom line business results.  A tall order however at the very least the framework is compendious enough to give it a go and flexible enough to be useful.  As with any other framework the proof will be in the deployment because as with most frameworks the IT-CMF does not tell you how to address gaps.  As we all know there are lean and agile approaches to solving problems and there are heavy and bureaucratic approaches.

 

 


[1] http://www.computerweekly.com/news/1280097086/Early-adopters-report-big-savings-from-IT-CMFIT-CMF:  A Framework, A Certification and More

Thomas M. Cagley Jr

On Thursday morning I checked into a testing center to test the knowledge and skills that are central to understanding and leveraging the IT Capability Maturity Framework (IT-CMF) and also to certify.  When I completed the exam I tweeted my results to the world and got resounding response of  . . .”so what exactly is IT-CMF?” and “why do we need another model?”

The IT-CMF is a framework that provides a management roadmap to optimize business value derived from IT investments. The framework is stewarded by the Innovation Value Institute (IVI) based at the National University of Ireland in Maynooth.  The Innovation Value Institute was co-founded by NUI Maynooth and Intel in 2006 (http://ivi.nuim.ie/about/index.shtml).

The IT-CMF consists of a five-stage maturity model that acts as structure for tracking IT improvement efforts. The use of five stages to describe frameworks seems to have become ubiquitous, someday a bit of creativity will be applied and we will see framework described as a Möbius strip!  Regardless of the lack of creativity in structure, the model is nothing if not compendious.   The framework draws its power from a set of 33 critical processes that describe a set of interlocking best practices. The specific recognition of some of my pet peeves like architecture helped sway my opinion of the framework.  The IT-CMF is organized into four foundational macro processes:

Managing IT like a Business

Managing the IT Budget

Managing the IT Capability

Managing IT for Business Value

While focused on IT, the IT-CMF approaches the definition of value on a broader plane which includes the business from product to finance.  The breadth is reflective of other modern frameworks such as the CMMI or agile albeit the IT-CMF takes a broader approach.  The IF-CMF will challenge the user to step away from development life cycles and think about IT from an Entrepreneur’s point-of-view.  I will admit that this understanding did not immediately leap off the page.  The stated goal to more firmly align IT investments with bottom line business results connotes a structure that could be oppressive.  The ability to apply each critical practice independently (ish) provides a good deal of flexibility that more structured frameworks can lack.

 

The question you might ask is why another model is needed in an already crowded world? Based on the potential overlap in the myriad of frameworks a second question must be broached, can the IT-CMF be effectively integrated into a multi-modal process improvement framework?

An assessment using the IT-CMF framework provides a view into an IT organization’s maturity across the core processes.  The IT-CMF supports four types of assessments; Executive, Critical Process, Cluster and Industry Benchmark.  Each type of assessment delivers different levels of results   Assessment results taken together with comparisons to industry benchmarks and best practices, will highlight a company’s key maturity gaps and value-creation opportunities.

ComputerWeekly[1] noted that “CIOs spend 80% of their budgets “keeping the lights on”. But their most intractable problem is to get the best return possible from the other 20% of their budget.”  The IT-CMF is a framework that seeks to more firmly align IT investments with bottom line business results.  A tall order however at the very least the framework is compendious enough to give it a go and flexible enough to be useful.  As with any other framework the proof will be in the deployment because as with most frameworks the IT-CMF does not tell you how to address gaps.  As we all know there are lean and agile approaches to solving problems and there are heavy and bureaucratic approaches.

 

 

Metrics Minute: Return on Investment (ROI)
Thomas M. Cagley Jr.

Audio Version:  SPaMCAST 163

Definition

ROI is a standard measure of project profitability that is the discounted profits over the life of the project expressed as a percentage of initial investment.  ROI is a classic financial metric that when applied to projects, is typically used as a technique for project acceptance or, in retrospect, as a tool to evaluate overall performance. When applied in agile projects, ROI can be used to determine priority of epics, themes or even stories.

Formula

ROI = (Average Net Benefits) ÷ (Initial Costs)

Average Net Benefits is the residual value that results from a project after adding total benefits for a specific period and subtracting all expenses for the project for that period.

Initial Costs includes any expense related to the initial development of the project.

Usage

The primary use of ROI is to compare a potential project to other possible projects in order to determine which makes the most sense to pursue (like Return on Assets – ROA).  Projects with a higher ROI are more likely to be included in the portfolio of projects.  When applying ROI to agile projects I suggest at the very least using ROI as a tool to prioritize epics and features into a release plan.

Alternately ROI can be used as a tool to evaluate continued investment in a product or a portfolio of products.  In this case the metric would be periodically recalculated to determine whether an overall project portfolio represents an efficient use of assets or if segments of the portfolio offer high a higher return.

Issues

There are several issues with the calculation and use of ROI as a decision tool including poor math, failing to make an apples-to-apples comparison or in extreme cases the combination of poor math and bad comparisons.  The most significant issues are:

Return on Investment is a financial tool designed to help build a business case to support decision making by evaluating the forecasted impact to your organization’s bottom line.  ROI or any other financial analyses should not be your only evaluation criteria.  High ROI’s may be an artifact of wishful thinking or creative accounting for benefits.  Rigorous follow-up helps stop this type of behavior.

A second issue is failing to account for the time value of money.  The time value of money suggests that a benefit or cost today has a higher absolute value than the same value or cost in the future.  Techniques like Net Present Value normalize benefits and costs in the future based on a discount rate (such as the presumed interest rate over the period or the organizations internal rate of return).

Comparisons of factors like ROI for two projects must be across similar timeframes so that an apples-to-apples comparison can be made.  In environments that are rapidly evolving shorten the comparison period to favor projects that deliver benefits earlier rather than later.  The comparison problem can be exacerbated when used for the smaller work packages usually seen in agile projects. Gross value may be more easily determined for work packages like stories.

A final issue is that intangible benefits may affect the rational for project selection which forces organizations to address valuation of intangible benefits. This issue is a relative of the “numbers don’t tell the whole story” argument. As in the discussion of comparing work items above, as the work packages become more granular, it may become more difficult to accurately evaluate intangible or non-business value in order to calculate ROI. Create rules for how you will allocate or value intangible benefits and costs. Valuation of intangible benefits requires a judgment call and unless that judgment is based on rules or guidelines those judgment calls will be open to question or manipulation.

Related Metrics

  • Return on Equity (ROE)
  • Return on Assets (ROA)
  • Payback Period
  • Internal Rate of Return (IRR)
  • Net Present Value
  • Total cost of ownership

Criticisms

Criticisms of ROI, like the issues noted earlier, revolve around how the metric is used.

This first criticism is that comparisons across different types of projects are difficult due differing cost and benefit structures that effect ROI.  An example of problems that occur when comparing differing types of projects would be making an investment decision about an infrastructure project versus a project that generates revenue. It is usually more difficult to identify and monetize the benefits of an infrastructure project which makes these types of projects more difficult to fund even though they may be required to enable projects with a more classic benefit stream. This criticism is true; however, I would suggest this is a failure to enforce a linkage between infrastructure projects to the impact on revenue streams.

A second criticism: Valuing intangible costs and benefits is at best an inexact science and is open to internal political games. This criticism is the most troublesome. There isn’t general agreement on how to value the intangible.   I would suggest not using a one-size-fits-all solution; rather I would suggest ensuring that valuations of intangible cost or benefits are consistently evaluated based on a risk adjusted discount rate. The higher the risk, the higher the discount rate would be set. A formal approach to measuring or estimating risk will be required to avoid manipulation of the analysis.

A third criticism of ROI centers on using a single point in time to qualify projects. The calculation of ROI generally reflects a snapshot of at particular moment in time. Unless you live under a rock it is difficult to forget that the business environment is – - – dynamic. The idea that an ROI calculation can be done and evaluated once to ensure that the organizations portfolio of projects continually maximizes value is ludicrous. To deal with the issue I suggest periodic revaluations of the portfolio to ensure that the business environment does not change enough to require changing the composition of the project portfolio.  Note: I would suggest that there is a need re-evaluate projects more often if they in the gray area between acceptance and rejection.

 

Audio Version:  SPaMCAST 161

When talking about metrics to measure agile processes one should begin by defining “why” you want to measure followed immediately by “what” we will do with the data when we know it. As I have noted before, there are only two reasons to measure: The first is to generate specific behaviors and second to predict the future.  Organizational goals provide the rational for what to measure and the type of measure determines whether it drives behavior or provides direction.

Change only occurs when three states are satisfied. There needs to be a trigger (1), those that are being asked to change must have the ability (2) and the motivation (3) to change.  Measurement as a tool to predict the future can provide the trigger and motivation for change.  Goals are a mechanism to order how we interpret the data generated as we measure but they are passed through many filters until they arrive at the individual where action happens.  The hierarchy begins with strategy and then is interpreted organizationally and operationally until individuals place their stamp on it and act.  Transparency and balance are required to help ensure that each interpretation stays true to the organizational goal.

Effective measurement is a balance.  On one side collection of the measures which are then synthesized into metrics.  Collection signals what is important to the organization which is balanced by the insights, actions, change and transformations that are generated.  Balance in measurement is much akin to the second law of thermodynamics.  To paraphrase, for every measure there must be an equal and opposite reaction.   A simple metrics model to enforce balance includes metrics in four categories

      1. Productivity
      1. Quality
      1. Predictability
      1. Value

The model help focus measurement in order to deliver value based on actions taken to make the business better. 

Agile measurement requires embracing seven philosophies as they relate to measurement.  Agile measurement must:

  • Reinforce desired AGILE behavior
  • Focus on results
  • Measure trends
  • Be easy to collect
  • Include context
  • Create real conversation
  • INCLUDE ONLY WHAT IS ABSOLUTELY NEEDED

If measurement incents behavior, incent the right behavior!  As an example, developing a tasks level schedule for the whole project then measuring tasks completed against that schedule flies in the face of the daily planning and re-planning required to support effectiveness of self-managed and self-organized teams.   

Interim steps are far less important than results.  Does the code work?  What did we finish?  Can it be delivered? By focusing on results we are far less likely to miss the big picture.  I recently read a status report that proudly announced that the team had met 95% of the dates during the current release.  The one they missed was the final delivery date . . .

In most cases any individual observation is a reflection of the process.  Deming used the term “common cause”.   The only way to address a “common cause” problem is to change the process.  Trends are useful in describing the capacity of the process (predicting the future).  I use a rule that individual observations are studied only when they are three standard deviations away from the mean. In other words, study individual observations when they are different enough to matter; otherwise, reflect on the trend of the observations.

Collecting information, any information, takes time and effort.  Firstly, any effort that you take away from delivering functionality reduces ITs value to the company.  Secondly, there is always a resistance to work that is perceived as overhead.  Automate as much as possible to reduce the impact and make sure the information collected can be consumed at the point of collection, as well as in the hallowed halls of the PMO.

What does a number really mean?  It has been said that a set of metrics have never solved a problem and while I don’t think in absolutes I would suggest that numbers without context are far less valuable and more open to highly variable interpretations that will require even more effort to contain and manage.

Generating a conversation is a corollary to the need for context.  Conversation causes data to be synthesized and contextualized so that common organizational knowledge is generated.  Knowledge is the real value. 

Only measure what is absolutely needed.  For every metric I would ask you to think about what you will do with the information you are requesting and then how perceived overhead will affect acceptance before you act.

Principles keep us honest and focused on delivering value to the organization.  Principles that an organization develops, adapts and adopts are a reflection of what the organization values.

A subset of all of the metrics in the world makes sense in the agile environment.  The pallet the Metrics Minute suggests is as follows.  The term pallet is used as a considered opinion, a pallet suggests selecting those that are really needed to paint the picture that has value. The measures in the pallet are as follows: 

      • ROI (Value)
      • Customer Satisfaction (value)
      • Team Satisfaction (value)
      • Business Value – Burn-up (Value)
      • Velocity – Burn-down (predictability)
      • Sprint Escape Rate (Predictability)
      • Test Case Pass Rate  – Automated (Quality)
      • Technical Debt (Quality)
      • Work-In-Process (Productivity)
      • Cycle Time (Productivity)

Each of these metrics will be (or have been) covered in detail in other Metrics Minute entries so that you can determine if these metrics are useful in answering the questions you have about your agile development program. Rarely will any individual project, team or organization need to use all of the metrics in the agile metrics pallet.   The pallet is just . . .  a pallet. Pick only those that you need for balanced view of your development process.  Create your own color by blending metrics.

Software Process and Measurement Cast 158 Quick Reminder

Peter Taylor has also provided two copies of his book The Lazy Project Manager for Software Process and Management listeners. I will draw two names at random (not random names) on November 27th and Peter will send the winners a copy. To participate send an email to spamcastinfo@gmail.com with the subject line “Lazy Project Manager.” Include you full shipping address.

Listen at http://t.co/Fzu29Dzt

Next Page »

Follow

Get every new post delivered to your Inbox.

Join 2,106 other followers