Transactive Memory and Agile: An Overview

I spend a lot of time studying individuals and teams in a quest to help them learn how to deliver more value. One of the most common failure scenarios I observe is that once organizations have reached a decent level of effectiveness, someone will leave and the whole thing will come tumbling down. I am not trying to suggest that turnover and change should be stamped out (I actually think it is healthy), but rather something else is amiss. Several years ago I was able to study the implementation of an agile scaling methodology in a Fortune 100 company. The study looked at quality, productivity, and cycle time across 15 product lines. In every case, the metrics fell during implementation (this was expected) then improved spectacularly, over 80% improvement in every metric, the year after implementation. Unfortunately, things got murkier when the consultants supporting the change were withdrawn and sent elsewhere. The metrics fell by 30 to 50% from the high watermark. I am not arguing that consultants should be permanently ensconced in any organization – it is unhealthy but rather something else is amiss. Recently I forgot to call my father on his birthday. My wife remembers things like special events and then clues me in…if she is around. This time she was not. The whats amiss in all three of these scenarios is a reflection of the risks of transactive memory. 

 

Story Maps Across The Product Life Cycle

A life cycle is a series of floors!

Story mapping is a technique for visualizing and organizing a product backlog.  Story maps are useful for identifying a minimum viable product, for planning releases, for finding holes in the features product management needs and even for finding extraneous functionality that finds its way into every grouping of work. Story maps are so useful that they often thought of as a silver bullet. However, they are not a tool for every scenario that a team (or team of teams) might find itself facing.  All software products follow a fairly typical product lifecycle. Software products are created, enhanced and extended, maintained and then retired. While every piece of software follows this path, not every team participated in every stage of the life cycle. Story maps are not equally useful in each stage.

 

Budgeting, Estimation, Planning, #NoEstimates and the Agile Planning Onion

There are many levels of estimation including budgeting, high-level estimation and task planning (detailed estimation).  We can link a more classic view of estimation to  the Agile planning onion popularized by Mike Cohn.   In the Agile planning onion, strategic planning is on the outside of the onion and the planning that occurs in the daily sprint meetings at the core of the onion. Each layer closer to the core relates more to the day-to-day activity of a team. The #NoEstimates movement eschew developing story- or task-level estimates and sometimes at higher levels of estimation. As you get closer to the core of the planning onion the case for the #NoEstimates becomes more compelling.

 

Uncertainty in Planning/Budgeting/Estimating

Uncertainty is a reflection of human’s ability to think about and then worry about the future.  The future, whether tomorrow or next week generates cognitive dissonance because we are afraid that what will happen will be at odds with our mental model of the future. Budgeting, estimation, and planning are tools to rationalize away uncertainty; however, they have a complicated relationship with uncertainty.  For example, in some scenarios some uncertainty helps to prove the veracity of an expert, and in other scenarios uncertainty can generate cognitive dissonance with the assumption of certainty built into budgeting, estimation, and planning tools organizations use.

 

A Process For Using Storytelling To Generate The Big Picture

Some forms of storytelling are more formal than others

Storytelling is a tool with many applications.  Generating a high-level narrative project is useful for any project to help get people on the same page and keep them there over the life of the endeavor (or at least until the story changes). Establish the big picture before diving headlong into defining what will be delivered.  A simple storytelling process is shown below:

 

Getting Stuff Done: Planning

 

A framework is just a framework without planning!

Personal Scrumban establishes a framework for conquering the chaos that day-to-day life can throw at you. However having a structure, even a structure with multiple classes of service, does not get the most important pieces of work in the queue when they need to be in the queue. Planning is required. Weekly and daily planning exercises, similar to sprint planning and the daily stand-up, are useful for taming the backlog and adapting to the demands of real life.

I begin all weekly planning sessions with a quick backlog grooming session (note: when new items are added to the queue during the week, grooming can be performed). In personal Scrumban, the goal of backlog grooming is not get team consensus (no need for the Three Amigos). Rather the goal is to ensure each backlog item that might need to be tackled in the next week has been broken down so that there are one or two immediate next steps identified. The first step in backlog grooming is to ensure that all work items (or stories) have been classified by class of service. For example, if one of the work items was “Review cover art for the Hand-Drawn Slide Saturday Ebook,” the work item should be classified in the Podcast/Blog class of service. Classes of service act as a macro prioritization and assigns the work to the relevant time slice in any given day. The second step is sizing, just like in Scrum, the immediate next steps should be of a size that can be accomplished in a single sitting. The information gathered in execution will used as part of daily planning or during the next weekly planning session.

Weekly planning is comprised of getting work items in priority order and then deciding which needs to be dealt with during the upcoming week GIVEN what is known when planning occurs. If you have not already established a work-in-process limit (WIP), set one for each class of service. A WIP limit is the amount of work you will allow yourself to start and actively work on at any point. Work is only started if there is capacity to complete the task. Prioritize up to the WIP limit or just slightly past the limit in each category. Remember if as you complete tasks in a category (and you have time) you can refresh the backlog by prioritizing new items. I generally do my weekly planning every Sunday evening so that I am ready to begin the when I roll out of bed on Monday.

Daily planning is exactly like a daily stand-up meeting, with two minor twists. In Scrum, the daily stand-up meeting starts the day with each team member answering the three famous questions:

• What did I complete since the last meeting?
• What will I complete before the next meeting? and
• What is blocking progress?

The three questions provide a framework to make generate laser focus on what is done and what needs to be done. The twists

In personal Scrumban, as in normal Kanban, completed work items would have been moved to the completed column of the Kanban board as soon as they done, however this is a good time to ensure that has occurred. The twists relate to how new items are dealt with and time allocation. During planning, work items that will be accomplished during the next 24 hours should be moved to in-progress. Given the nature of daily planning, if new work items have been discovered and prioritized into the backlog, they then become part of the standard planning process. The stand-up also provides time to reflect on anything that will block accomplishing the planned work items. A second twist to the stand-up process is a reassessment of the time slices being provided to each class of work. For example, if a critical work product needs to be completed, time from a more discretionary class of service can be reallocated and the work items in this category can be put on hold.

A weekly planning session provides a stage to address the week. To paraphrase Helmuth von Moltke the Elder, no weekly plan stands first contact with Monday. The daily stand-up provides a platform to re-adjust to the nuances of the week so that you can stay focused on delivering the maximum value possible. Without planning, all personal Kanban is a framework and a backlog of to-do items.  Planning puts the energy into the framework that provides the guidance and reduces stress.

 

What Is The Difference Between A Schedule and A Plan?

A schedule is not a plan but a plan might have a schedule in it!

The first two organizations I worked for called the project schedule the ‘project plan’. A little later when I went to work for an organization that approached project management more formally, I was initially confused when a Gantt chart stopped being a project plan and my trusty plan was replaced by a document indicating how things would be approached rather than what would be done. I still occasionally conflate the concept of a project schedule with a project plan. While the two tools are related, they are different and serve different purposes.

A project plan is a deliverable used to document planning assumptions and as a vehicle to communicate the approved of scope, cost and schedule. Some form of a schedule is typically included in the plan. Inclusion of an early schedule establishes a link between the two deliverables. The Project Management Institute (PMI) indicates that the project plan is a formal, approved document. Formal project plans can include a wide array of sub-plans, including a risk management plan, quality plan or communication plan. Formal, classic project plans can be quite significant documents requiring a lot of effort to prepare. A plan and all of the sub-plans provide a platform for a project manager and stakeholders to develop a common understanding of how a project will be approached and establish roles. Many Agile projects use the Agile Team Charter to set expectations for how the project will be approached at the team level.

A cautionary note: writing and getting a plan signed-off does not ensure that all parties have developed a common understanding. Interaction and conversation are critical steps to developing a common language for the project.

Project schedules come in many forms ranging from simple approaches, such activity lists and time tables, to highly complex forms that include task network-based schedules and Critical Path Methods (CPM). A common thread in most schedules is that features, tasks and activities (or some subset) are documented and connected as a tool to guide the team and communicate progress. Agile teams use prioritized backlogs and release plans as schedules and while other methods use techniques such as milestone charts, task lists, Gantt chats and/or CPM (this only scratches the surface). Schedules act as tools to guide activities in a project, to answer the “when” questions and to help answer the “how much will this cost” questions.

Plans are a mechanism to help teams and project leaders consider how the project will be approached, to define roles and to begin to establish a common understanding between everyone involved. Project schedules reflect how the work will get done and when it will get done. Schedules reflect tactical planning, while plans take a more strategic view. Like planning, all projects use some form of scheduling technique. Team charters, backlogs, release plans, iteration backlogs, task lists or Kanban boards or project plan documents and detailed project schedules, reflect the difference in our approach and philosophy.

 

Five Reasons People Hate Schedules

The schedule is not the enemy when used correctly.

I recently asked a group of the Software Process and Measurement listeners why they hated schedules. The focus of the question was not on plans, which are documents that provide strategic direction but rather schedules which are more task oriented. The respondents were a mixture of scrum masters, project managers, process improvement leaders and change leaders primarily in software development, enhancement and support. I should note that by being readers of a process blog and/or listeners to process-related podcast the respondents marked themselves as lifelong learners and perhaps a bit outside of the norm (in a good way). However, the top five answers were:

1. They are generally wrong. Schedules, especially anything over a longer time horizon, establish an expectation. These schedules reflect best guess and best intentions and rarely standup to what as teams wrestle with delivering value. There are all sorts of techniques to try to anticipate schedule drift, like adding padding. These techniques are tacit admissions that the schedules are generally wrong.
2. Schedules prescribe how a problem is to be solved. A detailed schedule is the embodiment of a solution; listing the tasks that specify what is to be done and when it needs to start and be completed. However, as most projects progress, the solution as it was originally conceived, evolves. This renders a detailed schedule moot.
3. Schedules are someone else’s idea of what should happen. Project managers or tech/test leaders are often tasked with creating schedules (this even happens on Agile teams when someone else breaks tasks down and assigns the work). Schedules are created with input or buy-in from the team doing the work yielding animosity and stress from the team. Release and iteration planning are Agile’s solutions to these problems.
4. Schedules reduce team behavior. One attribute of an Agile team is supportive behavior. Teams commit to work and then, when needed, swarm to tasks and features so that the team can meet its commitment. Detailed project schedule commits team members to performing specific tasks in a specific order.  The schedule gets in the way of team members being able to use self-organizing techniques like swarming.
5. Detailed schedules take a huge amount of upkeep. One respondent suggested that for any project scheduled to take more than a year to complete, one person should be allocated to maintaining the schedule. That included chasing team members for updates, resolving conflicts and in general being a pain. Other respondents were less specific, but indicated that the cost of the schedule was more than the value.

The question generated responses that were oriented detailed project schedules typically found in projects managed using classic project management techniques.  A few respondents pointed out the value of detailed project schedules. Some of the benefits included the ability to distribute and direct work across distributed teams and to facilitate a discussion of when the project would deliver. It should be noted that these responses came from more command and control oriented organizations. Respondents with a background in Agile tended to point out that while they did not feel that a classic project schedule made sense, the combination of product backlog and sprint level task list was a necessity.

 

 

Plans

A map is a plan.

All projects should have some sort of plan. Whether that plan is a classic project plan and schedule or a prioritized backlog and a release plan. A plan helps answer stakeholder questions and, perhaps more importantly, it reflects the philosophy of the project. In order for a plan of any type to be helpful, the plan and philosophy must be possible. Bob Ferguson (Listen to SPaMCAST 240) said that there were ways to detect a plan what was not possible. Several of the rules of thumb that Bob suggested (augmented with a few of my own) are:

1. Difficult work is done late. This is on both of our lists. Teams that avoid addressing difficult or technically complex stories backload risk, which can impact a project’s ability to deliver value. Conceptually this problem should be harder in Agile, assuming stories or features are prioritized in value order and the inter-relationship between features is factored into the value discussion.
2. Learning is not explicitly planned. Any project that is creating new features or a new product should have prototyping built into the process used to gather requirements and build a backlog. Experiments/prototypes also can and should be used to prove solutions that are complex or cutting edge (at least for the team). This item was on Bob’s list and not on mine; it is now.
3. Rate of story completion is not feasible. If the plan can’t be completed given the team’s (or teams’) level of velocity or productivity, then it is a bad plan. Plans that specify the amounts of functionality to be delivered, the date of delivery and the budget to be spent have credibility problems, however when they are developed using wishful thinking productivity rates they enter into the impossible range. This one is on my list and not Bob’s (in your face Bob).
4. Belief that the plan — is THE Plan. A plan, schedule or backlog that does not change for a project of any moderate or large size is wrong or if it actually works is  the reflection of sheer dumb luck (Harry Potter reference – reread Harry Potter and the Philosopher’s Stone.). Anyone that falls into the trap of absolute belief that any project deliverable can be created once and referenced forever will find delivering value difficult at best.
5. Not involving the business. The real product owner(s) must be part of the product to act as an active conduit of business acumen into the team to minimize wait and search time. All too often business stakeholders have been taught treat the boundary between IT and the business as a demilitarized zone where information hand-offs occur on a periodic basis. This behavior makes planning and maintaining any sort of plan difficult at best which slows the project down. IT teams often elect proxy product owners from inside the IT boundary leading to the same result (I heard this termed all of the responsibility and none of the authority). Proxy product owners can’t provide the level of feedback on priorities and the plan that the business can.

Plans have value only if they are current and only if maintaining plans, schedules, backlogs and the release plans do not become the goal of the project. Planning helps teams to develop a strategy for delivering value, but they must be allowed to change. Change is inevitable because we are learning both personally and about our project everyday. Not using what we learn is silly!

 

 

Sprint Planning and Non-Story Related Tasks

Burn down chart

I have been asked more than once about what to do with tasks that occur during a sprint that are not directly related to a committed story.  You need to first determine 1) whether the teams commit to the task, and 2) whether it is generally helpful for the team to account for the effort and burn it down.  Tasks can be categorized to help determine whether they affect capacity or need to be planned and managed at the team level.  Tasks that the team commits to performing need to be managed as part of the teams capacity while administrative tasks reduce capacity.

Administrative tasks.  Administrative tasks include vacations (planned), corporate meetings, meetings with human resources managers, non-project related training and others.  Classify any tasks that are not related to delivering project value under administrative tasks. One attribute of these types of tasks is that team members do not commit to these tasks, they are levied by the organization. The effort planned for these tasks should be deducted from the capacity of the team.  For example, in a five person team with 200 hours to spend on a project during a sprint (capacity), if one person was taking 20 hours of vacation the team’s capacity would be 180 hours.  If in addition to the vacation all five had to attend a two hour department staff meeting (even an important staff meeting), the team’s capacity would be reduced to 170 hours.  Administrative tasks can add up, deducting them from capacity makes the impact of these tasks obvious to everyone involved with the team.  Note: in organizations that have a very high administrative burden I sometime draw a line on the burn down chart that represents capacity before administrative tasks are removed. 

Project-related non-story tasks. Project-related non-story tasks are required to deliver the project value.  This category of tasks include backlog grooming, spikes and retrospectives.  There is a school of thought that the effort for these tasks should be deducted from the capacity.  Deducting the effort from capacity takes away the team’s impetus to manage the effort and the tasks. This takes away some of the team’s ability to self-organize and self-manage. The team should plan and commit to these tasks, therefore they are added to the backlog and burned down. This puts the onus on the team to complete the tasks and manage the time need to complete the tasks. As example if our team with 170 hours of capacity planned to do a 10 hour spike and have three people perform sprint grooming for an hour (total of 13 hours for both), I would expect to see cards for these tasks in the backlog and as they are completed the 13 hours would be burned down from the capacity.

Tasks that are under the control of the team need to be planned and burned against their capacity.  The acts of planning and accounting for the time provide the team with ability to plan and control the work they commit to completing.  When tasks are planned for the team that they can’t control, deducting it from the overall capacity helps the team keep from over committing to work that must be delivered.