Individually, each boy is a project. Together they're a program to be managed.

Individually, each boy is a project. Together they’re a program to be managed.

Scaling Agile project management to large, complex endeavors requires an Agile Program Manager to address the big picture coordination of programs.  Program management is the discipline of coordinating and managing large efforts comprised of a number of parallel and related projects. Scrum leverages a concept called the Scrum of Scrums to perform many of the activities need for program management.  Agile Program Management is not just repurposed project management or a part-time job for a Scrum Master.

Agile Program Managers coordinate and track expectations across all projects under the umbrella of the program, whether the projects are using Agile or not. Coordination includes activities like identifying and tracking dependencies, tracking risks and issues and communication. Coordination of the larger program generally requires developing a portfolio of moving parts at the epic or function level across all of the related projects (epics are large user stories that represent large concepts that will be broken down later). Agile Program Managers layer each project’s release plans on top of the program portfolio to provide a platform for coordinated release planning. Techniques like Kanban can be used for tracking and visualizing the portfolio.  Visualization show how the epics or functions are progressing as they are developed and staged for delivery to the program’s customers.

Facilitating communication is one the roles of an Agile Program Managers. The Scrum of Scrums is the primary vehicle for ensuring communication.  The Scum of Scrums is a meeting of the all of the directly responsible individuals (DRIs) from each team in the program. The DRI has the responsibility to act as the conduit of information for his or her team to the Agile Program Manager and other DRIs. The DRI raises issues, risks, concerns and needs. In short, the DRI communicates to the team and the Scrum of Scrums. The Scrum of Scrums is best as a daily meeting of the DRIs chaired by the Agile Program Manager, however the frequency can be tailored to meet the program’s needs.  A pattern I have seen used to minimize overhead is varying the frequency of the Scrum of Scrums based on project risk.

Another set of activities that generally fall to the Agile Program Manager is the development and communication of program status information. Chairing high-level status meetings, such as those with sponsor or other guidance groups, is a natural extension of the role. However this requires the Agile Program Manager to act as a conduit of information by transferring knowledge from the Scrum of Scrums to the sponsors and back again. Any problem with information flow can potentially cause bad decisions and will affect the program.

It is important to recognize that Agile Program Management is more than a specialization within the realm of project management or a side job a Scrum Master can do in his or her spare time.  Agile Program Managers need to be well versed in both Agile techniques and in standard program management techniques because the Agile Program Manager is a hybrid from both camps. Agile Program Managers build the big picture view that a portfolio view of all of the related projects will deliver. They also must facilitate communication via the Scrum of Scrums and standard program status vehicles.  The Agile Program Manager many times must straddle the line between both the Agile and waterfall worlds.

Scaling - going from one project to two (or more!)

Scaling – going from one project to two (or more!)

The Scrum of Scrums (SoS) is a technique for scaling Scrum and other Agile techniques to larger efforts. On paper, the SoS is a simple extrapolation of the daily Scrum or stand-up meeting that has become a hallmark of the practice of Agile. A typical stand-up meeting occurs on a daily basis so that all of the team members of can coordinate and plan daily activities. Classically the team uses the three question technique to organize the meeting. In a similar manner, a typical Scrum of Scrums acts as a focal point to help synchronize a team of teams or even teams of teams. There are four basics of a SoS that need to be understood before any nuances can be addressed.

  1. Who Attends: There are two basic schools of thought in picking SoS attendees. The first (and most common) school suggests that the Scrum Master attend the SoS. The Scaled Agile Framework Enterprise (SAFe) is an example of a methodology that leverages the Scrum Master during both the planning event and during the program increment. Alternately, the second group takes a more egalitarian approach (possibly more pragmatic) allowing the each team to select an attendee that can most easily convey or understand the current issues the team and the larger group are having at the time. For example, if design decisions are at the front, perhaps team members with the most UX acumen would make sense. In this scenario attendees would vary over time.
  2. Who Facilitates: Small SoS groups, for example a handful of teams that are co-located, may not require facilitation. The SoS can self-organize and execute the meeting with little overhead. However as the number of participants increases (I bound SoS meetings using the same 5 -9 members rule used in Scrum teams) or as the distribution of the members becomes more varied facilitation becomes more important. Facilitators help the team use the SoS practice, ensure logistics are set up and champion the schedule. In larger efforts, a program manager often delivers facilitation, or if practicing SAFe the Release Train Engineer performs the facilitation role.
  3. Frequency: Scrum of Scrums often follows the same pattern as the daily Scrum/stand-up meeting. A second frequency pattern is risk-based; the frequency of the SoS meetings varies depending on need. Early in a project the SoS meets daily as teams are forming, norming and as early decisions are being made. The SoS meeting become twice weekly in the middle of the project and then returns to daily as a release approaches.
  4. Format: Daily stand-up meetings commonly leverage the classic three question approach (What did I do? What am I going to do? and What are my blockers?). The Scrum of Scrums generally follows a VERY similar format with each participant answering the following four questions:
    1. What did my team do since the last meeting?
    2. What will my team do before we meet again?
    3. Is my team experiencing blockers?
    4. Is my team going to put something in another team’s path?

The meeting follows the same round robin approach with each participant interacting with each other. The facilitator (if used) should never be the focal point of the meeting, nor should the SoS devolve into a purely a status meeting.

The daily stand-up and the SoS are very similar meetings. Both meetings are held to share information, coordinate activities and to identify problems. The scope of SoS meeting is broader than a single team with the meeting providing coordination and planning activities within and across teams.

Next: Using the Scrum of Scrums to Scale Planning and Scrum of Scrum Anti-patterns

OLYMPUS DIGITAL CAMERA

Most Agile frameworks are built on the basis empirical models of process control. These, in turn, are based on periodic observation and adaptation. Each instantiation of the process may be different due to the circumstances any specific team faces. The empirical mode is often compared to the defined process approach, which posits a standard process for all teams that evolves based on the needs of the collective whole rather than a team or small group of teams. Scaled Agile Framework Enterprise (SAFe) program increments are no different. Program increments are similar to a large Scrum sprint beginning with planning and completing with a demonstration and retrospective, known as inspect and adapt. The empirical process of inspect and adapt is one of the core principles in SAFe. At the team level, each sprint culminates with the classic demonstration and retrospective. The same type of discipline is leveraged for program increments. PI level inspect and adapt is composed of three components:

  1. The teams involved in the program increment demonstrate the system with the new features, architectural changes and non-functional requirements that have been completed during the PI. The goal of the demonstration is to evoke feedback based on the overall current status of the overall application from the Agile release train (ART) stakeholders.
  2. Metrics Review. Program metrics are reviewed and discussed as a team. The goal of the metrics review and the ensuing discussion is to ensure that everyone involved understands performance based on tactical experience and measured performance. The combination of experience and measurement helps the team see through their individual cognitive biases.
  3. Retrospective/Problem Solving Workshop. The problem solving workshop provides a platform for the PI participants to first identify the issues experienced during the increment, then to identify which of those that they would like to solve and finally to develop a corrective action plan. SAFe recommends that a structured, root cause analysis is used to analyze the issue and define a corrective action plan (the preferred method uses the “five whys” and Ishikawa Diagrams/Fishbone diagrams – both of these are tried and true techniques used in the quality and process improvement arena). The goal of the process is to generate tangible improvement actions that can be taken into the release planning event for the next program increment.

The inspect and adapt process in SAFe is a means of generating feedback on the product that is being developed and the processes being used to do the work. Feedback is a means of ensuring the Agile release train stays on track. Inspect and Adapt provide feedback so that changes can be made the trajectory of the product being build and to make changes in how work is being done.

Program increment (PI) objectives are summaries.

Program increment (PI) objectives are summaries.

Program increment (PI) objectives are summaries of business and technical goals at multiple levels within a program increment.  In the Scaled Agile Framework (SAFe) there are two levels of PI objectives. Team PI objectives reflect the summary of the specific business and technical goals that have been planned during release planning. Program PI Objectives are at a higher level of abstraction, reflecting the summary of the teams PI objectives to describe the overall program increment. The process of generating the PI objectives serves three primary purposes. They are to:

  1. Create alignment. Teams in the ART synthesize the visions (business context, product and architecture visions) presented at the beginning of the planning event along with the prioritized backlog to generate business and technical stories which are then planned. Stories and spikes are added and planned by the team until they reach capacity. When process of planning is complete the work that has been planned is summarized into the team PI objectives. PI objectives are communication vehicles for the team to share what they are doing to those outside the team and for other teams and stakeholders (those outside the team boundary) to consume and understand. The PI objectives are a stand in for a laundry list user stories that allows all of the ART teams to compare objectives.  The comparison process helps to ensure teams are aligned so that the program increment vision can be delivered. Program PI objectives are generated by synthesizing the teams PI objectives into program PI objectives. Program PI objectives are another tool to ensure all of the teams are aligned.
  2. Generate agreement of feasibility. The process of synthesizing the stories into team PI objectives and then from team PI objectives into program PI objectives is a mechanism to communicate what each team believes feasible in the program increment time frame. As team-level PI objectives are communicated and synthesized into program PI objectives, the ART has another opportunity to expose dependencies or to ensure that dependencies are understood.
  3. Set Expectations. PI objectives, whether at the team or program level, are commitments to perform. PI objectives can be thought of as line in the sand based on plans generated by the teams that are committed to doing the work (rather than plans and objectives made for the teams).

The release planning process is a mechanism for teams to plan the work they are going to do in a program increment. PI objectives are a mechanism to summarize the detail that planning creates so that everyone involved in the Agile release train can agree upon what will be delivered. At the end of a program increment those involved in the increment can reflect on the objectives both at a program and team level and use what was learned to continuously improve.

Lots of varied roles!

SAFe’s release planning event is has been considered both the antithesis of Agile and the secret sauce for scaling Agile to programs and the enterprise. The process is fairly simple, albeit formal. Everyone involved with a program increment (an 8 – 12 week segment of an Agile release train) get together and synchronize on the program increment objectives, plan and then commit to the plan. Who is the “everybody” in a release planning event? The primary participants include:

  1. The Scrum Teams.   Release trains are typically comprised of 50 – 125 people with Scrum teams of 5 – 9 people comprising the lion’s share. Scrum teams are comprised of a Scrum master, product owner and technical personnel. The teams plan, identify risks and dependencies, share knowledge and commit to the plan.
  2. Release Train Engineer. The RTE facilitates the release planning meeting. The RTE is often called the Scrum master of Scrum masters. I have also heard the RTE called a cat herder.
  3. Product Management. Product management provides the vision for the program increment. They interpret the product roadmap and program epics providing direction.
  4. Business Owners. Business owners are the voice of the business. They interact with the teams in the ART to influence and generate an alignment between the program-level program increment objectives and the individual team-level program increment objectives.

Others that are involved include:

  1. CTO, Enterprise Architect and System Architect. These roles provide the architectural vision and an assessment of the Agile environment including changes since the beginning of the last program increment.
  2. Customer Representatives. Customer representatives are an adjunct to the product management personal providing interpretation of the vision.
  3. Release Managers. Release managers assist in planning, managing governing releases. The release manager may be involved in several ARTs.
  4. Engineering Development Managers. The development managers responsible for the applications involved in the release train need to be aware and provide support and knowledge capital to help ensure success.
  5. Tech Lead and Representative Team Members, Developers, QA/Test, Documentation, etc. Representatives from groups the ART must interface with during the program increment need to participate and have decision making power to commit personnel and resources needed to support the ART.
  6. VPs and Executives Responsible for Product Strategy, Delivery and/or other Affected Operations. Anyone that would typically need to review, provide support or resources or approve a plan or direction should be at hand during the release planning event.

It has been said that it takes a village to raise a child. It takes an equally complex number of roles and participants to plan and then generate commitment to that plan.

OLYMPUS DIGITAL CAMERA

The release planning event in the Scaled Agile Framework Enterprise (SAFe) is a two-day program-level planning event that focuses the efforts of a group of teams to pursue a common vision and mission. As we have noted, the event includes participation by everyone involved in the Agile release train (ART), participation is in-person (if humanly possible), occurs every 8 – 12 weeks and has a formal structured agenda.   The agenda has five major components:

  1. Synchronization.  This component seeks to get everyone involved in the ART on the same page in terms of:
    1. Business Context
    2. Product Vision
    3. Architectural Vision
    4. Technical Environment and Standards

Each of these subcomponents provide the team with an understanding of what they are being asked to do and the environment they are being asked to operate within. The flow begins by grounding the team the business context for the program increment (the 8 -12 weeks). Each step after the business context increased in technical detail.

  1. Draft Planning. Leveraging the context, vision, architecture and environmental standards, the teams develop a draft plan. We will explore the process in greater detail in a later essay, however this where the team breakdown the stories they will tackle during the program increment, breaks them down, exposes risks and identifies and minimizes dependencies.   Draft planning usually consumes the second half of the day. The Release Train Engineer will gather the scrum masters together periodically during the draft planning process to ensure teams are sharing dependencies and understand the direction each is heading.
  2. Management Review and Problem Solving. At the end of draft planning, any significant problems with the scope of the program increment, staffing or other resource constraints are generally apparent. After the majority of team has completed their work for the day the management team (including RTE and scrum masters) meets to share what was learned and make the decisions needed to adjust to the constraints. This is must be completed before the beginning of day two.
  3. Final Planning. The teams review the adjustments made during the during the management review the previous evening as a whole group and then break out into teams again to continue planning converting the draft plans into final(ish) plans. Plans are finalized when they are accepted by the business owners.
  4. Review, Re-planning and Acceptance. When the teams plans are finalized they are reviewed by the whole team, the risks are ROAMed, the whole team votes on acceptance (a form of peer review and acceptance), any rework is performed on the plan and finally a retrospective is performed and next steps identified.

The release planning meeting operationalizes a program increment. A program increment represents 8 – 12 week planning horizon within a larger Agile Release Train. The large scale planning event helps keep all of the teams involved in the ART synchronized. The release planning meeting might be SAFe’s special sauce.

Their are meetings of all types!

Their are meetings of all types!

The Scaled Agile Framework Enterprise (SAFe) organizes work using a hierarchy of value stream, agile release train, program increment, sprint and teams. In order to make the structure work, SAFe adds a few events to the standard set of team-level Scrum meetings (sprint planning, daily stand up/Scrum meeting, demonstrations and retrospectives.  The additions include:

  1. Release Planning Meeting. The release planning meeting is used to plan and kick-off a program increment. The meeting is typically held over the course of two days and involves everyone involved with the program increment. The release planning meeting is one of the keystones of the SAFe framework.
  2. Scrum of Scrums: The Scrum of Scrums (SoS) is a meeting of the Scrum Masters using a format that is very similar to the daily stand-up meeting. The SoS is a common tool for scaling Agile and has is part of the Scrum cannon. This meeting is generally facilitated by the release train engineer.
  3. Scrum of Product Owners (optional): In a similar vein as SoS, a Scrum of product owners. The same general format of the classic three questions can be applied from a product owner perspective. The goal of the meeting is generally to keep the product owners informed and involved.
  4. Release Management Meeting: The release management meeting reviews progress toward a developing releasable product. Attributes that are reviewed include scope, quality, and progress toward the release roadmap, quality and potentially other factors.
  5. System Demo: The system demo provides a demonstration of the integrated software. This is typically a demonstration to the business and other key stakeholders. The system demo occurs at the end of each sprint generally after the sprint demo.
  6. Program Increment Solution Demo (Part of the activity SAFe calls Inspect and Adapt): The PI demo reflects everything that was developed (and is done) during the program increment. All release planning participants are part of this demo. The PI solution demo connects the loop with everyone that was involved in planning and also provides a starting point for everyone involved in the next planning event.
  7. Problem Solving Workshop (Part of the activity SAFe calls Inspect and Adapt): The problem solving workshop is a macro version of a retrospective. The workshop generates an improvement backlog that is an input into the next release planning cycle. The problem solving workshop provides all of the teams with time to identify and prioritize problems that may affect the entire Agile release train.

All of these events bear further exploration, however it is important to note that when scaling Agile so that important  large business needs are tackled in a timely fashion, scale comes with a cost. The cost is the need for control structures. Agile has many attributes that contribute to discipline and control such as short feedback loops, iterative planning, demonstration and more, but as the number of teams working toward a common goal gets larger additional mechanisms are generally needed when cadence and synchronization are not sufficient.

If you're running at a different cadence, someone is always falling behind.

If you’re running at a different cadence, someone is always falling behind.

The Scaled Agile Framework Enterprise (SAFe) uses the concept of a hierarchy to translate business strategy and needs to the work a Scrum team does on a daily basis. The hierarchy begins with value streams that are actualized by agile release trains (ART). ARTs are a long lived team of teams this is organized to deliver the support the value stream. While the value stream provides a goal and the ART the long term vehicle for the that support additional structure is needed before teams are set loose. As anyone that has watched teams interact knows, without structure shenanigans will ensue. The program increment provides a structure to connect the long term vision and the scrum teams. Program increments are similar to sprints, but at higher level.

The scrum teams within an ART use sprint cadence as a synchronization and motivational tool to develop solutions in a coordinated manner. Cadence provides short feedback cycles that provide focus. Program increments aggregate a fixed number of sprints together to peruse a common theme or goal such as a group of related features that might comprise one or more releases. The program increment process provides a framework to synchronize planning and development through group planning, group demonstrations and larger scale retrospectives. The program increment functions in much the same manner as the Sprint in Scrum. Here are the attributes of a program increment:

  1. Time-bound planning cycle. Program increments are typically 8 – 12 weeks, four two-week or three-week sprints.
  2. Common theme/goal. The program increment acts a container to focus development on the features and functions needed to deliver common theme or goal. The program increment and release plan do not have to synchronize. Using an example of a hypothetical customer facing website. A program increment could be focused of developing an integrated shopping cart solution with releases being made during the 8 – 12 weeks as major components reach done.  The mantra of SAFe and Agile development is, “develop to cadence, release on demand.” Said more bluntly, develop using sprints and program increments to provide cadence and synchronization and be ready to release when the business needs the functionality.
  3. Ceremonies demarcate program increments. Much like sprint planning and demonstrations and retrospectives mark the beginning and the ending of a sprint, program increments are bounded by ceremonies. The release planning meeting marks the beginning of the program increment and the “inspect and adapt cycle” marks the outer boundary.

A program increment is a time-boxed container with a common theme or goal that helps the multiple teams involved in an ART to stay synchronized. Teams further enhance their synchronicity by integrating frequently, participating in Scrum of Scrums, management dependencies as well as team and system-level demonstration. The duration of the program increment is often syncronized to release schedules and plan, however the two concepts do not have to be synonymous. Agile development is about building implementable functionality every sprint, the macro cadence of the program increment allows that functionality to be bundled into features which can be released as planned or as needed.

The Release Train Engineer is part of the crew that guides the train.

The Release Train Engineer is part of the crew that guides the train.

As groups of coordinated work get larger the tools, techniques and roles needed to keep the work coordinated expand. Complexity requires more controls to keep the train on the tracks. The Release Train Engineer (RTE) is one of the techniques the Scaled Agile Framework Enterprise (SAFe) uses to keep the 50 to 125 cars (number of people in an Agile Release Train or ART) on the track. Formally RTE is a scaled scrum master, however practically the role tends to take on a broader footprint. It could be viewed as a mixture of Scrum master and program leader. The roles and responsibilities of a RTE include:

  • Provides guidance to the release train. While the ART and the teams that are part of the train are predominately self-managing and self-organizing, the RTE provides guidance to help teams adapt to the environment.
  • Organizes, runs and facilitates the release planning meeting. The release planning meeting  is a two day meeting that includes all of the people involved with the ART. The meeting is crucial meeting that kicks-off and drives each program increment.
  • Assists in tracking the ART. In many organizations tracking and reporting fall on the shoulders of the RTE. The function is better placed in a Program Management Office with the RTE providing assistance. Let the RTE focus on facilitating and leading rather than on owning standard administration tasks.
  • Chairs the scrum of scrums (SoS). As a leader and the scrum master of scrum masters, the RTE is perfectly positioned to ensure that teams interact and share cross-team issues and risks by facilitating the SoS.
  • Resolves and/or escalates impediments. Scrum masters facilitate the resolution of impediments within their span of influence. Once an impediment is outside of a scrum master’s span of influence the RTE steps in to remove or escalate the impediment to someone that can remove the impediment.
  • Facilitates intergroup communication and dependency resolution. Much of the role of the RTE is facilitating communication between groups. In a perfect world all teams would have the proper understanding of what was going on in other teams that they could impact or could impact them, however many times the environment precludes that possibility. The RTE helps to keep information moving so that the teams can focus on delivery.
  • Facilitates process improvement at the ART level. While teams pursue improvements based on retrospectives, the RTE facilitates ensuring the overall processes are continually being refined. An example of one of process improvement events that the RTE leads is the retrospective that caps the release planning meeting.
  • Cajoles, leads and In general, helps make stuff happen. All leaders use their social influence to attain a goal. RTE’s are no different. SAFe (and most other frameworks) use goals that can be traced from business value to the work a team delivers. Work in ALL organizations is goal oriented. Leaders help everyone involved focus on those goals.

The release train engineer has a hand on the throttle of the agile release train, but just like the engineer on the freight train they are only one part of the process that that decides speed and direction. Locomotive engineers inspect their trains, monitor performance, communicate and interact with team members and passengers. They support getting the train to its appointed destination not only on-time but safely.

A value stream is a set of activities that are intended to create and deliver a consistent set of deliverables that are of value to customers. Using the train metaphor, value is the cargo the train delivers.

A value stream is a set of activities that are intended to create and deliver a consistent set of deliverables that are of value to customers. Using the train metaphor, value is the cargo the train delivers.

The Agile Release Train (ART) in SAFe is the primary high-level tool used to organize activity to deliver value. Other frameworks (Agile and classic) use projects and programs to organize around the delivery of value and value’s alter ego, funding. ARTs are used to deliver, enhance and support the functionality needed for business facing value streams to exist within an organization.

A value stream is a set of activities that are intended to create and deliver a consistent set of deliverables that are of value to customers. Using the train metaphor, value is the cargo the train delivers. The term consistent in the definition is important to delineate an ART from a typical project or program. The Project Management Institute defines a project as a temporary group activity designed to produce a unique product, service or result. A project or program might initiate or support a value stream in a non-scaled Agile organization, however the focus is on a temporary endeavor rather than on support an ongoing or consistent endeavor. The long-term focus makes it significantly easier to embrace Agile principles.

Value streams reflect a business orientation that integrate strategic themes and needs into how an organization delivers value. The products and services of most organizations tend to be long lived, therefore value streams have the same long-term orientation. SAFe ARTs organize activity around value streams. In practice a value stream in an organization can be large enough to support multiple ARTs. A value stream is a requirement for an ART whether the ART develops a new value stream, enhances or supports an existing value stream, without access to business value it hard to generate strategic alignment within the organization. When ARTs (or any significant part of an organization) are out of strategic alignment, support in terms of budget, people and resources will be difficult to acquire. This will lead to disillusionment with the ART. Using our train metaphor, the engines, train cars and crew will be dispersed to other trains. Without strategic alignment an organization will not be able to support the structure of an ART and stable teams that are needed.

Value streams present the whole path an organization takes to deliver value. The phrase “concept to cash” is often used describe the breadth of vision that a value stream must have. Value stream taking the big picture takes a on a whole business context rather than purely a software development project or programs. To adjust to the breadth of vision ARTs need to incorporate the needs of business process, architecture, systems as well as application software. Agile release chains make sense as long as they serve a real value chain.