Greater scale, more problems?

Greater scale, more problems?

In 2016, scaling Agile is a contentious topic.  Frameworks and techniques for scaling are often lambasted as semi-Agile or perhaps even backdoor waterfall techniques. Occasionally you still hear that if a piece of work is too big for one team to complete in a reasonable period of time, it should be broken down or just not done. Rather than throw the baby out with the bathwater, many organizations have taken a more pragmatic approach and adopted techniques to scale Agile. These techniques reflect the differences between a single-team Agile project and a scaled project.  Careful observation can trace most of these perceptions back to a single difference.

Why do organizations perceive scaled Agile projects as different than a project completed by a single team? I recently asked a group of developers from banks, insurance companies, startups and government agencies during a social gathering at Podcamp Toronto to list the attributes they thought were different. The list was quite long and included:

  1. Higher levels of coordination.
  2. Need to synchronize deliveries.
  3. More communication mechanisms.
  4. A need for common architectures.
  5. More measurement and metrics.
  6. More standards (and more enforcement).
  7. More stakeholders.
  8. More teams.
  9. More risk.
  10. More reporting.
  11. Backlogs: More than one.
  12. Backlogs: Longer list of requirements and more work to manage the backlog.
  13. Contention between features and functional layers.
  14. Contention for who is working on a given program at any one time (the person standing next to me was a devotee of DevOps).

The list could have been longer if we had more time and paper to take notes.  It would be easy to suggest that the primary reason is #8: more teams and, therefore, more people.  While more people is a contributing factor, the more basic issue is philosophy and economics. The primary reason scaled Agile is different is the conflict between the goals a team embraces to deliver a sprint, iteration or release and the goals that the larger project embraces to deliver a larger group of functions.

Agile teams, with their product owner, decide which stories they will accept during an interaction, and then self-organize to deliver value. Team-level planning, whether a team is using Scum or some other version of the planning game, is a process of setting goals and making public commitments.  Goals and commitments are probably related to the overall goal of the project or product but are rarely perfectly synchronized.  To make matters worse, as soon as real life begins to rear its ugly head, teams optimize plans and goals to address the conditions they are facing.  This is a fact of life that very few IT professionals would dispute.

Why does this mismatch occur even when everyone knows it can happen? Taking a page from the theory of the economic man and extrapolating it to the team level we immediately see that a team will act out of self-interest to maximize the chances of meeting the team’s goals. It is easy to recognize the behavior as a form of local optimization often seen in process improvement efforts that are not using system thinking. Any approach to scaling has to address the potential for local goal optimization at the expense of the high-level project or product goals. 

The attributes suggested by the cadre of Torontonian developers are a reflection of trying to control for the variance in goals between and amongst teams.  The attributes are a second order abstraction.  For example, common architectures and synchronization mechanism are tools that help keep teams working the same direction.  When scaling Agile organizations need to address the steps they take to move from teams to teams-of-teams in a manner that solves for the basic human need for goal attainment.