No Mowing Sign

The Environment is Complex

Having been involved in the world of buying, building, maintaining, and testing software for many years, one of the longest running conversations between everyone involved with delivering value is the impact of complexity on cost, effort, quality and even on the ultimate solution to business problems.  The concept of complexity and the impact of complexity is unfortunately – complex.   The importance of developing an understanding of complexity is complicated by a lack of a crisp definition and a confusion of the topic with the concept of complicated.  The difference between complicated and complex is not a mere nuance; the distinction will affect the options we perceive are available to solve any specific problem.

Most of the dictionary definitions of complexity start us down the path of unraveling the complex from the complicated.  For example, Merriam-Webster defines complex as “a whole made up of complicated or interrelated parts.” The operational definition of complexity that I have found useful in the systems world is the interaction of components in which the outcome is not perfectly predictable based on the known or measured inputs.  This contrasts to the concept of a complicated system.  A complicated system is comprised of one or many parts that might be hard to understand or analyze, but once understood the relationship between input and outcome is predictable.  The path of a hurricane is complex while the path of a rocket is complicated. Perhaps more germanely, projects are complex while software programs after they are written are generally only complicated. Note the distinction between what is complicated and complex changes over time based on knowledge and computing power, someday the behavior of hurricanes will become merely complicated.

Often the main wild card that shifts a piece of work from complicated to complex is people.  The interactions of complicated components coupled with people whose behavior is driven by relationships, self-organization, interconnections and even evolution loosens the relationship between inputs and outputs. Paul Gibbons, in his book The Science of Successful Organizational Change (See Chapter 4, Page 104 or SPaMCAST Re-read), defines complexity as being comprised of three features.

  1.       Multiplicity – Lots of moving parts.
  2.       Nonlinearity – The input and output are not linearly related.
  3.       Emergence – Behaviors arise based on rules that are novel.

Software-centric work is rarely simply a complicated process; there are too many chaotic people involved and too much discovery required to deliver value. The big gotcha is that many process improvements in software-centric environments are based on a belief that mirrors the definition of complicated rather than complex systems.  Changes tend to focus on influencing processes and constraints as a tool to elicit improved productivity and efficiency. Constraints are often used (and are useful) to keep complicated processes on course but if the process you are addressing includes people or discretion, it is complex.  Constraints alone may not have the desired outcome over the long-term. During the 1990’s many process improvement programs helped organizations codify and harden their software lifecycles without regard to nonlinearity and emergent behavior. In the short run teams and organizations saw improvement from using constraints alone. However, in the long run, the outcome was that very little improvement occurred and more disturbingly, time-to-market slowed, costs increased and projects still failed (this could have been influenced by interaction with other complex systems).  This was one of the contributing factors that lead to the Agile movement.

Agile and the concepts of experimentation, failing fast and learning combined with the knowledge of the distinction between complicated and complex systems provide hope that change can improve the delivery of value inside and outside the organization. We recognize that we work in a larger system that is not just complicated and complex but sometimes even chaotic.