Audio Version:  SPaMCAST 137

The world is complex; it is comprised of a myriad of processes each with their own slice of complexity. We all fear being sucked into whirlpool of complexity as evidenced by the laundry list of best selling books that purport to provide the tools needed to take control of our personal and professional worlds. In the process improvement world, models whether statistical or flowcharts are used to represent order. These tools are used to characterize the software development and maintenance processes we use on a day-to-day basis. Models are abstractions, generalizations of the real world that are important so that we can understand, even approximately, how the real world works and how people in the real world react to inputs. Abstractions of abstractions however, can be a bridge too far. By a bridge too far I suggest that simplifications of simplifications can become extreme enough cause those enticed by the simplicity these simplifications provide to make incorrect decisions. An example is the use of a single number to represent productivity (an industry average for example) when even a quadratic equation is a simplification that can border on uselessness. A small project will have a fundamentally different rate of productivity than a large project; there is no single number that represents productivity. A better representation is a curve that varies based on project size and other attributes. Why are we afraid of complexity when we know the world around us is complex?

Examples of the impact of adding complexity to a system or product abound.  Media distribution provides an example how complexity can slow or stop general implementation of a new product or process.  The following table shows four media distribution models, the cost and level involvement of users to acquire the media and the rate of adoption.  Note that degree of complexity and involvement are user perceptions and are reflection of reset paradigms based on the last successful change in the environment.

Media Type               Cost                Effort              Adoption

Radio                          Free                Low                Fast

Television                 Free                Low                Fast

Cable                          Subscription Moderate       Slow

Netflix Streaming    Subscription Low                Fast

Perceived complexity influences adoption rates therefore marketers strive for product and message simplicity.

JRR Tolkien wrote of the rings of power.  Three for the Elves, seven for the dwarves, nine for men, and one, The One Ring to rules them all.  Whether in literature or real life we, as change agents, are seduced by the simplicity of using metaphors to represent complex scenarios for the same reasons as our cousins in marketing; they are easy to understand and sell.

Methodologies and frameworks are examples of metaphors and abstractions.  Waterfall, agile, Scrum, Kanban, CMMI are all examples of more or less complex abstractions of the real world.  In the software measurement world industry average curves showing behavior over a range of projects is one level of abstraction while the single number industry average that ignores the range of projects in an organization is another. Why when we know we live in a complex world do we first crave the simplest metaphor then worse, take action based on the simplest metaphor ignoring the complexity? Is it just a marketing convenience?

Basic human psychology drives a need for certainty, many times we think in binary terms; black and white.  The certainty we seek is unfortunately unattainable which causes conflict within ourselves and within the communities we join or are forced to join.  Managing the conflict that the need for certainty generates is a cost that does not need to be borne if we accept a rational dose of uncertainty.  The cost of conflict is generated by two flavors of mechanisms.

1.  Compensation

2. Conflict avoidance

There are similarities and differences between the mechanisms.

Deepak Chopra has pointed out that in quantum physics, matter fluctuates between a wave and a particle, only “collapsing into manifestation” in response to an observer. Not only do differing levels of awareness lead people to perceive the world differently, but through our awareness and conscious and unconscious layers of intention, we all participate in creating what manifests. This is an uncomfortable level of uncertainty for anyone seeking to change an organization.

To compensate for the lack of certainty, we create theories about how things occur. As we seek to understand uncertainty the natural extension is to formulate explanations.  Explanations are mechanisms to shift variability back to a deterministic mode.  Models are a mechanism to explain variability.  The thought is that what we can explain, we can control.  Whether our model is true or untrue is not as material as the exercise of creating theories and models to compensate for a lack of order in the outside world.  Mike Copeland suggests that Humans choose to create theories that have been elevated to “knowns” or beliefs to lessen the uncertainty of ambiguity.  It is as if the creation of models can create order and relationships where causality does not exist.

Models, theories and abstractions are mechanisms to compensate for the disconnection between a need for certainty and the real world which is far more uncertain.  They are mechanisms created that compensate and deliver the appearance of certainty.  I would suggest that abstractions continue layer upon layer until a mass audience can relate to concept a quickly.  The problem with this compensation technique is that as simplifications are simplified, so they more consumable they lose explanative power and decisions made for simplifications of simplifications are significantly riskier.

Conflict within organizations can represent an interesting dichotomy; constructive conflict and destructive conflict.  Constructive Conflict can help forge ideas, removing the scale revealing the core of the idea by forcing people to confront possible defects in a solution so that the best idea survives and adds value.  However, for all of the possible good constructive conflict can deliver, destructive conflict is always injurious to the organization.  The distinction between constructive and destructive is often hard to discern increasing the risk of any perceived conflict.  The difficulty discerning the difference and the down side risk of getting the distinction wrong makes being conflict avoidant a viable strategy.  Talking in abstractions and abstractions of abstractions are tools to avoid conflict.  This is true for a number of reasons.  The first is that the more abstract an idea is, the more easily the similarities of world views can be emphasized smoothing over potential divisions within a group.  In competitive environments abstractions are a means of withholding information or as a tool to emphasize conformity.   In all of these cases rather than risking conflict even though it might be constructive, abstractions are used to defuse conflict.

Abstractions are simplified representations of the real world.   Abstractions are appealing because their simplicity allows an emotional connection (the simpler the more powerful) but in their simplicity they lose their power to describe a more uncertain world.  Complexity appeals to intellectual thought and has higher explanatory power at the cost of emotional connection. Simple abstractions and models are tools for dealing with fear of uncertainty, a tool for conflict avoidance and finally as a tool to diffuse anger but what they should not be is the driver of strategic direction. My advice is to temper emotionalism with the complexity and uncertainty of the real world.