No Silver Bullet is the 16th installment of the Re-Read Saturday of the The Mythical Man-Month by Fred P. Brooks. No Silver Bullet is the longest of the essays, and even includes an abstract and introduction. In this essay, Brooks discusses hard parts of software development and how most of the productivity gains of the previous decades were focused around improving the processes around software development rather than addressing the really hard parts. The hard parts are capturing and translating requirements into a design. In the abstract Brooks makes four suggestions:
- Buy the software that can be bought.
- Use prototyping to get requirements.
- Grow software organically.
- Identify and develop the great conceptual designers of the rising generation.
Commercial, off-the-shelf software, prototypes, iterations and great people — I have never heard of any of these. Just kidding. These four suggestions are the core strategies that define how most internal IT organizations operate in the 21st century. While the interleaving years has not spawned a silver bullet (albeit I can regale you about several that have been tried) it doesn’t mean that we should stop looking. The process of looking creates an atmosphere of continuous process improvement yielding increased productivity.
Brooks broke the problem in two parts. The first is the essence of software development, the difficulties that are driven by the nature of business problem translated into software, this an intrinsic attribute. The second are the accidental difficulties are driven by how software is produced; the extrinsic attributes of software which are usually core to the process. The hard part of building software is the specification, design, and testing of the conceptual construct, the code. The hard part is not the labor of building and testing the representation of business problem. In other words, coming up with the concept of the physical construction of the code is not the hard part, but rather the hard part is figuring out where to start. It boils down to complexity.
There are four inherent properties of software:
Software is complex because each part is different. Even when software looks similar, assemblies of code often serve radically different business goals. I have often used the analogy of software modules and Lego, but in reality the analogy fails because reuse is a myth at a high level. One the reasons macro level reusability has been hard is complexity. Software is not really a box of Legos that can be easily assembled. Simply put, complexity is intrinsic to developing software, if we remove complexity from the description of what is being built, we risk losing the essence which is critical to understanding. Unfortunately, complexity causes problems both technically and from a management perspective, making communication in all of its forms more difficult. All of this yields problems in the software.
Much of the complexity faced in development is arbitrarily generated. The interaction of stakeholders and business needs generate layers of apparently arbitrary decisions that are often times at odds with a less complex approach. The impact of layers of decisions makes building software difficult. Consider coding a program to implement any country’s tax code. Madness!
Once upon a time I sat in a management meeting and listened to a pitch for an application that would address the company’s needs now and forever. What was funny was that the director made the pitch with a straight face. One of the three constants in life is change (along with death and taxes). Whereas hardware can be replaced, software applications are difficult enough that they must be engineered to change and evolve. The need to build software that can change and evolve increases the degree of difficulty.
Software can’t be perfectly visualized. That fact hinders communication, which makes building software harder. The visualizations available, such as flow charts, are abstractions that remove some level of complexity. In statistical analysis it is often said that the simpler the model, the lower explanatory power. Imperfect visualizations increase complexity by making it harder to understand what needs to be built, and therefore more difficult to understand.
Over the history of software development, there have been several improvements that make the process (the accidental attributes) of building software easier. Brooks specifically calls out three:
- High-level languages which are closer to natural language.
- Computer time sharing which shortens the time span between an action and the ability to see results.
- Unified programming environments that allow programs and libraries to be used together, reducing integration complexity
In addition we could add methods and frameworks, such as Scrum, Extreme Programing, DevOps, Continuous integration and the Capability Maturity Model Integration to name just a few. Each seeks to make the development process easier, more consistent and repeatable. None address the intrinsic difficulty of the solving the actual business problem.
One of the aspects of Brooks’ essays that I enjoy is that he is never doom and gloom, and in No Silver Bullet, Brooks does not disappoint. Brooks suggests several slivers of hope (his words) being pursued by the software development community. They include:
- Languages that encourage modern design techniques and modularization, such as Ada.
- Object-oriented (OO) programming, which reduces accidental complexity by shifting the focus to the essence of design.
- Artificial intelligence, although Brooks suggests that AI will have little effect as he does not see how it will change the process programming.
- Expert systems as represented in rules bases and inference engines will separate application complexity from the act of programing. Expert systems may also be a tool to distribute good practices.
- Automatic programming also known as code generation.
- Environments and tools continue to improve based on research and evolution.
- Workstations used by developers are becoming more powerful.
The “slivers of hope,” with the possible excepting of OO, have had only pockets of success. However the research and experiments with these items and others, such as Agile methods, have created an environment in which continuous improvement is expected.
Brooks rounds the essay out by returning to four measures that address the essence of the problem: the difficulty in capturing the business problem in a design. He suggests:
- Whenever possible, don’t build from scratch, rather buy and assemble solutions.
- Rapid prototyping is a better mechanism to generate an understanding of what to build. The goal is to generate better technical requirements.
- Develop incrementally (and get feedback). Brooks suggests that we grow rather than build software.
- Identify and use great designers. Development is a people-process, therefore improving the people translates directly into higher quality and productivity.
For many of us the admonition that there is no silver bullet is not a surprise. What is a surprise is that every few years a new silver bullet is touted in the marketplace. Very few of these silver bullets address the essence of what makes developing software hard. In most cases they addressing the accidental attributes that will improve the quality and cost profile, but will not deliver exponential levels of change.
Previous installments of the Re-read of The Mythical Man-Month