Expectations and environment affect behavior.

Expectations and environment affect behavior.

When I was a manager of QA (testing), my manager more than once shared the expectation that testing should have found all of the defects in a piece of software before it was put into production. I actually think that expectation found its way into my annual objectives just prior to my changing jobs. Not only is it not possible to find all defects, the responsibility for defect removal must be shared by developers and testers alike. Organizational management can impact test efficiency and effectiveness in many ways, however three are fairly common.

  1. Unrealistic expectations – One of the principles of testing is that exhaustive testing is impossible.  The over-emphasis of removing all defects or perhaps slightly less dangerous – all significant defects – takes the focus away reducing risks. Testers and test managers must use testing and other techniques (reviews, for example) to focus the available time and effort they have on what is important and risky. Developing an understanding of potential impact and possibility of problems (risk) is needed to target testing resources. That is impossible when trying to find all possible problems.
  2. Undervaluing /Underfunding testing – One the indicators of how an organization values test (and testers) is funding. Funding includes people, tools and other resources. Organizations can have enough testers and but not the proper logistics to test effectively or efficiently.  Another common indication that testing is considered a commodity or not a core function is when an organization outsources testing.
  3. Failure to pursue root causes of defects – Testing shows that defects exist (another principle of testing). Testing does not create defects; said another way defects come from somewhere else. The only way to stem the tide of defects is to change how the software is developed by improving the development process (inclusive of people, process and tools) by finding the root cause of defects and then improving the process. If process improvement does not include the development process the rate of defects production will not change. Ignoring the lessons from defects found in testing dooms the organization to repeating the mistakes that led to those defects.

Organizational management can have a direct impact on the effectiveness and efficiency of testing.  The actions that are taken are very rarely because they want buggy software, but rather because they are unsure of the role, scope and goal of testing in the development and maintenance of software.  As a general rule, the role of testing whether through reviews requirements, design and code or by executing cases is to find defects. Software is complex enough that exhaustive testing is either not possible or not cost effective, therefore the goal of testing is to reduce the risk to the users of the software (and to the organization) that defects that will be delivered. The organization has the responsibility to provide the resources and tools needed to meet the goals and strategy of the testing that is needed to meet the organization’s goals.

Advertisements