Let these principles be a caution!

Let these principles be a caution!

In Testing Principles Part 1: This is not Pokémon we noted that we need to strive to find ways of not injecting defects into systems, because while testing can find many defects, it will never be able to remove them all. The next set of principles suggests both where and when to work and then finally leaves us with a strong and sobering reminder. The next set of principles includes:

4. Early testing. Testing includes activities that execute the product (dynamic testing) and activities that review the product (static testing). In software, most people would recognize dynamic testing, which includes executing test cases and comparing results.  Static testing includes reviews and inspections in which a person (or tool in some cases) looks at the code or deliverable and compares it to requirements or another standard. Reviews and inspections can be applied to any piece of work at any time in the development life cycle. Reviewing or testing work products as soon as they are available will find defects earlier in the process and will reduce the possibility of rework.

5. Defect clustering. When you find one defect in a deliverable there will generally be others waiting to be found. The clustering can be caused by the technical or business complexity of the product, misinterpretation of a requirement or other issues (for example, left to my own devices I would get the “ie” thing wrong when spelling word – thank you static testing tool: spell check). Given that defects tend to cluster, if the budget for testing isn’t unlimited then spending more time on areas where defects have been found is a good risk mitigation strategy.

6. Testing is context dependent.  The type of testing that will be the most effective and efficient is driven by the type of product or project. For example, a list of user stories can be reviewed or inspected but can’t be dynamically tested (words can’t be executed like code). Programmers will unit test code based on their knowledge of the structure of the code (white box testing) while product owners and other business stakeholders will test based on understanding of how the overall product will behave (black box testing).

Principle 7 could have been included in part one, however it serves well as a final cautionary tale.

7. Absence of errors fallacy. Just because you have found and corrected all of the defects possible does not mean that the product being delivered is what the customer wants or is useful. My personal translation of this principle is that unless you build the right thing, building it right isn’t even an interesting conversation.

The seven testing principles lead us to understand that we need to focus our efforts by building the right product, using risk to focus our limited resources and find defects as early as possible. Testing in an important part of delivering value from any project, however it is never sufficient.  If you remember one concept based on the seven Testing Principles it is that we can’t test in quality or value.  Those two attributes require that everyone on the project consider quality and value rather than putting that mantle on the shoulders of testers alone.