Different philosophies yields different perspectives.

Different philosophies yields different perspectives.

The approach an organization takes to Agile user acceptance testing (UAT) depends on a number of factors. These factors generate the environment that will define how the organization approaches Agile testing.  The types of factors that influence Agile testing can be as global as whether customer satisfaction or efficiency drives behavior, or as tactical was whether manual or automated testing is used. While the potential factors that influence an organization’s Agile testing philosophy can be numerous, generally three factors are the most definitive: the influence of lean, use of test frameworks and the stance on text automation.

Lean: Agile and lean philosophies have become difficult to untangle. Philosophically, Agile can be represented by short, complete cycles with frequent customer feedback and an acceptance of the potential for change that feedback creates.  Philosophically, lean represents the relentless pursuit of efficiency. A common combination of the two philosophies is the use of the single-source information practice, where information is captured once and then reused. Acceptance test cases represent a perfect scenario to merge Agile and lean philosophies. Requirements are captured once and written as acceptance tests. This application of the single source philosophy avoids documentation (avoidance of a specific requirements artifact), reduces the possibility of transposition or misinterpretations between requirement artifacts, and can reduce the effort needed for bi-directional traceability. Organization that have not embraced a lean philosophy along with Agile may well leverage separate documentation techniques for stories and acceptance test cases.

Test Frameworks: Frameworks, such as Cucumber, provide a structure to think about acceptance tests.  For example, the “given, when, then” structure used in Cucumber provides a mechanism to not only focus on writing tests that can be automated, but also tests that are readable. Another framework for defining tests is “arrange, act, assert,” however this is generally used as a unit test pattern rather than an acceptance test pattern.  Readable tests increase the likelihood that more than just the most technical team members will be actively involved, which will result in better tests and higher delivered quality. Test definition frameworks like Cucumber are are generally linked to the use of test automation tools, however use frameworks to provide structure and discipline to acceptance criteria and testing even if you are not using a test automation tool.

Automation:  Test automation can be a game changer. Automation of acceptance testing can help increase the overall productivity of acceptance testing and increase the efficiency. Automation generally means that you can run more tests in less time than a manual tester.  Automated tests can also be rerun as the code base changes. However, automation is not cost free (we will explore this in depth later in the week), requiring time and effort to develop and maintain testing scripts (and possibly to pay for the software). In every case, the use of test automation is a balancing act between cost and benefit. Remember that automated testing is a great tool for efficient acceptance testing, but ultimately automation will need to be combined with manual techniques like exploratory testing to provide the broadest mirror of the production environment.

How an organization approaches Agile acceptance testing in the short run will be driven by the organization’s philosophies on lean, frameworks and automation. None of the decisions that an organization makes in these categories are trivial. A lean focus will guide an organization toward minimalistic techniques and methods, which influence how teams work and test.  Adopting standard frameworks for acceptance enhances communication within the team. Testing automation is a great tool but will require time, effort and money in order to implement.  Organizational philosophy will determine which paths are possible and in the end which path is taken.