A count of the Pokemon in my local park yesterday!

In today’s complex software development environment, it is easy to see every data collection issue as a complex automation problem. For example, I recently had a discussion with a team that is trying to determine how they would know whether a coding standard change they were contemplating would be effective. The team proposed capturing defects the coder/developer pairs found in TFS. The plan was to enter each defect found into the tool. After a discussion, the team decided to adopt a much simpler data collection solution with very low overhead. One of the classic quality tools is a tally sheet or check sheet. A check sheet is a form that used to collect information as it happens as a set of checkmarks or tally marks on the form. Check sheets are a very simple data collection tool. They are often a great way to pilot data collection before spending time and effort on automation or adding overhead to work. A check sheet is a form of the histogram where data where data collection and categorization occurs at the same time as visualization. (more…)

Tight stairway to the sky

Block Out Distractions

As the New Year’s celebration approaches, it is useful to reflect on how we can become more efficient and effective.  Continuing breaking down nine of the techniques I have found useful over the past year we explore a selection ranging from saying “no” to time boxing. The goal is to not only get more done but to get the right things done.   (more…)

Different philosophies yields different perspectives.

Different philosophies yields different perspectives.

The approach an organization takes to Agile user acceptance testing (UAT) depends on a number of factors. These factors generate the environment that will define how the organization approaches Agile testing.  The types of factors that influence Agile testing can be as global as whether customer satisfaction or efficiency drives behavior, or as tactical was whether manual or automated testing is used. While the potential factors that influence an organization’s Agile testing philosophy can be numerous, generally three factors are the most definitive: the influence of lean, use of test frameworks and the stance on text automation.

Lean: Agile and lean philosophies have become difficult to untangle. Philosophically, Agile can be represented by short, complete cycles with frequent customer feedback and an acceptance of the potential for change that feedback creates.  Philosophically, lean represents the relentless pursuit of efficiency. A common combination of the two philosophies is the use of the single-source information practice, where information is captured once and then reused. Acceptance test cases represent a perfect scenario to merge Agile and lean philosophies. Requirements are captured once and written as acceptance tests. This application of the single source philosophy avoids documentation (avoidance of a specific requirements artifact), reduces the possibility of transposition or misinterpretations between requirement artifacts, and can reduce the effort needed for bi-directional traceability. Organization that have not embraced a lean philosophy along with Agile may well leverage separate documentation techniques for stories and acceptance test cases.

Test Frameworks: Frameworks, such as Cucumber, provide a structure to think about acceptance tests.  For example, the “given, when, then” structure used in Cucumber provides a mechanism to not only focus on writing tests that can be automated, but also tests that are readable. Another framework for defining tests is “arrange, act, assert,” however this is generally used as a unit test pattern rather than an acceptance test pattern.  Readable tests increase the likelihood that more than just the most technical team members will be actively involved, which will result in better tests and higher delivered quality. Test definition frameworks like Cucumber are are generally linked to the use of test automation tools, however use frameworks to provide structure and discipline to acceptance criteria and testing even if you are not using a test automation tool.

Automation:  Test automation can be a game changer. Automation of acceptance testing can help increase the overall productivity of acceptance testing and increase the efficiency. Automation generally means that you can run more tests in less time than a manual tester.  Automated tests can also be rerun as the code base changes. However, automation is not cost free (we will explore this in depth later in the week), requiring time and effort to develop and maintain testing scripts (and possibly to pay for the software). In every case, the use of test automation is a balancing act between cost and benefit. Remember that automated testing is a great tool for efficient acceptance testing, but ultimately automation will need to be combined with manual techniques like exploratory testing to provide the broadest mirror of the production environment.

How an organization approaches Agile acceptance testing in the short run will be driven by the organization’s philosophies on lean, frameworks and automation. None of the decisions that an organization makes in these categories are trivial. A lean focus will guide an organization toward minimalistic techniques and methods, which influence how teams work and test.  Adopting standard frameworks for acceptance enhances communication within the team. Testing automation is a great tool but will require time, effort and money in order to implement.  Organizational philosophy will determine which paths are possible and in the end which path is taken.