Definition of done

Definition of done

In “User Stories Are For Technical, Operational and Risk Stories Too!” we made the case that user stories are not just for customer facing user interfaces.  But, we only partially answered the question:

“I would like to see some sample user stories in infrastructure projects such as network migration or upgrade, storage provisioning, or server provisioning.  I think storage provisioning would be a little bit complex due to hardening and other security requirements.”

The nuance in the question centers on the phrase “hardening and other security requirements.” There are two approaches that I often use for stories with nonfunctional requirements.  The first (and not my favorite) is through a final hardening sprint. The second approach is splitting the user stories into thin slices and then incorporating the nonfunctional requirements into the definition of done. (more…)

Definition of done

Definition of done

In “User Stories Are For Technical, Operational and Risk Stories Too!” we made the case that user stories are not just for customer facing user interfaces.  But, we only partially answered the question:

“I would like to see some sample user stories in infrastructure projects such as network migration or upgrade, storage provisioning, or server provisioning.  I think storage provisioning would be a little bit complex due to hardening and other security requirements.”

The nuance in the question centers on the phrase “hardening and other security requirements.” There are two approaches that I often use for stories with nonfunctional requirements.  The first (and not my favorite) is through a final hardening sprint. The second approach is splitting the user stories into thin slices and then incorporating the nonfunctional requirements into the definition of done.

The first solution is leveraging a final hardening sprint. In this scenario an integrated solution is assembled, tested and tuned. Many organizations wait to perform security testing until they have a fully integrated system. In some cases, such as a high security environment or scenario where entire solution is not available to integrate into a single environment, this approach is difficult to avoid. The difficulty with this approach is that it puts off gaining valuable feedback on security and integration until very late in the project. Finding problems late in the project can imperil delivery or cause defects to be pushed into production (Go fever is the pressure to implement even though there are known problems).

The second solution begins by incorporating hardening and security requirements into the story’s definition of done. The definition of done is the requirements that the software must meet to be considered complete. Adding hardening and security requirements to the definition of done will cause the team to add specific tasks to both ensure the requirements are built into the solution of the story and then prove they are implemented. The added tasks will either require the team to accept fewer stories into the sprint or to more thinly slice each story.  This approach is optimal in organizations that continually integrate and test functionality as it is built, however I recommend it for any organization even when an overall final test is required.

A better approach is to combine the both options.  For example, a manufacturer of financial hardware I spoke with recently develops hardware and software on different development tracks which requires a final integration, security and a stress test sprint. In order to avoid potential shipping delays, which are viewed as potentially catastrophic to the business, security and hardening requirements were added to the definition of done for both hardware and software stories. Teams built testing appliances and harnesses to generate early feedback on the nonfunctional requirements.     

Security and hardening are nonfunctional requirements that affect how hardware or software requirements are built, tested and delivered.  These nonfunctional requirements are tasks, not separate stories.  Task and process requirements are better suited as components in the definition of done that are applied to stories and help teams plan their work.

In need of some stabilization...

In need of some stabilization…

One of the central tenants of Scrum is the principle of delivering potentially shippable, functional software at the end of every sprint. This is an extension of the Agile Manifesto’s principle of delivering working software frequently. One generally accepted interpretation of this principle is that the software in production should be implementable at the end of the sprint. In order for this to happen, stories must be coded, tested (in various ways) and the broader user community needs to be convinced that it will work. The second most common interpretation of the principle focuses on the word potentially in the phrase ‘potentially shippable software’. Stressing potentially helps teams and organizations rationalize building a backlog of finished work that is released periodically or running software through final overall set of acceptance tests.

The solution some organizations adopt when they need a final set of tests is to use a hybrid practice called a hardening sprint. For example, Rumadak commented on the entry, Agile User Acceptance Testing: What, Who and Why, that “We generally have one sprint before release for regression and UAT.” I see three primary reasons for leveraging a hardening sprint. In these special cases planning and executing a hardening sprint makes sense due to legal or environmental constraints. They are:

  1. Some teams are contractually mandated to do a final acceptance test before production turnover. This scenario typically occurs in arm’s length outsourcing projects or in situations where you are purchasing commercial, off-the-shelf software.  The goal in this scenario is ensure that the organization buying the software is getting what they paid for. Note: In some highly regulated environments a final acceptance test is required.
  2. Some highly complex environments make integration and acceptance testing in a production-like environment difficult. Therefore waiting until the end of a project to do final integration and acceptance testing may be only option (in the short term). For example, when I worked in the credit card processing world, where projects sometimes needed a full test of the file maintenance cycle, including user interaction, which took roughly 36 to 48 hours and generally occurred on weekends due to the complexity of the technical environment and the resources that had to be available to run the tests.  In that case, running a daily, weekly or even bi-weekly cycle would have been prohibitive.
  3. Some types of testing require the overall system to be effective.  For example, overall stress and security testing not only requires as close to a production-like environment as possible, but also needs all of the functionality that will be delivered to actually test.

In a hardening sprint the team will run tests and will fix anything that breaks. A hardening sprint is not a backdoor to continue developing new stories or functions. The use of hardening sprints for either acceptance testing or integration testing does not preclude running integration and acceptance tests as part of every sprint.  The discipline of writing acceptance tests as part of every story and using those tests as criteria to determine whether a story is complete makes sense. Running acceptance tests every sprint makes sense even if a final overall hardening sprint is required.  Running acceptance tests as early as possible means teams will find problems that effect customer usability earlier, reducing the possibility of significant rework and  project  delays that might occur if we just wait for the hardening sprint.