One of the most common measures of software quality is a count of defects. Conceptually it goes as follows: the more defects generated (and found), the lower the quality of the software. The process of finding, categorizing, counting and resolving defects not only improves the quality of the delivered software but is useful for improving the processes used to create the defects (and the software). The defect management approach is common because it can be simple or complex, require little or a lot of effort to execute, or predictive or reactive depending on an organization’s need.
A simple defect management process would include:
- Information Capture: Define the information needed to report and identify defects and the tools needed for reporting and storage. Information typically includes the defect name, description, who is impacted, priority and contact information. Different defect management approaches will require different data. Only collect the minimum data needed.
- Prioritization: Typically the second step in the process is to triage the defect and then to set the priority for resolution. Every organization I have ever been involved with has their own set of criteria for determining priority. In general, prioritization can be simply stated as the more significant impact a defect causes, the higher the priority fixing the defect will be.
- Scheduling: Once the defects have been prioritized, they can be scheduled for resolution. I recently was given an overview of a support group that used a Kanban tool with three classes of service (high, medium and low priority) to manage the workflow. Scheduling was performed through a combination of sequencing the backlog, class of service and work-in-process limits. Every organization has a scheduling process (albeit some are as simple as measuring the amount of screaming).
- Resolution: Fixing the software defects includes all of the steps of software development, from analysis, design, coding, data changes as needed, but will ALWAYS include testing to ensure the defect is fixed and nothing else is broken.
- Reporting: Reporting can include defect aging reports (how fast defects are being fixed), occurrence reports, and root cause analysis reports. Reports are only constrained by the imagination of those involved in reporting and analyzing the defects. Trend reporting is often the most powerful reporting tool.
- Process Improvement: The first reason for collecting defect data is to ensure the defect gets fixed. The second most important reason for collecting defect data is to change how work is being done. Defect data is a tool to identify where something is going wrong and then to learn from the data.
The basic process is simple and nearly every team or organization leverages some variant of this process. Agile teams often capture defects as stories on their backlog and use the grooming and sprint planning process to prioritize and schedule defect corrects. The complexity of the defect management process and the level of effort needed support the process is due to the collection of defect data. Some organizations require defects to be collected and recorded during every review, test and then after the software is deployed. Additionally, the level of detail recorded on the defect varies immensely. The more data and times defects are recorded is directly related to the amount of effort needed for defect management.
The number and types defects generated and delivered by software projects is often used as a measure of quality. Delivered defects are a simple measure of software quality all software organizations should collect and report. Collecting and reporting defects at other levels can be useful for trending and process improvement; however, it comes at a cost in terms of effort, cost and occasionally poor behaviors. Defect management is a great first step, but rarely provides a full picture of software quality.
Next week we will explore other frameworks and approaches and the some of the downsides of quality management frameworks.