At its heart, the defect management approach to defining software quality is focused on identifying, categorizing and counting defects. Quality attribute models expand the definition of quality from the occurrence of defects to a framework of attributes. For example, ISO/IEC 25010:2011 describes an attribute model comprised of eight quality characteristics. The characteristics are:
- Functional suitability
- Performance efficiency
In the ISO model each of the characteristics can be further broken down into sub-characteristics to provide additional depth to the definition of quality. All quality attribute models are based on a broader framework than the defect management approach. Using a broader framework provides teams and organizations with more information than simpler models due to the sheer number of attributes in the model. However, more information comes at a cost. Quality attribute modes are less easy to implement, require more data collection, and often require more effort from the development team.
Implementation first requires defining metrics that are a reflection of the quality attributes within an organization. Once the organization has defined metrics, implementation requires training, support, and buy-in. The number of measures and metrics and the difficulty of explaining and understanding them are negatively correlated with ease of implementation.
Fixed quality models like ISO/IEC 25010:2011 require a significant number of metrics to cover each of the categories. The amount of data increases the amount collection and analysis effort. The effort and cost of any measurement program should be evaluated against the benefits generated through the collection and analysis of the data.
Collecting data often requires effort from project teams either to collect, interpret or defend data. Developers often misinterpret time spent on software measurement as time that does not deliver value. However, measurement is a core step in every process improvement model.
All quality models are a balance between cost and benefit. The ISO standard addresses this balance by providing a “quality in use” model that includes five components:
- Effectiveness is a measure of how successful a project is in producing the desired result.
- Efficiency is a measure of the impact of the software on the amount of work performed per unit of effort.
- Satisfaction is a measure of whether the software meets a need or desire.
- Safety is measure of whether the operation of the software causes or reduces danger.
- Usability is a measure of the ease of use, functionality and learnability.
Implementing the “quality in use” model can be accomplished with a much more succinct set of metrics. Measures to provide feedback on the five characteristics often include common measures and metrics such as defects and customer satisfaction. A model that leverages common metrics is easier to understand and requires less effort to implement.
Attribute models collect substantially more data than defect management models and are therefore more powerful. However, they are harder to implement and run. One compromise organizations should consider is the “quality in use” model which has fewer moving parts and tends to be more easily understood. Many organizations harvest the best parts from attribute, use and defect models to build their own starting points for defining how to measure software quality in IT organizations.
Thursday: A pallet of metrics that can be used for the quality of use model.