Which size metric makes sense for a testing organization is influenced by how testing is organized, where testing is incorporated into the value delivery chain, and whether the work is being done for a fee.
How the people are organized to identify needs, develop software and deliver value will influence which sizing technique will be appropriate for testing. There are two basic competing models for organizing testers. The two models are independent test groups and testers embedded into the team. Variants of the later include testers as part of the team and testers as matrixed members of the team. In an independent test model, developers complete their work (usually after unit testing) and then “throw” the work over the wall to the independent testers, whereas in the embedded version the work does not have to pass over a boundary.
Independent testing teams generally focus most of their efforts on planning and executing tests (these types of tests are often termed dynamic testing and include system testing through user acceptance testing). In this organizational scenario, testing teams size their work independently of the development team. Size is used to predict when work will be completed or for how much work will be accepted by the team. Metrics that specifically leverage or count testing deliverables such as test cases or test case points are typically used.
Testers embedded in the team generally align to the same size metric as the development personnel (the decision of which size metric is often decided by the development personnel). Testers plan their work as part of the overall team or at worst, concurrently with development tasks. In this scenario, all three categories of size metrics are used. Examples of physical, functional and relative size metrics used include a number of requirements, IFPUG function points or story points.
How testing activities, both dynamic and static (static testing includes techniques such as reviews and pair programming) are incorporated in the value delivery chain influence which size metric will be used to plan testing activities.
The more integrated into the development process testers are the more likely the team will plan together using either a functional metric (function points) or a relative measure (tee shirt sizing or story points). Team’s that are using any of the test first techniques (e.g. TDD, BDD and ATDD) are examples of scenarios in which testing is highly integrated into the value delivery chain. The higher the level of integration of testing into the development process the more apt testers will be to leverage sizing metrics that reflect the ultimate functional deliverable versus counting individuals physical items.
Similar to the scenarios in which testing is an independent group, when test activities are segregated to a separate phase, test teams often leverage physical size metrics (number of requirements or components) or hybrid sizing methods such as test case points.
Work for fee
Outsourced testing is an extreme version of the independent test group. Outsourced test groups that price by project, face all of the same issues as teams doing any outsourced piece of work. With the exception of open, time and material contracts, the testing team needs some basis for the estimate that allows them to complete the work and make a healthy profit margin. Just like many development groups that have begun to leverage cost per function point, testing providers are experimenting with cost per test case point.
The size metric a test group leverages is rarely a random choice. Many of the influences are outside of the individual testers span of control. Organization culture influences how testers are involved in the development process and whether they are segregated into separate teams. The best option for a size metric is the one that helps a team know how much work they can commit to completing and when that work can be completed well.