Test case points is only one approach to determining the size of work that needs to be tested. The other measures fall into three broad categories. The categories are:
Physical Measures. This category represents a count tangible “things” like requirements or test cases. The assumption is that there is a relationship between the count of the physical item and the effort or the duration of testing. For example, there might be a relationship between the number of test cases need for a project and the amount of effort needed to execute and review those test cases. Measurement approaches that count physical items are generally an easy approach to generating a measure or metric. The most immediate problem with counting physical things is that individual items are generally not the same size. Therefore simple counting does not reflect the range of sizes.
Functional Measures. This category uses a set of rules to determine software size by assessing the software based on a set of rules focusing delivered functionality. IFPUG Function Points (and other function point methods) are examples of size measures in this category. The assumption made when using this category of metrics is that the functional size of the software components is related to the duration and effort required for testing. Function point counts are a good reflection of the functionality; however, projects are often a mix of functional and nonfunctional requirements. In scenarios where the ratio of functional to non-functional is out of the ordinary, functional measure may not be useful.
Relative Measure. Measures in this category use the measurer’s perspective as a framework to assess size. Story points and tee shirt sizing are examples of relative size measures. This form of measurement makes that same basic assumption that every other size category makes; that size is related to effort. The way the metric is used is typically different. Instead of using size to estimate how much effort a piece of work will require, relative measures are typically used to gauge how much work can be done in a fixed time by a fixed group. The development of relative measures engages the whole team, through techniques such as planning poker. The process which helps the team determine size while learning about the user story being measured. Issues with this type of measure center around the need for the team to be stable or the need for reporting size for use in other measures.
Test case points are a hybrid approach. The method begins with test cases (a tangible thing) and then counts the steps, verification points, interfaces and factors in whether baseline data is needed. These counts within a count are used to adjust the size of the test case in order to more closely relate size to effort. This approach uses concepts from the physical and functional approaches.
Each of these sizing approaches has pluses and minuses. When and where the value of these metrics outweighs the effort to generate these metrics is a subject of much debate as is which size measure or metric you choose. Where you fall in this debate is heavily influenced by how you are organized, where you fall in the value delivery food chain and whether the work is being done for a fee.
We will address this topic next!