How do you measure out of the ordinary packages?

Independent testing groups are often asked how long and how much effort is required to test a piece of work.  Several size estimation techniques are actively in use in many organizations.  Each of these techniques begins by deriving size either based on a set of rules or through relative sizing.  Size, once derived, is used to estimate effort.  Effort is then used to generate cost, staffing and duration estimates.  The first sizing technique is“Test Case Points”

Test Case Points are a unit of measurement generated from the the testable requirements based on a set of rules.  The process is straightforward:

  1. Identify the testable requirements in a piece of work.  Use Cases or technical requirements documents are used for identifying testable requirements.  
  1. Identify the complexity of each testable requirement.  Test case points evaluate four factors to determine complexity:
    1. The number of test steps. The number of execution steps needed to arrive at an expected (or unexpected) outcome after all preconditions have been satisfied.
    2. The number of interfaces to the other requirements. A simple count of the number of interfaces in the test case.
    3. The number of verification points. A simple count of the points in the test case that the results are evaluated for correctness.
    4. Need for baseline test data. An evaluation of whether data needs to be created to execute the test case.  


Once all of the simple, medium and complex test cases are identified, they are summed by category.

  1. Weight each category.  

  1. Sum the weighted categories together to yield the total test case points

The goal of test case points is to use size to generate an estimate.  Every version of test case points I have worked with uses a set of factors to adjust the size as part of the sizing process.

  1. Develop an estimation adjustment weighting based on a set of factors (for those familiar with IFPUG Function Points this adjustment is a similar process to the one for determining the value adjustment factor). The factors are:
  1. Count or Single Factor Adjustment Factors
    Factor 14 – Operating System Combinations (simple count)
    Factor 15 – Browser Combinations (simple count)
    Factor 16 – Productivity Improvement from Second Iteration Onwards (percentage)
  2. Factors that leverage a combination of fixed factor and complexity weighting
    Factor 1 – Domain Knowledge & Complexity
    Factor 2 – Technical Know How
    Factor 3 – Integration with other Hardware Devices such as Handheld Devices, Scanners, Printers
    Factor 4 – Multi-lingual Support
    Factor 5 – Software/Hardware Setup
    Factor 6 – Environment Setup
    Factor 7 – Build Management
    Factor 8 – Configuration Management
    Factor 9 – Preparation of Test Bed
    Factor 10 – Stable Requirements
    Factor 11 – Offshore/Onsite Coordination
    Factor 12 – Test Data Preparation
    Factor 13 – Network Latency

 

  1. Generate an estimate using the following formula

Weighted Test Case Points X Adjustment Factor X Historical Productivity Rate

In many cases, organizations generate estimates for types of work separately using the adjustment factors that that would affect the type of work.  An example of a type of work is test case generation.  Factor 5, software/hardware setup, would not be predictive of the effort for setting up test cases.

The process for deriving test case points is fairly straightforward (steps 1 – 4).  The process of turning the test case points into an estimate is more complicated. Next, we will develop a short example and examine the strengths and weaknesses of the process – some which are very apparent and other are not.