Test case points is only one approach to determining the size of work that needs to be tested. The other measures fall into three broad categories.  The categories are: (more…)

Size Matters. . .But Which Size

If size matters then you must measure.

If size matters then you must measure.

Size matters.  I will provide a moment of silence to let the jokes and requisite tittering die down.  In software development the size of the software being built or changed matters because size affects productivity, size affects risk and size affects how projects are managed.  I suggest that these reasons are just the tip of the iceberg of why size matters.  Because size is such a major determinant in how a project is run, I would suggest we can’t leave the knowledge of how big the software component of a project is to shear guess work.  We must measure software size.  Unfortunately there are numerous measures of size and each of these measures has its own strengths and weakness making selection a chore.  Finding the right measure or at least the right category is more than an academic discussion unless you want to adopt a measure that does not meet your needs.

Fortunately the landscape of software size measures can be simplified by consolidating all of the available size measures into three categories.  The first category (and oldest) is that of physical measures, second is functional measures and the third is relative measures. Each of these categories is valid in specific circumstances.  Each category contains measures that are valuable as a tool to answer specific questions.  Each category waxes and wanes in its explanatory power across portions of the development life cycle.  Lets explore each in a bit more detail in anticipation of creating a simple selection algorithm.

Relative size measures use the measurer’s perspective as a framework to assess size.  These measures are much akin to stepping off a distance and declaring it to be so many yards or feet.  The measure is relative to size of the measure’s stride (your stride and mine are probably different).  Relative measures are fine for an approximation but fail to deliver precision or comparability.   Relative measures provide their greatest explanative power when there is the least amount of information or when there is no standard measure available. One of the most common uses of relative measures has historically been in budget activities when only a small amount of information is known; analogies are the most common type of relative measure used in in the budgeting process.   In recent years story points (a relative measure) have been used by some organizations as a means to to develop an approximate size during the development of requirements.

Functional size measures evaluate software size by assessing the size based on a set of rules focusing on “user” recognizable functionality.  Most standard functional measures focus on sizing only what was requested (and later delivered).  The focus on sizing business functionality means that functional measures can be used as soon as requirements or stories are identified.  The ability to size requirements based on a common set of rules allows the transition from relative to functional measures; from perceived size to rules based size. The transition from relative measures to  functional measures is also marked by an increase in the the number of rules required to determine size (and therefore to some extent the amount of effort required to determine size) while at the same time the level level of  abstraction typically decreases (what is being measured is closer to what is being delivered).  An example of the decrease in level of abstraction is shown in the the level of detail of IFPUG Function Points.  IFPUG Function points are determined by measuring and counting five types of components (external inputs, external outputs, external inquires, internal logical files and external interface files).  Examples of functional measures include IFPUG Function Points, Cosmic Function Points and Use Case Points to name a few.

Physical measures of size count tangible “things” like lines of code, modules, objects or typical software components.  The rules for counting physical size measures tend to vary by language, technology or are tied directly to specific technologies.  Because of the variance in the rules aggregating data in non-homogeneous organizations tends to be problematic.  For example, what does comparing 10 lines of Java and 10 lines of ASP.net mean or worse yet one object, three pages of documentation and 1,000 lines of generated Java?  Physical measures reach their zenith in explanative and predictive power during the specific activities that create the physical item being counted; code during coding, test cases during testing or pages of documentation during the creative writing phase called user documentation.
Size matters for estimation and planning regardless of methodology or technique.  Whether a project uses a relative, functional or physical measure matters less than measuring work and using that measure to create information.  Which method you use depends on organizational and project culture.

Michael Milutis
Executive Director, ITMPI


As promised, here is the link to the recording of Tom’s webinar:


You can access this at no charge, and as often as you want, between now and October 31st.

We apologize for the technical problems that many of you had during the live version of
Tom’s webinar.

For a listing of all our recordings, please visit http://www.itmpi.org/library

For our live webinar calendar, please visit http://www.itmpi.org/webinar

To see a listing of all webinar recordings currently available at no charge, please
visit us at http://www.twitter.com/itmpi

Agile estimation using functional metrics is designed to cover the product and release rings of Cohn’s planning onion using a synthesis of parametric and Delphi estimation techniques with the emphasis shifting from parametric to Delphi as events dictate.    The technique leverages the ability to size requirements to develop parametric estimates and then dovetails collaborative techniques to refine those estimates based on memories and self-knowledge.

The process flow is as follows

Stage One

  1. Identification of functional requirements (or stories)
  2. Sizing using Quick and Early Function Points
  3. Simple Parametric Estimation

Stage Two – Sprint or Iteration Planning

  1. Break Requirements into More Granular Pieces (if needed) and Refine Size
  2. Team Level Re-estimation of Requirements Using Delphi Techniques
  3. Team Level Commitment

Size is a critical component for developing an estimate and for planning however size and estimates are not synonymous.  Size is merely a step along the path from point A to point B.  As we move along the path size will be revisited twice.

The sizing process begins by segregating the functional requirements from the non-functional requirements.  The functional requirements are then sized using Quick and Early Function Points (QEFP).  Function points for all their warts are the easiest way to consistently size software requirements.  This is accomplished by focusing on the basic building blocks of functionality found in all software projects. The QEFP method leverages the relationship between action verbs and transactions to identify the transaction functions found in function points and the subject of the requirement to identify the data functions found in function points.  The application of this technique is similar to sentence diagramming that you learned during grade school or high school.  This relationship between words and size has been observed and investigated over the past few years by a number of different people within the functional sizing community including myself (see “Turning Perfect Good Words Into Numbers” originally presented at the IFPUG Functional Sizing Summit at http://www.davidconsultingroup.com).  Function points or any functional metric has at its heart the goal of converting requirements or stories into a number in a consistent manner.  The number then must be interpreted based on the abilities of the team or organization.  At the level of the overall project, the technique described in this paper leverages parametric estimation.  A simple parametric estimation equation could be:

Y =-(7^-6*(X^2))-Z*X+26.587


X = Size in function points

Y = Productivity rate for the type of project

Z = Behavior or Process Index

The result is a productivity rate for the project.  Collection of historical data on a selection of projects will be required to build an equation.  I would further suggest that you will need to augment internal data with external data to increase the validity of the estimation equation.  Note the factor can be applied to a disaggregated requirement, epic or story.  This estimate is created for organizational planning purposes.

Stage two begins as sprints or iterations are kicked off.  The sprint teams (we will use SCRUM terminology) breakdown the stories into pieces that can be accomplished during the sprint, then re-size the pieces using the QEFP method.   The goal of using QEFP at this point is to take one source of variance out of the estimation discussion that will be had when the Delphi method is applied.  This focuses the group on expanding the team level self-knowledge needed to coalesce on an appropriate level of effort needed to complete the story.  For example the QEFP technique has been combined with planning poker into a process that was quickly learned by the sprint teams.  The results were a marked reduction in stories that were not completed during the sprint they were committed to in.  By removing size as a variable the team that initially piloted this method indicated they were better able to focus on discussing team capabilities and technical considerations when doing their initial sprint planning.

Real life examples will help drive how organizations synthesized what could be considered competing methodologies into something greater than the sum of the two parts.

Part 4

Case One:

  • Firm:   Small custom technology organization
  • Project Types:  Internal and external projects
  • Culture: Highly collaborative
  • Current Methodology:  Mixed waterfall and SCRUM

Other notes:  All external projects are bid with many using a fixed fee structure.  Internal projects were continually re-scoped to fit the internal development budget which changes as the economy waxes and wanes.  Prior to rewriting the estimation process and leveraging functional metrics approximately 30% of bids were successful and budgets tended to be a suggestion.  The lack of estimation success meant that there was a significant risk of losing money if the business was won and if the business was not won going out of business in the long run.

The firm adopted Quick and Early Function Points for sizing the backlogs for all projects, both internal and external.  Where backlogs were not being used to manage requirements they were developed.  A quick baseline was developed to determine a productivity factor.  The productivity factor was then used to translate individual stories into effort.  Each team spent a day reviewing how they worked together to generate a baseline of self-knowledge and trust.    Collaborative story level estimation was redone using planning poker.  It became apparent quickly that a lot of disaggregation was needed to actually estimate the backlogs.  After applying QEFP and the productivity factor to the in-flight projects the firm progressed to applying QEFP to all bids.

The results were that won bids increased 20% and negative misses were nearly eradicated.  A negative miss was defined as underbidding on a fixed bid contract.

Case Two:

  • Firm:   Large software development firm
  • Project Types:  Internal projects (software for resale)
  • Culture:  Hierarchy, classic command and control
  • Current Methodology:  Mixed waterfall  (but SCRUM recently introduced)

Other notes:  The methodology in the environment was predominantly classic waterfall with central PMO.  Just before we readdressed the estimation process a team had implemented SCRUM and some components of extreme programming (xP) this was done in sort of a guerilla fashion.   One very large project was consuming the majority of the organizations resources.  Significant requirements were still being discovered after construction had begun.   Estimates had been developed based on a bottom up process very early in the project and they were of questionable validity.  The top managers just returned from begging for more money from the board of directors.  The project was being capitalized.

The solution in this case was for the company to more firmly embrace the SCRUM framework for project management at the team level.   A product backlog was developed and Quick and Early Function Points were adopted to size the backlog.  The sizing process exposed a number of functional blind spots (function points can be leveraged as form of analysis).  Team members were trained in using QEFP which allowed them to size new stories or re-size changed stories at an individual sprint planning level.  The impact of these changes was to allow the PMO to size and preplan the backlog with development leaders and the primary product owner.  The full product owner team selected stories to formulate sprints during the sprint planning meetings.  Teams re-evaluated (sized and estimated) the selected stories and then committed to the stories they could do.

The initial result was improved product owner satisfaction, involvement allowed them to be part of the solution.  After the first few sprint there was an increased perception of estimate consistency both at the product backlog and sprint team level.  It was also noted that the teams that had been using the using SCRUM before adopting the new estimation methods had a reduced number of stories that had not completed at the end of the sprint.  The teams attributed this improvement during retrospectives to a better capacity to size and commit to work that they’re actually able to accomplish during the sprint.


Agile methods have matured and are now being integrated into many different approaches to the development of software.  Estimation has been problematic for all methods; agile to plan based therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper.   Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile estimation.

The fourth step in our checklist for selecting a size metric is an evaluation of the temporal component. This step focuses your evaluation on answering the question, “Is the metric available when you need it?” When do you need to know how big a project is depends on what you intend to do with the data (that goal thing again). The majority of goals can be viewed as either estimation related (forward view) or measurement related (historical view). Different sizing metrics can be initially applied at different times during a projects life. For example Use Case Points can’t developed until Use Cases are developed, Lines of Code can’t be counted until you are deep into construction or at the very earliest in technical design.


Figure 4

The major dichotomy is driven between estimation needs and measurement needs. As Figure 4 suggests determining size from requirements (or earlier) will require focusing on functional metrics. Functional metrics can be applied earlier in the process (regardless of methodology) because they are based on a higher level of abstraction that is more closely aligned with the business description of the project. Developing estimates or sizing later in the in the development process opens the possibility of more physical metrics which are more closely aligned with how developers view their work.

Knowing the measurement goal that the size metrics you are selecting actually supports is a big deal (extra credit points). The goal has to be a key contributor to the decision process.

While knowing the goal your size metric will support is important it will help you make a cut based on four macro attributes between organization specific and industry defined metrics and between physical and logical metrics. For example, if benchmarking against other firms or industry data is required to attain your measurement goal using organizationally defined metrics would be less viable. Similarly if you have a heterogeneous software environment then selecting a functional metric would make more sense than using a physical metric (logical metrics normalizes varied technology).


Figure 1: Focus

The second checkbox is whether the measure has an externally defined and documented methodology. Why is definition important? Definition is the precursor to repeatability and consistency which allows comparability. Consistency and repeatability are prerequisites for the ability to generate data needed to use the scientific method such as Six Sigma and tools used to support Kiazen. Finally, an external definition reduces the amount of effort that is required to construct and implement measurement programs.

Even where a definition exists a wide of nuances are possible. Examples of the range of definitions begin with the most defined, the functional precision of ISO functional metrics to the less defined methodology of Use Case Points which began with a single academic definition and has evolved into many functional variants. The variants seen in UCP are a reflection of having no central control point to control methods evolution which we will explore later in this model. The range of formality of definition is captured in Figure Two.


Figure Three consolidates the view of formality of definition with the delineation between logical and physical metrics. Each measure has strengths and weaknesses. The first two items in our checklist are macro filters.


Once the macro filter is applied each subsequent step in the checklist will narrow the field.

All jokes aside size matters. Size matters because at least intellectually we all recognize that there is a relationship between the size of product and the effort required to build. We might argue over degree of the relationship or whether there are other attributes required to define the relationship, but the point is that size and effort are related. Size is important for estimating project effort, cost and duration. Size also provides us with a platform for topics as varied as scope management (defining scope creep and churn) to benchmarking. In a nutshell, size matters both as an input into the planning and controlling development processes and as a denomination to enable comparison between projects.

Finding the specific measure of software size for your organization is part art and part science. The selection of your size measure must deliver the data need to meet the measurement goal and to fit within the corporate culture (culture includes both people and the methodologies the organization uses). A framework for evaluation would include the following categories:

· Supports measurement goal

· Industry recognized

· Published methodology

· Useable when needed

· Accurate

· Easy enough

The next installment of this essay will discuss evaluating whether the size metrics supports the measurement goals and the topic of industry recognition.