There are lots of options, but you only need a few!

There are lots of options, but you only need a few!

Deciding what to measure in an IT organization is like letting a child loose in a candy store. There are so many things to measure and so little time! The natural tendency is measure everything. I once was shown a list of 248 (I counted them) measures for a 50-person development and maintenance group (I did not count them).  It was crazy and required a staff of three people to collect and crunch all that data. In the frenzy to measure and collect everything the logic is often that it does not matter whether the information will used now, we might need it later. This is usually a sign that the organization needs to step back and reevaluate. I suggest a simple measurement pallet that supports a wide range of organizational goals (We will discuss syncing measurement to organizational goals in a future article.) A basic IT measurement pallet can start with these six measures (this is a Swiss Army knife approach):

  • Size: Begin by measuring the size of functionality that is being delivered. I recommend IFPUG Function Points for functional size; however other functional measure such as COSMIC, NESMA or MARKII function points will also work. While none of these measures are perfect, they are currently the best means to determine how “much” functionality is being delivered.  More advanced organizations might add measures of non-functional requirements to further broaden your understanding of what is being delivered.
  • Cost: Your IT organization should understand how much each project costs, which means the organization is going to have to invest, or at least understand, good cost accounting techniques. Decisions about whether or how to allocate corporate and IT overhead to projects or whether to include hardware cost even if it has to be apportioned are important decisions in order to understand total cost of a project.
  • Effort: Measure the actual amount of effort it takes to deliver the project. Actual means just that how much time is needed, not what the amount you will bill or don’t dump hours in a project that is ahead of schedule. Effort, time accounting, is a close relative of cost accounting and needs to be approached with equal discipline. One final note, collect effort in hours. Anything lower is too much overhead and anything higher generates a lot of measurement error.
  • Duration: Collect the start and end dates of the project.  I also suggest collecting interim dates, such as release dates. One common issue with determining duration is having a definition of what marks the beginning and end of a project. For example, is the research to investigate concept feasibility part of the project or not?  If your project has a warranty period (the development team is supporting the implementation and fixing production defects), is that part of the project or part of maintenance?  Each organization needs to have a common set of policies to ensure everyone understands what start and end means.
  • Defects: I recommend counting all post implementation defects.  If needed, organizations can get fancy and account for defects found during the development process. Post implementation defects are a reflection of the quality the customer actually feels. One simplification I often recommend is that instead of spending huge amounts of time to figure out which project caused a defect, credit the defect to the project that last touched the application (they should have found the defect while regression testing).
  • Satisfaction:  Measure customer satisfaction using simple survey or interview approaches (five-ish questions). Focus on trying to identify whether the overall satisfaction to any customer group is changing at a high level.  When you see a move in the data, you can spend time on a more in-depth investigation. More advanced approaches should leverage techniques such as net promoter.

A simple measurement pallet can be used to generate a wide range of metrics, for example effort and size can be used to generate productivity.  Size and duration can be used to generate velocity or time-to-market metrics.  More importantly, by tackling this simple pallet organizations can develop the discipline needed to generate value from measurement.

Advertisements