Software Measurement


A count of the Pokemon in my local park yesterday!

In today’s complex software development environment, it is easy to see every data collection issue as a complex automation problem. For example, I recently had a discussion with a team that is trying to determine how they would know whether a coding standard change they were contemplating would be effective. The team proposed capturing defects the coder/developer pairs found in TFS. The plan was to enter each defect found into the tool. After a discussion, the team decided to adopt a much simpler data collection solution with very low overhead. One of the classic quality tools is a tally sheet or check sheet. A check sheet is a form that used to collect information as it happens as a set of checkmarks or tally marks on the form. Check sheets are a very simple data collection tool. They are often a great way to pilot data collection before spending time and effort on automation or adding overhead to work. A check sheet is a form of the histogram where data where data collection and categorization occurs at the same time as visualization. (more…)

Listen Now
Subscribe on iTunes
Check out the podcast on Google Play Music

SPaMCAST 449 features our interview with Jasveer Singh.  We discussed his new book, Functional Software Size Measurement Methodology with Effort Estimation and Performance Indication.  Jasveer proposes a new sizing methodology for estimation and other measurement processes.

Jasveer Singh holds a Master of Technology degree in Computer Technology from the Indian Institute of Technology, Delhi, and has studied Executive Master in Management at École de Commerce Solvay, Brussels, Belgium.

He has about 30 years of valuable senior-level international experience in the ICT area and has worked in the top IT/Telecom equipment manufacturer, operator, consultancy, and service companies in different countries (Bharat Electronics Limited, Alcatel, Siemens Business Services, WorldCom, Logica, and Sigos in India, France, Australia, Belgium, and Germany). A significant part of this experience has been in the management of software development (analysis, design, coding, testing), system design, quality assurance/control, and project management while working with different programming languages, object-oriented technology, database management systems, etc. His in-depth experience in these software domains led him to realize the improvements needed in the currently available methodologies for software size measurement and to develop the Functional Software Size Measurement Methodology with Effort Estimation and Performance Indication (FSSM) which is a thorough methodology and great help for software projects.

Currently, he is based in Belgium and is the director of EUSFP.

E-mail: js@fssm.software

LinkedIn: https://www.linkedin.com/in/jasveer-singh-11230a12/

FSSM book: http://www.wiley.com/WileyCDA/WileyTitle/productCd-1119238056.html

FSSM online book: http://onlinelibrary.wiley.com/book/10.1002/9781119238126

FSSM website: www.fssm.software

Re-Read Saturday News

This week we wrap up our re-read of  Holacracy: The New Management System for a Rapidly Changing World by Brian J. Robertson which was published by Henry Holt and Company in 2015. The concepts in Holacracy are an important addition to the discussion of management, governance, and leadership in the 21st Century.  Read or re-read this week’s installment for more thoughts and comments!   

Catch up on the all of the Holacracy entries: (more…)

Listen Now
Subscribe on iTunes
Check out the podcast on Google Play Music

SPaMCAST 445 features the return of a favorite, Capers Jones.  It is always fun to talk with someone with their own page in Wikepedia.  Capers and I talked about his new book, A Guide to Selecting Software Measures and Metrics. Capers is passionate about software quality and measurement. Capers said, “High-quality software is not expensive. High-quality software is faster and cheaper to build and maintain than low-quality software, from initial development all the way through total cost of ownership.” from The Economics of Software Quality.  As usual, Capers was engaging, educational and controversial.  Spending time with Capers is always a learning experience!

Capers biography is long and storied.  Let it be said that Capers is a serial author, public speaker, pundit, guru and deep thinker.  Check out his Wikipedia page or Linkedin.

Capers can be contacted at capers.jones3@gmail.com.

Capers first appeared on SPaMCAST 3 and  last appeared on SPaMCAST 53

Re-Read Saturday News

This week we tackle Chapter 7 of Holacracy: The New Management System for a Rapidly Changing World by Brian J. Robertson published by Henry Holt and Company in 2015.  Chapter 7 shows how to generate alignment between roles, circles, and the overall organization.  Lots of inspect and adapt talk this week.

Catch up on the first four entries in the re-read

Week 1:  Logistics and Introduction

Week 2: Evolving Organization

Week 3: Distribution Authority

Week 4: Organizational Structure

Week 5: Governance

Week 6: Operations

Week 7: Facilitating Governance

Week 8: Strategy and Dynamic Control

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

 

A Call To Action

If you got a new idea this week while listening to the podcast, please give the SPaMCAST a short, honest review in iTunes, Stitcher or wherever you are listening.  If you leave a review please send a copy to spamcastinfo@gmail.com.  Reviews help guide people to the cast!

Next SPaMCAST

SPaMCAST 446 will feature our essay on questions.  Questions are a coach and facilitator’s secret power!   Do you have a favorite go-to question you like to ask?  Care to share?

We will also have columns from Gene Hughson and Jon M Quigley (and maybe more)!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

 

Not quite a Google bus

Not quite a Google bus

Labor, raw material, and capital productivity are easy concepts to understand.  For example, labor productivity is the ratio of the products delivered per unit of effort.  Increasing the efficiency of labor will either increase the amount of product delivered or reduce the amount of labor needed.  Raw material or capital productivity follow the same pattern. The issue is that while labor, raw materials, and capital explain a lot of the variation in productivity, they do not explain it all. And in software product development other factors often contribute significantly to productivity improvement. Total factor productivity (TFP) is not a simple ratio of output to input, but rather is a measure that captures everything that is not captured as labor, capital or material productivity. Factors included in total factor productivity include attributes such as changes in general knowledge, the use of particular organizational structures, management techniques, or returns on scale. The components in TFP are often the sources of productivity changes in software development.  (more…)

26096385160_0723a35c74_o

In simplest terms, productivity is the ratio of output per unit of input.

Almost every conversation about change includes a promise of greater productivity.  In simplest terms, productivity is the ratio of output per unit of input.  While the equation is for calculating productivity is straightforward, as we have discussed, deciding on which outputs from an organization or process to count is never straightforward. The decisions on the input side of the equation are often equally contentious.  Three critical decisions shape what measures will be needed to supply the inputs used to calculate productivity.   (more…)

Listen to the Software Process and Measurement Cast (Here)

The Software Process and Measurement Cast features our interview with Charley Tichenor and Talmon Ben-Cnaan on the Software Non-Functional Assessment Process (SNAP).  SNAP is a standard process for measuring non-functional size.  Both Talmon and Charley are playing an instrumental role in developing and evolving the SNAP process and metric.  SNAP helps developers and leaders to shine a light on non-functional work required for software development and is useful for analyzing, planning and estimating work.

Talmon’s Bio:

Talmon Ben-Cnaan is the chairperson of the International Function Point User Group (IFPUG) committee for Non-Functional Software Sizing (NFSSC) and a Quality Manager at Amdocs. He led the Quality Measurements in his company, was responsible for collecting and analyzing measurements of software development projects and provided reports to senior management, based on those measurements. Talmon was also responsible for implementing Function Points in his organization.

Currently he manages quality operations and test methodology in Amdocs Testing division. The Amdocs Testing division includes more than 2,200 experts, located at more than 30 sites worldwide, and specializing in testing for the Telecommunication Service Providers.

Amdocs is the market leader in the Telecommunications market, with over 22,000 employees, delivering the most advanced business support systems (BSS), operational support systems (OSS), and service delivery to Communications Service Providers in more than 50 countries around the world.

Charley’s Bio:

Charley Tichenor has been a member of the International Function Point Users Group since 1991, and twice certified as a Certified Function Point Specialist.  He is currently a member of the IFPUG Non-functional Sizing Standards Committee, providing data collection and analysis support.  He recently retired from the US government with 32 years’ experience as an Operations Research Analyst, and is currently an Adjunct Professor with Marymount University in Washington, DC, teaching business analytics courses.  He has a BSBA degree from The Ohio State University, an MBA from Virginia Tech, and a Ph.D. in Business from Berne University.

 

Note:  Charley begins the interview with a work required disclaimer but then we SNAP to it … so to speak.

Next

In the next Software Process and Measurement Cast we will feature our essay on product owners.  The role of the product owner is one of the hardest to implement when embracing Agile. However how the role of the product owner is implemented is often a clear determinant of success with Agile.  The ideas in our essay can help you get it right.

We will also have new columns from the Software Sensei, Kim Pries and Jo Ann Sweeney with her Explaining Communication series.

Call to action!

We are in the middle of a re-read of John Kotter’s classic Leading Change on the Software Process and Measurement Blog.  Are you participating in the re-read? Please feel free to jump in and add your thoughts and comments!

After we finish the current re-read will need to decide which book will be next.  We are building a list of the books that have had the most influence on readers of the blog and listeners to the podcast.  Can you answer the question?

What are the two books that have most influenced you career (business, technical or philosophical)?  Send the titles to spamcastinfo@gmail.com.

First, we will compile a list and publish it on the blog.  Second, we will use the list to drive future  “Re-read” Saturdays. Re-read Saturday is an exciting new feature that began on the Software Process and Measurement blog on November 8th.  Feel free to choose you platform; send an email, leave a message on the blog, Facebook or just tweet the list (use hashtag #SPaMCAST)!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, neither for you or your team.” Support SPaMCAST by buying the book here.

Available in English and Chinese.

Productivity is number of clothes sewed per hour.

Productivity is number of clothes sewed per hour.

Definition:

Labor productivity measures the efficiency of the transformation of labor into something of higher value. It is the amount of output per unit of input: an example in manufacturing terms can be expressed as  X widgets for every hour worked. Labor productivity (typically just called productivity) is a fairly simple manufacturing concept that is useful in IT. It is a powerful metric, made even more powerful by it’s simplicity. At its heart, productivity is a measure of the efficiency of a process (or group of processes). That knowledge can be tool to target and facilitate change. The problem with using productivity in a software environment is the lack of a universally agreed upon output unit of measure.

The lack of a universally agreed upon, tangible unit of output (for example cars in an automobile factory or steel from a steel mill) means that software processes often struggle to define and measure productivity because they’re forced to use esoteric size measures. IT has gone down three paths to solve this problem. The three basic camps to size software include relative measures (e.g. story points), physical measures (e.g. lines of code) and functional measures (e.g. function points). In all cases these measures of software size seek to measure the output of the processes and are defined independently of the input (effort or labor cost).

Formula:
The standard formula for labor productivity is:

Productivity = output / input

If you were using lines of code for productivity, the equation would be as follows:

Productivity = Lines of Code / Hours to Code the Lines of Code

Uses:
There are numerous factors that can influence productivity like skills, architecture, tools, time compression, programming language and level of quality. Organizations want to determine the impact of these factors on the development environment.

The measurement of productivity has two macro purposes. The first purpose is to determine efficiency. When productivity is known a baseline can be produced (line in the sand) then compared to external benchmarks. Comparisons between projects can indicate whether one process is better than another. The ability to make a comparison allows you to use efficiency as a tool in a decision-making process. The number and types of decisions that can be made using this tool are bounded only by your imagination and the granularity of the measurement.

The second macro rational for measuring productivity is as a basis for estimation. In its simplest form a parametric estimate can be calculated by multiplying size by a productivity rate.

Issues:
1. The lack of a consistent size measure is the biggest barrier for measuring productivity.
2. Poor time accounting runs a close second. Time account issues range from misallocation of time to equating billing time to effort time.
3. Productivity is not a single number and is most accurately described as a curve which makes it appear complicated.

Variants or Related Measures:
1. Cost per unit of work
2. Delivery rate
3. Velocity (agile)
4. Efficiency

Criticisms:
There are several criticisms of the using productivity in the software development and maintenance environment. The most prevalent is an argument that all software projects are different and therefore are better measured by metrics focusing on terminal value rather than by metrics focused on process efficiency (artesian versus manufacturing discussion). I would suggest that while the result of a software project tends to be different most of the steps taken are the same which makes the measure valid but that productivity should never be confused with value.

A second criticism of the use of productivity is a result of improper deployment. Numerous organizations and consultants promote the use of a single number for productivity. The use of a single number to describe the productivity the typical IT organization does match reality at the shop floor level when the metrics is used to make comparisons or for estimation. For example, would you expect a web project to have the same productivity rate of a macro assembler project? Would you expect a small project and a very large project to have the same productivity? In either case the projects would take different steps along their life cycles therefore we would expect their productivity to be different. I suggest that an organization analyze their data to look for clusters of performance. Typical clusters can include: client server projects, technology specific projects, package implementations and many others. Each will have a statistically different signature. An example of a productivity signature expressed as an equation is shown below:

Labor Productivity=46.719177-(0.0935884*Size)+(0.0001578*((Size-269.857)^2))

(Note this is an example of a very specialized productivity equation for a set of client server projects tailored for a design, code and unit testing. The results would not representative a typical organization.)

A third criticism is that labor productivity is an overly simple metric that does not reflect quality, value or speed. I would suggest that two out three of these criticisms are correct. Labor productivity is does not measure speed (although speed and effort are related) and does not address value (although value and effort may be related). Quality may be a red herring if rework due to defects is incorporated into productivity equation. In any case productivity should not be evaluated in a vacuum. Measurement programs should incorporate a palette of metrics to develop a holistic picture of a project or organization.

3068483640_328b020efa_bGluttony is over-indulgence to the point of waste.  Gluttony brings to mind pictures of someone consuming food at a rate well beyond simple need.  In measurement, gluttony then can be exemplified by programs that collect data that has no near-term need or purpose.  When asked why the data was collected, the most common answer boils down to ‘we might need it someday…’

Why is the collection of data just in case, for future use or just because it can be done a problem?  The problems caused by measurement gluttony fall into two basic categories.  The first is that it wastes the effort of the measurement team, and second because it wastes credibility.

Wasting effort dilutes the measurement team’s resources that should be focused on collecting and analyzing data that can make a difference.  Unless the measurement program has unlimited resources, over collection can obscure important trends and events by reducing time for analysis and interpretation.  Any program that scrimps on analysis and interpretation is asking trouble, much as a person with clogged arteries.  Measures without analysis and interpretation are dangerous because people see what they like in the data due to clustering illusion (cognitive bias). Clustering illusion (or clustering bias) is the tendency to see patterns in clusters or streaks of in a smaller sample of data inside larger data sets. Once a pattern is seen it becomes difficult to stop people from believing that the does not exist.

The second problem of measurement gluttony occurs because it wastes the credibility of the measurement team.  Collecting data that is warehoused just in case it might be important causes those who provide the measures and metrics to wonder what is being done the data. Collecting data that you are not using will create an atmosphere of mystery and fear.  Add other typical organizational problems, such as not being transparent and open about communication of measurement results, and fear will turn into resistance.   A sure sign of problems is when you  begin hearing consistent questions about what you are doing, such as “just what is it that you do with this data?” All measures should have a feedback loop to those being measured so they understand what you are doing, how the data is being used and what the analysis means.  Telling people that you are not doing anything with the data doesn’t count as feedback. Simply put, don’t collect the data if you are not going to use it and make sure you are using the data you are collecting to make improvements!

A simple rule is to collect only the measurement data that you need and CAN use.  Make sure all stakeholders understand what you are going to do with the data.  If you feel that you are over-collecting, go on a quick data diet.  One strategy for cutting back is to begin in the areas you feel safest [SAFEST HOW?]. For example, start with a measure that you have not based a positive action on in the last 6 months. Gluttony in measurement gums up the works just like it does in a human body; the result of measurement gluttony slows down reactions and creates resistance, which can lead to a fatal event for your program.

 

14545519494_12ab1ba776_kSloth plagues many measurement programs as they age.  As time goes by, it is easy for practitioners to drift away from the passionate pursuit of transforming data into knowledge. Sloth in measurement programs is typically not caused by laziness. Leaders of measurement groups begin as true believers, full of energy. However over time, many programs fall prey to wandering relevance. When relevance is allowed to waiver it is very difficult to maintain the same level of energy as when the program was new and shiny. Relevance can slip away if measurement goals are not periodically challenged and validated. An overall reduction in energy can occur even when goals are synchronized, if there is a conflict on how the data will be used and analyzed between any of the stakeholder classes (measurement team, management or the measured). Your energy will wane if your work results in public floggings or fire drills (at the very least it will make you unpopular).

The drift into sloth may be a reflection of a metrics palette that is not relevant to the organization’s business, therefore not likely to produce revelations that create excitement and interest.  This can cause a cascade of further issues.  Few metrics programs begin life by selecting irrelevant metrics, except by mistake, however over time relevance can wander as goals and organizational needs change.  Without consistent review, relevance will wane and it will be easy for metrics personnel to lose interest and become indifferent and disengaged.

In order to avoid or reclaim your program from sloth due to drifting goals; synchronize measurement goals with the organization goals periodically.  I suggest mapping each measurement goal and measure to the organizations goals.  If a direct link can’t be traced, I suggest that you replace the measure.  Note: measurement goals should be reviewed and validated any time a significant management change occurs.

When usage is the culprit, your job is to counsel all stakeholders on proper usage. However, if management wants to use measurement as a stick, it is their prerogative. Your prerogative is to change fields or to act out and accept the consequences. If the usage is a driver for lack of energy, you probably failed much earlier in the measurement program and turning the ship will be very difficult. Remember that it pays to spend time counseling the organization about how to use measurement data from day one rather than getting trapped in a reactionary mode.

The same symptoms occur when management is either disinterested (not engaged and not disposed positively or negatively toward the topic) or has become uninterested (disengaged). The distinction between disinterested and uninterested is important because the solutions are different. Disinterest requires marketing to find a reason to care; to be connected.  A stakeholder that has become uninterested needs to be reconnected with by providing information so their decisions matter.  Whatever the reason for actively disengaging or losing interest, loosing passion for metrics will sap the vitality of your program and begin a death spiral.  Keep your metrics relevant and that relevance will provide protection against waning interest. Metrics professionals should ensure there is an explicit linkage between your metrics palate and the business goals of your organization.  Periodically audit your metrics program.  As part of the audit map the linkages between each metric and the organizations business goals.  Make sure you are passionate about what you do.  Sharing your passion of developing knowledge and illustrating truth will help generate a community of need and support.

Synchronizing goals, making metrics relevant and instilling passion may not immunize your metrics program from failure but they will certainly stave off the deadly sin of sloth. If you can’t generate passion or generate information and knowledge from the metrics program to generate relevance consider a new position, because in the long run not making the change isn’t really an option..

Can't see forest for the trees

Can’t see forest for the trees

The first deadly sin is pride. In the cannon of deadly sins, pride is the sin from which all other spring. In the world of metrics programs, the sin of pride is when a metrics program settles on a single metric that is used to reflect the value or well-being of a project, a group or organization. Examples of abound of metrics programs that fixated on cost or productivity to the exclusion of a broader palette of metrics and work attributes.  Most metrics professionals quickly learn that one metric cannot be used for all projects.  If you can’t easily answer the question, “Does this relate?” each time you use a metric and for each metric you use, the information generated through measurement and analysis will provide little or no value.  The goal is to understand the differences between groups of work so that when comparisons are made, you can discern what is driving the difference (or even if there is a difference).  Comparing package implementations, hardware intensive projects or custom development is rational only if you understand that there will be differences and what those differences mean.   The bottom line is that rarely does a single metric deliver that deep level of understanding that generates value from measurement.

Another example of the single metric syndrome generated by the sin of pride occurs when an organization uses a single metric to value performance in a contractual arrangement.  While entire contracts are rarely stipulated on a single metric, it easy for a single metric to be given disproportional weight due to the framers’ lack of understanding or a disconnect between the framers and the people that administer the contract.  Poor understanding of the relationship between the numbers and the concepts they represent is akin to failure in punctuation when writing.  The resulting meaning can be garbled as the contract is negotiated, implemented and managed.  We won’t get into an existential argument over whether something is a sin if it is inadvertent; the result is the same. Garbled concepts can lead to a single metric focus which once discovered will beg to be taken advantage of.  This usually causes an overemphasis on a specific portion of the value chain, such as productivity being emphasized over time-to-market, quality or cost.

Next Page »