A measurement program is like building a wall. Make sure you have all your resources in place.

A measurement program is like building a wall. Make sure you have all your resources in place.

Part of the Simple Checklist Series (Resources Part 1)

Beginning or continuing a measurement program is never easy. Many times measurement programs begin because an organization or individual thinks it necessary for survival or to avoid pain. Measurement can be thought of as a balance between the effort to collect and report measurement data and the value gained from applying what is learned from the measurement data.  Measurement programs targeted only the at gathering and reporting part of the measurement program will languish in the long run. On the other side of the equation, i.e. measures need to be used in order to generate the value needed to eclipse the effort of collection and reporting. Everyone must be educated on how to use measurement data and then continually asked to use the data. Both sides of the equation are necessary. The simple Measurement Readiness Checklist will be useful for any major measurement initiative, but is tailored toward beginning a measurement program.  The checklist will provide a platform for evaluating and discussing whether you have the resources, plans and organizational attitudes needed to implement a new measurement program or support the program you currently have in place.

I have divided the checklist into three categories: resources, plans, and attitudes.  Each can be leveraged separately. However, using the three components will help you to focus on the big picture. We will address each component separately over the next several days.

Scoring

This checklist can be used as a tool to evaluate how well you have prepared for your measurement journey. The following questions are the evaluation criteria.  To use the checklist, answer each question with high, medium, low and not present (with one exception). Each question will contribute points toward the total.

Section and Question Weights:

Resources: Forty-two total points. Each component contributes up to 7 points (7, 3, 1, 0).

Plans: Eighteen total points. Each component contributes up to 6 points (6, 3, 1, 0).

Attitude: Forty total points. Each component contributes up to 8 points (8, 4, 2, 0).

Note that where support and implementation projects would need to take a different angle we will point out any possible nuances.

Resources

Resources are the raw materials that you will consume on your measurement journey.  As with any journey having both the correct resources and correct amount of resources will make the journey easier.  Just think of trying to canoe from New York to London for a meeting; the wrong resources can make the trip difficult.

Management Support: When initially implementing a measurement program, support from management is the most critical resource.  This is the time when measurement seems to be all effort, cost and bother.  Later, as value is derived, support can be less visible.  Note that the more management support you have across the whole IT structure, the easier it is to get a measurement program on its feet and keep it there.

Scoring

7 – Senior management is actively involved in guiding which measures and metrics are collected and how they are used.  Senior managers stop people in the hall to discuss progress in collecting and using measurement data. Discussion of progress is an agenda item at all management-staff meetings.

3 – Senior and middle managers attend formal measurement informational meetings and talk about the need to support the measurement initiative.

1 – A senior manager or two attended the kick-off meeting, then relocated en mass to Aruba, leaving the middle managers in charge.

0 – The measurement initiative is a grass-roots effort.

Support Note:  Whether you are answering from a support or implementation perspective does not matter.  Management support is important.

Change Specialist: Measurement is a form of organizational change that typically requires skills that are not generally found in an IT department. The skills needed to start and perpetuate a measurement program include sales, marketing and communication.

7 – An organizational change specialist has been assigned as a full time resource for the project.

3 – An organizational change specialist is available within the organization and works on many projects simultaneously. The specialist may or may not have experience with IT change programs.

1 – Someone on the team has helped craft an organizational change plan in the past.

0 – Organizational change websites are blocked and your best bet is buying a book on Amazon using your own cash.

 Support Note: A change specialist is needed for ALL change programs regardless of whether we are discussing implementation or generating ongoing support.

Expertise: A deep understanding of measurement will be needed in a dynamic IT environment.  Experience is generally hard won. “Doing” it once generally does not provide enough expertise to allow the level of tailoring needed to deploy a measurement program in more than one environment. Do not be afraid to get a coach or mentor if this is a weakness.

7 – The leaders and team members working to implement and/or support the measurement program have been intimately involved in successfully implementing measurement in different environments.

3 –At least two team members have had substantial involvement in implementing a measurement program in the past, in a similar environment.

1 – Only one SME has been involved in a measurement program and that was in another environment.

0 – All of the team members have taken basic measurement classes and can spell measurement, assuming they can buy a vowel.

 Support Note:  You can never have a measurement program without someone with (or without access to) measurement knowledge.

We will finish the resource part of the checklist tomorrow.

Audio Version:  Software Process and Measurement Cast 149

Change is never easy and is begun many times because an organization or individual deems it necessary for survival or to avoid pain. Survival and pain avoidance while powerful can lead to accounting only for the negative side of preparation for change. On the other side of the equation, a positive side , everyone aware of the consequences of not changing versus is everyone aware of the benefits of changing.  Both are necessary pieces of knowledge as the intellectual benefits persuade, while pain avoidance sells.  The simple CMMI Readiness Checklist will be useful for any major change initiative but is tailored toward the requirements for implementing a framework like the CMMI.

I have broken this checklist into three categories; resources, plans and attitudes.  Each can be leveraged separately. However, using the three components will help you to focus on the big picture.

Scale

The simple checklist can be used as a tool to evaluate how well you have prepared for you CMMI journey using the questions as evaluation criteria.  To use the checklist, evaluate  each question  on a scale of high, medium, low and not present (with one exception). Each question will potentially contribute points toward the total which can be used to evaluate preparation.

Section and Question Weights:

Resources: Forty two total points. Each component contributes up to 7 points (7, 3, 1, 0).

Plans: Eighteen total points. Each component contributes up to 6 points (6, 3, 1, 0).

Attitude: Forty total points. Each component contributes up to 8 points (8, 4, 2, 0).

Resources

Resources are the raw materials that you will consume on your journey.  As with any journey having both the correct resources and correct amount of resources will make the journey easier.  Just think of trying to canoe from New York to London for a meeting; the wrong resources can make the trip difficult.

Management Support

Support from management is critical as we have discussed in past  checklists but so is support from your peers and from the teams that will be using the processes.

Score

7 – Senior management is actively involved in guiding and using the outputs of the CMMI.  Senior managers stop people in the hall to discuss progress and recently process implementations. Discussion of progress is an agenda item at all managers staff meetings.

3 – Senior and middle managers attend formal CMMI informational meetings and talk about the need to support the CMMI initiative.

1 – Senior managers attended the kick-off meeting, then relocating in mass to Aruba, leaving the middle managers in charge.

0 – The change initiative is a grass-roots effort.

Cash

Change costs money. Costs can include consultants, training, travel and an odd late-night pizza or two.

7 – A reasonable budget has been established and the implementation team can draw from the budget for planned expenditures.  Emergency funding can be attained to handle issues.

3 – A reasonable budget has been established and approved; however, access must be requested and justified for all expenditures.

1 – Any time that money is required funding must be requested and approved.

0 – Donations are sought in the organization’s lunchroom on a periodic basis (consider a PayPal donation button on your homepage).

Effort

Even if you have bales of cash, developing and implementing processes will require effort. Effort will be required from many constituencies including the process-improvement team, management and from the teams using the process, just to name a few.

7 – A reasonable staffing plan has been established and the change program is the only project the assigned resources have been committed to.

4 – A reasonable staffing plan has been established and the change initiative is the highest priority for the assigned resources.

1 – All resources are matrixed to the change initiative and are also assigned to other projects with high priority.

0 – You have all the effort you need after 5 PM and before 8 AM and during the working hours of company holidays.

Change Specialist

Organizational change requires skills that are not generally found in an IT department. The skills needed include sales, marketing and communication.

7 – An organizational change specialist has been assigned as a full time resource for the project.

3 – An organizational-change specialist is available within the organization and works on many projects simultaneously. The specialist may or may not have had experience with IT change programs.

1 – Someone on the team has helped craft an organizational change plan in the past.

0 – Organizational change websites are blocked and your best bet is buying a book on Amazon using your own cash.

Projects

Change requires something to impact.  The organization needs to have a consistent flow of projects so that changes are not one-shot attempts.

7 – Projects are constantly beginning that will provide a platform for implementing process changes.

3 – There are numerous projects in the organization; however they typically begin early in the year or on some other periodic basis that makes waiting a necessity if you are not ready exactly on time.

1 – The organization does only a small number of projects every year.

0 – The organization does one large project every year.

Calendar Time

Calendar time is a resource that is as important as any other resource. Severe calendar constraints can lead to irrational or bet-the-farm behaviors which increase risk.

7 – The schedule for implementing the CMMI is in line with industry norms and includes time for tweaking the required processes before appraising.

3 – The schedule is realistic but bare bones. Any problems could cause delay.

1 – Expectations have been set that will require a compressed schedule; however, delay will only be career limiting rather than a critical impact on the business.

0 – The CMMI implantation is critical for the organization’s survival and is required on an extremely compressed schedule.

Expertise

A deep understanding of the CMMI (or any other framework for that matter) will be needed to apply the model in a dynamic environment.  Experience is generally hard won. “Doing” it once generally does not provide enough expertise to allow the level of tailoring needed to apply the model in more than one environment. Do not be afraid to get a mentor if this is a weakness.

7 – The leaders and team members working to implement the CMMI have been intimately involved in successfully implementing the framework in different environments.

3 –The leader and at least one of the team members have been involved in implementing the CMMI in the past in a similar environment.

1 – Only the leader of the CMMI program has been involved with implementing the CMMI in an other environment.

0 – All of the team members have taken the basic CMMI course and can spell CMMI assuming they can buy a vowel.

Plans

Planning for the implementation of change can take many forms — from classic planning documents and schedules to backlogs.  I would suggest the structure of the plan is less of a discussion point than the content.  There are several plans that are needed when changing an organization. While the term “several” is used this does not mandate many volumes of paper  and schedules rather that the activities required are thought through and recorded, the goal is known and the constraints on the program have been identified (in other words the who, what, when, why and how are known to the level required).

Scale and Scoring

Plans: Eighteen total points. Each component contributes up to 6 points (6, 3, 1, 0).

Organizational Change Plan

The Organizational-Change Plan includes information on how the changes required to implement the CMMI will be communicated, marketed, reported, discussed, supported, trained and, if necessary escalated.

6 – A full change management plan has been developed, implemented and is being constantly monitored.

3 –An Organizational-Change Plan is planned but is yet to be developed. .

1 – When created the Organizational-Change Plan will be referenced occasionally.

0 – No Organizational-Change Plan has or will be be created.

Backlog

The backlog records what needs to be changed in prioritized order. The backlog should include all changes, issues and risks. The items in the backlog will be broken down into tasks as they are selected to be worked on.  The format needs to match corporate culture and can range from an agile backlog to a Microsoft Project Schedule.

6 – A prioritized backlog exists and is constantly maintained.

3 – A prioritized backlog exists and is periodically maintained.

1 – A rough list of tasks and activities is kept on whiteboard.

0 – No backlog or list of tasks exists.

Governance

Any change program requires resources, perseverance and political capital. In most corporations these types of requirements scream the need for oversight (governance is a code word for the less friendly word oversight). Governance defines who decides which changes will be made, when they will be made and who will pay for the changes. I strongly recommend that you decide how governance will be handled and write it down and make sure all of your stakeholders are comfortable on how you will get their advice, counsel, budget and in some cases permission.

6 – A full-governance plan has been developed, implemented and is being constantly monitored.

3 –A Governance Plan is planned but is yet to be developed. .

1 – When created, the Governance Plan will be used to show the process auditors.

0 – Governance . . . who needs it!

Attitude

When you talk about attitude it seems personal rather than organizational, but when it comes to large changes I believe that both the attitude of the organization and critical individuals (inside or outside the are important.  In many cases we must assess more than just one or the other. As you prepare to address the CMMI, the onus is on you as a change leader to develop a nuanced understanding of who you need to influence within the organization (see recent blog). The check list will portray an organizational view; however, you can and should replicate the exercise for specific critical influencers.

Scale and Scoring

Attitude: Forty total points. Each component contributes up to 8 points (8, 4, 2, 0).

Vision of tomorrow

Is there a belief that tomorrow will be demonstratively better based on the actions that are being taken? The organization needs to have a clear vision that tomorrow will be better than today in order to positively motivate the team to aspire to be better than they are.

8 – The organization is excited about the changes that are being implemented.  Volunteers to help or to pilot are numerous.

4 – Most of the organization is excited about most of the changes and their impact on the future.

2 – A neutral outlook (or at least undecided) is present.

0 – Active disenchantment with or dissension about the future is present.

Minimalist

The view that the simplest process change that works is the best is important in today’s lean world.  In many cases heavy processes are wearing on everyone who uses them and even when the process is okay today, entropy will add steps and reviews over time which add unneeded weight.  Score this attribute higher if the organization has a process to continually apply lean principles as a step in process maintenance.

8 – All processes are designed with lean principles formally applied.  Productivity and throughput are monitored to ensure that output isn’t negatively impacted.

4 – All processes are designed with lean principles formally applied; however, they are not monitored quantitatively.

2 – All processes are designed with lean principles informally applied.

0 – Processes are graded by the number of steps required with a higher number being better.

Learner

A learner is someone that is learning understands that they don’t know everything and that mistakes will be made. They understand that when made, mistakes are to be examined and corrected rather than swept under the carpet. Another attribute of a learner is the knowledge that the synthesis of data and knowledge from other sources is required for growth.  In most organizations an important source of process knowledge and definition are the practitioners — but not the sole source.

8 – New ideas are actively pursued and evaluated on an equal footing with any other idea or concept.

4 – New ideas are actively pursued and evaluated but those that reflect the way work is currently done are given more weight.

2 – The “not invented here” point of view has a bit of a hold on the organization, making the introduction of new ideas difficult.

0 – There is only one way to do anything and it was invented here sometime early last century.  Introduction of new ideas is considered dangerous.

Goal Driven

The organization needs to have a real need to drive the change and must be used to pursuing longer-term goals. The Process Philosopher of Sherbrooke argues that being goal-driven is required to be serious about change.  In many cases I have observed that a good focused near death experience increases the probability of change but waiting that long can create a negative atmosphere. A check-the-box goal rarely provides more than short-term localized motivation.

8 – The organization has a well-stated positive goal and that the CMMI not only supports but is integral to attaining that goal.

2 – The pursuit of the CMMI is about checking a box on a RFP response.

0 – CMMI is being pursued for no apparent purpose.

Conviction

Belief in the underlying concepts of the CMMI (or other change framework) provides motivation to the organization and individuals.  Conviction creates a scenario where constancy of purpose (Deming) is not an after-thought but the way things are done. Implementing frameworks like the CMMI are long-term efforts — generally with levels of excitement cycling through peaks and valleys.  In the valley when despair becomes a powerful force, many times conviction is the thread that keeps things moving forward. Without a critical mass of conviction it will be easy to wander off to focus on the next new idea.

8 – We believe and have evidence that from the past that we can continue to believe over time.

4 – We believe but this is the first time we’ve attempted something this big!

2 – We believe  . . . mostly.

0 – No Organizational-Change Plan has been created.

Scoring

Sum all of the scores and apply the following criteria.

100 – 80   You have a great base; live the dream.

79 – 60   Focus on building your change infrastructure as you begin the CMMI journey.

59 – 30   Remediate your weaknesses before you start wrestling with the CMMI.

29 –   0   Run Away! Trying to implement the CMMI will be equivalent to putting your hand in the garbage disposal with it running; avoid if you absolutely can!