Discipline and GNGB Introduction Part 2

Plan Based, Agile or Just Doing ‘Stuff’:

There seems to be confusion in the some corners of the software development world. Those afflicted with ‘the confusion’ have concluded that plan based methods can be replaced with random activity, ‘winging it’. Winging it can not the means to address problems in the real world. Discipline is required to consistently address real problems. Real problems means someone is watching. Discipline requires saying what you will do then being able to execute (whether promising to pick up the kids or finger paint the Mona Lisa). The how is a discussion point whether you use plan based methods or function based methods is a market choice not a discipline choice.

Good Numbers Go Bad – Introduction Part 2

The impact of measures and metrics is dependent on how closely they are linked to business goals and organizational strategy. The closer the linkage between business goals and measures and metrics the higher the probability that value will be derived. Metrics that are specifically tailored to address the organizations business context must not only deliver information as well as add value (information does not equal value). For example, if the business’s goal is to reduce cost, the metrics must measure the reduction of cost of production and the attributes that drive cost. Measuring other areas might be more appealing intellectually (for example quality and productivity) however they will not produce the value that is sought. Identifying and ensuring proper linkage helps ensure the value of measurement thought support of the ultimate business goals

Realistically business goals are typically not as cut and dry as reducing cost. Goals of market leaders (or aggressive newcomers) are typically targeted on towards expanding opportunities (creating disruptive innovation) and/or sales (expansion). Linking metrics to these types of goals can be addressed a two levels. The first is a macro view, measuring output (innovations or sales) and at a micro level, measuring the critical steps that lead to innovation or sales. If you are going to use the micro level strategy, focus only on measuring the most (most critical = small number) critical steps in the process. Measuring too many items can lead to death by measurement, an affliction that can be avoided. The goal of measurement is actionable information not death by numbers even though the later might be tempting.

Measurement programs represent the scaffolding for the analysis and presentation of data. The basic goal of a measurement program is organizing data so it can be interpreted more easily. Frameworks bring structure and organization to an overloaded information worker with allows them to first notice the data then to extract its information content. The quality of the framework will act as a governor on the level and speed of information extraction. The choice then becomes not whether a framework is needed but rather the efficiency of structure and the filters that will be applied. A viable framework of structure and discipline provides an anti-structure. Lack of a frameworks or an anti-structure belief is a will cause a large number of measurement problems, a prescription for making “Good Numbers Go Bad”.

 

Which Software Size Metric Should I Use? Part 2 (draft)

Knowing the measurement goal that the size metrics you are selecting actually supports is a big deal (extra credit points). The goal has to be a key contributor to the decision process.

While knowing the goal your size metric will support is important it will help you make a cut based on four macro attributes between organization specific and industry defined metrics and between physical and logical metrics. For example, if benchmarking against other firms or industry data is required to attain your measurement goal using organizationally defined metrics would be less viable. Similarly if you have a heterogeneous software environment then selecting a functional metric would make more sense than using a physical metric (logical metrics normalizes varied technology).

Figure 1: Focus

The second checkbox is whether the measure has an externally defined and documented methodology. Why is definition important? Definition is the precursor to repeatability and consistency which allows comparability. Consistency and repeatability are prerequisites for the ability to generate data needed to use the scientific method such as Six Sigma and tools used to support Kiazen. Finally, an external definition reduces the amount of effort that is required to construct and implement measurement programs.

Even where a definition exists a wide of nuances are possible. Examples of the range of definitions begin with the most defined, the functional precision of ISO functional metrics to the less defined methodology of Use Case Points which began with a single academic definition and has evolved into many functional variants. The variants seen in UCP are a reflection of having no central control point to control methods evolution which we will explore later in this model. The range of formality of definition is captured in Figure Two.

Figure Three consolidates the view of formality of definition with the delineation between logical and physical metrics. Each measure has strengths and weaknesses. The first two items in our checklist are macro filters.

Once the macro filter is applied each subsequent step in the checklist will narrow the field.

 

“You Get What You Look For, So Look!”

“You Get What You Look For, So Look!”

Thomas M. Cagley, Jr

 

During a discussion of genes and metabolism on “NPR’s Talk of the Nation: Science Friday” on September 7^th, Dr. Jonathan Graff stated, “you find what you are looking for.”    While we all can recall stories of serendipity like the creation of Ivory Soap, I think there is a more than a kernel of truth to the statement therefore to find the vein of gold rather than the occasional gold nugget you will:

1.  Have to be looking.
2. Apply the scientific method to your analysis.
3. Visualize the results.

 
Early in the development of measurement programs there is a tendency to place more significance on accumulating data than is warranted.  This tendency stems from the need to show progress, pointing to a big pile of data can be considered a sign of progress.  The second reason for this tendency the belief that knowledge cannot be extracted until some magical quantity of data is accumulated.  While I am firmly aware that statistical significance is linked a sample size, I would strongly suggest that in waiting, measurement organizations do it themselves a disservice.  The disservice stems from shying away from taking the necessary steps into developing hypotheses, analyzes and reporting (assuming the results are served with appropriate caveats).

 
Yes, I’m suggesting diving into the deep end of the measurement and analysis pool.  The dive in action serves more than one purpose. The first purpose is that analyzing the data will allow the measurement program to test its processes.  Are the processes what is needed to develop an understanding of the data and to visualize the results?  If process changes are needed, we can agree that it is easier to make a change earlier rather than later.  Diving in allows the program to begin organizational education based on real data.  Training in my book includes not only how to use the data but also how to question and interpret strange patterns. Pattern recognition is a normal tool used to interpret data.  Real data makes teach pattern recognition easier.  Beginning to report data immediately also puts pressure on the organization to clean up the data.  I once had a manager that stated the quickest way to make sure that data it was good was to force managers to use the data or to explain why they did not.  I thought he was crazy when I first heard his strategy.  The data (which was horrible to begin with) quickly of began to approximate reality when the project managers were required to use and explain the data.  The process proved that that the organization was serious.  Finally, not waiting insurers that you don’t wait forever; it is way too easy to wait for more data or for it to get a little better. 

 
The value to information is insight, not the physical accumulation of data, pure and simple.  We all can agree that just having data does not provide significant knowledge.  Measurement programs need to embrace knowledge creation as a central core value.  Knowledge can be interpreted and applied to a situation or scenario so that decisions can be made.  The ability to make rational decisions with proper information has value that can be identified, measured and shared.

 

Human Interaction During Testing on Two Continents

Testing is the means of proving that the development process has understood and built what was required.  In its purist form it provides a set of proofs that validate the ‘whole’ development life cycle.  Regardless of the test model used there is one overriding goal; to deliver the best product possible within the constraints of time, budget and scope.  Meeting the goal of testing requires a strong level of interaction and communication between the testing and development teams.  This requirement becomes an imperative when testing occurs on two continents.  

One day I woke up to a gorgeous sunrise then the phone rang and I inherited the testing component of a development project.  The project was well along, with development nearly complete with the external system and user testing looming.  Development and testing were to be done on two continents.  In a nutshell, the project was troubled (the usual cast of characters were at work; over budget, late, feuding sub-teams and subject to runaway requirements).  As soon as I heard the words; project, testing and soon, a mental assessment checklist popped to mind: 

• Had test planning been performed and reviewed (and by whom)?
• Were there test cases that actually proved the requirements?
• Had the developers been a party to the creation of these cases?
• Had user acceptance tests been defined?
• What were the criteria for acceptance (functionally, quality and date)?
• When was the last time that the test managers (and/or the project manager) actually talked, rather than emailed each other?
• What was driving the current project delivery date?
• Was there a phone list of project participants?
• What time was it in
  India?

 

Testing is a human activity built on communication and interaction both with the project deliverables and the participants.  My first stop was the world clock at (www.timeanddate.com/worldclock) and then the phone list.  The test plan was my third stop.  If done correctly it would provide the basis to synchronize the test teams.  The test plan is a core management communications and negotiation tool.  The plan compiles and communicates the information about testing activities to provide a common way for all involved parties to understand and track test activity.  The plan lines up resources so that even a manager can know what will be done, when it will be done and who will do it.  The critical components of a test plan to drive multiple testing groups on two continents are: 

• An overall statement defining the minimum level of testing that will be acceptable.
• Test ownership
• The application components and subcomponents with their relationships
• A test schedule (with effort, duration and resources)
• The test metrics
• The acceptance criteria
• The communication plans

 

Defining the minimum level of testing for the project creates baseline, a tool to define what the quality goal of the project really is and then to resist cutting testing below a level that would meet the goal. 

In any project it is important to determine the types of tests to be done.  Examples include regression testing, usability testing, system testing to name a few, however, when testing is done with more than one team it is imperative to determine who will be responsible for each test.  A single point of contact is (or at worst a single team) must be identified to lead and be responsible for a specific test.  Note, this does not mean that only one team can be involved just that someone needs to be in charge.  This is critical to making sure communication and information flow can be managed. 

Components and the schedules are related items.  Knowing the components and subcomponents of the application being built or modified and their relationships provides the knowledge needed to create a road map to plan testing and for building a schedule for actually performing the test.   

Metrics provide a means to monitor testing while in progress, rather than simply relying on verbal status.  Verbal status, while important, can easily be misinterpreted or represent progress that is out of date.  Metrics provide information and knowledge that cuts across any cultural boundaries.  The metrics should describe software functionality tested, degree of code coverage and progress against a schedule which are core management issues.  Publishing test metrics provides a language to synchronize not only the test teams but the project team, as a whole.  

Acceptance criteria serve two purposes.  First, acceptance criteria define when a project is complete.  Second, if created and documented early in a project (before coding to take a page from the agile methodologies) they serve as a communication tool for validating requirements.  Acceptance criteria provide a language of acceptance that cuts across communication barriers.   

Providing a plan for the compilation and communication of complete information about the plans, progress and problems for testing allows both continents to provide support and anticipate changes in their own schedule. Interaction, communication and knowledge are important components to making testing happen and becomes even more important when more than one team is involved or when testing done on two continents.  With adequate information, testers are better able to plan and perform their tasks more efficiently and to get an accurate picture of the system. Did I mention communication was important and that it must happen over the entire project life cycle? 

Testing is frequently unpredictable, and unpredictability, the bane of managers, creates painful schedule surprises and Maalox moments. This is especially true when multiple teams in multiple locations with multiple cultures are involved.  Communication and coordination based on planning that combines all of the different perspectives of testing and system development  (schedules, functionality, code and problem resolution) makes it is possible to understand and manage software testing.  Communication based on a plan lets you manage the testing process on one continent or two rather than to allowing  it to manage you. 

 

Involvement, Part 2

Associations and organizations are driven by people: both members and non-members.  Whether you are a member or non-member there is a connection through a common thread of software measurement, process improvement, agile methods or beer brewing.  Regardless of the thread, the commonality binds the group together so that it can form a community.  In associations the backbone of the community is the volunteer who maintain and administer the programs.  An example is the volunteers that are the core of the International Function Point Users Group (IFPUG).  The volunteers have made IFPUG the premier software measurement association.  The diversity of the volunteers, who are truly international coming from almost all continents and all types of organizations (corporate, academia, government and consulting), shows how people of all races, creeds, colors can create a community greater than the simple sum of the parts.   

Within associations, each volunteer has as chance to make his or her mark on the organization he or she has chosen to be involved in.  By making their mark they have the ability to reap the benefits of involvement both in terms of altruism (giving back to the community) and well as greater opportunities to meet and network with other in thier field.  Involvement is important for the organization and the individual, without the symbiosis the relationship would wither over time.  Get involved in a group; volunteer.  Professions or hobby, it really is all about the people 

 

Why Are Requirements . . Part Two –

This is a work in progress. . . I am working on this while I print my taxes.  Thoughts on the essay in progess?

 

***** 

 

The factors driving the people category are typically the most volatile and a seemingly least controllable of the variables within the requirements process.  This essay will focus on the ‘people’ category with subsequent assay is focusing on process, environment and solutions.

 

People have a major impact on the vagaries of requirements.  All of the strengths and weaknesses that individuals and groups bring to the table will influence the final product,  the requirements.  There are numerous definitions of ‘requirements’ however in the final view of what the functionality developed by the project should do. A few of the more problematic contributors to requirements variance are:

 

1. Lack of experience

2. Human nature

3. Communication

4. Organizational politics

 

Two types of experience are germane to this discussion of the requirements development process.  The first is knowledge of the problem space from a business perspective. Without knowledge of the problem space, the requirements developed may not practically address what the project should do. Knowledge and experience with the problem space is critical for effectiveness there fore many techniques to manage this risk have been developed.  One technique that has been developed to ensure problem space knowledge is to ensure access to the business partner.  This level of access is a central tenet of the most of the agile methods is to keep knowledge and experience close at hand. The second category of experience is experience with the requirements process itself. Without experience gathering, recording and managing the requirements process the information gathered is apt to be more costly than necessary and less valuable than needed.  Agile methods use coaching to reinforce this knowledge and experience while other methods use training and processes.  The goal is the same in either case, efficiency and effectiveness.

 

An Estimate Is Just A Number, Right?

A version of this essay will be / was presentend on the SPaMCAST Six podcast (www.spamcast.net).

 

An Estimate Is Just A Number, Right?

Thomas Cagley

We’ve all been asked for an off-the-cuff estimate and gotten burnt. What is more problematic is that we continue the behavior even after being burnt. Worse yet, even though we know this form of behavior is destructive, we feel we have no choice.  Not answering is not allowed. The behavior is rationalized by viewing the off-the-cuff estimate as a time honored IT tradition.  What is wrong with tradition?  A first estimate is just a number right?   We forget that an estimate, even in the most benign case sets expectations . . . Once answered the resultant number, range of numbers or frame of reference creates a set of boundaries in the requestors mind and as importantly in your mind. The changes which will be required as knowledge overtakes unknowns will require more effort then if you’d gotten the phrase, “I’ll get back to you….” out of your mouth.  

Estimates are driven by many factors.  I would like to focus on the grand-daddy of all factors.  The most critical factor in the quality of any estimate (at least in my opinion) is requirements.  The love/hate relationship between the hard cold number of an estimate and the fuzzy concept that is requirements begins even before a project starts.  It begins with the intial hallway conversation.  How accurate can an estimate for a project be when all you have is a one line description or at best a paragraph in an email (either case might be the poster child for an elevator speech)?  The goal of requirements is to describe the problem in terms of a solution state, what the project is supposed to do when it is complete.  Knowing what you want to end up with is the knowledge that makes creating an estimate possible (let alone what makes creating the project).  Changes in the knowledge of the project team and the conceptual ideas describing the solution before requirements follow a non-linear path.   The one paragraph description is jump into the future without all of the facts.  Jumping to the end without all the facts reduces the probability of an estimate being correct to a level equal to any small random number.  As importantly the process of jumping to the end engenders the behavior of not listening by putting up barriers which makes the inevitable change difficult to embrace.  Training yourself to not listen reduces your ability to hear requirements that are outside the paradigm used to generate the initial estimate.  Barriers to improved knowledge and recognizing change reduces the ability to create accurate estimates. Providing a bad estimate early in a project creates a catch 22 for the project manager.  When change occurs it reduces the project manager and estimators credibility which leads to an aggressive defense behavior by not hearing anything that endangers the original position. 

There are many strategies to address the requirements-estimation conundrum.  Methods such as estimation funnel (a process strategy), estimating by phase (a project methodology strategy) and the planning components of agile methods (a different project methodology strategy) will address significant portions of the conundrum.  These techniques work by relating knowledge growth to accuracy.  Regardless of the logic of relating estimates to knowledge, breaking the underlying belief in an estimate as an absolute is not for the timid.  Improving the relationship between requirements and estimation within given company will be a factor of the organizational culture in effect. 

When is an estimate just a number?  Maybe the question should be when is an estimate just a number?  When is an estimate not a promise? When is not a contract with your boss, customer or both? The crass answer might be when you can the answer the following questions; when will it be done, how long it will take, or how much will the cost with mere suggestions. Or maybe when changes to requirements or processes can occur without an impact to the delivery date, cost or the hours you’ll have to work.  Bottom line is an estimate is just a number when it’s wrong and you know it.

 

Traceability: Interpreting The Model (A Beginning)

After assessing the three components (customer involvement, criticality and complexity), count the number of yes and no answers for each of the components.   Plotting the results is merely a matter of indicating the number of yes answers on each axis.  For example if  an appraisal yields

Customer Involvement:   8 Yes  1 No

Criticality:                        7 Yes  2 No

Complexity:                     5 Yes  4 No

 

Could be shown graphically as:

 

 

 

 

The Traceability model is premised on the idea that as criticality and complexity increases the need for communication intensifies.  This communication becomes more difficult as customer involvement becomes more arms length.  Each arm of the model influences the other to some extent.  In circumstances where customer involvement is high there are many different planning and control tools that are available to a project than when they are not.  The inter-relations of the outcomes of the assessment on each axis will suggest a different implementation of traceability.  Continuums are difficult to implement therefore for ease of use, I suggest an implementation with three basic levels of traceability (the Three Bears Approach); Papa Bear or formal / detailed tracking, Mama Bear or formal with function level tracking and Baby Bear or informal (but disciplined) / story based tracking. 

Interpreting the axises.

 

Transpose the axises you have plotted to the model shown earlier (see example below).

 

 

I will continue on interpreting the model in the next installment along with the final version of the essay ‘Model, Model Everywhere.’

 

 

Using and Interpreting the Model (I changed it a bit)

Putting the Model Into Action

The traceability scaling model proposed in this paper is based on an assessment of project attributes that are encompassed by three common concepts. 

• Customer Involvement
• Complexity
• Criticality

A thumbnail definition of each of the three concepts begins with the concept customer involvement.   Customer involvement is defined as the amount of time and effort applied to a project in a positive manner by the consumers of the project.  The second concept, complexity, is a measure of the number of properties a project has that are outside the normal expectations within the organization the project is being done (the norm is relative to the organization or project group rather than to an external standard).  The final concept, criticality, is defined as the quality, state or degree of being of the highest importance (again relative to the organization or group).  We will unpack these concepts and examine them in greater detail as we peal away the layers of the model. 

The model can be visualized as follows.

 

 

Interpreting the model:

The Using the traceablity model is premised on an assessment of the team and project attributes.  The evaluation/assessment is driven by a defined set of common attributes (discussed in the next section). The model premises that as criticality and complexity increase, the need for traceability increases (increased formality and granularity).  The impact of reducing customer involvement (good involvement) has a similar impact.    

The assessment process can be done by the team or project governance structure but to be effect the communication must be open and honest.  A means to integrate the assessment process into your current methodology by doing it as a part of your formal risk assessment.  For smaller projects using it during project planning is another avenue for minimizing the impact of another assessment for the project.  I strongly suggest using cross functional groups to provide the breadth of knowledge needed to apply the model.  Cross functional groups are most useful for large projects and/or projects with sub-projects.  Use a facilitator for large groups or when the outcome will be contentious.  In larger projects where the answers may vary, thoughtful discussion will be required to determine whether the lowest common denominator will drive the results or whether a mixed approach is needed.  When a portion of the team meets the criteria for full tracing the need should not be overruled to meet the profiles of other subprojects. 

 

SPaMCAST and the Process Improvment and Measurement Blog

Earlier this week I launched a podcast to help provide a lightening rod, an epicenter for dialog for software process improvment and measurement.  The Process Improvement and Measurement blog includes many of the discussions that will be converted into essays.  I invite you to comment and interact with the materials in both this blog and the the podcast.  Start a discussion  or rant.  Help generate a community. 

The Software Process and Measurement Cast, SPaMCAST is a podcast (time sliced internet radio) that explores the varied world of software process improvement and measurement.  The cast covers topics that deal with the challenges found in information technology organizations as they grow and evolve.  SPaMCAST typically follow a format that begins with a commentary or essay followed by an interview and listener comments.  The interviews will address the wide range of software process improvement or measurement topics.  The list of topics and interviews that are currently scheduled include:

• Process and Product Quality Assurance
• Estimation
• Software measurement
• Function Points
• Other Functional Metrics
• Outsourcing the Measurement Process

 

 SPaMCAST - Software Process and Measurement Podcast   
SPaMCAST.libsyn.com or iTunes!