Traceability: A Radical Approach (Parts 2 and 3)

Sorry  for the delay — I will have the figures added soon!

Putting the Model Into Action

The model for scaling traceability proposed in this paper is based on an assumption that there is a relationship between customer involvement, criticality and complexity.  The relations yields the required level of documentation required to achieve the benefits of traceability.  The model leverages an assessment of project attributes that define the three common concepts.  The concepts are:

•    Customer involvement in the project
•    Complexity of the functionality being delivered
•    Criticality of the project

A thumbnail definition of each of the three concepts begins with the concept of customer involvement, which is defined as the amount of time and effort applied to a project in a positive manner by the primary users of the project.  The second concept, complexity, is a measure of the number of project properties that are outside the normal expectations as perceived by the project team (the norm is relative to the organization or project group rather than to any external standard).  The final concept, criticality, is defined as the attributes defining quality, state or degree of being of the highest importance (again relative to the organization or group doing the work).  We will unpack these concepts and examine them in greater detail as we peal away the layers of the model.

The Model

Figure 1 - Overview of the Traceability Model

The process for using the model is a simple set of steps.
1.    Get a project (and team members)
2.    Assess the project’s attributes
3.    Plot the results on the model
4.    Interpret the findings
5.    Reassess as needed

The model is built for project environments. Don’t have a project you say!  Get one, I tell you! Can’t get one? This model will be less useful, but not useless.

Who Is Involved And When Will They Be Involved:

Implementing the traceability model assessment works best when the team (or a relevant subset) charged with doing the work conducts the assessment of project attributes.  The use of team members acts to turn Putt’s theory of “Competence Inversion ” on it head by focusing project level competencies on defining the impact of specific attributes.  The use of a number of team members will provide a basis for consistency if assessments are performed again later in the project.

While the assessment process is best done by a cross functional team, it can be also be performed by those in the project governance structure alone.  The smaller the group that is involved in the assessment the more open and honest the communication between the assessment group and the project team must be or the exercise will be just another process inflicted on the team.  Regardless of the size, the assessment team needs to include technical competence.  Technical competence is especially useful when appraising complexity.  Technical competence is also a good tool to sell the results of the process to the rest of the project team.  Regardless of the deployment model, diversity of thought is generated in cross functional groups that will provide the breadth of knowledge needed to apply the model (this suggestion is based on feedback from process users).  The use of cross functional groups becomes even more critical for large projects and/or projects with embedded sub-projects.  In a situation where the discussion will be contentious or the group participating will be large I suggest using a facilitator to ensure an effective outcome.

An approach I suggest for integrating the assessment process into your current methodology is to incorporate the assessment as part of your formal risk assessment.  An alternative for smaller projects is to perform the assessment process during the initial project planning activities.  This will minimize the impact of yet another assessment.

In larger projects where the appraisal outcome may vary across teams or sub-projects, thoughtful discussion will be required to determine whether the lowest common denominator will drive the results or whether a mixed approach is needed.  Use of this method in the real world suggests that in large projects/programs the highest or lowest common denominator is seldom universally useful.  The needs for scalability should be addressed at the level it makes sense for the project which may mean that subprojects are different.
Assessing Customer Involvement:

Customer involvement can be defined as the amount of time and effort applied to a project by the customers (or user) of the project.  Involvement can be both good (e.g. knowledge transfer and decision making) and bad (e.g. interference and indecision).  The goal in using the traceability model is to force the project team to predict both the quality and quantity of customer involvement as accurately as possible across the life of a project.  While the question of quality and quantity of customer involvement is important for all projects it becomes even more important as agile techniques are leveraged.  Customer involvement is required for the effective use of agile techniques and to reduce the need for classic traceability.  Involvement is used to replace documentation with a combination of lighter documentation and interaction with the customer.

Quality can be unpacked to include attributes such as competence; knowledge of the problem space, knowledge of the process and ability to make decisions that stick.  Assessing the quantity attributes of involvement requires understanding how having multiple customer and/or user constituencies involved in the project outcome can change the complexity of the project.  For example, the impact of multiple customers and user constituencies’ on decision making, specifically the ability to make decisions correctly or on a timely basis, will influence how a project needs to be run.  Multiple constituencies complicate the ability to make decisions which drives the need for structure.  As the number of groups increases, the number of communication nodes increases, making it more difficult to get enough people involved in a timely manner.   Although checklists are used to facilitate the model, model users should remember that knowledge of the project and project management  is needed to use the model effectively.  Users of the model should not see the lists of attributes and believe that this model can be used merely as a check-the-box method.

The methodical assessment of the quantity and quality of customer involvement requires determining the leading indicators of success.  Professional experience suggests a standard set of predictors for customer involvement which are incorporated into the appraisal questions below.
These predictors are as follows:

1.    Agile methods will be used                        y/n
2.    The customer will be available more than 80% of the time         y/n
3.    User/customer will be co-located with the project team            y/n
4.    Project has a single primary customer                    y/n
5.    The customer has adequate business knowledge            y/n
6.    The customer has knowledge of how development projects work         y/n
7.    Correct business decision makers are available                y/n
8.    Team members have a high level of interpersonal skills            y/n
9.    Process coaches are available                    y/n

The assessment process simplifies the evaluation process by using a simple yes-no evaluation.  Gray areas like ‘maybe’ are evaluated as an equivalent to a ‘no’.  While the rating scale is simple the discussion to get to a yes-no decision is typically far less simple.

Agile methods will be used:  The first component in the evaluation is to determine whether the project intends to use disciplined agile methods for the project being evaluated.  The term ‘disciplined’ is used on purpose.  Agile methods like xP are a set of practices that interact to create development supported by intimate communication.  Without the discipline or without critical practices, the communication alone will not suffice.  Assessment tip:  Using a defined, agile process equates to a ‘Y’, making it up as you go equates to an ‘N’.

Customer availability (>80%):  Intense customer interaction is required to ensure effective development and to reduce reliance on classically documented traceability.  Availability is defined as the total amount of time the primary customer is available.  If customers are not available, lack of interaction is foregone conclusion.  I have found that agile methods (which require intense communication) tend to loose traction when customer availability drops below 80%.   Assessment Tip: Assess this attribute as a ‘Y’ if primary customer availability is above 80%.  Assess it as an ‘N’ if customer availability is below 80% (which means if your customers are not around 80% of the time normally during the project without very special circumstances rate this as a No).

Co-located customer / user:  Co-location is an intimate implementation scenario of customer/user availability.  The intimacy that co-location provides can be leveraged as a replacement for documentation-based communication by using less formal techniques like white boards and sticky notes.  Assessment Tip:  Stand up look around, if you don’t have a high probability of seeing your primary customer (unless it is lunch time), you should rate this attribute as an ‘N’.  Leveraging metaverse tools (e.g. Secondlife or similar) can be used to mitigate some of the problems of disparate physical location.

Project Has A Single Customer:  As the number of primary customers increase, the number of communication paths required for creating and deploying the project increases exponentially.  The impact that the number of customers has on communication is not a linear, it can be more easily conceived as a web.  Each node in the web will require attention (attention = communication) to coordinate activities.  Assessment Tip: Count the number of primary customers, if you need more than one finger, assess this question as an ‘N’.

Business Knowledge:  The quality and quantity of business knowledge the team has to draw upon is inversely related to the amount of documentation-based communication needed.  Availability of solid business knowledge impacts the amount of background that needs to be documented in order to establish the team’s bona fides.  It should be noted that it can be argued that sourcing long term business knowledge in human repositories is a risk.  Assessment Tip:  Assessing the quality and quantity of business knowledge will require introspection and fairly brutal honesty, but do not sell the team or yourself short.

Knowledge of How Development Projects Work:  All team members, whether they are filling a hardcore IT role or the most ancillary user role, need to understand both their project responsibilities and how they will contribute to the project.  The more intrinsically participants understand their roles and responsibilities the smaller the amount of wasted effort a project will typically have to expend on non-value added activities (like re-explaining how work is done).  Assessment Tip:  This is an area that can be addressed after assessment through training.  If team members can not be trained or educated as to their role, appraise this attribute as an ‘N’.

Decisions Makers:  The project attribute that defines“decision makers” is the process that leads to the selection of a course of action.  Most IT projects have a core set of business customers that are the decision makers for requirements and business direction.  Knowing who can make a decision (and have it stick) then having access to them is critical.  Having a set of customers available or co-located is not effective if they are not decision makers (‘the right people’).  The perfect customer for a development project is available, co-located and can make decisions that stick (and very apt not to be the person provided).  Assessment Tip:  This area is another that can only be answered after soul searching introspection (i.e. thinking about it over a beer).  If your customer has to check with a nebulous puppet master before making critical decisions then assessment response should be an “N”.

High Level of Interpersonal Skills:  All team members must be able to interact together and perform as a team.  Insular or other behavior that is not team conducive will cause communications to pool and stagnate as team members either avoid the non-team player or the offending party holds on to information at inopportune times.  Non-team behavior within a team is bad regardless of the development methodology being used.  Assessment Tip:  Teams that have worked together and crafted a good working relationship typically can answer this as a “Y”.

Facilitation: Projects perform more consistently with coaching (and seem to deliver better solutions), however coaching as a process has not been universally adopted.  The role that has been universally embraced is project manager (PM).  Coaches and project managers typically play two very different roles.  The PM role has an external focus and acts as the voice of the process, while the role of coach has an internal focus and acts the as the voice of the team (outside vs. inside, process vs. people).  Agile methods implement the role of coach and PM as two very different roles, even though they can co-exist.  Coaches nurture the personnel on the project; helping them to do their best (remember your last coach).  Shouldn’t the same facility be leveraged on all projects?  Assessment Tip:  Evaluate whether a coach is assigned if yes answer affirmatively.  If the role is not formally recognized within the group or organization, care should be taken, even if a coach is appointed.

In preparation for scaling total the number of Y’s and N’s.  Scaling will be discussed after all components are described.

 

Traceability: A Radical Approach

Note:  This is the essay from SPaMCAST 66 at www.spamcast.net – Comments are welcome!

Introduction

Traceability is an important tool in software engineering, however, it is both hard to accomplish and requires a focused application to derive value.  Topping off the value equation, traceability is a core tenant of the CMMI.  The importance of traceability is derived both from its use as tool for the management and control of requirements and the need to prove traceability to attain a CMMI rating.  Controlling and understanding the flow of requirements puts a project manager’s hand on the throttle of the project by allowing he or she to understand (and control) the flow of work through a project.  In most cases, some degree of control is important, and with control mechanisms in place, the question becomes when does the control generated represent the proper hand on the throttle or a lead foot on break?

The implementation of traceability sets the stage for the struggle over processes mandated by management or the infamous “model”.  Developers actively resist process when they perceive the effort to not be directly leading to functionality that can be delivered.  Work not linked to the delivery of functionality is not perceived as delivering value to their customers.  In the end, traceability, like insurance, is best when the most obvious reasons (uncontrolled project changes and delivering functionality not related to requirements) for tracing are avoided.

Identifying both the projects and the audience that can benefit from traceability is paramount for implementing and sustaining the process.  Questions that need to be asked and addressed include:

• Is the need for control for all types of projects the same?
• Is the value-to-effort ratio from tracing requirements the same for all projects?
• What should be evaluated when determining whether to scale the traceability process?

Scalability is a needed step to affect the maximum value from any methodology component, traceability included, regardless of whether the project is plan driven or agile. A process is needed to ensure that traceability occurs based on a balance between process, effort and complexity.

The concept of traceability acts a lightening rod for the perceived excesses of CMMI (and by extension all other model-based improvement methods).  This paper explores a possible approach for scaling traceability.  The approach described in this paper bridges the typical approach (leveraging matrices and requirement tools) with an approach that trades documentation for intimate user involvement.  The approach described in this paper uses a simple set of three criteria (complexity, user involvement and criticality) to determine where a project should focus its traceability effort on continuum between documentation and involvement.

Agile methods seem to be at odds with the use of traceability.  The defining attributes of agile processes start with the encapsulation of development work into smaller pieces (terms used might include time-boxing, iterations or stories – albeit these terms are not exactly the same).  An integral component in agile methods is that real time communication is emphasized over written documents.  Techniques such as team co-location and including users as part of the team are leveraged to enhance intimate, constant communication and the team esprit de corp.  Projects using agile methods typically measure progress through the creation of working software (see http://en.wikipedia.org/wiki/Agile_methods for further descriptions of agile methods) rather than status reports, schedules and documentation.  The perception of agile software development (the perception is driven by published articles, unattributed stories and the occasional inappropriate application of the method) can cause groups using agile methods to be viewed as less disciplined.  My opinion is that when agile methods are implemented poorly, this view is less a perception than a fact.  This perception is put more eloquently in the WIKIPEDIA entry, which ends with the following comment, “Combined with the preference for face-to-face communication, agile methods produce very little written documentation relative to other methods. This has resulted in criticism of agile methods as being undisciplined[1]”.  The question presented to organizations is: can traceability be pursued when a project is using agile processes?

The principles behind the agile manifesto focus on the satisfying customers thought early and continuous delivery of software. As noted before, the agile methods stipulate that the most efficient and effective method of conveying information to and within a development team is face-to-face conversation[2]. Assuming the primacy of satisfying the customer, the process for each project must be assessed and tailored to most effectively and efficiently deliver functional software.  The level of business to customer involvement determines whether agile methods can be useful.  The levels of complexity and criticality affect the requirements for communication and the ability to simplify (maximize the amount of work not done).  Traceability becomes a tool that can be leveraged to bridge the gaps caused by less than perfect involvement, a complex project and increased criticality.  The model proposed in this paper provides a means to apply traceability in a scaled manner so that it fits a projects need and is not perceived as a one size fits all approach.

---

[1] http://en.wikipedia.org/wiki/Agile_software_development, third paragraph, 12/4/2007.

[2] http://agilemanifesto.org/principles.html, 11/20/2007

 

Synopsis Traceability A Radical Approach Based on User Involvement

‘Model based software process improvement’ and ‘process discipline’ are phrases that can chill the blood of most software engineers even when uttered forty feet away.  Applied incorrectly the perceived trappings of process discipline are viewed as overhead which gets in the way of ‘real work’.  The processes that are perceived to be the most offensive to developers are those concentrated on controlling their behavior or providing oversight of their work.  When the CMMI® is interjected into the process landscape, traceability becomes one of the lightening rods typically identified in the overhead discussion.

So, avoid the lightening rod, right?  Avoiding either the lightening or lightening rod is easier said than done if an appraisal to the CMMI® model is in your future.  Traceability[1] is a core tenant of the requirements management process area within the CMMI®.  Why is the effort to create and implement the processes needed to support traceability worth the investment? Simply put, traceability allows project personnel to know that what was planned was installed and what was installed was planned both at the end of the project and as the project progresses.  Having that knowledge is hard to argue with.  In some cases, knowing whether a requirement was planned or delivered is an imperative.  The problem is that “doing traceability” is not very easy.  Taking someone’s word that what was planned was delivered is so much easier.

Even when the traceability is deemed important, the perceived value of traceability depends on the effort required from the groups involved in gathering and using the data (you get what you put into the process).  As they deploy traceability, some of the burning questions all process engineers must answer are:

• Do the benefits of tracing accrue to all groups equally (assuming they exist)?
• Is the effort required to “do traceability” less than the value that will accrue for tracing requirements?
• Can the process be scaled so that all projects and all groups can derive more value than effort?

Traceability is a difficult concept to define a value proposition for all types of projects and an equally difficult concept to sell to all of the stakeholders in the world of projects.

“Traceability:  A Radical Approach Based on User Involvement” suggests an approach to scaling traceability based on balancing user involvement, risk and control needs.  This approach provides a means to balance the effort required to trace requirements with the value of doing it.  As a side benefit, this approach makes traceability a saleable concept.

---

[1] “Requirements traceability refers to the ability to describe and follow the life of a requirement, in both forwards and backwards direction (i.e. from its origins, through its development and specification, to its subsequent deployment and use, and through all periods of on-going refinement and iteration in any of these phases.)” Gotel O.C.Z and Finklestein A.C.W., “An analysis of the requirements traceability problem”, in Proceedings of ICRE94, 1st International Conference on Requirements Engineering, Colorado Springs, Co, IEEE CS Press, 1994

 

Traceability: A Radical Approach Based on User Involvement – Synopsis

Synopsis

‘Model based software process improvement’ and ‘process discipline’ are phrases that can chill the blood of most software engineers even when uttered forty feet away.  Applied incorrectly the perceived trappings of process discipline are viewed as overhead which gets in the way of ‘real work’.  The processes that are perceived to be the most offensive to developers are those concentrated on controlling their behavior or providing oversight of their work.  When the CMMI® is interjected into the process landscape, traceability becomes one of the lightening rods typically identified in the overhead discussion.

So, avoid the lightening rod, right?  Avoiding either the lightening or lightening rod is easier said than done if an appraisal to the CMMI® model is in your future.  Traceability[1] is a core tenant of the requirements management process area within the CMMI®.  Why is the effort to create and implement the processes needed to support traceability worth the investment? Simply put, traceability allows project personnel to know that what was planned was installed and what was installed was planned both at the end of the project and as the project progresses.  Having that knowledge is hard to argue with.  In some cases, knowing whether a requirement was planned or delivered is an imperative.  The problem is that “doing traceability” is not very easy.  Taking someone’s word that what was planned was delivered is so much easier.

Even when the traceability is deemed important, the perceived value of traceability depends on the effort required from the groups involved in gathering and using the data (you get what you put into the process).  As they deploy traceability, some of the burning questions all process engineers must answer are:

• Do the benefits of tracing accrue to all groups equally (assuming they exist)?
• Is the effort required to “do traceability” less than the value that will accrue for tracing requirements?
• Can the process be scaled so that all projects and all groups can derive more value than effort?

Traceability is a difficult concept to define a value proposition for all types of projects and an equally difficult concept to sell to all of the stakeholders in the world of projects.

“Traceability:  A Radical Approach Based on User Involvement” suggests an approach to scaling traceability based on balancing user involvement, risk and control needs.  This approach provides a means to balance the effort required to trace requirements with the value of doing it.  As a side benefit, this approach makes trace

---

[1] “Requirements traceability refers to the ability to describe and follow the life of a requirement, in both forwards and backwards direction (i.e. from its origins, through its development and specification, to its subsequent deployment and use, and through all periods of on-going refinement and iteration in any of these phases.)” Gotel O.C.Z and Finklestein A.C.W., “An analysis of the requirements traceability problem”, in Proceedings of ICRE94, 1st International Conference on Requirements Engineering, Colorado Springs, Co, IEEE CS Press, 1994

 

Agile Estimation Using Functional Metrics Parts 3 and 4

Agile estimation using functional metrics is designed to cover the product and release rings of Cohn’s planning onion using a synthesis of parametric and Delphi estimation techniques with the emphasis shifting from parametric to Delphi as events dictate.    The technique leverages the ability to size requirements to develop parametric estimates and then dovetails collaborative techniques to refine those estimates based on memories and self-knowledge.

The process flow is as follows

Stage One

1. Identification of functional requirements (or stories)
2. Sizing using Quick and Early Function Points
3. Simple Parametric Estimation

Stage Two – Sprint or Iteration Planning

1. Break Requirements into More Granular Pieces (if needed) and Refine Size
2. Team Level Re-estimation of Requirements Using Delphi Techniques
3. Team Level Commitment

Size is a critical component for developing an estimate and for planning however size and estimates are not synonymous.  Size is merely a step along the path from point A to point B.  As we move along the path size will be revisited twice.

The sizing process begins by segregating the functional requirements from the non-functional requirements.  The functional requirements are then sized using Quick and Early Function Points (QEFP).  Function points for all their warts are the easiest way to consistently size software requirements.  This is accomplished by focusing on the basic building blocks of functionality found in all software projects. The QEFP method leverages the relationship between action verbs and transactions to identify the transaction functions found in function points and the subject of the requirement to identify the data functions found in function points.  The application of this technique is similar to sentence diagramming that you learned during grade school or high school.  This relationship between words and size has been observed and investigated over the past few years by a number of different people within the functional sizing community including myself (see “Turning Perfect Good Words Into Numbers” originally presented at the IFPUG Functional Sizing Summit at www.davidconsultingroup.com).  Function points or any functional metric has at its heart the goal of converting requirements or stories into a number in a consistent manner.  The number then must be interpreted based on the abilities of the team or organization.  At the level of the overall project, the technique described in this paper leverages parametric estimation.  A simple parametric estimation equation could be:

Y =-(7^-6*(X^2))-Z*X+26.587

Where:

X = Size in function points

Y = Productivity rate for the type of project

Z = Behavior or Process Index

The result is a productivity rate for the project.  Collection of historical data on a selection of projects will be required to build an equation.  I would further suggest that you will need to augment internal data with external data to increase the validity of the estimation equation.  Note the factor can be applied to a disaggregated requirement, epic or story.  This estimate is created for organizational planning purposes.

Stage two begins as sprints or iterations are kicked off.  The sprint teams (we will use SCRUM terminology) breakdown the stories into pieces that can be accomplished during the sprint, then re-size the pieces using the QEFP method.   The goal of using QEFP at this point is to take one source of variance out of the estimation discussion that will be had when the Delphi method is applied.  This focuses the group on expanding the team level self-knowledge needed to coalesce on an appropriate level of effort needed to complete the story.  For example the QEFP technique has been combined with planning poker into a process that was quickly learned by the sprint teams.  The results were a marked reduction in stories that were not completed during the sprint they were committed to in.  By removing size as a variable the team that initially piloted this method indicated they were better able to focus on discussing team capabilities and technical considerations when doing their initial sprint planning.

Real life examples will help drive how organizations synthesized what could be considered competing methodologies into something greater than the sum of the two parts.

Part 4

Case One:

• Firm:   Small custom technology organization
• Project Types:  Internal and external projects
• Culture: Highly collaborative
• Current Methodology:  Mixed waterfall and SCRUM

Other notes:  All external projects are bid with many using a fixed fee structure.  Internal projects were continually re-scoped to fit the internal development budget which changes as the economy waxes and wanes.  Prior to rewriting the estimation process and leveraging functional metrics approximately 30% of bids were successful and budgets tended to be a suggestion.  The lack of estimation success meant that there was a significant risk of losing money if the business was won and if the business was not won going out of business in the long run.

The firm adopted Quick and Early Function Points for sizing the backlogs for all projects, both internal and external.  Where backlogs were not being used to manage requirements they were developed.  A quick baseline was developed to determine a productivity factor.  The productivity factor was then used to translate individual stories into effort.  Each team spent a day reviewing how they worked together to generate a baseline of self-knowledge and trust.    Collaborative story level estimation was redone using planning poker.  It became apparent quickly that a lot of disaggregation was needed to actually estimate the backlogs.  After applying QEFP and the productivity factor to the in-flight projects the firm progressed to applying QEFP to all bids.

The results were that won bids increased 20% and negative misses were nearly eradicated.  A negative miss was defined as underbidding on a fixed bid contract.

Case Two:

• Firm:   Large software development firm
• Project Types:  Internal projects (software for resale)
• Culture:  Hierarchy, classic command and control
• Current Methodology:  Mixed waterfall  (but SCRUM recently introduced)

Other notes:  The methodology in the environment was predominantly classic waterfall with central PMO.  Just before we readdressed the estimation process a team had implemented SCRUM and some components of extreme programming (xP) this was done in sort of a guerilla fashion.   One very large project was consuming the majority of the organizations resources.  Significant requirements were still being discovered after construction had begun.   Estimates had been developed based on a bottom up process very early in the project and they were of questionable validity.  The top managers just returned from begging for more money from the board of directors.  The project was being capitalized.

The solution in this case was for the company to more firmly embrace the SCRUM framework for project management at the team level.   A product backlog was developed and Quick and Early Function Points were adopted to size the backlog.  The sizing process exposed a number of functional blind spots (function points can be leveraged as form of analysis).  Team members were trained in using QEFP which allowed them to size new stories or re-size changed stories at an individual sprint planning level.  The impact of these changes was to allow the PMO to size and preplan the backlog with development leaders and the primary product owner.  The full product owner team selected stories to formulate sprints during the sprint planning meetings.  Teams re-evaluated (sized and estimated) the selected stories and then committed to the stories they could do.

The initial result was improved product owner satisfaction, involvement allowed them to be part of the solution.  After the first few sprint there was an increased perception of estimate consistency both at the product backlog and sprint team level.  It was also noted that the teams that had been using the using SCRUM before adopting the new estimation methods had a reduced number of stories that had not completed at the end of the sprint.  The teams attributed this improvement during retrospectives to a better capacity to size and commit to work that they’re actually able to accomplish during the sprint.

Summary

Agile methods have matured and are now being integrated into many different approaches to the development of software.  Estimation has been problematic for all methods; agile to plan based therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper.   Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile estimation.

 

Agile Estimation Using Functional Metrics Part 2

Uncertainty:

A lot has been written about uncertainty, mostly from the point of view of requirements, however the impact of uncertainty extends further than requirements into factors that can be purely technical (whether specific coding languages can do the job) to the complexities of the real world (cue the changing economy as an example).  If we change our perspective to completion of the project, I propose that we will all admit that level of project uncertainty is substantially reduced the closer you are to completion of the project.  Moving back toward the beginning of the project where most estimation exercises occur one simple truth becomes apparent, knowledge dispels uncertainty.  We need to gather better knowledge whether by leveraging history, mathematical algorithms and/or project specific information to make better estimates.   Integrating agile techniques for knowledge capture in projects are tools for reducing uncertainty.  Techniques to use include incorporating a user or user proxy on the team, focusing on short pre-defined time horizons, implementing processes that foster communication and periodic re-planning.

Self knowledge:

Two psychologists, Joseph Luft and Harry Ingham developed a construct to understand personal awareness.  The tool named Johari’s Window[i] divides personal awareness into four different categories, as represented by its four quadrants: open, hidden, blind and unknown. The lines dividing the four panes are like window shades, which can move as an interaction progresses.  The concept is adaptable to teams. Team level blind spots complicate estimation, planning and ultimately performance.  Techniques to improve a team’s self knowledge include forming stable teams, fostering intimate communications and ensuring retrospectives actually happen often.  These tools minimize what isn’t known by the team and surface misunderstandings quickly.

Consistency of Method

There are several types of estimation that can be leveraged during any project.

Estimation Types:

Let’s quickly review the four most popular estimation techniques in very broad terms.  The four are:

• Analogy
• Bottom-up
• Parametric
• Delphi

Estimation by analogy starts will the selection of a similar project (building a new bathroom based on the results of building a bathroom last week) which acts as a central metaphor for the new project.  The estimator will then decide how closely the two projects resemble each other and whether there are any mitigating circumstances that will affect the effort required to finish the project, how much the project might cost and how long it will take.   Based on these differences the estimator will apply a correct factor and voila, an estimate is created.  The second general category of estimation techniques is the bottom- up estimate.  An estimate of this type typically starts by identifying a set of technical deliverables (a shower stall, sink and pipes for bathroom with a shower if building a house) or work breakdown structure (the tasks needed to build a bathroom for our house).  The identified low level deliverables are estimated based on some form of history and then rolled up to higher and higher levels until the cost or effort for the entire project is known.  The third category estimation is called parametric estimation.  This form of estimation builds and leverages statistical relationships between historical data and one or more variables that define scope such as functional size (the number of square feet in the bathroom times the productivity of the builders).  The fourth type of estimation techniques can be grouped broadly in to the category of Delphi.  The central theme of Delphi techniques is the use of collaborative techniques to leverage group think to decide upon an estimate (the plumbers, electricians and carpenters get together and use a process to come to consensus on how much time is required to build the bathroom).  These techniques work best when the requirements being estimated can be stated a level of granularity that can be understood by those participating in the estimation session.

Mike Cohn has described the planning continuum using an onion analogy[ii].  Where strategy is the outer layer followed by layers for portfolio, product, release and iteration segments as you approach the onion core.  I suggest that there is no one tool or technique perfectly suited for each level in the onion.

Integrating Cohn’s planning onion into our earlier conversation of budgeting, estimation and planning,   I would suggest that strategy and portfolio levels are budgeting tasks.  The product and release layers are estimation tasks where as tools like white parametric estimation make sense.  Iteration and day-to -day organization are planning tasks where planning tools like schedules, kanban boards and standup meeting make sense to direct activity.  The method we are introducing in this paper combines the use of functional metrics and tools in the estimation step in a manner that fosters usage in the planning layers of the onion.  This set of techniques provides a consistent strategy and answers the changing information needs as the project evolves.  All this, while providing a collaborative environment for both the team and the client.

---

[i] http://en.wikipedia.org/wiki/Johari_window, June 14, 2009

[ii] http://www.mountaingoatsoftware.com/presentations/51, June 14 2009

 

Discipline and GNGB Introduction Part 2

Plan Based, Agile or Just Doing ‘Stuff’:

There seems to be confusion in the some corners of the software development world. Those afflicted with ‘the confusion’ have concluded that plan based methods can be replaced with random activity, ‘winging it’. Winging it can not the means to address problems in the real world. Discipline is required to consistently address real problems. Real problems means someone is watching. Discipline requires saying what you will do then being able to execute (whether promising to pick up the kids or finger paint the Mona Lisa). The how is a discussion point whether you use plan based methods or function based methods is a market choice not a discipline choice.

Good Numbers Go Bad – Introduction Part 2

The impact of measures and metrics is dependent on how closely they are linked to business goals and organizational strategy. The closer the linkage between business goals and measures and metrics the higher the probability that value will be derived. Metrics that are specifically tailored to address the organizations business context must not only deliver information as well as add value (information does not equal value). For example, if the business’s goal is to reduce cost, the metrics must measure the reduction of cost of production and the attributes that drive cost. Measuring other areas might be more appealing intellectually (for example quality and productivity) however they will not produce the value that is sought. Identifying and ensuring proper linkage helps ensure the value of measurement thought support of the ultimate business goals

Realistically business goals are typically not as cut and dry as reducing cost. Goals of market leaders (or aggressive newcomers) are typically targeted on towards expanding opportunities (creating disruptive innovation) and/or sales (expansion). Linking metrics to these types of goals can be addressed a two levels. The first is a macro view, measuring output (innovations or sales) and at a micro level, measuring the critical steps that lead to innovation or sales. If you are going to use the micro level strategy, focus only on measuring the most (most critical = small number) critical steps in the process. Measuring too many items can lead to death by measurement, an affliction that can be avoided. The goal of measurement is actionable information not death by numbers even though the later might be tempting.

Measurement programs represent the scaffolding for the analysis and presentation of data. The basic goal of a measurement program is organizing data so it can be interpreted more easily. Frameworks bring structure and organization to an overloaded information worker with allows them to first notice the data then to extract its information content. The quality of the framework will act as a governor on the level and speed of information extraction. The choice then becomes not whether a framework is needed but rather the efficiency of structure and the filters that will be applied. A viable framework of structure and discipline provides an anti-structure. Lack of a frameworks or an anti-structure belief is a will cause a large number of measurement problems, a prescription for making “Good Numbers Go Bad”.

 

Involvement Versus Focus; That Is The Question

Involvement Versus Focus; That Is The Question
Thomas M. Cagley Jr

Most process improvement programs extol the virtue of focusing on the needs of the organization or a specific class of users (usually senior or middle management). All said and done the virtue of focus is important but it is no longer sufficient to make change actually happen. The “organization” is being taken over by the World of Warcraft generation. This new crowd is beginning to believe they need to be involved if they are going to be asked to change how they are going to work. An example of that new expectation of involvement is reflected in the agile movement. Teams control how they do work and to an extent the work they will accept into the sprint. To the new generation, focus without involvement has come to mean big brother. Telling this new generation what to do whether you have their best interest in mind is deemed a paternalistic action, and for those with an expectation of being involved in their future, paternalism is at best grating and perhaps degrading. What does involvement mean? What is the new requirement to help an organization mature in terms of discipline and capability?

The American Heritage Dictionary defines involvement as “the act of sharing in the activities of a group”. At its core, I would suggest the concept boils down to collaboration between involved parties to achieve a common goal. I think we can all agree upon the ideal of using collaborative groups to conquer many types of projects. Workers acting in a concerted manner in real life leads to pressure on several organizational fronts in order to achieve this ideal. In a future essay we will explore another of my simple checklists (see www.tcagley.wordpress.com) to implementing involvement but in this essay I would like to discuss the impact of a common goal. In many organizations the management structure in use is the command and control methodology. In this method decisions are made at the top of the management pyramid then get transmitted to line management and then to project teams at the epicenter of work. In this model, management knows best because of experience, advanced degrees, having gone to a seminar or just divine right. To those being affected it doesn’t matter. The collaborative model suggests a different model in which management sets forth the goal then engages all parties to chart the course forward. Would the classic waterfall implementation of the CMMI survive a collaborative approach or would a more interesting approach to the discipline of software engineering emerge? I suggest that in combination the structure of the CMMI and the collaborative nature of AGILE methods are the natural outcome in today’s methodological environment. Note, I would suggest the pressure of the economy would add the philosophy of lean to the mix yielding a three legged stool of the CMMI as a framework and Agile and lean philosophies as the implementation techniques.

Change without involvement might work in the short run if you are bent on using a management philosophy of command and control but if you can step away from that model to embrace a more collaborative approach a better solution will be generated. A solution that yields discipline, collaboration, effectiveness and efficiency. Focus is a good thing but consider focusing on involvement rather than on using focus as a tool to speak for others.

 

Agile Estimation Using Functional Metrics, Part 1

Agile Estimation Using Functional Metrics, Part 1
by Thomas M. Cagley Jr.

The term agile has come to mean many things to many people.  The definitions and connotations range from how work is organized within a project to a description of the speed at which work is completed or alternately a radical rethinking of organizational culture.   Regardless of how you define agile I would suggest that we all would agree that agile methods are now maturing.  Part of the process of maturing is the incorporation of best practices from other methods and frameworks creating a hybrid.  The fringe is influencing the center and the center is influencing the fringe.  The hybrid is at once better than any of the absolutes and threatening to those who believe in absolutes.

Estimation has been a lightning rod for the discussion all methods (agile, waterfall, iterative or water fountain) with the issues of predictability and standardization radiating outward.  Because of the controversy this is an area where a wide range of hybridization has always occurred.  Organizations adjust techniques to fit governance structures, culture and risk profiles.  There is no one size fits all solution.   This paper provides a path for incorporating the use of function points into agile estimation techniques.  The process will yield an estimation process that combines one part functional metrics, one part parametric estimation techniques with two parts agile estimation (heavily influenced by Mike Cohn).   I would suggest that functional metrics provide a path for incorporating the best practices of robust software sizing with the collaborative techniques championed by the agile community in a manner that increase standardization without ignoring the principals of the Agile Manifesto.

Budgeting, Estimation and Planning

I’d like to begin this discussion by challenging your preconceived notion of estimation as compared to the activities of budgeting and planning .  These three concepts are sometimes thought of as being synonymous however I believe it is important to understand just how different these concepts are.  Each has different inputs and outputs, uses different tools and techniques and is generally used by different groups within the organization.

A quick overview of the macro differences are:
•    Budgeting
o    Defines how much we have to spend based on the influence of scope
o    Tends to ignore the cone of uncertainty
•    Estimation
o    Presents an approximation of effort and duration based on size and project nature
o    Focused by the cone of uncertainty (a range based on knowledge)
•    Planning
o    Defines tasks and allocates resources
o    Focused on the narrow part of the cone of uncertainty (a much smaller range)

Estimation, planning and budgeting might be related but they are certainly not the same.  The use of functional metrics in agile estimation is targeted at the estimation layer of this three layer cake but provides support for planning.  Developing a basic understanding of the components of estimation (we are going to ignore budgeting as bastion of guesses) and its relationship to sizing is critical to using these techniques.

Estimation

Estimation is several parts science and a least one part magic.  This strange confluence of science and magic defines the transformation of requirements size, skills, people and equipment into how much the project will cost and how much effort it will take .  The whole process of transformation is bound by a cone of uncertainty.  Uncertainty builds boundaries around the false precision of the estimate providing a range around the estimate based on what is known and unknown.  Collaborative estimation techniques are good at increasing team knowledge while reducing the amount of self-deceit that can occur when knowledge is discussed.

The amount of art increases as the estimation discipline is replaced by the planning discipline.  The art of planning matches specific tasks with people thru a process of assignment.  In a perfect world estimates and planning would be able to be done together in seamless workflow but estimates happen generally earlier in the project lifecycle before you can decompose work into tasks which is required for planning.

The simplest form of any estimation model, human or tool based is a mathematical mash-up of size (implied or counted), team and organizational behavioral attributes and degree of difficulty (technical complexity) applied to a productivity signature.  As the level of sophistication in the mathematics increases tools SEER, SLIM or KnowledgePlan make sense.  Other methods raise level of collaboration and do any of the required math in the heads of the participants.  These techniques include Delphi, analogy or planning poker. The process in this paper splits the difference leveraging collaboration to increase participation and self knowledge while suggesting the use of a simple spreadsheet based parametric models to increase consistency and standardization.

Sounds simple, right?  Estimation has been a nagging pain in every IT manager’s backside since a user asked how much a project would cost and when it would be done.   We have gotten pretty good at budgeting using techniques like “x number of people times 20 hours in a day and you’ll get something next year” methods.  It’s when we try to figure out how much functionality will be delivered in real life that  things start to break down or least get very, very complicated.

There are three main categories of problems that cause estimation to be problematic in the real world.
1.    Uncertainty: how much do you know about what you’re building?
2.    Self knowledge: what you do really know about yourself and your team?
3.    Consistency of method: do you have a process for estimating?

 

Relevance

Essay from SPaMCAST 58 (short and sweet)

Relevance is power

Irrelevance is desperation

Relevance is perceived

So is Irrelevance.