Sorry for the delay — I will have the figures added soon!
Putting the Model Into Action
The model for scaling traceability proposed in this paper is based on an assumption that there is a relationship between customer involvement, criticality and complexity. The relations yields the required level of documentation required to achieve the benefits of traceability. The model leverages an assessment of project attributes that define the three common concepts. The concepts are:
• Customer involvement in the project
• Complexity of the functionality being delivered
• Criticality of the project
A thumbnail definition of each of the three concepts begins with the concept of customer involvement, which is defined as the amount of time and effort applied to a project in a positive manner by the primary users of the project. The second concept, complexity, is a measure of the number of project properties that are outside the normal expectations as perceived by the project team (the norm is relative to the organization or project group rather than to any external standard). The final concept, criticality, is defined as the attributes defining quality, state or degree of being of the highest importance (again relative to the organization or group doing the work). We will unpack these concepts and examine them in greater detail as we peal away the layers of the model.
The Model
Figure 1 - Overview of the Traceability Model
The process for using the model is a simple set of steps.
1. Get a project (and team members)
2. Assess the project’s attributes
3. Plot the results on the model
4. Interpret the findings
5. Reassess as needed
The model is built for project environments. Don’t have a project you say! Get one, I tell you! Can’t get one? This model will be less useful, but not useless.
Who Is Involved And When Will They Be Involved:
Implementing the traceability model assessment works best when the team (or a relevant subset) charged with doing the work conducts the assessment of project attributes. The use of team members acts to turn Putt’s theory of “Competence Inversion ” on it head by focusing project level competencies on defining the impact of specific attributes. The use of a number of team members will provide a basis for consistency if assessments are performed again later in the project.
While the assessment process is best done by a cross functional team, it can be also be performed by those in the project governance structure alone. The smaller the group that is involved in the assessment the more open and honest the communication between the assessment group and the project team must be or the exercise will be just another process inflicted on the team. Regardless of the size, the assessment team needs to include technical competence. Technical competence is especially useful when appraising complexity. Technical competence is also a good tool to sell the results of the process to the rest of the project team. Regardless of the deployment model, diversity of thought is generated in cross functional groups that will provide the breadth of knowledge needed to apply the model (this suggestion is based on feedback from process users). The use of cross functional groups becomes even more critical for large projects and/or projects with embedded sub-projects. In a situation where the discussion will be contentious or the group participating will be large I suggest using a facilitator to ensure an effective outcome.
An approach I suggest for integrating the assessment process into your current methodology is to incorporate the assessment as part of your formal risk assessment. An alternative for smaller projects is to perform the assessment process during the initial project planning activities. This will minimize the impact of yet another assessment.
In larger projects where the appraisal outcome may vary across teams or sub-projects, thoughtful discussion will be required to determine whether the lowest common denominator will drive the results or whether a mixed approach is needed. Use of this method in the real world suggests that in large projects/programs the highest or lowest common denominator is seldom universally useful. The needs for scalability should be addressed at the level it makes sense for the project which may mean that subprojects are different.
Assessing Customer Involvement:
Customer involvement can be defined as the amount of time and effort applied to a project by the customers (or user) of the project. Involvement can be both good (e.g. knowledge transfer and decision making) and bad (e.g. interference and indecision). The goal in using the traceability model is to force the project team to predict both the quality and quantity of customer involvement as accurately as possible across the life of a project. While the question of quality and quantity of customer involvement is important for all projects it becomes even more important as agile techniques are leveraged. Customer involvement is required for the effective use of agile techniques and to reduce the need for classic traceability. Involvement is used to replace documentation with a combination of lighter documentation and interaction with the customer.
Quality can be unpacked to include attributes such as competence; knowledge of the problem space, knowledge of the process and ability to make decisions that stick. Assessing the quantity attributes of involvement requires understanding how having multiple customer and/or user constituencies involved in the project outcome can change the complexity of the project. For example, the impact of multiple customers and user constituencies’ on decision making, specifically the ability to make decisions correctly or on a timely basis, will influence how a project needs to be run. Multiple constituencies complicate the ability to make decisions which drives the need for structure. As the number of groups increases, the number of communication nodes increases, making it more difficult to get enough people involved in a timely manner. Although checklists are used to facilitate the model, model users should remember that knowledge of the project and project management is needed to use the model effectively. Users of the model should not see the lists of attributes and believe that this model can be used merely as a check-the-box method.
The methodical assessment of the quantity and quality of customer involvement requires determining the leading indicators of success. Professional experience suggests a standard set of predictors for customer involvement which are incorporated into the appraisal questions below.
These predictors are as follows:
1. Agile methods will be used y/n
2. The customer will be available more than 80% of the time y/n
3. User/customer will be co-located with the project team y/n
4. Project has a single primary customer y/n
5. The customer has adequate business knowledge y/n
6. The customer has knowledge of how development projects work y/n
7. Correct business decision makers are available y/n
8. Team members have a high level of interpersonal skills y/n
9. Process coaches are available y/n
The assessment process simplifies the evaluation process by using a simple yes-no evaluation. Gray areas like ‘maybe’ are evaluated as an equivalent to a ‘no’. While the rating scale is simple the discussion to get to a yes-no decision is typically far less simple.
Agile methods will be used: The first component in the evaluation is to determine whether the project intends to use disciplined agile methods for the project being evaluated. The term ‘disciplined’ is used on purpose. Agile methods like xP are a set of practices that interact to create development supported by intimate communication. Without the discipline or without critical practices, the communication alone will not suffice. Assessment tip: Using a defined, agile process equates to a ‘Y’, making it up as you go equates to an ‘N’.
Customer availability (>80%): Intense customer interaction is required to ensure effective development and to reduce reliance on classically documented traceability. Availability is defined as the total amount of time the primary customer is available. If customers are not available, lack of interaction is foregone conclusion. I have found that agile methods (which require intense communication) tend to loose traction when customer availability drops below 80%. Assessment Tip: Assess this attribute as a ‘Y’ if primary customer availability is above 80%. Assess it as an ‘N’ if customer availability is below 80% (which means if your customers are not around 80% of the time normally during the project without very special circumstances rate this as a No).
Co-located customer / user: Co-location is an intimate implementation scenario of customer/user availability. The intimacy that co-location provides can be leveraged as a replacement for documentation-based communication by using less formal techniques like white boards and sticky notes. Assessment Tip: Stand up look around, if you don’t have a high probability of seeing your primary customer (unless it is lunch time), you should rate this attribute as an ‘N’. Leveraging metaverse tools (e.g. Secondlife or similar) can be used to mitigate some of the problems of disparate physical location.
Project Has A Single Customer: As the number of primary customers increase, the number of communication paths required for creating and deploying the project increases exponentially. The impact that the number of customers has on communication is not a linear, it can be more easily conceived as a web. Each node in the web will require attention (attention = communication) to coordinate activities. Assessment Tip: Count the number of primary customers, if you need more than one finger, assess this question as an ‘N’.
Business Knowledge: The quality and quantity of business knowledge the team has to draw upon is inversely related to the amount of documentation-based communication needed. Availability of solid business knowledge impacts the amount of background that needs to be documented in order to establish the team’s bona fides. It should be noted that it can be argued that sourcing long term business knowledge in human repositories is a risk. Assessment Tip: Assessing the quality and quantity of business knowledge will require introspection and fairly brutal honesty, but do not sell the team or yourself short.
Knowledge of How Development Projects Work: All team members, whether they are filling a hardcore IT role or the most ancillary user role, need to understand both their project responsibilities and how they will contribute to the project. The more intrinsically participants understand their roles and responsibilities the smaller the amount of wasted effort a project will typically have to expend on non-value added activities (like re-explaining how work is done). Assessment Tip: This is an area that can be addressed after assessment through training. If team members can not be trained or educated as to their role, appraise this attribute as an ‘N’.
Decisions Makers: The project attribute that defines“decision makers” is the process that leads to the selection of a course of action. Most IT projects have a core set of business customers that are the decision makers for requirements and business direction. Knowing who can make a decision (and have it stick) then having access to them is critical. Having a set of customers available or co-located is not effective if they are not decision makers (‘the right people’). The perfect customer for a development project is available, co-located and can make decisions that stick (and very apt not to be the person provided). Assessment Tip: This area is another that can only be answered after soul searching introspection (i.e. thinking about it over a beer). If your customer has to check with a nebulous puppet master before making critical decisions then assessment response should be an “N”.
High Level of Interpersonal Skills: All team members must be able to interact together and perform as a team. Insular or other behavior that is not team conducive will cause communications to pool and stagnate as team members either avoid the non-team player or the offending party holds on to information at inopportune times. Non-team behavior within a team is bad regardless of the development methodology being used. Assessment Tip: Teams that have worked together and crafted a good working relationship typically can answer this as a “Y”.
Facilitation: Projects perform more consistently with coaching (and seem to deliver better solutions), however coaching as a process has not been universally adopted. The role that has been universally embraced is project manager (PM). Coaches and project managers typically play two very different roles. The PM role has an external focus and acts as the voice of the process, while the role of coach has an internal focus and acts the as the voice of the team (outside vs. inside, process vs. people). Agile methods implement the role of coach and PM as two very different roles, even though they can co-exist. Coaches nurture the personnel on the project; helping them to do their best (remember your last coach). Shouldn’t the same facility be leveraged on all projects? Assessment Tip: Evaluate whether a coach is assigned if yes answer affirmatively. If the role is not formally recognized within the group or organization, care should be taken, even if a coach is appointed.
In preparation for scaling total the number of Y’s and N’s. Scaling will be discussed after all components are described.
