The streets of Bangalore

The streets of Bangalore

Why do organizations outsource work? It certainly is not to eviscerate the workforce of the company; it is to get more with less. More functionality, more productivity, more quality, more profit and more market share.  The first group of myths cover the “Theory of More.” The Theory of More is built on the expectation that more functionality will be delivered faster and with higher quality.  Central to most of the myths in this section is: How do you know you are getting the more you wanted?

Myth:  “We only have one goal in mind for outsourcing and that is all we need to measure”.  Organizations occasionally get fixated on a single result and measure this result to the exclusion of all others.  This pattern almost always generates irrational results. For example, a major outsourcer noted that the easiest way to increase productivity was to reduce the work that they allowed to enter the IT shop they had acquired (the relationship between the amount of work entering the shop and productivity is statistically measurable and predictable). The contract was focused on productivity, but the outsourcee certainly did not expect to get less functionality delivered when the contract was signed. By accepting less work the developers are able to focus on one thing at a time rather than multitasking yielding higher productivity (effort per unit of work).  A better approach is to leverage measurement techniques like the balanced scorecard.  A single metric can’t define value without being prone to gaming (intentional or unintentional).  The use of any scorecard requires deft selection that enhances the behavior which both partners anticipate and wish to promote.

Myth:  “Productivity of outsourced work is higher than internal projects.” Changing productivity is a common goal of sourcing decisions.  This myth is a variant of the infamous “the grass is always greener…” truism. The classic definition of the productivity equation is the amount of output created per unit of input; a software example would be the number of function points per month of effort.  Data from multiple consulting organizations indicate that outsourcing productivity is lower but the cost basis substantially lower.  The second definition (false definition) substitutes monetary value for physical units of output, or the dollar value of software created per effort month.  Reducing the cost of software would increase the net value.

Myth:  Deciding on an outsourcer based on  a specific CMMI® maturity ratings because it is believed that  a higher maturity level equates to higher productivity. When productivity is measured as units of output per unit of work, there have been numerous studies of organizations assessed at CMMI Maturity Level 5 assessments that show lower productivity than comparable non-CMMI Level 5 organizations.  This strongly suggests that the relationship is not direct or that other factors might be effected by the attaining the high levels of discipline required to achieve CMMI Level 5.

Myth:  “The reduction in labor costs created by off-shoring is all that matters when developing a cost profile.”  This myth has been debunked in many articles.  Labor costs are only a component of the costs that should be considered.  Communication and travel costs must be layered into any sourcing equation.  Other areas that affect the bottom line costs include differences in quality (cost impact can go either way), oversight costs (add-on cost),  infrastructure costs (add-on cost) and the cost of lower productivity.  If the sourcing equation contained labor as the only component, there would be little or no discussion.

Myth: “The quality of software delivered by outsourcers is higher than internal projects.”  Unlike productivity, where the answer depends on the definition used, the data on quality collected in project assessment does not bear this myth out when quality is defined as defects delivered.  While there are many anecdotal stories of higher quality when quality is measured by external consultancies, the data is lower than equivalent internal projects.  One explanation of the differences between the anecdotes and measured data (other than ascribing it to salesmen) is that troubled projects tend to be measured (only a slight overstatement).  Another is that the measures of quality and satisfaction have been intertwined.

Myth: “We can agree on a contract now and then agree on how to measure it later.”  The success criteria for the sourcing decisions need to specifically documented, agreed upon and measured.  Success criteria typically are complex and have more than one single driver.  Understanding the relationship between individual success criteria is mandatory.  The interrelationship can cloud the impact of a sourcing decision.  An example is the inter-relationship between work entry and productivity.  Slowing the entry of work to the team or “shop” will cause an immediate increase in productivity.  You need to ask yourself whether your success criterion elicits the right response for your needs and provides predicative data.  To quote a dear friend of mine, Gail Flaherty, “How will they know?”  Contractually, any sourcing agreement must  address how everyone will know whether or not the goals are being addressed up front.

Myth:  “Measure what is easily seen so it won’t add significantly to the overhead of the contract.”  Tracking success or performance should not be limited to easily quantifiable data.  Based on your goals for sourcing work, determine the factors that predict success.  For example aren’t behavioral components such as morale, experience and turnover typically significant predictors of success which can provide explanative information for performance?  These are hard to measure but important. CMMI Ratings and quantitative data are merely proxies for behaviors which explain how people actually work.

Myth:  “Measure everything so we can control the vendor and see problems before they happen.” Measurement is like chocolate; there is a right amount that satisfies (and may actually have some health benefits) and an amount that will weigh you down.  Finding the balance that allows you to monitor and control your sourcing arrangement without giving away the benefits is still an art.  It should be pointed out that if you have made the effort to negotiate and stipulate success criterion in the contract, you must take the time to measure.  Just don’t overdo it!

Myth:  “Productivity and knowledge capital of the firm at large will be unaffected by sourcing work outside the organization.” This is a myth for no other reason than no one really knows if it is true or not.  Productivity has been the backbone of the U.S. economy since the mid ‘90s.  Three major components drive productivity: capital deepening, knowledge and multifactor productivity (MFP).  MFP reflects the joint effects of many factors including research and development (R&D), new technologies, economies of scale, managerial skill and changes in the organization of production.   Contracts must  contain methods for capturing and returning knowledge capital so that it can be used to continue to fuel the productivity engine.

Myth:  “Choice fuels competition, innovation and efficiency.” There are many assumptions built in: perfect knowledge, free competition or low barriers to entry and exit — to name a few.  The degree to which these are true influences the benefits derived from choice (or at least the risk of making a choice).  Organizations that have outsourced all or part of their IT organization need to understand and identify the benefits and risks of their choice (in a perfect world they would lock in the benefits and mitigate the risks) early in the process due to the barriers that contracts and dislocation of jobs entail.  Understanding how your contractual decisions impact basic economic tenants can’t be ignored.

A corollary to this myth is “More choices yield a better decision.”   Too many choices can create a scenario where analysis paralysis occurs or, equally as evil, a fractured non-optimal answer.  This type of crisis creates a condition where it becomes more and more difficult to make a decision because of the fear of losing out on something better.

Untitled1Outsourcing has been used as a tool for organizations to manage many factors ranging from cost containment to capacity management.  The use of sourcing as both a tactical and strategic tool is not a short-term phenomenon.  Time and the accumulated experiences of many have lead to the accumulated lore based on opinions, facts and urban myths.

Outsourcing is a double-edged sword.  Even though sourcing has become a political hot potato, it is a tool for managing demand, a balance sheet or as a last resort a means to make short-term numbers The topic has it share of proselytizers.  Regardless of why an organization decides to consider OS, they need to go into the process with their eyes open.  In this highly charged arena, it has become difficult to determine what is true or merely opinion (and more importantly whether knowing the difference matters).

Most of the myths that have evolved around sourcing options are constructed upon individual grains of truth.  Unfortunately, many grains of truth have been wrapped in hysteria or marketing spin, swelling them to many times their original size.  Sifting through this morass by applying the correct level of due diligence is important when deciding on how to source your IT activities.  Failure to perform the necessary due diligence is a failure of fiduciary responsibility.  Five meta-categories of myths will be used to isolate and expose the myths of outsourcing:

  • Theory of More – Sourcing options are a tool to get higher productivity, faster time-to-market, better quality or some combination of the three.
  • Laws of Determinism – Higher levels of process or discipline will lead inevitably to a specific outcome of sourcing decisions.
  • Impact of External Pressures – Business pressures spawn specific types of sourcing decisions and can be addressed either through strategic or tactical sourcing decisions.
  • People As An Asset? (Can I Sell Them?) – People and organizations are highly intertwined in  both the sourcing decision process and chances for success.
  • Been There, Done That! – Each sourcing scenario is based on a set of beliefs and a willingness to address risk however each is very different.

Understanding the myths within each high level category and will provide you with a means to winnow through the chaff into a set of actionable truths.

Human Interaction During Testing on Two Continents
Thomas M. Cagley Jr.

Audio Version on SPaMCAST 175 

Testing is the means of proving that the development process has understood and built what was required.  In its purist form it provides a set of proofs that validate the ‘whole’ development life cycle.  Regardless of the test model used there is one overriding goal; to deliver the best product possible within the constraints of time, budget and scope.  Meeting the goal of testing requires a strong level of interaction and communication between the testing and development teams.  This requirement becomes an imperative when testing occurs on two continents.

 

One day I woke up to a gorgeous sunrise then the phone rang and I inherited the testing component of a development project.  The project was well along, with development nearly complete with the external system and user testing looming.  Development and testing were to be done on two continents.  In a nutshell, the project was troubled (the usual cast of characters were at work; over budget, late, feuding sub-teams and subject to runaway requirements).  Reviewing how the testing had been distributed had to be reviewed to determine whether the most efficient use of the project’s limited remaining resources was occuring.  Of course this begged the question whether resources could be redistributed.  As soon as I heard the words; project, testing and soon, a mental assessment checklist popped to mind:

 

  • Had test planning been performed and reviewed (and by whom)?
  • Could the test cases actually prove the requirements?
  • Had the developers been a party to the creation of these cases?
  • Had user acceptance tests been defined when the stories were created?
  • What were the criteria for acceptance (functionally, quality and date)?
  • When was the last time that the test managers (and/or the project manager) actually talked, rather than emailed each other?
  • What was driving the current project delivery date?
  • Could I impact how the work had been distributed (sources and teams)?
  • Was there a phone list of project participants?
  • What time was it inIndia?

 

Testing is a human activity built on communication and interaction both with the project deliverables and the participants.  My first stop was the world clock at (www.timeanddate.com/worldclock) and then the phone list.  The test plan was my third stop.  If done correctly it would provide the basis to synchronize the test teams.  The test plan is a core management communications and negotiation tool.  The plan compiles and communicates the information about testing activities to provide a common way for all involved parties to understand and track test activity.  The plan lines up resources so that even a manager can know what will be done, when it will be done and who will do it.  The critical components of a test plan to drive multiple testing groups on two continents are:

 

  • An overall statement defining the minimum level of testing that will be acceptable.
  • Test ownership
  • The application components and subcomponents with their relationships
  • A test schedule (with effort, duration and resources)
  • The test metrics
  • The acceptance criteria
  • The communication plans

 

Defining the minimum level of testing for the project creates a baseline, a tool to define what the quality goal of the project really is and then to resist cutting testing below a level that would meet the goal.

 

In any project it is important to determine the types of tests to be done.  Examples include regression testing, usability testing, system testing to name a few; however, when testing is done with more than one team it is imperative to determine who will be responsible for each test.  A single point of contact is (or at worst a single team) must be identified to lead and be responsible for a specific test.  Note, this does not mean that only one team can be involved just that someone needs to be in charge.  Determining ownership is best driven by its relationship to the development lifecycle, unit testing is best owned by the development team. Understanding the relationship between types of tests and the development lifecycle is an important piece of knowledge in the ownership decision process.  Oversight of the process should always be driven by the organization that sourced the project.  This is critical to making sure communication and information flow can be managed.

 

Components and the schedules are related items.  Knowing the components and subcomponents of the application being built or modified and their relationships provides the knowledge needed to create a road map to plan testing and for building a schedule for actually performing the test.

 

Metrics provide a means to monitor testing while in progress, rather than simply relying on verbal status.  Verbal status, while important, can easily be misinterpreted or represent progress that is out of date.  Metrics provide information and knowledge that cuts across any cultural boundaries.  The metrics should describe software functionality tested, degree of code coverage and progress against a schedule which are core management issues.  Publishing test metrics provides a language to synchronize not only the test teams but the project team, as a whole regardless of organization or culture.

 

Acceptance criteria serves two purposes.  First, acceptance criteria define when a project is complete.  Second, if created and documented early in a project (before coding to take a page from the agile methodologies) they serve as a communication tool for validating requirements.  The definition of acceptance criteria must be led by the organization owning the project and ‘users’, all parties should participate but ownership stays at home.  Acceptance criteria provide a language of acceptance that cuts across communication barriers.

 

Providing a plan for the compilation and communication of complete information about the plans, progress and problems for testing allows both continents to provide support and anticipate changes in their own schedule. Interaction, communication and knowledge are important components to making testing happen and becomes even more important when more than one team is involved or when testing done on two continents.  With adequate information, testers are better able to plan and perform their tasks more efficiently and to get an accurate picture of the system. Did I mention communication was important and that it must happen over the entire project life cycle?

 

Testing is frequently unpredictable, and unpredictability, the bane of managers, creates painful schedule surprises and Maalox moments. This is especially true when multiple teams in multiple locations with multiple cultures are involved.  Communication and coordination based on planning that combines all of the different perspectives of testing and system development  (schedules, functionality, code and problem resolution) makes it is possible to understand and manage software testing.  Defining how to source testing is important to all outsourced projects, the test plan or test architecture is a decision making tool and a tool to synchronize activities between sources.  Communication based on a plan lets you manage the testing process on one continent or two rather than to allowing  it to manage you.

Outsourcing:  Metrics and Governance
Thomas M Cagley Jr.

Successful outsourcing arrangements require that all parties pay continuous attention to the goals of the arrangement.  The phrase ‘continuous attention’ is nearly an oxymoron given differing corporate cultures, short attention spans and competing channels of information.  Outsourcing contracts often contain one or more tools intended to focus attention (typically as a portion of the governance convents). The tools currently in vogue include metrics, service level agreements, balanced scorecards and the prescription of CMMI®[1] maturity levels.   In many cases these tools are used as weapons to interrogate and control the parties in the contract.

The goal of governance is to manage and reduce risk. A better practice is to construct governance covenants so that they build and strengthen the capabilities of both parties. CMMI, metrics and scorecards done correctly can fall in this category while at the same time mitigating risk. The choice of tool and the construct of the governance convents are as critical as the financial portions of the contract. All too often, the development of the governance processes is by contract personnel without a through understanding of process or metrics. The lack of process or metrics knowledge is exacerbated when the idea is championed by parties on one side of the table or other (Note: Someone has to champion the concept or most outsourcers will run, not walk, away from the idea.  Each side of the negotiation needs to have a metrics counsel.). These scenarios lead to agreements governed by convents that incent unbalanced behavior, that do not add capabilities or strengthen the relationships between parties to the contract.  Examples can be found across a range from contracts which pay penalties and bonuses on a single metrics (typically indexed productivity or time-to-market) to contracts with scores of embedded metrics which sap time and effort from delivering value. Soliciting input from and the involvement of your organizations process and metrics personnel (or consultants if you do not have these capabilities) is a best practice that does not find its way into many contract negotiations.

There are many techniques that can be used to facilitate developing a balanced approach to using metrics in the governance process. Goal driven metrics techniques are processes for developing useful metrics for the core of contract governance.  A few of the techniques that have been found useful include:

  1. Goal, Question, Metric (GQM)
  2. Practical Software Measurement (PSM)
  3. Goal-Question-Indicator-Measurement (GQIM)

 

A basic tenant of all of these techniques is to begin with an understanding of your business goals. A through and honest appraisal of the reasons why you are outsourcing will allow you to build a set of measures that have a chance at allowing you to know whether those goals have been met. The set of measures will also provide a filter to discriminate between what data will provide information and which will provide extraneous noise (this requires a careful and thoughtful understanding of data analysis).  Without the use of goal driven approach it is possible that the contract will incent behavior that does not accurately reflect the goals, targets and objectives of executive management (an old adage points out that you get what you measure).

As noted earlier the two worst-case scenarios typically are a focus on a single goal (cost reduction, increased productivity or better quality) or the development of scorecards with too many contributors (if you need an econometric model to understand the impact of metric you certainly have gone too far). Kaplan and Norton’s work on the Balanced Scorecard has certainly driven the point home that a single view of performance is at best not very useful and more than likely dangerous.  At the same time balanced and overly complex do not have to be synonymous concepts.

Dashboards and scorecards are excellent summarization techniques however they are not adequate to develop knowledge or to direct action. A granular level of detail will provide organizations with the facility to have event-centric and experience-based interaction with data in order to generate actionable insights rather than to rely purely on personnel interactions. Granular, query-able data will allow you to develop mechanisms to defeat the tyranny of average, which looses the ability to identify special cases (both good and bad).

Continuous attention does not have to be an oxymoron.  It does mean all parties in an outsourcing arrangement must decide on what is important when the contract is being framed (and have the guts to change when the business dictates). This process provides a basis from which to develop a well-grounded metrics program that will let you know if you are attaining your goals while increasing capability and value.


[1] Capability Maturity Model Integration® and CMMI® are registered trademarks of Carnegie Mellon University.

Attention As An Asset In Outsourcing
SPaMCAST Version

Thomas M. Cagley Jr.

Organizations manage many assets; some tangible such as people, buildings and hardware and their processes, and some that are intangible such as knowledge, goodwill and attention. The process of outsourcing transfers many tangible and intangible assets from one organization to another on a contractual basis. Outsourcing, however does not transfer the need for attention to paid to transferred assets by everyone in the equation. This is unlike the direct sale of an asset, and in practice it is easy to loose track of attention. Outsourcing agreements require the development of mechanisms that provide focused, constant, balanced attention to the process and transferred assets. Without this type of attention the gains that were promised in the deal can melt away to be replaced with higher costs, lower quality and miserable satisfaction levels (for everyone involved).

Many outsourcing agreements recognize the need to manage attention through the inclusion of many techniques, such as metrics, service level agreements and stipulations for benchmarking against process standards, such as the SEI Capability Maturity Model Integrated (CMMI). These tools, at least initially, provide the focus and attention they are designed to address. Entropy becomes the worst enemy of attention-focusing devices during the life of an outsourcing agreement.

Based on my observations, numerous outsourcing agreements problems begin in a state of hyper-focus. Everyone is interested in how the new agreement is working and evolving (the reasons are not always benign). The hyper-focus stage is not sustainable and typically reduces productivity and slows the new organizations ability to bring functionality to market. As the relationship matures and trust is built the mechanisms for providing focus need to evolve.

The monitoring portion of the focus tends to relax (the agreement is no longer the next new thing) and the focus moves to capabilities (compliance should not be ignored, just de-emphasized). This stage of the agreement is far more sustainable; the attention paid to the contract by all parties’ swings back to a more rational level. The pendulum swing needs to settle at this point. The contract needs to provide provisions that allow for refinement of how transparency and insight are going to be managed. Without these types of provisions trouble will occur (I have been amazed at the number of large deals that do not include provisions for orderly evolution). Trouble ranges from the aggressive, contract conflict to the passive-aggressive, flooding the monitoring processes with information. The passive-aggressive pattern is seen more often as conflict begins and is a leading indicator of more serious issues yet to come.

The insidiousness of flooding the system with information is based on the axiom that as information increases so does its demand for attention. Data becomes overwhelming, swamping the ability to create information and drive decisions. Kernels of truth are obscured; the analysis to determine what is actually happening becomes more difficult. In extreme cases overload occurs (additional stress) which can cause both parties to back away from constructive measurement when negotiations are reinitiated on the governance components of the contract. The time before the reinstitution of balanced governance leaves the outsourcee blind to potential impacts to the cost model they have forecasted for the deal. Contracts must include covenants that allow for evolution to be effective over the entire lifecycle.

Attention focusing devices such as benchmarking against the CMMI coupled with a simple set of balanced metrics (including indexed an indexed productivity metric) are typically found in agreements that have evolved to a point where capabilities and capacity are important to both parties. These types of devices help focus attention on areas that are business critical and can be sustained for long periods of time. The ability to sustain focus is important since as soon as focus/attention begins to waver it is difficult to re-establish. They are also positive views that help organizations and individuals preserve and extend their ability to deliver value to both organizations. Observed best practices to address this issue include including personnel that are experienced in outsourcing governance while the contract is being negotiated (not just the sales team) and including measurement professionals as part of the team when developing the contract.

.

Hebert Simon wrote, “What information consumes is rather obvious: it consumes the attentions of its recipients. Hence a wealth of information creates a poverty of attention.”

Nature abhors a vacuum, attention follows a similar pattern. Stop trying to control or maintain attention and it eases away finding other areas to settle in, much to the chagrin of those who understand W. Edwards Demming’s point about constancy of purpose.

Outsourcing agreements typically do not explicitly recognize that attention is one of the assets that are transferred as part of an outsourcing agreement. Inexperienced outsourcees are always surprised that an arrangement actually requires a substantial investment to manage attention in the governance components of a contract and that managing attention evolves over the lifecycle of the contract.

An update version of this essay that will be podcast on Software Process and Measurement Cast 19  (www.spamcast.com)

Stockholm Syndrome and Outsourcing, Updated

Thomas M, Cagley Jr

Managing risk is one of the keys to success in an outsourcing arrangement.  There are many control mechanisms used to manage outsourcing deals.  Control mechanisms can range from full-scale contract offices, PMO’s, metrics, scorecard reporting, audits, CMMI assessments and on-site oversight teams.  In real life, typically these are applied in  combination.

Cultural disconnects are a major contributor to problems in outsourcing that increase as the distance between client and outsourcer grows.  Sounds like a truism, however examples abound even today as organizations fall prey to misunderstandings driven by cultural disconnects.  The misunderstandings that can occur can range for differences in semantics to deep-seated cultural differences.

A tool to manage / minimize this type of disconnect is to co-locate an on-site account management team with the outsourcer.  This type of arrangement provides an avenue to mitigate cultural differences, translate both intent and words and mainly to build trust.  All positives; however darker possibilities exist and as personal observations prove they do happen!

On-site management of outsourced projects provides a number of impressive benefits that other forms of control do not provide.  The first and most important of these is a simple visible presence that reinforces that work is important.  Secondly, an onsite presence can provide a bridge between cultures (both organizational and sociological cultures).  Further, a presence provides a mechanism for translation, and for ironing out semantic differences quickly and efficiently.  Finally, an onsite presence is a basis to build a common history and understanding which yields trust.

A powerful tool with equally powerful drawbacks: what happens when an onsite lead or team looses perspective?  I observed a team where a loss of perspective had occurred.  I participated in an assessment of team that supports a group of outsourced applications.  During initial discussions it was impossible to determine who worked for the outsourcer and who worked for the onsite account management.  Collaboration you might ask?  True, but only if the arrangement is structured as such and all parties perceive it that way which it was not the case.  The on-site team had lost perspective and aligned themselves with organization they were overseeing, an application of the ‘Stockholm Syndrome’.

When an onsite team gets too close, they loose perspective, and they begin to believe they are part of the company they supposed to interface with.  When perspective is lost who will they advocate for the project or how will a critical point of translation be interpreted?   Even if the closeness is merely an appearance, it will be difficult for others to understand how to act.

How do the best make co-located teams work?  The best observed application of the techniques begin with the sourcer deploying a cross-functional team.  The skills that are required include project management, business and systems analysis.  The very best include personnel with both facilitation and negotiation skills (negotiation is more typical). All team members require a strong sense loyalty to their company.  Note that if the work is ’offshored‘ (not just outsourced), then the team members must have command of the local language.  The rational for teams rather than an individual is two-fold:  The first is that a team can field more skills.  The second is that a team is far less likely to “go native” than an individual (teams create their own support structure).  Note, using teams is a best practice only if the amount of work supports it.  Smaller outsourcing agreements may not have the luxury, which means they must roll all of these skills into a single individual.

Despites the downside risks, co-locating sourcer and outsourcing teams of any size are a best practice.  How organizations structure their co-location program to keep the personnel fresh and useful is what separates the wheat from the chaff.  Observed tactical best practices to maintain the crispness of on-site teams include:

Ø Rotation of personnel (not everyone at once unless there is only one person) re-enforces the attachment to the parent company.  A secondary form of rotation includes making a spot on the team a step on a job progression.

Ø Leveraging PPQA reviews provides an assessment of whether the outsourcers  processes are being followed.  Non-compliances are identified and an action plan is put in place for remediation.

Ø External audits, using models such as the CMMI, ITIL or ISO Standards, provide a far more formal reading of whether processes are followed (typically with more consequences if they are not).

On-site teams are a best practice for reducing the risk of an outsourcing agreement, but it is a best practice that has a downside unless they are carefully managed.