Check Mark

Check!

Understanding customer satisfaction is important. It might be more important for a product or services sold to someone outside the firm because of the link between satisfaction and sales —  no customers = no revenue. Understanding customer satisfaction for internal IT groups, groups that support the value chain, are often given short shrift. However, careers and budgets are influenced by satisfaction. Making sure you have the right approach and logistics for measuring internal customer satisfaction is critical for being successful. The questions described in earlier blog entries in this theme (see below) can be used as a really simple checklist. Review the questions below and answer them with either: yes, no, or no clue.  (more…)

Soap, Shampoo, Towel and Rubber Duckie

Everything you need for a proper bath!

The fourth category of considerations for an organization that is primarily focused on internal applications to think about before they start measuring customer satisfaction is self-sufficiency. However, before we start, after the first article in this theme, I was asked whether the overhead of the four considerations would put teams and individuals off from talking to their clients, customers, and stakeholders. The simple answer is no. Conversations with individuals about their satisfaction with your efforts are important feedback tools. Sprint Reviews and Demos are events that are structured to create those conversations.  Conversations and formally measuring customer satisfaction are not the same thing. Neither should preclude or interfere with the other but rather doing both will provide different types of information. If you are not talking with your stakeholders, I will put it more succinctly: you probably have a career issue that measurement will not fix.   (more…)

 

Rock piled by the shoreMost internal IT organizations do not have a lot of experience as professional customer research personnel, but they have to get a handle on how their work is perceived. Before tackling the collection and analysis of how customers and clients perceive their work there are four considerations, we take a deeper dive into three today. (more…)

One Way Stop Sign

Measuring or assessing customer satisfaction is a fact of life for products and services for organizations that deliver to their customers. I receive several every day. Each text, email and phone call asking my opinion tells me that my opinion matters. The process of determining whether customers are happy is a form of attention. Internal customers are not always paid the same compliment, this is a rectifiable mistake. There are multiple ways to collect customer satisfaction data (a sample of techniques are in Customer Satisfaction Metrics and Quality) the next four segments of the blog are not going to focus on data collection techniques, but rather on internal customer satisfaction measurement rationale and infrastructure. Spending time upfront to understand whether what you are doing solves a problem or is a sustainable process is important. None of this is easy and doubly so because most collecting and analyzing the data aren’t marketing or market research personnel. There are four areas that need to consider before you send your first survey or schedule your first stakeholder interview. (more…)

On a scale of fist to five, I’m at a ten.

(This is lightly re-edited version of a post from 2016 — I have been on planes for two days going hither and yon, therefore, we are revisiting quality.)

Quality is partly about the number of defects delivered in a piece of software and partly about how the stakeholders and customers experience the software.  Experience is typically measured as customer satisfaction. Customer satisfaction is a measure of how products and services supplied by a company meet or surpass customer expectations. Customer satisfaction is impacted by all three aspects of software quality: functional (what the software does), structural (whether the software meets standards) and process (how the code was built). (more…)

On a scale of fist to five, I'm at a ten.

On a scale of fist to five, I’m at a ten.

Quality is partly about the number defects delivered in a piece of software and partly about how the stakeholders and customers experience the software.  Experience is typically measured as customer satisfaction. Customer satisfaction is a measure of how products and services supplied by a company meet or surpass customer expectations. Customer satisfaction is impacted by all three aspects of software quality: functional (what the software does), structural (whether the software meets standards) and process (how the code was built).

Surveys can be used to collect customer- and team-level data.  Satisfaction is used to measure if products, services, behaviors or work environment meet expectations.  (more…)

How do you calculate value?

How do you calculate value?

IT value is an outcome that can be expressed as a transaction; a summation of debits and credits resulting in a number.   Unfortunately, even if we can create a specific formula, the interpretation of the number is problematic.   Measures of the economy of inputs, the efficiency of the transformations and the effectiveness of specific outputs are components of a value equation, but they only go so far.   I would like to suggest that customer satisfaction makes interpretation of value possible.

Those that receive the service determine the value, therefore value is specific to the project or service. In order for value to be predictable, you must assume that there is a relationship between how the product or service is created and the value perceived.  When we are assessing the value delivered by an IT department, which is part of a larger organization, it is rare that we are allowed the luxury of being able to declare that the group we are appraising is a black box.  Because we can’t pretend not to care about what happens inside the box or process, we have to find a way to create transparency so that we can understand what is happening. For example, one method is to define the output of the organization or processes.  The output or product can viewed as  a synthesis of inputs, raw materials and process.  The measuring the efficiency of the processes used in the transformation process is a typically measure of value add. The product or output is only valuable if it meets the users need and is fit for use. Knowing the details of the transformation process provides us with the knowledge needed to make changes. While this sounds complex, every small business has had to surmount this complexity to stay in businesses.  A simple value example from the point of view of a restaurant owner follows.

  • A customer enters the restaurant and orders a medium-rare New York Strip steak (price $32.00)
  • The kitchen retrieves the steak from the cooler and cooks the steak so that it is done at the proper time and temperature.  (The inputs include requirement for the steak, effort of waiter and kitchen staff.)
  • The customer receives a medium-rare New York Strip steak

From the restaurant owner’s point of view the value equation begins as the price of the steak minus the cost of steak, preparation, and overhead.   If the cost of steak and servicing the customer was more than the price charged, an accounting loss would have resulted and if the costs were less  . . .  an accounting profit.  The simple calculation of profit and loss provides an important marker in understanding value, but it is not sufficient.   For example, let’s say the customer was the restaurant reviewer for a large local newspaper and the owner comp’ed the meal AND the reviewer was happy with the meal.  The owner would derive value from the transaction regardless of the accounting loss from that single transaction.  As I noted earlier, customer satisfaction is a filter that allows us to interpret the transaction.   Using our example, if the customer was unhappy with his or her steak the value derived by the restaurant will be less than totaling of the accounting debits and credits would predict.  While a large IT department has many inputs and outputs, I believe the example presents a path for addressing value without getting lost in the complexity of technology.

In a perfect world, IT value would be established in a perfect market place.  Customers would weigh the economy, efficiency, effectiveness and customer satisfaction they perceive they would garner from a specific development team to decide who should do their work. If team A down the could do the work for less money and higher quality or deliver it sooner, they would get the work.   Unfortunately, perfect market places seldom exist and participants could easily leverage pricing strategies that internal organizations would not be able to match.  The idea of a project level market place has merit and benchmarking projects is a means of injecting external pressure that helps focus teams on customer satisfaction.

Measuring IT value, whether at a macro or project level, needs to be approached as more than a simple assessment of the processes that convert inputs into products or services that the business requires.  Measure the inputs and raw materials, measure and appraise the processes used in transformation and then determine the user’s perception of the output (where the customer and user are different you need to understand both points of view).  Knowing the value of all of these components while having your thumb on the filter of customer satisfaction will put you in a position to not only determine the value you are delivering (at least over the short-term), but to predict how your customers are perceiving the value you are delivering.  Remember forewarned is forearmed.

Would you be satisfied if you paid for five cabbages, but got only four?

Would you be satisfied if you paid for five cabbages, but got only four?

Estimates and customer satisfaction are intertwined because they set expectations. Estimates set the development team’s expectations about resources that will be available to deliver the project, including effort, people, calendar time.  From the customer’s point of view, an estimate will generate expectations about what will be delivered, when it will be delivered and what the project will cost.  The dark side of estimation is that once an estimate is even whispered, it quickly become set in stone and gains the power to dissatisfy. 

A project estimate is an outline of the resources that the team or estimator needs to deliver a given scope of work.  Regardless of the ultimate veracity, most organizations require an estimate for a project to secure funding and team resources.  Estimates create parameters that the team(s) will operate within.  Missing an estimate, whether due to lack of information, increased scope or changing business decisions, will impact other projects.  This ripple effect impacts other projects as overall portfolio is adjusted to deal with changing realities. This leaves very few within the IT department satisfied. 

IT customers and stakeholders have been trained that they need to ask how much a project will cost before they begin and many have been trained that answer is at best a guess.  However, it is a guess to which they will hold the team.  This leads teams to pad the estimate in a vain attempt to not to have break the news that something that was not anticipated (and probably could not be anticipated) has caused the estimate to be wrong.  The project estimate, which is typically generated at the point in a project where the team knows the least about what will be needed to deliver the customer’s vision, generates a substantial anchor bias.  The anchor puts teams and their customers in an almost immediate customer satisfaction hole that is difficult to remediate. 

Most customers and stakeholders have purchased large ticket items, like a car, and paid the negotiated price.  A significant proportion have purchased products that either had to be customized for them, including clothing (suits or prom dresses) or houses.  Rarely do the prices vary significantly from the negotiated price. But, those of us in the business would challenge the relationship between these examples and developing, enhancing or supporting software. However most of customers and stakeholders would not be impressed with the distinction.  When pressed they will be happy to tell you that they can find a supplier to meet their needs for a fixed price (and someone will always say yes).

Why should project teams care about customer satisfaction?  Simply put:

  • Customers supply the budget or pay the bills.
  • Selling to happy customers costs far less than finding new customers.
  • In the long run ALL customers can change suppliers (internal or external).
  • Happy customers promote your team or organization.
  • Satisfaction leads to less in-fighting and better relationships.

In the end, unless we can change the estimation landscape, we will need to get better at estimation if we want to keep our customers satisfied. In part this means that we need to include more people in the process and to hone our ability to predict the future.  

Measurement is no longer optional

Measurement is no longer optional

Measurement is a topic that most IT practitioners would rather avoid discussing. Many practitioners feel that it not possible to measure software, or that all that matters is whether the customer is satisfied. IT managers tend to have a point of view that incorporates customer satisfaction and at least a notional view of how efficiently they are spending their budget. When managers or other leaders do discuss the topic of measurement, their arguments for measurement tend to begin intellectually.  Arguments begin with statements like, “we need to measure to ensure meet a model like the CMMI,” or “we need to measure to build knowledge so that we can estimate.” While these are good reasons to measure, many measurement programs are being driven by more basic and powerful need: the need to demonstrate efficiency. The demand for IT continues to explode in every organization I speak with. The problem is that IT, whether developing, enhancing or maintaining software, is expensive. Couple that expense with the cost to acquire and maintain hardware and the budgets of some IT organizations become larger than moderately industrialized countries, and are growing. The pressure that growing IT budgets create on the bottom line means that efficiency can no longer be a dirty word in IT organizations. If efficiency is, or is about to become, important then organizational measurement can’t be far away.

Much of the effort in the development field over the past ten to twelve years has been focused on effectiveness and customer satisfaction, as evidenced by the Agile movement. Efficiency has begun to creep back into the conversation under the auspices of Lean. Measurement programs focused on customer satisfaction now must be refitted to discuss time-to-market (how much time is needed get a unit of work to market), productivity (how much effort is needed to get a unit of work to market), cost efficiency (how much does a unit of work cost to come to market) and quality per unit of work. All of these metrics and the measures are based on need to be comparable and combinable across the whole of the IT organization.

A roadmap to develop measurement program can be as straightforward as: Defining Goals and Values

  • Developing Common Measures
  • Mapping the Linkage Between Goals and Common Measures
  • Identifying Measures and Metrics (Including Gap Analysis)
  • Validating Metrics to Needs, Goals and Values
  • Developing Metrics Definitions
  • Mapping Metrics Data Needs
  • Defining an Overall Dashboard

In 2014, many IT organization budgets are beginning to recover and grow; however, because of the huge backlog in demand for IT services and products the need for IT department to efficiently use their budgets is not going to change. What is going to change is the need for individual teams to solve their own measurement quandary, because all IT work needs to be combined, compared and evaluated. Solid measurement programs have to balance efficiency, effectiveness, quality and customer satisfaction.  The process starts with understanding the goals of the organization and then ensuring what gets measured and demonstrate progress against those goals.

We should be getting back to normal on the blog starting now.  As many of you know I have working on a project management book with Murali Chemuturi which has now gone to the publisher.  While I know there is more work on it in the future I intend to start catching up starting now!

IT Value and Customer Satisfaction
Thomas M. Cagley Jr

IT value is an outcome that can be expressed as a transaction; a summation of debits and credits resulting in a number.   Unfortunately, even if we can create a specific formula, interpretation of the number is problematic.   Measures of the economy of inputs, the efficiency of the transformations and the effectiveness of specific outputs are components of a value equation but they only go so far.   I would like to suggest that customer satisfaction makes interpretation of value possible.

I believe that value is perceived by those receiving the service therefore tends to be specific to the project or service, to be predictable there must be an assumption that there is a relationship between how the product or service is created and the value perceived.  When we are assessing the value delivered by a department like Information Technology (IT) which is part of a larger organization, it is rare that we are allowed the luxury of being able to declare that the group we are appraising is a black box which means we have to create transparency.  Because we have to create transparency into the appraised organization we have to define value as a synthesis of inputs and raw materials, the efficiency of the processes used in the transformation process (where value is typically added) and finally whether the  output meets the users need and is fit for use.  While this sounds complex, every small business has had to surmount the complexity to stay in businesses.  A simple value example from the point of view of a restaurant owner:

  • A customer enters the restaurant and orders a medium-rare New York Strip steak (price $32.00)
  • The kitchen retrieves the steak from the cooler and cooks the steak so that it is done at the proper time and temperature.  (The inputs include requirement for the steak, effort of waiter and kitchen staff.)
  • The customer receives a medium-rare New York Strip steak

From the restaurant owner’s point of view the value equation begins as the price of the steak minus the cost of steak, preparation, overhead and the free tap water.   If the cost of steak and servicing the customer was more than the price charged, an accounting loss would have resulted and if the costs were less  . . .  an accounting profit.  The simple calculation of profit and loss provides an important marker in understanding value but it is not sufficient.   For example, let’s say the customer was the restaurant reviewer for a large local newspaper and the owner comp’ed the meal AND the reviewer was happy with the meal.  The owner would derive value from the transaction regardless of the accounting loss from that single transaction.  As I noted earlier, customer satisfaction is a filter that allows us to interpret the transaction.   Using our example, if the customer was unhappy with his or her steak the value derived by the restaurant will be less than totaling of the accounting debits and credits would predict.  While a large IT department has many inputs and outputs I believe the example presents a path for address value without getting lost in the complexity of technology.

In a perfect world, IT value would be established in a perfect market place.  Customers would weigh the economy, efficiency, effectiveness and customer satisfaction they perceive they would garner to decide who should do their work.  Unfortunately, perfect market places seldom exist and participants could easily leverage pricing strategies that internal organizations would not be able to match.  The idea however has merit and benchmarking projects is a means of injecting external pressure without the potential for pricing abuse.

Measuring IT Value whether at a macro or project level needs to be approached as more than a simple assessment of the processes that convert inputs into a product or service the business requires.  Measure the inputs and raw materials, measure and appraise the processes used in transformation and then determine the user’s perception of the output (where the customer and user are different you need to understand both points of view).  Knowing the value of all of these components while having your thumb on the filter of customer satisfaction will put you in a position to not only determine the value you are perceived to be delivering (at least over the short-term) but to predict how your customers are perceiving the value you are delivering.  Remember forewarned is forearmed.