Editor's note: Howard Waddell is president of Decision Resource, Inc., a Miami, Fla., research firm.

Often, after a customer satisfaction survey has been completed, organizations don't know what to do with the information that has been collected. If a product or service attribute is rated "good," how much worse is it than another rated "excellent"? If your customers wait in line for service for 15 minutes rather than an expected 20 minutes, should you be content when they tell you the wait was "better than expected"?

There are several rating scales that are routinely used for customer satisfaction measurement (CSM). Unfortunately, most have significant weaknesses when used in practical situations.

Performance scales

Performance scales allow customers to rate product or service attributes with choices that might include superior, excellent, good, fair, and poor. With a scale such as this, "good" certainly doesn't look very good at all. Even "excellent" is something less than "superior." What is one to make of data that comes from a scale that suggests that excellent is not good enough?

Experience with performance scales has also shown that many customers tend to be overly generous in their evaluations of products and services. This positive response bias is most likely when customers know they have no anonymity or when they perceive that a well-meaning employee's raise or job may be on the line. When positive response bias exists, average scores tend to be high and problem areas may not be clearly identified. This becomes only too clear when customers switch to a competitor's product soon after reporting that a product was "excellent" in every way.

Satisfaction scales

This scale permits the customer to indicate how satisfied he or she is with the various attributes of a product or service. Questionnaire choices might include very satisfied, somewhat satisfied, neither satisfied nor dissatisfied, somewhat dissatisfied, and very dissatisfied. Like the performance scale, this scale lends itself to positive response bias. Furthermore, if the rating scale has no neutral position, forcing those who are neutral to take a position artificially, customers will, more often than not, take the more positive position, compounding the positive bias.

Expectations scale

One of the major cruise lines apparently borrowed directly from its research reports when it began using the advertising slogan "Exceeding Expectations." If my expectation is that I will get seasick, what is their slogan telling me?

The expectations scale allows customers to evaluate product and service attributes with such phrases as "much better than expected," "better than expected," "about as expected," "worse than expected," and "much worse than expected." The problem is that these responses are simply devoid of meaning unless the expectations are also known. If the expectations are known, then the volume of data that must be digested before meaningful conclusions can be drawn may have increased by a factor of two or more. And when expectations are negative (think airline food), what conclusions can be drawn from a "better than expected" rating? Does it mean the customer is happy? Maybe, maybe not.

The expectations scale can also confuse rather than enlighten when the mix of new to old customers changes. The expectations of these two groups can be very different. The new customer may have expectations that are not realistic. The established customer knows exactly what to expect. Should these two groups legitimately be consolidated? If not, how does one easily compare the responses of the two groups?

The expectations scale presents one other problem. When ratings change over time, it is difficult to know whether performance has changed, expectations have changed, or they both have changed. What is a product manager to do if his or her performance evaluation is in any way driven by customer satisfaction ratings based on this scale?

Requirements scales

There are two different types of requirements scales. In the first, the customer has choices that might include "exceeded my requirements," "met my requirements," "nearly met my requirements," and "failed to meet my requirements." This scale does not work well for attributes such as billing accuracy or product taste that might never exceed customer requirements. Furthermore, this scale is not particularly suitable for evaluating levels of performance that are subject to variability such as employee courtesy or in-stock inventory.

The other requirements scale gives customers choices such as "always meets my requirements," "usually meets my requirements," "occasionally meets my requirements," "rarely meets my requirements," and "never meets my requirements." While this scale is suitable for attributes that are variable, it is doesn't make sense for attributes that are fixed, such as the number of service locations or the size of a gas tank.

A better scale

In addition to the individual weaknesses of the rating scales that have been discussed so far, they all have one common weakness. None of them tell the business manager, "Here's what you need to improve." One can only infer from the customers' answers what corrective action they feel is required. The process of making deductions about the customers' responses can be avoided altogether if the customers are asked in a more straightforward manner, "What do I need to improve?" The answer to this question leaves much less room for interpretation.

While there may be a variety of ways of asking what needs to be improved, the method that has been used by our firm over the past several years is quite simple and has worked very well. It is also a method that has been gaining some popularity in both the business and academic communities.

The method can best be illustrated by example. In a recently completed survey for an electronics parts distributor, customers were asked the following question: "For each of the service factors listed below, please indicate how much improvement, if any, you feel is needed by [company name]." Twenty-eight service attributes were listed on the questionnaire, and for each the choices were "no improvement needed," "some improvement needed," and "considerable improvement needed."

This "improvement needed" scale offers numerous advantages.

  • Most important, the scale elicits responses that do not require interpretation by management. The answers are not ambiguous.
  • It is simple for the customer to understand. He or she is not required to interpret what responses such as "satisfied" and "very satisfied" might mean.
  • It is simple for the customer to use. With three easily understood choices, the customer can move through the questionnaire quickly, avoiding fatigue and irritability.
  • It does not invite charitable responses. In fact, our experience (and the experience of others who use this scale) indicates that it invites a high level of candor. Rare is the customer who says "no improvement needed" to all or even most of the attributes being evaluated. In the recently completed survey for the electronics parts wholesaler, a company whose customers are very loyal, the distribution of combined responses for all attributes listed in the "improvement needed" question was as follows: 57 percent - no improvement needed; 33 percent - some improvement needed; and 10 percent - considerable improvement needed. For one of the attributes, 27 percent of the customers said considerable improvement was needed. There was no mistaking what the message here was.
  • It fits a wide variety of product and service attributes, including those that are subject to variability, those that are not subject to variability, and those for which performance cannot exceed requirements. The one weakness it shares with the other scales discussed here is that it cannot identify if a repeated activity, such as catalog distribution or billing, should be done more often or less often.
  • It does not have the numerous shortcomings of the expectations scale (i.e., that scale's inadequacies in evaluating performance relative to low or unknown expectations, changing expectations, or different levels of expectation within the customer base.)
  • Finally, for the reasons just cited, the scale is ideal for employee satisfaction measurement surveys as well.

More straightforward

With the traditional scales used for customer satisfaction measurement, the business manager is only getting an indirect and, perhaps, ambiguous answer to the question: "What aspects of my product or service are most in need of improvement?" The way to find the answer to that question is to ask it in a more straightforward way. The "improvement needed" scale allows him or her to do that.