Editor’s note: John Goodman is president of, and David Beinhacker is measurement manager at, TARP, an Arlington, Va., research firm.
Would you pull off the road to fill your gas tank if you could already see that needle was showing full? Probably not. However, we have found that many companies are essentially doing just that by executing surveys to gather data on questions they already have the answers to.
The opportunities for saving the company serious money lie in eliminating two areas of research. The first is measuring transactions where it is known that the process is stable and performance is very high, e.g., 98-99 percent accuracy. The second is using surveys to measure quality where there are existing internal metrics that report quality more accurately and quickly. In both situations, executing surveys is a complete waste of money!
Needle in the haystack
A major financial services company recently started to rethink its approach to measurement of customer satisfaction and quality. It found that it had 80 different ongoing transaction surveys for the full range of important customer transactions.
Of these, it found that about 30 transactions were accurately executed, meeting customer expectations at least 98 percent of the time; for some transactions, expectations were met 99.5 percent of the time. The question then is, for these transactions, is a survey the right way to measure performance? When a survey is used in a near-perfect service environment, you are basically looking for the needle in the haystack. In that scenario, 98-99.5 percent of all measurement resources are deployed in an area where there is no problem!
Relying upon a complaint-handling system offers a better approach. If the one customer out of 100 who receives faulty service is unhappy with the transaction, there is, on average, a 50 percent chance he or she will complain. Therefore, by monitoring complaints, the company can monitor the quality of the process at a greatly reduced cost. If complaints rise significantly, you have proof that something is wrong. A major hotel chain found that it successfully delivered clean bathrooms 99.6 percent of the time. To monitor their continued success, rather than survey 10,000 people to get back 1,000 surveys four weeks later, of which four would have bathroom complaints, the chain chose to put a sticker in the bathroom and on the room desk highlighting the guest services hotline. Customers’ call rates would let the chain know if quality had slipped.
Are you answering a question your colleague answered yesterday?
The second instance of resource waste is when research is used to gather information already available from other internal data sources. A major home repair services company was spending millions of dollars surveying customers to see if the repairman showed up on time. However, the company’s call center tracked data on exactly how many times customers had phoned to complain about repairmen who were late or no-shows. For high involvement issues like a home visit, 90 percent of consumers will call if the repairperson is not there at the appointed hour. Why survey to gather data you already have?
If a company has detailed or continuous inspection of its deliverables, then surveying is both untimely and less accurate than these other sources. For example, a bank was surveying customers on their satisfaction with the uptime of the ATMs. However, the bank already had internal IT metrics that measured ATM uptime to four decimal places. In another example, a bank was monitoring five calls per month for all 300 reps in a call center, a total of 1,500 observations per month. Additionally, it was fielding a random survey of a sample of 300 calls per month, trying to determine if the reps were using the consumer’s name and a series of other operational dimensions. While the survey can give good data on the overall impact of the call center on customer satisfaction, the call monitoring data will give much more timely and actionable information on which types of calls are being handled best and where improvements are needed. The survey cannot provide much in the way of diagnostics beyond a very macro level.
Use them intelligently
So, is the answer to discontinue surveys? Not at all; just use them intelligently to do what they do best: identify dissatisfaction.
Some recommendations:
1. If you execute eight out of 10 transactions excellently, don’t do a random sample of all transactions. Focus your survey on those transactions that you do poorly and periodically sample the rest. Neiman Marcus Direct generally gives great telephone customer service but it has found that there are two types of transactions that are challenging. It specifically surveys customers each month who require those types of transactions to see if the company is improving its performance while surveying a much smaller sample of the rest of the callers.
2. Make sure the data doesn’t already exist. We have found that departments often operate in silos, unaware of the research being done or data being collected by their co-workers. Sharing existing research and internal metrics not only gives immediate access to actionable data but does so without incurring the costs of conducting surveys.
3. Ask other departments what data they have describing the customer experience and use their data to track improvement. Use a survey to show the impact of the improvement. At MacKenzie Financial, a survey identified several opportunities to improve client satisfaction. Each operations area then took action, including identifying existing metrics that described the company’s daily success at delivering quality service. Research was then used to confirm the result of the actions: a double-digit improvement in client satisfaction and loyalty.