Skip to: Main Content / Navigation

  • Facebook
  • Twitter
  • LinkedIn
  • Add This

Big data: boon to improving customer experience, bane of researchers?

Article ID:
June 2013, page 34
John Goodman, Peter North and David Beinhacker

Article Abstract

The authors explore examples of the use of big data to improve the customer experience and offer tips for researchers to avoid being left in big data’s wake.

Taking the good with the bad

Editor's note: John Goodman is vice chairman of Customer Care Measurement and Consulting, Alexandria, Va. Based in Chevy Chase, Md., Peter North is principal of True North Digital, a digital marketing consulting firm. David Beinhacker is director of research at Customer Care Measurement and Consulting.

While big data (BD) is a hot topic in the realms of marketing segmentation, marketing research and customer experience, your view of it likely depends on your vantage point. For those in segmentation and customer experience, big data offers many exciting and varied ways to listen to, learn about and analyze your customers and non-customers alike. For those in marketing research, however, big data looms as a threat that could put you out of business, as technology vendors have created data-driven utilities that ostensibly replace much of the satisfaction-tracking functions traditionally performed by market research companies.

Big data is most often used for marketing purposes but it has powerful customer experience applications that are often overlooked and underutilized. This article first suggests a broad definition of BD. It then suggests incorporating a broader range of data inputs into BD than most companies use. These expanded inputs, in turn, lead to valuable opportunities for improving the customer experience process. The article then suggests the biggest opportunities and threats created by the broader definition and range of inputs. Finally, it addresses how to best play the politics of big data and recommends specific actions you can take to assure you are driving the BD bus as opposed to being run over by it.

New forms

In 2012, Gartner updated its definition of big data to “high volume, high velocity and/or high variety information assets that require new forms of processing to enable enhanced decision-making, insight discovery and process optimization.” While there are dozens of definitions, Gartner’s is adequate because it highlights the complexity stemming from many sources and new forms of processing to make decisions and discover insights.

The term “many sources” means that in the customer experience context, this data can be produced by any electronic source but it also exists on pads of paper in call centers and within stories related in focus groups. So, in the customer experience context, any information or data describing an instance or aspect of a customer experience is a possible part of the customer experience BD constellation. The challenge is recognizing the source and translating it into a compatible format and classification. The new approach to processing implies sorting through the mass of data to find the nuggets that are important. Making decisions and discovering insights requires more than data-crunching; actionable outcomes arrive from what Shah, Horne and Capella call “informed skeptics” who apply informed judgment to analysis.1

Customer experience BD consists of the usual suspects – contact data, customer survey data, employee survey data, purchase data, customer demographics and economic data (usually purchased from aggregators). In addition, there are now many “squishier” data sources such as digital recordings, social media and videos, as well as ad hoc employee input. Finally, there is a massive amount of transaction and quality data, such as process failure data (e.g., the part was not in stock or the package was not delivered on time), that, in our experience, is not even viewed as customer experience data. However, it is often some of the richest in terms of describing the key facets of the customer experience.

In the past, the squishy data was only sampled by quality analysts or researchers because it was too massive. Similarly, the transaction and quality data was only reported in aggregate and in most cases was downplayed. For example, to say that 95 percent of all appointments were met sounds impressive until you know that there were 3,000 appointments in a day so 150 customers waited at home in vain each day. If you then could tie those missed appointments to call data, survey data and repurchase data, you could start calculating how much revenue and word-of-mouth damage occurred in addition to the extra cost of rescheduling or expediting visits.

A number of opportunities

Big data can be used in a number of ways and presents a number of opportunities for the customer experience – almost all involving making business more proactive. It is traditionally used to segment customers for marketing offerings, such as how Amazon uses it, suggesting cross-sell and up-sell. The new innovation is proactively setting and resetting expectations. In a November 2012 survey of more than 600 companies, conducted as part of World Quality Month, the American Society for Quality found that the single biggest concern of businesses was setting proper customer expectations2.

The survey polled more than 600 quality and customer service experts worldwide, who said that managing customer expectations (29 percent) and communicating with customers (20 percent) are the top challenges in maintaining quality service.

Other challenges include: educating customers about products and services (16 percent); providing customers with timely service (13 percent); and training and retaining good staff (12 percent). What is fascinating with all five of these items is that they all can be at least partially remedied by big data.

Given that no one reads contracts or directions anymore (we find that less than 2 percent of any audience says they fully read their homeowners insurance policy, for example), expectations-setting must be achieved via highly tailored and “just-in-time” communication. One insurance company tailors the welcome letter to customers to highlight only the three provisions that have proved to be the most likely to create unpleasant surprises for that specific type of customer. Likewise, a utility uses integrated data to communicate via each customer’s preferred communication channel (text, e-mail or cell phone) to update power outage information by individual neighborhood. Market research should be working to identify these information needs (including exactly when they will be needed) and provide this information to the operations side of the business – that is, steering the big data bus.

The second and third issues (communication and education) have huge opportunities for enhancement via big data. Customers can be contacted through their preferred communication channel about the most relevant information, which will get their attention. Both of these issues were combined in a recent test at a West Coast power utility. We found that sending an e-mail saying, in effect “Your bill is going to be an unpleasant surprise and we’re concerned!” got a 54 percent open rate across more than 30,000 customers – an extremely high open rate (most industry e-mail campaigns have open rates of no more than 10 percent). Once opened, the e-mail offered energy usage data for the first 10 days of the billing period and projections showing that the ultimate bill would be $50 or 30 percent higher than expected. It then offered three energy-saving tips and links to sign up for more support tools. Click-through rates and adoption of the offered tools were very high and a number of consumers called with compliments, saying it shows the utility really cares about helping consumers conserve and keep their bill down. Getting such a positive response from customers from such negative news is a testament to the vast potential of customer communication and education.

Providing timely service via data is what one of the authors (Goodman) calls delivering “psychic pizza.” That is, ringing the doorbell and saying, “Here is the pizza you were about to order!” This process preemptively addresses predictable needs or service questions that customers haven’t even asked yet. For example, an East Coast utility has addressed the perennial point of pain of arranging a home repair visit by getting the consumer’s preferred channel of communication, confirming the afternoon before in an automated manner via that preferred channel and informing the customer that the service tech will call at 8:00 a.m. to tell the consumer where they stand in the daily queue. This delights the customer and eliminates the cost of up to three inbound phone calls: 4 p.m. the day before (“Is he really coming tomorrow?"); 8 a.m. the day of the visit (“When is he coming?”); and the frantic call if he is not there by 10:30 a.m. (“Is he really coming before noon because I have to go back to the office!”).

Retaining good staff

Big data can also assist in retaining and training good service staff by both preventing frustratingly recurring customer “dumb questions and inquiries” (which are reduced by voice-of the customer and customer education activities) and by providing easily accessible, flexible answers to frontline employees so that they can be more successful in handling difficult issues. One financial services company provides service staff with flexible solution spaces and talking points based on customer history and value and the specific circumstances – there can be four answers to the same problem based on the big data algorithm.

Half of all voluntary turnover among good employees is due to the employee being frustrated by their own lack of effective tools and answers. They often say, “I’m not getting paid enough to take flack for things that are not my fault and that I can’t explain.” Big data can provide them with clear, believable, defensible explanations that leave the customer feeling that they were treated fairly.

Market research should be identifying where education is needed and evaluating the effectiveness of customer education as well as the effectiveness of response guidance in producing customers who are satisfied and feel that they have been treated fairly and with respect.

Serious threats

But for all of its beneficial applications, big data also presents some serious potential threats to the quality of the customer experience and to the job security of market research professionals and vendors.

The threat to the customer experience is using the data in a way that offends or scares customers. One hotel chain found that too much personalization offended customers. A customer noted “The fact that I ordered scotch on one trip does NOT mean I want extra scotch stocked in my mini-bar – in fact I’m not sure that I even want what and how much I drink in your records!” The point is to be responsive without being creepy. To avoid making a mess of the experience you cannot take the data and mechanistically act on it. As Shah et al. say in their Harvard Business Review article, leaven data with judgment and avoid formulaic, one-size-fits-all actions.

The threats to researchers’ job security are much more serious and certain. First, most CRM systems execute automatic e-mail satisfaction tracking campaigns, which is fast becoming the response medium of choice. This eliminates the need for outside survey firms to conduct satisfaction tracking surveys. Additionally, the CRM systems can automatically integrate the survey data back into the customer records for use in both analysis and future interactions with the customer. Likewise, most telephone automated call directors are equipped with computer telephone integration (CTI), which facilitates measurement of first-contact resolution by allowing easy identification of repeat calls from the same phone number – again, no need to ask the same introductory questions of a customer who has already provided the answers.

A second threat is that CRM, when tied to operational data, can also report customer experience more complexly and accurately than the customer themselves can on a survey. For example, a delivery company has operational data indicating exactly how many packages missed their connection and therefore were not delivered on time. The company also has call center data describing the incidents where the customer called in to complain. Finally, it has satisfaction survey data collected after the fact. The CRM data, with the operational data included, will provide a more complete record of the total number of customers encountering any particular problem and, after the fact, allow analysis of the impact of the incident on loyalty and actual sales – which is often more credible than customer-stated intention to repurchase. Again, this suggests a possible decline in the need for surveys.

The third threat lies in the contact center and is embodied in the speech analytics tools now being brought online. These tools, when properly tuned, can ascertain satisfaction, replacing both call-quality monitoring and satisfaction surveying. While this is troubling for survey companies and the market research department, it is disastrous for the quality-monitoring staff, because we predict that within three years, their jobs will completely disappear. Finally, these same speech analytics tools can analyze phrases and sequences to identify the best ways to pitch a product to improve sales and close rates.

You are probably reacting angrily, in one of two ways. First, you say that these tools are not accurate and won’t be any time soon. Three years ago you were right. Now, some of the tools are becoming very accurate. Second, you are confident that the tools will be too expensive to be adopted. Again, in the past, they were a half-million dollars and up. The prices have dropped dramatically and now compete with the costs of labor-intensive activities like call monitoring. Further, once they are purchased for one function, they will be available to perform all the other functions as well.

Now, before you start redrafting your résumé, take heart. Most of the vendors don’t know how to effectively use the brilliant tools they are being offered – so you have a few years. But you should prepare to incorporate them into your toolkit.

Learn to drive it

So how can you take the wheel of the big data bus and learn to drive it rather than getting run over by it? First, some perspective. Big data has been around for almost 20 years, starting with Amazon personalizing product recommendations based on previous purchases. In the late 1990s, CRM systems promised a customer experience nirvana almost before customer experience was first introduced by Joe Pine in the mid-1990s. The ongoing challenge of BD from a customer experience perspective is to get the BD tools to truly (and cost-effectively) enhance the customer experience and to do so without seeming creepy and/or violating privacy. Most of the vendors cannot provide the savvy judgment needed to capitalize on these challenges, which provides the primary opportunity for marketing research.

For marketing research to gain a leadership position in the use of big data, researchers must:

  • explore and understand all the capabilities of BD to provide and take action on business intelligence and how it can be practically used for enhancing the customer experience, like the delivery of psychic pizza to both improve loyalty and word-of-mouth as well as decrease costs;
  • understand at least topline approaches to the technical integration of the various tools like CRM, CTI and speech analytics;
  • provide the cross-functional bridge between marketing, operations and finance, exerting the necessary political finesse to keep everyone focused on the goal of enhancing the customer experience;
  • evaluate how effective the implementations are in really impacting customer experience; and
  • create the capability to analyze how the tools impact costs and revenue in a transparent manner, one that the CFO and CMO will accept.

Capitalizing on big data will require hard work and political savvy and there is a good chance you will end up with a smaller staff. But the work you do will have more impact and the jobs that remain will be more sophisticated. And, as you gain experience working with and synthesizing all of the disparate data sources available to you, you’ll enhance your own skill set in ways that will position you for whatever comes next.


1 Shvetank Shah, Andrew Horne and Jaime Capella. “Good data won’t guarantee good decisions.” Harvard Business Review, April 2012, p. 24.

2 “Naughty or Nice, ASQ offers tips for getting better customer service during the holidays.” Press release.

Comment on this article

comments powered by Disqus

Related Glossary Terms

Search for more...

Related Events

August 30-31, 2016
Burke Institute will hold a seminar, themed 'Unlocking Consumer Insights: Neuroscience, Non-Conscious Measurement, Measurement of Emotion,' on August 30-31 in Chicago.
September 12-14, 2016
RIVA Training Institute will hold a course, themed 'Fundamentals of Moderating,' on September 12-14 in Rockville, Md.

View more Related Events...

Related Articles

There are 2635 articles in our archive related to this topic. Below are 5 selected at random and available to all users of the site.

General Mills marketing research decides cookbook cover
"Betty Crocker's Cookbook" has sold over 22 million copies, but as the flagship of their publishing line, General Mills Marketing experts needed to figure out a cover that could keep the book selling strong. A variety of techniques were used to figure out what book cover would sell best.
JCPenney pinpoints its customers
In order to fully understand the needs of their customers, JCPenney has initiated a series of studies called Consumer Feedback. These studies give JCPenney a clear picture of the needs, attitudes and behaviors of their customers.
Rating scales can influence results
A summarized excerpt of a U.S. Department of Commerce study testing the merits of a seven-point rating scale versus a 10-point rating scale.
Quest research pays off for United Way
In the past, marketing research was too expensive for many United Way organizations. But all that has changed, thanks to a new research program called Quest. By utilizing innovative survey techniques and technology, Quest allows United Way organizations to improve communications, identify key services and improve fundraising easily and inexpensively.
Continuing analysis of shopping habits in San Diego
The "Continuing Analysis of Shopping Habits in San Diego," or CASH, is a clearinghouse for consumer information in San Diego County. The data is a service of The San Diego Union and The Tribune, a San Diego newspaper, and employs telephone interviews to collect extensive information on consumers throughout the area.

See more articles on this topic

Related Suppliers: Research Companies from the SourceBook

Click on a category below to see firms that specialize in the following areas of research and/or industries


Conduct a detailed search of the entire Researcher SourceBook directory

Related Discussion Topics

Hybrid methodology
04/25/2016 by Dora Kicsi
Survey Incentives
03/23/2016 by Alex Hales
Confidence interval Definition
03/03/2016 by Alex Hales
Survey Incentives
02/21/2016 by Blaine Ung
TURF Simulator with Shapley Value
02/10/2016 by Amit Zaveri

View More