Editor’s note: Kevin Gray is president of Cannon Gray, a marketing science and analytics consultancy. Koen Pauwels is distinguished professor of marketing at Northeastern University.
Marketing researchers can now access more data (and a greater variety of data) and do so faster than ever. Advances in statistics and machine learning enable us to discover important insights in data never before possible. We can now answer questions that not long ago we might not have thought of. Qualitative research has also benefitted from recent developments in information and communications technology.
But when it comes to obtaining useful research results, technology isn’t everything. What if we’re answering the wrong questions? Let’s look at a few examples of what we mean.
Q: Our KPIs have been improving but sales have been flat. Is there something wrong with our tracking survey?
A: Why do you believe these KPIs are related to sales? Do you have empirical evidence that they are?
Q: Our social media tracker tells us overall sentiment is down, especially among young men. Panic!
A: Are young men your target customers? If so, what do they say offline, which often has more impact?
Q: Our data tell us sales among young women are up, so why are total sales down?
A: How important are young women in your current and desired customer base? What drives total sales?
Even with masses of data and sophisticated analytics, much precious time and budget can be squandered if we answer the wrong questions. Humans are natural satisficers and information misers. These tendencies are reinforced in today’s complex marketing world in which managers feel pressured to make many decisions very quickly.
In our busy daily lives, we tend not to be introspective or to examine our assumptions – or perhaps even recognize that we have made crucial assumptions. Furthermore, though they may have been overhyped, many cognitive biases have been revealed by psychologists which can seriously interfere with decision-making. Recent research demonstrates how big data by itself may not solve these biases, and even make them worse unless specific questions are asked and answered.
Asking the right questions
So how do we know we’re asking the right questions to begin with? In the following paragraphs we’ll break down several steps you can take.
1. Do your homework. If we haven’t already done so, we should prepare a comprehensive but readable summary of our marketing activity in the past several years and, to the extent possible, that of competitors. Industry trends and macro factors such as changes in economic conditions and regulations should also be covered. This document should be updated at least once a year.
2. Conduct a data audit. Data audits are highly recommended. We should list up all potentially relevant data we have or can easily obtain. They may consist of past marketing research reports, industry data, sales figures from various sources, customer transaction records, website activity, social media data, call center activity or government statistics. This will need frequent updating.
3. Determine market drivers. It may be possible to use some of these data in econometric modeling to obtain a general picture of what is driving our sales and market share. We may also be able to make post hoc forecasts of what our sales should have been to help us understand whether we are under or over performing. This need not be an intricate marketing mix model, and even correlating two variables at a time at various leads and lags can be illuminating, if interpreted with care.
Some common errors
There are some common errors we can avoid. One is to define our category too narrowly. Many categories overlap considerably – beer, cola and sports drinks may compete in some situations, for instance. Breakfast cereals may compete against breakfast bars, yogurt, toast or skipping breakfast altogether. One of us hasn’t owned a car for years thanks to convenient public transportation. In short, if sales are declining, perhaps it’s not the company we regard as our main competitor that is the culprit. On the other hand, our category may be growing in part because it’s been stealing sales from adjacent categories, and we may not be aware of this.
As to metrics, a typical problem is to track everything we can in real-time, whether or not it helps our decision-making. The most useful KPIs lead and convert into sales and can be influenced by our marketing actions. Paid search clicks, website visits and social media sentiment may not mean what you think they mean. Clicks may be accidental; website visits are mostly from current customers not in the market for an update or new product (as Microsoft found out); and social media sentiment may not reflect the consumer conversations that really matter for your brand. Always aim to evaluate whether a proposed KPI is truly driving performance and can be improved by the employees who are being judged on this KPI.
Another error is to focus on a narrow group of consumers as our core target in the absence of evidence. As it turns out, targeting decisions are sometimes quite arbitrary. To return to an example cited earlier, on the surface, it may seem logical that we should aim at young women, but we must be very careful when making assumptions about people, including why they use (or don’t use) our category. Concentrating on young women may have been a bad decision and it could be that the bulk of our sales is made by other people. They may be infrequent purchasers but comprise the vast majority of people using our category. Usage and attitude surveys can shed light on the realities of our brand and category.
Related to this is the supposition that most people are highly loyal to a single brand and highly involved with a category. Brand managers easily fall prey to this myth! Some other important questions we can ask ourselves are:
- Are we ignoring the upper part of the sales funnel?
- Are we mostly subsidizing behavior that was going to happen anyway?
- Have we eroded our brand equity by heavy discounting?
MR fundamentals and common sense
We need to scrutinize our decision-making at a fundamental level. It’s easy for managers to get hung up on some hot business fad that will be forgotten in a year or two and confuse buzz with business.
Some things to think about include:
- What might be causing the patterns in data that have grabbed our attention?
- Are these patterns real or chance fluctuations?
- Are the data we have data complete and accurate enough for our purposes?
- Are we thinking too much in black and white instead of weighing the odds?
- How much of our decision-making is just our gut and not evidence-based?
- Do we use research to justify decisions we’ve already made and ignore findings that do not support these decisions?
It’s all too easy to allow data, technology, preconceptions, habits and fads to govern decision-making and lose sight of our business objectives, marketing research fundamentals and common sense. We hope this article will be a reminder to be on guard against this.