Editor’s note: Michaela Mora is president of Relevant Insights LLC, a Euless, Texas, research firm. This is an edited version of a post that originally appeared here under the title, “10 key pieces of advice on how to do and use market research.”
The 2015 MSMR Alumni Research Conference that took place in April at the University of Texas in Arlington, opened with a presentation by Susan Schwartz McDonald, president and CEO of NAXION. During the presentation, Schwartz tried to summarize her experience doing market research in 10 key learnings. I found myself nodding at each point because she described what many professional market researchers (myself included) have experienced.
I kept thinking that these nuggets of wisdom should to be shared with those coming to the market research field either as providers or users, so I have summarized them here:
- Know the difference between business problems and research problems, and frame them well. Business problems are not the same as research problems. Research questions exist in the context of business problems and should provide answers to solve them. The research question – such as, which product is preferred – can be used to solve business problems, such as which product should be invested in and where the growth opportunity are for the business,among others. Both providers and users of market research need to understand the business problem to frame the research questions correctly and provide relevant answers.
- Do not waste resources to research what is already known.Fear and internal politics have been behind a lot of market research done to validate what’s already known. There is knowledge in the organization that sometimes is discarded and new research is requested as part of a cover-your-ass strategy. Believe me, I’m all for using research for hypothesis validation but some questions do not require additional research.
- Work hard to become a professional market researcher. With easy access to online survey tools, may individuals outside of MR believe that no special skills and knowledge are needed to work as a market researcher. However, as Schwartz said, “Amateurism has contributed to cynicism about survey research.” There are a lot of bad surveys out there producing garbage data just because those designing them don’t have the most basic knowledge about the principles of market research. If you are new to the field, please find training or hire professional market research to help you.
- Focus on the input more than on the output when it comes to developing tools. Our field is extremely sensitive to the garbage-in-garbage-out principle. Unfortunately, there is a race to develop tools that can capture, process and present data often based on concerns more about the output than the input. For example, I have met many clients who believe user-friendly online survey tools equate to good surveys. The tools may have many features to develop good surveys but the tools don’t write the surveys themselves. Market researchers are responsible for that, and designing good surveys seems easier than it is. Badly designed surveys, as Schwartz said, make your tools look bad.
- Look to minimize measurement biases. All market research methods are susceptible to what is called the observer effect in which the measurement of a phenomenon is affected by the measurement itself. A simple example of this can be found in the order in which we ask questions, as each question can potentially prime the response to the next, and the results can be very different when the order is changed. This is why we have to be cognizant of the advantages and disadvantages of different data collection and analysis methods to minimize measurement errors.
- Ask questions people can answer. It has become a trend to despise survey research under the belief that people can’t articulate what they want, can’t remember behaviors, etc. The things is that people can provide useful information only if you ask questions they can really answer, and many surveys have questions that don’t allow for that. Some clients, due to budget constraints, short timelines or simply a lack of knowledge want to use surveys for everything, but this method is not a good fit for all types of business and research questions. Again, knowledge of market research principles and methods is needed to select the best approach and design the right types of questions in order to provide useful and relevant information.
- Be aware of the difference between statistical significance and practical significance. Focusing on statistical significance can be misleading if we don’t take into account the practical significance of the results. Take market segmentation for example. A solution can uncover significantly different but very small segments that would be enormously costly to reach. Sometimes, clients only want to do research in certain niche segments without understanding them in the context of the general population, missing invaluable insights into the practical significance of targeting such segments. If you find statistically significant differences, it doesn’t always mean they have practical value.
- Don’t fall in love with any one market research technique. This is probably the most dangerous kind of love in the market research field, often driven by level of comfort and knowledge of the many different tools available to market researchers. We need to be open to trying new techniques as the business and research problems dictate, without ignoring others proven useful in the past. Each research approach, qualitative or quantitative, provides different types of insights. As Schwartz said, “Research fashions come and go but what you need is a whole wardrobe.”
- Don’t discount what your customers say when statistical models don’t agree with them. We know that adding more variables increases model fit, so use hold-out samples for validation and be aware of over-fit models. I agreed when Schwartz said that customer insights are only as trustworthy as your data and your willingness to hear bad news. When things don’t match, check your math and your intellectual honesty.
- Use market research to learn how to think like your customers. Users of market research are not exempt from confirmation bias. As Schwartz advised, “Don’t look only for answers but seek to develop empathy.” We need to be aware of this bias and recognize when our interpretation of the results are driven by preconceived ideas, and it is robbing us from really understanding our customers. We need to use market research to walk in our customers’ shoes – to really understand where they are coming from – in order to provide products and services that are relevant to them.
In the end, Schwartz said it is our mission as market researchers to interpret, integrate and extrapolate. We need to interpret the results in the context of the business problem and help clients make decisions. By the same token, clients need to be open and consider market researchers as partners by collaborating and providing business insights that only someone inside an organization may have, otherwise interpretations are made in a vacuum, integration would not be possible and extrapolations can be simply wrong.