How will ChatGPT change the marketing research industry?
Editor’s note: Kevin Karty is CEO of Intuify, a Boston-based market research firm.
Throughout my 20-year career in data science, I was an AI skeptic. When companies bragged about the AI in their software, I cringed in embarrassment. At best, it was glorified machine learning. At worst, flat out deception.
Inevitably, the hype went bust: AI-backed hedge funds underperformed human managers (who in turn underperformed index funds). Sentiment coding tools still couldn’t understand phrases like, “That game is sick!”
When I first tried ChatGPT 3.0 (we’re on 4.0 at the time writing this article) I expected to be disappointed. I was shocked. ChatGPT is going to force radical change in our industry. No, it won’t destroy marketing research, but there will be winners and losers. The difference will be determined by how well companies and individuals adapt.
Image adapted and updated from Pablo Villalobos, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, Anson Ho and Marius Hobbhahn. “Machine Learning Model Sizes and the Parameter Gap.” ArXiv [Cs.LG], 2022. arXiv. http://arxiv.org/abs/2207.02852.
Before diving in, let’s first understand the issue: How are large language models (LLMs) like ChatGPT different from traditional AI?
- Model size: LLMs are massive … like a billion times bigger than the models that won the Netflix Prize back in 2009.
- Emergent learning: At some point, LLMs began to develop capabilities they weren’t trained to do. ChatGPT “taught itself” to do advanced chemistry.
- Zero shot (and few shot) learning: LLMs don’t need proprietary training data. Want ChatGPT to summarize hotel reviews? Just ask it! Is the data really new? Give it some examples and it can learn on the fly.
LLMs represent humanity’s first significant step away from narrow AI toward general AI.
What does this mean for me, my company and my job in marketing research?
AI’s implications are far reaching. Everyone will be affected. In this article I’ll share a list of some changes we can predict in the next few years.
Your company will expect more from you and you’re going to need LLMs just to keep up.
By some estimates, 73% of marketers in the U.S. use some sort of generative AI to create content for their jobs – writing e-mails, blog posts, taglines, etc. Reasonable estimates indicate that these tools enhance productivity by 37% to 60%. We’re already at a place where if you don’t use ChatGPT and similar tools, you are at a distinct disadvantage compared to your peers.
Creating mediocre market research content will become easy.
Want a quick starter survey to learn more about healthy dog food? Tell ChatGPT to write it. It’s basic, but it will get you started. Then you can use ChatGPT to refine the questions one by one. Remember the days when you would spend hours searching on Google to create a list of barriers to adoption? Sifting through blogs, pasting ideas into excel, then cleaning them up and simplifying them? ChatGPT can knock that out in minutes. Need a starting point for an IDI or focus group script? Just ask!
Cheating on surveys is going to get even worse.
Cheating in survey panels is already a massive problem. Research from the CASE initiative gives us estimates as high as 30% to 40%, with higher rates in low incidence surveys. P&G shared an example of a failed product launch for Crest that was green-lighted only because of massive cheating in a product test. With ChatGPT, it’s becoming increasingly hard to detect cheaters using open ends, red herrings and other methods. Think it’s easy to detect a bot from a real person? Type “Jailbreak ChatGPT” into Google. We’ve gotten to the point where our company voice-validates every respondent on important surveys. Every. Single. One. And within five years, deepfake technology could make it easy to fool that too.
Qual research at scale will finally become a reality.
Imagine a day when we don’t need 100 Likert scale questions for a brand tracker or positioning study, or when Top 2 Box concept scores are replaced by just asking people what they think in their own words. That day may already be here. ChatGPT is the most effective open-ended coding tool that has ever been created. It can recognize sarcasm and slang from context with incredibly high accuracy, detect emotional content and build relevant topic models that make sense. All with zero training data. We’re already able to process thousands of voice and video files quickly, automatically and with almost no human intervention to extract near human-quality insights.
ChatGPT will level the playing field.
Remember when Qualtrics paid over a billion dollars to acquire Clarabridge for its best-in-class topic and NLP models? That tech is already obsolete. I spent six years of my life getting a Ph.D. and learning to run fancy statistical models. Now my teenager plays with AI for fun. Before ChatGPT, companies would have forced employees to adopt AI tools. Now, companies are asking their employees to slow down AI adoption.
LLMs will make a company’s domain expertise easy for everyone to access.
Large companies have vast amounts of redundant information, often with only a few chosen individuals carrying institutional knowledge. LLMs will finally democratize that. Plenty of startups have promised AI that will organize and index a company’s market research data in unified databases so that it’s easy to find, but the effort to implement and maintain those systems is self-defeating. LLMs will literally skip that process. Point a pre-trained LLM at a stack of documents, have it “read” them, then type in questions and get answers without even reading the source documents.
These are just a handful of the use cases we’ll see.
Will everything change for market research? What should I do about AI now?
First, don’t freak out. Market research isn’t going away. “Virtual respondents” that replace real people in surveys and interviews are fraught with problems like bias and regression to the mean. LLMs still need human managers and guardrails. Many large companies are slowing down adoption until they address security and privacy concerns, which will give everyone a bit more time to adapt.
Second, learn as much as you can! Experiment with it because it isn’t a fad that’s going to disappear. Sign up for a free account with ChatGPT or Bard. Google “best ChatGPT prompts for market research” and try some of them out. Stay up to date with webinars, conferences and lunch and learns.
Third, elevate your personal unique value proposition. LLMs will devalue certain roles: summarizing reports, coding open ends, writing survey content and simply knowing random facts. However, this will free us up from tedious tasks to focus on strategic thinking that turns insights into organizational action. Successful market researchers will increasingly become strategic advisors. This isn’t a new trend; LLMs are simply accelerating it.
There’s good reason to stay optimistic, at least for the next five years. Beyond that, the future is much harder to predict.
We are certainly living in exciting times – dangerous but also full of opportunity. We need to help each other adapt. Together, we’ll get there.