By Huey Yii “Daphane” Tan and Sean Jordan
Over the last two decades, journalists and reporters have seen their work augmented or outright replaced by automated systems, the utilization of which has allowed many news organizations to trim both reporting and editorial staff in favor of generating cheaper, cleaner content authored by robotic algorithms. The human footprint in journalism is fading as technology gets better every year and it is not unthinkable that one day, much of the content published as news will actually be written by artificial intelligence algorithms, with human reporters merely providing the data and editors overseeing the final results.
As a similarly data-driven industry, the field of marketing research is also beginning to grapple with these changes. While journalists cover real-world events, marketing researchers investigate and report on business problems. Both industries are focused on gathering data and both are held to high standards in terms of how they report what they’ve uncovered. Both are also facing increasing pressure to provide useful reports faster, more cheaply and with fewer errors. And both are under increasing pressure to leave the data gathering and assembly to automated and AI systems and to focus instead on finding the facts needed to tell a compelling and insightful story.
A shift in the industry
This change isn’t necessarily negative. Less than a century ago, the rise of appliances for washing clothes and dishes led some to question if housewives might become bored and dissatisfied if so many of their domestic chores could be so easily automated. Such an idea is laughable today. Nevertheless, the transformation of these appliances from optional to essential within a single generation illustrates how quickly broad cultural acceptance towards automation can happen. And whereas data gatherers were once tasked with spending a significant amount of time collecting, preparing and cleaning data for detailed analysis, today’s automated tools save researchers and analysts from these rote tasks and allow them to spend more time on the substantive work of uncovering insights. The more intelligent these tools become, the more they take the tedium out of the process and make it easier for researchers to find the answers they seek.
Granted, most researchers don’t need to be persuaded that AI systems are a benefit. In a 2018 survey conducted by Qualtrics with marketing research professionals, 93% of those surveyed said they view AI as an industry opportunity instead of a threat and 80% said AI will make a positive impact on the market.
Despite broad appreciation of the benefits of AI, its potential downsides have not gone unnoticed. The same report found that many researchers are also considering more training or a career change to safeguard their jobs against AI. Qualtrics (which itself has incorporated many AI-based automated tools into its research platform) predicts that AI will take over data analysis within 10 years. Furthermore, technologies such as automated statistical analysis, text analysis and natural language processing are expected to significantly impact industries that market researchers serve.
The role of the researcher of tomorrow is to get used to delegating analysis to AI systems. “AI will have an impact on how data is collected. AI will have an impact on how data is analyzed,” says Aaron Fransen, vice president, financial services practice for MaritzCX. “But AI will not have an impact on the fundamental need to identify how to solve business problems when it comes to making business decisions.”
In other words, the robots aren’t coming to replace marketing researchers. AI is evolving to work alongside human problem solvers.
The human element
As long as there is a need to understand consumer behavior, the market research industry will never be obsolete. However, the nature of conducting research in the coming decades will hardly resemble today’s practices because technologies utilizing AI touch every single element, from defining the research problem to writing the final report.
Today, only a portion of marketing research involves AI. Humans still determine the entire upfront process, which includes defining a problem, setting objectives, selecting a research method and determining a sampling plan. Where automation does come into play is optimizing a survey, gathering responses, managing field work, producing charts and visualizations, analyzing open-ended comments and generating reports. Even so, many of these tasks still require a considerable amount of human input and some functions (such as AI text analytics) aren’t nearly reliable enough yet to give human analysts the day off.
However, within five to 10 years, it is quite likely that AI processes will be employed to handle these tasks, from helping to set objectives to writing appropriate survey questions, to automatically generating the appropriate analyses (complete with robust printouts and explanations of the findings in plain English). AI will also allow researchers to delegate tasks such as localizing surveys for different regions, panel onboarding, outlier detection and data cleaning to bots designed to handle these jobs efficiently and cheaply. And as these AI processes are linked into libraries of validated surveys and established best practices, future researchers may become far less concerned with the task of data collection than they are with what to do with the data once it’s collected.
“Yes, it’s possible to achieve 100% data automation in the MR industry but just because it’s possible doesn’t mean it is advisable,” says Paul Herdman, global head of customer experience, for NICE inContact. “Could we fully automate data collection and insights without ever speaking to another human? Yes, with today’s technology it is fully possible. However, technology is an enabler, not the solution for the MR industry. We still need smart, capable, highly trained people to frame the business problems logically, interrogate data sources that are most relevant to their business and ultimately make the right decisions on how to act. Technology enables us to investigate more data, ask more questions and get results faster and cheaper but each business is too unique to leave analysis and decision-making to algorithms alone.”
Part of the toolbox
Technology will never replace experience and common sense. In fact, humans are needed to validate the accuracy of AI-produced findings. Therefore, market researchers may need to invest in understanding the impact of technology on their jobs. “If you don’t understand at a basic level how the technology works, how any software program you’re using works, how can you know the outcome it gave you was the right one?” says Herdman.
AI technology is least likely to kill strategic jobs, according to Qualtrics. Marketing research veterans who don’t have the time or energy to learn new tricks can breathe a sigh of relief and focus on the value they bring: common sense and experience. AI and technologies will become a part of the research industry’s toolbox but we still need experience and business knowledge for higher-value tasks like establishing hypotheses, writing coherent survey questions, validating AI-produced findings and communicating findings to stakeholders or clients.
Furthermore, AI-powered market research is not as simple as a robot churning out models or finished reports. There is always going to be a need for a “captain on the ship,” says Joseph White, senior director of marketing science for MaritzCX, one whose expertise helps set up the problem for the algorithm, implements the algorithm in the system, trains the system to do what it needs to do and, most importantly, validates the system for accuracy. Similarly, report writing requires researchers to craft several versions of the report, complete with text for different outcomes. “Automation is a tool for the thinkers but it can never replace the thinkers,” says Jeff Minier, executive director at Kynetec.
Robust skill set
In a world of growing automation, researchers need to develop and cultivate a robust skill set that can incorporate these tools. Whereas marketing research was once the domain of practitioners with a significant amount of training in statistics or the social sciences, tomorrow’s marketing research leaders may instead need to be experts in utilizing tools and technology to effectively manage the automated marketing research process. “The tools in people’s hands today are different from those in the past,” says Fransen. “Therefore, researchers need a high degree of aptitude for utilizing technology.”
Because of the inevitability of AI augmentation, incoming market researchers will likely benefit from boosting their technology skills, particularly learning how to write code to maximize the benefits of using a research platform. For example, in the field of syndicated research, having some firsthand programming knowledge in database tools such as PHP or SQL allows researchers to develop a better understanding of multiple sources of data, organize them efficiently and provide better insights to their clients or managers. And while DIY tools for online surveys have made it easier than ever for supplier-side and corporate researchers alike to author complex surveys, some experience with JavaScript can go a long way in streamlining survey design. And knowledge of languages such as Python or R can allow a researcher to conduct sophisticated custom analyses without having to manually prep or reconfigure data for more standardized tools.
Of course, it is possible to have a healthy and long career in marketing and research without ever learning how to code and it’s quite likely the AI tools of tomorrow will even be able to assist in writing whatever code or scripts a researcher might need. “If there’s a researcher in the middle who is knowledgeable about things on both ends – the qualitative stuff or the super quant stuff – the better he’ll or she’ll be at framing up the problems, solving those issues, creating a good research design, analyzing numbers and turning them into insights,” says Minier.
The future of marketing research
Despite all of the changes journalists are facing in their own industry, the truth is that the profession of journalism is not in danger of being replaced by automation. Rather it is likely that the journalists of tomorrow will play a far different role than those of today, becoming content experts and cultivating relationships with their readers and sources. They will provide input into algorithms that help them to write their stories more capably and efficiently and to augment their own reporting with related data sources that can help their readers to develop a deeper understanding of the truth.
Likewise, the mission of a marketing researcher will continue to be to properly define a business problem, understand the research design and methods required to solve that problem, collect the right types of data needed to close informational gaps and then to synthesize that information into insights and solutions. The researchers of tomorrow are not likely to look back on the processes of today and wish they had more time to spend on the tedium of collecting and analyzing data. Rather, they’ll probably wonder how anyone could have played the role of a researcher without the aid of AI.
About the authors
Huey Yii “Daphane” Tan is a Ph.D. candidate in marketing at University of Tennessee Knoxville and a master of marketing research graduate from Southern Illinois University Edwardsville. Sean Jordan is research director at the Research & Planning Group, a St. Louis, Mo., research firm. He is also an instructor in qualitative research at Southern Illinois University Edwardsville’s master of marketing research program.