Editor’s note: Roddy Knowles is director, product and research methodology at Research Now, Plano, Texas.  

Automation is not a new concept to the marketing research industry, yet in the past couple of years the term has gone from seldom discussed to common parlance. However, there is a degree of ambiguity around what automation means and how it relates to the future of the research industry. To some, it signals a bright future: smarter research done faster and in a more cost-effective manner. To others, it is met with trepidation: new technology that will soon take the place of skilled researchers and project managers. Both are valid viewpoints, though to understand what automation is and why it matters for the future of marketing research, a more nuanced understanding is needed.

Improving efficiency 

Conducting marketing research used to be a drawn-out and arguably inefficient process. The most obvious change occurred roughly 15 years ago when the industry transitioned from phone- and paper-based surveys to online data collection. Data collection firms pioneered the use of software algorithms to recruit and sample research participants at a massive scale and at speeds that were previously impossible. Before the shift to online, data would be collected and tabulated in a manual way (albeit with some computer assistance), banners would be printed out and researchers would look over the results and create a report. By the time the insights were shared with clients, the data was often stale – possibly unable to meaningfully impact business decisions. Through the years automation has made analysis quicker and cheaper. Things like running tabs, coding open-end questions and creating charts became simpler and taken for granted.

Fast-forward to where we are today, a world where technology and automation have altered the landscape. Insights are delivered digitally at the speed of business. Even with all of the changes, if we take a step back and look at a typical research project today, we can point to areas that need improvement. It’s common to approach every research project as its own unique creation, essentially building it from scratch. While there may be some borrowing from prior research, researchers will redesign whole questionnaires as standard practice. Instead of reviewing and improving upon previously-defined sampling parameters, they will re-create new audiences with custom screeners. There has been a fair amount of resistance to automating the front end of the research project – specifically design – as to some it feels like it is infringing on the domain of the researcher. After all, shouldn’t it be the researcher who tailor-makes custom research? I believe there will continue to be a place for truly custom research but many studies can benefit from an automated approach.

Approaching automation

While there are many ways to approach automation, one framework I like to use follows this three-step process:

  • Identify the types of recurring studies you conduct, those that address the same objective with the same approach. These are the best candidates for automation. Common examples include ad testing, concept testing and customer satisfaction work.
  • Define the template or standardized approach. You might review existing questionnaires for inspiration, look for an off-the-shelf tool designed by an expert researcher or consult with a research provider to create a custom and repeatable solution. Don’t give up if you can’t immediately find the perfect templated solution. I’ve seen many clients find success by picking and choosing different predefined modules to very quickly create a tailored and future-ready solution that meets their needs. 
  • Scope the research outputs. Because one of the benefits of automation is standardization, it’s critical to understand what you’re getting at the back end of your projects. Make sure dashboards and/or reports allow you to gain efficiencies in both analysis and data sharing.

Let’s look closer at the last step – research outputs. One benefit of automation is the ability to easily compare across studies. When research is standardized, it not only reduces errors but also allows for quick comparison and measurement of results against a pre-built set of norms. Approached properly, automated research should eliminate problems that have plagued researchers around having to ask the same question multiple different ways in different studies. If you’ve experienced the waffling that comes after someone asks how the results of this study compare to a similar study conducted last year, you know what I’m talking about. Having proper benchmarks or norms in one place to measure current and future results can be a lifesaver.  

Having an automated or standardized approach to research can allow researchers to use tools that help them to do their jobs faster and better. Automation can make some of the skills previously required for researchers obsolete for day-to-day work, becoming required more as theoretical principles rather than skills brought into practice. For example, someone can field a study without having to sit down and write a questionnaire (using a template); program a survey or even liaise with a programmer (the questionnaire is already programmed); run data tables (done through a reportal or avoided); or create charts (done in a dashboard and can be exported). The ramifications here are twofold. First, it means that those with relatively limited experience can conduct research. Research need not be daunting nor unduly complicated. Second, it means that the researcher can be put to better use. Researchers and insights professionals can spend time crafting stories with their data rather than running tables or coding open-ends. Time can be shifted from manual, behind-the-scenes work to making data impactful. After all, isn’t this what researchers should be doing?

Transforming research

Many researchers have been harnessing automation to revolutionize the marketing research industry. There are numerous examples of this. Artificial intelligence is being used to support bots that can conduct qualitative chat sessions as if they were human moderators. Surveys are being modularized or chunked in near real time and back-end data imputation happens with the push of a button. Data sets can be linked and integrated, with limited human intervention, so that observed behavior can be connected with survey data and save participants from having to answer so many darn questions. And the list goes on. It’s worth keeping an eye on and experimenting with these relatively nascent but fast-developing technologies fueled by automation because some of them will fundamentally transform how we conduct research. 

However, what may make the most sense for you or your organization is to focus on where you can leverage automation now to improve how you run research projects. Ultimately, automation frees up the researcher from tedious implementation and allows for more time and brainpower spent on impacting business decisions and thinking on a strategic level.