Producing consistent results through quant+qual integration

Editor’s note: Hubertus Hofkirchner is CEO ofPrediki Prediction Services Vienna. This is an edited version of a post that originally appeared here under the title, “New qualitative: Tapping crowd intelligence for advertising efficiency.”

“Qualitative is the better-suited research method, bring it back” is a frequent complaint of advertisers (like John Zeigler, DDB group chairman) when their work undergoes quantitative pre-testing. Yet, the long-term trend shows an unabated decline of qualitative in the wake of the success of online quantitative and automated digital research.

Proponents of qualitative criticize that quantitative data provides too little insight into consumers’ reasoning and emotions. They propose to intensify traditional focus groups and face-to-face interviews. Before reverting to these methods we should consider why they lost ground in the first place and whether innovations provide a better alternative for advertisers.

Marketers and advertisers have very good reason for joining the method debate. A recent study on 30 years’ worth of data from nearly 1000 case studies – done for the IPA’s Effectiveness Awards – found that emotional advertising is twice as effective as less creative campaigns that primarily address the rational mind. A campaign’s immediate success is less important because the long-term effects make all the difference.

This raises several questions: Does the decline in qualitative disregard the IPA’s findings? Are pre-testers using the wrong methods? What are some other research methods – which are clearly on the rise, according to ESOMAR’s GMR 2014 industry report?

The distinction between quant and qual stems from traditional methods being good at either one or the other with full coverage needing both. Today’s business environment however, puts enormous pressure on companies and managers to be more agile and cost efficient. Tighter budgets and deadlines do not combine well with the luxury of conducting two sequential projects. Doing them in parallel is not a good solution, either. The cost still remains a problem and the studies cannot build upon one another and little value is added.

Separate projects create their own issues. If the qualitative findings fit with what a marketer already knew quantitatively, he will ask himself “Good but so what?” When they do not, his puzzled question will be, “Which one is right?”

The benefit of solely qualitative projects may be lost if verbatim responses get summarized into shallow insights and fail to wash out the gold nugget called actionable insight. It relies on a central – potentially weak – point, the individual researcher whose analysis is subject to personal perception, individual interpretation or even fear when encountering results which the client may dislike.

Today, qualitative or quantitative research is no longer a contradiction as new methodologies can integrate the two. One such example out of ESOMAR’s growing “other” category is the prediction market method. Its trading mechanism produces a stream of quantitative data while in parallel, the participants’ market talk generates rich topical conversations. The two are inherently linked: positive or negative market price movements show directly if a piece of market talk is an insight or nonsense.

Experiments in psychology have shown that people can predict other’s emotions and behavior better than their own (see Puleston/Hofkirchner: “Predicting the future” – ESOMAR 2014). The answers to, “Will I clean up?” versus “How many of us will clean up?” are 50 percent and 15 percent, on average, the empirical result is 13 percent. Therefore, proper prediction market questions always ask for other consumers’ actions, emotions or intentions but never the respondent’s own.

Each respondent in a prediction market becomes a research analyst himself, as the challenge is to be right about the future. This decentralized approach opens different perspectives with correspondingly deeper interpretation of everybody’s verbal and numeric responses. Crowd intelligence emerges and market price movements provide the feedback which trains the panel’s collective IQ and sparks creativity. The environment feels democratic. Respondents feel a strong impulse to help the client.

Advertisers can now get two viewpoints at once, combining the rational and the emotional into a consistent whole. Artfully structured question designs ensure that respondents forecast and discuss both near and far-horizon KPI’s to facilitate differentiation between short-term novelty effects and long-lasting emotional ones.

The future lies in the new integrated methodologies which produce consistent quantitative and qualitative results with only one project.