Editor's note: Michael Conklin is chief methodologist at MarketTools Inc., San Francisco.
What’s the real business impact of marketing research? The question of return-on-investment (ROI) for research has emerged in recent years as one of the industry’s most-discussed topics. Some companies are already pushing suppliers to prove ROI on a per-project basis. A study from Forrester1 last spring predicted that within the next three years, more than half of market insights professionals expect to have to quantify their ROI to the business.
ROI measurement is a particularly thorny issue for market researchers. Traditionally, return-on-investment is thought of as a strict financial calculation assessing profit against the costs of investment. But the hard part of calculating the ROI for marketing research is the “R,” which is usually measured in dollars generated.
Many market research projects endure a long and complex process from initial definition to final execution, which means the specific value of the output can be difficult to determine. That value is also dependent on how companies use their research data to develop actionable plans that directly impact business results. The critical MR industry dilemma is this:
- ROI is easiest to measure when: results are focused on the short-term; baseline performance standards are known (uncertainty of business success is low); and investments are small and easily changed.
- The value of marketing research is highest when: large investments are at stake; long time horizons are involved (as in strategy development); and uncertainty of business success is high.
- For tactical market research – such as late-stage concept tests, graphics tests or messaging tests – the numbers can be readily measured and ROI can be easily calculated. Even so, it’s not always easy to obtain the business data to make the traditional ROI calculation against market research results. Sales and operational metrics are often not shared with the insights department and certainly not shared with research suppliers. How then can we “prove” ROI with this dearth of information?
For strategic, longer-term market research – such as A&U studies, segmentation or migration analysis – the business impact is tremendously difficult to measure. Because of this, a strict focus on ROI can lead organizations to conduct fewer strategic research projects, which can ultimately have a negative impact on long-term business performance.
If we focus on what is easy to measure, we explore the areas where research is least valuable. The critical issue is how to measure the true ROI for marketing research in more complex strategic situations.
Relevancy of insights
At an “ROI on MR” conference last summer, some of the major client organizations attending the event suggested that a different ROI – relevancy of insights – and their impact on business outcomes might be a different, more desirable ROI for which to strive. In order for insights organizations to showcase the value of market research, we need to strongly align research results with recommendations that directly impact business decisions. Too much research sits on shelves because no actionable plans arise from the data.
This approach would elevate the strategic role of MR and insights organizations, and highlight the benefit of exposing suppliers to the wide variety of business issues that prompt research projects. It would let business decision makers better leverage their research suppliers’ knowledge and equip them to better match research results and insights with unique business opportunities. By directly relating insights to alternative decision-making options, as well as correlating customer preferences to bottom-line purchases, the value of research will be boosted.
Pay-for-performance
The concept of pay-for-performance is also emerging as a way to assess the value of market research. Pay-for-performance is used a lot in the advertising world – for example, a company might commission an agency to create and run an ad campaign for an agreed-upon cost of $X million, setting specific reach or recall metrics for the ad. If the ad reaches the target metric, the company will be paid the full amount; if not, it will receive a percentage of the dollar amount based on the actual performance of the ad against goals.
Coca-Cola has developed a pay-for-performance system2 for its market research vendors that has internal stakeholders evaluating each research project the company conducts – based on a set of criteria covering a number of dimensions, including a quality judgment of the insights provided as well as more interpersonal aspects like the ease of working with the supplier on the project. Points are awarded to grade each project; at the end of the year Coke adds up the total points for each vendor and rewards them with a portion from a bonus pool based on those points.
Although this is a long way from measuring the dollar return of each project, this pay-for-performance system does a number of crucial things:
- It provides a consistent framework for evaluating projects. This is important because it gives the supplier a clear understanding of what the client values in a research project.
- It provides incentives for the supplier to provide more of the services the client values instead of just enough. Since better insights on a project will result in larger bonuses at the end of the year, there is incentive to put the most senior and or insightful people on that client’s business.
- It works with current practices, so that projects are still competitively bid and repeat business is still at risk if performance is poor. But, because the system provides incentives outside of the specific project in the form of an annual bonus, vendors are motivated to make the extra effort once a project is underway to prove they are better than the competing suppliers at delivering what the client values.
Actually different?
Although a pay-for-performance system uses more broadly-defined criteria for success than monetary impact, it’s a step in the direction of measuring the ROI of market research. But how is this actually different from the current state of affairs?
Currently, suppliers design studies so that the resulting research insights have value for the client’s organization (even though we can’t really know how much actual value there will be since researchers aren’t involved in the execution based on the data). Because the sales process is long and complex, research suppliers already have the incentive to develop repeat business from clients – and the best way to create repeat business is to deliver valuable insights on every study. In a way, research suppliers already operate in a pay-for-performance system, in that poor performance yields poor repeat sales.
Acknowledging the need
The discussions going on today aren’t the first time the MR industry has tackled the ROI question. But it is the first time that the industry is unified in acknowledging the need to prove the relationship of insights to outcomes. In the meantime, clients and researchers together can achieve a win-win scenario by creating a formalized evaluation system and tying incentives to meeting the evaluation criteria. It’s still a long way from true ROI, but, in my view, does a better job of achieving transparency and aligning the interests of both the supplier and the client.
References
1http://blogs.forrester.com/richard_evensen/11-04-01-market_insights_professionals_what_are_you_worth