Do we have a problem with innovation?

Editor's note: Sheila Wilson is research associate and Tim Macer is managing director at meaning ltd., a London-based research technology consultancy that carried out the study on which this article is based. A white paper on the report can be downloaded (registration required) from

Technology is continuing to transform the market research landscape, with data collection modes on the point of a very surprising switch for second place after online. Yet it appears MR has something of an innovation problem, especially over technology. These are some of the contradictory messages to emerge from the FocusVision 2015 Annual MR Technology Report by our firm, London-based meaning.

For 12 years now, we have been testing the temperature of technology as used across market research in our annual technology survey. We interview just over 200 research companies of all sizes around the world each year (206 in 2015), using a 12-15-minute online interview among those responsible for, or influential in, the technology directions of their research company.

For 2015, we focused on the research industry’s relationship with innovation, attitudes and experiences around blending qualitative and quantitative research and we also revisited what is happening with communities. We also asked our usual tracking questions about modes channels and technology choices being made across the industry. Here, we present some of the highlights in our exclusive annual review on the report for Quirk’s.

Mobile is trending and so is CAPI

Figure 1: Approximate proportion of work represented by each interviewing mode or combination of modes.

Every year, we ask our participants to consider what proportion of their quantitative work they attributed to each mode. In Figure 1, we show the major modes and also self-completion on mobile devices (not including SMS research), which looks like it is becoming a major mode.

Back in 2006, while Web was already ahead, CATI and paper between them still accounted for almost half of the volume of work. Less than a decade later, Web dominates, while CATI and paper are fading into obscurity and CAPI looks set to overtake CATI in volume very soon. Mobile self-completion has grown from barely anything to nearly 5 percent in in the last six years.

It is mobile technology, both tablet and smartphone, that is driving growth not just in mobile self-completion but CAPI too. Consumer tablets with built-in data communications have transformed the economics of switching face-to-face interviewing projects from paper to CAPI. As for CATI, it now seems that market research companies are at last moving through the complex process of divesting themselves of their CATI operations and the pain of switching CATI trackers to Web. In another question, we also observe continued rapid growth in participants taking conventional online surveys on mobile devices.

The message for market researchers and software developers alike is that mobile, with all its challenges, is now defining the direction of travel in data collection.

The complex weave of distribution methods

We have been asking MR companies what proportion of projects involves each of the main distribution methods since 2006. The one constant over time is PowerPoint’s stranglehold, while other methods continue to jockey for a position in the mix (Figure 2). With so much research today being delivered without necessarily culminating in a debrief before an actual audience, we have long questioned whether PowerPoint deserves to be the industry’s de facto standard for communicating research findings to decision-makers.

Figure 2: Percentage of projects involving these deliverables or distribution methods to the client.

Just as technology is transforming data collection, at this point we would like to report to you that companies are also shifting towards high-tech delivery methods. Interactive analysis and digital dashboards can deliver insights in ways that are more engaging, more actionable and more embedded into the research buyer’s own business. But in fact, they are not showing clear signs of taking off – although we have detected evidence since 2012 that larger companies are increasingly likely to use them.

Perhaps this will change. In another question, we ask what type of new technology research companies are looking to buy in the coming year. What comes out on top are report-publishing and delivery software.

Communities find their niche

Having explored online research communities in 2009, when they were still in their infancy, and then again in 2012, we wanted to see how they were doing in 2015.

The results show communities are now well-established in around a quarter of research companies. Only 17 percent of companies were operating at least one in 2009 and in 2012 this was unchanged at 16 percent. Three years on, it has now reached 24 percent; 41 percent of larger firms and 16 percent of smaller firms. But 18 percent of smaller firms say they are planning to introduce one. The number is also higher in the Asia-Pacific region: 32 percent run them, against 24 percent in North America and 22 percent in Europe.

Figure 3: The number of research communities being run.

The actual number of communities being run has also grown (Figure 3). In 2009 only 6 percent of companies had more than one: that has now doubled to 12 percent but it’s still rare to find anyone operating more than a handful. Just 8 percent have more than 10 and only 4 percent have more than 20.

In 2009, most companies were making do with their panel software to run communities. At that time, there was little else on the market. Now, most firms are using entirely different software from their panel software (Figure 4).

Figure 4: Type of software used to operate communities.

In further questions (not shown here) we asked about the software capabilities operators need to run their communities and how well these requirements are met. These too show considerable improvement across the board since 2012, though support is judged not as good as needed for several key features, including mobile apps for community members, co-creation tools, real-time focus groups and, with the lowest rating, text analytics.

The benefits of blending qual and quant

This year, for the first time, we asked some questions about blending qualitative and quantitative research. In the question in Figure 5, we asked what advantages companies saw in mixing qual with quant.
At a time when clients are demanding more from their research spend – and especially for it to have more impact on their business – it is encouraging that our participants are of the view that blended research offers a good way to achieve this. The top three benefits cited all check those boxes. Cost and efficiency benefits are less easily realized, it seems.

Figure 5: The advantages of blended research.

In the answers to other questions, four out of five companies said it was either “essential” or “moderately im-portant” to be able to provide blended qual and quant research and more than twice as many firms said it was “easy” rather than “difficult” to do some blended research with the technology they use. This surprised us, since most software on the market is predominantly qual or quant in its approach.

In addition, participants were asked how many quantitative projects incorporated some qualitative elements and vice versa (Figure 6). These show that blending qual and quant is widespread, though there are variations by region and company size. It is also clear that qual elements are more likely to be injected into to a quant intervention rather than the other way around.

Figure 6: Proportion of quantitative research projects containing elements which use qualitative methods or are analyzed qualitatively. And proportion of qualitative research projects containing elements which use quantitative methods or are analyzed quantitatively.

However, this overview ignores around a quarter of the sample who could not give an estimate, and among those who did, that there are some very divergent practices among companies, with a handful of stating that they blend research on all or nearly all of their projects.

Software developers may be surprised to learn the extent to which researchers are blending qual and quant and should take on board the observation that a third of companies don’t find it easy to do. Also, respondents indicate the benefits of speed and cost are proving to be elusive.

MR’s innovation problem

An industry that fails to innovate is likely to be one that will fail to survive. Today, research companies report they are under unprecedented pressure from clients to deliver more for less money. And, clients are increasingly turning to other sources of insight, from data science and forecasting to social media analysts and DIY surveys. These realities risk doing to MR what Expedia did to the travel agency.

For 2015 we decided to look at how research companies are responding to the innovation challenge. Worryingly, our conclusions are that the industry really does have an innovation problem, which is not apparent if you attend industry events showcasing innovation.

We asked our participants – all technology decision-makers or influencers in their respective firms – how innovative they thought the industry is as a whole and also how innovative they considered their own company to be. We calibrated the scale from 1 for “highly conservative” to 10 for “highly innovative” and presented it as a slider.

Overall, the score for the industry was a very mediocre 5.6 – the boundary between innovative and conserva-tive being 5.5. Companies rated themselves slightly more innovative, with a mean score of 6.1. Both scores were a fraction higher among larger companies but by global region there was no noticeable difference. We also found just over a quarter of companies rated themselves to be less innovative than their own score for the industry as a whole.

Figure 7: How are companies choosing to innovate?

The question shown in Figure 7 explored the ways the companies were attempting to innovate. Innovation can be either continuous (or incremental) or it can be discontinuous or potentially disruptive. Companies ideally need to be open to both. We also looked for evidence of active investment (e.g., putting money into it), rather than opportunistic change (e.g., picking up an idea at a conference).

The approaches mentioned most are high on incremental improvement and low on more radical, disruptive change. They are also notably low on methods involving investment. Only one in five of companies placed one of the active investment methods at the top (i.e., “provide time…”, “dedicated team” or “specific budget”).

We wondered how much the fear of disruptive innovation making MR a thing of the past might be playing out among companies. We asked if technology-driven disruptive change is viewed as an opportunity or a threat. The verdict overall (Figure 8) is a bit of both. Optimists generally outweigh pessimists by two to one but many are ambivalent.

Figure 8: What does disruptive innovation represent for the market research industry?

The picture from Asia-Pacific is very different to that of Europe and North America, with a far greater proportion seeing it as a “major threat.” In the previous question, it was also firms in Asia-Pacific that were more likely to be investing in innovation and also pursuing more discontinuous innovation approaches.

Logically, doing better what you already do can only take you so far – yet this is what the survey appears to show much of the industry is relying on, to ride out the predicted storm.