Don't fear disruption, marketing research firms. Embrace it.

Editor’s note: Based in London, Richard Thornton is the global sales and operations director at Cint, a Stockholm research firm.

Without a doubt, the trifecta of big data, new technology and social media has disrupted the traditional market research industry as we know it. A study on global market research growth recently published by ESOMAR showed a considerable upswing of 23 percent in the market for new approaches, while spending on traditional practices only grew by 2 percent over the three-year period measured. This would hint that a Big Bang-level disruption is on the way for the industry.

However, the answers become less clear when it comes to responding to this upheaval and weighing the positives against the negatives. On one hand, the significant upsurge in depth and access to consumer insight across the Internet, whether by piecing together a digital footprint or analyzing conversations taking place on social networks, has put our industry in the spotlight – this sudden abundance has created a growing thirst for information as more and more companies recognize the value of insights for decision-making at all levels.

On the other hand, the competition posed by these new players challenges the fundamental business model of many market research firms and those involved with data collection. In order to survive, these agencies must deliver more for their clients, all while operating within restraints posed by downward pressure on pricing and reductions in budget and resources. To meet this challenge, market research agencies need to enhance efficiency, deliver more value for all parties involved in the process and look for savvy solutions.

Rather than fear the new technologies behind this shift in the industry, firms should embrace them. Change may also be forced. Many organizations today are under increased pressure in terms of extracting value out of their existing business models while maintaining operational margins and profitability. One of the most effective ways to do this is to focus on parts of processes and ways of doing things that can be automated, normally through technology. In market research, we are seeing this play out in the way organizations up and down the value chain are reviewing their strategies and workflow when it comes to supply chain management and procurement.

Seamless API integration options, for example, present a valuable opportunity to streamline the entire process of sourcing and delivering insights. (API stands for “application programming interface” – a set of programming instructions for accessing Web-based software so software components understand how to interact with each other). By standardizing some steps and automating others, API “connections” between different platforms and software can save time and energy every step of the way. Additionally, these solutions offer increasing levels of control, transparency and flexibility when accessing intelligence through open panel marketplaces – all of which can combine to help agencies save time and plan expenditures more strategically, thus helping them to deliver on-budget.

For those agencies managing their market research panels, the true costs of conducting research can sometimes be hard to predict and quantify, ranging from recruitment and sourcing costs, which are generally on the up, to the resources involved in ongoing maintenance and management involved in keeping a panel asset healthy and engaged. This is why many buyers of sample are looking towards sourcing from more open, platform-like marketplaces, whilst suppliers are increasingly understanding the benefit of hooking up to such platforms to create a more efficient distribution of their excess “inventory” of respondents.

By accessing reputable double-opt-in research only panels through a hub of open market research communities, agencies can source sample in a very transparent and controlled environment, with fixed pricing, accurate feasibility around delivery and with the opportunity to tap into millions of people’s opinions extremely quickly. Further potential efficiencies can be gained by blending samples from different panels, either by combining insights from the agency’s own panelists with answers from another or by incorporating responses from a few trusted panels representing different demographics to create a more comprehensive picture at a lower comparative cost. This can also help address single source bias within a panel or particular supply source.

As the industry adjusts to this new landscape of information sourcing, one key ingredient remains the same: quality. This applies not only to the data being gathered and the insights being parsed but also to the experience of everyone involved in market research interactions. Engaged and committed respondents, especially those who have had positive experiences previously when responding to surveys, participate more frequently and provide more valuable intelligence than those who feel disenchanted, disinterested or noncommittal. Not rocket science of course but it is still extremely frustrating to see a research process in many cases where very little consideration is given to the respondent experience, from questionnaire design to appropriateness of methodology, all the way down the supply chain to how a survey is presented to a respondent.

With the increasing sophistication of survey technology, there has been a remarkable proliferation of online panel communities but they are not all created equal. Many stand-alone survey Web sites offer payments and rewards for anyone to sign up and complete surveys, without stringent screening procedures or techniques in place to ensure honest and reliable answers. While it may be inexpensive to access information from these types of panels, it often can be difficult to identify the companies responsible for running the sites, the entities administering the surveys and the detailed demographics of people answering them. This “catchall” approach and anonymity can undermine trust and potentially cast doubt on the validity of the results.

Fortunately, there are also an increasing number of publishers, brands, non-profits and other organizations that have created open market research panel communities of readers, consumers, supporters and other individuals who feel an affinity for their work and a connection to their brands. As these groups use their panels for their own research purposes, they have a vested interest in keeping the experience relevant, engaging and non-intrusive. Many of these types of panels are also now connecting to panel platforms and marketplaces to offer up further survey opportunities and rewards for participation, which enables the broader market research community and beyond the benefit of tapping into these valuable and affinity-based panels to gather market insight.

Just because the industry is changing doesn’t mean market research agencies must reinvent the wheel. In fact, studies have shown that many of the most successful ideas are not completely original but rather a combination of current knowledge or practice with a twist of novelty thrown in. This means that coming up with a savvy response to the changing climate should involve building on the inherent strengths of longstanding processes, products and experience – then adding a layer of innovation and mixing things up a bit to create a dynamic solution. Change should be embraced as a challenge to celebrate the traditional methods, the power of past successes and the knowledge gained with time and practice by integrating them with the exciting new opportunities flourishing in the industry today – forming innovative offerings that succeed with the just the right combination of the old and the new.