SPONSORED CONTENT
JD Deitch
Chief Revenue Officer
P2Sample
Times are changing and so is the sample industry. So why do we keep focusing on things that don’t matter anymore? While our end goals remain the same – wide, diverse reach; quality; and dependability – how we get there has changed drastically. The fact is that most people are operating on dated views of the relevant issues and have only a dim understanding of how the proverbial sausage is made. We need to get beyond simplistic and dated views of the space and address the fundamental changes brought on by the digital age. To truly be able to judge suppliers, here are the factors you need to consider.
Old thinking around reach
In the past, reach meant big proprietary panels and numbers in panel books, which often had little to do with real-world project feasibility. Many still believe this is how the industry runs and nothing could be further from the truth.
Reach today: Variety of recruitment sources. Due to the fact that true probability sampling is expensive, difficult and nearly impossible to achieve, our industry uses the relative ease of convenience sampling and has done so for years. With this in mind, we can take a step back and look at our recruitment methods specifically with a grain of salt. Whereas many still feel double opt-in panels are the “gold standard,” we have the opportunity to reach a deeply profiled, broad and diverse audience by using automation and various forms of recruitment.
Skepticism whirls around the idea of river sample, which earned a bad name a couple decades ago due to fundamental problems in execution. Yet, even companies that are the staunchest advocates of more “acceptable” techniques like double opt-in panels are now openly admitting they use real-time sample sources. The truth is that whether respondents have joined a panel or are just “dropping by” to do a survey in real-time, sample matters little. Both recruitment methods have pros and cons. What really matters is that we garner real, verifiable, engaged respondents from multiple sources to create diversity.
Automation allows this. From online forums and blogs to search engines, shopping sites to social networks, from gaming sites to app stores, it is possible to recruit a wide variety of real, verified people from hundreds of sources while managing quality and costs. When automation is properly implemented in the recruitment process, it improves everything else downstream, from feasibility estimates to response times to economics for suppliers and buyers. A limited number of recruitment partners (and not simply the use of much-maligned real-time sample) should raise a red flag, as representativity can plummet in this scenario.
Reach today: Beyond the panel book. Sample suppliers need to be far more transparent about recruitment – their processes, technology and more – far beyond what’s included in the traditional “panel book.” Not only should suppliers freely answer questions about respondent recruitment and quality assurance but they need to offer greater detail on the actual feasibility of the respondents garnered for any given project. There are several ways to get a better understanding of a supplier’s capabilities. For example, we have implemented a dynamic, clickable, constantly-updated map that clearly illustrates how many of our 40+ million opted-in panel members in each of the 150 countries that we serve have been active in the past 90 days. This is just one solution that takes a step toward the transparency that the industry needs to demand in order to improve accuracy and results.
Old thinking around quality
Gone are the days we can accept suppliers’ assurances that they have proprietary systems that handle these things. We must ask them to prove it and require concrete, dynamic and sophisticated methods that leverage advanced technology on both these fronts.
Quality today: Treating respondents right. When we talk about respondent experience in the market research industry, it spans the entire respondent lifecycle and doesn’t begin or end with survey design as many would have us believe. Sample suppliers can play a large role by taking proactive measures to ensure respondent engagement and satisfaction. A lot of this hinges on giving respondents greater control over their own experiences, even rating surveys while in-field.
Sample suppliers are not immune from practices that create negative user experiences. Whatever the cause of these bad experiences, suppliers must directly manage the process in-field to make it stop. This includes close real-time monitoring of projects, capturing data from field statistics to respondent ratings of the study and dealing with it accordingly. Automation is our best path to unlocking these capabilities. For example, studies where respondents are able to complete a survey efficiently and with minimal pain should see greater sample flow. Conversely, bad studies progressively need to have their sample flow restricted, be quarantined for triage and even shut down if they are bad enough.
The bottom line is that we all want to reduce needless respondent churn and waste while gaining better data. Use machine learning and AI to target engaged and qualified respondents. Machine learning and AI can play role in this by boosting respondent targeting on the front-end, further helping find the survey for which the respondent is most likely to qualify for and complete. Thoughtfully applying technologies like these are meaningful steps toward optimizing the respondent experience.
Quality today: Fighting fraud from multiple angles. Fraud is not just a series of “bots” that can be thwarted by traditional methods. In fact, as technology advances, decidedly smart, sophisticated and savvy human fraudsters are increasingly able to break through the minimal defenses on which our industry has historically relied. Dynamic fraud detection can drastically improve fraud mitigation rates. Static techniques include things like: e-mail and address verification; Captcha blocks that require human intervention; honeypots that trap machines; and open-end questions that weed out spammy answers. These are not sufficient, even when used in concert.
Again, advanced technology can save the day here. Artificial intelligence (machine learning and deep learning in particular) can sift through the massive amounts of data and detect in real-time the ever-changing profile of fraudsters. Because this technology is self-learning, it will continue to find new patterns and anomalies in those patterns as long as it is fed billions of data points, is provided with a large amount of historical data to help it learn and is paired with humans who provide a solid foundation of industry knowledge. Dynamic methods like this are essential to fighting fraud today.
Old thinking around dependability
Yesterday’s views around dependability for sample were rudimentary: if your study was falling short, you wanted to be able to reach out to your vendor for more. You paid handsomely for this: a top-off of 10 respondents might trigger a $1,000 project minimum or empty reassurances that help was on the way.
Dependability today: automation, backed by smart people. A huge part of field failure arises because detection of that failure was a manual effort. Automation can now proactively detect and address problems immediately, alerting people that action needs to be taken before it’s too late. Now, instead of needing the humans to simply detect the problem (typically when it is too late), the humans can add good judgment.
End-to-end automation improves quality, reach and dependability
If you take nothing else away from this article, take this: Everything should be automated when it comes to sample. Automation actually has the potential to take us far beyond simple time savings and solve many other common problems when properly implemented. As we covered above, it has the power to vastly improve reliability, efficiency, global reach and feasibility and dependability. Though currently underutilized in the industry, automation is what matters. When functions from recruitment through bidding and feasibility to project execution are automated, sample companies can achieve the height of operational efficiency. This technology plays a role in every aspect of the sample continuum.
In fact, P2Sample now offers the industry’s only programmatic panel and sample management platform that is maximally automated from end-to-end. This means, with the launch of our feasibility API last fall, we have a fully automated approach from start to finish, plus machine learning investments in both respondent experience and fraud mitigation. Benefits to this kind of approach are vast and affect every aspect of sample we discuss in this article, boosting agility, effectiveness and economics.
Instead of focusing on old, outdated methods, we should start demanding automation that goes beyond common limited applications. Automation can have massive benefits for process and outcomes, as well as better data and agility, when sample companies use this technology throughout both their internal and external processes.
Capabilities are far greater
Are you still holding on to old thinking surrounding data quality? Think again. Capabilities surrounding sample reach, quality and dependability are far greater than they were in the past. We need to start taking a closer look at approaches to these key issues and recognize sample for what it is: a core pillar of what we do as market researchers, providing the foundation for quality insights. Especially with greater usage of automation, sample suppliers are ready to be a true partner in evolving and meeting the changing needs of the industry. These are hard changes though, which is why many suppliers have fallen back on incremental improvements that lead to the perception of commoditization. We must demand solid partnerships and sample companies need to illustrate a commitment to change in order to improve data quality.
So why do we hold on to old ways of thinking? Technology advancements across the board allow us to deliver better results across the board. Let’s agree to start focusing on new, technology-driven ways of achieving fundamental goals and this will bring our industry forward.
About the author: JD is a 20-year veteran of the market research industry who currently serves as CRO of P2Sample. He began his career client-side in financial services, then went on to hold senior global positions at The NPD Group and Ipsos. Prior to P2Sample, he held executive positions at AYTM and Bakamo.Social. JD is a frequent speaker and a thought leader in the insights industry as it evolves to tackle the challenges of the digital age. He has a Ph.D. in political science with distinction from The American University and a B.A. from the University of Pennsylvania. He lives in France with his wife and two sons.