SPONSORED CONTENT
Can surveys for market research be trusted?
For the practice of survey research, in many respects, it is both the best of times and the worst of times. Never has the world been more invested in data-based decision-making. We are addicted to data and for good reason: All else equal, the more data, the better the decision. And the more effective the intervention, the more powerful the insight. Sure, big data has (perhaps) supplanted the role of survey research for some use cases and applications. But there is plenty of data-demand to go around and surveys continue to be the best tool to uncover the attitudes, opinions and psychology that drive consumer and other behaviors.
Yet while we reap the riches of a data-driven world, we face a crisis in confidence. Survey response rates are at an all-time low and convenience (nonprobability) samples continue to display higher error properties than other data collection strategies. But more critically, the survey research field met its watershed moment in the large-scale errors experienced in the 2020 election. As they say, fool me once (errors during the 2016 election), but fool me twice (2020)?
We cannot avoid the reality anymore: survey research is experiencing biases that are deeper than ever before. In the past, it was generally accepted that these were demographically driven and survey weighting took care of that. Today? We are starting to understand that biases are attitudinally driven as well. For example, people low in social capital, connectedness and/or trust appear to be missing in survey data. Insofar as what you are measuring is associated with these factors (e.g., the elections), bias is all but guaranteed.
The field of survey research is robust and adaptive. We have survived many changes in technology and culture, and many talented methodologists are making great strides to meet the new challenge head on. At NORC, where I serve as a chief scientist and SVP, we have utilized our AmeriSpeak panel and developed TrueNorth, the most effective tool available today in reducing bias in convenience samples for market research. We, along with others, have taken a long, hard look at the 2020 election and are developing tools to ensure far greater accuracy in the future.Â
The art of survey science is advancing and will continue to prosper, grow and, most importantly, find methods to ensure the data it produces is valid and reliable. Marketing departments and corporations are making decisions of real consequence and require the data upon which such decisions are made to be accurate and true. Survey research is up to the challenge and will serve marketers and business decision makers for decades to come.
David Dutwin