Editor’s note: Patrick Stokes is founder and CEO of Rep Data, a data collection solutions firm.
There are many factors that weigh into achieving data quality in market research, an issue that is persistent, evolving and complex. One of the best ways to tackle this challenge is by building up our knowledge about all the intricate challenges surrounding quality, so we can improve quality with things like advanced technology, better sourcing practices, expert intervention and more. If researchers truly want to trust data, we must understand every element throughout the research process that is impacting the quality of our insights.
Our latest research-on-research, conducted in partnership with DM2: Digital Marketing and Measurement, explored how audience demographics, characteristics and behaviors affect outcomes. We asked one overarching question: “How are today’s survey respondents affecting your data quality?” The project examines the potential for response differences based on traditional audience demographics such as age and gender; the type of device used to answer the survey questions; self-reported behaviors and personality types; and other key indicators.
To measure quality, we used a methodology that leverages trackable, quality-oriented question sets used for many years to determine sample provider and respondent quality and characteristics. The longevity of these question sets provided data that gave significant benchmarks for the United States, from 50K+ interviews in the past year alone. In addition, some standard questions from sources such as the U.S. Census were included to give a foundation for outside comparisons. In all, we collected n=1,800 responses on three different survey lengths (7, 12 and 18 minutes), with completes divided evenly by survey length and device type (mobile vs. tablet/laptop/desktop), as well as reflecting consistent age and gender quotas.
What we found w...