Editor's note: Jerry Savage is the proprietor of Seattle-based ResearchSight.
The widespread availability of online survey technology has opened important doors for product developers and marketing managers who seek to develop customer-centric strategies that will help them succeed in a challenging marketplace. However, the Internet is still a largely uncharted social space that harbors the well-intentioned as well as others who participate in surveys merely to make money or to get inside information on new product and marketing concepts. Indeed, the Internet could be likened to the Wild West of primary market research: a place of great opportunity and innovation, yet one which also poses many unique challenges.
It seems fair to conclude that there are two critical elements of any quality survey: 1) a sample that is representative of the population being studied and 2) survey questions or “items” that garner valid and reliable responses from participants. The quality of the sample is arguably the more important of the two and it is precisely this issue which poses some of the biggest challenges in online survey research. While online surveys using established consumer panels as a sample source are an enticing option for companies that want to survey various segments of the marketplace, cheaters (i.e., individuals who lie in surveys to acquire an incentive or get information) can pose a serious threat to data integrity and undermine the generalizability of results.
As such, questions of sample integrity are at the core of methodological questions about how to use the Internet to acquire accurate and actionable business intelligence. While not all companies are faced with exactly the same questions, there are strategies that can be applied by most analysts and managers when analyzing the integrity of online survey samples:
1. Be diligent. The first step to ensuring sample integrity in any online...