Questions you should ask when selecting a panel provider
The November Quirk's features a directory of research companies that provide panels and panel software. Based on contributions and information from Dallas research company e-Rewards Inc. and ESOMAR, Amsterdam, the Netherlands, here are some questions to ask that can help you select the panel provider that's right for the project at hand.
Recruitment
There are two widely-used methodologies used to recruit panel members into an online panel. Open-source recruitment allows for anyone with access to the Internet to enroll into a panel. Closed, or by-invitation-only, recruitment is a method of inviting only pre-validated individuals or individuals who possess known characteristics to enroll into a market research panel.
Opting in
The opt-in process might indicate the respondents’ relationship with the panel provider. Double opt-in enrollment describes the process by which a check is made to confirm that the person joining the panel wishes to be a member and understands what to expect.
Sources
Is the panel actively managed or database-driven? Actively-managed panels consist of panel members who voluntarily opt in to participate in a panel and receive invitations for online research surveys. Database contact lists consist of large-scale repositories of e-mail addresses obtained from various sources. Little to no prior agreement and/or consent to receive market research survey invitations has been given by individuals on these lists. Actively-managed panels have been shown to produce higher response rates and data quality.
Compensation
How are panel members compensated? What are people told during recruitment? How does this vary by length of interview, respondent characteristics, etc? The type of rewards and proposition could influence the type of people who agree to answer a questionnaire or join a specific panel and can therefore influence sample quality. Panel providers utilize two primary methods for incentive fulfillment: guaranteed cash/reward incentive or sweepstakes prize drawing. Cash/reward incentives are awarded upon survey completion. The sweepstakes incentive method awards respondents by entering their name into a prize drawing for completing a survey. Prize drawings are done periodically and the number of eligible contestants varies. Guaranteed cash/reward incentives have been proven to yield higher response rates.
Profiling
How often is profile information updated? What profile data is kept on panel members? How often is this data updated? Extended and up-to-date profile data increases the effectiveness of low-incidence sampling and reduces pre-screening of panelists. If you are seeking to reach a low-incidence or hard-to-reach group, a panel provider with extensive targeting is better equipped to fulfill your request.
Contact frequency
How often are members contacted or invited to participate in surveys? Panel providers that frequently blast their panel members with survey invitations risk falling into the category of junk mail in the minds of individuals. Moreover, panel providers that allow panel members to participate in multiple surveys within a short period of time can increase the number of hyperactive and professional respondents that complete surveys. Seek a sample provider that limits the number of times a panel member is invited to participate in surveys over a given period of time.
Quality control
What guarantees are there to guard against bad data (i.e., respondent cheating or not concentrating)? Do you have an identity confirmation procedure? Do you have procedures to detect fraudulent respondents at the time of registration? To ensure the quality and reliability of the data, choose a panel provider that employs a quality management system. Quality management systems vary by panel provider but consist of tools and processes to identify and eliminate duplicate members and undesirable respondents from their panel.
Special thanks to e-Rewards Inc., Dallas, and ESOMAR, Amsterdam, The Netherlands, for providing input on the questions.