What's my motivation?

Editor’s note: Christopher De Angelis is vice president of sales at Survey Sampling International, Fairfield, Conn.

Research professionals have a common interest in truly understanding research respondents. We have a shared goal in respecting, protecting and creating goodwill among the survey respondent base. One of the interesting things about an Internet sample panel is that, by its very nature, a panel encourages two-way communication. Survey Sampling International (SSI), as part of its panel management practices, has an ongoing commitment to understand and respond to panelists’ likes, dislikes, frustrations and motivations.

While the reactions of panelists to different offers and opportunities are varied, patterns emerge in the Internet survey experience. Research with panelists provides insight into the format, presentation and style elements that garner the highest-quality results. This article deals with understanding the behavior of Internet panel respondents and the factors that combine to create a successful interview experience for both respondents and researchers.

Motivations for joining an Internet research panel

Press the lever, get a banana. Press the lever again, get another banana. This characterizes the way some people view panelist motivations and reward systems. But is it really an accurate view of what motivates panelists? In April 2004, 6,000 SurveySpot panelists responded to one or more panel management surveys. These surveys were administered by SSI to measure satisfaction with the panelist experience and to help provide additional metrics for managing the SurveySpot panel most effectively. One of the questions SSI asked was:

“What is the primary reason you joined SurveySpot?”

  • 22 percent said to influence decisions and the designs of products and services.
  • 26 percent said they like to share their opinions with others.
  • 47 percent said it was the opportunity to make some money while giving their opinion.
  • 2 percent fell into the “other” camp.
  • 3 percent were not sure.

Almost half of all panelists said they were intrinsically motivated to join a panel and share their opinions. Panel members respond positively to the opportunity to see how their opinions relate to the opinions of others. Not only is this feedback conducive to increased participation, it’s less costly than direct incentives. However, this motivation is a fragile thing, and can be easily damaged if trust is broken as a result of lengthy surveys, technical problems or other factors that burden the respondent.

Motivations for taking a survey

SurveySpot panelists provided feedback on their experiences and motivations:

“What is the most important factor in determining if you will take a SurveySpot survey?”

  • 23 percent said an interesting topic.
  • 22 percent said an invitation received at a convenient time.
  • 21 percent said the prize or reward offered.
  • 18 percent said the length of the survey.
  • 7 percent were not sure.
  • 8 percent said “other.”

Topic and convenience, not surprisingly, were among the most important factors. This is consistent with the findings of other research studies such as the 2003 CMOR Respondent Cooperation and Image study. The influence of prizes or rewards was a top choice, but not a stronger influence than either convenience or interest in topic. Almost one in five said survey length was of importance in deciding to take a survey.

“In your opinion, what is the ideal survey length?”

  • 2 percent said less than two minutes.
  • 21 percent said two to five minutes.
  • 44 percent said six to 10 minutes.
  • 21 percent said 11 to 15 minutes.
  • 3 percent said 16 to 25 minutes.
  • 0 percent said 26 minutes or more.
  • 8 percent said there is no ideal length.
  • 1 percent were not sure.

One thing is clear from this and other research SSI has performed on Internet studies: Respondents overwhelmingly prefer short surveys. Fully 88 percent of panelists in our survey indicated a preference for surveys lasting 15 minutes or less. If you have a short survey, state the length in the invitation and this will encourage participation. Invitations for surveys that advertise a “short” or “brief” survey that takes 20-25 minutes generate a lot of panelist frustration. Be honest about the actual length of the survey experience. Respondents are saying, “I want to share my opinion, but I prefer to do it for surveys that are interesting to me, received at a convenient time, and are of a reasonable length.”

There are several forces working against longer surveys, including: lack of interest, higher mid-survey abandonment rates, and increased incentive costs. Remember that incentives/rewards were a motivation for joining a panel for less than half of survey respondents, and a factor in responding to a specific survey only about 20 percent of the time.

Further support for the finding that incentives and rewards are not the primary motivation for some panelists is suggested by responses to the following question:

“What is the primary reason you choose NOT to take part in a SurveySpot survey?”

  • 35 percent said it was an inconvenient time.
  • 6 percent said the subject was not interesting.
  • 25 percent had trouble accessing the survey.
  • 2 percent said the instructions were confusing.
  • 16 percent said the survey length.
  • 9 percent said “other.”
  • 7 percent were not sure.

The fact that 25 percent of respondents indicated they had trouble accessing a survey suggests that there is work to be done to make the survey experience smoother for willing respondents. Increasing server capacity to accommodate higher volumes of active panelists with open concurrent sessions would be in the best interest of everyone involved in the research process.

Another view of panelists’ motivations for taking surveys is found in the following results from reward testing. SurveySpot uses a $10,000 monthly prize drawing in which over 100 cash prizes are awarded to panelists every month. The program rewards panelists who complete surveys and those who attempt to complete a survey but do not qualify with entries into the monthly $10,000 prize drawing.

In this test, the invitation to take the survey stated both the survey length and the incentive being offered. A control and two test cells were randomly selected from across the panel. The objective was to measure click-to-start rates of panelists opening and starting the survey with three promised incentive offers:

  • an entry into the $10,000 monthly prize drawing,
  • an entry into the $10,000 monthly prize drawing plus a $300 project-specific prize drawing from among all completes, and
  • an offer of $1 for completing the survey.

Click-to-start rates for the $10,000 prize drawing plus the $300 project-specific drawing were about 5 percent higher than for the $10,000 prize drawing alone. Click-to-start rates for the $1 incentive alone were about 8 percent higher than for the $10,000 prize drawing alone.

In essence, the needle moves - albeit slightly - when a small project-specific prize drawing or a $1 incentive is offered. The monthly $10,000 SurveySpot prize drawing offers the intrinsically motivated panelist and the panelist interested in an opportunity to make some money while sharing opinions reason enough to begin a survey. Research agencies benefit by not needing to shoulder the added costs of incentives for shorter surveys with topics of general interest. Further, panelists themselves remain motivated to take surveys without requiring an incentive every time.

Panelists matter

The information that flows from respondents provides the insight and direction our clients need for successful, actionable survey research. Respondent needs are simple: Tell it to them straight. Keep surveys less than 15 minutes long. Provide feedback, if possible. Offer an opportunity to be rewarded, and make sure if you lead respondents into the survey process they can access the survey and make their opinions heard.

Remember that the basic premise of Internet-based research is that it stems from an opt-in willingness of respondents to be part of the process. On the Web, respondents are just one mouse click away from doing something else. It’s up to the research professional to ensure they don’t exercise that option!