In one of the more enjoyable work-related trips I’ve taken recently, I traveled to Budapest this past April to attend an ESOMAR conference on panel research. Late last year we decided to add a directory of panel research providers and an accompanying editorial emphasis to our 2005 publication schedule. So I was happy to see that ESOMAR had added a panel research event to its always-solid conference lineup and doubly happy that it was in Budapest !
The chilly, rainy weather was perfect for conference-going but what I heard and saw over the two-day event was anything but gloomy. (The rain lifted later in the week and allowed my wife and I a few rewarding days of sightseeing in a fascinating city. If you’ve never been, you must go. Friendly people, a rich history, great public transportation and memorable food and drink - what more could you ask for?)
At over 200 people, attendance was about double what organizers had expected - testament no doubt to the ever-growing role of panels in the research process. With response rates to telephone research falling and Internet adoption rates climbing, panel research and other online approaches represent a golden opportunity for the industry. Presenters and attendees in Budapest repeatedly acknowledged this fact and there seemed to be a strong drive to get things right at the outset, establishing panel recruitment and management best practices to ensure the long-term health of the methodology.
Kees de Jong, the event’s program chairperson and CEO of Bloomerce Access Panels, set the tone with his opening remarks, raising a number of important points. Perhaps the most important was this: panels are people. While panel-building and maintenance require the delicate balancing of three factors - money, quality and resources (the respondents) - everything the industry does should take the respondents into consideration. Data quality is key, as is an efficient cost structure, but without respondents to take the surveys, nothing else matters.
The conference had good case history examples of companies using panels, including Hewlett-Packard, DHL, British Sky Broadcasting and General Mills. Jeff Hunter, director of consumer insights for General Mills, spoke from the client’s perspective on the evolution of panel research. The food-products maker has long used mail panels but has gradually moved to the point where 80 to 85 percent of its panels are now online, Hunter said. Perhaps because of the speed and flexibility of online panels, Hunter said the firm does more strategic research now. The mix used to be one-third strategic and two-thirds tactical and now it’s two-thirds strategic and one-third tactical.
Some lessons General Mills has learned: despite the Web’s visual capabilities, packaging research is something best done in a store setting or at least offline; in an online panel, respondent tenure matters - research on research has shown that responses change over time; targeted proprietary panels - which function like online communities - show promise; many online panel members are members of multiple panels and more work needs to be done to gauge the impact of this fact, both on biases that may be introduced and on respondent fatigue.
Conference tracks included papers on the effects of new technologies, developing and keeping panelists, and panel management and quality issues. Pete Comley, chairman of U.K.-based Virtual Surveys, presented a research-on-research paper that examined findings from various studies on why people participate in research panels. He identified four respondent segments: helpers, opinionateds, professionals and the incentivized. Helpers, as the name suggests, enjoy doing surveys and will do them in many cases without payment. They skew older and enjoy being part of an online community. Opinionateds want their views to be heard and enjoy expressing their views. They will sometimes do surveys without compensation. The professionals are the most motivated by financial or other tangible rewards. They don’t care as much about the survey topic. They skew male and tend to do the most surveys and work with multiple survey firms. The incentivized group is the most motivated by prizes and other awards for completing surveys. They are younger and do a lot of surveys.
Given that respondents have so many reasons for completing surveys, Comley suggested that panel firms think about some kind of targeted incentive approach, which would give respondents micro-payments for each survey and other “softer” benefits tailored to their preferences. He said that using sweepstakes as the main motivator should be avoided, as research showed that many respondents doubt the prizes are ever awarded since they themselves have never won one of the larger prizes. The screening process should also be improved so that respondents who don’t qualify learn their fate early and don’t have to waste time answering several questions only to find out they don’t meet survey guidelines.
The most interesting portion of Comley’s presentation was his proposal for a common incentive scheme, under which panel firms would work together on a system that would monitor respondents (and perhaps reduce the number of surveys completed by the “professional” respondents) and also serve as a national register of those willing to take part in online surveys. This approach would reduce respondent acquisition costs (panel firms are paying repeatedly to recruit the same individuals on multiple panels) and also make it harder for people to join the same panel more than once.
Thought-provoking
De Jong and the rest of the conference organizers should be congratulated for assembling such a substantive and thought-provoking conference. After two days of listening to bright, inquisitive researchers explain and explore ways to make a part of their profession better, it’s hard not to come away impressed and energized. If the rest of the industry puts the same care and commitment toward panel research as I witnessed in Budapest , the methodology should have a bright future indeed.