Editor’s note: Christine Holt is research director of the InVision Qualitative division of C&R Research, Chicago. Liria Barbosa is senior qualitative analyst in the firm’s LatinoEyes Qualitative division.
There’s no question that there are many respondents out there, participating in qualitative and quantitative studies, who are being contacted too often. It’s likely that some of these individuals are “trained” - they have been exposed often enough to screeners, groups or surveys that they know how things work and what question’s coming next. We’ve been stopped many times after groups and had respondents say, as a means of praise, “Wow, that was really fun. So different from the other ones I’ve done.” Really? That always catches our attention because we wonder exactly what they’ve been exposed to, how long ago and how often.
But what worries us more than the frequent participant who’s earnest in sharing their point of view, who’s qualified honestly, and has done a few groups before, is the lady at the table who hasn’t qualified honestly, doesn’t really use the product in question, and is more interested in whether time is about up so she can collect her check and be on her merry way.
These folks, the professional respondents, don’t contribute to the research process. In fact, it’s difficult for them to actually participate because they often don’t have any familiarity with the issues at hand. This, in turn, signals the moderator, unwittingly, to try to draw them out, which detracts from getting fuller responses from the “real” participants.
Even worse, though, is if they do take an active role in the group. They’re unable to provide honest feedback, which can give us a false sense of what’s happening - not to mention their potential impact on other respondents. Still other professional respondents can be quite aggressive, since they may feel they are experts on the focus group process. They may give answers they believe will move the group along or that will satisfy the moderator, “Is that what you were going for?” In contrast, the earnest respondent is participating because she wants to have a voice, to have an impact, to feel a part of something bigger than her. Plus, she can be a real asset in the room because she’s familiar and comfortable with the process. She can jump right in and participate fully.
What to do?
So what can we, as qualitative researchers and moderators, do to ensure that we’re at least getting earnest respondents at the table, even if some of them have participated before? First and foremost, we must take care to screen carefully and specifically. Organizations such as ESOMAR provide us with guidelines regarding past participation screening for qualitative research. So, as an industry, we typically screen for no participation in other groups in the past six months. ESOMAR also recommends that we screen for no participation in the past two years in the same or a related category. It also recommends that we cap the total number of group experiences to five in the past five years. And finally, to increase our odds of not seating that less-than-earnest respondent, ESOMAR suggests that we strive for half of each group to be virgins to the process - no past group participation.
While these are great recommendations, we all know that in the real world it’s not going to be feasible to follow them to the letter - particularly the last recommendation about seating newbies. If we make this a recruiting requirement, our costs are going be substantially higher and the time it takes to finish recruiting will be longer than our clients can tolerate.
So, we rely on our qualitative partners, the recruiters and facilities, to help reduce the incidence of professional respondents by being diligent about maintaining accurate and objective records in their databases regarding potential respondents’ past participation and reported dishonesty. It’s our responsibility as moderators to share with the recruiter or facility any concerns we have about any respondent, based on things we observed or heard in the groups. In turn, it’s the facility or recruiter’s responsibility to follow up on those concerns, including removing that individual from the database.
We can also help to discourage the professional respondent by requiring all respondents to have their ID checked upon arrival at the facility and by formally questioning them about whether they work with other facilities. As moderators, we can ensure that our screeners are fully blinded and open, offering as few clues as possible to our purpose. We can ask respondents to bring the product to the groups or require them to complete homework about the product that demonstrates their level of involvement - and refuse to seat those who forget their assignments. Facilities and recruiters could more proactively work to expand their databases (to bring in “new blood”), control contact frequency and actively purge or put on hold the most heavily used respondents or those with whom there have been issues. In the ideal world, facilities would be more compelled to participate in existing sample validation programs sharing contact information on the respondent level with each other. Of course this sometimes happens on a voluntary basis now, but an industry-wide initiative along these lines would be a big step forward.
Positive impact
In the meantime, by focusing on the “professional” respondent, and not overly concerning ourselves with the earnest respondent with some history of past participation, we can truly make a positive impact on the net result of the focus group. We can be satisfied that we have exercised an appropriate level of control in an effort to seat as productive a group of consumers as possible. Then we can focus on the real task at hand: understanding their impressions and motivations toward the category or product at hand to help our clients bring the most consumer-focused products or service to market as possible.