There are benefits to both

Editor’s note: Matthew Towers is the founder of Towers Research Services, San Francisco. He wishes to thank Jeff Walkowski and David Van Nuys for their help preparing this article.

When bulletin-board focus groups (BBFGs) started gaining popularity, some qualitative researchers were quick to point out why online groups are not the same as “real groups,” implying that online groups were somehow inferior. Many articles and discussions focused on comparing BBFGs with in-person focus groups, extolling the virtues of each and but often highlighting the deficiencies of BBFGs. Champions of online groups countered with long lists of potential benefits as an apology for the “deficiencies” of online research. However, in practice, the trade-offs one does make are very often offset by the advantages of online groups.

In writing this article, the intention is not to diminish the validity or usefulness of focus groups but to challenge some of the assumptions and beliefs about in-person groups and to demonstrate some of the less publicized attributes of bulletin-board groups. Most experienced, trained moderators know how to use non-verbal cues, reduce the impact of dominant respondents, make respondents comfortable in an artificial environment, probe properly, and encourage timid respondents to participate. However, this does not mean that these challenges do not exist, just that good moderators have developed techniques for coping with them.

Challenges of in-person groups

Research observers need to see respondents’ facial expressions and body language to gain an in-depth understanding of what respondents are thinking.

As a researcher, RIVA- and NLP-trained, I find it an interesting challenge to read respondents. Sometimes I do get non-verbal clues that indicate when it would be effective or inappropriate to drill down, but the flow of the conversation is my greatest hint about when to probe or change direction. Unless one is particularly adept or well-trained at reading body language, how accurate is the interpretation is likely to be? When it comes time to write a report, I report what my clients and I hear or what I can glean from audio and videotapes.

The ability to manage groups is an essential part of being a good moderator; reading body language helps moderators do this. Reading non-verbal cues may help to get more in-depth information from respondents, but it is not the only way to extract the same level of information and it is not without the risk of misinterpretation.

Focus groups are a neutral environment.

While qualitative researchers are aware that facilities are an artificial environment, most research is still conducted in a traditional facility setting. The only people for whom this is a natural setting are facility staff, moderators and their clients. Facilities provide a controlled (mostly) environment where research can be conducted with a minimum of influence from extraneous variables. In this controlled environment, a respondent’s physical context and at least some of the psychological context, is lost. Online research allows respondents to choose where and when they participate. Many studies have been conducted for technical products where the normal environment for the respondents would have been in an office or at home in front of their computers. While online research venues may or may not be artificial, focus group facilities certainly are.

Face-to-face interviews are the most effective way to stimulate response from a group.

While this may be true in general, it comes at a cost. Observers often hang on every word one respondent says and ignore the rest of the group. At times observers or group members disregard a respondent’s contributions because they look different or act strangely. People often pay more attention to outspoken, articulate participants, and ignore the rest without any proof that gregarious people buy more.

It is the moderator’s job to manage how observers react to and use information presented during groups. Good moderators can minimize, but not eliminate the effect of the dominant respondents. Good moderators can encourage shy respondents to come out of their shells, but can they make respondents think faster or instantly become more comfortable in groups? Good moderators cannot remove prejudice; at best they can make observers aware that it exists.

Focus groups rely mostly on top-of-mind thinking.

Sometimes research clients insist on packing the discussion guide with too many questions, leaving moderators with little time to probe in-depth. In such a group, moderators ask participants to respond to a question or other stimulus, and eagerly wait for the response. Many moderators tend to anticipate and reward quick thinking. Do quick thinkers buy more products? Are their perceptions more accurate? This is not an inherent fault of focus groups, but it is a reality that many, if not most, moderators face.

People who are articulate have more valid points to make.

Focus groups rely almost solely on verbal information from respondents. Respondents who are more comfortable writing than speaking are largely left out of the equation or at least marginalized. In fairness, it is true that with online groups there might be a bias towards respondents who are more comfortable writing than speaking, though it is often said that Americans’ number one fear is speaking in public. The same is not said of the fear of writing in public.

Unspoken advantages of bulletin-board groups

Appearance doesn’t matter; content does.

By using an online methodology, neither you nor your client can introduce any of your appearance-based biases into the research process. Ethnicity, attractiveness and accents do not flavor the way moderators manage and interpret groups.

Messages to individual respondents don’t interrupt groups.

With bulletin boards, a moderator can privately contact individual respondents without putting them on the spot. Moderators can send praise, ask respondents to tone it down, or eliminate them from a group without interrupting the flow of the group or creating a tense situation.

There is no chance to misread body language.

While BBFGs don’t allow moderators and observers to read body language, they also don’t allow moderators to misread body language. The focus of bulletin-board groups is on the actual language and content of the responses.

Respondents can take the time they need to respond to questions.

For the more timid or contemplative respondents, BBFGs provide an environment where these more reticent participants can respond in their own time frame. In focus groups, some participants respond quickly, others follow. BBFGs can be set so that responses appear simultaneously; thus speedy respondents don’t necessarily lead the discussion. Additionally, because BBFGs take place over a few days, moderators have time to review responses and develop very specific probes.

Respondents can do homework during the group.

Respondents can be asked to do several homework assignments during the group. They can be asked to review other Web sites, look at exhibits and comment on them without being influenced by others. They can read concept statements at their own pace, so they can actually think about them. Additionally, moderators can ask respondents to share photos of products or of their home and office settings.

Observers can clarify and adjust questions during the group.

Observers can watch the groups online, adjust product descriptions, clarify moderator misconceptions, and add questions with much less disruption than in-person groups.   Moderators need not interrupt the group while they try to manage additional questions and clarification from clients in the back room.

Sensitive topics can be handled privately.

Sensitive topics can be introduced individually or to a group so respondents can respond in the manner with which they are most comfortable. Some projects require two methodologies because some of the subject matter is sensitive. With an online methodology, moderators can often combine what would have been two projects: IDIs for sensitive material and groups for non-sensitive issues. With a bulletin-board group, the moderator can send or receive private messages and share responses without attribution to a specific respondent. Additionally, respondents have greater anonymity, and may be willing to talk about sensitive subjects that they might not normally discuss face-to-face in a group.

Participants can go offline to find answers.

How many times has a moderator asked a question in a group or IDI where the respondent could have answered if they were able to look it up or ask a co-worker? Not all answers are or should be top-of-mind, and what is answerable varies by respondent. For example, consider questions such as: How many servers are on your network? What type of firewall do you use? What is in your closet, garage or medicine cabinet? How is your home office set up?

Of course you can ask these questions in a focus group but the response is based on the participants’ memory at that time in that setting. Moderators rely heavily on the respondent’s ability to rapidly recall accurate memories. Imagine how much more accurate these responses could be if the respondent was able to look in his/her closet, office or network to get the answer. This is quite easy to do during bulletin-board groups.

Respondents can easily share digital content with the moderator, clients and fellow participants. Most bulletin-board software allows respondents to upload files and links to Web sites. This can be a fast and fun method to explore to learn.

New ways

As mentioned at the beginning, the intention of this article is to challenge the way we look at the difference between online and in-person groups. There are many aspects of in-person groups that are invaluable - such as the personal interaction between the moderator and their clients, the energy of a group, and non-verbal information that in-person groups can provide - but as with any methodology, in-person groups also have limitations. While bulletin-board groups have their own set of challenges, they also offer new ways to overcome some of the limitations of traditional focus groups.