Always in style
Editor's note: Andrew D. Cutler is principal at Integrated Marketing Associates, a Bryn Mawr, Pa., research firm.
Not too long ago, I wrote an article for Quirk’s ("What can Web do for you?", January 2011) rhapsodizing about the array of technologies that are now available to help market researchers provide research solutions more quickly, efficiently and/or cheaply than traditional in-person research may permit. Many readers may have inferred that I am strongly advocating for the use of these technologies. Well, they were partially correct: I am – but only when going high-tech makes more sense than choosing in-person research.
Even in this age of gee-whiz-will-miracles-never-cease technological marvels, there are often research projects that are still going to be best served via plain, old-fashioned, eyeball-to-eyeball, in-person research. No amount of Internet bandwidth or cyber-power is going to replace the insights that can emerge when a researcher sits down in the same room as the research participant(s) and skillfully elicits their feedback.
In-person research offers several benefits over any other approach. Let’s take a look at some ways that in-person research trumps technology.
Detecting nonverbal cues
In-person research provides a more nuanced understanding of participants’ responses, particularly with regards to body language and other nonverbal cues that may not be apparent via the Internet or telephone.
You’ve heard the claims: 93 percent of communication is nonverbal. (How the heck do they measure this stuff so precisely, you might ask?) Whether this figure is true or exaggerated, the fact remains that a great deal of what we communicate is nonverbal.
One of the biggest nonverbal channels is the use of tone and inflection. This is why even a very simple sentence (e.g., “That’s great”) can have multiple meanings, depending upon how the words are spoken. Is the speaker being sincere or using sarcasm? If they’re being sincere, how much enthusiasm do they have? Which word do they emphasize? All of this affects the meaning of the statement.
Since qualitative research is generally conducted via spoken words, tone and inflection are usually evident. Regardless of whether the research occurs in-person or by telephone, this nonverbal information is part of the data that an experienced researcher will capture.
But here’s the catch: At least as much nonverbal communication is through facial expressions and body language. Thus, a large amount of data is lost in the traditional phone interview. The use of Webcams, as I discussed in the aforementioned Quirk’s article, helps to remedy this problem by training a camera on the respondent and then transmitting their image to the moderator and any clients who are watching. However, depending upon how close the camera is to the respondent, one typically has to choose between a close-up of their face, which leaves body language largely hidden from view, or a broader shot, which makes it difficult to see more subtle facial expressions.
Webcam technology, then, is an imperfect solution to the problem of data loss that comes with conducting research in a setting other than in-person.
Managing stimuli
The moderator has complete control over the exposure that respondents have to individual stimuli.
Web-assisted interviews do provide the moderator with a great deal of flexibility in terms of how and when stimuli are shown. An internal, password-protected Web site that we often use at my firm allows us to place any PowerPoint slide in front of the respondent and then to remove it when appropriate. In fact, we can show as many slides as we wish, with precise control over the duration of exposure to each slide. Respondents cannot download or print out the slides so confidentiality is assured.
But again, there is no substitute for the real thing. If we want to show multiple slides simultaneously, for example, we cannot do so without shrinking each of the slides so that they all fit on the computer screen, rendering each slide more difficult for the respondent to see on their computer’s desktop. By contrast, in the context of in-person research, our only limitation is the size of the physical desktop.
Furthermore, some stimuli do not lend themselves to online technology. For example, a pharmaceutical sales aid (a multipage document that pharma representatives use when meeting with health care professionals) can be awkward to test online because it is a physical booklet. For example, respondents cannot readily flip from page seven to page two if they want to compare the information on separate pages. As for testing devices or other 3-D objects? Fuhgeddaboudit! There is simply no technology (at least, not yet) that effectively replicates our ability to, for example, have an asthma patient hold a new inhaler in their hands and examine it.
In addition, during in-person research, clients will sometimes ask us to test stimuli on the fly. In the middle of an interview, there will be a knock on the door and we will be politely handed a visual concept or promotional message that the client has quickly come up with, based on what the respondent is saying. Such last-minute additions to the research are difficult, if not impossible, to implement when research is occurring in the context of Web-assisted phone interviews.
Staying focused
In-person interviews provide assurance that respondents are not multitasking during interviews and that other persons are not present and/or influencing the responses.
It’s no secret that respondents interviewed on the telephone often are doing something else at the same time, whether it’s washing dishes or grooming their dog. This is problematic, for obvious reasons. A distracted respondent is a respondent who provides less-useful, less-accurate and less-reliable feedback.
Even if the respondent is trying to give you 100 percent of his or her attention, consider the fact that they may be at home (with their kids running around) or in a coffee shop (with noisy patrons ordering decaf lattés) or at the office (with co-workers interrupting them periodically). Again, the quality of the interview inevitably suffers.
Occasionally, unbeknownst to the interviewer on the other end of the telephone, another person will be sitting next to the respondent and (deliberately or otherwise) influencing his or her answers. This, of course, contaminates the feedback and compromises the quality of the data.
Webcams again provide a partial fix to these problems. Because a Webcam reminds respondents that they are being watched, they are usually on their best behavior and hence will generally wait to catch up on e-mails or clip their fingernails until after the interview is over. Still, the distractions of colleagues and family members may inevitably arise.
There is, in short, no substitute for conducting research in the controlled laboratory setting of a focus group facility (assuming respondents are told in advance to turn off their phones!).
Stakeholder attendance
Kicking off the research with in-person interviews enables the project team to attend the research and collectively decide on any modifications and adjustments to the discussion flow and areas for probing.
One of the most reliable patterns in this business is that, for many clients, the first day of research, and in particular the first interview on the first day of research, holds the greatest interest and importance. This is certainly understandable; it is at the beginning that clients are most anxious to see how the respondents feel about, for example, advertisements that have been painstakingly developed over the past several months. Will it be a hit, like the agency predicts? Or is it going to alienate some customers, like the product manager fears? And, equally importantly, are things on the right track? Is the moderator asking respondents the right questions? Is the client ultimately getting the information that they need to make the right decision?
For this reason, on day one the back room in the focus group facility is often filled with a large audience of stakeholders in the project. When the moderator goes into the back room at the end of the interview (to solicit follow-up questions), he or she often walks into a room that is abuzz with discussion, analyzing what the respondent said and what implications it has. These discussions often generate follow-up questions and/or modifications to the stimuli. As a result, the interview structure, content and focus tend to evolve and by the end of the day things may be very different from where they were at the beginning. It is a productive and exciting process.
Now imagine the same project executed via telephone interviews (Web-assisted or not), with the project stakeholders scattered across different sites. Teleconferencing is set up to enable everyone on the team to listen to each interview and then, at each interview’s conclusion, to discuss what the respondent said and what it means. The process of collectively assessing the feedback and deciding on modifications to the discussion guide and stimuli can still occur but is inevitably less fluid than an in-person discussion at a focus group facility. For example, the creative director cannot quickly show the rest of the team an alternative visual image that she has sketched out on her notepad.
More importantly, in a teleconference, the body language and facial expressions of the team members are lost entirely (or partially, if everyone is on videoconference) and this loss of information can affect the degree to which each team member understands what his or her colleagues are trying to communicate. By gathering the stakeholders into a room together, in-person research can prevent these miscommunications from occurring and ensure that the team is in full alignment regarding the next steps.
Role-playing
In-person research also enables respondents to demonstrate exactly how they would talk/interact with another individual.
In many research projects, one objective is to understand how the audience being interviewed (e.g., health care professionals) interacts with another audience (e.g., patients) and in some cases, how they would use an object (e.g., a medical training device) during these interactions. Role-playing, with the moderator serving as a surrogate for the third party, is often a useful exercise in these types of research situations.
Let’s imagine that research is needed to understand how diabetes educators would train patients to self-inject themselves, using a pen-needle device. Would you opt to conduct this research using a Webcam and a telephone? Probably not. Research of this sort obviously works best when it is in-person. Savvy moderators and clients will likely want to enact a mock patient visit, during which the respondent (in this case, a diabetes educator) demonstrates precisely how they would train a patient (role-played by the moderator) to use the device.
Similarly, imagine that you wanted to understand how physicians believe pharmaceutical reps should optimally use a specific sales aid when visiting physician customers. Specific questions would include: What pages would they use? Which information on each page would they focus on? How much time would they spend on each chart? In what order would they present the information? What would they say as they’re using the document? Again, an in-person role-playing exercise is likely to provide more information than any other approach can offer.
Certain advantages
In-person research offers certain advantages over technology-based research – advantages that, in certain research contexts, makes it the best methodological solution. The face-to-face approach has – and probably always will have – everlasting value. Wise researchers and clients will avoid the siren call of technology when it is best to do so.