Editor's note: Bernard Schwartz is a freelance qualitative research consultant.

I recently completed two research assignments in sequence. The first was a positioning study for a once-daily AIDS medication; the second tested advertising concepts for the twice-daily drugs threatened by the new once-a-day products. Now here's the interesting part: In the first study, most respondents told me that they are not excited by once-daily dosing, as it is not important for drugs in the AIDS category. In the second study, where respondents (also AIDS physicians) were asked to consider advertising concepts for products with twice-daily dosing, they told me the exact opposite - that once-daily dosing is a major breakthrough, and that they could not wait for the new once-a-day products to become available.

How could it be that the research findings of these two projects are so different, one endorsing the benefits of once-a-day dosing, the other finding that it isn't important? Could it be that the structure and content of the interviews themselves influenced the outcome? Does the manner in which we ask questions convey clues that "lead" respondents' answers?

Always avoid taking the lead

Market research professionals take pride in their objectivity; as neutral third parties we are not beholden to any point of view. Researchers, and many on the client side as well, are concerned about anything that might provide respondents with hints as to how the interviewer or client would like the question answered. Clearly, if interviewers were to communicate a preferred answer or a specific point of view, they would compromise the objectivity of the research and close off the possibility of receiving honest answers. Leading respondents in any form is a long-standing taboo within our profession.

Experienced researchers carefully monitor the phrasing for questions to determine if semantic clues are present that might telegraph or cue an answer. If the interviewer were to ask "Why do you think that inferior product is better, for heaven's sake?" it is all too easy to identify this as a leading question. Neutral probes such as "Tell me about that" or "Please explain" can help avoid giving out clues regarding the interviewer's viewpoint. Another example of leading the respondent might be "If they raise the price of Product X by 40 percent, will it influence your usage?" What right-thinking respondent wouldn't say "Of course it will!"

At times, avoiding leading respondents can be a complex task. It is possible to lead with facial expressions, hand gestures, and the tone of one's voice. Raised eyebrows or pursed lips - even when unconsciously done - may cause a respondent to think that their answers are not the "correct" or expected answers, and experienced interviewers take great pains to avoid these non-verbal cues.

In the context of a specific question, there are several formal devices that can help avoid leading. These include adding the phrase "if at all" to questions. For example, "How, if at all, would this product change your thinking about...," allowing for a full range of possible answers including "It wouldn't change at all." Another device is to carefully ask questions from both a positive and a negative point of view: "What are the advantages of this product?" and then "What are its disadvantages?"

Some interviewers phrase probes in the voice of other respondents when there apparently contradictory responses from different respondents. For example, "Other respondents have told me they really like that feature because it does ____. How do you feel about that?" Or again, "In other interviews, respondents told me that they use Product G because it has much better [attributes] than this one. How do you respond to that?" This technique allows one to probe answers in a value-neutral manner, by asking them to compare their experiences to those of others rather than challenging respondents' views outright.

Some tests of objective research design

In designing qualitative market research, the following should be considered in crafting discussion guides as well as during the interview when we are revising and fine-tuning interview questions to fit individual circumstances.

  • Does the context of a question lead respondents to think that the interviewer "owns" or prefers approval of the product or attribute?
  • Could the answer to the question, as it is phrased, easily go toward any of the possible answers, or even to answers you haven't anticipated? Are you getting a range of responses to the question, or only one answer?
  • Do you have a basis for comparing respondents' general views exclusive of the product being evaluated and in the context of the stimuli? For example, "Earlier, you told me that you didn't think this attribute was important, what is different now?"

Leading with the research design

In qualitative research, it is possible that research designs themselves - or the flow and order of interviews - may bias the outcome of the research by leading respondents toward a predetermined outcome.

The two research projects described above followed a fairly traditional research design. First, some general questions were asked to orient the respondent to the subject at hand, followed by a detailed evaluation of a specific product or service of interest to the client. In these two projects, the stimuli were a set of advertisements and a blinded product profile respectively. This is certainly a common format for market research, used in positioning studies, product assessments, communications testing, medical practice surveys and a host of other qualitative research projects.

In sequence, respondents in the two projects:

1. Were asked general questions about their current treatment preferences, product selection and usage - with an emphasis on questions relevant to the features and benefits of the product- or concept-to-come. Up to now, conventional wisdom has been that it is preferable to ask some general, preliminary questions rather than just springing the stimulus on respondent. By asking preliminary questions, researchers hope to get the respondents thinking about the issues at hand. In actual practice these preliminary questions are sometimes included with little or no expectation that they will yield new or significant findings.

This section of the discussion often serves to:

  • help the interviewer place respondents on a continuum of preferences: pro or con, conservative vs. radical, user vs. non-user, early vs. late adopter;
  • conduct background research on current consumer preferences, behaviors, and attitudes toward the product category or general area of the research; or
  • "orient" the respondent to the subject at hand.

2. Were asked to read a description of a blinded product or to review ads or other media relevant to the client's product.

3. Participated in a discussion centered on a consideration of specific product features (after the product profile or media were shown), for example, "How important is once-daily dosing to you?" "How might it affect your treatment approach?"

Why this might not always be the optimal way

My concern is that this research format may lead respondents contextually by giving not-too-subtle hints as to what they think we want to hear. In effect, the context of the research and the shape of the interview may, at times, imply a desired response.

Qualitative researchers often deal with respondents who are well-educated, savvy individuals. More often than not, within the first few minutes of questioning respondents are mentally trying to determine:

  • Who is sponsoring this study and what is the sponsor's agenda?
  • What is the level of commitment or involvement of the moderator with regard to the product? Does he/she work for the manufacturer?
  • How much does the person sitting across from me really know about the topic?
  • Will I "look smarter" if I approve of the product or if I disapprove?

At the point that an interviewer hands a product profile or other stimulus to respondents, he or she may be tacitly assumed to champion or represent the product. No matter how objective the researcher is and how carefully questions are phrased, respondents may assume that the researcher has a stake in making the product look good.

By the same token, once a series of general questions are posed, discussion of the product or stimulus is frequently biased by those questions. It is more difficult to get unbiased information about the product or service under evaluation after respondents have considered the preliminary questions (and received obvious hints as to the interviewer's agenda). This conventional approach to research doesn't always work, and in fact, may be leading the respondent within the context of the discussion. This being the case, it is helpful, at times, to have respondents evaluate the stimulus or concept before they form an opinion as to the interviewer's agenda - cold, with no preliminary warm-up questions.

Structuring interviews so a stimulus is shown and evaluated first, before the more general discussion, may be an effective approach at times. While this research design flies in the face of tradition, I have done it with success. After evaluating the product or concept in the first section of an interview or focus group, one can the explore respondents' current, baseline preferences in a later section, in the context of, e.g., "How, if at all, would this product change your current practices or preferences/purchasing habits/usage?" "What are your current practices and preferences?" By changing the order of the interview, one can evaluate current preferences in the context of the stimulus:

  • What impact (if any) will the product have on your current purchasing patterns?
  • How (if at all) will it change your current practice, usage, approach?
  • How does this compare to current products?

Perhaps this is a more relevant approach for many research projects.

Clients can also play a leading role

According to Murray Simon, a seasoned moderator and researcher, at times, clients have a strong vested interest in their product and in the research process. It takes a great deal of self-discipline for researchers - and the clients - to remain neutral and sensitive to the possibility of leading respondents despite findings they feel are negative. The market researcher must be the voice that keeps clients focused on the need for, and importance of, objectivity in the research process.

Simon recalls a research project that clearly illustrates this point. One of his clients had expended a considerable amount of money to conduct a head-to-head comparison between their drug and four older, competing brands. The results were disappointing in that they indicated that the client's drug was comparable to, but no better than the competing products.

In an effort to justify their substantial research investment, the client developed advertising that emphasized a corporate commitment to research as exemplified by this head-to-head trial while downplaying the actual results.

It backfired! Advertising concept testing with physicians resulted in very negative reactions, with a majority of respondents stating that the concepts merely point out the fact that the client's new product is no better than those already currently in the market.

Simon had to resist his clients' intense efforts to introduce leading questions in an effort to make the study outcome more positive. Finally, he invited the clients to work with him to find a way to salvage the research project. After much discussion, and in an effort to inject a more balanced perspective into the proceedings, the concept was modified halfway through the study. The modification involved the addition of one word - unsurpassed (as in, "unsurpassed efficacy when compared to the other available therapeutic agents"). With the introduction of that one word, attitudes changed from strongly negative to distinctly positive - "This is important information. You should make sure physicians know about this!"

Simon was able to both resist his clients' suggestions that respondents be led toward more favorable answers to interview questions, and to persuade them that respondents were already giving true and useful answers, just not the ones the clients wanted to hear. Alternatively, he was successful in encouraging them to explore other ways to get the message across that were more effective.

Don't take the lead in your research designs

There are dozens of ways to lead respondents, for example, poorly phrased questions and giving subtle clues as to what we they think we might want to hear. It is a constant challenge to avoid leading in all forms, both in content and in context. But it is an effort that must be undertaken in order to conduct interviews that elicit unbiased and untainted responses.