Editor's note: Martha P. Wilson, M.A., Ph.D. (candidate) is director of Wilson Associates, a San Francisco research firm. Francine Simon, M.A., market research analyst, Kaiser Foundation Health Plan, Oakland, Calif., contributed to this article.

At a recent conference, in response to a comment on . the science of qualitative research, a market researcher was heard to say, "Science? I only think of science when I think of quantitative research." This perspective is quite common. Many of the people doing qualitative research have little or no understanding of the "research" or scientific aspect of their work. In fact, a significant number of qualitative researchers have no research training.

This means that day after day, year after year, decisions are being made by health care organizations, toothpaste conglomerates, clothing retailers and an infinite number of other businesses based on qualitative work that may not be credible research. It's information, but is it data?

So what turns the collection of information into research? What transforms information into data? And why is it important? The answer is straightforward. It is the use of the rules of scientific inquiry, known as scientific method, to guide the work at hand.

If there is no scientific method used to conduct the work then it isn't research. The idea behind research of any kind is that information based on research is more reliable and credible than information gleaned subjectively. And yes, even qualitative research requires scientific method for it to qualify as research.

What does the qualitative researcher (or the client seeking good qualitative research) need to know to conduct true qualitative research? They need to understand the fundamental principles of the scientific method and have the ability to implement them in everyday practice.

There are essentially six basic steps involved in scientific inquiry for qualitative research:

1. Problem formulation

2. Research design

3. Sampling

4. Data collection

5. Analysis

6. Reporting

Carrying out each of these steps requires attention, knowledge and training. These steps are intimately related and critically interdependent. Without one the other step is inadequate and the work loses its status as research. The brief description of these steps is designed to highlight some methodological issues and problems.

1. Problem formulation

Ideal]y, a great deal of thought goes into the identification and formulation of the topic to be researched. This may include the development of an actual hypothesis to be tested or it may involve setting the parameters for exploratory research. In either case, it must clarify what is being measured or tested and why. It's critical to define the terms for the research at the outset to ensure that what the respondents mean and what the researchers mean are the same thing. Concepts such as "customer satisfaction" or "product attrac-tiveness" should be clearly spelled out before being included in the research instruments.

Problem formulation involves a thorough review of similar research and literature available on the topic and then requires a systematic construction of the problem to be researched. It is most common to specify the actual, measurable objectives of the research during this process. Once this step is complete, the researcher is ready to begin the research design.

2. Research design

Problem formulation and research design are probably the most neglected areas of qualitative research. "Let's do a focus group" frequently substitutes for these comprehensive steps.

Designing the research first involves weighing the value of a variety of qualitative and quantitative data collection techniques. The researchers choose the data collection techniques that are most effective in meeting the research objectives with the least amount of error and researcher bias. (Unfortunately, the economics of focus groups is more often the reason for selecting them than their actual value in producing the most reliable data.)

Issues of reliability, credibility and replicability are considered and documented for later inclusion in the methods section of the final report.

Having chosen the technique(s), the researcher designs the instruments, which might include a moderator's guide, a guide to field procedures, a questionnaire, an interviewer's guide or observational guidelines and procedures.

3. Sampling

The ability to obtain the particular sample often determines which data collection techniques to use. The design of the sample is usually part of the research design phase. It is so important to qualitative research and so neglected that it is prudent to highlight it as a separate, but integrated step in the research process.

Sampling consists of designing the selection process for the study participants to determine who gets selected, why and how. There are myriad sampling techniques but all share the same goal: minimizing the chances of getting respondents who do not reflect the target population. Sampling also minimizes the chances that the findings are accidental or coincidental.

A major problem in the field is that focus group research is largely reliant upon databases maintained by focus group facilities. In some cases, these databases have become pools of self-selected, recycled participants, some of whom participate in focus groups and interviews several times a month. In essence, they become "professional research subjects" and as such, their feedback is highly suspect as they come to adapt to the focus group culture and learn to say what they think the facilitators and clients want to hear. There is no reason to believe such databases are in any way random or otherwise representative of the population to be studied.

Simply put, the databases maintained by most facilities are not appropriate for scientific sampling. This means that there is bias built in to the universe used to select participants.

Sampling is designed to factor out bias and limit error in the type of respondent. The best sample is one that both provides access and limits the possibility of including people in the sample who shouldn't be. Note that with focus groups and most interviewing, the sample size is too small and not randomized to make generalizations about larger populations. Even so, the sample should be carefully selected from the universe of people identified during the formulation of the problem.

4. Data collection

Data collection involves the administration of the instruments selected and finalized during the design phase of the project. It is done under firmly controlled circumstances prescribed by the design to insure consistency and replicability. This means, for example, if you wish to compare responses, all of the questions in an interview are asked of each interviewee in the same way.

Focus group moderator guides are data collection instruments. Often moderators use guides as just that, guides. This means that across groups the questions may not always get asked in the same way with the same wording. Thus, comparative analyses cannot, from a scientific standpoint, be made using the findings of a series of focus groups. (They are made routinely, but are probably not accurate.) For purposes of reliability, the questions must be asked in the same way for comparable groups.

Of course, moderators argue that the nature of qualitative research allows us a great deal more flexibility than quantitative research. The beauty and uniqueness of qualitative work is its lack of structure and seemingly limitless ability to explore the issues. This is not in contradiction to the requirement for structure according to a scientifically derived method. In fact, asking the question the same way every time provides the scientific structure and then allows the moderator to explore the answer, once it is given, in as many creative ways as possible. Thus, the creative aspect works hand in hand with the structure.

5. Analysis

Analysis in qualitative research is, more than any other step, not very well defined. In quantitative research, analyses are highly reliant on statistical techniques, while in qualitative research its most accurate form is simple description with leeway for subjective interpretation.

It is important for descriptive analyses to include all responses and for each response to be characterized as equally important. There is nothing in qualitative research that allows one respondent's answer to be more important than another's. In fact, the researcher must guard against clients who try to prioritize the responses based on what the client likes or dislikes, wants to hear or doesn't want to hear.

One of the most common errors in qualitative research is to fall into quantifying the responses. It is misleading to report numbers or percentages (e.g., 80 percent felt that the product was wonderful) because seldom are focus groups, interviews or observations representative of the target or the general population. Generalizations cannot be extracted about the general population from small group interviews or from focus groups. One can assume that people similar in attitude and behavior to those in the room will hold similar viewpoints but this requires really knowing who the participants are.

The goal of the analysis is to organize and categorize the findings in a way that increases our understanding of the responses in the context of the population under study. This means that the "data" must be analyzed and interpreted in the context of the originally defined problem and research objectives.

6. Reporting

Reporting qualitative findings requires the inclusion of the purpose of the research, a description of the research including the reasons for selecting the techniques used, a description of the sampling techniques and a discussion of the recruitment methods. The latter should include a brief discussion of the number and type of people who self-selected versus those who refused to participate.

The findings of qualitative research are most accurate and effective when delivered with a caveat regarding their usefulness. This discussion should highlight both the nature and limitations of qualitative techniques and focus on their value in providing "flavor" and increased understanding. The audience and/or client should be cautioned against making major decisions based solely on qualitative findings. Instead, they should be encouraged to combine the findings with other quantitative and qualitative research results to be sure that they have a solid basis for decision making.

Increasing the scientific method

How can you increase the reliability and credibility of qualitative market and social research? Increase the use of the scientific method. Does that infringe on the nature of qualitative research and limit its creative and exploratory capabilities? No. In fact, it can enhance these crucial aspects by providing the credibility the research and the findings deserve.

Qualitative methods were never intended to be without science or structure. The notion that "If it's qualitative, anything goes" defies the very fact that we're trying to conduct a unique type of scientific research. If it is to be called research it must be based in science not whimsy, gut feelings or budgets.

The unstructured nature of qualitative research is both its strength and its weakness. It's strength lies in the ability to probe the respondent's thoughts, behavior, motivations and lifestyle. It provides a rich array of information and often provides the context that quantitative research can't. But its weakness is that its limited structure makes it subject to a great potential for error. Its much more susceptible to researcher/client bias and therefore requires objectivity and systematic processes.

While there are numerous things we can do to improve the quality of qualitative research, these ten guidelines have been selected to initiate an ongoing dialogue in the field:

1. Remember that qualitative research is best for providing an understanding of the complexities of the issue(s) at hand rather than offering conclusive findings. Both clients and qualitative researchers must refrain from treating the findings as conclusive without including both literature review and other research.

2. Qualitative research is most effective when integrated into a larger project which includes a healthy quantitative component. One of the best uses of focus groups is to test the initial drafts of telephone, mail or intercept survey instruments. This allows the client to get feedback about the questionnaire, about whether or not participants were likely to actually complete it and about their understanding of what each question is intended to measure. The focus groups, being relatively inexpensive compared to the implementation of a survey, allow the client an opportunity for refinement before investing significant time and money fielding the instrument.

3. Many people conducting qualitative lack training in research methods. Becoming conversant with the scientific method through market research or social science courses at your local community college or university in research methods can only improve the professionalism in the field.

4. Qualitative researchers must educate clients on the proper use of qualitative findings. We can do this through our initial discussions of design, through the questions we ask about the data collection techniques that the clients have asked us to use and through the oral and written reports we provide. Of course, if we have the opportunity to actually design the research, we can make it a habit to present our designs in the context of the steps of scientific inquiry.

5. The true strength of qualitative research lies in its research design and its theoretical framework. The findings can be validly interpreted within that framework and only within that framework. The soundness and potential replicability of the findings is dependent upon the steps of scientific inquiry. And, like dominos, each step profoundly influences the balance and integrity of the other.

6. Discuss the potential for bias with your client. Highlight the ways that the client's and/or the researcher's preconceived ideas can produce particular results. Identify ways to minimize them and build these approaches into the design.

7. When reporting, include in the methods section a description of the sampling technique and the recruitment process with a discussion of the number and types of people who self-selected into the process and the number and types of people who selected out and why. This means carefully documenting who refuses to participate in the study. If at all possible, obtain minimal demographics.

8. Spend more time finding out who is really in your focus group, individual or small group interviews, observational settings, etc. Collect not only demographics but administer other data collection techniques to find out about lifestyle, decision making, buying habits, etc. Knowing your respondents provides a very solid context for analysis and interpretation. Rather than trying to extrapolate to larger groups through generalizations, tell your clients the kind of people you have as respondents and extrapolate from there.

9. Purchase sample whenever possible from reputable organizations. Avoid using facility databases when possible. Develop healthy rationale for your sample design and stick to it.

10. If you want to know the proportion of the population that feels a particular way or engages in a particular behavior, use quantitative research methods. The findings will always be more reliable.

Qualitative research is finally taking its rightful place in the research arena after decades of being frowned upon by the scientific community. To establish a permanent foothold for it in social and marketing research, we must increase its credibility, maintain its naturally fragile integrity and treat it as a serious form of scientific research.