Editor's note: H. Catherine Bueter is currently employed by Hewlett-Packard, where she has held the positions of product manager and marketing programs manager for the firm's San Diego Color Imaging Division.

Recently, when Quirk's Marketing Research Review surveyed its readers on their usage of various research methods, almost two-thirds (68.9%) of the respondents said they had used focus groups in the past year, while slightly less than one-fifth (17.9%) said they had used panels. I feel that panels are an extremely effective market research tool, and the survey confirmed my belief that panels are under-utilized by the marketing research community.

A panel can be an effective alternative to focus groups, which I feel are relied on too heavily and sometimes used inappropriately. Here's some key data on how panels work, in case you aren't familiar with this research method.

What is a panel?

A panel is a group of people who represent a target market and are recruited and pre-qualified to participate in market research studies on an on-going basis.

How do panels work?

Panel members are recruited (either by mail or, more typically, by phone) and are asked to participate. They are told that they will be called upon several times for their input about a given product category. Most panel members agree to participate because they believe they are helping improve the quality of products offered by the manufacturers. As an added incentive, it is common for panel members to receive a gift for their participation. Membership is also anonymous; members do not know each other.

One of the key ingredients for the success of a panel is its on-going maintenance. It is essential for the members of the panel to be balanced demographically and according to other key variables (like region, sex, psychographics, etc.). This is to ensure that the panel is representative of the population under investigation. It is the research company's job to ensure that the integrity of the sample is maintained.

What do panels offer the researcher?

Panels basically offer a pre-qualified audience that is matched to the demographics of the target audience. Since they are in a sense "ready for action" at any given time, they offer you the following advantages:

  • Quicker survey results. One of the most time-consuming aspects of research is the process of deciding who you want to interview, developing the screening questionnaire, and choosing the sample. With a panel, you have already made these decisions and pre-selected your respondents.

  • Higher respondent participation rates. Panel members are more likely to respond to the research instrument they are given because they have a sense of obligation. This is accomplished at the very beginning when they are recruited. They feel they are making a contribution, and therefore, when they are called upon to provide input, they are likely to respond.

  • Opinions that are projectable to your target market. By definition, the panel members are representative of the population you are trying to influence. Again, this is a key part of building a panel. It is essential that the panel be broad enough to encompass the range of people in your target audience. It is usually worthwhile to get consensus from all interested parties in your organization on this part of the process. Otherwise, you end up adding people at a later date, which can be costly and time-consuming.

  • Efficiency. Efficiency is increased because your panel members have been carefully screened and qualified, and are ready to respond at any given time. This will significantly increase your actual number of "qualified" respondents. In other words, you go after fewer respondents, which results in a higher percentage of completed interviews.

  • The ability to track behavior over time. With a panel, you can track the behavior of the same individuals over time. This is especially useful for determining patterns of consumption, buying, and general product usage. Having access to the same group of people over a prolonged period of time is an excellent way to evaluate the effectiveness of certain stimuli (i.e., advertising) on an individual's behavior.
  • Potential cost savings (set up costs are spread over several surveys). The costs for finding the people to interview often are a significant portion of the research budget. These costs are diluted with panels because of the initial investment in their set-up. Because you only really design the sample once, you do not have to repeat these costs every time you do a survey.

Why aren't panels used more often?

There may be the belief that panels are extremely costly and therefore difficult to justify. Well, it is true that there are significant costs involved in setting up and maintaining a panel. However, it's important to look beyond the sheer dollars involved and analyze panels from a cost/benefit point of view.

A panel should be seen as a long-term solution, not as another singular research project. Look back over the past year and analyze the types of research projects you've done. For example, tally up all the dollars you've spent in the past year on other research methods and ask yourself, "Is this too much, and were there times when some other methodology would have been more appropriate?"

One way to examine this is to compare the kinds of marketing issues that are typically explored in focus groups and panels. Focus groups are helpful for investigating issues that benefit from the group synergy that the format offers. These include:

  • new product concept development;
  • understanding customer language;
  • generating hypotheses about customer behavior.

Quantitative studies (which include panels as a subset) are most appropriate for providing answers to specific questions such as:

  • Should we introduce product X? What is the level of purchase interest? What is the optimal positioning? How much should the product cost?
  • Which positioning (A, B, or C) is best for our product or product line?
  • Who is the target audience for our product?
  • What benefit should we promote in our advertising?

All of these are specific marketing questions that need definite, precise answers. And yet, at times, a research tool is used that only allows you to explore, not confirm, the findings. Why? The answer is not that complex when you compare focus groups to quantitative research methods.


   

Number
of people

Average
Cost

Prep.
Time

Time to
results

Client
involvement

Methodology

   

   

   

   

1 focus group

10-12

$5-6k

2 weeks

2 weeks

Intrusive

Quantitative survey

100+

$15k+

2 weeks

4-6 weeks

Passive

At first glance, the advantages of focus groups over quantitative methods are time and money. But after being on both sides of the marketing research fence (I spent several years conducting primary research before shifting to the user community as a product manager) I believe that the real reason for the predominant use of focus groups is their "intrusive" nature. By intrusive I mean that the product manager physically sees her/his customers, hears their reactions to the products and is able to develop a sense of how they feel about the products.

From my experience, I have noticed that product managers are much more comfortable when they receive information first hand, straight from the respondent's mouth. And we all know that people remember more when they not only hear information but are also exposed to visual reinforcement of the data-which is just what happens in a focus group.

So your product manager loves focus groups because they provide valuable feedback. But how do you convince him that there are times when other, less intrusive forms of research are better- suited to obtaining the kind of information he needs?

One suggestion is to provide examples of how using an inappropriate research tool can lead to the wrong decision. The "cost" of bad decisions ultimately will more than offset the cost and time savings that a research method might seem to offer.

But let's look at the problem you will face if you push for a quantitative survey. Let's say a product manager chooses to do a quantitative survey to answer some questions about what features printer users want in a new color printer. As his researcher, you meet with him and develop the specific questions to be asked. The two of you then review the sample specs and the questionnaire with the research agency.

After the survey is approved, the questionnaire is mailed out and about 4-6 weeks later, a huge 3-inch, 3-ring binder arrives with more number than the product manager could ever possibly use! (I have even seen it happen that the product manager is on a different product by the time the research is finished!)

So now you have all this data and you get questions like, "But how do they feel about being able to add color to their typed memos?" You reply with, "43.4% said on a 1-5 scale, with a 1 being 'not at all important' and a 5 being 'extremely important,' that the importance of being able to add color to memos was a 4.8." You then hear, "Well, how many colors would they want?" You respond, "We didn't ask that question." Frustrated, your product manager leaves with unanswered questions and decisions that still need to be made about the new color printer.

However, if you maintained a panel study in which printer users were asked their perceptions of the new color printers, you could then turn to the product manager and suggest that you simply add that question to the next survey. You've accomplished several things here. You've enhanced your credibility in the mind of the product manager and secondly, you've established that you can find solutions to any question that she/he might have.

"Wow, this sounds great," you may be saying to yourself, "but what's the catch?" There is one major consideration with respect to setting up a panel: initial set-up and on-going maintenance costs. However, once again refer back to your past year research studies and ask yourself if some of the money in your research budget might be more efficiently used elsewhere.

Another potential problem is respondent overuse, which creates a "professional respondent" bias. But there is a cure for this-simply establish a maximum number of research projects per year in which a given person can participate. This will eliminate the potential of any one person becoming an expert in a given area.

It is important to keep in mind that you are offering a contribution to improve decision-making in your company. It is a challenge, but isn't that what research is ultimately all about? Given this, it seems worth some time and effort to investigate ways to provide the proper methodology to answer marketing questions. Because we all know that some data really isn't better than no data at all.