Editor's note: Steve Bernstein is manager, market research at Consolidated Freightways, Inc. in Menlo Park, California.

Focus groups are an effective method to help marketers generate new product ideas, ideas for product repositioning, new marketing programs, and the like. But as all skilled market researchers know, focus group results are always tentative, never definitive. Results come in the form of new hypotheses about underlying forces at work in the marketplace. These hypotheses are supposed to lead marketers to a range of alternative new product, service, or program possibilities.

Thus focus group data are most appropriately used as input to the design of another research project, ultimately some sort of quantitative test, the results of which can be projected to the whole target population. As an example of the way the process should work, imagine that a small company wants to spur growth by introducing a new product into a new market. As a result of information gleaned from focus group research, they develop five alternative product concepts. They test them all quantitatively, rolling out the one that has the highest probability of maximizing profit.

The company is so happy with the results that management decides to apply the process to every new product investigation. Despite testing, once in a while they introduce a product that disappoints them. But their success rate meets their goals, so they stick with a consistent approach.

Ideally, the process works that way. The dark side of focus group research, however, is when companies rush to market with a product based solely on focus group data, neglecting follow-up quantitative testing. Only through blind luck does this approach succeed.

Not nearly as harmful, but probably very common, is the partial breakdown of the process at the point where the focus group data are used to design the quantitative test.

Several obstacles impede the consistent, efficient translation of focus group data to promising, fully-refined, testable marketing concepts. First, the data from the groups are often subject to distortion and misuse. This problem is widely understood among market researchers. They strive to eliminate the pitfalls by preparing observers in advance and providing proper warning during the presentation of results. Despite best intentions, however, it is impossible to avoid some misuse.

Second, because of its loosely-structured nature, qualitative information is prone to loss. Some focus group sessions are stupefyingly dull; attention spans lapse. The seeds of many new ideas are never noticed; others are simply forgotten. Though a well-trained market research analyst can reap more from a set of groups than an untrained observer, the problem still exists.

This becomes all the more important when one considers how expensive the information is. In the world of business-to-business market research, focus groups typically cost $400 to $750 per respondent. The average amount of "air time" each respondent gets is about 12 to 15 minutes.

The final obstacle to the efficient translation of focus group data is the amount of time that passes between the completion of a series of focus groups and publication of the research provider's report. Enthusiasm and energy, the horsepower of creativity, often wane during the ensuing weeks. Early "topline" findings are a partial solution, but they are necessarily sparse, compounding the loss-of-data problem mentioned above.

Follow-up workshops and championships can help solve these problems by getting focus group observers to discuss in an organized fashion what they saw and heard in the focus group, with the ultimate goal of creating workable new product concepts.

How do workshops and championships work?

If qualitative and quantitative research are the muscle and bone of new product research, then workshops and championships are the tendons that bind them together. The following recipe for workshops and championships may need to be modified to accommodate diverse company cultures.

The general idea is this: using the focus group moderator's guide to shape the follow-up workshop discussion, a facilitator leads the observers through two exercises in parallel. The first exercise debriefs them on each section of the moderator's guide, asking them to interpret and react to what was said in the focus group. The second exercise is creative idea generation-brainstorming. The workshop produces many new product concepts, but by the end, the best five or ten are left. After the workshop, volunteers "champion" each alternative, refining each into a well-considered, viable new product concept ready for quantitative testing.

Workshop structure

All focus group observers should be required to participate in the workshop and championship. Many marketers love to watch focus groups; setting participation in the entire process as a prerequisite to focus group attendance is an effective way to get cooperation.

The workshop should be conducted two to five days after the final focus group. Sessions can last from a half a day to two full days. When picking a date for the workshop, strike a balance between letting participants digest what they learned and debriefing them while the experience is fresh. Of course, they should bring notes they made during the focus groups.

The workshop facilitator should use the first ten minutes or so to describe the entire process. Everything, including the purpose of the focus groups, the workshop, the championship, and finally the quantitative test, should be covered. The facilitator must make clear that the ultimate goal is to produce one very promising new product concept, but that the intermediate goal-that is, the goal of all steps leading up to the test-is to produce "finalists." This is very important. The truth is that if the workshop and championship steps end with only one promising idea instead of several, the exercise has been a waste of time. At the end of this introduction, the facilitator must establish two important ground rules:

  • During debriefing, participants should limit their comments to what they learned during the focus groups only. Otherwise the workshop can too easily digress into a forum for demonstrating how much people think they know about the market. Encourage the participants to keep each other honest.
  • While brainstorming, participants should help create an environment which encourages the creation of ideas and discourages the evaluation of ideas. This is essential to productive idea creation- don't worry about redundant or seemingly absurd ideas.

Immediately after the introduction, a brief warm-up exercise will help loosen up the participants. Participants should bring a list of the five most surprising or interesting points they heard during the focus groups, as well as the five most important points. The workshop facilitator allows ten to twenty minutes for an open-ended, perhaps even chaotic, discussion of what participants bring in with them.

After the warm-up, the workshop participants review what they learned as the respondents were discussing the first topic of the guide. Then the participants brainstorm new product concepts based on the review. Next, the group discusses what was learned in the second section of the guide, then it brainstorms that section, and so on. The moderator's guide should be broken up so that each section can be debriefed and brainstormed in about 40 minutes or less. Generally, debriefing takes longer than brainstorming.

Experience shows that side-by-side debriefing and brainstorming tends to produce many more new concepts than debriefing the entire moderator's guide first, then brainstorming.

During the discussion, the facilitator should keep legible notes of the debriefing and the brainstorming on a flip chart. (A better tool is a scrolling dry-erase board that allows the facilitator to make copies directly from the writing surface.) It is not necessary to write out new product concepts in detail. Capture just enough of each idea to remind everyone of it later.

If the workshop is run correctly, the group will end up with a long list of new product concepts, ranging from the ordinary to the absurd. Many ideas will be redundant, or at least overlapping, because during the workshop, evaluation of ideas was inappropriate. At the end, however, the group should reduce the list by combining redundant ideas and discarding unfeasible ones.

The facilitator must keep participants honest. Before discarding unfeasible ideas, be sure they are truly unfeasible, not merely unpopular. A seemingly poor idea can often be crafted into a commercially appealing concept if someone is willing to champion it.

Try to weed the list down to between five and ten new concepts. Because of the structure of the championship, there must be at least as many participants as concepts. Also, if the group cannot in good conscience reduce the list to ten, then concept testing may be restricted to multivariate approaches only. This really depends on how your company does its quantitative testing. Generally, the fewer alternatives you must test, the wider the range of market research approaches from which to choose.

Championship structure

The last task in the workshop is to assign champions to each of the surviving new product concepts. The term "champion" is used in the classical sense, as in the story of the Trojan War, when Achilles was the Greek champion and the Trojans advanced Hector as their champion. Each new product concept must have its champion.

Rather than assign participants to champion particular concepts, first ask for volunteers. The process will work much better if people are allowed to work on concepts they like. If there are enough participants, try to get teams, rather than individuals, to champion ideas. Keep teams to about the same size, and if you have participants from different levels within the organization, maintain a mix of higher and lower level people on each team.

During the one or two weeks following the workshop, each champion refines his or her alternative into a well-considered, viable, new product concept, ready for quantitative testing. Inform your champions that, unlike the workshop, all sources of information for this exercise are fair game, including video and audio recordings of the focus groups, flip charts from the workshop, data from past market research projects, secondary information, etc. They shouldn't feel limited at all. The only restriction is that champions aren't allowed to change the fundamental idea to which they've been assigned. The refined concept, in other words, has to resemble the original idea from the workshop.

The championship phase culminates in a meeting where each champion presents a fully-refined concept to the other champions. No outsiders are allowed at the meeting. At the end of the presentation, the champion must defend the concept while the other participants uncover and explore weaknesses through question-and-answer.

Despite the intensity this arrangement may produce between the presenters and the other attendees, the environment is relatively safe for two reasons. First, everyone will be vulnerable when presenting their own new product concept during the meeting. Second, because they have made it this far through the process together, the participants will have formed a common bond through the shared experience of the focus groups and the workshop. Though the intensity may be high, personal risk is naturally modulated. The feedback is meant to be used for final refinement, and the ultimate output from the whole process is a set of fully-refined, ironclad alternatives, suitable for final quantitative testing. Any of the alternatives should be viable, since each has been considered so thoroughly.

Benefits of the workshop and the championship

Recall the three problems often associated with focus group research described at the beginning of this article:

  • Focus group data are subject to distortion and misuse.

  • Because of the loosely structured nature of qualitative research, focus group data are prone to loss.

  • Full analysis of focus group data takes time. In the interim, the enthusiasm of the participating marketers can wane, diminishing the quality of their work.

The workshop process addresses these problems. First, by forcing observers to debrief "publicly," that is, in the presence of the other observers, they tend to be more responsible about what they say and think about the experience. If in describing respondent behavior or opinions they go beyond what really happened in the focus groups, the other participants, including the facilitator and the researcher, will keep them honest. This minimizes distortion and abuse.

Second, by airing all that was reamed during the focus groups, participants benefit from each other's perspective. Any one observer is likely to miss much while watching a focus group. Taken together, however, the observations of all the participants should capture almost everything said. An open debriefing makes the focus group a much more efficient and rich market research tool.

Finally, by conducting the workshop and championship on the heels of the focus groups, your group takes full advantage of the enthusiasm and energy generated.

Anecdotal evidence suggests that a typical concept test of several alternatives almost always includes at least one candidate that is not truly viable. A market researcher only learns this when, to everyone's surprise, one of these "filler" alternatives wins the test. When this happens, the credibility of a valid test is often undermined because Marketing cannot roll out the winner (since it is not viable), and the highest-scoring viable alternative is tarnished by losing.

Worse yet, this type of disaster usually reflects poorly on the in-house market research staff, and rightly so. It should be the market researcher's responsibility to test only well-considered, workable alternative concepts. In advance of the test, the marketing staff should be made to grasp the possibility that any of the alternatives may win. The market researcher must ensure this, because a test that includes unfeasible alternatives is improperly designed.

This is the purpose of the championship. The output from the championship is guaranteed to be viable. Furthermore, each alternative has at least one strong supporter in the marketing department. This locks in the design of the quantitative test and greatly enhances the likelihood that the result can and will be implemented.

In addition, because the fundamental reason for doing any market research is to reduce uncertainty, untrained focus group observers (aka impatient marketers) can find focus groups to be frustrating. Focus groups do not really resolve anything. Because they urgently want to eliminate any uncertainty, these observers are often tempted, inappropriately of course, to boil down their observations into one message or one product idea. This problem is skirted in two ways. First, at the beginning of the workshop session the facilitator describes the process, so participants should expect to enter the quantitative test with several strong alternatives. Thus they should be comfortable with some uncertainty. Second, after the championship, the remaining alternatives will be strong . Though everyone may have their personal favorite, no one will be able to simply assert that one alternative is the one that customers will like best.

Possible risks and disadvantages

That the qualitative research provider's analysis is not available during the idea creation phase is a possible weakness of the process. The in-house researcher should decide whether the value of immediate debriefing outweighs the loss of the fresh perspective and thorough analysis provided by a good qualitative researcher. Depending on timing, the provider's analysis could still be used to kindle creativity during the championship phase.

One possible solution is to have the qualitative researcher facilitate the workshop and the championship. This at least reaps the advantage of his or her fresh perspective. However, the market research staff should carefully consider providing this service themselves. It is an opportunity that should not be passed up lightly.

Insider offers advantages

An in-house facilitator's vantage point is much better than that of an outside facilitator. The in-house person will be more in tune with the way the company works and the personalities of the participants. Participants will feel more at ease discussing sensitive new product concepts in front of an insider. An insider tends to be more available to help the participants than an outsider, who may be distracted serving other clients in other industries and markets.

In the description of the workshop/championship process, there is a presumption that market researchers and marketers participate side-by-side in all aspects of the research process up until the quantitative test. For many firms, the market research and marketing functions are well-integrated. In other firms, especially those that sell products and services to other businesses, the two functions may not be accustomed to working closely together. The benefits of drawing market researchers and marketers together in such firms are obvious. A well-managed workshop/championship is a contribution to the firm. Managing the process and facilitating the sessions is a way in-house market researchers can add value beyond presenting study results and making recommendations.

In these times of hand-wringing about the real value of market research, we need to demonstrate the power of the market research process. Contrary to what many in-house market researchers might think, the value added is not solely in being a conduit of information, but also in the management of this process. In-house researchers are uniquely positioned to provide this service since it is the only thing missing from the array of services offered by most full-service research firms.

The author thanks David Shaw of the Hewlett-Packard Company for his valuable input to this article.