Editor's note: Robert Miller, Ph.D., is a senior vice president of Cincinnati-based MarketVision Research, Inc., with responsibility for the Statistics and Marketing Sciences Group.

Conjoint or trade-off analysis has become one of the most valuable marketing research methods for guiding new product development as well as reassessing features and benefits of existing products or services. The value of conjoint analysis is its ability to prioritize the consumer's decision-making process in order to understand the value of a product's individual features or benefits. In short, trade-off research forces respondents to select one product feature while at the same time trading off this feature for others. In addition, trade-off analysis permits simulations of product offerings in order to gauge customer acceptance of new products and services. The results provide insights and guidance for the introduction of new products, the restaging of current product lines, and the development of product extensions.

According to Jim Tumbusch, a pioneer in conjoint research and senior analytical consultant in MarketVision's Statistics and Marketing Sciences Group, "although trade-off research has become widely accepted, some managers need to be more comfortable in their understanding of the methodology and how trade-off research can be used for product development and product enhancement decisions."

To increase a marketing manager's understanding, the process of conducting conjoint research will be discussed here to facilitate greater effectiveness in conducting conjoint research leading to more actionable results. More specifically, the following key topics will be covered:

  • effectively preparing for a conjoint study
  • clarifying product attributes and levels
  • the value of pilot testing
  • efficient data collection
  • making results clear and actionable for management

Effectively preparing for a conjoint study

One of the most critical steps in conducting a conjoint study is to carefully develop a list of product attributes (or features/benefits). Although this process is often time consuming and challenging, the creation of carefully worded attributes is the key to a successful conjoint study.

We have found that the best approach is to include product management and marketing in developing the attribute list while in a setting where free discussion can occur about every possible feature or benefit. This process often requires multiple meetings and a well trained researcher to clarify each of the product features and to fully challenge the thinking of all participants so that each attribute and level is clearly stated.

For example, one of our clients, a national health care plan provider, wanted to configure different health care plans to meet the needs of several target segments. Among the attributes we examined were monthly cost of the plan, availability of dental and/or prescription coverage, and number of doctors and hospitals covered under the plan. The prescription coverage attribute had three levels: no prescription coverage, covered at 80% of expenses exceeding the first $100, and each prescription costs $4. The process of defining these attributes provided an opportunity for all members of the product team to consider every aspect of the product. It provided MarketVision with good insights into the client's existing level of product knowledge and understanding while allowing the participants to consider all possible features/benefits and ways the product could be improved. Moreover, this approach unified the product management team and forced a rethinking about all product attributes.

It is often important in the early planning stage for the marketing research company to provide guidance. For instance, our firm conducted a large business-to-business conjoint study to provide input in the development of a new industrial product. Since research and development could create a wide range of products to satisfy the technological needs of our client's customers, a large number of product development decisions had to be made. Each of these decisions involved real world trade-offs on items such as safety, product stability, weight, packaging, and costs. This client was relying on our firm to guide the product development team through a complex matrix of decisions. Since there was unfamiliarity with trade-off research, our firm was asked to show the benefits of this approach as well as how the outcomes of this type of research would aid the product development team. According to Don McMullen, MarketVision president, "We needed to partner with the client and offer the right research solution within a tight time frame and a limited budget."

Our firm's first task was to educate management and product development on the capabilities and limitations of trade-off research. Clear examples of how this research would provide answers to specific research questions became vital. For instance, examples of the report format and data output were provided so that the final report would be fully understood. This education process clarified client uncertainties about the actionability of results. It also provided an opportunity for our firm to:

  • Explain the conjoint process in detail.
  • Show through examples what the output would look like.
  • Demonstrate ways the information could be incorporated into the product development decision-making process.
  • Provide evidence on how this technique has been validated.

Clarifying attributes and levels

To clarify product attributes and usage, focus groups with target users should be conducted. Focus groups provide insights into how users think about the product, terms and language used, as well as identification of key benefits or features. Often it is useful to "tease out" responses by utilizing special techniques or exercises to promote discussion of product benefits. Mini-groups (of four to six respondents) may also be used to assure more independent thinking and more thorough probing of various product attributes. Equally important are client debriefing sessions in which an understanding of user perceptions are discussed and charted.

In a recent conjoint study completed for a health care products company, our firm conducted focus groups to determine existing product feature usage, terms used, and to identify potential new features or benefits. The groups assisted our firm in fine tuning product enhancements that were later included in the conjoint study. The groups also identified several new features not being considered and clarified terms and phrases that would be used in the questionnaire and product feature descriptions.

Pilot testing

The attribute list and levels should not be considered finalized until pilot testing is complete. During this process, a small number of respondents are invited to take the interview and share their comments and thought processes as they complete it. One of the key benefits of pilot testing is to identify areas of misunderstanding by respondents. This provides a method to evaluate how respondents interpret the attributes and how well they understand the instructions. The pilot respondents are also interviewed after they finish the survey to review their interpretation of terms and to assess overall reactions. Also, a consistency test is run on their answers to evaluate respondent understanding and overall comprehension. In a study completed for a durable goods company, our firm discovered that several questions only worked after careful rephrasing and changing question order sequence. The pilot test effectively prevented problems that would have occurred later when the study was fielded.

Efficient data collection

Central location, on-site visits, mall intercept, and diskette mailings are the most common ways to conduct data collection. Completing conjoint interviews by phone is only recommended with products for which there are a small number of product features or attributes which are easily understood when described over the phone. An overriding concern about phone interviewing is respondent attentiveness and comprehension.

Central Location and Mall Intercept. The key to successful central location or mall intercept interviewing is interviewer and supervisor training. The data collection staff needs to understand the general purpose of the study, be familiar with the interview, know how to answer questions from respondents, and be trained to recognize if the conjoint program is not functioning properly. Careful coordination needs to occur when concept boards or other product support materials are utilized. Often, when a product is futuristic and conceptually difficult to understand, a videotape is produced which shows or describes the features or benefits of the product. The videotape also assures consistent presentation of the product features across all respondents. In a study our firm completed for a tool manufacturer, it was necessary to show a videotape because the product was very futuristic and completely unfamiliar to the targeted respondents. The videotape, which demonstrated the product, allowed a quick understanding of how the product was to be used and its key benefits. It also provided a means for consistent presentation of the product benefits across all respondents.

On-Site Visits. This approach is most appropriate when the project is a business-to-business study or when working with high-level professional respondents. Our firm recruits all respondents by telephone and mail, followed by in-person visits at the respondent's office or work place, using laptop computers. If the research topic is conceptually challenging or complicated, respondents are sent a carefully worded description of the research purpose as well as a prelisting of attributes and features so they are prepared for the interview and their time is used efficiently. Business-to-business respondents also need an opportunity to comment on the product with an open-ended question so that their detailed understanding of the product or service will be reported back to the client. In a study completed in the chemical industry, our firm was faced with scheduling difficulties and geographical separation of client locations. In addition, it was critically important to have certain respondent companies represented in the study. To meet this challenge, our firm equipped a team of trained researchers with laptop computers and transported them to the respondents' respective plant locations to complete the interviewing process. As a result, all key accounts were included in the research as well as all customer segments.

Diskette Mailings. It often makes sense to conduct a diskette mailing when respondents are geographically dispersed and as a cost savings measure. The challenge of diskette mailings is return rate and sample self-selection. In addition, since interviews are completed independently, there is little opportunity to provide guidance during the interviewing process. A solution to return rate is to pre-recruit (identifying computer model and format) and to offer meaningful incentives once completed diskettes have been received. As a way to respond to sample self-selection, reminder letters, postcards, and telephone follow-up should occur with targeted respondents. Finally, it is helpful if overnight mail is used to emphasize the importance and urgency of the study. An excellent way to give guidance to respondents is by providing brief yet clear instructions as well as a 1-800 telephone number boldly printed on the diskette labels. It is extremely important that the marketing research company have on hand the staff available to answer questions and, if necessary, walk respondents through the interview. Using these techniques, our firm normally achieves a 60% to 75% return rate.

Making results actionable for management

One of the most challenging aspects of any conjoint study is presenting the results in a way that makes comprehension easy and at the same time shows clear direction for product management decision-making. As a way to enhance understanding, our firm uses visual translations of the utility scores, usually by market segment. In addition, product managers must understand how individual features or benefits will produce the greatest product acceptance and at what cost. These results can only be achieved by a thorough educational process showing how the results should be interpreted and utilized by the product development team.

As an example, in a study completed on an over-the-counter health care product, the client needed to make decisions about adding staff, manufacturing capacity, and enhancing the product distribution network. The conjoint study our firm conducted enabled these important decisions to be made with confidence and provided new strategic direction on a product issue not identified beforehand.

Conjoint analysis is a powerful technique that when used properly can be an effective tool for management decision making. Although this method is widely accepted, attribute level development and data collection continue to require expert ability and supervision. Moreover, marketing researchers need to provide innovative ways to present conjoint results that can be tied directly to the product decision-making process. These results must be technically sound, easily understood, and actionable. In a recent presentation to a group of regional business leaders, Don McMullen summarized the role of marketing research for the 1990s as follows: "We are prepared to go beyond the 'traditional' role of market research. The issues we are now being asked to address carry us outside the old boundaries of marketing research into marketing, sales, product development, logistics, and general management. It isn't enough to merely provide design expertise and summarize output. We are now looked upon to develop a broader range of talent and expertise in an effort to push forward the business functions that marketing research is expected to support. Conjoint analysis gives us an excellent tool to do so."

More than any other research technique, conjoint analysis is one which can positively affect marketing efforts by adding greater certainty to marketing decisions.