Editor’s note: Michael Lieberman is founder and president of Princeton, N.J., research firm Multivariate Solutions.

To some degree, all decisions involve choice. Individuals choose among different alternatives; commuters choose among possible routes and methods of transport; shoppers choose among competing products based on attributes such as price, quality and quantity.

Unlike with traditional polls and surveys, choice model predictions can be made over large numbers of scenarios within a context, to the order of many trillions of possible scenarios. Choice modeling is the most accurate and general-purpose tool currently available for making behavioral predictions, and human decision-making is regarded as the most suitable method for estimating consumers’ willingness to pay for quality improvements in multiple dimensions.

In marketing research, the most common types of choice models are forms of conjoint analysis, such as discrete choice modeling or paired-comparison analysis.

One option for choice modeling is maximum difference (max-diff) analysis (or scaling). Max-diff is based on customer choice or trade-off instead of typical rating-scale responses. Max-diff is the multinomial extension of the traditional method of paired comparisons (Thurstone 1927, David 1988). Whereas a paired-comparison question asks a respondent to make a binary choice, max-diff has the respondent specify “best” and “worst” choices from sets of three or more objects.

Most importantly, max-diff allows the researchers to test a large number of attributes without having to resort to unwieldy, large orthogonal models or the complicated process of adaptive conjoint analysis. What can be accomplished with 10 relatively simple max-diff choice scenarios replaces up to 70 paired-comparison analyses.

A maximum-difference choice model is easily administered, has multiple levels of analysis and is a very effective tool in establishing the relative priority of such items as:

  • potential message for a new product;
  • features or benefits of a service;
  • which extras to include in a loyalty program;
  • which political message most resonates with different segments of the public;
  • fundamental customer interests and activities;
  • unmet/future needs.

Max-diff can eliminate the awkwardness of a large set of customer choices, move respondents rapidly through the survey, ease respondent fatigue and remove problems of clustering of attributes at either the top or bottom end of discrete scales. Moreover, max-diff reveals the descriptive results that companies are looking for and can be applied to large databases to predict future customer behavior.

This article will run through examples of a max-diff, presenting representative case studies showing the three levels of output that a max-diff analysis can deliver.

Maximize its loyalty program

Our fictional hotel chain, Malone Gardens, one of several chains owned by Colossal Hotels Inc., is seeking to maximize its loyalty program. It has identified 12 loyalty benefits that it’s considering offering its guests:

- free hotel nights;

- experience getaways;

- dream vacations;

- premium merchandise;

- airline miles;

- bonus points;

- complimentary health club

   privileges;

- hotel room upgrades;

- reward planner services;

- spa or golf packages;

- partner car-rental privileges; and

- shopping and dining.

Malone Gardens is conducting a survey across a sample of approximately 2,000 potential Malone Gardens guests to see which of the 12 rewards would maximize loyalty to the brand. Examples 1 and 2 show potential choice scenarios.

During the course of the Malone Gardens survey, each respondent might see eight to 10 or more of these randomized scenarios (if a respondent were to see all possible matches, there would be 495 choices to make). A key benefit of max-diff is that, unlike conjoint analysis, all respondents are not required see the same scenarios as each other in order for the analysis to be performed. Thus, each respondent in the sample may see completely different sets of max-diff choices. This has no effect on the analysis.

The first outputs are general charts (such as Figure 1) that show percentage differences for each attribute. We refer to these as high/low visuals.  Among scenarios with each reward present:

  • a reward scores +100 if “most appealing”
  • “not chosen” scores zero
  • a reward scores -100 if “least appealing”

For each attribute, scores are calculated based only on those respondents for whom the attribute was included in the choice set.

As Figure 1 shows, evidently, Malone Gardens customers prefer immediate gratification.

The next step is to show these results visually. Figure 2 shows a chart of descending positives. The larger the contrast between “most appealing” (blue), and “least appealing” (red), the more desirable the attribute. Figure 2 shows the six most desirable rewards.

The next phase of output from Malone Gardens is to estimate the impact if certain packages of options are offered in the loyalty program. The max-diff data is built perfectly to establish this.

In a discrete choice model, the data is set in an array in which an attribute variable is 1 if the attribute is present in the choice scenario and 0 if not. The dependent variable is 1 if that attribute is chosen, 0 if not. This is a multinomial logit regression model. It is used to simulate real-world decision outcomes.

In the case of Malone Gardens, the logit model is run and conjoint utilities are constructed. These utility scores are shown in Figure 3. The practical value of the utilities is that they allow Malone Gardens to estimate the bump in loyalty program enrollment with the addition of one or a combination of loyalty benefits.

We instructed Malone Gardens to ask respondents the likelihood of their signing up to become a Malone Gardens Priority Club member. Malone Gardens then asked the same question after the max-diff choice exercise. As expected, there was a rise in enrollment in the Malone Gardens Priority Club - the gap.

We do not use the gap as a dependent variable in any regression exercise. Rather, the gap provides endpoints for the max-diff regression model, which we now fit between the gaps.

Figure 4 shows the max-diff simulator before the choice exercise. In this “before” scenario, on average, customers are 5.6 out of 10, or 56 percent, more likely, to join the Malone Gardens Priority Club.

Figure 5 shows what happens when Malone Gardens offers a combination of three benefits: free hotel nights, hotel room upgrades and experience getaways.

After incorporating the max-diff results, the likelihood to join Malone Gardens Priority Club has risen to 7.3, or 73 percent, a jump of 17 percent. The max-diff simulator allows Malone Gardens to model literally hundreds of what-if scenarios, simulating any individual or combination of loyalty awards. 

Within the context of this max-diff approach, individual-level utilities can be calculated. The max-diff equation produces a logistic regression model. When a respondent’s choices are set within the regression model, an odds-ratio for each respondent is created using the formula of 1/(1+e-z).

For example, if a given respondent’s choices are added across the maximum difference questions, then applied to the logistic regression model, the respondent might produce an odds-ratio of .6.

When compared against the maximum likelihood to join Malone Gardens Priority Club (100), this would give the respondent an odds-ratio of 100*.6=60, or a 60 percent chance of joining Malone Gardens Priority Club based on the choices made in the max-diff exercise.

Test price sensitivity

In our second example, the client, a casual-dining restaurant, needed to test price sensitivity for 11 main courses, each at four price points. Figure 6 shows the main courses and the price points tested.

Given the large number of attributes, a conventional Grange-Gabor price wheel or discrete model would be unwieldy. Instead, we employed max-diff. Using an orthogonal design, we settled on a 15-run design to determine which attributes would be present in each of 15 choice questions. The order of questions was randomized, and price levels were randomly assigned to each attribute, in each question, for each respondent.

Simply put, respondents saw 15 scenarios with varying menu choices at varying prices and were asked which they would most likely choose and which they would be least likely to choose.

This approach allows us to construct a discrete-choice model similar to the logit model described above. We can construct utilities for each item (though not individual conjoint utilities) and a price function, and then build a simulator to model item demand (by exponentiating the utility of all items in the market, then dividing by the individual items). Figure 7 shows a simulator that contains seven menu items at varying price points.

Examining the outcome, the restaurant’s management can see that beef ribs, at their lowest price point, have a demand of 24 percent, and that pricing the double BBQ bacon cheeseburger above $11 will drive demand down below 10 percent.

Using latent-class analysis, or cluster analysis, respondents could be grouped into segments. That is, by clustering different groups, we can see which prefer steaks and chops, vs. seafood, vs. trendier dishes such as teriyaki chicken.

Simple and effective

There are many applications for the accurate measurement of choice modeling. Max-diff scaling is not new but is emerging as a simple and effective option for choice models. It can be used to assess the extendibility of a brand’s loyalty strength and brand equity, to refine a brand’s communication efforts by identifying segments in which a brand’s image is strong, and/or as a way of monitoring the competition. Whatever its application, this method of choice measurement is a sophisticated tool that helps keep marketers one step ahead.