Editor’s note: Bob Beaulaurier is partner, and Marlene Holm is an account executive, with Market Decisions Corp., a Portland, Ore., research firm.

If you’re faced with a situation where you need to test reactions to a number of variables or trade-offs, the analytical hierarchy process (AHP) is a method that works well when there are few variables to test. Testing five or fewer variables is the optimum and you likely can do the number-crunching yourself. It works particularly well when helping manufacturers determine how to bundle product features, how to determine which brand goes with a particular product or which payment options to offer, etc.

AHP allows the researcher to take several decision attributes or known variables that go into a decision process and, by polling respondents, determine which are the most important. Decision variables or product features are paired against each other and respondents pick the feature they like best among the two variables/features, e.g., cost vs. reliability. Respondents can choose between two choices or variables at a time. Factors are then stacked up against each other on a scale from 1 to 100, which roughly equates to percentages or impact on a 100-point scale among the factors tested.

The main advantage of this technique is that it can effectively determine the relative importance of items used as criteria for making a decision and show which choice might have the best “play.” Traditional research methods often fall short because you can only test one feature against another. It’s also easy to use and requires no fancy software or proprietary techniques.

“When asked to rank or rate a list of things according to some criterion, such as preference, value, risk or cost, one might be able to rank their order and even to assign some numbers to their relative positions on the list,” wrote David Hallowell, a founding partner of Rockland, Mass., consulting firm Six Sigma Advantage, in an online article. ”However, two problems arise in that simple scenario. First, whatever measurement scale is chosen is just ordinal at best. A rating of 10 does not mean the preference, risk or whatever for an item is twice that of an item rated 5. (One might be tempted to treat the numbers as a ratio scale, but there really is no basis for it.) Second, when there are more than a few items on the assessment list, it gets hard to keep all the prioritization considerations in one’s mind at the same time - making it hard to think about and to complete the task. AHP takes that simple-enough looking prioritization problem and makes it simpler and more meaningfully measurable. It reduces the list into pairwise comparisons and asks for a ratio assessment of each pair.”

For example, if one were analyzing which home luxuries were the hot buttons for home buyers, one might trade off “three-car garage” versus “central air conditioning” versus “view of the city” versus “gated community” versus a “gourmet kitchen.” Using AHP the relative performance of the home luxuries could be compared. The question might look like:

Q1. If you were choosing between home options, please select the one that is more important to you:

1. Central Air Conditioning               

2. Gourmet Kitchen

With AHP, each of the attributes, in this case home luxuries, would be compared against all other home luxury items. This means you know the relative performance of each attribute versus all other attributes. Conjoint users often do not have direct comparisons, which makes it difficult to use the analyses for different situations. With AHP it is easy to take out variables and rerun the analysis because you have data for each variable. So a home developer in this example could use the analysis again when going to a location that did offer city views, for example, to see which attributes would be the top three to include in a marketing brochure.

Similarly, AHP can be easily utilized for subsegments. This is particularly useful when you do not know who the target audience is ahead of time. Again, this may be more difficult with some other analyses which do not collect data for each attribute and it could turn out that your target market is interested in a subset of variables that were not compared to each other directly. In the luxury home example, because we asked about “view of the city” against all other variables we can take it out of the analysis. Similarly, in a place like Phoenix where central air conditioning is a given in any home, we could take that out of the analysis in that market and see the relative performance of the remaining variables.

AHP also helps capture both subjective and objective evaluation measures, providing a useful mechanism for checking the consistency of the evaluation measures and alternatives suggested by the team, thus reducing bias in decision-making.

Getting started

AHP is basically a five-step process:

1. Identify the factors of the decision.

2. Compare each factor against every other factor, indicating how much more (or less) important Factor A is to the decision than Factor B.

3. Change those “soft” comparisons to numeric values.

4. Place the numeric values into a pairwise comparison matrix, with the As in the rows and the Bs in the columns. Note that each answer results in two cells being filled.

5. Normalize the matrix to get the weights.

For example, if one team within a company believes that Factor A is much more important to customers than B, B and C are of roughly equal importance, and D is slightly more important than either B or C and moderately less important than A, the analytical hierarchy process can show the relative weights (in percent) of how important Factors A, B, C and D are.

This also gives a starting point for tweaking a decision. For instance, suppose Factors B and C, instead of being exactly equal, were a couple of percent different. You can test to see how relevant that difference is. Once you have the weights of these factors, you can run what-if scenarios and simulate their impact on the bottom line.

Using the home luxury example, suppose that “view” and “gated community” perform relatively equally. It may be possible to focus on promoting the “gated community” aspect instead of property views, which might give an equal punch to marketing efforts for a lower cost since properties with views are often more expensive for developers to purchase and for homeowners in terms of property taxes, etc. Of course, a caveat is that underlying these decisions may be other factors (in this case security and location, location, location), so some care and consideration also goes into transferring the analysis to other situations.

How the technique works

Each of the cells in the pairwise comparison matrix is the quotient of the row it represents by the column it represents (row divided by columns). The first cell in the upper left-hand corner is A/A, which will always equal one - same for B/B, C/C. The second cell in the first column of the matrix is row B, column A. Here, find the question where Factor A is compared to Factor B. Then, take the number of respondents choosing Factor B and divide that by the number choosing Factor A for that question. This quotient is then put into that cell.

This procedure is followed for all of the cells. For each cell, there is also a cell with its inverse in the matrix. Recognizing this saves looking up the question twice when finding the quotient for each of the cells. Next, the columns of this comparison matrix are added. The sum is used to weight that column and each of the factors in that column are divided by the sum. The resulting matrix is called an adjusted matrix. The average of each of the factors’ rows in the adjusted matrix is a simple mean. It is gathered by adding the cells in the row and dividing that sum by the number of cells in the row. This average is the weighting that respondents put on that factor on a scale of 1 to 100. This procedure can be applied to more or fewer features or attributes.

As an example of AHP in action, say a utility company wanted to determine which type of payment method (by check, by phone or online) its customers most preferred. To determine the relative strength of payment options customers were asked to choose from the following:

1. Check vs. pay-by-phone

2. Online vs. check

3. Pay-by-phone vs. online

All options are traded off against each other as shown in pairwise comparison matrixes 1 and 2.

Note that we really only have choices for three items in comparison matrix 3. As shown in matrix 1, three of the options are comparing one variable to itself (A vs. A, B vs. B, and C vs. C). B vs. A is the same as A vs. B, so we do not need to ask that question twice. We recommend rotating the order to avoid any ordinal bias when administering the questions. Adjusted comparison matrix 1 shows the ratio of respondents indicating their choice was the row over selecting the column.

The adjusted comparison matrix is a fancy name for a table that weights the relative performance of the attributes tested so that they value 100 points by dividing the column totals from the pairwise comparison matrix. Pairwise matrix 3 shows the numbers utilized and the results are shown in adjusted matrix 2. Since most decision makers and market researchers are very comfortable with 100 points, this is an appropriate way to show the relative strength of each of the attributes. In short, the adjusted comparison matrix weights each variable’s contribution out of 100 points.

The results in adjusted comparison matrix 2 show that the relative performance of paying by check (70) still significantly outperforms the online (16) and pay-by-phone (14) options for this particular utility company.

Not just a gut feeling

AHP offers the following benefits:

  • It converts “soft” assessments (used by people) into “hard” values (that can be used by programs). Example: In choosing a car, John likes the V-8 power and Jane likes four doors best. With AHP we can look at car buyers overall and see what percent of the time they choose V-8 or four doors over another feature. We can even run separate AHP analyses for males versus females so that we are not just taking product managers’ or executives’ gut feelings for what the marketplace wants.
  • It allows for shades of gray. Continuing with the car theme, suppose the V-8 motor is not an option. Traditional research approaches are difficult to use when you pull out variables once you get to the analysis phase. With AHP, because of multiple paired comparisons, you can run the analysis without the V-8. Each variable is compared against each of the other variables, so you know definitively from each respondent which option they would choose because no variable comparison is left out of the analysis as with conjoint or some other common analyses.

Similarly with our utility payment options, using the same data that is already collected by comparing each pair against every other pair, you could see the percent choosing “paying by check” over “paying online” if “pay by phone” were eliminated from the options. Therefore it gives you the ability to simulate different options and different costs associated with different options.

By using AHP, we can often go right to the behavior and the activity people would choose, avoiding all of the other variables that a fancier analysis might include. This can be particularly helpful if there are lots of variables, parameters and baggage that people use to make their decision.

A simple AHP analysis isn’t always better, but it can be straightforward to use. Though we advise you to always consult and support your market research professional, AHP is something that most qualified researchers can do with a simple spreadsheet without buying sophisticated software.