Editor’s note: Howard Waddell is president of Decision Resource, Inc., Miami.

A survey of an organization’s customers might reveal a number of individual product or service elements that are in some way deficient. If that is the case, then management must decide which of these elements to address first. One approach is to try to fix the things that customers say are most in need of fixing as indicated by low performance or rating scores. However, some of the weakest elements may not be among those most important to the customer.

A simple solution to this problem would be to ask the customers how important each of the various product and service elements is. Unfortunately, a too-common response is that everything is important. Consequently, the problem isn’t solved.

In this article, we will explore a fairly straightforward way of determining the elements that have the greatest impact on overall customer satisfaction. From this it is a short step to setting priorities for action.

During the spring of 1997, our company, Decision Resource, conducted a mail survey of investors with a net worth of $1 million or more. One of the primary objectives of the survey was to determine the individual service elements that were most closely correlated with overall customer satisfaction and, of these, which ones were most in need of improvement.

Altogether, 225 questionnaires were mailed to people (mostly men) in households with a net worth of at least $1 million. Of these questionnaires, 97 (or 43 percent) were completed and returned. The statistical information which follows is based on these responses.

The investors were asked to provide an overall rating of their stock broker or financial advisor. Their responses are shown in Table 1.

Table 1 
Overall Rating of Broker/Advisor

Superior

19%

Excellent

19%

Good

37%

Fair

25%

Poor

--

TOTAL

100%

The rating scale used on this survey is one we use quite often. There are several reasons we like it. First and foremost is that it tends to coincide with how people judge a product or service in their own minds. If you gather together a group of laypeople and ask them to devise a product or service rating scale, it’s unlikely they will come up with the kinds of scales many market researchers like to use. (How likely would it be that such a group would develop a 10-point scale anchored at the ends with "excellent" and "poor"?)

The second reason we like this scale is that it has an odd number of points. It’s not good to force fence riders to jump to one side or the other, particularly when it’s been repeatedly shown that most of these people, if forced, will be more generous in their evaluations than less generous. Third, this scale includes a "superior" category. If people are, by nature, generous with their evaluations, at least in a marketing research context, there must be a way to differentiate those who would rate an organization "excellent" and those who would rate it as "virtually unrivaled." "Superior" does this for us as is shown in Table 1.

In addition to obtaining an overall rating of the type just described, the researcher can also measure overall customer satisfaction by asking the customer if he or she would recommend the product or service to a friend. In those surveys where we ask such a question, our choices are usually limited to three: "Yes, enthusiastically," "Yes, but with qualifications," and "No." In this particular survey we did not ask this question. In hindsight, we should have. The survey revealed that 44 percent of the investors first became aware of their broker or financial advisor because he or she was recommended to them by someone they knew. In this segment of the investor market, the ability to get referrals appears to be critical to success.

After providing the overall rating, the investors were asked to indicate how much improvement was needed by their broker or advisor for each of several service elements. The choices were "No improvement needed," "Some improvement needed," and "Considerable improvement needed." There are a number of reasons we prefer this "improvement needed" scale. Foremost among them is that the answer given is a message to management that requires absolutely no translation. Contrast that with answers such as "Exceeded my expectations," "Met my expectations," and so on. These answers are meaningless unless you know what the expectations are. Even then it’s not obvious what to do with the information. If you go to the post office expecting to wait in line for 20 minutes, but only have to wait 15, just how satisfied are you?

Table 2 shows the average ratings of 14 individual service elements. The scores were developed by assigning a numerical value for each of the three answers: three for "No improvement needed," two for "Some improvement needed," and one for "Considerable improvement needed." (A fourth option, "Don’t know/Not applicable" was also available to the investors. These responses were not included in the calculations of average ratings.)

Table 2 also shows the correlation (r2) between each service element and the overall rating. The correlations were mathematically derived directly from the survey data. This correlation is an indicator of the degree to which the individual service elements influenced the overall rating. It is also an indicator of how important each service element is to the investors’ overall assessment of their broker or advisor.

Table 2 
Ratings of Individual Service Elements and Their Correlations With The Overall Rating

Rating (1-3) 

Correlation (0-1) 

Explaining investment risk

2.88

0.13

Frequency of account statements

2.87

0.06

Willingness to correct problems quickly

2.83

0.07

Accuracy of account statements

2.79

0.06

Variety of investment products offered

2.76

0.16

Following instructions

2.76

0.11

Prompt reporting of buy/sell transactions

2.75

0.11

Understanding your financial objectives

2.69

0.22

Frequency of contact

2.61

0.20

Simplicity of account statements

2.50

0.01

Recommending what is best for you

2.50

0.26

Quality of buy recommendations

2.22

0.35

Quality of sell recommendations

2.10

0.28

Commission rates

2.00

0.44

Average

2.59

0.18

A quick look at the table shows that there is no strong relationship between the individual ratings of the various service elements and the correlation figures. To help sort out the information in Table 2, a 2x2 grid was set up to separate the service elements into four groups. This grid is shown as Table 3. 

Table 3
Below average correlation Above average correlation
Above average rating Accuracy of statements
Frequency of statements
Correct problems quickly
Variety of investment products
Following instructions
Prompt buy/sell reporting
Understanding your objectives
Frequency of contact
Below average rating Simplicity of statements Recommending what's best for you
Quality of buy recommendations
Quality of sell recommendations
Commision rates

The lower right box in the grid identifies those service elements that are the primary problem areas for this upscale segment of investors. These four items received below average ratings and had an above average degree of influence on the overall rating (as measured by the correlation coefficient).

Service elements listed in the upper right box are the industry’s strengths. These two items received above average ratings and had an above average degree of influence on the overall rating.

Elements listed in the upper left box received above average ratings but were not highly correlated with the overall rating. No additional resources need be applied to improving industry performance in these areas.

The one service element ("simplicity of statements") listed in the lower left box received a below average rating but did not have much influence on the overall rating. Correction of this problem does not need to be a priority.

An analysis of the type described has one weakness that should be pointed out. If all of the major competitors in the market have the same deficiency, say, "Simplicity of statements," then this service element will not differentiate one competitor from another. Consequently, the correlation between "Simplicity of statements" and the overall rating will be small. If this is the case, there may be an opportunity for the astute brokerage house to further differentiate itself from the competition by creating statements that are readily understood. A review of the data itself will show how much response variance there is for each of the individual service elements.

The data used in this article show how all the upscale investors surveyed view their individual brokers and financial advisors. However, the strengths and weakness indicated are those of the industry, not of the individual brokerage houses or advisors. The very same techniques can be employed to look at the customers of specific organizations to determine what the important strengths and weaknesses of those organizations are. Priorities for action can then be set.