Editor's note: Lori Laflin is the director of data processing at CJ Olson Market Research, Minneapolis. David E. Hyatt is the customer results and service manager at Decotiis Erhard Strategic Consulting, Colorado Springs, Colo.

Customer satisfaction and customer retention are the cornerstones of many business mission statements and strategies. This is a good thing. Businesses in this country and around the world are focusing on their customers like never before. Rather than Henry Ford's "any color as long as it's black," philosophy, today's executives are developing quality initiatives and taking to heart catchphrases like "mass customization." This focus on the customer requires that businesses actually talk to their customers to determine what it is they want as well as how effectively goods and services are being delivered. The result is that many people who never had to think about survey research have suddenly found themselves in a position of responsibility for the company's entire customer satisfaction program. And, while anybody can write questions, it is much more difficult to write good or meaningful questions.

To have an effective survey instrument (i.e., one that provides actionable information) and make the best use of the information you collect, it is important that the items developed have the following characteristics:

  • they measure attributes that are relevant to the customer;

  • they measure attributes that affect behavior;

  • they measure those attributes reliably; and,

  • they produce actionable results.

So, how do you do this? This article discusses how to develop survey items with the above characteristics. While we'll focus on customer satisfaction research, many of the principles discussed are just as relevant to other research applications.

Generating content

In general, there are four ways to generate survey content or items: make it up, use research literature, copy existing surveys and use focus groups and qualitative interviews.


Making it up, while not the most scientific or best way, is probably the most common method of generating survey content. While this method is often relatively inexpensive up front because "we know what's important to our customers," it is often the most costly in the long run. If the items developed do not reflect things important to customers or don't result in actionable information, the survey project will be a waste of your valuable time and money. Therefore, we recommend using one of the other methods.

Rather than making up the content, there are two sources that can be quite useful in the development of survey content: research literature and existing surveys. There are numerous articles in which the authors discuss customer satisfaction and survey content. Distilling the essence of these articles and the content areas they discuss and then developing a survey to measure them for your own purposes can be quite effective. These articles are often available over the Internet or at your local library. Existing surveys might include something done previously at your company or done by a competitor. While these can provide you with good background and help point you in the right direction, consider the following: the survey content used for one study is not necessarily relevant to yours. For example, one survey might assess customer satisfaction with a call-in help line. While you may have a call-in line, perhaps your calls are of a more technical nature or your representatives deal with different types of callers. So, using the questions from the existing survey to ask about the help line on your survey would be a waste of time. Make sure that the questions you ask are relevant to both your audience and your organization and that you can actually act on the suggestions you get from the survey results. (Also, be careful copying survey content - you can run into copyright problems.)

The final way to develop content is the one we recommend for most applications: using qualitative data collected through focus groups and one-on-one or small-group interviews. The remainder of this article will discuss the techniques you can use to develop an effective survey instrument. Whether you use focus groups or one-on-ones or some combination will depend primarily on the people you need to talk to (where are they, who are they and so on). For more help choosing the appropriate qualitative research method, contact a reliable research supplier.

Using focus groups and one-on-ones and small-group interviews

In the remainder of this discussion we will use the following terms: "construct," "content area," and "item." A construct is a concept that is being used for scientific purposes (and we all know that designing surveys is a science). It represents an intangible idea, such as satisfaction or happiness. You can't see it per se, but you can see the implications of it. For example, you can't see happiness but you can infer the state of someone's happiness by looking at them and noting their behaviors. In survey research, the content of your survey will be designed to reflect certain constructs. Because constructs are usually complex, and often reflect more than one idea, a survey content area refers to a group of items that reflect one idea. For example, if we are trying to measure the satisfaction of airline passengers, we know that there are many things that could affect their satisfaction: registration and ticketing, baggage handling, timeliness of arrival and departure, in-flight amenities and so on. If we want to accurately gather a complete picture of these passengers' satisfaction with the airline, we need to ask them questions about each of these content areas. Finally, the items are the individual questions for which you would like respondents to provide information. Each content area will be made up of several items.


When you have chosen to conduct qualitative research to develop a survey instrument, one of the first decisions you will need to make is: Who should we talk with to determine the relevant constructs and content areas? There are five potential groups of respondents: customers, employees, executives, lost customers, and prospects or non-customers. Each group will provide a unique perspective. Customers, lost customers and prospects can talk about what's important to them; employees, particularly those on the front lines, can provide information on what satisfies or dissatisfies customers; and, executives will provide information on long-range goals (also, it's nice to keep them in the loop to gather support for your project). It is critical, at the bare minimum, to talk to customers.

There is a wide variety of questions you can ask in the focus groups, but the key to a successful focus group is to remember that it is, in essence, a group discussion. The moderator is there to guide the flow of the conversation, but the real information comes from having participants think about the responses of others and comment on them. Ask customers what makes them happy in a particular situation and what makes them unhappy. Ask them what would cause them to change providers. Ask them what made them try you in the first place and if their initial perception was correct. Ask them about your competitors' strengths and weaknesses. Finally, get specific examples of positives and negatives and make sure they clarify anything ambiguous. Rather than "I had to wait a long time," ask them to tell the story that surrounds their statement: how long did they have to wait, what did they do while waiting, how did it make them feel, and so on. Be sure you are capturing all of these great ideas on a flip chart or using a tape recorder.

Returning to our airline industry example, groups could be conducted with current customers (perhaps some with business travelers and some with pleasure travelers), with front-line employees including ticket agents and attendants, with people who prefer other airlines, and with executives. It's generally easier for participants to focus on problems rather than on what has gone well; however, encourage them to think carefully about positive experiences. Speaking from personal experience, we recall being delayed in Denver with a flight we thought would never leave. The United Airlines gate agents kept the waiting passengers entertained with a group game of trivia. The first to shout out the correct answer got a $50 travel voucher. What impressed us was not the token reward, but rather that the airline considered the passengers to be their responsibility even before boarding the aircraft and showed some concern for our inconveniences.

Now what?

After conducting the focus groups, the real work begins. The discussions will need to be analyzed. To examine the results of the qualitative research, take a look at the stories you were told -- think about cutting up transcripts or highlighting them to isolate specific stories. For each story you hear, ask yourself, "What are they really saying?" For each unique story you hear (some may overlap), complete one of the following sentences:

  • To delight this customer, (this must happen or this condition must be satisfied).

Again referring to our airline example, we may have heard several stories from passengers about delays, cold or inedible food, or lost luggage and from those have generated specific statements such as:

  • To delight this customer, the luggage must arrive with the flight and be delivered within x minutes of de-planing.

  • To delight this customer, the ticketing agent must proactively offer flight options.

  • To delight this customer, the flight should depart on time (actually leave the ground, not just push back from the gate).

  • To delight this customer, the seating area must be clean.

And so on.

After you have generated your statements, group similar ones together and remove any redundancies. Look at the groups you have put together. What is the theme that holds each group together? These themes will become your content areas. This can be hard to do at first. Essentially, you need to take a step back from each of the specific statements and determine the underlying idea behind them. For example, if you heard stories about rude flight attendants, rude ticket agents, rude customer service reps, and lack of customer service support for late planes, you have probably identified a content area of customer service. Each of those stories talks about different groups of people, but they all deal with customer service.

Examine your groups and your content areas. Does it seem that you have covered everything and, conversely, do some of your areas overlap? Refine these as necessary. The content areas become the sections of the survey. Additionally, all those statements you created become the core of your survey items. In our airline example, the following information was gathered. To delight the customer, the ticketing agent must:

  • be able to answer questions and provide solutions,

  • proactively offer flight options,

  • explain the carry-on requirements,

  • be courteous and polite; and,

  • conduct the transaction professionally.

The content area is "satisfaction with ticketing agent" and, using our statements, the questions write themselves:

How satisfied are you that the ticketing agent you spoke with . . .

  • . . .was able to answer your questions?

  • . . .provided solutions to your travel restriction problems?

  • . . .offered flight options?

  • . . .explained the carry-on requirements?

  • . . .was courteous and polite?

  • . . .conducted the transaction professionally?

(Sorry, but questions about which scale to use are beyond the scope of this article.)

Most cost-effective

In conclusion, although qualitative research can seem like a costly way to generate survey content, in the long run it can be the most cost-effective. Rather than relying on gut instinct or previous studies that may or may not apply to your situation, you are "asking the experts." Through this process you are increasing confidence that the survey items you generate are good items and that they reflect attributes that are important to customers. Also, this process should help you marshal employee and executive buy-in to the survey process, particularly if their opinions were sought up front. Furthermore, studies have shown that survey response increases when the items are important to the customer. This step can help you achieve a higher response rate than you might have achieved otherwise.


Finally, one last instruction - now that you have gone through all this work to develop your survey content, make sure that you make the best use of the information you collect and actually act on the findings!