How to become a survey expert

Editor’s note: Marcie Levine is president of Saja Software, Inc., a Longmont, Colo., producer of survey software.

Let’s say your CEO or boss has just returned from a conference emphasizing the importance of gathering information in today’s competitive business arena. One case study that has been brought to their attention highlights a company that improved sales by making product decisions based upon research from existing and potential customers through a series of opinion surveys on their Web site. In fact, your boss has now decided it would be a great idea to survey current customers about a newly launched product. Before you know it, you find yourself responsible for this fantastic and exciting new project. There’s only one small problem: You’ve never created a survey and have no idea where to begin. It may seem overwhelming at first, but by following these eight steps you can be well on your way to becoming a survey expert.

Step #1: determine the objectives

It is important to pin down the objectives before beginning the survey process, because they are the reference points that guide the survey. Your objectives will influence the number of questions while shaping content and administration. Some questions to ask that will help determine the objectives of the survey include:

  • Why is this survey being done? What problem needs solving?
  • What information does the person requesting the survey need to know?
  • How will the response data be put to use?

When making decisions on the target audience, the demographic questions, and the survey structure, you can return to the statement of objectives to ensure that what is being asked will achieve the desired result.

Step #2: obtain corporate commitment

Your department will probably use the information from your survey to change or shape programs, products or services and even creative approaches. To produce the right kind of data that will help steer these decisions, it’s crucial to make sure that the people who requested the survey are committed to, and involved in, the survey process.

You need to write a plan that includes key actions, dates, roles and responsibilities, as well as other resources needed to administer and communicate the survey’s results. The chances of a successful survey are reduced unless you have clearly communicated the plan to the people involved.

Step #3: identify and know how to contact customers (the target audience)

When identifying customers and determining how to reach them, some useful questions to ask are:

  • How big is the potential customer base?
  • Will all the customers be surveyed, or only a portion of the market?
  • If only a portion of the customers will be surveyed, will you survey a random sample of the group or a specific subset of the group?
  • Who is responsible for providing or obtaining the names and addresses of the customers, if names are required?
  • What demographic data will be needed?

The answers to these questions will help you develop a mail, phone number or e-mail list. It will also help determine the first communication vehicle for you - a memo, an e-mail, a letter to the customer’s home, or a phone call. Depending on who will be responsible for contacting the prospects, how confidential the responses need to be and how prospects will be instructed to return questionnaires, the survey process may be conducted differently.

You can obtain the names and addresses of your target audience from several sources: customer lists, mailing lists available through list brokers, or membership lists available through associations.

Step #4: constructing the survey

Like most jobs, thorough and careful preparation of the survey will pay off in the end. Write out the kind of outcome, information and analysis that you want to obtain, and then design your survey based upon these specific needs -- always keeping the respondent in mind. Be careful not to bias a survey to show the data that you want to see, but be certain to design it so you can get the information needed.

Typically, there are several parts to survey question design:

A) Type of information sought. Questions should be designed carefully in order to gather the desired information. As in all professional correspondence, thoughtful vision, clarity of language and semantics are critical to good communication. Think through the objectives of the survey when determining what topics of information to include in the survey.

B) Actual question structure and word choice. Surveys may include closed-ended questions (e.g., "Rate from poor to excellent your view of the ease of ordering from our company"), open-ended questions (e.g., "What do you like the most about the products we offer?") or a combination of both types of questions.

When writing or editing questions, you should scrutinize each question according to the following guidelines. Continue rewriting each question as needed, without altering the original purpose of the questions. Remember to:

  • Keep each question simple and single-minded.
  • Keep each question as short as possible. If it takes too long to fill out, customers won’t return your survey.
  • Use understandable and clear language.
  • Be specific.
  • Do not talk down to the customer.
  • Avoid bias.
  • Avoid conjectural questions or situations.
  • Keep response types to a minimum.

Bear in mind that customers answer survey questions by different scale types (i.e., excellent, good, fair, poor). Most users will select only a few types for any particular survey to limit confusion on the part of customers and simplify the analysis process. Try to keep the number of scale types small, and survey design will remain simple.

C) Survey construction. Once you have determined the type of information needed, the question structure, the word choice and the scale types, it is time to combine the pieces of the survey into a complete document.

Issues that need to be addressed include determining if you want to group any demographic questions together at the beginning or end of the survey and deciding if you want to group the survey questions by topic or mix them up randomly.

If possible, you should have a small sample of customers take a draft survey to ensure that the questions are understandable, and that the survey takes a reasonable length of time to complete.

Step #5: administering the survey

You should consider the options for survey administration early in the design process. Choices include mailing a hard copy (paper) survey; running the survey over a network, your Web site or via e-mail; or proctoring the survey over the telephone. Today’s survey software products typically give you several options.

Hard copy surveys, also known as paper-and-pencil surveys, are easy to administer to people that can be brought together in one place. Where technology is either not available or is intimidating to some of the customers, a hard copy survey is a good alternative. The potential downside to a hard copy survey is the cost and time of data entry to get the survey responses ready for analysis.

Computer-administered surveys can be quick and easy, and eliminate the need for data entry for analysis of answers. In today’s world, we are accustomed to electronic communication: e-mail, the Web, an intranet or network. Many customers will find it much easier to respond to your survey on-line. Just make sure that customers are comfortable using a computer so participation isn’t lessened because of technology intimidation.

Other questions regarding survey administration include those dealing with confidentiality and anonymity. For some surveys these concerns may not be large, but for others they will be very important. You need to ensure an open and honest approach to this issue so that the integrity of the survey does not become compromised. Confidentiality and anonymity can be enhanced by explaining to customers the steps that have been taken to ensure confidentiality and anonymity.

For all target audiences, there are good times and bad times to give surveys. For example, don’t plan to administer a survey during the last week of the quarter crunch if you want to reach marketing executives. If you are doing a phone survey, when will potential customers most likely be available?

Finally, don’t forget to tell customers the date their surveys must be completed. Some organizations use a small incentive (a coupon for a product or service, even a dollar bill) to increase participation.

Step #6: compiling the data and analyzing the survey responses

It is time to gather the data and understand what the customers have told you. There are four sequential steps to this process.

1. Analyzing the answers of all customers. You usually want to see first how the total population of customers answered your survey questions. Some survey software products allow you to complete this step with a few keystrokes; others may be somewhat more complicated. If you have done a relatively lengthy survey that covers several topics, you will probably want to look at topic averages in addition to responses to the individual questions.

As you do your analysis, keep in mind the many statistical measures of dispersion and central tendency. Will the mean be enough information or will you need to know the standard deviation of the responses? Will you need the percentage breakdown of each response? Ask yourself what would be meaningful to you and your customers. Think about the level of statistical understanding of the client and the customers.

2. Analyzing subsets of customers. Once you know how the total population answered, you will want to look at how subsets of the population responded compared to the total population. How did men answer compared to women? How did people in the 45–54 age group respond compared to those in the 35–44 group? What quality issues are reported by first-time purchasers of your product? Knowing this kind of information helps you provide better products and services, identify new markets or respond to operations problems.

3. Analyzing by topic or question. Sometimes you will want to know how a total population or a subset answered a specific question: "How many times in the last decade have you bought a Ford?" You can examine the breakdown of the customers not only by frequency but also by demographic category. Good survey software should allow you to analyze your data in many ways.

4. Graph the analysis. The ability to present data in easy-to-understand graphic formats is important. Charts and graphs allow you to see and understand the data quickly. Ultimately this may help you communicate valuable information to key executives and members of the marketing and creative teams.

Step #7: preparing to communicate survey results

As we stated above, it is important how you communicate the results of the survey to your boss, department or client and, depending on your agreement, to the customers who participated in the survey. Communicating the results to customers reinforces that their input was used.

Successful communication requires a consistent and persistent plan. Most organizations use a variety of formal and informal communication methods that suit their culture, such as newsletters, e-mail, and meetings. Customers, suppliers, employees and other groups may require different forms of communication.

Step #8: creating action plans

The point at which surveys prove their value is when the information you have gathered is used in a constructive way to improve operations, products, programs, creative, offers, etc. Developing an action plan helps implement those improvements. An action plan should include:

  • An evaluation of survey data and consensus of what action needs to be taken, and with what priority.

  • A listing of activities that need to be carried out for each priority item.

  • The names of the people responsible for implementing the plan.

  • A timeline for each plan’s accomplishment.

Once you put the action plan in place, it is important to periodically re-visit the plan. Be sure that the work is getting done. At some point, you can conduct the same survey again and evaluate if the areas identified as action items are actually improving.

Planning and commitment

Successful customers surveys require planning and a commitment from all parties involved. By following these eight steps you can make the process of designing, administering and analyzing surveys a positive one. In the end, you will obtain valuable and desired information from your customers that may be used as the foundation for major change within your company. These steps will hopefully make your job less stressful and your life a little easier.