Editor’s note: Jon Buchwald is founder and owner of Applied Business Intelligence, Inc., a San Rafael, Calif., business intelligence consulting and software firm. He can be contacted via his Web site at http://www.biztelligence.com.

The World Wide Web is becoming an increasingly popular medium for conducting market research. It allows smaller businesses that previously lacked access to a large base of survey respondents to carry out the opinion-based studies necessary for crafting a successful marketing strategy. Unfortunately, few surveys found on the Web are likely to provide good quality data upon which a strategy can be based.

While electronic (Web and e-mail) surveys are a valuable tool for gathering data, they have their fair share of built-in caveats, and must be constructed and structured carefully. Here are a few basic rules of electronic surveying that can help researchers and entrepreneurs get the most out of their questionnaire:

  • Be respectful of your respondents’ time, and eyesight. Most respondents are on the Web for reasons other than your survey, and are likely to have a relatively low attention span. If the survey length is greater than your respondents’ expectations, then you will suffer from a high breakoff rate. Keep your survey brief, and focused on what you want to learn. Observe a friend take the survey, and obtain feedback on its length and structure.

The look and feel of your survey is important, and simple is better. Survey respondents may suffer from eyestrain if your survey is too long or contains bright colors or graphics. Eyestrain is a potential source of respondent dissatisfaction, which may lead to poor data, and an increased breakoff rate. Avoid bright backgrounds; use cool colors. Try to avoid forcing respondents to scroll all over the survey. If a survey is lengthy, then break it into a few smaller pages, rather than having one huge page.

I once saw a 100+ question e-mail survey, in which respondents wrote comments like "Stop this!" in their open-ended responses.

  • Don’t assume that your phone survey will be successful on the Web. Many researchers assume that a survey designed to be administered by phone can be typed into an HTML form verbatim, and placed onto the Web. Phone surveys are controlled, and can be lengthy, whereas Web surveys are uncontrolled, and should be brief.

Before putting a phone survey on the Web, write down what you are trying to learn, and prioritize it. It is likely that a verbally administered survey will be too long for most Web or e-mail respondents. Make plans to cut or consolidate questions, in order to give your Web survey a concise look and feel.

  • Keep in mind that electronic surveys are uncontrolled. If you are conducting a survey by e-mail or the Web, then you have no control over the amount of time your respondent takes to complete the survey, or what (s)he does between the first question and the last, nor can you rely on the respondent’s state of mind, or sobriety. If response time is critical, or you need to be reasonably confident that your respondents had minimal interruptions while taking the survey, then set up a CGI script that records the time the survey is first accessed, and the time it is submitted ("cookies" combined with hidden fields also work nicely). Throw out responses with too large a time gap.
  • Don’t rely solely upon electronic surveys; mix your survey techniques. Web based surveys are inherently biased towards a more technically oriented segment of the population, and the fact that they are uncontrolled may lead to lower quality data. When possible, obtain some of your responses (>= 20 percent) using a controlled technique, such as a phone or live interview, and look for differences between responses obtained using different methods. Also, when possible, call some of your electronic respondents, and verify and/or probe their responses.
  • Remember that browsers and modems are not created equal. If you are using graphics, Java, or a script like JavaScript or VBScript, then some respondents’ browsers will not be able to cope with them. A respondent with a slow modem may become frustrated if there are lots of graphics to load. Frustration can lead to biased responses, and increase your breakoff rate.

If you require graphics in your survey, then ask for the respondent’s browser type and connection speed, and look for response differences across browsers and modems.

  • Avoid using canned questions. Some survey creation tools come with question libraries, supposedly designed by "experts." Avoid these, except for the most basic pieces of information (e.g., gender). Using canned questions is tempting, since it requires less thinking than carefully specifying what information is most important. However doing so will decrease the quality of the data you obtain. If you are able to articulate what you want to learn, the questions to ask will become clear.
  • Don’t expect respondents to flock to you...unless you attract them first. Asking Web surfers to take a survey is not unlike interrupting a TV program to ask viewers to take a survey. Your response rate is not likely to be high unless there is some sort of incentive. Drawings for a cash prize often work nicely.

Don’t forget to make potential respondents aware of your survey. While taking out a banner ad with a search engine may get you some respondents, it is expensive. Posting notices to electronic bulletin boards and newsgroups, and sending e-mail to potential respondents can be more cost-effective. Get permission before soliciting on a newsgroup or bulletin board.

If you are soliciting responses, then be sure you know who you are asking. Posting notices to a few specialized bulletin boards or newsgroups may inadvertently bias your results.

  • If you are conducting a drawing, then be sure to conform to state and federal sweepstakes laws. Some states prohibit various types of drawings and contests. Provide a link to a complete set of contest rules from your survey site. Have a qualified attorney review the rules for appropriate disclaimers and language. If your winner is from a state that prohibits such drawings, then you may have to choose an alternate winner.
  • Be aware of what your survey communicates, and take advantage of it. Surveys not only gather data, they communicate information to potential clients. If your survey is sloppy or poorly structured it will hurt your reputation. A survey can be used to convey positive information about yourself (e.g., Please select the feature of our product that you think is the most valuable...), or negative information about competitors (e.g., Which of the following products has the poorest reputation for quality...Why?).

If you mention a competitor in a question, avoid making any undocumented direct statements about them. Use hypothetical cases, or ask questions about your respondents’ opinions of their drawbacks. Ask respondents to elaborate on responses that are positive about you or negative towards your competition. It helps them to formulate good ideas about you, and arguments against your competitors.

  • Respect your respondents’ privacy. Use blind distribution lists to mass e-mail your respondents, and never give out their names, e-mail addresses, etc. If people get "spammed" as a result of taking a survey, then it hurts everyone who uses the Internet for market research, including yourself.

The Internet can be a cost-effective way to obtain quality information about both your clients and competitors. Following these 10 basic rules will help you to make the Web a powerful resource in your market research efforts.