Editor’s note: Joel Zeiler is executive vice president at Monroe Mendelsohn Research, Inc., New York.
Frequently characterized as a poor alternative to telephone, personal, and even online interviews, mail surveys are given little thought by most survey researchers. Mail surveys have been criticized for their low completion rate, lack of flexibility and self-selection bias. In fact, the main and only benefit we as researchers associate with mail surveys is usually their low price - viewing mail surveys as the methodology to consider only when funding is unavailable for personal, telephone or online interviewing.
Are there drawbacks to mail surveys? Sure!
One major one is that they are not conducive to fast turnaround. While questionnaires can be mailed out quickly, high completion rates are best achieved by allowing respondents to choose a convenient time to fill out the survey. This process is generally expedited by sending out follow-up postcard reminders. But mail surveys usually require more in-field time to complete than online, telephone or personal surveys.
Another shortcoming is the inability to ask very specific or aided questions because they may affect the results of earlier questions. Some respondents will read through the entire questionnaire before filling it out, while others will go back and add or change previous answers based on questions found later in the survey.
Lack of depth in open-ended questions is another mail survey drawback, but no more so than with an online survey. Because an interviewer cannot offer follow-up probes, mail survey open-ends frequently have very short and general responses.
Many advantages
Do these drawbacks justify the negative image of mail surveys? Absolutely not!
While mail surveys are, in fact, usually less expensive than other interviewing methods, one should reconsider and not overlook the many other advantages mail surveys offer.
Possibly the most important benefit of mail surveys is the ability to reach groups that are not easily accessible by personal, telephone and online interviews.
Some of these groups include business professionals or affluent adults, who are more likely to have unpublished telephone numbers, to receive a profusion of spam and other junk e-mails, and less likely to be home when called or to spend time completing a long interview when reached.
Also included in these hard-to-reach groups are physicians, who are not likely to take time from busy office hours to complete a telephone interview. Others may not have Internet access or may choose to use their non-working computer hours more productively.
However, everyone gets their mail, and if properly approached and compensated, these hard-to-reach people may be more likely to sit down and spend time to complete a survey at their convenience.
A comprehensive mail study we’ve conducted every year for the past 25 years has averaged a completion rate of about 55 percent - a level higher than that achieved by most telephone and personal interviews, and far superior to the 5 percent to 10 percent levels common to online surveys. Our 2004 completion rates for mail surveys with physicians average above 50 percent. And completion rates in the 70 percent to 80 percent range are achievable among less difficult-to-reach target groups.
Mail surveys also provide respondents with the opportunity to look up information or conduct “pantry checks” or audits of prior bills, expenditures, income levels and many other items.
The promise of anonymity is another benefit. While personal and telephone interviewers may indicate that all information collected will be kept confidential, blind mail surveys absolutely guarantee it. This is particularly important to someone who may be asked to provide information of a confidential or personal nature.
And, despite their drawbacks, mail surveys have withstood scrutiny in terms of their accuracy and reliability, and the process compares favorably to the sampling flaws of some newer, more technically advanced methodologies. To this end, mail surveys, when conducted properly, have passed the acid test as evidence in legal proceedings.
Right alternative
It is hoped that this article will help shed some light on why mail surveys should not be a researcher’s choice of last resort, but rather the right alternative to personal and telephone interviewing. The benefits of mail surveys should be weighed along with those of other interviewing modes in terms of the research issue, the target group and other project parameters.