Editor's note: Thomas J. Lacki is the director of decision support systems with the National Research Services Group of Maritz Marketing Research. Based in Minneapolis, he received his Ph.D. in human information processing and statistics from the University of Minnesota.

The process of conducting survey research is one of collecting information on the attitudes, perceptions, behaviors, and characteristics of a group of individuals and inferring that others-those not interviewed- share the same attributes. This procedure involves sampling from a universe, and is the foundation of any good research study. It is therefore of critical importance that both the sampling procedure and the sample be of the highest quality.

Recall the 1936 presidential election poll performed by Liberty magazine, in which a landslide victory was erroneously predicted for Alf Landon over Franklin D. Roosevelt. The sample in this study was a set of listed telephone numbers drawn from directories, and the survey results were fatally flawed because the sample-those individuals with telephones-was not representative of the universe of voters, only some of whom had telephones at that time. The effect on Liberty magazine was severe: it soon went out of business. Employing a poor quality sample can have similar adverse, albeit less severe, consequences ranging from improper representation of the area being surveyed to increased interviewing costs.

In the evaluation of competing "brands" of telephone sample available from different suppliers, there are several features to consider when making a purchase decision. Many criteria are well-known and obvious, such as the expertise of a company and the completeness of the database. Beyond these, however, are concerns which are subtle yet equally important to a marketing research professional. Three of these will be briefly examined below.

Length of the random digit telephone number seed

In the generation of random digit dialing (RDD) samples, it is important to ask whether the last three or the last two digits of the telephone number are being randomly generated. The term "telephone block group" is frequently employed to denote the initial set of digits forming the base onto which the random number is appended, but its length is not always prominently discussed in the technical literature provided by sampling suppliers.

From an extensive analysis of our database, it has been estimated that a three digit procedure will be between 29% and 13.5% less efficient than a two-digit procedure on the average for unweighted (Type A) and weighted (Type B) samples, respectively. Indeed, the drop in efficiency is expected to be 25% or greater for 1,676 counties for a Type A sample and 410 counties for a Type B sample. A model has been prepared from which it is predicted that such a drop in efficiency will increase the cost of using each telephone number by 19 cents-which can be several times the original purchase price. It is imperative that a knowledgeable buyer be aware of this difference and its impact on the total cost of a research study.

"Hidden" costs

Efficiency and its real effect on the total price of the sample is but one of the "hidden" costs associated with purchasing a telephone sample. Second, the advertised price of a sample must be adjusted by a variety of additional costs. These can greatly influence the actual purchase price of a sample, but are rarely fully appreciated. Some of the common hidden charges include:

  • job set-up fees;
  • cell/quota set-up fees;
  • additional fees for listed samples;
  • additional fees for targeted samples;
  • coding of records (e.g. appending an MSA designation);
  • fee for delivery of sample on magnetic tape or PC diskettes;
  • addition fee for failure to meet contract/job minimum quantities;
  • fee for same-day service.

The net influence of these accessory costs can be substantial, especially when there are a number of areas within the job that are customized, such as a collection of trade areas defined by ZIP code radii. Incremental fees can, in fact, double the actual cost of each telephone number. Again, in comparing the products of various sampling system suppliers, the knowledgeable buyer will look at the actual cost per telephone number-the total of the invoice divided by the sample quantity ordered.

Suppression of businesses from residential samples

The method used to eliminate known business telephone numbers from a residential sample varies significantly from supplier to supplier. Many sampling companies simply do not attempt to perform this service and ignore the increase in non-qualified dialings it produces for the customer. Some eliminate entire exchanges that have a high likelihood of being located within business districts- unfortunately, this both removes any residential number which may be in those same exchanges and fails to capture those businesses in less commercial areas. Yet others use an algorithmic approach based on a set of "rules of thumb." For example, one such rule might classify any telephone number in which the last four digits are in consecutive ascending (e.g. 612-927-1234) or descending (e.g. 612-927-4321) order as a business.

A collection of five such rules of thumb were applied to our national database of business telephone numbers to determine the extent to which they would correctly classify business telephone numbers. The result? A poor showing at only 12.8 %. In other words, a true search of a business database is nearly eight times more effective than an algorithmic approach.

An buyer who is aware will recognize that costs increase as efficiency decreases and purchasing a sample employing a superior methodology for the elimination of known business telephone numbers will impact the quality and cost of the research.

Attempting to alleviate these problems (and avoid the fate of Liberty magazine!) Maritz Marketing Research has created its own sampling system. In brief, this service uses a database of over 90 % of all available working listed telephones in the United States, and is updated every six months. Obtained from a large and respected list compiler, it is an accumulation of information from more than 20 different sources. The capabilities include the production of the following types of telephone samples: random digit, listed, and targeted (e.g. high income).

Each of these types of samples can be drawn for areas defined by virtually any geography characteristic. For example, samples may be based in postal geography (ZlPs), census geography (MSAs, counties, tracts), commercial geography (ADI's, Nielsen markets) or telephone geography (area codes, exchanges). Alternatively, custom geography definitions, such as trade areas, may be used.

Summary

Three issues have been raised that need to be considered when evaluating the supplier from whom a telephone sample is to be purchased: the length of the RDD telephone block group seed, "hidden" costs, and the method for eliminating business telephone numbers.

It is true that all telephone numbers are equal-a phone number is a phone number is a phone number-but, some are clearly more equal than others because of increased quality and correspondingly reduced dialing costs. Telephone samples are not a commodity. Next time you consider purchasing samples, add the above criteria to your list for evaluating competing products.