A measured response

Editor's note: Jerry Thomas is president of Decision Analyst, Inc., Arlington, Texas.

The promise of advertising is great. It's an opportunity for a brand to tell its story directly to the ultimate consumer, to build awareness and project a powerful brand image, to create and build brand equity, to bypass the trade and circumvent competitors. In actual practice, however, the promise of advertising is seldom realized. In fact, the opposite is true. Media advertising is probably the most inefficient, least productive expenditure in a typical company's marketing budget. Why is advertising's potential not realized?

First, few companies do basic strategy research to develop a creative blueprint to guide the development of their advertising. Second, few companies pretest their advertising creative to make sure it has a chance to work. Third, even fewer companies track their advertising once it's "on air" to measure the effects of the advertising over time. Advertising tends to be created in an informational vacuum and is rarely evaluated in any consistent, systematic way thereafter. In effect, there is no reliable feedback loop, so the advertising muddles along from year to year, never getting any better.

No wonder that many companies have grown weary of traditional advertising and have shifted media dollars into sales promotion and direct-response marketing activities - where effects tend to be immediate, easy to see, and easy to measure. The strategic potential of advertising is just as great as ever, perhaps even greater, since so few companies seem to understand how to create and deploy consumer advertising that really works.

Advertising success

To successfully utilize advertising in the marketing mix, three types of research are essential:

  • Strategy research. How advertising works differs from product category to product category, and from brand to brand within a category. This means that each brand must develop an understanding of its consumers and their motivations to serve as a template for creative development.
  • Advertising pretesting. Once the advertising creative is developed (either rough or finished), it's really important to pretest the advertising. Pretesting helps identify outstanding commercials and flags under-performing commercials. More importantly, pretesting provides guidance to the improvement of the commercial, and to the improvement of all future commercials. However, pretesting is not perfect nor foolproof. Pretesting cannot perfectly predict on-air success.
  • Advertising tracking. Once commercials are aired, the only way to know if the advertising is working is tracking research. It's the ultimate acid test of advertising effectiveness. As it's used here, the term tracking research refers to telephone interviews among a representative sample of target-audience consumers. These interviews can be continuous (i.e., a certain number of interviews are conducted every day or every week throughout the year) or pulsed (i.e., the interviewing is conducted in waves at discrete points in time, say every three months or every six months).

The tracking questionnaire

A well-designed advertising tracking questionnaire should include the following essential measurements:

  • Unaided and aided brand awareness. The creation and maintenance of brand awareness is one of the most fundamental (and most valuable) goals of advertising. Advertising can be effective if it does nothing more than create brand awareness.
  • Unaided and aided advertising awareness. These tend to be diagnostic measures. These measures help indicate if the changes in brand awareness or market share are related to the advertising itself. For example, if brand awareness is trending up over time, and advertising awareness is trending up, then we can reasonably assume that the advertising is having positive effects. But, if brand awareness is rising while advertising awareness is declining, then one would have reason to suspect that the advertising might not be responsible for the improving brand awareness.
  • Advertising message recall. What messages and ideas from the advertising do consumers remember? Do the remembered messages correspond to the advertising messages that the advertising was intended to communicate? Advertising message recall is measured by an open-ended question, to which respondents give unaided, spontaneous answers. This question helps determine if the intended messages are getting through to consumers. Advertising message recall also provides an indication of consumer memory distortion and learning effects over time. That is, once a commercial starts running, consumers do not remember everything in it equally. Some elements stick in the memories of consumers, and other elements fade away. Knowing the elements that have the highest memory value is of great benefit in improving future creative executions.
  • Brand image. Advertising can shape and magnify a brand image over time. This is one of the most important strategic benefits of advertising. However, if you include brand image rating questions in the tracking questionnaire, don't expect to see any meaningful changes in a brand's image in the short run. Typically, it takes a minimum of one to two years of consistent advertising to cause a measurable change in brand image. The full effects of brand-image shifts play out over 10- to 50-year time intervals (sufficient time to let the stubborn and the rigid of mind pass away).
  • Brand trial and usage. Trial of the subject brand and major competitive brands is a useful measure to track. Usage of the subject brand and the major competitive brands (i.e., how often, what size package, etc.) allows market share estimates to be calculated and tracked over time. Moreover, both trial and usage are valuable analytic variables. The survey results can be cross-tabulated by triers versus non-triers, users versus non-users, and light users versus heavy users. Including a measure of volumetric usage of the target brand is always important, because one of the possible effects of the advertising is an increase in frequency of brand purchase (i.e., an increase in the volume or amount of the brand consumed).
  • Demographics. Key demographics such as geography, age, sex, education and income should always be included. These variables are extremely valuable in analyzing tracking survey results and in defining the optimal target market for a brand.

    The following measurements might also be considered for inclusion in an advertising tracking questionnaire. Typically, these are not must-have questions, but at times one of these optional questions can be very useful.
  • Aided advertising message recall. Sometimes researchers will include a list of all major copy points, and ask consumers who are aware of the advertising whether the advertising communicated each point. This can be an effective way to measure messages conveyed by the advertising. One must be careful in how this question is posed, and how it is interpreted, because consumers have a tendency to claim that they recall all aided messages - even messages not actually in the advertising itself.
  • Aided commercial recall. Typically, a campaign consists of several commercials. By reading a brief description of each commercial to respondents, the level of recall for each commercial can be determined. This is not an exact or perfect measure, but it can provide a first approximation of the impact of each commercial. Once it has been confirmed that respondents have seen a specific commercial, it is then possible to ask follow-up questions, such as a) the number of times each commercial was viewed, b) whether respondents remembered the name of the brand advertised in each commercial, and c) some type of simple rating of each commercial. Again, these are not perfect measures, but each can tell us a little about the commercial's on-air performance.
  • Promotion awareness and usage. If promotion plays a significant role in the marketing plan, then it could be useful to track awareness of a brand's various promotions and consumer participation in those promotions. These questions can be unaided and/or aided, and questions about competitive promotions can be included as well.
  • Market segment characteristics. These are typically questions to identify important market segments, to refine one's ability to analyze the tracking survey data. For example, questions about price sensitivity, cents-off coupon usage, preference for shopping at certain types of retail outlets, propensity to participate in promotions, etc. can be useful cross-tabulation variables. These questions might reveal that the advertising is doing particularly well among certain groups of consumers, but is not reaching other segments of the consumer market.
  • Media habits. These questions can be simple and few, or complicated and many. Generally, it is best to limit media questions to a few important measures, such as the amount of time the respondent spends "consuming" various types of media, or types of programming watched most often. The media questions can be analytic variables and can help refine media strategies. Since so many good syndicated sources of media data are available, it usual doesn't "pay" to add a lot of media questions to an advertising tracking study.
  • Lifestyle/psychographics. These types of attributes or statements can allow us to analyze tracking data by lifestyle or psychological segments. Typically, lifestyle and/or psychographic measures are of limited value in an advertising tracking study. First, the correlation between lifestyle/psychographic market segments and marketing-relevant consumer behavior tends to be low (i.e., these measures don't work very well). Second, time limitations on questionnaire length tend to preclude the inclusion of sufficient lifestyle/psychographic measurements to provide statistically reliable results.

Sample definition

Once you have decided what questions to ask in your tracking study, two critical decisions remain to be resolved: sample definition and continuous versus pulsed interviewing.

It is wise to define your sample broadly, to make it as inclusive as possible. For example, even if your target market were defined as consumers 25 to 34 years old, it would still make sense to track all consumers 18 to 64 (or older) in age. The broader definition of the sample is a safety net, because the demographics of a market can change over time. If you define your sample too narrowly, you run the risk of the sample becoming obsolete. Likewise, define the product category you are tracking as broadly as possible. Also, always set quotas for gender, so that you do not under-represent men.

Continuous interviewing offers a number of advantages over pulsed. Continuous provides a complete record of consumer measurements over time - with no gaps or missing time periods in the data. The quality of interviewing tends to be higher with continuous surveys, since the same interviewers work on the project day after day. Continuous tracking smoothes out the effects of short-term disturbances such as adverse publicity, new product introductions, bad weather, etc., whereas pulsed tracking can be biased strongly if some negative event occurs just as a wave of interviewing is conducted. Continuous tracking is a better monitor of competitive information, since the interviewing is ongoing and not biased to the media schedule of one brand (as tends to happen in pulsed interviewing).

Continuous tracking data can be analyzed in relation to other continuous data (sales, advertising expenditures, market share, etc.), normative standards can be set, and predictive mathematical models can be derived. That is, it is possible over time to develop a model for a specific brand that explains the relationship among media expenditures, tracking variables and market share (given sufficient time and data). Such an understanding is the Golden Fleece that marketing executives seek.
Pulsed tracking is not without some advantages. Pulsed tracking is less expensive than continuous tracking. Pulsed interviewing can be concentrated into a short time interval to provide highly precise before/after measurements for specific flights of advertising, and the waves of interviewing can be precisely timed to coincide with media schedules.

Regardless of whether you choose continuous or pulsed interviewing, several guidelines should be followed to ensure that the tracking data is comparable from time period to time period:

  • Maintain constant methods. The questionnaire, the sample definition, the training of interviewers, the editing, coding and tabulation procedures must all remain constant from time period to time period. Any change in methods is very likely to cause perturbations in the survey results and destroy the comparability of data between different time periods.
  • Stay with one research company. Once you have found a research company with a) financial stability and b) good quality-control systems to do your advertising tracking, stay with that company. If you change research companies every year or two, the tracking data will not be comparable across time. Small differences in methods (i.e., interviewing training, callback policies, editing and coding conventions, etc.) from research company to research company will almost always destroy data comparability.
  • Stick with advertising tracking. The tracking research will grow in value from year to year, as you learn more and more about the long-term effects of your advertising. You must stick with your tracking measurements year after year to fully realize the maximum strategy insights into your brand, your market and your advertising.

If you consistently pursue the three keys to successful advertising (strategy research, pretesting and advertising tracking), you will gradually increase the yield from your advertising investments year after year - and leave your competitors groping in the darkness.