Hmm...let me think about that

Editor’s note: Norman B. Leferman is president of Leferman Associates Inc., a Stamford, Conn., research firm.

Steven J. Hellebusch’s By the Numbers column in the April 2006 issue of Quirk’s (“Total unaided awareness?”) provided a good primer on the subject of brand awareness measures and some of the issues of measuring it under alternative data collection modalities - e.g., telephone or in-person interviews vs. online or mail surveys. The purpose of this article is to expand upon those thoughts and to raise several other issues which should be considered when building a tracking study questionnaire.

What does data cleaning do to unaided brand awareness?

The prior column correctly explained that unaided measures are those which compel a respondent to recall a brand name on their own, without any assistance from the interviewer or questionnaire. As such, unaided questions are asked in an open-ended fashion, e.g., “What brand(s) come to mind when you think of ready-to-eat cereals?”

Hellebusch also noted that all unaided questions should necessarily be asked before one starts to probe with aided questions (such as, “Have you ever heard of a ready-to-eat cereal called Honey Nut Cheerios?”) to assess total brand awareness as the sum of the unaided and aided measures.

Here’s the problem or issue to be considered When one asks an intermediary question to assess unaided advertising awareness (e.g., “What brand(s) of ready-to-eat cereals have you seen advertised in the past three months?”), that probe, by itself, is a form of aiding that will jog someone’s memory to expand their recall of brands. That is not necessarily bad. But, when data are cleaned to give unaided brand awareness credit to a brand that had not been recalled by a respondent until probed about advertising, it really changes the definition of the data The cleaned data no longer reflect unaided brand awareness in a pure sense.

• What do multiple mentions do to tracking sensitivity?

Many years ago my colleagues and I observed that many survey participants tend to be lazy in answering unaided questions. While they might name two or three or four brands in a category, they rarely name all of the brands on an unaided basis that they are capable of remembering. For this reason, it is not uncommon for interviewer administered questionnaires to continue probing for “Any other brands?” or “What other brands of ready-to-eat cereal can you name?”

However, despite these instructions, interviewers and respondents get tired or frustrated by these challenges and rarely result in survey participants naming all the brands on an unaided basis that they can name.

In product categories that only have a few brands this is not a particular problem. However, for categories like ready-to-eat cereals or shampoos, there are not only dozens of brands, but there are also many line extension products (e.g., Cheerios, Honey Nut Cheerios, MultiGrain Cheerios, Frosted Cheerios, Berry Burst Cheerios, Team Cheerios, etc.).

Without good and complete interviewer probing, newer products can’t be found on the unaided brand awareness radar screen at all. As such, tracking studies that are built around assessing brand progress via unaided brand awareness are doomed to failure no matter how much is being spent on advertising.

Combine this with the differential probing skills of changing crews of interviewers and it is sometimes likely that unaided awareness can go down rather than up simply because the interviewers in Wave I did a better, more complete job of probing than the interviewers administering the Wave II survey.

One solution that some researchers have adopted is to report unaided brand awareness data on a share-of-mentions basis. In so doing, any differences in probing skills are seemingly washed away. But are they?

Let’s say that in results from Wave I we found that the average respondent offered the names of six brands of ready-to-eat cereals on an unaided basis, but that in Wave II the average participant only named five brands. On a theoretical basis where all people using the newest brand in the category would name it on an unaided basis, the new brand should have the same (or greater) share of mentions in Wave II as it did in Wave I.

However, if people name brands on an unaided basis with any kind of relationship to how much they have heard about brands in their lifetime, then any new brand is at a distinct disadvantage from an unaided brand awareness perspective. So, if the Wave I group of interviewers got consumers to name more brands than the Wave II team, a new brand would have had a lesser chance to get mentioned, at all. No amount of data manipulation (e.g., analysis of share-of-mentions) can help.

• The effects of advertising and other forms of exposure on measures of unaided brand awareness are cumulative.

If interviewers do a great job of eliciting multiple mentions of brands from their respondents it is likely that they will get people to remember brands which may no longer be on the market To some extent that is a direct result of the fact that the remembered brand may have been around for a long time and or supported with advertising for many years. When a new product gets introduced it must truly do something outstanding to get recognized and remembered. Even more, it must supplant an established brand from the forefront of the mind.

As such, the introduction of a new product to a very established product category must “turn the world upside down” to be mentioned in a survey, particularly if the interviewers don’t do an adequate job of getting people to mention all of the brands that they can think of.

Even though there is a forgetting/ decay curve for unaided brand awareness that means that people will forget brands that are not advertised or promoted, a long established brand may continue to foster more unaided brand awareness mentions than a newcomer to the category.

• Choose the sensitive metric and ask that question first.

If the purpose of a tracking study is to measure progress against some goals, then the researcher would be well advised to make sure that the selected metric is both sensitive and meaningful.

It has been a common practice in many tracking studies to always ask questions in a certain order based on a theoretical model that trial and usage of a brand follow brand awareness. As such, many, if not most tracking study questionnaires use the following sequence:

1. Measure unaided brand awareness

a. Collect first brand mentioned

b. Probe for other unaided mentions

2. Measure brand trial

a. Collect brand used most often

b. Probe for other brands used recently (in unaided fashion)

c. Probe for other brands ever tried (also in unaided fashion) It is only after those “important” data have been collected that surveys probe for advertising awareness and collect aided measures:

3. Measure unaided ad awareness

4. Probe for aided brand awareness

5. Probe for aided, ever trial

6. Probe for aided ad awareness

And, as Hellebusch noted, it is a common practice to edit responses backwards during data processing so that a brand name that has been volunteered in response to the unaided ad awareness is given credit for unaided brand awareness, even though it may have been omitted when brand awareness was being sought.

Perhaps we would circumvent some of the data editing issues and eliminate some of the cloudiness of tracking data if we simply changed the order of the questions to focus on the sensitivity.

 — Let’s ask unaided ad awareness first. This would focus the respondent’s attention on what has been seen or heard advertised recently and would, thereby, deemphasize recall of brand names that have not been advertised in years. For the advertiser and their agency, putting this measure first in a survey enhances their chances of seeing the needle move.

— Let’s ask about brand usage next. This would require that a brand that is currently being used in the home has more awareness presence than one that happens to be most prominently remembered because it has spent the most dollars on advertising over the life of the brand.

— Then, lastly, ask for unaided mentions of other brands. Since each of the foregoing questions has been asked in an open-ended fashion we have not violated our ability to come up with a true measure of unaided brand awareness and we have not pestered the respondent ad nauseam to keep naming brands. Moreover, we may have also ensured that each interviewer has done a better job of eliciting multiple brand mentions.

The following sequence would be better and more sensitive to tracking changes in the marketplace:

1. What, if any, brands of shampoos have you seen or heard advertised in the past x months? (PROBE) What other shampoos have you seen/heard advertised in the past x months?

2. What one brand of shampoo do you tend to buy most often?

3. What, if any, other brands of shampoos have you bought in the past x months? (PROBE) What other shampoos have you bought in the past x months? 

4. What, if any, other brands of shampoos can you name? (PROBE) What other shampoos can you name?

5. (FOR EACH UNNAMED BRAND ASK) Have you ever seen or heard of a shampoo called (BRAND)?

6. (FOR EACH UNNAMED BRAND ASK) Have you ever bought a shampoo called (BRAND)?

7. (FOR EACH UNNAMED BRAND ASK) Do you recall seeing or hearing any advertising for a shampoo called (BRAND)?

This sequence also reduces the need for data editing.

While there are likely some purists reading this article who feel that asking the ad awareness question first might bias what respondents report to be their most-often brand, we feel that the absolute bias is more than compensated for by improved tracking data. Moreover, if the bias is a function of increasing ad awareness, then advertisers should also be happy to see an increase in brand preference even if it does not directly translate into sales.

Taken together, we hope that the issues and suggestions raised here not only improve the sensitivity of future tracking studies, but also contribute to better interviewing and enhanced levels of respondent cooperation.