Determining what works and why

Editor's note: Frank Buckler is founder and CEO of Success Drivers, a Santa Barbara, Calif., research company.

Countless studies show that the money spent for advertising itself only explains 25 percent of market impact (ARF 2016). The remaining 75 percent of variations are due to the creative content of the advertisement or campaign. There are numerous ways to address this question through research. All of them, however, have significant limitations. Because of these limitations, the majority of ads perform below expectations. Even when an agency produces a top-performing ad, in most cases it can’t easily replicate this success.

Most brands test creative ideas in pre- or post- copy test surveys. Quantitatively, they measure how well creative content performs. The surveys ask respondents for explicit qualitative and quantitative judgments. Typical questions include “Why did you like the spot?” or “What made you remember the brand?” These kinds of qualitative questions are supposed to measure why a commercial or other marketing vehicle does not perform well. Researchers compare successful advertisements with unsuccessful spots and try to identify properties that are obviously different. In practice, though, there are always multiple properties that overlap and cause variations. Objectively, there is no way of reliably pinning down the reasons for success with descriptive or correlation-based data analysis. It is a good exercise for generating hypotheses but it does not provide reliable knowledge.

Even brain-scanning (fMRI or EEG), facial recognition, eye tracking or implicit association testing do not solve the problem. Those methods more reliably measure the responses to advertising. What still is unsolved are these two questions:

1. What is the causal impact of emotional and attitudinal responses to outcomes?
2. Which of dozens of creative tactics and techniques within an ad has which impact on responses in which conditions?

Question one needs to be answered in order to make recommendations more reliable. Answering question two will make recommendations much more tangible and enable us to avoid ambiguous recommendations – which would be expected to better translate into market success.

To answer these and other important ad-effectiveness questions, we need to measure the impact that emotions have, the influence that creative techniques have and we need to measure their effect in a controlled, causal manner to be sure that the insights generated will be useful. An approach we’ve developed consists of three components: copy testing, ad profiling and AI-powered key driver analysis.

Copy test. In online surveys, we tested hundreds of commercials in different product categories. We measured the instinctive emotional response to an ad; we measured with implicit measurements how the ad is perceived and how attitudes toward the brand changed. Further, we assessed the impact on awareness, learning and purchase consideration. We designed the questionnaire to not just be applicable for TV commercials but any other advertising format and any other product category.

Ad profiling. Along with the copy test, which measures a consumer’s response, is it important to understand what exactly influences those reactions. This is why we conduct a content analysis. In this, experts quantify all creative elements of an ad using a codebook of nearly 200 codes. First, we categorize which overarching emotional schemes are used, e.g., Disney equals “family love is most important,” Nike equals “tenacity leads to success,” banks equal “we are your friend,” etc. Then we code which creative vehicles (such as celebrities, music or spokespersons) are leveraged. If we can understand which of those very specific creative components work, creative teams will be more successful in translating recommendations into effective ads.

Key driver analysis based on AI. The impact of creative components and tools cannot directly be read in data. Simple correlations and comparing figures can produce spurious findings. What it takes is a causal key driver analysis. For that, we leverage a self-learning, AI-based methodology to better explain why customers buy.

Particularly, we apply the universal structure modeling (USM) methodology, a technique that has been widely scientifically published (see the list of references at the end of the article). The method builds on the idea of PLS-path modeling but instead of conventional linear methods, it leverages machine learning in the form of a Bayesian neural network. With this, it can be flexible in how it sees variables relating to each other instead of assuming linearity, independence of drivers and perfect distribution properties. USM also comes with simulation techniques that eliminate the black-box property of neural networks. (If an implementation of USM is not available, one could still use PLS-path modeling or Bayesian networks from standard software packages to receive useful results.)

Aspirations were high

Our aspirations were high when we started researching the creative drivers of advertising in six different product categories: food, spirits, home décor and accessories, OTC medications, insurance and banking. We performed online copy tests of over 500 commercials. The ad profiling revealed first that in a category only about 30 to 40 out of 130 emotional triggers are commonly used and it was the same with creative vehicles – every category has two or three techniques that are used in over 80 percent of spots.

Reviewing the performance KPIs of the copy tests, we found a similar takeaway in every product category: An ad should be liked (the implicit associations with the ad are positive), linked to the brand (otherwise customers will not know what to buy) and it is beneficial if viewers state that they learned something from the ad. The most interesting finding was that positive emotions are mandatory to make consumers learn a rational message.

In addition, we found that:

Positive emotions are the key for driving impact. Positive emotions are not just productive but mandatory to make consumers process your ad. Just as important, three types of negative emotions – anger, contempt and disgust – must be avoided at all times.

Overall, the emotional connection turned out to be an essential element of the advertising instead of just a desired final outcome. Emotions are a response to emotionally-relevant content. Producing content that has emotional value to the consumers makes them listen to it and remember it.

The big question still is what kind of emotion drives sales. By measuring the impact of Paul Ekman’s seven emotions on purchase intent, we found a pattern that is valid for all product categories: A commercial needs to leave its audience happy. Surprise is an emotion that is effective in getting to happiness but potentially negative emotions need to be handled with care.

Many award-winning commercials emotionally engage an audience but they do so by leaving the viewer sad (e.g., you feel sorry for someone in the story) or frightened – neither of which will effectively drive sales. Also, anything that creates feelings of disgust or contempt will kill an ad’s performance. And it’s not the type of obviously distasteful situations one might think of; even scenes where actors grab wet, used tissues or where they consume food which the audience doesn’t like can make viewers respond with disgust and can turn them off of ever buying the advertised product.

It pays to go against the current. The emotional triggers (the overarching emotional messages of a spot) that are most-used in many product categories typically don’t work! The triggers that do work are believable and have a simple and direct link to what the advertised brand can do for the customer emotionally.

Interestingly, what is emotionally relevant to customers is not always what you would expect and we found in some categories that the two or three most commonly used emotional triggers are ineffective. For example, banks mostly advertise that they are a trustworthy “friend.” Pharmaceuticals promise relief. And liquor brands promise they’ll help you have a great time in the company of others. All of those emotional values make perfect sense but unfortunately are measurably not effective yet they are used in most ads.

Instead, we found for OTC pharmaceuticals that brands need to step out of the comfort zone to drive impact. Ads that use the “loser” trigger – where someone in the spot has exaggerated misfortune or clumsiness – are consistently outperforming the industry. The technique triggers the physiologically measurable type of happiness called schadenfreude, which makes the viewer feel superior, sending them the message “Don’t be one of those losers who doesn’t take the right medication.” Well-applied, it is a goldmine.

For banks, it might be true that they need to gain trust but this is not achieved by claiming trustworthiness. A more effective emotional trigger here is to use “family love,” to show that the services are serving the most important thing customers care for – their loved ones.

Also, with spirits brands, most people would agree that the spots that claim the brands deliver “a great time in company with others” work well. But evidence shows something else. You can have fun with friends with any kind of drink. Instead, consumers choose brands simply because they are expecting some indulgence.

Old-school works. Creative agencies are driven by the search for the new and the cutting-edge. But our studies have found that many fashionable new tactics can hinder ad effectiveness while many old-school approaches are true performance boosters. Further, we have found that simpler is often better, especially with creative elements in ads. Commercials typically use music to engage the viewer emotionally yet many use meaningless, unrelated soundtracks that fail to make a difference. Even worse is when brands try to educate and convey a message using ineffective vehicles such as metaphoric storytelling, text and voiceover techniques. And while many brands leverage celebrities, we have found that it often harms the performance of ads. Celebrities need to be closely associated with the brand otherwise they can distract attention away from the brand and fail to support brand-building and brand-recall.

A key question in advertising is: How can we ensure that the audience gets our message? Will they see that we are cheaper, work better, will make them slimmer, etc.? Again, the evidence in this study is a call for simplicity. If you want to convey a message use the simplest and most direct way to bring a message across: put a spokesperson on-screen who looks into the camera and tells the viewer very briefly what needs to be told.

Music is known to drive emotional engagement. But what we found is that a simple technique will let a commercial outperform its peers: Use a well-known song that has lyrics which correspond to the commercial’s message. For example if your spot is about enjoying a nice day in spring, use the hit “I Can See Clearly Now” for its references to “bright, sunshiny days.”

Everyone can use this approach

The relevance of the new approach is obvious. Suddenly, we are able to set stricter guidelines for creatives in their work. Comparable results can be achieved by anyone without our help. Many research companies offer emotion-based copy tests. With some effort every brand can develop its own coding framework with 20 percent effort that covers 80 percent of content. Finally, the AI-based key driver analysis that we recommend is scientifically published and available to the public as well. Even the use of conventional key driver analysis would be better than using descriptive- or correlation-based approaches.

The clients that funded this multi-client-multi-category study now use results in multiple ways:

• Copy test: They use the applied copy test for future assessments. The driver model built in this study is applied to this new data to arrive at more specific recommendations.
• Deep-dive modeling: Some clients now extend the study with in-market-success data. In this way you can model the algorithmic relationship between creative tactics and short- and long-term sales figures.
• Alignment workshop: All this knowledge must be translated into creative briefings that will end up in more powerful ads. With the studies’ findings users can enable creative strategy workshops and enrich the discussion with spot examples out of a database of commercials that have been analyzed.

Have we been successful in identifying the DNA of successful advertising in our multi-category study? No, we did not find that Holy Grail. But at least we raised the bar and achieved insights with greater clarity and market impact.

Selected papers on USM

Buckler, F. and Hennig-Thurau, T. (2008), “Identifying hidden structures in marketing’s structural models through universal structure modeling: an explorative Bayesian neural network complement to LISREL and PLS.” Marketing Journal of Research and Management, Vol. 4, Issue 2, pp. 47-66.

Turkyilmaz et al. A. (2013), “Universal structure modeling approach to customer satisfaction index.” Industrial Management & Data Systems, Vol. 113, Issue 7, pp. 932-949.

Garbe, J.N. and Richter, N.F. (2009), “Causal analysis of the internationalization and performance relationship based on neural networks – advocating the transnational structure.” Journal of International Management, Vol. 15, Issue 4, pp. 413-431.

Oztekin et al. A. (2011), “Development of a structural equation modeling-based decision tree methodology for the analysis of lung transplantations.” Decision Support Systems, Vol. 51, Issue 1, pp. 155-166.

Turkyilmaz, A., Temizer, L., Oztekin, A. (2016). “A causal analytic approach to student satisfaction index modeling.” Annals of Operations Research. doi:10.1007/s10479-016-2245-x.

Rigdon, E.E., Ringle, C.M., Sarstedt, M. (2010). “Structural modeling of heterogeneous data with partial least squares,” in Naresh K. Malhotra (ed.) Review of Marketing Research, Volume 7, Emerald Group Publishing Limited, pp. 255-296.

C.N. McIntosh, J.R. Edwards, and J. Antonakis. (2014). “Reflections on partial least squares path modeling.” Organizational Research Methods, 17, 210-251.

Henseler, J., Hubona, G.S., & Ray, P.A. (2016). “Using PLS path modeling in new technology research: Updated guidelines.” Industrial Management & Data Systems, 116 (1), 1-19.