Listen to this article

Have you considered System 1 testing? 

Editor’s note: Raina Rusnak is consumer research and insights lead for Peapod Digital Labs, a company of Ahold Delhaize USA. Rusnak drives consumer research and insights services for each of the Ahold Delhaize USA retail companies: Food Lion, Giant Food, The GIANT Company, Hannaford and Stop & Shop.  

It never even crossed my mind to try a System 1 approach in any of my research.

As insights professionals, we aim to get inside the minds of consumers. What do people think? What will their purchasing behavior be if we make certain changes? How do we develop and maintain a competitive advantage over competitors to stay relevant in an ever-changing landscape of constant disruption? Businesses often asks us to pull out crystal balls to divine the future, but in the form of proper, defensible research. We naturally pause to consider appropriate research tools available to help answer the business question at hand. Occasionally, though, we find that our usual toolkit doesn’t seem to fit the business need.

Logging more than 20 years in the industry – including time on both the client side (where I currently sit) and supplier side – I’ve routinely been able to address business questions using traditional quant surveys. Once we clean out the speeders, straightliners and bots, we tend to assume positive intent from the quality respondents that remain. In most cases, consumers aim to be truthful in their responses, but we all possess biases that sometimes hide in our own blind spots. 

Implicit association test 

The journey into my first adventure with an implicit association test (IAT) began with a traditional quant survey, like most of my research does. We directly asked respondents to identify aspects of the grocery delivery experience that were most important to them. The results confirmed what we already knew: customers expect core foundational pieces of the delivery experience, including picking an accurate and complete order, delivering items in good quality condition, offering a competitive price and arriving on time. These imperative core elements simply overshadow other aspects of the delivery experience. 

We had concrete data to point to – other aspects of the delivery experience become irrelevant by comparison. But something about the results nagged at me. I presented findings to executive leadership with the numbers to back up the message, but my gut told me we still needed to dig deeper. For the first time in my career, I went back into the field almost immediately. Not because I didn’t trust the numbers – I did. Rather, I suspected that when holding the core principles of grocery delivery constant, other elements of the process could prove differentiating factors in the customer experience – interests locked below consumers’ consciously stated preferences. To get at that information, we needed to try another approach.

Determined to get the research right, our re-fielding included three different approaches: a max-diff, a benefit hierarchy and – here’s the new part for me – an implicit association test. I’d read about System 1 testing to get around inherent bias, but I’d never actually employed it.

Measuring implicit preferences and biases 

Thinking about IAT as a high-level concept, pause with me to consider a quick visual. What image comes first to mind when you read the word “flossing?” Depending, perhaps, on your generation (or the presence of kids in your household), did the word conjure mental images of dental hygiene, or a person repeatedly swinging their arms back and forth to music? Both are correct. In fact, both images may have come almost instantly to mind for many readers. Can you even say which image popped into your head first? When implemented correctly, IAT helps to measure implicit preferences and biases – what occurs in customers’ minds before they think about it for even a second. As consumers make purchasing decisions, particularly in a brick-and-mortar setting, System 1 is often at play. Cheetos may not be on your grocery list, but sometimes they may end up in your cart – an impulse-buy on your stroll down the chip’s aisle. Sure, it could just be your depleted willpower or a special treat, but it could also be a gut reaction that happens without conscious thought.

From a business perspective, when deciding whether to invest millions in offering a particular aspect of a service, it’s critical to understand if the service makes a difference to customers, whether consciously or not. Will the investment pay off or is money better spent in other ways? Here’s where IAT helped us.

IAT complements traditional studies  

We exposed respondents to three different images, controlling for the single piece to which we wanted reactions. Two were extremes on either end with the final image falling on middle ground. Each image was displayed alongside a series of positive and negative attributes, asking respondents to press a button quickly if they considered the image and the word to be associated. If they saw no connection, respondents did not press the button. The quicker the reaction, the stronger the association. The results of our study, measured in milliseconds, revealed all positive attributes for the first image, largely negative association for the other extreme, and mixed reviews for the middle image, but with a notable drop-off in positive sentiment.

We learned that once we control for the important pillars of the experience, the aspect of delivery that shows up as not all that important to consumers from a stated importance perspective is a differentiating factor. We would have missed that without the results of our IAT study, recommending that the business allocate funds in a different way.

An important caveat—implicit testing does not replace quant. Rather, IAT complements a traditional study. The potential uses and implications of IAT are wide. Consider using to test one brand against competitors, two innovative ideas against each other, brand perceptions relating to attributes, or how well your products fit within your brand portfolio – possibilities abound.

Three thoughts to chew on as you determine your future methodological approaches. 

  • Trust your gut. If your traditional quant survey shows results that don’t pass your sniff test or don’t align with your hypothesis, listen to it. Be careful not to manipulate your research to seek confirmation on your hypothesis – keep the research pure. But… 
  • Keep digging. Be courageous enough to recommend the business keep going. Try a new methodology. Of course, IAT may not be the right tool, but (assuming you have budget) try something different from your original approach to be certain you’re getting the true answer. If additional data confirms your original research findings, great, you can have more confidence in developing the go-forward strategy. If, on the other hand, the research takes you in a different direction, well, you may just find a hidden gem that will help grow the business. 
  • Be engaging with the respondent. Respondent fatigue is real. Introducing fun, interactive approaches to surveys can heighten the quality of your data and get you closer to real business answers. IAT is an engaging approach that keeps drop-off low and quality seemingly high.

While this was my first use of IAT as a methodology, having it as part of my toolkit rounds out my ability to offer clear answers and direction for exceeding customer expectations and helping to grow the business.