AI as a tool
Editor’s note: Heather Myers, the founder and CEO of Spark No. 9, is an accomplished entrepreneur and strategic leader with a background in corporate strategy, technology and media. Find Myers on LinkedIn.
AI personas are rapidly becoming a standard tool in marketing, product development and innovation. They can generate plausible customer responses to ideas, messaging and concepts quickly and at a low cost, making them an attractive alternative to traditional research. But an important question remains: how well do these simulations predict real-world behavior?
AI personas are built on patterns in human language, historical data and inferred preferences. Yet people frequently say one thing and do another. Stated preferences often diverge from actual behavior once real-world constraints appear – such as attention limits, pricing or other context shifts that influence decision-making.
This raises an important implication: Simulations of human thinking, built from human-generated data, may reproduce those same distortions at scale.
Why real-world testing still matters most
This challenge is not unique to AI. It reflects a long-standing limitation in market research – the gap between stated intent and observed behavior, often referred to as the say-do gap. Surveys, focus groups and even ethnographic approaches rely on stated preferences in controlled environments rather than direct observation of real-world choice.
So why aren’t companies testing more in the real world?
Real-world testing has often been difficult because it typically requires pilots or pre-launch builds – meaning teams must construct what they intend to test before they can measure it. As a result, many organizations rely on traditional research methods or default to internal opinions and stakeholder consensus when direct evidence is hard to generate.
Experimentation in the real world is now faster and cheaper
Real-world experimentation no longer requires the time or investment it once did. One of the most powerful mediums for testing ideas already exists: digital advertising platforms.
Platforms like Meta, Google and TikTok can be leveraged as live testing environments. In many cases, they are already where customers first encounter new ideas, products and messaging, making them a natural fit for evaluating what actually resonates.
If someone clicks, engages or converts on an ad representing a new idea, that is real-world validation of interest or relevance. If they do not, that is also a meaningful signal. In both cases, behavior – not stated intent – is what is being measured.
These platforms also allow for precise audience targeting. Teams can define segments based on demographics, interests and behavioral characteristics, creating a close representation of their intended customers and reaching them at scale.
The other major shift is creative speed. With modern AI tools, teams can now generate large volumes of ad variations – different messages, concepts, visuals and value propositions – in minutes rather than days or weeks. Each variation becomes a distinct hypothesis and test input.
Once campaigns are deployed, these ideas are no longer opinions or simulations – they become behavioral tests. Because these platforms operate at scale, meaningful patterns can emerge quickly. Within days and with minimal media spend, teams can generate statistically significant signals and compare how different ideas perform in real conditions. This turns existing marketing infrastructure into a real-time behavioral research system.
AI personas in practice
This is not to say AI personas do not have value. They are useful for generating hypotheses and exploring possible responses in a simulated environment. These simulations help teams think faster and more broadly, but they should not be treated as a substitute for behavioral evidence.
The most effective approach is not choosing between AI and real-world testing but using them in tandem. AI generates hypotheses. Real-world behavior tests them.
In the end, the question is not what sounds plausible. It is what people actually choose.