Qual at scale, culture at risk
Editor’s note: Rebecca Bukele is an independent qualitative research consultant and bilingual moderator based in Miami, experience in qualitative research, multicultural insights and consumer behavior. After more than a decade in corporate research leadership roles, she transitioned to independent consulting and freelance moderation, partnering with brands, agencies and research firms across the U.S. and globally. Bukele has moderated hundreds of focus groups, in-depth interviews, ethnographies and online communities in English and Spanish, with particular expertise in U.S. Hispanic and multicultural audiences. She is a RIVA-trained Certified Qualitative Moderator and holds a master’s degree in market research and consumer behavior from IE Business School, as well as a bachelor’s degree in commerce from the University of Virginia. Find Bukele on LinkedIn.
AI is changing qualitative research quickly. We can conduct more interviews, translate conversations almost instantly, summarize hours of dialogue and identify patterns across hundreds of participants in a fraction of the time it once took.
As a qualitative researcher, I’m interested to see where these tools take our industry. I’m also cautious.
After more than 14 years conducting qualitative research, particularly among U.S. Hispanic and multicultural audiences, I keep coming back to the same concern: As we get better at scaling conversations, we may also get better at smoothing over some of the cultural nuances that make those conversations valuable in the first place.
Because in multicultural qualitative research, the words themselves are only part of the data.
For years, we have reminded clients that U.S. Hispanics are not a monolithic group. The statement has become so familiar that it almost feels like a cliché. But in research, the implications are significant.
A first-generation Colombian in Miami, a third-generation Mexican American in Texas and a Puerto Rican raised in New York can all be coded “Hispanic” while having very different relationships with language, identity, culture and the U.S. mainstream.
Even within the same person, those relationships aren't necessarily fixed. Someone may express or experience their cultural identity differently at work than at home, or when speaking with a parent vs. a friend.
That complexity doesn't always fit neatly with one of AI's greatest strengths: finding patterns.
Technology is very good at asking, in effect: What do these people have in common?
As researchers, we also need to ask what gets obscured when we group them together.
Sometimes the outlier is the insight
Imagine testing a new brand positioning with 30 Hispanic consumers. Most participants describe it using similar language, while a handful react very differently.
It would be easy to treat the dominant response as the finding and the others as outliers. But I would want to know more about those outliers.
Are they all U.S.-born? Did they grow up bilingual rather than becoming bilingual later? Does the positioning resonate with participants who feel closely connected to their country of origin but feel stereotypical to participants who navigate their Hispanic and American identities differently?
Those differences may tell us that the segmentation we brought into the research isn't capturing something important.
This is also where confirmation bias can creep in. If we enter a Hispanic study expecting certain cultural values to matter – family, tradition, community, collectivism – we can usually find evidence that they do.
AI can make finding that evidence even faster. The harder job is noticing the person who doesn't fit the story and figuring out why.
Respondent response style: Agreement isn't always agreement
Another complication is response style.
Cross-cultural research has long examined acquiescence, social desirability and other response biases. The takeaway isn't that any ethnic group responds in one predictable way. Culture, language, education, acculturation, power dynamics and the research environment can all influence how comfortable someone feels disagreeing or criticizing an idea.
It's one reason I tend to be suspicious of easy consensus in a focus group.
Suppose I'm testing a new advertising concept, and participants keep telling me they like it.
I could ask, “Why do you like it?”
But sometimes I get much more from changing the task entirely:
- “Imagine this campaign launches tomorrow and completely fails. What went wrong?”
- “You have to convince the brand not to run this. What would you tell them?”
That gives participants a different kind of permission. Someone who was reluctant to criticize the concept directly may suddenly identify three things that don't work.
This distinction becomes important as AI moderation grows. A perfectly logical follow-up to “I like the ad” might be, “You said you liked the ad. Tell me more about what you liked.”
But sometimes “I like it” is exactly the response I don't want to take at face value.
Good probing isn't always about getting someone to elaborate on an answer. Sometimes it's about finding another way into it.
Code-switching isn't noise. It's data
Language may be where I find this most interesting.
AI has made translation dramatically easier. A discussion guide can move from English to Spanish almost instantly, and Spanish-language transcripts can be made accessible to an English-speaking research team just as quickly.
That's incredibly useful. But knowing what someone said is not always the same as understanding why they chose to say it that way.
Consider a bilingual participant evaluating a healthcare message. She has been speaking primarily in English, the language she uses at work and throughout much of her day. Then she starts describing how her mother would react to the message and, without being prompted, shifts into Spanish.
That catches my attention.
Or imagine someone discussing a new food product in Spanish. When asked to describe the product, he uses the expected Spanish vocabulary. But when I ask how he would tell his friends about it, his response becomes a fluid mix of Spanish and English.
I don't necessarily see that as poor Spanish or a translation problem. It may be a much more authentic representation of how that consumer actually communicates.
And it gives me somewhere to go.
“You switched into Spanish when you started talking about your mom. Tell me about that.”
Or: “I noticed you described the product differently when you imagined talking to your friends. Which version feels more like you?”
Now we're learning something beyond whether the participant understood the words. We're getting into audience, identity and context.
I've also seen language choices reveal problems with the research materials themselves. A brand may insist on testing a beautifully translated Spanish term, while participants repeatedly abandon it for the English term they actually use in everyday life.
Technically, the translation may be perfectly correct. But if consumers don't naturally speak that way, that's the more useful finding.
Research on bilingual language use reinforces why these shifts deserve attention. Code-switching can be related to topic, social context, group membership, identity and emotion. It is a natural feature of bilingual communication, not simply linguistic interference.
The switch itself can tell us something.
The transcript is not the interview
There is another challenge with scaling qualitative research that concerns me: The temptation to treat the transcript as a complete representation of what happened.
Imagine we're testing messaging around financial security.
A participant spends several minutes confidently explaining that she makes financial decisions independently and doesn't need anyone else's approval.
Later, I ask her to imagine actually moving $20,000 of her savings into the product we're discussing.
She pauses.
Her posture changes. She laughs a little.
Then she says:
“I'd probably talk to my dad first.”
On paper, I have two contradictory statements.
Sitting across from her, I have something much more interesting.
I can go back to what she told me earlier:
“You said before that you make these decisions on your own. But when the money became real, your dad entered the picture. Help me understand that.”
That conversation could uncover something far more useful than whether she likes the financial product. Maybe there is a difference between how she thinks about her independence and how she actually makes high-stakes decisions. Maybe family authority changes when financial risk increases. Maybe “independence” itself means something different than the research team assumed.
These are the moments qualitative researchers are talking about when we say we are reading the room.
We're listening to the words, but we're also paying attention to hesitation, facial expressions, posture, nervous laughter, energy and tone.
Sometimes the most interesting part of an answer is the five seconds before someone gives it.
AI can introduce bias, too
AI is sometimes positioned as a way to reduce interviewer bias, and there are real advantages to consistency. But removing a human interviewer doesn't remove assumptions from the research.
Researchers already know to watch for confirmation bias, leading questions, anchoring and premature interpretation. With AI, some of those risks may simply show up in different places: the way the system has been trained, the patterns it privileges, the follow-ups it selects or the assumptions built into the research design before the first participant ever answers a question.
If a participant gives an unexpected answer, an AI interviewer might generate the statistically or semantically logical follow-up.
As a moderator, my reaction may be less sophisticated: Wait. That's weird. Why did she answer it that way?
I don't want to lose that reaction.
If everyone is agreeing with the premise of my discussion guide, I may deliberately attack the premise. If someone contradicts something they said 20 minutes earlier, I want to connect those two moments. And if halfway through the interviews I realize that my carefully constructed line of questioning is based on the wrong assumption, I need the freedom to abandon it.
There is structure to good qualitative interviewing, of course. But there is also improvisation, curiosity and a willingness to follow something you weren't expecting to find.
Cultural expertise belongs upstream
This is ultimately why I think the rise of AI makes culturally fluent researchers more important, not less.
The value isn't simply that a human moderator can come up with a better-sounding probe than an LLM.
Moderation was never the whole job.
Cultural expertise should influence who gets recruited, which markets are represented, how groups are segmented, what methodology is appropriate, which language participants are allowed to use and whether the concepts we're testing make cultural sense in the first place.
Earlier in my career, I worked on multicultural studies where seemingly small methodological decisions changed the research entirely.
In one study, a planned written exercise had to be redesigned after the moderator pointed out that members of the community could speak their heritage language but did not necessarily read it. In another, research was conducted in a familiar community space rather than a traditional facility because accessibility and trust mattered more than having the perfect backroom. Neither issue would have been solved by better probing once the interviews began. The important decision happened earlier.
The same principle applies to Hispanic research.
If the sample has already been defined, the methodology selected and the English guide finalized before the Spanish-language or culturally fluent researcher enters the conversation, we've brought that expertise in too late.
The question isn't whether AI can ask questions
AI can ask questions. And as the technology improves, it will get better at probing, interpreting speech and recognizing behavioral cues. I'm genuinely interested to see what that makes possible for qualitative research.
I don't think the useful debate is human vs. AI.
The more interesting question is what kind of understanding we're willing to call qualitative insight.
If the objective is to ask hundreds of people questions efficiently and identify recurring themes, AI gives qualitative researchers reach we simply haven't had before. There is enormous potential in that.
But some of the best insights I've encountered in research have come from the moment when the pattern breaks.
The participant who answers differently from everyone else.
The perfectly translated Spanish phrase nobody actually uses.
The person who switches from English to Spanish when the conversation turns to her mother.
The enthusiastic reaction that changes when a hypothetical decision suddenly involves real money.
The pause that makes you decide not to ask the next question in your guide.
None of those moments are particularly dramatic on their own. They are easy to miss, especially when the goal is to process more conversations, more quickly. But someone has to notice that something changed and be curious enough to find out why.
That's where I see the qualitative researcher's role becoming even more interesting. As machines get better at capturing, organizing and scaling what people say, we can spend more of our energy figuring out what deserves a second look.
AI may allow us to have more conversations than ever before.
Our job is to make sure scale doesn't make us less curious about the conversation that doesn't fit.