Beyond the Hype: Unlocking Conversations at Scale with AI Moderator
Editor's note: This article is an automated speech-to-text transcription, edited lightly for clarity.
Qualitative research can be expensive and time consuming. Voxpopme set out to solve this problem though AI moderation.
This session from the June 11, 2025, Quirk’s Virtual Sessions – DIY Research series, introduces Voxpopme’s AI-moderation platform and goes in depth into the creation.
Betsy Shaak, head of product at Voxpopme not only illustrated the simplicity of the process but why the organization created the platform and what thoughts went into it.
Session transcript
Joe Rydholm
Hi everybody and welcome to our session, “Beyond the Hype: Unlocking Conversations at Scale with AI Moderator.”
I’m Quirk’s Editor, Joe Rydholm. Before we get started, let’s quickly go over the ways you can participate in today’s discussion. You can use the chat tab to interact with other attendees. You can use the Q&A tab to submit questions for the presenters during the session, and they will answer as many questions as we have time for during the Q&A portion.
Our session today is presented by Voxpopme. Enjoy the presentation!
Betsy Shaak
Hello. Thank you so much for joining today's webinar. We are going to jump right into things, no time to waste.
Thank you guys for joining Voxpopme’s webinar. Today we're talking a little bit about some hype moments. We're talking about a long conversations at scale with AI moderation, and I really want to go into a little bit more depth than just a marketing or sales pitch.
So, without further ado, let's get into it.
So, great to meet you. My name is Betsy. I am the head of product at Voxpopme. I've also been very close to the development of our AI moderation product for the last two years. So, I promise in the next 15 minutes you'll see why this matters as a methodology, what we've developed for clients who are using Voxpopme and learn a little bit more about our approach to building this technology because that part is super important.
A little bit about what we've actually built.
We are tackling AI moderation from a market research focused box. Voxpopme has been in this space for about 12 or13 years. We really know market research. We know how to do insights and we're really excited to apply those learnings to the AI moderation space. The goal here, real people, real interviews that you can scale utilizing AI.
What I want to mention here is it's scaled with AI, not replaced by AI. I still don't think we're in a spot where we can say, ‘one-for-one, you should go and just do AI moderation and no human should review it, or no human should do an IDI side-by-side.’ But I think that there's a lot of exciting things about this technology.
We're kind of approaching it with a market research background, but a responsible market research background as well because we kind of know the truth that sometimes you just can't afford the time, money, people power, etc. that it takes to do IDI. Sometimes you would love to talk to 15 or 50 people, but your budget is only $500. The established way of doing in-depth interviews, it's just, it's fantastic, but it's also not for everybody. This can be slow schedule, be hard to scale, be very resource intensive. Can you find the moderator for this niche topic? It needs a big budget. Sometimes it requires people to come in person to a place which is also really hard and you're taking them out of their environment as well with that.
The truth is that not every team has the headcount budget for the amount of insight that they need to make the important decisions. So, many fallback on something like quant that lacks the depth because they can't do qual.
What we're really trying to do with this AI moderator product that we've launched is help you skip some of the busy work so that you can really focus on the things that you really need to focus on. We want to cut out some of the calendar juggling. If you've ever run a study over multiple time zones, it is a real pain in the butt.
Manual moderation can be really valuable. I do still think that there's a place for the human in the loop here, but sometimes you don't have either the moderators with the knowledge, the time or the budget for that. So, you start to lose that depth completely.
Then lastly, obviously the staring at raw footage element of this. I remember a time not very long ago when you would do IDI and you took a pen, and you wrote down your notes. Then you prayed that you remembered enough at the end to know what your notes meant. This has really changed the analysis side.
So, how did we build this in a way that's one responsible but also built into the way that you work today?
The first element that we've done when we approached AI moderation at Voxpopme was video first research.
We really believe in the value of seeing and hearing from real people, not only for quality concerns, but also the depth that you get out of this. We didn't really want to focus on something that was text heavy. So, we focused on a video first approach.
The next is we were looking at some of these other ways that AI agents and voice agents are interacting. We saw that they weren't natural, responsive conversations.
So, the way that we have approached building AI moderation at Voxpopme, is really about comfort and flow. We want to make it feel like a natural conversation. We want at any time in the conversation you could ask a question, ‘how long is this going to take’ or ‘can you repeat that last thing you just said’ just like you would with a person.
Next thing is we wanted to make sure that it was made with researchers in mind, but also made with research.
Voxpopme has over 10 years of call experience. We also, ADA tested this with many research teams, and we learned a lot along the way. We have our background in qual, of course, and moderation as a company, but sometimes it's almost best to build these things with the customers, and that's exactly what we did here.
Then next, the scalable analysis pieces of this are super, super important. It is so important that those are in there so that you could skip some of the ‘slog,’ as that's written here, but the manual work and just the brain toll that takes that part of it. I want to be able to let researchers dig deep where they need to and pull back when they need to.
That pull and push of this analysis experience is really important because sometimes you don't have the time to go back and watch 15 hours of video. Sometimes you just need a quick response. Other times you're in a situation where we really need to dive deep into this topic and we really need to understand that topic really well and you need to have a chat experience with that concept.
So, how it actually works, which is probably the most important thing.
There's a lot of marketing hype in this area right now. Rightfully so because there's a lot of value coming out of these tools. But we wanted to show you how this actually works in practice.
When you go into the Voxpopme platform, you can create a project from scratch. In this case we're about fitness routines. Then put in your objectives, what are the main things that you're trying to achieve with this? Then you can choose, ‘okay, do I want it to be an asynchronous piece of research or maybe I want to use an AI moderator or a human moderator or both.
And set up flow super easy. Click on the AI moderator and look, there's a participant experience right there. That's what the participant experience looks like to talk to one of these moderators.
Then once you've figured out, ‘okay, this is the methodology that I want to go into,’ you can do things like name your interview room, the title of the room that the person is going to be in. Choose an AI moderator voice, which is super interesting to test and just to see the impact of that research.
Then my favorite part about it is how we have implemented the discussion guide pieces of this, which is that you can let the AI write your discussion guide based off of your objectives.
If you're somebody you had to have, I don't know, five or 10 people, look at your discussion guide and approve it. You can also just upload that into the platform.
I think in this demo video, we're letting the AI write it for us.
What it's going to do is take your objectives and see, ‘okay, how could I ask these questions in a way that's going to get these objectives filled?’
Then it's also going to give you the estimated time for how long this is going to take. What we didn't want to do was say ‘oh, you want it to be 45 minutes? Let's trap somebody in a room for 45 minutes.’
No, sometimes you want to just get answers to your question, and I think that it's super important, the ability to say, if you get through this in 18 minutes, maybe it's just a fast talker like me, or maybe it's because the AI moderator realized that on the first question you already answered the second and third question, that's okay.
We wanted to give that flexibility for how you can interact with your guide and give you an estimation on how much time this is actually going to take.
Here we're also showing the upload flow.
If you have a guide that you're already working on or you've already gotten approved, you can upload that as well, which is super important.
Then the last step of this is the audience bits. It’s how you connect that piece of work with participants.
You can either source those participants through us on the Voxpopme side, or if you have your own sample of people that you already talked to or you're going with partners, you can absolutely do that as well.
Screeners, of course, everybody knows how this stuff works at this point.
Then segmentation and boom, it's live.
That is really how quickly it was. That wasn't a super sped up video. If anything, it was slowed down a little bit so that I could talk over it in some areas.
But, as you can see, this is really quite easy of a setup. If you want to get super complex and you want to go really crazy with screener questions or you want to add some more condition logic to your prompts or the questions that the moderator asked, it could take you a little bit.
Yes, absolutely, you can go as in depth as you want with this, but it's also really exciting to see how this is empowering teams to be more agile with their research.
The fact that you could launch this side-by-side with a human moderator and get the scale that you need with AI moderation, but the human understanding as well, I think it's a nice compliment to both.
So, in that scenario where you've launched 50 or 500 moderated sessions, whatever, that's great, but now what if you have 500 hours of people talking about their experience with Red Bull. Unless you have an intern that wants to stay on the weekends and drink a couple of Red Bulls themselves, you're not going to be able to get through that.
The other side of this is if you're using a tool that doesn't appropriately show evidence and show video evidence of the findings, then how are you supposed to trust that it's not a large language model hallucinating or something?
The way that we built the analysis side of the Voxpopme house is that it always shows evidence on every single thing that it brings back to you and it allows you to check it.
We want to make sure that not only do you know that the insights are there, but also that they are built off of trust.
The end goal of this is that once all these interviews are done, the Voxpopme side of the house really kicks in. So, themes, chat clips, quotes, trends, then you're moving from the raw feedback to a story as quickly as possible.
The end result of this, what we're seeing is that whole idea of, ‘well, we just don't have time to qual.’ Your teams are really empowered to add this into research and add that depth into research. Which has been really quite cool to see because it's just resulting in a better experience for everybody.
How can you use AI moderation?
This is a question I've been getting a lot lately. We have some more content that's coming out about this, but what situations do we see market researchers using AI moderation? That's really interesting.
The first one we see a lot of internal projects run, which is kind of fun. Everything from, ‘Hey, I'm going to do these human IDIs, interact with the AI moderator and see what you think of this discussion guide,’ all the way to, ‘we want to show everybody this methodology so that we can start to become more comfortable with it and understand that.’
We also see clients who are just launching internal projects for fun. Tell me about the last time you went to one of our stores. What did you think about it? Those internal projects we've seen a lot of. I think it's a really great way to introduce the methodology and feel a little bit more comfortable with it.
The second one I see a lot is side-by-side. So, part human, part AI.
In that case, it's usually humans who do maybe something like five IDIs and then they're going to scale that to 20 with AI moderation or they have the AI doing moderation and they're really leaning on the human to get more of the analysis pieces together. Either way, we see a lot of humans working with these agents really well.
Third, one that is super interesting to me is sensitive topics. One of my favorite stories from this was that we had one study that was working on a segment of a population. It was menopausal women, and the lead researcher was male.
These are just topics that your moderator coming into the room is going to bias the conversation. Sometimes it is better to have an AI moderate things. Sometimes we see that participants are a little bit more honest as well with the AI moderator. We're also seeing that sometimes it's a little bit easier to talk about some sensitive topics with AI moderation and then get into the really common ones.
Buying motivations by our personas, segmenting with AI moderation is really interesting usage and attitudes.
Then the last one that I'll mention is one of my personal favorites, current events.
I have some examples for you on the screen, but the current event topics around GLP-1s like Ozempic or tariffs, these are things that really work well with AI moderation because they are a little bit of a sensitive topic, because they could have political connotations, or in the case of Ozempic, there's cultural connotations about weight.
We see a lot of these current event topics ran also because it doesn't take you four weeks to launch the study. Today you could go and talk to a hundred people who are on Ozempic about why they chose Ozempic and how it's influencing their decisions at the grocery store. And you could get those a hundred responses, I would say probably within 48 hours depending on how great your screening criteria is. And it's like unlocking this really quick turn research about current events, which is super interesting.
As we build research tools in this AI first or AI side-by-side world, what are the things that the Voxpopme team is thinking about?
I think it's important to share this because we want to make sure that these are built responsibly, of course, but also in a way that they will actually want to get used. Aproduct that's just built for the sake of building it isn't helping anybody.
As we're going down this path to improve this product, we're thinking about keeping conversations at the forefront of insight. So, how do we get scaled conversations? How do we get more people to give real opinions?
There's a lot of talk about fraud in the industry and we want to make sure that if you're making a decision, you can check like, ‘Hey, we're making a decision because of this insight, and here are the videos of people saying this thing that's really important.’
The next is the participant experience in AI first research.
I showed you a little bit of the participant experience earlier of what that looks like. We really care about this because of a couple reasons. One, fatigue.
With AI moderation, there's going to be a lot more people running these types of studies. We want to make sure that people actually want to do it. Right now, we have a lot of people, participants, who give really good feedback on the participant experience, and we want to keep that.
Then two, because sometimes recruits are really hard and you want to make sure that if you are able to find that really small segmented group that you're trying to capture in, I don't know, people who don't have the internet purchased in Indiana, you want to make sure that they have a really good experience when they go through something like that.
Then next, the grounding of the analysis, evidence and truth.
I'm really keen on keeping evidence and truth at the center of this. So, if you're making a decision, we want to go with this style of packaging, we want to make sure that if you're making that decision you not only get that insight, but then you also have the evidence and you're confident that it was significant and the right decision to make.
Then lastly, customization and specification. We want to be able to customize things more.
The level of probing is one that's always up for debate and customization on that, customization of name, customization of experience, customization of everything from your participant experience to how you do the analysis for a concept test. We're really thinking about how we can customize this to get these quick turn insights ready for everybody from brands to innovation teams.
Then, of course, building AI tools for market researchers with market researchers.
If there's a situation where you're thinking about, ‘maybe I want to try this or maybe I want to more in-depth demo, or maybe I just want to see the participant experience, reach out to us. Reach out to me on LinkedIn, I'd love to talk to you about it. Look right there, it's a QR code for just that thing.
Right now, we're offering the ability to run a pilot study with AI moderator with Voxpopme. It's the ability for you to actually try to launch one of these yourselves. Choose your audience. You can upload your guide or let the AI write it for you. Then you can launch your study, and you can just review the insights. It's really that simple.
Then of course, yeah, you can use your phone to scan this QR code to connect with me on LinkedIn.
Honestly, we are really keen to have some conversations about this methodology, and we're constantly looking at how this can impact market research and how we can build it responsibly and also in a way that provides the most value. So, even if you're saying, ‘Hey, I don't think that this is a good idea, and I don't think that AI moderation is for me.,’ I want to talk to you too. I want to understand why.
If you're thinking, ‘oh, man, I could have really used this on a study that we did last month.’ Let us know. Reach out.
Maybe you don't have something that you're running right now, but you want to look back and say, ‘Hey, how would this have been different if I used this technology? ‘Because sometimes that can be even more helpful than running something brand new.
Or if you're thinking, ‘Hey, what does it look like to run an AI moderator versus something like a video survey or an AI moderator versus a human moderator?’ We can absolutely do a side-by-side together. The team at Voxpopme would love to do that. It's one of our favorite types of projects to run.
That being said, I hope that you never have to skip qual again. I hope that these tools just continue to get better and that you get to a point where Qual doesn't become a nice add on or a side-by-side rate. I hope it becomes something that you get to do in every single aspect of your work.
That is a wrap. We'd love to answer any questions. There's my e-mail and also you have my LinkedIn on the previous page. Love to chat about this.
Thank you guys so much for your time. I really, really appreciate it. I hope you have a great day. Bye.