How Conversational AI Can Speed Up Your Time to Insight
Editor's note: Automated speech-to-text transcription, edited lightly for clarity.
On June 21, 2023 Human Listening™ gave a webinar on conversational AI and how it can help to speed up time to insight, especially with qualitative research. They talked a lot about the chatbot's ability to have a conversation as if it were two humans talking to each other. The speakers also talked about the best ways to analyze the data collected through conversational AI.
Watch the full video or read the transcript below from June’s Wisdom Wednesday session with Human Listening ™.
Webinar transcription:
Joe Rydholm:
Hi everybody and welcome to our webinar from “Text to Talk, how conversational AI can Speed Up Your Time to Insight.”
I'm Quirk’s Editor, Joe Rydholm and before we get started, let's quickly go over the ways you can participate in today's discussion. You can use the chat tab to interact with other attendees during the session and you can also use the Q&A tab to submit questions to the presenters during the session and we'll answer as many of them as we have time for during the Q&A portion.
Our session today is presented by Human Listening™, enjoy the webinar.
Chris Barry:
Hi, I'm Chris and I run the Human Listing™ Business for the Evolve Group. I've spent my career on both agency and client side in insights and analytics roles, connecting stakeholders to their consumers, customers, members and employees to help deliver better commercial outcomes.
Human Listening™ is an AI powered insights platform focused on creating a meaningful engagement with your consumers. It has a suite of applications to do this from scripting and deploying surveys to analyzing data and visualizing it in dashboards.
I'm joined on the webinar today by Gabe McElwaine, VP of sales for Human Listening™. Gabe has been in the insights industry for many years, more recently, bringing AI based solutions to the market to help drive growth for organizations.
Today we are here to talk to you about conversational AI and EVE, our conversational AI companion with applications to build control and integrate conversational AI into your research program. You can uncover deeper insights and emerging trends that will help you make smarter business decisions.
After we finish the presentation, we'll be online to cover any questions you may have.
We started Human Listing™ as we continually heard complaints around standard quantitative surveys. I think everyone here understands that surveys alone just aren't getting the job done for all the reasons that you're seeing on screen.
Now, not only are the results companies get less and less reliable, but participants are less and less interested in clicking boxes and choosing responses that someone else has scripted. So probably more of a rhetorical question, but who here really loves doing online surveys?
I'm pretty sure we've all heard presentations over the years on the difficulties of surveys from programming to respondent experience to bots and everything else. If we met now and I wanted to get to know you, I wouldn't ask you to do a survey. I'd have a conversation with you and begin to build a relationship. So why do we force our customers to do it this way?
And when you think about it, this problem isn't insignificant. When organizations want to have human to human interactions, they tend to run things like focus groups, but they don't scale.
And when you're looking to support business cases, organizations often lean to quantitative methods which will give you the what but not the why. They lose the human engagement.
Human Listening™ delivers human engagement at scale. In essence, conversational AI can be the unifying element that bridges the gap between knowing and understanding your target audiences. And whilst we've been using other large language models to build the application of conversational AI and market research for over five years now, it's the leap in both acceptance and skepticism that places us here with you today. How do we harness the power of conversational AI in this industry? That is latching on to open AI's GPT models.
It's an important topic and no doubt the technology and the chat experience will continue to enable the elicitation of better and richer data.
I think we've all seen the box on the left for a simple verbatim response at the end of a survey. I know I'm guilty of entering something like “nothing else to add” or “everything's great,” which isn't helpful to anyone.
But on the right is a quick glimpse at how open-ended verbatims via conversational AI can come to life. Not only does using a tool like EVE get you better and more informed responses, it also gets the respondent more engaged both in their responses and with your brand.
This is a good time to step back and think about what conversational AI actually is and how the structure of it gives us a more robust view of a mindset simply than using chat GPT.
So I'm gonna hand over to Gabe now who's gonna run you through the rest of the presentation.
Gabe McElwaine:
So when I say chatbot, this is probably the first scenario that comes to mind for many of you, yet another hurdle to overcome just to get some help from someone who knows the answers to your questions. Responding without acknowledging context has got to be one of the most frustrating aspects of interacting with virtual chats.
What conversational AI allows researchers to do is stop thinking about quant is good for this and qual would be better here. Conversational AI allows for research to be structured in a way that helps to understand what's really important in real time for both participants and the researchers. We've all likely heard of and used plenty of qual/quant solutions over the past few years where conversational AI can start to lead to a real fusion of methodologies.
This idea that we can explore with a minimum of structure rather than being reactive or simply looking to validate or disprove hypotheses. There's more time for exploration of concepts and ideas. Doing this at a quant friendly scale allows for text analytics to measure the insights given.
We've also found that conversational AI tools draw out more detailed responses, particularly on difficult topics like financial interpersonal dynamics or even death where respondents aren't worried about opening up to an interviewer in an IDI or in front of strangers in a focus group.
And what are the underpinnings of conversational AI?
They begin with large language models and there are scores of them with insanely large amounts of data to model. The one we're all likely most familiar with is open AI's GPT-3 and GPT-4, which are only a couple of the number of large language models out there. There are many differences that OpenAI claims it can do that others cannot. I'm not gonna get into that today, but suffice to say that if you've tried it out, it can perform a wide range of natural language processing tasks and can deliver a very good one way chat experience.
We'll discuss how that's a little bit different and more detailed as we continue on.
So we ask ChatGPT what it thinks it is and it defines itself as an advanced large language model.
Large language models are predictive modeling exercises using machine learning to predict the next word in a sequence. As we referenced in the previous slide, they're trained on enormous data sets.
GPT-3, the old tech from way back in January ingested over 400,000 books to build its predictions. GPT-4 has scaled that up 10,000 fold. Making the outputs sound human-to-human is one of the biggest challenges LLMs need to overcome, which is where ChatGPT comes in. A fine tuned version of GPT-3 and -4, using the same architecture and pre-training, but specifically adapted to perform well in conversational language tasks.
So what does this mean in terms of the amount and types of data that can be collected?
Think of survey data and it's pretty much a binary exercise, this or that, this or not that, some of this and not some of that. Conversational data has far more layers and holes to go into probing for more detail and true insight.
One of the key benefits of conversational AI and platforms like Human Listening™ is taking all that unstructured data and putting it together into a coherent and cohesive structure so that researchers can begin to understand very quickly what issues are rising to the surface.
But conversational AI does more than simply structure the verbatims by intuitively following up on responses participants give to the conversational AI that asked questions. There are multitudes of different types of conversations that can happen all at once.
In these instances, even a serial rejector gives away that their spouse buys the cereal, allowing researchers to get insight into shopping behaviors that a typical survey question likely would've missed, or even pulling that conversation towards something your team might find more relevant in this example. Shifting from texture to health benefits.
In Human Listening™, our platform allows our clients to look at similarly tagged verbatim responses and understand if others are having similar experiences.
Of course, there are still best practices needed in order to truly benefit from conversational AI tools in order to make sure respondents stay engaged. The probing needs to have some sort of limit, but also things like the tone of the AI moderator and priorities of what the client is trying to understand so that the questioning doesn't go off on a tangent that doesn't correspond to a commercial need.
You may remember this slide from the beginning of the presentation. Now think about all the things we just laid out over the past few slides. Sure, this incident shows frustrations with the dairy section, but we all know that there are multiple threads of the shopper experience in a store.
Another group likely enjoyed the revamp produce section. Others had to wait in a really long line at the checkout. Maybe the bakery section introduced some new cakes that they really liked.
How many moderators and focus groups would you need to run to get an overall satisfaction? How much survey scripting would you have to do to understand what a seven out of 10 shopping experiences really meant?
Conversational AI allows all that to happen all at once in one unified dataset. Which brings us to analyzing all that conversational data that you've just gotten.
One of the great things about these conversational AI tools is that you can use just about any type of quantitative analysis technique with any conversational data that you get.
Things like exploring emotional aspects and feelings, responses of two groups quantifying what people are saying at a numerical level, exploring causal relationships and even identifying more complex relationships between topics are all things that can be done with this great combination of qual and quant data.
Here's another way of visualizing how a seemingly simple question of shopping preference can lead to millions of differing conversational threads.
I think as insights professionals, we all want to make sure that we have enough data with which we make decisions. So having the ability to aggregate these data sets into something usable and understandable is a key feature of conversational AI tools.
Since we seem to be on a breakfast and serial kick here, let's take a look at a quick view of a conversational AI interaction on our platform.
As you can see through the responses and through the questions being asked, there certainly is a natural flow to this conversation. Rather than doing what a traditional chatbot would do, forcing a conversation to stay in one area, then giving up when it wasn't getting the responses it understood, a well-executed conversational AI dialogue will respond to the participant's inputs, asking relevant questions and follow-ups without going too long and losing the participant's interest.
One of conversational AI's great strengths in uncovering unknown knowns and especially those unknown unknowns, which are the drivers of innovation and product satisfaction.
And of course, having all of this information at your fingertips is great, but it still needs to be presented in a way that can bring it all home for you and your clients.
Taking those quant attributes that are the underpinnings of large sample sizes like demographics, regionality and income, even country, and being able to filter those responses by metrics that are key to understanding your audience in more detail.
Of course, by tagging verbatim and coding them in real time as they're received, it helps free up researchers for more time acting on the data they see, rather than taking countless hours to summarize it themselves.
Putting all of this together in one unified database is a key aspect of leveraging conversational AI in your learning plans.
And where you can see generative AI and conversational AI working exceptionally well together is in developing summaries of large qualitative data sets, understanding the broad themes of differentiation between brands, in this case cereals, allow for fast ways of understanding consumer opinions and views, as well as the differences in brand perception that you see in this example.
And of course, conversational AI has broad applicability for all of your research goals, whether it's customer experience, employee experience, brand health concept testing, product testing or strategic research. Conversational AI is a great tool for you to use regardless of what you're looking to do with your learning plans.
Chris and I would like to thank you all very much for spending some time with us and Human Listening™ today. If you have any questions, please feel free to reach out to us and we'll be happy to talk to you.