Uncovering hidden insights
Editor's note: Rick Kieser is CEO of Ascribe. He can be reached at rick.kieser@goascribe.com.
Artificial intelligence (AI) has gone mainstream and there is simply no escaping reading, hearing and talking about it. No matter the industry, AI will impact it. Indeed, AI is already impacting such disparate industries as finance, health care, travel, national security and farming!
Marketing research has traditionally been hesitant and slow to adopt change as an industry. However, in the case of AI, we’ve already rapidly embraced many AI-based tools. To increase the value of these tools, researchers will benefit from understanding them and what they can do to help now and in the future to satisfy the ever-increasing demand for lower costs, greater efficiency and faster and better insights.
Why marketing research needs AI
Marketing research is the foundation of successful marketing, product development and customer service. But traditional research can be expensive and time-consuming. Simple surveys can cost between $15,000 and $45,000 and often much more, and B2B one-on-one interviews cost $300 to $500 each and more. Focus groups, on average, cost $8,000 each. And a simple project takes weeks to complete.
Further, traditional research is not designed to handle the mass of data now available in some businesses (e.g., retail, CPG and others). As a result, due to the cost and time investment needed, many companies are priced-out of using research completely. And many companies opt to use market research only for the most critical and risky projects, such as product launches.
As an industry, we must find ways to reduce the cost and time barriers to marketing research and make insights more available for businesses inundated with omnipresent feedback. AI is an essential piece of that solution.
AI 101: Moving to advanced generative AI
AI is a wide range of technologies that apply to many different situations and refers to the ability of machines to perform tasks that typically require human intelligence, such as learning, problem-solving and decision-making. Generative AI is a specific type of AI that uses algorithms to generate new content, such as text, images and audio, based on the data on which it has been trained. The main difference between AI and generative AI is that AI is a broad term encompassing many different algorithms. In contrast, generative AI is a specific type of AI that focuses on generating new content.
The earliest implementations of AI used a rules-based approach limited by a lack of context, a limited vocabulary and overreliance on patterns. Natural language processing (NLP) enabled machines to understand and work with human language. Other language models, such as BERT, provided pretrained models that can be fine-tuned for specific NLP tasks. With the introduction of machine learning algorithms such as generative adversarial networks (GANs), AI has become more powerful and capable of creating convincingly authentic content. More recently, rapid advances in generative AI and in large language models (LLMs) have provided the ability to produce even more creative results. As a result, generative AI has become a game-changer in many fields, including art, biology, programming, writing, translation and more.
While there is currently some experimentation with using AI to develop surveys and analyze survey datasets for comprehensive insights, most AI usage in the insights industry has historically been in the analysis of open-end comments. Generative AI already positively impacts the industry, especially in analyzing open-ends from surveys, social media, reviews and other sources. It has been used in the industry for nearly a decade. It is always important to remember that AI's value is to increase the speed of delivery and reduce expense, not to eliminate critical human oversight, validation and interpretation.
Where are we at currently?
In November 2022, generative AI was launched globally. Because the developers understand that researchers must always use results to deliver meaningful insights, the companies developing these tools are trying to understand how researchers want to use generative AI and what they want the tools to accomplish. As a result, there has been a flurry of applications using generative AI within the open-end analysis space and the tools are evolving rapidly to be not only easy to use but also effective in providing better analytics and insights.
One important evolution was to advance the results produced from topics (simple, one-word facts) to themes (more descriptive ideas with more human-like insights). Because important insights are often found in open-end responses, analysis that results in themes allows the researcher to find those insights faster without all the manipulation necessary with topics. For example, a topic-based analysis for a clothing manufacturer might give the result “zipper,” and you would have to look deeper to find the issue. Themes, however, would return “the zipper is broken.” Similarly, for a restaurant, a topic results might be “food,” where themes would give you "the food is delicious" or “the food is cold,” telling you immediately what action you need to take. Themes are all-encompassing, while topics are narrow, and themes, as logical ideas, give you direction for subsequent actions.
Generative AI has also powered the development of several other advances:
- The ability to easily net themes for increased insight gave even more power to generative AI open-end analysis by quickly facilitating human interpretation and combining similar themes.
- Adding sentiment analysis to themes also helps researchers quickly sort and communicate the brand or product’s status and track how it changes over time, significantly reducing analysis time.
- Segmentation analysis allowed researchers to understand better how perceptions differ by segments within the data.
- And finally, multilingual capability eliminated the need to translate responses before analysis.
These advances came from developers understanding what researchers needed and responding by building new and better capabilities with generative AI into their market research solutions.
Case studies
The latest evolution of generative AI is moving beyond delivering topics or themes to providing more comprehensive and illustrative summaries from the analysis. To shed light on this capability, here is a case study based on a one-question customer satisfaction survey about a health club. A total of 2,000 responses were analyzed.
- The first analytic query was, “Are customers generally satisfied with their experience at the clubs?” The generative AI tool responded: "No, customers are not generally satisfied with their experience at the clubs. While some customers have had positive experiences with the local employees, many customers have expressed dissatisfaction with the condition of the facilities, the lack of updates and the lack of amenities."
- The second query was,“What are some actions the owners can take to improve customer satisfaction?” which got this response: "1. Ensure that maintenance issues are addressed promptly, even on weekends. 2. Improve the cleanliness of the facility, especially in areas such as the women's bathroom, shower area, pool, jacuzzi and steam room. 3. Set all clocks correctly. 4. Install more fans to improve air movement in the facility. 5. Allow members to extend their trainer sessions at the rate they signed up for. 6. Reallocate funds from transforming and furnishing offices to improving the club."
- The next query was, “What are some things the fitness clubs are doing well?” Positives for the clubs are: "The fitness club is doing well in having a highly trained and professional personal training staff. They have also done a good job of making the gym look fresher and more inviting by adding plants. They have also been friendly to guests and offered free 10-day memberships to family and friends."
- The next query was limited to the segment of NPS detractors. The query asked what NPS detractors say about the clubs and generated this response: "NPS detractors are saying that the bathrooms are not clean, the personal training area is too small and uncomfortable, there is not enough variety in the extra workout classes, the markings on the plates on most machines are rubbed off, it is very hot and humid in the locker room and the ceiling fans in the basement are growing hair on them."
- Finally, a query about a specific area of the club, the pools, resulted in this response: "The biggest complaints about the pools are that the water is cloudy and too cold, the whirlpool is broken or not hot, the sauna is not very hot and the walls and floors of the pool area are peeling paint and have chipped and broken tiles. Additionally, there is no temperature regulator on the pool shower and no paper towels in the locker room.”
This type of analysis also applies very importantly to non-survey open-end comments. Here is an example analysis of 1,500 social media posts from visitors to the London Eye:
- The first query was, “What is this dataset about?” The response was: "This is about customer reviews of the London Eye experience. It includes information about the customers' experiences and recommendations and responses from the Customer Services Team."
- The second query was, “What are customers happy with?” The response was: "Customers are happy about the knowledgeable and humorous guides, the champagne experience, the great views and the ability to create good memories and take good photos."
- “What are the top three improvements the London Eye could make to improve people’s experience?” was the third question. AI generated the following suggestions: "1. Provide an audio commentary to give visitors more information about the landmarks they are seeing. 2. Make the wheel stop periodically to allow less-mobile or disabled visitors to board. 3. Provide a map or guide to help visitors identify the landmarks they are seeing."
- The next query was, “Looking only at reviews from families who visited the London Eye, what are their key likes and dislikes?” The results were: "Likes: Beautiful views, Spectacular views of London, Our tour guide was wonderful with descriptions and jokes. Dislikes: Overcrowded cabins, Lack of rules for how many people can get in the cabins."
- Another query, asked only of a segment of all social media reviews, was “What are people happy about in reviews posted since April 2022?” This list of the most frequent topics was AI-generated:
- Iconic, amazing 360° views of London
- Knowledgeable, approachable, fun tour guides
- Comfortable way to see the city
- Great historical facts and pointers for our trip
- 2-for-1 tickets
- Running commentary was really entertaining and informative
- Fun and educational trip
Generative AI is very good at summarizing text information but it is not good at quantification. And as researchers, we typically need to quantify the analysis results. When you need to build a code book, using themes is a good start, but to refine and develop the codes to be applied, you need human oversight, validation and interpretation. Applying AI themes analysis to the London Eye open-end comments successfully identified 138 codes (covering 96% of the total comments). After the themes were analyzed and aggregated by the researcher, the results were quantified as shown in Figure 1.
Further, once the comments are coded and quantified, they can be further analyzed to explore different segments of respondents and to generate additional insights, as in Table 1.
Summaries, which generative AI helps develop, are an essential advancement in open-end analysis solutions because they give researchers enough detail to quickly understand the verbatim responses while saving time in wrangling the data into meaningful and valuable information. However, to turn verbatim comments into quantitative information, you need a human being to massage the results of the generative AI analysis and communicate it to the end client. It is important to remember that the key benefit of generative AI is not to eliminate human review and interpretation but to put human time and talent where it is most valuable: in validating, evaluating and communicating essential insights to clients. Generative AI will continue to evolve and improve but human oversight will always be needed. In any event, generative AI unleashes the power of summaries to quickly comprehend what open-end comments are telling you.
Evolve and change
As our industry continues to adopt and embrace generative AI, the research solutions will evolve and change to meet our needs better. At this point, they deliver sound value to the market research organizations using them, saving time and labor and helping researchers put their effort where they are most valuable. If you haven't tried solutions that analyze open-ends, or if you tried them and rejected them previously, now is a great time to take a look at these platforms. Generative AI for open-end analysis is here to stay. If you need to analyze open-ended comments, you can't afford not to jump on board.