Talking back to the bots

Editor's note: Keith Bossey is SVP on research company GfK’s financial services team, New York. Gavin Lew is managing director of GfK’s user experience practice. He is based in Chicago.

In industries across the business spectrum, amping up customer service with digital technology is not just an option, it’s an imperative. Apps, mobile-friendly Web sites and artificial intelligence (AI) programs promise huge savings for companies of all kinds, as well as increased convenience for customers in need of help. 

But this digital win-win can only be realized when consumer-facing businesses implement change carefully and monitor closely.

The financial services industry is facing this dilemma head-on. On the one hand, providing truly personalized customer service to millions of consumers – some of whom may have very little money to save or invest – is simply not feasible. But increasing the industry’s use of non-human service reps poses big risks for banks and investment firms, who traffic in the one thing people tend to worry about most: their money.

Let’s face it: Banks are not noted for managing customer relationships or public relations well. Unexpected fees and charges, changing terms and conditions, plus a financial meltdown or two – the individual customer has little reason to think highly of Mr. Big Bank. 

In this difficult climate, data holds multiple promises for banks and other financial institutions. With access to a host of information on any one consumer, or segment, financial services (FS) companies should be able to increase personalization through adept use of AI and other digital services. Recent GfK research shows that, while roughly 85 percent of consumers want their banks to give them financial tips, only about 30 percent are actually getting that service.

If AI can deliver these kinds of much-needed benefits well, then banks can save money and boost customer satisfaction – a huge win-win. But the drive toward innovation needs to be accompanied by a willingness to measure often and respond nimbly to problems as they arise.

Simulates human conversation

One very prominent and growing example of digital personalization (and efficiency) in FS is the chatbot, a computer program that simulates human conversation. Using artificial intelligence, the application can answer simple questions and gather information, all the while limiting the need for the intervention of a real person. If you have ever “spoken” to someone while interacting with your bank’s Web site or app, that probably was a chatbot – or involved a chatbot “enhancement” to a human interaction.

A recent GfK survey found that roughly 18 percent of the U.S. population say they have interacted with a chatbot at least once – and the reports are not always positive. Among the highest-value customers (with incomes of $200,000 a year or more), incidents of negative chatbot experiences were twice as common as positive ones. Fully 15 percent of this important consumer segment said they had walked away unhappy after using a chatbot, while just 7 percent were satisfied with the exchange. 

These high-income consumers tend to be among the most open to – and anxious for – digital enhancements for managing their money and investments. One-third (34 percent) say that it is more important for a bank to have good online services and/or a high-quality mobile app than it is to have helpful staff in the local branch or office; this compares to 29 percent in the general population and 27 percent among those earning under $100,000 annually. 

But the biggest earners and investors are also more likely to say that their financial needs are “too complex” to be met by a mere interface. An inescapable fact for banks is that roughly two-thirds of all consumers, and almost three-quarters of high-asset investors (investable assets over $500,000), say it is still very or extremely important for banks and other financial firms to have a real person whom customers can speak to when they have problems. 

Find the right mix

The challenge for banks and other institutions, then, is to find just the right mix of human presence and digital efficiency. We know that 24/7 service and paying less for human advisors are appealing benefits for bank customers yet they also want to feel confident they will be able to “speak to someone” when the need arises.

The answer, as you might have guessed, is consistent, finely-attuned feedback. Chatbots represent a core customer experience issue, one that directly affects satisfaction and, potentially, loyalty. While studies show that customers tend to be reluctant to switch banks, it is also true that money has never been more portable. 

When we think about ways to measure and improve upon chatbot experiences, traditional surveys may be the first things that spring to mind – because marketing researchers have been relying on them for so long. And it is true that a traditional survey sent out immediately after a chatbot interaction could capture some overall impressions and feelings about the exchange. We might ask: “Was the experience satisfactory?” “Did you get what you needed?”

The true source of satisfaction with chatbots, though, is in the dialogue itself. Did the interface remind you repeatedly that it is not human – or was the difference barely apparent? To design a great chatbot experience is essentially to engineer a conversation – and that is much more difficult than one might guess. In a recent study¹, French train service Voyages-sncf.com sought to create a chatbot that would help travelers buy tickets and get information – essentially providing the same interactions that a human ticketing agent would. 

But the train company quickly learned that basing its bot’s actual word choices and dialogue strategy on the conversations people have with flesh-and-blood ticket sellers was a huge mistake. Without the context of the train station, the waiting line, eye contact, physical gestures and a host of other factors, the digital interactions were completely different. For example, you would never get to the front of a ticketing line and say, “I’m here to buy train tickets” but you might well start a chatbot conversation that way.

This and thousands of other examples of dialogue and interaction show the promise of a user experience (UX) approach to evaluating and improving chatbots. UX is an artful science, capturing and making sense of all the feelings, ideas, preferences, perceptions and responses that accompany the use of anything. Usually achieved through one-on-one observation, UX has been applied countless times to people’s interactions with Web sites, medical devices, automobile systems – pretty much anything you can think of.

But an ongoing challenge for UX is scalability. Focusing on the interactions of 10 or 20 customers with a bank’s app, Web site or bot will be revealing; but it cannot truly diagnose whether problems are widespread or fairly limited. Financial institutions with millions of customers worldwide need solutions that can reliably detect not just individual reactions but also the scope and magnitude of specific issues. They also need a compact way to get at the brand value aspects of user interactions – answering not just “Did the system work?” but also “How did the interaction make you feel?”

One scalable user experience measurement model, first defined and validated by GfK in 2014², is UX Score. Through a series of 10 questions, UX Score captures three dimensions of user experience – usability (task-oriented qualities), usefulness (self-oriented) and aesthetics (Figure 1).

In tests, UX Score proved to be a solid predictor of Net Promoter Score (NPS), purchase intention and market share. Scores can also be compared to competitors in the same category to understand strengths, weaknesses and opportunities for gaining advantage. 

In September 2015, GfK applied the UX Score model to studying customers’ interactions on three platforms – ATM, mobile app and PC-based online – with four major banks. By developing a clear picture of successful experiences and less-than-satisfying ones, the research could help banks understand where to spend crucial resources to improve customers’ feelings about their brands.

Using a nationally representative sample, the study found that roughly two-thirds of respondents overall felt the different touchpoints had “exactly the features” they were looking for – with mobile apps scoring slightly lower than ATMs and PC-based Web sites (see Figure 2). But only one-third said that the touchpoints had “interesting new features,” with mobile apps scoring slightly higher here.

Across all four banks, the three types of touchpoints scored well on usability but fell short on engagement (see Figure 3).

Four imperatives

Where do we go from here? It is clear that self-service touchpoints, powered by AI, are here to stay – and the proportion of interactions that rely on these digital channels will only continue to grow. Financial firms will therefore have four imperatives where a quantitative UX approach can be valuable:

Innovation: As products and services become digital experiences, traditional approaches used to measure concepts and identify improvements will become increasingly irrelevant. Actual user testing (qualitative) and UX Score measurement (quantitative) can have more immediate impact.

Customer experience tracking: While measures such as NPS will continue to be used for many channel interactions, and to understand customer experience at the relationship level, it will be more insightful to have quantitative measures integrated into tracking programs or have distinct tracking mechanisms for digital touchpoints.

Customer experience improvements: Deep dives into outages at these digital touchpoints can be assisted by the use of approaches such as UX Score as a pre/post measure, in both qualitative settings and larger-scale quantitative initiatives.

Competitive assessment: In industries such as financial services, where differences in product features and benefits are often small, the experience of delivering services becomes the brand differentiator. UX represents the delivery on the brand promise and assessing its success allows financial firms to look for areas of personal vulnerability or strength as well as opportunities to steal share from competitors.

In line with expectations

As companies rush to digital and AI as sources of personalization and efficiency, it is imperative that customers’ experiences are in line with expectations – usable, useful and aesthetically appealing. An innovative UX approach to this need can be both cost-effective and truly actionable if implemented with care and wisdom.

References

¹Pascal Lannoo and Frédéric Gaillard. “Conversational commerce and chatbot Design.” Presented at UX Masterclass, Shanghai. April 20, 2017.

²Bosenick, Tim, and Raimund Wildner. “How to measure user experience … and to calculate its ROI.” Digital Dimensions 2014, pp. 1-9. Published by ESOMAR.