Are you satisfied? The evolution of customer satisfaction measurement
Editor's note: This article is an automated speech-to-text transcription, edited lightly for clarity.
Customer satisfaction is a popular metric to measure, with many methods to do so. During a session in the 2025 Quirk’s Event – Virtual Global, speakers discussed the evolution of customer satisfaction metrics.
Samir Saluja, founder and managing partner at DeriveOne and Greg Timpany
business intelligence at Dairy Max, presented several methods of measuring customer satisfaction, like NPS, ACIS and more. The pair also walked through how the metric could be affected by AI within the next 10 years.
Session transcript
Marlen Ramirez
Hi everyone and welcome to the session, “Are you satisfied? The evolution of customer satisfaction measurement.” I'm Quirk’s News and Content Editor, Marlen Ramirez.
Before we get started, let's quickly go over the ways you can participate in today's session. You can use the chat tab to interact with other attendees during the session, and you can use the Q&A tab to submit questions for the presenters during the session.
Our session is presented to you by Dairy Max and DeriveOne. Enjoy the presentation!
Greg Timpany
Good afternoon, everybody. This is Greg Timpany.
Samir Saluja
And Samir Saluja here from Derive one.
Greg Timpany
We're going to be talking today about the evolution of customer satisfaction measurements.
Let's go ahead and dive on in.
Alright, I always like to start these presentations off with the definition. So, what is satisfaction?
It's a confident acceptance of something as satisfactory, dependable or true, courtesy of dictionary.com.
We're going to take a brief history lesson here today. The question on the table is, when did customer satisfaction measurement become a thing?
Well, according to a 2012 Quirk’s article, authors Vavra and Pruden indicated that customer satisfaction measurement, as we know today, really began to gain ground in the late 1980s and early 1990s.
Now during this period, it was a considerable challenge to convince management that it would be beneficial to actually listen to what consumers were saying. The reluctance around that initially was not cost concerns, but the fear of being told exactly how well or how poorly they were doing and the uncertainty of how to respond to the information. This is still true today, by the way.
Now the key, as it has always been, is to have a plan. What do we do with this information? Including how we're going to be sharing it back with the customers, so they know that we're actually listening to what they have to say.
Let's kind of ground ourselves here with a very simple but performative model here. This is the performance versus expectations model of customer satisfaction.
Now, authors Grapentine and Soorholtz remind us that satisfaction as we know it is really a dance between our pre-experience expectations, the big E, if you will, and our post experience perceived performance. Positive differences there between performance and expectations will serve to increase customer satisfaction where the opposite is true for any negative differences. So, you really want to make sure that the performance of the product or service exceeds those expectations if you really want to drive satisfaction.
Alright, now we're going to take a look at some of the common methods that are currently being used across the market to measure customer satisfaction.
Now, this first method is one that we all know is Likert scaling. It's an old time favorite, originally developed by Rensis Likert. He was an organizational psychologist and administrator for the U.S. Department of Agriculture.
Likert scaling here again, as you can see on the chart there, how satisfied are you with your current position? ‘Extremely dissatisfied’ on one side, ‘extremely satisfied’ on the other. This is an example. What we know is a single item balanced measure. In other words, it's bidirectional. It's a five-point satisfaction measure in this case.
Now, use of the midpoint or that neutral option is recommended unless you are looking to have the respondent give you a thumbs up or a thumbs down response. And that's typically implied in concept testing. Or if you're testing the applicability perhaps of a spokesperson, you really need a yes, no kind of answer.
The second example here is a unidirectional Likert scaling.
Now you can see on the far left, you’ve got ‘not at all likely,’ and then you have four gradations of likelihood. Again, it's not bidirectional, it's not balanced, but it is a common method for using scales, particularly in satisfaction measurement.
Now, unidirectional scaling can also be used for other dimensions such as purchase intent, credibility or importance, and certainly of course satisfaction.
Now, anytime I use the Likert scaling for measuring satisfaction, I like to try to extend its abilities here. We do that through driver analysis.
Driver analysis affords you that opportunity to understand really what's driving trust or satisfaction either overall or by key subgroups, if your sample is large enough. I typically use the Shapley value analysis as of, again, one of several statistical techniques that you can use. But it allows you to gauge the impact of a series of independent variables on a dependent variable such as satisfaction, likelihood to purchase or trust, and in a particular industry or company.
The example I have here comes out of my work in the dairy industry. You can see that again, a consumer's perception that dairy products are safe to consume is the most important driver for industry trust. And you see, I've got a few others that are quite impactful as well.
Now, when I talk to the marketers in my organization, I tell them to focus on these key areas that will actually increase trust. And why do I do that? Because we know from long time data that trust is strongly associated with both present and future product consumption.
So again, we measure trust quite frequently in the dairy industry. Driver analysis is one of the best ways to understand what's actually moving the needle.
All right, let's take a quick look at some of the pros and cons of Likert scaling.
On the left side, long history of use in marketing research and social sciences, It’s one of the easiest scales to program, certainly well understood by respondents, and again, ideally suited for developing multiple items scales.
Some of the downsides. If you are using it in a grid format, then you must pay attention to the actual number of items being tested to prevent respondent fatigue. It is very much prone to straight lining. That's why you'll commonly see a sort of a red herring line item in there to try to prevent straight lining. And again, respondent's not using all of the points of the scale. This is a particular issue if you're looking into multicultural research.
Let's take a look at the one question to rule them all otherwise known as the Net Promoter Score (NPS). Well, back in 2003, Fred Reichheld published his seminal work, “The One Number You Need to Grow” in Harvard Business Review.
If you're currently using NPS, well, you're certainly in good company. Over two thirds of Fortune 1000 companies actually use NPS as part of their measurement program. Proponents of the technique will tell you that it correlates with revenue growth relative to competitors in a specific industry.
At its core, it's a single item question that asks respondents how likely they are to recommend a company, product or service to a family member, friend or colleague.
You all have probably seen this chart before, but basically NPS is the percentage of promoters on the right minus the percentage of detractors in the second group, the passes. Well, those are the folks we actually want to focus attention on to try to move them up to scale, but that's another story.
Now, again, you hear me talk a lot about it. How do we extend these measures?
One of the ways I like to extend NPS is weaving it into what's known as the Burke’s Secure Customer Index. This is the percent of respondents you score nine or 10 across all three dimensions. These are your secure customers. What are those dimensions? Likelihood to recommend? That's your NPS question. Overall satisfaction and the likelihood of continuing to do business with.
Now these again, are all scaled using zero to 10, so there's no scaling issues, but customers that fall within that nine to 10 bracket for all three of those measures, they're secure. They're not likely to leave you anytime soon.
Now, this extension does allow you to capture NPS, but it also gives you overall satisfaction. And again, the likelihood of continuing to do business with somebody. It's ideally suited for deeper analysis. If you can look at those demographic breaks and attitudinal data that you may have collected. And if you're lucky enough to be able to append back to a customer database, then you can certainly use it as a measure to increase targeting efficiency for your direct marketing efforts.
Some of the pros and cons of NPS.
Well, it's simple. It's one and only one question.
It does facilitate an easy comparison against competitors across industries. It makes it very useful there, and even CEOs can leverage it and easily explain to the stakeholders in calls that they may have.
Now, on the downside, it doesn't give you any indication of what needs to be improved to grow the promoter percentage unless you ask targeted follow-up questions. So, if you're relying on NPS, I will implore you to use targeted follow-up questions to find out what you can do differently to improve the needle, so to speak.
It does have a strong positive bias with a negative skew. Basically that means we're potentially overstating satisfaction.
It's a single item measure, which can be less reliable over time than multi-item measures.
Our third method that we're going to take deep dive into is the American Customer Satisfaction Index (ACSI). It's a national cross industry measurement. Well, how does it work?
It's been in place for over 30 years. It was developed initially at the University of Michigan. Over 350,000 consumers are surveyed annually. It includes multiple measures across multiple industries and even the public sector. It's a multi-equipment econometric model with manifest variables on the right, overall satisfaction expectancy, disconfirmation and comparison to an ideal product of service.
Here's your three questions again. What's your overall satisfaction with our product of service? Again, this is a balanced measure, very dissatisfied, very satisfied. To what extent does it meet your expectations and how well did it compare with your ideal type of offer?
The basic model looks like this.
You've got customer expectations, perceived quality, perceived value on the left. You've got the ACSI score in the middle. That's your satisfaction measure. Then you try to tie that to outcomes, like number of customer complaints, how loyal are the customers and what is their tolerance for varying prices?
Now, some of the pros and cons of this particular measure.
Well, it's certainly a well-designed model. It does provide tracking key competitors across industries. Being an econometric model has significant diagnostic capabilities. It's correlated with not only key macroeconomic indicators such as GDP, but also with higher stock performance and customers have actually been known to use it in making their purchase because it is publicly available.
The results on the downside. The cons if you will, potential cost because you're buying into the model, you're not necessarily doing it yourself. Now, although the three primary questions are relatively easy to implement, the behind-the-scenes complexity and the proprietary nature of the model prevents easy application of it. It's subscription based, so you'd have to buy into a subscription.
Alright, I'm taking you into a new way of thinking about satisfaction. This is the best ever combined with the improvement needed scale.
In 1992, article by Peterson and Wilson was documenting the customer satisfaction measures that lean toward a strong positive bias with a negative skew. Essentially that means it's likely that the measures we've been relying on have actually overstated customer satisfaction with too many top box responses.
Now, Garver and his 2024 Quirk’s article highlights the variation seen in NPS where promoters, for example, felt the product of service they purchased was only good or average. They weren't ecstatic about it.
On the other hand, he came across passives that would certainly recommend the product to service. And then he also saw those that they would not recommend the product to service no matter how satisfied they were. So again, this leaves some gaps in the NPS model when it comes to diagnostics and actually measuring the truth of the matter.
So, what can we do to improve this?
Well, Garver’s best ever scale uses a historical best/worst comparison in addition to comparison to the average product of service.
On the far left, you can see the worst ever, one of the worst ever. And then it gets into how does it compare to average? On the right hand side, one of the best ever, the best ever, that's your best ever scale.
Following that up with the stated improvement scale. This can be applied to a single attribute, or it can be applied to all your attributes as they relate to your operation, pricing, delivery, customer service, et cetera.
Again, stated improvement scales here are reverse ordered. So, you'll notice on the left no improvement needed, which is different certainly than what we typically use in measurement here.
Now, Garver’s research did point out however that this switch, again, switching to the best item, best outcome being first did not cause any respondent error. So that's good from a diagnostic perspective.
Alright, I'm going to take a look here at a few key metrics for any customer experience scale.
The three metrics that we want to use to measure or compare are lower satisfaction mean scores, fewer top-box responses, and a more normal distribution, less negatively skewed.
Now Garver and his research compared NPS, ACSI, best ever and stated improvement. What he noted was that the best ever and improvement needed skills have lower mean scores, especially when compared to NPS and the ACSI satisfaction measure. They have reduced skewness and notably lower top-box scores, which means we're less likely to over inflate our satisfaction measures.
Following the trend here, some pros and cons.
Well, the pro side, we mitigate negative skewness and reduce top-box percentage. So, this dampens overinflated satisfaction scores. It's easy to understand the output. It certainly gives consumers an avenue to express their thoughts on meaningful areas for improvements.
Why ask what their satisfaction is if you're not going to ask them, what can we do to improve it?
This measure gets around that. It can certainly be combined with other measures such as NPS or ACSI measures. I would suggest how we're placing the new questions after any existing CX questions in order to minimize order effect bias.
Downside. Resources will be needed to bring these new questions into an existing survey you have. So, it takes time and possibly some money to update your surveys and its certainly going to take time to socialize the new measure and build up enough data for KPIs to be developed.
Alright, now I'm going to turn it over to Samir. He's going to talk you through a 10-year horizon and how we might expect customer satisfaction measurements to evolve with AI. Samir.
Samir Saluja
Thanks, Greg.
We're going to switch gears a little bit here. Building off how Greg was showing the evolution of the methodologies and different types of metrics to really how we think about measurement. I'm sure everybody's probably inundated with requests or thinking around how I'm going to implement AI into whatever CX metrics you're currently deploying.
So really just thinking about the next 10 years, how we like to think about it is just trying to understand that there is a lot of hype there and understanding what's real and what's hype, just a framework.
In the next decade, real gains will come from responsible AI. This whole idea of trust, especially in customer care and insights. So, emotion reading technology is largely hype currently. And even beyond the hype, it's increasingly illegal. I mean at least in the EU currently. It's just something to look out for.
New review laws and tougher enforcement mean we'll need better integrity practices in terms of reading what our reviews look like online for different products and services. There's governance frameworks like NIST and ISO that will help build AI responsibly. Again, really kind of snapping into this whole idea of trust.
CX measurement itself is evolving with a movement towards more continuous consented data from conversations, behaviors and reviews summarized by AI and tied to outcomes that we can test. So, thinking about some of the measurements that Greg talked about and the need to ask follow up questions. The acquisition of that data is going to be more continuous and evolving over the next 10 years.
That's a lot. That's kind of a really broad remit there.
So, how do we think about adopting these new innovations?
Adoption should be systematic. The key thing to think about is you need to start small. There might be pressure to implement AI from various constituencies, but you need to start small and you really need to measure real results against some of the proven methods that Greg had talked about already gone over metrics that our industry has been using.
When you're thinking about implementing AI, only scale those experiments that you're seeing actually work. That means keep innovating in a safe manner. Do it in an efficient and audible manner. Make sure you A/B test everything so you don't just assume that if you're implementing an AI solution and that it's working. You should A/B test and set some guardrails for proven methods especially in regulated industries. But to balance innovation and accountability, you need to A/B test everything.
So, thinking about even just the next six to 12 months, these are some practical low risk steps that we would think about.
Focus on again, where you can measure results, like what measurements do you already have in place, like faster responses or cleaner review data and deflection and detection are measurable outcomes that can directly show operational and reputational gains.
So, if people are coming into support queries online, how often are there questions being answered? How often are you detecting those sort of satisfaction issues that are popping up?
Those are things that you can measure.
Just one last, putting it into a swing thought. Obviously, if you're playing golf, you can't really think about too many things at once. But really the final swing thought is that innovation should be really evidence-based.
As I mentioned, when AI helps your metrics and aligns their governance, scale it. If it doesn't stop fast, apply the scientific method. Make sure you have proof before you go into scaling production.
In other words, only adopt innovations that show real measurable value, better customer satisfaction, faster insights that operate within transparent, accountable systems that you understand.
It sounds kind of clear, really kind of simple, but people get into a rut, especially with AI. It's important to just always go back to that when you're talking to your constituents.
And that's it. So, Greg, take it away.
Greg Timpany
Roger that. Let's wrap things up here.
Here's some final tips and takeaways, things I want you to remember.
You want to stay relevant, you want to use qualitative data or prior research to identify the attributes that customers deem most important. Please do not ask respondents to rate attributes that are not relevant to their purchase experience. It only upsets them and may cause them to leave your survey and certainly would lead them,, potentially to giving you bad data. So keep it relevant.
Don't lose sight of the past if you are using NPS, ACSI or some other measure of satisfaction. There's value in that historical data, even if you're considering adding a new measure. So, hang on to that.
You want to stay present. The NPS and the ACSI measures facilitate competitor comparisons and benchmarks. This has value and you can certainly do some of that. Take that thought about staying present and looking at satisfaction measures of your competitors. You can certainly incorporate that using just about any measure we've talked about so far.
Lastly, we want to close the loop. Now it's absolutely critical that we let our customers know that we hear them, and we'll be taking appropriate action based on their feedback.
Now, it's also vital to involve the employees in the rollout or modification of any satisfaction measurement program because they have that frontline facing relationship with the customers. They need to know what their role is. So, make sure you keep both the customers in the loop and your employees. So, everybody is interacting fairly, everybody knows what's going on.
Last but not least, we definitely want to say thank you again. My name is Greg Timpany. My partner in crime here, Samir Saluja, with DeriveOne. I want to say thank you for taking some time to listen in today and we'll be available for any questions.