Editor's note: Michael S. Garver is a market research consultant and professor of marketing at Central Michigan University, Mt. Pleasant, Mich.
Identifying attribute importance is a critical objective of customer experience (CX) research, and key driver analysis (KDA) is often conducted to achieve this objective. To implement KDA, researchers statistically infer the importance of product and service attributes that “drive” overall customer satisfaction. KDA results help prioritize the most important attributes to customers, which will influence strategic plans and the priority of improvement efforts. If KDA results are biased, then practitioners may invest scarce resources to strategically leverage or improve the wrong attribute.
There are a number of KDA research issues and limitations that can bias KDA results, yet many CX researchers ignore them or are simply not aware of their existence. The purpose of this article is to put forth KDA best practices to help CX researchers obtain more accurate and valid KDA results.
To accomplish this, key aspects of defining a CX model are put forth, which includes defining attributes at different levels of abstraction as well as identifying unique and relevant dependent variables for KDA. Then, recommended statistical techniques (i.e., relative weight analysis, dominance analysis and correlated components regression) are introduced that overcome the problems of multicollinearity and deliver more accurate and valid KDA results. Then, a multistep KDA approach designed to overcome the problems of attribute redundancy is introduced, along with a KDA process to incorporate multiple dependent variables. Finally, the importance of segmentation in the KDA process is addressed.
The CX model
Typically, CX research objectives are to identify attribute importance, satisfaction and improvement opportunities. To fulfill these objectives, most CX surveys start by asking survey respondents for their overall satisfaction or loyalty, followed by a section of questions asking for customer satisfaction ratings on a number of product and service attributes. Attributes are the characteristics, features or components of a product or service. Product and service attributes may include evaluations of software, technical support and training. A CX model hypothesizes that attribute satisfaction (independent variable) leads to or causes overall satisfaction (dependent variable). Properly specifying a CX model is the foundation for obtaining accurate and valid KDA results.
CX model attributes
When developing a CX model, it is critical to develop a complete and comprehensive understanding of the CX attributes, meaning that researchers identify all of the important key drivers of the customers’ experience. If important attributes are not included in the CX model, then omitted variable bias occurs, which can severely bias the KDA results. Because KDA statistical techniques typically have a relative nature to their results (i.e., importance scores), omitted attributes from KDA can bias the results.
Once identified, CX attributes need to be defined and grouped according to their level of abstraction (i.e., overall category, dimension and subdimension). At the highest level of abstraction, CX attributes would be defined at the overall category level, typically worded to portray an overall evaluation of the attribute. For example, participants rate overall “software” or “technical support.” Overall category-level attributes represent what academic researchers define as a theoretical construct. The next level of abstraction would include dimension-level attributes, where these attributes are under the umbrella of the overall category-level attribute. Dimension-level attributes represent more concrete and more specific dimensions of the overall category-level attribute. For example, software speed, software ease of use and software functionality would represent more specific dimensions of the overall category attribute, overall software. Taking this logic even further, dimension-level attributes may have subdimension attributes. For example, software functionality may have subdimension-level attributes such as printing, reporting and filtering. All of these different levels of attributes represent the construct of software and they accomplish this goal at different levels of abstraction. To obtain accurate and valid KDA results, it is critical that CX attributes are defined and grouped by their hierarchy of abstraction.
While it is important to understand the CX model from the customers’ perspective, it is also necessary to align the CX model with the company’s strategic plans and objectives. For example, strategic plans often suggest that a firm wants to have the fastest response time or the best customer service in the industry. Are these attributes aligned with the CX model? Are these attributes measured on the firm’s CX survey? Aligning CX model attributes with strategic plans and objectives will help CX research results gain more visibility and traction with top management and produce more actionable results.
Given the current survey environment, there are significant constraints surrounding the number of attributes that can be included on a CX survey. If researchers are limited in this way, then it is recommended to only use overall category-level attributes. If CX researchers can include more attributes, then we recommend using dimension-level attributes of the most complex and most important constructs in addition to the overall category-level attributes. Subdimension-level attributes are the most specific and are typically the most actionable attribute, yet subdimension attributes can greatly increase survey length.
CX model desired end-states
Selecting the most appropriate dependent variables for KDA is important because employing different dependent variables for KDA will likely result in significantly different attribute importance scores. The Net Promoter Score and the American Customer Satisfaction Index are commonly used dependent variables for KDA but are these the best dependent variables for every company?
What are the desired end-states of the customer experience? What do customers need or want to achieve from their interaction with your product or service? This can be very different for different types of customers purchasing different types of products and services. For services that are hard to evaluate, the desired end-states of customers may focus more on trust or strength of relationship as opposed to satisfaction. For products or services that are complex, the desired end-states of customers may focus more on “easy to do business with” or low customer effort as opposed to loyalty. For different types of customers, overall value might be a more appropriate end state.
The key point is that CX researchers need to select the most appropriate dependent variables for KDA that are aligned with the desired end-states of the customer experience. Too many times, CX researchers go on autopilot and select overall customer satisfaction as the dependent variable for KDA, when more meaningful dependent variables are more relevant to their customers.
When selecting dependent variables for KDA, it is critical to align the CX model with the company’s strategic plans and objectives. If practitioners have strategic goals of creating satisfied customers who are perceived as trustworthy partners that deliver an excellent overall value, then the dependent variables for KDA should also represent these goals. In this example, the dependent variables for KDA should represent the following constructs: overall customer satisfaction, trust and overall value. Given the typical complexity of the customer experience as well as strategic goals of a company, it is common that CX researchers should implement multiple dependent variables for KDA.
Implementing qualitative research to develop the CX model
Developing a sound CX model is the foundation for an accurate and valid KDA. All CX models should be grounded in reality by implementing qualitative research with target customers. By conducting in-depth interviews and focus groups, qualitative researchers can discover customers’ desired end-states as well as different paths and types of interactions customers experience to achieve their desired end-states. An important goal for qualitative research is to learn what attributes (at varying levels of abstraction) are essential to reaching the customers’ desired end-states.
How do you choose the best statistical tool for KDA? Historically, many CX researchers have used multiple regression to implement KDA, yet this statistical technique has significant research limitations when conducting KDA with CX data. Multicollinearity, or high correlation among the attributes, is the most significant limitation of multiple regression for KDA, and academic research has demonstrated that the sign, size and significance of standardized beta coefficients (i.e., importance scores) from multiple regression can be severely biased and skewed when multicollinearity is present. CX data is often plagued by high levels of multicollinearity. For this reason, multiple regression is not recommended to conduct KDA.
To effectively deal with multicollinearity for KDA, academic researchers have created a number of alternative statistical techniques, yet relative weight analysis (Johnson 2000) and dominance analysis (Budescu 1993) are recognized as two of the best KDA statistical methods. Simulation research studies have demonstrated that relative weight analysis and dominance analysis are the best KDA statistical techniques and that they produce very similar results (Zhao, Mahboobi and Bagheri 2017). In these same studies, multiple regression performed very poorly.
Correlated components regression (CCR) is a relatively new technique that can be used for KDA. While correlated components regression has not been compared to relative weight analysis and dominance analysis, this technique has certain advantages. Correlated component regression was developed to more effectively handle high levels of multicollinearity and overfit regression models, specifically with high dimensional data where the number of independent variables may be higher than the sample size. While the above statistical techniques assume that the correct attributes have been already selected, CCR excels at identifying the appropriate attributes through a cross-validation process.
In most situations, relative weight analysis or dominance analysis are recommended as the best statistical tool to conduct KDA. However, correlated components regression may be chosen in certain situations as the primary statistical tool or as a complementary technique when selecting CX attributes for the analysis. Describing dominance analysis, relative weight analysis and correlated components regression in detail is beyond the scope of this article. Instead, the purpose here is to inform readers that these statistical tools represent best practice for conducting KDA. For more information about how to use these statistical techniques for KDA, please refer to Garver and Williams (2019) for an overview of relative weight analysis; Garver and Williams (2017) for an overview of correlated components regression; and Brusco, Cradit and Brudvig (2018) for an overview of dominance analysis.
A multistep KDA process to solve attribute redundancy
In practice, it is common that CX researchers would enter all of the CX attributes (overall category, dimension and subdimension attributes) into a KDA and interpret the attribute results across the different levels of abstraction. This process will often lead to attribute redundancy in the KDA model, which can severely bias KDA results. Attribute redundancy occurs when two or more attributes represent the same construct or overall category and these attributes are entered individually into KDA.
To overcome the problem of attribute redundancy, a multistep KDA process is recommended. Implementing this process, importance scores are first acquired for the constructs or the overall category-level attributes. In the second step, importance scores are acquired for dimension-level attributes within a construct or overall category-level attribute. Following this KDA process, CX researchers will first identify which constructs are most and least important, followed by which dimension-level attributes are most and least important within the construct. If subdimension-level attributes are employed, an additional third step would be added to this KDA process.
To illustrate the multistep KDA process, a simple KDA model will be used as an example. For this KDA model, let’s assume it contains three overall constructs: software, technical support and training. The attribute of software is represented by an overall category-level attribute, software, as well as three dimension-level attributes that include software speed, software ease of use and software functionality. The constructs of technical support and training are solely represented by overall category-level attributes.
The multistep KDA process begins by running KDA at the construct or overall category attribute level of abstraction. To accomplish this task, researchers would either use overall category attributes or form a composite variable for each construct. To form a composite variable, attributes at different levels of abstraction are combined to represent the construct. For example, to form a composite variable for the construct software, the overall category attribute of software would be combined with dimension-level attributes including software speed, software ease of use and software functionality. Because technical support and training are the only attributes representing their respective constructs and they are already at the overall category level of abstraction, these attributes can be entered individually into the first step of the KDA process.
Implementing the first step of the KDA process, importance scores are acquired for the constructs or overall category-level attributes and constructs are identified as most and least important. For example, assume that the software construct (composite variable) received 60% of the total importance while technical support (overall category-level attribute) received 30% of the total importance and training (overall category-level attribute) received 10% of the total importance. Interpretations from this KDA are that software is much more important than either technical support or training. Technical support is the second most important construct and training is the least important construct.
In the second step, the researchers would then run KDA on the dimension-level attributes of a given construct versus the same dependent variable. For example, software speed, software ease of use and software functionality are three dimension-level attributes that represent the software construct. In this step, these three attributes only would then be run in the KDA against the same dependent variable. Because technical support and training attributes are at the overall category level, these constructs are not analyzed in the second step. Assume that in the second step, software ease of use received 50% of the importance, software functionality received 30% and software speed received 20%. From this analysis, researchers know that software is the most important construct (from the first step of the process) and that software ease of use is the most important software dimension-level attribute, with speed being the least important software dimension-level attribute. If subdimension-level attributes are employed, then a third step would be implemented by running KDA with subdimension-level attributes of a given dimension against the same dependent variable.
To illustrate the problem of attribute redundancy, now assume that the multistep process is not followed and that all of these attributes are entered individually into the KDA. In this situation, the problem of attribute redundancy can severely bias the KDA results. For example, the software attributes (overall category and dimension-level attributes) are redundant and would likely share 60% of the total importance obtained by the software construct. Thus, the KDA results may include the following:
- Technical support (overall category attribute) = 30% of the importance
- Software (overall category attribute) = 20% of the importance
- Software ease of use (dimension attribute) = 20% of the importance
- Software functionality (dimension attribute) = 15% of the importance
- Training (overall category attribute) = 10% of the importance
- Software speed (dimension attribute) = 5% of the importance
Interpreting these KDA results, management might perceive that technical support (30% of total importance) is the most important attribute. Software and related attributes would likely be perceived as second most important, followed by training (10% of total importance). Due to attribute redundancy in this KDA model, the above results are biased and may mislead management on what is most important to customers. To prevent this problem from occurring, the multistep KDA process is strongly recommended.
KDA with multiple dependent variables
Employing different dependent variables for KDA may result in significantly different attribute importance scores. Thus, selecting the most appropriate dependent variables for KDA is critical. Given the typical complexity of the customer experience along with a company’s strategic goals, it is likely that multiple dependent variables should be employed for KDA. What is the KDA process when using multiple dependent variables?
If researchers are implementing multiple dependent variables for KDA, it is recommended that market researchers implement KDA with a multivariate dependent variable first, which will provide researchers with a more holistic understanding of KDA results. To run KDA with a multivariate dependent variable, researchers will need to create a composite variable consisting of all the relevant dependent variables. If relative weight analysis is the chosen statistical technique for conducting KDA, then this technique is designed to be run with a multivariate dependent variable and researchers will not need to create a composite dependent variable.
Then, it is recommended that market researchers implement KDA with each relevant dependent variable so that researchers can understand the unique relationships between the attributes and each dependent variable. If three dependent variables were selected, then the researchers would run three separate KDA. To keep the results palatable for management, it is important to focus on how attribute importance scores change with different dependent variables.
KDA segments: Customers are different!
Market segmentation is an important component in developing strategic plans. The logic is simple: customers have different needs and researchers should group together customers with similar needs (i.e., segments) and then tailor their offering to meet the needs of targeted segments. Best (2013) argues that many firms fall into the demographic trap when segmenting customers, which means that they rely too much on demographic segments that may not have different needs. Researchers suggest that firms should first form need-based segments (i.e., segments based on attribute importance scores) and then describe these segments with relevant demographic variables.
This advice is critical for CX researchers. If CX researchers believe that customers may have different needs, then they should use statistical techniques designed to uncover and identify different KDA segments. To accomplish this goal, latent class regression analysis can be run as a complementary analysis with the chosen KDA statistical technique (relative weight analysis, dominance analysis or correlated components regression). Similar to other KDA statistical tools, latent class regression analysis uses independent variables to predict a dependent variable. However, latent class regression analysis creates a unique KDA model for each customer and then segments those customers who have similar KDA results. The end result of latent class regression analysis is that CX researchers can identify the number of KDA segments as well as which customers belong to each KDA segment.
Latent class regression analysis is a powerful statistical tool, yet multicollinearity is also a key research limitation and weakness. Thus, it is critical to use latent class regression analysis in conjunction with a statistical tool (i.e., relative weight analysis, dominance analysis or correlated components regression) that is designed to overcome multicollinearity. In short, latent class regression analysis is run first to identify the number of KDA segments and then to identify which customers belong to each KDA segment. Then, the chosen KDA statistical tool is used to refine the attribute importance scores for each KDA segment. While a detailed description of this process is beyond the scope of this article, the interested reader can reference Garver, Divine and Nieto (2017) for more on this topic.
More accurate results
Obtaining accurate and valid attribute importance scores is critical for strategic planning and prioritizing improvement efforts. While there are a number of research issues and limitations that can bias KDA results, following the best practices put forth in this article can help CX researchers obtain more accurate KDA results.
References
Best, R. (2013). “Market-Based Management: Strategies for Growing Customer Value and Profitability” (sixth edition) Pearson.
Brusco, M.J., Cradit, D.J., and Brudvig, S. (2018). “An integrated dominance analysis and dynamic programing approach for measuring predictor importance for customer satisfaction.” Communications” in Statistics - Theory and Methods.
Budescu, D.V. (1993). “Dominance analysis: a new approach to the problem of relative importance of predictors in multiple regression.” Psychological Bulletin, 114(3), 542-551.
Garver, M.S., Nieto, D., and Divine, R. L. (2017) “A new approach to key driver analysis for CX research.” Quirk’s Marketing Research Review (October).
Garver, M.S., and Williams, Z. (2018) “Improving the validity of theory testing in logistics research using correlated components regression.” International Journal of Logistics Research and Applications, 21:4, 363-377.
Garver, M.S., and Williams, Z. (2019). “Utilizing relative weight analysis in customer satisfaction research.” International Journal of Market Research (June).
Johnson, J.W. (2000). “A heuristic method for estimating the relative weight of predictor variables in multiple regression.” Multivariate Behavioral Research, 35:1, 1-19.
Zhao, K., Mahboobi, S., and Bagheri, S. (2018). Revenue-Based Attribution Modeling for Online Advertising. International Journal of Market Research (May).