Editor's note: Carolyn Lindquist is product manager with Service Management Group, a Kansas City, Mo., research firm.
Written customer comments are a rich source of information about their experience with a brand. Text analytics provides the ability to leverage these comments to better identify aspects of the customer experience that drive satisfaction and loyalty. When customers provide feedback that needs immediate attention, text analytics can also be used to alert field and corporate managers. To achieve these results, our firm, Service Management Group (SMG), developed a process for soliciting text comments and analyzing the results for meaningful insights.
Text analytics can be very powerful with the right industry-specific settings; however, that power cannot be harnessed if customers do not leave rich comments. Looking at current processes, we discovered an opportunity to increase both the number of survey respondents that leave a comment and the quality of the comments themselves by improving the approach to open-ended questions. These new questions facilitate more detailed, emotion-filled responses while placing fewer burdens on survey respondents. After testing many ways to ask open-ended questions, three key elements emerged that improve the enriched open-ended feedback from customers.
1. Target your questions. As advocates for the service-profit chain, SMG’s goal is to move satisfied customers to highly satisfied. Unfortunately, in many cases satisfied customers were not providing enough information on how their experience could be improved. To elicit these details, open-ended questions need to be designed specifically to return actionable insights from this group of customers. Upon implementation of questions targeted to this group of satisfied customers, response rates increased by 12 percent and customers left richer, more detailed comments. Therefore, SMG recommends aligning different open-ended questions with customers based on their stated levels of overall satisfaction.
2. Ask why. Traditionally, survey questions have asked customers “What could we do to improve?” or “What did you like most?” To get longer, more sentiment-filled comments, ask customers why they felt a particular way about their experience. The tendency for “why” questions to be richer than “what” questions is evident through average comment length (see Figure 1).
For example, when asked what to improve, customers often provide comments such as “Staff friendliness.” These brief, less-thoughtful statements lack emotion and detail. However, when asked why they were not satisfied, customers tend to provide more elaborate responses such as “The staff was rude to me after I asked for their help,” which contains emotion and detail we can target with actions. For this reason, SMG recommends asking “why” versus “what” when designing open-ended questions (see table below for examples).
3. Be sensitive to placement. Typically, SMG asks open-ended questions near the end of the survey. This means respondents are first asked a series of closed-ended questions (rated using a scale) before they are offered a chance to provide written detail through open-ended comments. SMG theorized that placing open-ended comments at the beginning of the survey would lead to higher response rates because survey respondents may be less fatigued at this point. Our research actually showed a marked decrease in overall response rate when open-ended comments were moved from the end of the survey to the beginning. When beginning a survey with an open-ended question, respondents assume the survey will demand a longer time commitment. This leads the respondent to opt out of the survey. Equally notable was a 3.8 percentage point increase in highly satisfied responses across the closed-ended items (e.g., overall satisfaction, friendliness of staff or overall cleanliness) as shown in Figure 2. A change in survey scores is acceptable if it means the new scores are more valid (i.e., more reflective of the customer experience) but this was not the case. 
For example, when open-ended questions are placed at the end of the survey, customers write about their poor perceptions of staff friendliness as part of their open-ended comments and indicate dissatisfaction in their closed-ended comments. Conversely, if customers write about their perceptions of poor staff friendliness as part of an open-ended comment placed at the beginning of a survey, they often rate staff friendliness as satisfactory later in the survey. This illustrates that respondents providing written feedback early in the survey tend to pay less attention to survey content appearing later in the survey. Simply stated, once people write about their experiences, they feel that they have offered everything needed and quickly disengage from the survey process. This demonstrates the great importance of placing open-ended items at the end of the survey.
Lead to an improvement
In our experience, the data is best when adhering to three techniques: targeting the question to the customer’s satisfaction level; asking “why” instead of “what;” and placing the open-ended question near the end of the survey. Following these design principles for all open-ended questions will lead to an improvement in the quantity and quality of your customer feedback.