4 Reasons Why (Most) Customer Satisfaction Surveys Are Useless

If you’re relying on traditional customer satisfaction research to track and improve your performance, it’s almost guaranteed that you will not learn enough to make meaningful changes that will impact your business and bottom line.

A careful analysis of both structured (e.g. Likert scale) satisfaction data and unstructured (i.e. open-ended text comments) data reveal a couple key points that most companies and customer experience management “experts” frequently miss.

1. Most Customer Feedback Is Not Negative

Most of us incorrectly assume that unhappy customers are proportionately far more likely than satisfied customers to give feedback. However, the opposite is true. Generally speaking, 70 to 80 percent of customers who respond are likely to be pleased with your company. When presented with a 10-point satisfaction scale or 11-point likeliness-to-recommend scale (i.e. Net Promoter Score), most customers will give either a perfect or very good rating. The remaining 20 percent of customers are either neutral or very dissatisfied.

2. Structured Scale Data is Duplicative

There is little variance in structured customer experience data, and variance is what companies should be seeking. The goal of these surveys is to better understand where to prioritize scarce resources to maximize ROI, and to use multivariate statistics to tease out more complex relationships. Yet we hardly ever tie this data to real behavior or revenue. If we did, we would probably discover that it usually does NOT predict real behavior. Why?

Consider the typical set of customer survey questions:

  • Q1. How satisfied were you with your overall experience?
  • Q2. How likely are you to recommend the company to a friend/family member?
  • Q3. How satisfied were you with the timeliness of the experience?
  • Q4. How knowledgeable were the employees?
  • Q5. How friendly were the employees?

We carefully craft 20 questions focusing on what we think are important aspects of the customer experience, but customers view surveys differently than we do. The respondent has either had a pleasant experience or a not-so-pleasant experience. In the former case her outlook will be generally positive, and will carry over to just about every structured question she is asked.

Respondent 1: Respondent 1, who had a positive experience, answers the first two or three questions with some modicum of thought, but they really ask the same thing in a slightly different way, and therefore they get very similar ratings. Very soon the questions—none of which is especially relevant to Respondent 1—dissolve into one single, increasingly boring exercise. Since Respondent 1 did have a positive experience and she is a diligent and conscientious person who usually finishes what she starts, she quickly completes the survey with minimal thought, giving you the same top 1 or 2 box scores across all questions.

3. Structured Scale Data Doesn’t Provide Insight on How to Improve

The aggregated responses from structured scale survey questions won’t tell youhow to improve. For example, a restaurant’s customer satisfaction survey may help identify a general problem area—food quality, service, value for the money, cleanliness—but the only thing that the data reveals is that more research needs to be conducted.

To improve business results, you must include an open ended question in your survey. No other variable can be used to predict actual customer behavior (and ultimately revenue) better than the free-form text response to the right open-ended question. Text comments enable customers to tell you exactly what they feel you need to hear.

Respondent 2: Respondent 2 belongs to the fewer than 10 percent of customers who had a dissatisfying experience. He basically straight lines the survey like Respondent 1 did; only he checks the lower boxes. What he really wishes he could do is just quickly tell you what irritated him and how you could improve. Instead, he is subjected to a battery of 20 largely irrelevant questions until he finally (hopefully) gets an opportunity to tell you his problem in the single text question at the end. If he gets that far and has any patience left, he’ll tell you what you need to know right there.

Unfortunately, many companies don’t do much with this last bit of crucial information, instead focusing on the aggregate structured scale survey responses, all of which Respondents 1 and 2 answered with a similar lack of thought and differentiation between the questions.

4. Companies Ask the Wrong Open-Ended Question

Many companies make one final mistake. They ask the recommended Net Promoter Score (NPS) or Overall Satisfaction (OSAT) open-ended follow-up question: “Why did you give that rating?”

When you ask the 80 percent of customers who just gave you a positive rating why they gave you that rating, you will, at best, get a short positive comment about your business. The 10 percent who slammed your company will share the problem area, which is likely something you’re already aware of. This provides little actionable insight into how to improve the business in a way that matters to customers. What you really need is information that you don’t already know.

How to Get the Most Out of Customer Satisfaction Surveys

Let’s assume your company appreciates the importance of customer experience management and you’ve invested in the latest text analytics software and sentiment tools. You’ve even shortened your survey because you recognize that the Overall Satisfaction (OSAT) and most predictive answers come from text questions and not from the structured data.

You’re all set, right? NOT QUITE.

Instead of only asking the recommended “Why did you give that rating?”, ask a follow-up probe question like, “What, if anything, could we do better?”

This can be analyzed separately and then combined with the original open-ended question. When analyzed together, this one-two question combination provides a far more complete picture to the question about how customers view your company and how you can improve.

Regardless of the approach your company takes to customer experience management, the right combination of structured and unstructured data will provide the most comprehensive and actionable insight into the customer experience.


If you enjoyed this post, please share it with your network. Connect with us on Twitter @TopRightPartner.


Photo Credit: Survey Legend

Leave a Comment