An earlier version of this article was published in the Feb-April 2007 issue of our Leisure eNewsletter.

Download a PDF copy of this article by clicking here.

The Silliness of Most Customer Surveys

by Randy White, CEO
White Hutchinson Leisure & Learning Group

We’re sure you’ve experienced them. You frequent a business and are then asked to fill out a customer satisfaction survey. Now, even when you visit business websites to make a purchase or reservation, you often get a follow-up e-mail asking you to take a survey. All the surveys have a whole list of questions where they ask you to rate things from 1 to 10 or 1 to 7 or on some other scale. Most people find these surveys too long or figure it is a waste of time, because no one pays attention to the responses and nothing will change. That’s why only 10% of the customer satisfaction surveys are answered.

As an example, I stayed at a J.W. Marriott hotel in Hong Kong and received a two-page survey. It had 38 line items to be rated and only had one open-ended question asking to describe any problems I encountered (see later discussion on open-ended questions). Guess what? I didn’t fill it out. How would rating my room overall make it any better the next trip? Beats me, and that is exactly why I didn’t take what I consider as my valuable time to complete such a long survey that probably will not change things. Heck, for two years now I have been commenting in Marriott e-mail surveys about how the new desk chairs in Marriott Courtyards are too low for the desks. Think anything has changed in those two years? No. Have I received an acknowledgment from Marriott? No. If you want to see that silly Marriott survey, click here (P.S., we’re not trying to pick on Marriott. It’s just thathe first silly survey we ran across when writing this article was from that corporation, so it became the example.) Marriott did have a much better survey you received when you signed the check in one of its hotel restaurants. It was short enough to answer quickly and had a good open-ended question. Click here to see it.

There must be a lot of bean counters tracking all those surveys and ratings. The problem is, following customer satisfaction ratings doesn’t help you better please your customers or grow the business. In fact, the ratings can be downright misleading. See, if you are dealing with regular customers, it’s highly probable that the ratings will decrease over time. The reason is simple. The first time customers visit a business, it’s a new experience, so they more likely will be more impressed than after several visits, when the experience becomes old hat. What might have first exceeded their expectations becomes their expectations, so ratings decline. And over time, customers become less impressed with what they experience, and in fact, they probably start noticing imperfections. So, a business thinks it is getting worse, when that’s probably not the case. 

Surveys need to accomplish two important things:

  • Let a company know how well it is doing in creating loyal repeat customers that spread positive word-of-mouth and help grow a the business
  • Give a company insight on how it can improve.

Frederick F. Reichheld, Director Emeritus of the consulting firm Bain & Company, has been researching the correlation between customer loyalty and company profitability and growth and the most effective way to survey customer loyalty since the early 2000s. His work was first published in Harvard Business Review in the article The One Number You Need to Grow and more recently in his 2006 book, The Ultimate Question: Driving Good Profits and True Growth.

After administering tests to thousands of customers in different industries and examining their purchase industry, Reichheld determined there was one single question that could determine both customer loyalty and predict a company’s economic success and growth. It was this:

How likely is it you would recommend [company, product or service “XYZ”] to a friend, colleague or family member?”

A scale of 0 to 10 was offered, with 10 being highly likely.  Then he ranked responses in three categories of loyalty:

  • Promoters (scores of 9 and 10) – These people love the product, service or experience, invest in the relationship and generate 90% of positive word-of-mouth.
  • Passively Satisfied (scores 7 and 8) – These people have nothing negative to say, but if there was a better deal that came along from someone else, they’d defect in a minute.
  • Detractors (scores 0-6) – These are irritated and angry customers, the ones who complain and spread negative word-of-mouth.

Finally, he computed the Net Promoter Score (NPS) by subtracting the percentage of customers who were Detractors from the percentage who were Promoters.

Research by Reichheld, Bain and Satmetrix Systems shows that companies with high Net Promoter Scores are consistent with the most trusted and successful companies in the world. Companies with the highest NPS include USAA (82%), Harley-Davidson (81%), Costco (79%), Chick-fil-A (78%), Amazon.com (73%), eBay (71%), Vanguard (70%), SAS (66%) and Apple (66%). The average company sputters along at an NPS of only 5% to 10%. And many have negative scores, meaning they have more Detractors than Promoters.

The research shows that NPS leaders outgrow their competitors in most industries by an average of 2.5 times.

Think Net Promoter Scores don’t really sort out the successful growth companies from the others? Then just take a look at the scores for some of the airlines vs. their five-year growth, where Southwest Airlines enjoys the highest NPS score of 51% and the highest growth rate and profitability in the industry.

The other part of the equation of meaningful customers surveys is to get input on how you can improve your business based on the experience your customers are having (often far different than the experience you think you are giving them). This can be accomplished through a few simple questions you add to a survey for the NPS score. The key is to ask open-ended questions, instead of questions about rating different aspects of the customer’s experience. Open-ended questions are quite simple, such as;

  • If you were the manager of our business, what three things would you change?
  • How could we have made your visit with us better today?
  • What did you like least about your visit today?

One open-ended question about suggested improvements and one about the negatives of the visit is all it takes. That’s it.

A very insightful customer survey can have only three questions: one rating the likelihood of recommending your business to colleagues, friends or family to determine your NPS, and then two open-ended questions. You also might want to ask customers for their ZIP Code and how often they frequent the business so you’ll know where they come from and whether they are new or regular customers.

We know of one LBE that made it a practice to conduct a short exit survey with five customers every day to get guest input on their experience, and then the business immediately acted on any negatives. For example, if in probing about what could be better, a guest noticed the paint peeling on a wall, that night (yes, that night) the wall would be repainted. Surveys are meaningless if they are not immediately acted on. It doesn’t do any good to have a customer give you input on something that is negative about the business or might improve it if that suggestion just sits on a piece of paper for weeks or months. Meanwhile, thousands of guests can be having the same negative experience.

You don’t need formal surveys to get guest feedback. Every employee who interacts with guests can be an important source of information. Guests, especially frustrated ones who could defect and spread negative word-of-mouth, will often make comments to employees, usually those lowest down the chain-of-command. Create a culture where employees understand the importance of guest reconnaissance. Train staff not only to listen for those comments, but also to document them and immediately send them up the chain of command (better yet, empower employees to make the needed change).

Yes, some surveys are just silly and a waste of both customers’ and managements’ time and effort. A well-crafted survey will let you know where you stand in terms of customer loyalty and future business profitability and growth. More importantly, t will also tell you what you need to do to get better.