January 3rd, 2010
While attending the User Friendly Conference in Shanghai, China this year, we stayed at the Hau Ting Hotel. During one of the meals there, they handed us this card to rate their service.
Now, the translation issues of Chinglish aside, this card points out problems common with measuring satisfaction.
The first has to do with the polarity. Is a 5 good or bad? If one person rates something with a 5 and the next person rates it with a 1, can you really say they have opposite opinions? Or did they just read the scale differently?
The second has to do with scale. What is the difference between a 5 and a 4 (or a 1 and a 2)? Is there meaning there? Or, as is common, does the individual rate something a 4 because they believe they “never give out 5s”? (Many seasoned statisticians will compensate by always grouping the 4s and 5s together in their reporting, which, for satisfaction is probably the right thing to do.)
The third has to do with rating terms. I have no idea what they mean by “Nattiness”. But, even under the food category, there is “Portion”, which I think I understand—until I go to rate it. Do I rate a 5 is I’m happy with the portion size? Or if I think the portion size was large? Since it was a buffet, is it the size of the portion set out? Or the amount of food I put on my plate?
And finally, the fourth has to do with followup actions. If 80% of the respondents rated “Portion” with a 3, what would the restaurant do differently?
We want to know if our customers and users are satisfied with our efforts. And, if they aren’t, we need to know what to change. Creating an instrument to give us meaningful and actionable feedback is really difficult.
I’m working on a new presentation on measuring customer satisfaction. I’m calling it: Go Ahead! Make My Day! (Dana Chisnell suggested the subtitle should be “Feeling Lucky, Punk?”) Stay tuned…Tweet