Net Promoter Measures The Wrong Thing (or Why I Don’t Like United Airlines)

Jared Spool

December 12th, 2011

How likely am I to recommend United Airlines to someone else? If asked this question, I’d answer that it’s pretty likely, especially if that person lives here in the greater Boston area.

Of all the major airlines, United has the best service out of Boston. The only other options if you need to travel all over the country are American, Delta, and US Airways. Those three options deliver far worse service than United does.

This means, if I was included in a UA Net Promoter survey, I’d give them a 7 or above. That’s a good score for Net Promoter.

My score is a great demonstration of why Net Promoter doesn’t work. You see, I hate United Airlines. With a passion. As airlines go, they are really quite bad. I fly them almost every week and almost every trip, I have some experience with poor service and a bad relationship. Granted, there have been some trips where nothing bad happened, but nothing remarkably good happened either.

However, my trips with American, Delta, and US Airways are much worse. I will continue to fly United until someone better comes along, but I don’t expect that to happen any time soon. (I do like Virgin America a lot, and JetBlue or Southwest, but they don’t fly where I need them go as reliably as United, so I can’t use them often.)

I’m not recommending United Airlines because I like them. I’m recommending them because they are better than the other choices.

Net Promoter isn’t scoring my loyalty, because I’m not loyal. (I’m trapped, which is quite different.)

It’s not capturing my overall dissatisfaction with the airline. In fact, if everyone answers the survey for the same reasons I do, they look pretty good.

I think Net Promoter Score is an ineffective instrument for measuring how your customers feel about you. A better instrument is something more rigorous, like the Gallup CE11 Customer Engagement Score.

The CE11 has eleven questions, which we weight (as a Guttman Scale), including a Net Promoter-like referral question. But that referral question is weighted low, with questions like “Do you think [the brand] would take care of you if there was a problem?” or “I’m proud to be a customer of [the brand].” There are businesses that I’d score high on these other questions, but United Airlines wouldn’t be one of them.

These days, many of our clients are relying on the Net Promoter instrument (and its close brethren) to assess how they are meeting their customers needs. We warn the teams we’re working with to be careful — they may not be getting a complete picture of what’s happening and how their customers are experiencing their designs.

8 Responses to “Net Promoter Measures The Wrong Thing (or Why I Don’t Like United Airlines)”

  1. Daniel Szuc Says:

    Good reading.

    Suggest there is merit in all businesses reassessing what a customer “relationship” means, how they “value” that relationship and the way in which they “score” the health of that relationship.

    For example, one reason retail is hurting – http://www.theage.com.au/small-business/managing/blogs/the-venture/and-the-winners-for-poor-customer-service-are–20111212-1oq94.html because there has to be a compelling reason to get me in the shop in the first place, otherwise its all too easy to buy online.

    What does it mean for a business to build and nurture a relationship that is meaningful and respectful?

    rgds,
    Dan

  2. Ken Johnson Says:

    Loyalty is overrated anyway – we’re always waiting for something better to come along. The metric is still powerful because as long as you’re making recommendations, that company is ahead of the competition in important ways.

    NPS isn’t measuring how warm and fuzzy a company makes you feel. People are fickle, the competitive climate always changing – just ask Reed Hastings. But when your score begins to fall, you can start to ask the right questions and *maybe* right the ship before there’s a total disaster. It’s a simple, single question test for how well you’re doing relative to other options.

    Specifically, if United is doing just well enough to ensure their popularity, there is little motivation to innovate on their way of doing business.

  3. Darragh Says:

    A 7 on the NPS is not a particularly good score. It represents someone who is passive towards the product or service and can be poached by competitors. I think the description of your relationship with United reflects this.

    The NPS is designed to gauge how many of your customers would stay loyal to you in the short/medium future regardless of the quality (good or bad) of the competition, i.e. a 9 or 10. And the NPS does a fine job of producing this statistic.

  4. Bettina Says:

    The core question when doing a NPS survey is not the question that asks for the score you are giving but the why you are giving this score number. Did they not follow up with the why question? If that’s the case, then I agree that they missed the point. You can’t ask for a score and skip the why question all together.

    Also NPS doesn’t measure loyalty of customer but it’s leadership in something. The NPS score aims at aligning customer expectations and business goals.

    On a scale from 1 to 10 I would give a 10 for NPS surveys and I recommend NPS scores to any organization that needs to improve their overall performance and align their goals. At work we used it as part of weekly department ratings: as result it got the company humming within 8 months because everyone worked more focused.

  5. CloakerJosh Says:

    I found your article interesting, but I don’t necessarily agree wi your conclusion. I haven’t done a great deal of research on the subject, however I am familiar with Net Promoter Score system my company employs.

    Essentially, it’s as Darragh says in that a score of 7 or 8 is irrelevant. The 9s and 10s are counted up as Promoters, the 0-6s are subtracted and that’s the Net Promoter Score.

    Obviously when somebody has to pick an arbitrary score against a product or service it will be inheritly flawed, however I think it is a good tool to gain quick insight as it’s likely more people will spend the time to answer one questions as opposed to several (11 in your example). This allows for a bigger sample size, which ideally translates to a more accurate measure.

  6. Abhijit Thosar Says:

    Interesting… Enterprises need a qualitative measure to know the pulse of customers and act before it’s too late. NPS does a fine job… I have worked with some of the global banks… helping them build Customer Experience Dashboards and other multi-channel Experience matrices… Usability profession has failed to delivery something robust, real-time and reliable scale…

  7. Chris Eklund Says:

    Jared’s example actually might actually support NPS if you are comparing brands AND you collect and analyze the verbatim responses to the ‘why’ question. His 7 makes him a ‘passive’, NOT a 9/10 ‘promoter’. Passives are not much more loyal than detractors and won’t impact NPS. But, I bet if asked about Jet Blue and Southwest, that might push his score for those companies to a 9 or 10. Then relative to his United Airlines score (and verbatim) you’d see where you are falling short.

    However, I do agree that the Gallup CE11 score helps you to better understand the nuances of a customer’s CE score and what is contributing to the score given. Gallup has tied customer and employee scores to profitable companies much like NPS has. It might also give you more insights and be easier to extract actionable findings.

  8. Matt Says:

    Our company has recently taken on board NPS as a valuable metric. I agree with the article to some extent and with most of the comments here, that even a customer with a NPS response of 10 could always defect if a competitor deal is better than yours. But NPS is not the be all and end all of customer satisfaction. If your product or service is uncompetitive in the market, all your promoters will very soon become detractors based on lack of value or high prices. And no amount of NPS will fix that.

    Surveying customers and obtaining your NPS score is only one part of the equation. But let me ask you, Jared Spool, if United Airlines had ‘closed the loop’ by having someone contact you to discuss your score, would you have raised the point with them that you are only remaining loyal because you feel trapped? If you say yes, that clearly shows that ‘closing the loop’ with NPS is just as vital as obtaining the score itself, as United would then be aware they have a problem (particularly if a number of customers also feel the same way)

    Also on this, our company surveys customers using NPS periodically, and immediately after interation with our contact centre staff. We then contact them to find out as much as we can about their experience. More often than not, we reveal concerns from customers about such problems as phone wait times or silly mistakes – things that customers wouldn’t outwardly lodge a formal complaint about. This allows us to monitor NPS trends, movements and what customers are saying. This includes surveying the same customers before and after discussion, to see what difference closing the loop has made. You’d be surprised how many customers are thankful that someone has contacted them at the very least to say “Hey, we got your feedback and thank you”

    NPS is so much more than a score out of ten.

Add a Comment