5

There are some very detailed discussions about the validity of the NPS. But the take home message is that you can't really improve what you can't measure effectively, so it is still important to come up with metrics (preferably from a number of different sources).

One of the key recommendations is to change the question from whether you would recommend the product to something else. I was wondering whether anyone has tried this and what the result was?

UPDATE: Since I first asked this question many years ago, there have been more recent articles talking about the gaps in the NPS score and how other metrics can be used to fill in this like the Customer Effort Score and how it compares to the NPS score.

Are there more recent studies or research to support a more holistic measure of customer experience?

Currently the list of key metrics include:

  • 1
    That's an interesting question and I have wondered too if the wording is wrong for certain types of sales, for example technology that gives a competitive advantage. Why would you recommend this to your competitors? You wouldn't! If I was putting this together for our clients, I might try asking "If you had to make this purchase decision today, knowing what you know now, how likely would you go with Product X?" But I haven't done this, so that's why this is a comment and not an answer. – LindaCamillo May 27 '14 at 14:12
  • My personal beef with a question like "Would you recommend us to a friend" is it's almost never clear whether they are talking about spontaneous recommendation, or in reply to being asked. I routinely (before Covid-19) visited a major UK high-street shop for my lunch and frequently filled-in their "Win an iPad" surveys where they asked just such a question. I almost always answered "very unlikely" because I would never, out of the blue, say to a friend "You must go to XXXX... they're an amazing shop." – TripeHound Jun 15 at 14:11
  • @TripleHound my understanding with the NPS questionnaire is that it should normally be applied after a specific interaction with the user to provide the context that they are looking for the feedback from the users. – Michael Lai Jun 15 at 22:53
  • +1 for a very interesting question – greenforest Jun 21 at 21:35
  • @greenforest thanks, but it would have been nice to see some more answers :p – Michael Lai Jun 22 at 0:16
4

What you need to ask really depends upon what you want to know about the product.

This may not be what you're looking for but the SUS (System Usability Scale) might be a useful alternative.

The NPS is a very easy scale to administer which is an advantage over the SUS's 10 questions but this means it is a more subtle measure because it measures more than one thing. Commonly, an overall SUS scale is reported – in my experience, it's easily understood, but can miss some of the subtleties.

If you need to stick to a single-item scale, then one of the SUS questions might be an alternative. Question 9 ("I felt very confident using the service") seems useful.

  1. I think that I would like to use this service frequently
  2. I found the service unnecessarily complex
  3. I thought the service was easy to use
  4. I think that I would need the support of a technical person to be able to use this service
  5. I found the various functions in the service were well integrated
  6. I thought there was too much inconsistency with this service
  7. I would imagine that most people would learn to use this service very quickly
  8. I found the service very cumbersome to use
  9. I felt very confident using the service
  10. I needed to learn a lot of things before I could get going with this service
| improve this answer | |
1

Hope this helps - https://measuringu.com/business-software-ux2020/ https://measuringu.com/ux-consumer-software-2020/

The study shows 30 and 60% of the variation in Net Promoter Scores is determined by a product's usability. The UMUX-Lite also correlated highly with the SUS at r = .83 and the Net Promoter Score at r = .72.

| improve this answer | |
  • +1 It is good that you have provided the reference for a recent article, but the measures that have been compared for the software products evaluated were all around at the time that the question was first asked. The UMUX was first proposed in 2010 and UMUX-Lite is a cut-down version so it doesn't seem like there are any new metrics that is common or accepted. – Michael Lai Jun 21 at 4:11
1
+50

I was wondering about NPS quite a bit myself but never enough to research more in depth. This answer is to share what I learned about NPS.

There is research on the effectiveness of the classic 10-point-NPS question and how it could be improved.

A summary of a research paper on MeasuringU describes research and experiments especially with the scale (10-point vs. more or less and labels for the points).

It's probably best to read the text behind the link yourself, so just quoting the researchers' conclusion here:

We did find that reducing the number of scale points to 7-points generally improved the validity of the measurement. However, contrary to our expectations, assigning full-labels did not improve the validity, it rather produced weaker relationships between the scales and the validity criteria. … Our results show that different measures such as likelihood of recommendation, satisfaction and liking are interrelated and might be acting within causal chains.

The takeaway of the MeasuringU author is:

Similar to other studies, they found that the NPS and multiple metrics correlated with historical growth rates and changing the number of scale points and labels improved correlations, but not by much.

Some time ago I read (or heard) about problems with the typical NPS question ("How likely are you to recommend our service to a friend") and they resonated well with me. One problem is the point in time when this questions gets send out, typically by email. Directly after an interaction? If not, how long after. Does the customers still remember? Another problem is the question itself. It is asking about a potential future event (how likely are you) based on an interaction in the past. A better question to ask would be "Did you recommend our service to a friend" or "How often did you recommend our service to a friend", as this is easier to answer for the customer. (Unfortunately I can't find the source anymore)

| improve this answer | |
  • +1 nice summary of the NPS and issues. Did you find alternatives or a combination of UX metrics that work for your projects? – Michael Lai Jun 22 at 0:24
  • @MichaelLai I looked into the Google HEART framework: research.google/pubs/pub36299 It looks indeed very interesting and addresses the key aspects from you original question, but I haven't tested it in the field (yet). – greenforest Jun 22 at 7:54
  • 1
    @MichaelLai Looking at your original question again I only see now that HEART and other metrics were already listed. For your comment question: I found NPS combined with CSAT to be effective for measuring and communicating customer satisfaction and customer problems in a larger organisation, which helps the organisation to become more customer centric. – greenforest Jun 22 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.