Alternative questions/frameworks to the Net Promoter Score or better metrics of User Experience

Question

There are some very detailed discussions about the validity of the NPS. But the take home message is that you can't really improve what you can't measure effectively, so it is still important to come up with metrics (preferably from a number of different sources).

One of the key recommendations is to change the question from whether you would recommend the product to something else. I was wondering whether anyone has tried this and what the result was?

UPDATE: Since I first asked this question many years ago, there have been more recent articles talking about the gaps in the NPS score and how other metrics can be used to fill in this like the Customer Effort Score and how it compares to the NPS score.

Are there more recent studies or research to support a more holistic measure of customer experience? Are there alternative frameworks for measuring user experience out there not covered in this list?

Currently the list of key metrics include:

• CSAT: Customer Satisfaction Score
• NPS: Net Promoter Score
• SUPR-Q: Standardized User Experience Percentile Rank Questionnaire
• SUS: System Usability Scale
• TPI: Task Performance Indicator
• HEART Framework: a number of metrics combined - apparently used by Google

NEW: Just read about the USER framework by VMWare, which addressed some of the commonly asked questions about Google's HEART framework

Answer 1

This is an important question and the answer/solution could be different for different sectors, e.g., retailing, tourism services, and government services.

The original Net Promoter website suggests -

Complement NPS with other metrics and insights from various points along the customer journey, and you have a comprehensive, actionable view of your customer experience performance.

The question is - what metrics/tools/survey to use in a specific context for a comprehensive assessment of CX when NPS is low?

As part of a team, I am working on a research project for developing Customer Experience measurement tools (surveys) for omnichannel retailers. Our recent study has been published in the highly-ranked Journal of Retailing in early 2022, and it offers the Omnichannel Customer Experience (OCX) scale for comprehensively measuring CX in the omnichannel retailing context.

Conveniently, this journal article is open-access and it can be accessed freely @ https://doi.org/10.1016/j.jretai.2022.03.003

The OCX scale captures CX across nine distinct dimensions: social communications, value, personalization, customer service, consistency of both product availability and prices across channels, information safety, delivery, product returns, and loyalty programs.

When the comprehensive OCX survey complements an NPS survey, a customer-centric omnichannel retailer can identify which dimensions out of the nine key CX dimensions needs improvements.

The study also empirically validates positive relationships of OCX with managerially important outcomes, such as satisfaction with the retailer, omnichannel customers’ loyalty intentions, word-of-mouth intentions, share of wallet, and trust in the omnichannel retailer in a consumer goods setting. As such, when OCX survey is used to complement NPS, retailers could predict additional outcomes, such as trust and share of wallet, which are not captured by NPS. The survey questions for the outcome variables are available in the paper.

Answer 2

What you need to ask really depends upon what you want to know about the product.

This may not be what you're looking for but the SUS (System Usability Scale) might be a useful alternative.

The NPS is a very easy scale to administer which is an advantage over the SUS's 10 questions but this means it is a more subtle measure because it measures more than one thing. Commonly, an overall SUS scale is reported – in my experience, it's easily understood, but can miss some of the subtleties.

If you need to stick to a single-item scale, then one of the SUS questions might be an alternative. Question 9 ("I felt very confident using the service") seems useful.

1. I think that I would like to use this service frequently
2. I found the service unnecessarily complex
3. I thought the service was easy to use
4. I think that I would need the support of a technical person to be able to use this service
5. I found the various functions in the service were well integrated
6. I thought there was too much inconsistency with this service
7. I would imagine that most people would learn to use this service very quickly
8. I found the service very cumbersome to use
9. I felt very confident using the service
10. I needed to learn a lot of things before I could get going with this service

Answer 3

Hope this helps - https://measuringu.com/business-software-ux2020/ https://measuringu.com/ux-consumer-software-2020/

The study shows 30 and 60% of the variation in Net Promoter Scores is determined by a product's usability. The UMUX-Lite also correlated highly with the SUS at r = .83 and the Net Promoter Score at r = .72.

Answer 4

I was wondering about NPS quite a bit myself but never enough to research more in depth. This answer is to share what I learned about NPS.

There is research on the effectiveness of the classic 10-point-NPS question and how it could be improved.

A summary of a research paper on MeasuringU describes research and experiments especially with the scale (10-point vs. more or less and labels for the points).

It's probably best to read the text behind the link yourself, so just quoting the researchers' conclusion here:

We did find that reducing the number of scale points to 7-points generally improved the validity of the measurement. However, contrary to our expectations, assigning full-labels did not improve the validity, it rather produced weaker relationships between the scales and the validity criteria. … Our results show that different measures such as likelihood of recommendation, satisfaction and liking are interrelated and might be acting within causal chains.

The takeaway of the MeasuringU author is:

Similar to other studies, they found that the NPS and multiple metrics correlated with historical growth rates and changing the number of scale points and labels improved correlations, but not by much.

Some time ago I read (or heard) about problems with the typical NPS question ("How likely are you to recommend our service to a friend") and they resonated well with me. One problem is the point in time when this questions gets send out, typically by email. Directly after an interaction? If not, how long after. Does the customers still remember? Another problem is the question itself. It is asking about a potential future event (how likely are you) based on an interaction in the past. A better question to ask would be "Did you recommend our service to a friend" or "How often did you recommend our service to a friend", as this is easier to answer for the customer. (Unfortunately I can't find the source anymore)