NPS question after think aloud test

Question

I am planning to conduct an 1:1 think-aloud test on some wireframes in order to gather some findings for the usability and the value of a service. I would also like to add a question at the end of the test that I could use in the future to benchmark UI changes in the service.

I was thinking the NPS question "How likely is it that you recommend this service to a friend or colleague" but on the other hand I am a bit sceptical if this question is relevant on the first steps of building an app.

What I would really like to ask is something like "Would you use this app?"

Do you have any suggestions or insights regarding which is the correct question to ask for this case and setup?

Answer 1

As others said before, I wouldn't ask NPS too. Thinking Aloud are for finding problems and their solutions, but not for benchmarking.

For benchmarking you need to do summative tests - in a realistic enviroment with realistic tasks, real users and no observer in room, like in the "real (working) nature". Only in this circumstances it makes sense to ask for value of service. I recommend AttrakDiff http://attrakdiff.de/index-en.html as it asks for pragmatic and hedonic perception, is for free and has good reports. And fore sure SUS and the NPS derivate as @Tom Engh said.

Nevertheless I do always a SUS after my formative tests (thinking aloud). Because it gives me some hints about issues regarding percieved structure, complexity and confidence in use. I even take the SUS-score as an orientation for me but never tell anybody, because its not an objective benchmark

Answer 2

In my experience thinking aloud tests can tell you a lot about the usability of an application but little about the perceived value of it. You need to use other techniques (focus groups, surveys, interviews) to evaluate this quality. Therefore, I would not recommend asking this question following your thinking aloud test, as the user's experience during the test is likely to heavily color the response; if your wireframe has severe usability issues, people will say "no, I wouldn't use this" even though your system might be great with just a few adjustments.

Use thinking aloud trials for what they are meant for - identifying usability problems with an interface.

Answer 3

You can tack on any old questions onto the end of a thinking aloud session. If you've seen some issue keep coming up it gives the user a bit of time to just have a chat about it which can be quite enlightening.

However I'd be vary wary about attaching too much importance to any kind of 'scoring' at this point - the poor user is probably tired and thinking they are about to go and then they have to give some kind of evaluation, and they may not really be putting a lot of effort into it beyond a thumbs up or a thumbs down (ie a yes / no response).

Answer 4

There are other metrics like SUS score that may be more relevant. Jeff Sauro's blog on Measuring Usability covers the relative advantages and disadvantages of SUS and NPS. Specifically: http://www.measuringusability.com/blog/nps-ux.php

http://www.measuringusability.com/blog/nps-sus.php

http://www.measuringusability.com/sus.php