8

In the book, Rocket Surgery Made Easy Steve Krug recommends using 3 users (contra Nielsens 5 users) for a usability evaluation. Has this ever been researched by anybody.?

Improve this question
6
  • Does he not give rationale in the book itself?
    – JonW ♦
    Commented Mar 3, 2018 at 15:41
  • 1
    He gives an opinion. I am looking into wether anybody has actually tested his hypothesis against traditional usability evaluation methods, similar to those of Jakob Nielsen.
    – fynbo
    Commented Mar 3, 2018 at 15:49
  • 2
    The whole point the book is making is that it's better to compromise on rigour in order to gain deliverability in a resource constrained team. He isn't aiming for the same goal as Jacob Nielsen, the two recommendations aren't necessarily comparable.
    – Racheet
    Commented Mar 3, 2018 at 16:00
  • 1
    Thats my point. Without any objective data to back the claim up, it remains just a subjective opinion. From what I have researched, nobody has actually carried out an empirical study to back up the claim.
    – fynbo
    Commented Mar 3, 2018 at 16:16
  • 1
    Possible duplicate of Ideal number of candidates for user-testing
    – locationunknown
    Commented Mar 5, 2018 at 15:05

2 Answers 2

Reset to default
2

enter image description here

Krug most likely recommendeds three users for the same reason that Nielsen recommendeds no more than five users: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

The law of diminishing returns kicks in noticeably at around the third user, and after the fifth user the amount of new insights tails off significantly.

The image above is from Nielsen's article.

Improve this answer
2
  • 3
    That image is very misleading; it's presenting an (informed) estimate as data.
    – Robert Fraser
    Commented Feb 12, 2019 at 1:59
  • 1
    Neilsens original article assumes multiple iterations. It's 5 users per iteration, so 10 or 15 in total.
    – PhillipW
    Commented Feb 13, 2019 at 18:18
1

I believe he recommends 3 users because it allows for a 9am-12pm testing period once a month, with a debrief at lunch. In other words, it makes it easy to form a usability HABIT.

Improve this answer
1
  • Thanks for giving Krug's explanation. I think fynbo is asking if a second (possibly empirical) opinion on this recommendation exists.
    – Andre Dickson
    Commented Feb 12, 2019 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.