In the book, Rocket Surgery Made Easy Steve Krug recommends using 3 users (contra Nielsens 5 users) for a usability evaluation. Has this ever been researched by anybody.?

  • Does he not give rationale in the book itself? – JonW Mar 3 '18 at 15:41
  • 1
    He gives an opinion. I am looking into wether anybody has actually tested his hypothesis against traditional usability evaluation methods, similar to those of Jakob Nielsen. – fynbo Mar 3 '18 at 15:49
  • 2
    The whole point the book is making is that it's better to compromise on rigour in order to gain deliverability in a resource constrained team. He isn't aiming for the same goal as Jacob Nielsen, the two recommendations aren't necessarily comparable. – Racheet Mar 3 '18 at 16:00
  • 1
    Thats my point. Without any objective data to back the claim up, it remains just a subjective opinion. From what I have researched, nobody has actually carried out an empirical study to back up the claim. – fynbo Mar 3 '18 at 16:16
  • 1
    Possible duplicate of Ideal number of candidates for user-testing – locationunknown Mar 5 '18 at 15:05

enter image description here

Krug most likely recommendeds three users for the same reason that Nielsen recommendeds no more than five users: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

The law of diminishing returns kicks in noticeably at around the third user, and after the fifth user the amount of new insights tails off significantly.

The image above is from Nielsen's article.

  • 3
    That image is very misleading; it's presenting an (informed) estimate as data. – Robert Fraser Feb 12 '19 at 1:59
  • Neilsens original article assumes multiple iterations. It's 5 users per iteration, so 10 or 15 in total. – PhillipW Feb 13 '19 at 18:18

I believe he recommends 3 users because it allows for a 9am-12pm testing period once a month, with a debrief at lunch. In other words, it makes it easy to form a usability HABIT.

  • Thanks for giving Krug's explanation. I think fynbo is asking if a second (possibly empirical) opinion on this recommendation exists. – Andre Dickson Feb 12 '19 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.