8

In the book, Rocket Surgery Made Easy Steve Krug recommends using 3 users (contra Nielsens 5 users) for a usability evaluation. Has this ever been researched by anybody.?

6
  • Does he not give rationale in the book itself?
    – JonW
    Mar 3 '18 at 15:41
  • 1
    He gives an opinion. I am looking into wether anybody has actually tested his hypothesis against traditional usability evaluation methods, similar to those of Jakob Nielsen.
    – fynbo
    Mar 3 '18 at 15:49
  • 2
    The whole point the book is making is that it's better to compromise on rigour in order to gain deliverability in a resource constrained team. He isn't aiming for the same goal as Jacob Nielsen, the two recommendations aren't necessarily comparable.
    – Racheet
    Mar 3 '18 at 16:00
  • 1
    Thats my point. Without any objective data to back the claim up, it remains just a subjective opinion. From what I have researched, nobody has actually carried out an empirical study to back up the claim.
    – fynbo
    Mar 3 '18 at 16:16
  • 1
    Possible duplicate of Ideal number of candidates for user-testing Mar 5 '18 at 15:05
2

enter image description here

Krug most likely recommendeds three users for the same reason that Nielsen recommendeds no more than five users: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

The law of diminishing returns kicks in noticeably at around the third user, and after the fifth user the amount of new insights tails off significantly.

The image above is from Nielsen's article.

2
  • 3
    That image is very misleading; it's presenting an (informed) estimate as data. Feb 12 '19 at 1:59
  • 1
    Neilsens original article assumes multiple iterations. It's 5 users per iteration, so 10 or 15 in total.
    – PhillipW
    Feb 13 '19 at 18:18
1

I believe he recommends 3 users because it allows for a 9am-12pm testing period once a month, with a debrief at lunch. In other words, it makes it easy to form a usability HABIT.

1
  • Thanks for giving Krug's explanation. I think fynbo is asking if a second (possibly empirical) opinion on this recommendation exists. Feb 12 '19 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.