2

I am performing a heuristic evaluation of mobile apps. My worry is, as a single reviewer I am not able to find more than ~35% of the problems - as Nielsen suggests.

I am reviewing mobile apps.

Does the case of simple interface mean I can find more than 35% of the problems? i.e. does system complexity mean it's harder to find usability problems?

2

System complexity in UX is analogous to code complexity. Think of each option or interaction as another conditional or fork where a user may chose one of several actions.

If that principle holds (which I would suggest it does), you can consider much of the research on code complexity and code reviews as being relevant. All of those studies that I am aware of show that as the complexity of a system increases, so does the difficulty in getting full test coverage, and consequently the number of errors tends to increase.

This is one of the main reasons behind the drive for small modular libraries that are simpler and easier to test.

0

System complexity affects fixed and variable costs of evaluation, not a single individual's ability to find all usability problems.

  • Any references to studies about my question? – thevangelist Feb 21 '13 at 11:21
  • 1
    Well there is NNG's own analysis of 83 case studies although that doesn't specifically allow you to extract information about single user reviews, but does indicate that for a range of different projects (we presume of varying size) then the ROI is statistically best at around 5 testers. – Roger Attrill Feb 21 '13 at 12:25
0

You'll find that most of the heuristic evaluation frameworks/criteria/templates are fairly comprehensive in terms of the types of design and usability issues that are summarized.

Of course, having an exhaustive checklist of things to look out for doesn't guarantee that you will necessarily pick up on them all.

So basically you are trying to make sure that:

A. You include the right things to look for when doing the evaluation AND

B. You pick up on the right things when doing the evaluation

And since people have a tendency (or bias) to look for certain things and not others, that is to say, we can be ignorant of particular things for whatever reason, having more than one person helps to address this potential issue.

I think if you can perform the evaluation, it is at least worth the effort of going to a second person, but if you have a good list and a relatively objective point of view then you are still better off doing the evaluation than worrying about what you are going to miss. That will be a caveat for reporting on your activity.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.