Skip to main content
1 of 2
Vitaly Mijiritsky
  • 31.5k
  • 13
  • 85
  • 148

That's a very good question. You want your tests to be as realistic as possible. And you say that in real life there will be some instructional overlays in place. However, they will probably be displayed only during onboarding, so the real question is how learnable these features are - will the onboarding suffice to teach them about the new features. That is something that can't really be tested using regular usability testing, because learnability is directly affected by the frequency of usage, and you'd need to imitate that for your conclusions to be applicable.

Assuming you can't imitate real usage over time, I would try to guesstimate the usage you expect. If these features are meant to be frequently used (based on actual frequent user needs), you can assume that after initial onboarding they will be learned quickly - and then test them with the overlays, to establish whether the overlays are enough to get them through onboarding, from which point it should be relatively smooth sailing.

If the features are expected to be rarely used, you can't really rely on this learning curve, and then I would test them without the instructional overlays because they'd need to be intuitive enough on their own.

Vitaly Mijiritsky
  • 31.5k
  • 13
  • 85
  • 148