1

I am working on a startup with a small team - we have just produced a prototype of a key user flow that we will be testing with live customers for maximum insight to go back and iterate.

My question: How much context/introduction should I give to users (either about the product or task)?

I have never done a live usability test and do not want to lead them, but also want to make sure they know enough to know what they are trying to accomplish, and evaluate the design issues that arise from that.

  • The answer depends on what kind of usability testing you are conducting - thinking aloud? Performance tests? – Joshua Barron Jun 23 '14 at 16:56
  • @JoshuaBarron although there are a few small tasks involved in the flow that we could gauge performance on, I would say more along the lines of thinking aloud. It is still so early in testing that isolating any particular benchmarks would be premature in comparison to more general feedback. – jvform Jun 23 '14 at 17:38
1

The only context you should ever provide, in this situation, is the context that a user would arrive with. Anything else is going to confound your results. Many times, if your testers are properly screened, you won't require any explanation at all.

If you have a client who is providing a new testing mechanism for people with diabetes, and you sit a group of Type-I and Type-II diabetics down at a screen to make sure they understand the landing page + call to action, you probably don't need to give them any context. If they miss the boat, you've made a mistake.

On the other hand, if you're working with a college website and you want to make sure that students are able to find the new classes being added for the Fall semester, which were not offered before - well then you need to give them a bit of context. You've got college students at the school as testers, and that's good, but how much context is appropriate?

This is what several types of user testing are designed to do. You literally give the tester a task:

"Find the newly added courses for your major and register for one of them" (your context is now built in)

  • Reverse Card Sort: Locate it in a tree
  • Click testing: Locate the first-click correctly on a screenshot
  • Beta testing: Observe the person as they attempt to do it on a live site

Note, I didn't need to tell them what the site was about, because they already know, having been properly screened. I might need to tell them what I want them to try to accomplish, especially on a more complex site.

As a general rule though, I want to keep my explanation as simple as possible, in order to avoid polluting results and causing Type-1 or Type-2 errors.

  • Corrected a goof in the opening paragraph – Imperative Jun 23 '14 at 19:06
  • thank you, this is very helpful - generally what I was thinking (mainly beta test) but really appreciate the detailed answer. – jvform Jun 24 '14 at 12:51
0

If you're performing thinking aloud trials, you only need to provide minimal context. This is partly to reduce the chance that you bias the user's response. Ideally you will provide them with the standard usability test disclaimers (you are testing the system, not them; their responses are confidential, etc.) and just a little bit about the task you want them to accomplish, i.e., "Go ahead and try to accomplish X."

By doing this you can really see how usable your system is. You might need to provide clarification and prompts to users who are really veering off-course, but this should be kept to a minimum.

  • thank you. I found another thread with nice some good introduction verbiage as well - appreciate the answer. – jvform Jun 24 '14 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.