6

How do you determine the order of tasks in a usability test? Do you for example avoid putting tasks with similar paths one after another?

Improve this question
1
  • Ideally if the test group is large enough, you would split the tasks up so that different groups might perform a certain task. This can help to remove biases caused by the order in which a task is performed, but can also make it difficult to see usage patterns for individual users. You can also set up the test so that different users complete the task in different order and see if it has any obvious effects.
    – Michael Lai ♦
    Commented Mar 13, 2013 at 22:05

4 Answers 4

Reset to default
4

Put them in the order that's most realistic. It doesn't matter if repeated tasks will become easier due to parallels between them if they'll be repeated in real life. It does matter if a test reveals a piece of info that inflects a later task when the latter will usually be done first.

If there's no obvious order, you might ask your users what sorts of things they'd do after completing the current task, and see if that leads on to any obvious action.

Improve this answer
1
  • I would advise against taking the advice in this answer that has been marked as correct. Doing so will cause results to be subject to a compounding variable. It can take participants some time to warm up to what is required in usability testing, and they can also tire if the session lasts up to an hour. By randomising the order that tasks are presented, these compounding variables can be counterbalanced.
    – DigiKev
    Commented Oct 28, 2014 at 9:31
4

This is my general approach.

First I see if the things I want to test can be turned into a higher level task that touches upon the issues that we want to look at in a way that reflects 'normal' usage patterns. For example we might be interested in looking at whether people:

  1. Can register
  2. Can log in
  3. Can log out
  4. Can find the latest news page
  5. Can post a comment on a page
  6. Can edit their profile.

Might turn into:

  1. Can you post a comment on the latest news page? (requires them to register, find the latest news page, and post a comment.

  2. Can you log out

  3. Can you edit your profile (requires them to login and edit profile).

If groups of tasks are independent I'd likely randomise the order I present them to see if there are any dependency-related issues that pop out.

I'd also worry that if I have a lot of tasks with repetitions that I might be trying to get people to do things that they normally wouldn't do in a session with the system.

If your testing is set up in a way that the recruitment is an expensive task it's tempting to get a few users in for long test with many tasks. If the thing your testing is only normally used for 10m then you're then not testing a realistic scenario. It may be better to figure out ways to do more small tests with few tasks, than few long tests with many tasks.

Improve this answer
3
  • when you say randomise the order, would you randomise it for each participant or randomise one time for all participants?
    – Theman
    Commented Nov 26, 2015 at 22:31
  • 1
    For each participant. The idea is to see if doing task A makes task B more/less usable. To do that you need to have different ordering for different participants.
    – user597
    Commented Nov 28, 2015 at 15:21
  • If I wanted to gather quantitative data such as number of errors and I wanted to create statistics and graphs, would the fact that each participant not receiving the same order of tasks affect the reliability of my results?
    – Theman
    Commented Nov 28, 2015 at 20:08
0

I would suggest that you put the tasks which are most important or of interest to you first, simply because users might be most engaged and focused at the beginning, and as they get towards the end other factors (e.g. familiarity, fatigue, disinterest, etc.) sets in. But many of these factors can be mitigated by good experimental design to remove these biases.

Improve this answer
1
  • 1
    You briefly mention removing biases at the end of this post, but you begin by adding a large bias of presenting tasks that are most important to the researcher (not the user) first. You cannot be certain that users will be more engaged and focused at the beginning of a test. They may take time to warm up and understand what is required of them.
    – DigiKev
    Commented Oct 28, 2014 at 9:36
-3

Someone with more experience answer this, but I know in Steve Krug's usability tests the first thing he asks is what the site is. "Can you tell me what this website does?" or something to that effect.

Improve this answer
1
  • I think you are confusing a 5 second test with usability testing. This does not answer the question of the OP.
    – DigiKev
    Commented Oct 28, 2014 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.