2

I conducted a usability test where a user is given a set of tasks and we score them based on task completion (completed with ease, completed with minor difficulty, completed with major difficulty, and did not complete task).

However, recently I have encountered a problem where in some cases users never complete the task, so I have been marking them as incomplete. This can be due to running out of time so they never even were presented the task or they were taking so long that I had to move on from the task. In the case where they are taking long, one can argue that it should be marked as a fail since they failed to complete the task however another might argue that if given more time they would have figured it out and that we are testing the material and not the person.

A reason this is becoming an issue is because when I report out data it can be viewed not representative of the actual results. For example if we had a total of 8 users and for Task 1, 6/8 users passed but for Task 6, 4/6 users passed because 2 users never completed the task (due to having to move on but they had difficulty with it) how should I treat this data? What are the best practices for this scenario.

Improve this question
1
  • It depends on how you want to differentiate between 'incomplete' and 'fail'. You could argue that not being able to complete the task (for whatever reason) is no different to failing a task, if for example it was a time critical task and the userflow or interface is designed so that it took too long or was too confusing.
    – Michael Lai ♦
    Commented Jan 4, 2020 at 22:35

1 Answer 1

Reset to default
1

I would mark them as "Incomplete" and write down the reason and details for not being able to complete the task. I also do this for tasks where the facilitator had to step in and help the participant complete the task and move on the the next one. These incomplete cases can provide valuable insight.

**edit You'll have to define "Incomplete" in your study, so you can distinguish between an "Incomplete" and a "Failure".

Improve this answer
1
  • Exactly. In user testing you state the specific outcomes or behaviors and if you notice patterns where participants encounter issues, you create discussion points. You never "argue" about what has happened unless the user has explained what happened in an open question after the task.
    – Zoe Marmara
    Commented Feb 4, 2020 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.