2

I am conducting paper-prototype testing and manage to collect data about completion rates and satisfaction scores. However, I am finding it difficult to calculate the task times. Is there another metric apart from response time I can possibly use to evaluate paper prototypes?

Improve this question
3
  • Are you prototyping a computer interface? Can you elaborate on the tasks being completed?
    – Andy
    Nov 24, 2015 at 18:02
  • @Andy I am prototyping a tablet interface which will include various tasks such as changing from one language to another, selecting items from a list etc..
    – user1995
    Nov 26, 2015 at 7:04
  • Build a prototype on CogTool. This will give you task completion time estimates.
    – user1757436
    Dec 16, 2015 at 14:34

4 Answers 4

Reset to default
1

I like earlier responses, but in addition, I would point out that if users were not aware that time was an issue, you will have so much variability in your data that unless the effect is huge, it will not be significant.

In the future, you can use systems like Verify (verify.com) allow you to quantify completion rates and task times on individual screen mockups.

When it comes to additional quantifications, I would definitely do subjective rating, meaning this is important to the user. You could do it in absolute, or comparative to what they would otherwise use. You start by defining what are the important attributes of the experience (speed, effectiveness, trust, enjoyment, etc.) and ask people to rate the experience for those top 3-4 attributes.

Improve this answer
1

Sounds like you're trying to gather quantitative data from what's essentially a qualitative research method. I've only ever gathered usability information from usability tests. I'd expect low-fi prototypes to give even less-precise quantitative data than a finished system.

But you can discover some very useful qualitative data with low-fi prototypes. Mainly, you can discover problem areas and use your observations to improve your early designs. And, since it's a paper prototype, which allows you to easily create multiple versions, you can test them against each other.

If you're really interested in completion times, try a KLM-GOMS model, which provides exactly that, with surprising accuracy.

Improve this answer
0

I would ask respondents to estimate ease of completion for task (such as 1-very difficult -> 7-very easy) or ask them to map how easy they'd think it would be to do within a real app, if that's part of your goal.
Maybe like you, I've found too many potential distractors with paper-prototypes to make stopwatch time useful.

Improve this answer
0

You could count:

  • Number of times user has to ask for clarification, grouped by task
  • Number of times user has to ask for clarification, grouped by element
  • Number of times user "presses" or interacts with an element, only to realize it doesn't do what expected
  • Count the number of times users accomplish task X on the first attempt.

Asking users for their feedback on how hard something is can be useful, but users are not especially accurate in what they say, which is why data collected by observing them can be more useful. [1]

Sources:

[1] http://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.