Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I'm just wondering how high of a success rate does the outcome of a user-testing have to be before you can consider it successful. I know that 100% is the target but if time and budget are limited, what's the minimum acceptable score? Given that 100 is the highest possible score.

share|improve this question
5  
There are way to many unknown variables here to just throw out a specific number. The point of a usability test isn't necessarily pass/fail the product but rather to find areas of the product to improve. –  DA01 Jun 15 '10 at 2:56

3 Answers 3

How are you even scoring "success"? The purpose of user testing is to reveal weak points in a design, not to get a high score. Reducing the results to a number makes the entire process useless; why not just put it up on Mechanical Turk and see what comes of it?

In fact, you could argue if you get a good 'score', you are wasting everyone's time. It means the testing was done much too late in the process to be useful, or was unnecessary at that stage.

IMHO the most useful usability tests are the ones where users do get through all the tasks, but reveal many previously unknown things about your system along the way.

share|improve this answer
1  
I believe Nick Fine understood where I was coming from with this. It's more of finding out if you were able to do your job well after the project is completed. –  Gabriel Reinoso Jun 16 '10 at 1:38

It is a teeny weeny bit analogous to service level agreements (SLAs), or disaster recovery...."how much do I spend"....and the answer, with regards to UX matters, is "what's the cost to your organisation of failure".

If your application is mission critical (e.g. nuclear power station control panel) or a medical device (e.g. X-ray machine) then the cost of failure is massive and the pass levels need to be nigh on 100% for all users to prevent lethal mistakes.

If your application is eCommerce related, I'd put the success rate up at 90% or somewhere reasonable around there. You need your users to be able to achieve organisational goals (e.g. buy stuff, meet targets/plans etc) but also need to recognise that the audience will be significantly different from a medical or military application.

Much less than 90% and you have to question whether your interaction design is appropriate....imho :)

N

share|improve this answer

Why do you want to test?

Typically, usability analysts test a product's usability to see if the mental model works, if the workflow works, if the interaction (controls) are discoverable, if a task can be completed within a certain time, how many participants succeed at a given task (which is what you're after, I think), and so on.

The answer, as usual, is "It depends."

  • How critical is the task? (What's at stake if the user fails?)
  • How fast must the task be completed correctly to be deemed a success?
  • If the user fails, will they recognize the failure or will they believe the task succeeded?
  • If the user can recover from a failure (an error state), is that success with a longer time on task, or a failure?
  • And so on.

It's not unreasonable to expect a very high success rate. But it's also not uncommon to get lower success rates, if the GUI or design needs to be fixed.

In case you're new to usability testing, Gabriel, I'll add one tip. Make sure you give your participants realistic scenarios that don't lead the participants toward the desired outcome. For example, don't ask the users to "click the Save button," but instead ask them to "make sure you can use this document again, when you next use this program."

share|improve this answer
    
Thanks for the insightful comments and tips! –  Gabriel Reinoso Jun 15 '10 at 5:21

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.