I would like to do a discount usability test. What are the steps I should take to come to an insightful conclusion?
- How do I recruit the people?
- How do I setup the test?
- How do I define the tasks to be tested?
- How long should the test be?
-
What is a "discount usability test"? Do you mean you want to know how to do a usability test as cheaply as possible?– RahulCommented Aug 12, 2011 at 12:28
-
I mean those without a lab. The cheap ones. How do I recruit people? How do I set up the test?– tamimatCommented Aug 12, 2011 at 13:54
Add a comment
|
5 Answers
- How do I recruit the people?
You need to determine what kind of tests you'd like to perform. Just because you don't have a lab doesn't mean your ability to test is gone. There are very simple tests that include just observing a random sample while they are using your product. There are more intermediate tests involving a moderator/observer that take a little longer. It depends what you're trying to achieve as far as getting people recruited. For shorter tests, ask people in your organization (if possible) that aren't involved with the project to test it out. Assume they are going to say no, and keep asking until you get 3-7 test subjects. For longer or more formal tests, you may need to advertise that you're testing, and incentivize them with compensation.
- How do I setup the test?
On a budget? Setup a somewhat secluded area that is comfortable for your subjects with the device/platform you'll be testing. Make sure you have a timer available and some sort of matrix for tracking the user experience. We have an internal checklist with plenty of room for notes, and some examples of how to "frame" the task question without ruining the test.
- How do I define the tasks to be testet?
If it is something like a website, you'll determine the business goals and test the user on their experience while accomplishing that task. For example, it might be getting a user to register or go through a shopping cart. If it is an app, you'll need to determine what the key functions are of that application and determine tasks accordingly.
- How long should the test be?
Again, this depends on the tasks. My tasks, which are almost always web-based, take about 1-5 minutes each depending on the subject. I usually run batches of three tasks for informal testing (I say informal as anything that isn't in a lab) - this helps recruit people as it only takes 5-15 minutes of their time. When they're done, I always ask if they would be willing to participate in extended testing in the future.
Hope this helps!
I suggest you read Rocket Surgery Made Easy by Steve Krug.
In it he outline how to do simple low to no cost usability testing and interpret your findings.
I am reading this as being a basic test of a specific function within a site ( e-commerce or similar ). Which may make it problematic to test separately from other parts of the site. There is also a need to achieve the testing with relatively limited resources, which puts another set of restrictions.
How do I recruit the people?
Turn this around - who can you get to test this? Can you devise a simple online test that you could email to everyone you know ( probably not in this case ). Can you find a respresentative sample from the people you are working with/for? If you have to actually recruit random people from the public, it will cost you a lot. If you can use friends and family, or a local university students, or some group of free people, it will make a significant cost difference. But most of the marketing companies will help you recruit, if you pay enough.
How do I setup the test?
This actually should be your first question. What I think you are after is answering the question "how easy or natural will it be for the users of this site to achieve this task?" I have had people come to me with printouts of possibilities, asking "how would I do xxx", which works in some cases. I have produced task sets for people and asked them to complete them, which can also work ( but can also be complex ). You might want to consider mocking up the specific task as best you can, and allowing the subjects to try this out. However, if you are looking at various options, this needs a large number of people to utilise each one. The setup is extremely dependent on the usability question and what you want to put into producing a setup.
How do I define the tasks to be tested?
Assuming you have a question like "how easy is it for people to do this", then set them the specific task, in a constrained situation. If you are asking a wider question like "which of these works best" then you need a number of different tasks. Identifying your question is the core to identifying the tasks ( and pretty much all of the answers you are seeking ).
How long should the test be?
As long as necessary. You can either time the test, which will put pressure and constraints on the volunteers, or you can set then the task, and record how long it takes to complete. You should have some idea how long it should take, and if it is a commercial site, the task should be only a few minutes. If you are doing multiple tests in one go, you should realy limit your total time to 10 minutes, unless you are paying people, in which case you can keep them for longer.
You also need to consider what type of testing will work well. Again, if you have a number of different choices and options, you might find that a focus group would work well. Or you might want to mock up some of the options, allow a number of people to do them, and then run a focus group.
None of this requires a lab. I have done a number of these, as part of my study work, and have never needed a lab ( although I have used one when one is available ). Also, if you have an academic institution nearby, they might have a lab that you can borrow, for a reasonable agreement.
answered Aug 12, 2011 at 19:41
I'd try to avoid testing software you've been involved in developing. (ie if you are in a larger organisation try to test other people's software - and get them to do yours)
To do this properly there's a 'Point 6: How to moderate Usability Tests'
This gives some useful tips:
The key thing is that there's a subtle balancing act on knowing when to break the flow so that you can ask 'why did you do that' kind of questions. Moderating tests properly is hard work.
Regarding the data analysis part of discount usability testing, the method of Instant Data Analysis might be of interest to you. Or perhaps rather: How to reduce the cost of formal usability testing.
The method is described in this research paper (not free). In short it argues that instead of using several days or even weeks on analyzing usability test video data, test facilitators instead get together right after the last test to compare notes. This way a joint list of usability issues can be assembled rather effectively.
According to the paper:
... the experiment revealed that in only 10% of the time required to do the video data analysis, Instant Data Analysis identified 85% of the critical usability problems in the system being evaluated. At the same time, the noise of unique usability problems usually characterizing video data analysis was significantly reduced.
