What ways to conduct user research quickly and cheaply are there?

Question

Let's face it, in today's world - everything is moving fast and with approaches like agile, UX gets squeezed. It doesn't have to be this way and from what I hear many companies are using Sprint 0 approach to do some research before diving into develoment. However, I'm sure that doesn't solve every UX problem. With that said,

What ways to conduct user research quickly, cheaply and with results meaningful enough (and hopefully statistically significant) are there?

I'm curious if you do user research and how it works in your org (big or small).

Partly answering my question here but I know there are a lot of apps springing up geared towards testing/research, e.g. Loop 11, VerifyApp... Please include any of these in the answer where relevant.

Answer 1

You only need to test with five users. That's an oversimplistic point, and need is the key word, but the important point is that you can easily find most of the major issues with a small sample set. I think seeking statistical validity has you going down the wrong path.

You're not conducting peer reviewed research, you're conducting iterative tests on a single product. You don't need something you can run once and be 100% sure about, you should running tests every step of the way.

If you can't get five users each step of the way, test one user¹:

As soon as you collect data from a single test user, your insights shoot up and you have already learned almost a third of all there is to know about the usability of the design. The difference between zero and even a little bit of data is astounding.

Steve Krug's Rocket Surgery Made Easy describes great ways to conduct quick, simple usability tests on the cheap, without professionals. Again, these won't catch every possible usability problem in one go, but they'll allow you to test repeatedly to catch new usability problems introduced or uncovered after design changes.

From the description, Krug's book can (and will) help you:

• Test any design, from a sketch on a napkin to a fully-functioning web site or application
• Keep your focus on finding the most important problems (because no one has the time or resources to fix them all)
• Fix the problems that you find, using his "The least you can do" approach

Note for statistical validity you'll need a nice large sample, ideally of at least 30 so you can normalize results. Statistical validity is usually not a priority unless you're doing a simple survey as it requires a relative large number of participants for simple usability tests. Do not focus on this.

For specific tools, Usaura is a great quick and free resource to let you send quick tests to users over the internet. Note they don't let you test live interfaces but you can do some quick A/B testing on a visual layout of a page, ask some quick questions, or get a click map of where users think "where should you click to do...".

Answer 2

If you want to do just a click test or a five second test than http://usabilityhub.com/ is the right solution for you.

On the other hand for most usability tests, a paper, a laptop, and a webcam are more than enough. They are not so expensive.

Here you can find an article on this topic

Answer 3

Look at the book "The Lean Startup" by Eric Ries. Although the principles in this book are not specific to UX they could be applied to using UX in an agile organization. One of the basic principles is to experiment often in small increments using a split test. Make a small change to your app that you believe would provide a better UX and put half of your customers on the new design and half on the old design, collect metrics and compare. If the change has a positive outcome release it, if not throw it away. Move to the next test and iterate. Much easier to do on a web based application. The key is continuous incremental and iterative changes.

Answer 4

Sprint0 sounds very familiar :). Yes, Agile can be moving really fast. My frequent approach when there is no time to do usability test with the actual users of your system is to recruit in the company (no people working on the same project to avoid bias). Send an email and ask people to volunteer half an hour of their time. In my experience people find usability studies fun and usually getting enough people is not an issue.

Also you can try setting expectation for your dev team that design may still change; so if something is implemented you may feed additional changes into the design. How do you feed the changes into design? After usability tests you can create user stories that address identified usability issues and add them to the backlog or talk with a product owner so they can prioritize them.

I would be curious to know how others integrate UX into agile. I am a part of the agile team attending scurms, etc. In some other companies UX teams are more removed from the dev team.

Answer 5

Assuming you have coworkers, hallway testing is about as inexpensive as it gets.