Usability testing for open-source projects?

Question

I'm working (slowly) on an open-source project, and I was hoping to do some usability testing once it's a bit further along. Also, hopefully the GIMP devs will notice this question ;-). I'm not sure how much can be done for $0.00. I can con friends/family/coworkers into helping, and I can use a free screen recorder, but the project is targeted at a fairly limited audience (technical artists working on shaders), of which I know 0. I'm hopeful that maybe some users on the wide interwebs would be interested in helping, too. The problem there is how to connect with them. I've read articles on remote usability studies, however it tends to depend on pay-by-the-minute services and expensive screen sharing software. Anyone with experience doing testing on the very cheap have any suggestions?

Answer 1

Check out Skype's Screen Sharing feature (it's free). I'm planning on using Skype for this very purpose in a couple months.

I'll be writing a "Skype Screen Sharing with Steve" blog post to try to encourage participation. I'm thinking the invitation should be up front about the fact that you're looking for feedback and are curious about uncovering problems. But it's also an opportunity for the end user to learn the software better.

Answer 2

I had this problem as well a short while ago. What I did was write out a list of tasks and have participants record a screencast of themselves doing them, while thinking out loud. Later on when you watch the screencast, if there are parts that you would want them to re-do, you could ask them to do a second round.

What is the advantage for you in screensharing?

Answer 3

Read Steve Krug's Rocket Surgery Made Easy. It explains in plain English how you can do usability testing on a shoestring budget (or for free!) by using a common sense approach.

It's been known for years that usability testing can dramatically improve products. But with a typical price tag of $5,000 to $10,000 for a usability consultant to conduct each round of tests, it rarely happens.

In this how-to companion to Don't Make Me Think: A Common Sense Approach to Web Usability, Steve Krug spells out an approach to usability testing that anyone can easily apply to their own web site, application, or other product. (As he said in Don't Make Me Think, "It's not rocket surgery".)

In this new book, Steve explains how to: Test any design, from a sketch on a napkin to a fully-functioning web site or application Keep your focus on finding the most important problems (because no one has the time or resources to fix them all) Fix the problems that you find, using his "The least you can do" approach By paring the process of testing and fixing products down to its essentials (A morning a month, that's all we ask ), Rocket Surgery makes it realistic for teams to test early and often, catching problems while it's still easy to fix them. Rocket Surgery Made Easy adds demonstration videos to the proven mix of clear writing, before-and-after examples, witty illustrations, and practical advice that made Don't Make Me Think so popular.

Answer 4

Google Hangouts On Air is a good option for remote usability testing. It provides screensharing capabilities, allows uses to easily re-connect and the recording is automatically uploaded to YouTube.

The video quality for the recording is not the best, but is good enough to see what is going on if you know the prototype you are testing.

We create and distribute a participation form (using google docs) to easilly schedule testing sessions.

Some related materials:

• A video summary of our testing sessions for which we used Google Hangouts.
• A talk about testing with users worldwide (slides and video).