Using a survey to identify problems, for usability testing

Question

I'm new to UX research, and I am on a small UX project just for the sake of practice.

Before conducting a usability test, I understand that I need to know the user's pain points in certain products or services, but I don't have an access to this data. I decided to conduct a survey about a Facebook page, to profile the users and also to ask them to identify problems that I can use to construct the usability-test tasks, or scenarios.

My goal for this survey is to identify the major usability problems that users face while using the Facebook page. I just created a survey, and I'm not sure if I did it right.

Could any UX professionals comment on the survey I created?

Thank you so much! any comments would be helpful!

Answer 1

You've got the general idea

It's a great idea to ask users what they dislike and then to set about proving them right or wrong by conducting a usability test before you spend resources fixing the perceived problem. Users often complain about a symptom without recognizing the underlying design problem.

If this is what you intend to do, then your survey is a good step forward. (I'm leaving it to others to comment on your survey design.) If this is not what you intended to do, then please read on.

Typical usability testing

Typically, you would have a specific feature in mind that needs testing. If you don't have a specific feature in mind, then you might want to determine their goals—so what users do on the site. You could infer this by looking at site traffic, log data ("analytics"), by asking them, and so on. You might also ask your Product Management team or your Marketing team what their goals are for the site.

Note that "asking them" could involve a survey.

Then you would develop scenarios that require users to complete specific tasks related to those goals. After screening and scheduling your participants, you would observe and measure their performance. Or, if you use automated testing, you would allow the system to gather data that you review.

Your analysis of user performance will reveal the usability problems. This would result in recommendations that you prioritize. This typically considers cost to fix the problem, seriousness of the problem, and the company's marketing focus and Product Management goals.

A bit of reading

Here are some articles you might find interesting.

• Usability.gov is a fantastic resource for usability testing with instructions and supporting documents.
• Nielsen Norman group's overview of usability provides a solid introduction: What, Why, How, When, and Where to work on usability.

—

I hope that helps you move forward.

Answer 2

I'd skip the survey and just do the usability tests.

UX pros tend to distrust surveys since you're asking people to recall their behaviors. And people tend to answer questions in not quite honest ways.

For example, in your survey, I can try to remember what my top three features are, but I'm not sure what a feature is, and I might not want to admit that I spend a lot of time stalking my ex. And I could come up with some features I'd like to see, but never actually use them when they're developed.

Instead, just do usability tests - preferably in person - and observe users perform the most common tasks for the app.

If you have to use a survey... (1) Ask questions like this: "How many times in the past month have you used [Feature A]?" The answer choices can be ranges: Never, Daily, Weekly, Once a month.

(2) List the features people can choose from.

(3) I wouldn't use "Rate your satisfaction" questions. You and I might be equally satisfied with a thing, but I'll give it a 4 and you'll give it a 5.

Answer 3

If you don't know how to make a survey, don't reinvent the wheel. Use an existing survey. That way, your results gain authority ("I used a valid method"), you can compare your result to published results of others who used the same survey, and you save yourself work in general.

For a quick and dirty solution, the most common survey would be the SUS. The QUIS is also popular, but a bit oldschool in the freely available version. There are also PSSUQ, SAMI and a few others. I'm personally also fond of the Bargas-Avila questionnaire¹. It doesn't have a specific name, but it's nice, short, to the point, and was validated on a very large number of participants.

I also recommend reading a good book on the subject. "Measuring the user experience" is a favorite of mine, based on solid academic theory but written specifically for the practitioner. It is a very hands-on book and leads you around the multitude of traps you didn't see lurking on your way, for example: "What do I do with the mountain of data I just gathered so I transform it into knowledge?" There are also several others, you can research them and pick the one you like. Reading one of those has a really high return on investment of your time.

¹ Bargas-Avila, Javier A., et al. "Intranet satisfaction questionnaire: Development and validationof a questionnaire to measure user satisfaction with the Intranet." Computers in Human Behavior 25.6 (2009): 1241-1250.

Answer 4

If you're interested in UX in a more holistic way (and not just in usability), you might consider using the AttrakDiff scale (http://attrakdiff.de/index-en.html) or the User Experience Questionnaire (http://www.ueq-online.org/). Although both questionnaires were developed in academia, they are fully applicable in practice.