Questions in Usability Testing without giving any lead

Question

How to make good questions that asking if "Z" feature is easy to use or not in Usability Testing for web or app, without giving any leads to user?

Answer 1

When testing usability, think of which hypothesis you want to validate and which scenarios you have to work with. Based on those, create tasks that will help you measure how well you perform or solve those. Tip, for writing up hypothesis you can think of current problems and or opportunities, for example:

• Ex. problem: users don't read my articles. Hypothesis: 1. the articles are not discoverable 2. the headers of the articles are not interesting enough

What you would do for the usability test is to figure out whether your hypothesis are right or wrong and whether the design (or design changes) may help your problem.

Don't ask users directly why they don't read your articles to start with. You create tasks that may be solved by reading the article. They user tho may solve the task however they please. After this you can ask about how well they feel they could solve the task, missing info, what could work better, how else they could think of solving the task, etc. Get them to try those if they want to.

If they didn't discover the thing you wanted them to test, you can follow up by asking them to do that very thing and follow up with questions: -why do you do it in that way? -How well did it performed? -Would you have tried it like this if I didn't ask you to? why? etc.

This way you can learn more about the challenges your product may have, which expectations your users have and how to solve it.

There is lots of good reading here https://www.nngroup.com/articles/measuring-perceived-usability/ and https://www.nngroup.com/articles/leading-questions/

Quoting a few good practices from the second link:

"Do not rephrase in our own words. Participant: “I notice this picture here…” Researcher: “You mentioned that the picture was helpful. What about it did you like?” Improvement: “You mentioned the picture…?” Do not suggest an answer. “How well would this save time for you during your workday?” Improvement: “How might this affect your efficiency, if at all?” Do not name an interface element. “The related links on the side of the page here — where would those lead?” Improvement: “This area on the side of the page… [point to area]. What is that?” Do not assume you know what the user is feeling. “When you were struggling with this task, what was happening?” Improvement: “What was easy or difficult about completing that task?”"

Mind that tests are different based on whether you are looking for qualitative or quantitative data. Also mind that tests are different too when performed 1:1 or online. Can you see facial expressions? those will tell you a lot.

Give users enough time to think and don't rush them in replying to questions or lead them to tell you what you want to hear :)

Answer 2

Use task based activities that require the use of the feature you want to test and base your questions around the problem that our feature sets out to solve rather than the feature itself.

Lets say your new feature is an email subscription form. You might present you test candidate with a page or prototype and ask them something like "how would you get regular information about this product?" and then observe how they find and use your email form. Once you have notes about your observations of the user's performance you can then ask them how they found using that particular feature.

Answer 3

Lets create a made up example.

Your mobile app is about animal facts.

One question might be "What is the average weight of an icelandic horse?"

After the user has answered the question; he or she can rate on how easy it was to look up without giving the user any hints or tips.

Hope this helps.