I'm looking over a usability report, and one thing is jumping out at me: at the end of each section in which an identified usability issue was discussed, is a section entitled "Recommendations."

Fair enough.

Only it's not recommendations from the UX team that did the tests, it's recommendations they solicited from the usability test participants.

I've not seen this done before and I'm not sure what to think about it, so I thought I'd ask you lot what you think.

What are the pros and cons of getting design recommendations from usability test participants?

9 Answers 9


I don't think there are any real cons because you can always just ignore the recommendations.

The problem I see is that usually you won't get any new or useful ideas because (with some exceptions of course) users will only know what they want when they see it. Or as Henry Ford put it: "If I had asked people what they wanted, they would have said 'faster horses'."

  • 3
    :D thanks - in return here's my other often used quote: "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." - Antoine de Saint-Exupéry ;)
    – Phil
    Commented May 11, 2011 at 21:11
  • 6
    Another good one! Here is one of the ones I quote to my managers every so often... "You can use the eraser on the drafting table or a sledge hammer on the construction site. - Frank Lloyd Wright
    – jonshariat
    Commented May 11, 2011 at 21:17
  • Brilliant, never heard it - I have a feeling I'm gonna use this one a lot :) Thanks!
    – Phil
    Commented May 11, 2011 at 21:25
  • the con is you may be setting user expectations that their suggestions will be implemented
    – jk.
    Commented May 12, 2011 at 10:47
  • @jk: I thought about that too. But Gary is asking about usability testing so it's not like you're asking all your users.
    – Phil
    Commented May 12, 2011 at 10:56


enter image description here

  • :D +1 for the most funny answer ever!
    – Phil
    Commented May 12, 2011 at 12:14
  • Oh yes. Homer's vehicle came to mind while I was reading the reports, don't you worry about that.
    – gef05
    Commented May 12, 2011 at 13:12
  • Why doesn't this have hundreds of upvotes is totally beyond me :)
    – Mahn
    Commented Jul 11, 2012 at 17:06

Often the usability test participants will just volunteer comments and a good researcher will dig deeper as to why they like / dislike etc aspects of the site. I'd expect participant comments to be included to illustrate the recommendation from the UX team which did the test rather than being just a list of freestanding comments.

  • Nice thought about using it to illustrate decisions. That was, in essence, where I ended up with this. The report is written as "they said it so we'll do it" which I'm opposed to.
    – gef05
    Commented May 12, 2011 at 13:12

There really isn't a problem with it as long as you just take them as suggestions and not data or fact.

The reason is that users are not designers and thus do not know what the solution to their problem is. Its like asking some college guys how to better a fancy meal. They may know it tastes good or bad but they wont know how to fix it.


In participatory design and co-creation it's common practice to ask participants about their design ideas. Even if the solutions are not as good as what professional designers could think of, truly listening to what people say can lead to really valuable insights in what they want.

On the other hand, theoretically, having people think about design suggestions during a user test could influence their behavior. If you really need the best behavioral data you can get from a controlled experiment, you'd have to be very careful with introducing new factors. But, researchers in for example cognitive psychology have different goals than most people conducting a usability study.

Depending on who reads the report and their understanding of the process, listing the participants' suggestions as 'recommendations' could easily lead to misinterpretations (the UX team doesn't necessarily have to agree with what the participants think).

  • 1
    Yes, I think the distraction and the setting of expectations is an issue. The users were asked at the end of each task for suggestions - which means that after the first task was complete, they then knew that for all remaining tasks they were going to be asked for suggestions. Not an appropriate mindset.
    – gef05
    Commented May 12, 2011 at 13:19

I always feel like it is toeing a fine line when test participants start giving design advice about the product being tested.

In my experience you can easily get derailed from the tasks at hand because the participant's gaze changes when he/she starts seeing positive feedback about recommendations for the tested product.

I usually try to mark in my notes the areas which prompt suggestions and then go back to them to solicit the advice from the user after the real testing is done. This way the user's thoughts stay on subject while we solve the tasks in the test protocol, but potentially valuable insights aren't lost either.

When interpreting design advice I find that it's important to keep digging for what the root cause is for a user suggestion. Often the user's thought process will originate somewhere foreign to the practitioner and there's a big chance of misinterpreting if we just take the suggestion at face value.


When you ask usability test participants for opinions, comments, or feedback it is no longer a usability test but is now a user interview. In other words it is a focus group consisting of one user.

I think it is ok to ask for opinions at the very end of a test, but only after I have observed the participant try every task that I want to test. In fact this is exactly what we did recently, we ran one task then showed the participant a presentation and asked for opinions.


Because it's their personal opinion. You don't want to satisfy everyone with your design, because you can not. Some people will appreciate the design, some will not.

Maybe all the test participants that didn't sent you design recommendations like the design. For the next time you can even include a question: "What is your general feeling about the design of the website?" so that you get an idea.


Let's remember also that innovation is achieved best(my opinion) through user input in development and design fields. I mean, take a look at Apple, they opened the development of apps to all users, and now we have thousands of apps which are amazing to use, and they were ultimately created by users (approved by Apple developers of course).

So let's not assume that a user giving us recommendations is completely wrong, or does not know what they are talking about. I mean, if someone takes the time to write a recommendation, they most probably have thought about it and come up with some sort of an idea which they thought was legitimate.

So, do users know what they want? I believe that subconsciously they definitely do, so in order for them to realise what they want, they will have to have thought about it, and in this case, if someone has given you a recommendation, we can be sure that they have thought somewhat about it.

My suggestion would be to not get offended or put off if someone does give a recommendation, albeit you don't have to follow what they say, as pointed out before.


  • 2
    But note that said app designers had absolutely no input in development of the platform and hardware itself. Steve Jobs is well known for not soliciting focus group type feedback (which is a good thing, in Apple's case)
    – DA01
    Commented May 12, 2011 at 2:12
  • Yes, I must agree with DA01 on this. Jobs has a reputation for not looking to focus groups.
    – gef05
    Commented May 12, 2011 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.