7

I have a few different options for the UX of a tool. Each is quite different from the other.

They are currently represented as lo-fi wireframes. I would like to get feedback on them to decide which one(s) to move forward with without building prototypes of each.

Given I have access to a wide range of users would it be effective to simply show the users the different design options, explain how they work and then ask them which they think would work best (and why)? How would the quality of the results compare to a usability test on a prototype?

4 Answers 4

9

In short, no. People without the literacy needed to evaluate a design by looking at wireframes can't give you a good answer. The only way to get a real answer from your target audience is to test the design with them. Otherwise it's like a doctor asking a patient which treatment they prefer, rather than saying 'this is what you need.'

Testing doesn't have to be a big and expensive deal, either. You can go as simple as wireframing the screens and dialogs needed for a given task and asking them to 'tap' through them by pointing to screen elements and telling you what they are thinking as they do. Make notes about what they get right, and what wasn't clear to them, and revise. Read up on paper prototyping for some tips on how to go about this.

To go one step better, get the design prototyped so that your test audience can use it on a screen, and see how it goes. It's more time consuming, especially if you don't code yourself, but it's very effective.

You can also be quite limited with your test audience; 3-5 people will find the majority of problems, based on research by Jakob Neilsen. Even if you can only get one person who will be an actual user of the product, it will help you much more than asking them what they prefer. Testing is part of going the distance to producing good designs, and asking for preference seems almost likely to produce the opposite of a successful design.

5
  • Thanks. One of the challenges have with doing a usability test (maybe this is a breakout question) is that I am building a tool which will be used multiple times a day by the target user base. So my goal is to answer the question 'which design will work better when you have used this interface 1000 times?', I feel like a prototype test under lab conditions would make it very difficult to answer this question. My opposing designs differ quite significantly, even in the site map, so it is a big decision to have to reverse. Are there any techniques that might help me?
    – Jon White
    Oct 4, 2011 at 5:52
  • @JonWhite: Todd is right here. Asking customers isn't going to answer your question, but testing with them will very likely. I've never seen someone do a usability test with potential customers and not gain some insight.
    – JohnGB
    Oct 4, 2011 at 6:35
  • 1
    @JonWhite There's no way to know for sure which design will be best after frequent use, but it's really good that you're designing with that in mind. To answer the question, you'll have to make a choice and then re-test when the app is in production for a while, so budgeting for that is wise. For now though try a few rounds of wireframe/prototype testing with a week or so in between, and see how behaviour changes. Where do people get faster, what do they forget. It won't produce absolute answers, but will give good clues. Oct 4, 2011 at 17:39
  • @Todd thanks this is really good advice. I will try and setup some prototypes
    – Jon White
    Oct 4, 2011 at 19:07
  • I have broken this out into a separate questions as I belive there is a lot of value hidden in these comments, @Todd - I would encourage you to share your expertise on ux.stackexchange.com/questions/12323/…
    – Jon White
    Oct 4, 2011 at 19:37
2

You're designing a Tool (a Web Application) which means User Interaction can only be evaluated through....interaction. The degree of value that you will obtain from your user's looking at a low-fi wireframe will fall short of being valuable for any prototyping. In essence, you're asking for User Experience feedback on something that cannot be interacted with.

Your discussion with users should come before your shortlist of options and after the first phase of prototyping.

1
  • I did an extensive round of user research along with a UX contractor, so I feel I understand the user base and we have personas. We had not planned to do any usability studies until after we built, but now faced with this dilemma it looks like we will need to. And the UX budget has run out so I am on my own! Thanks for answer.
    – Jon White
    Oct 4, 2011 at 5:47
1

As Bill Buxton argues, showing different design options will help the user expressing critical thoughts: they can do so without risking to "throw over all your work".

So you could give some example tasks to "accomplish" (How would you try to do it?) on every design option, and then let them compare which they liked best, and even more importantly, why.

1

Users are notoriously bad at predicating what they may do in the future which is why I believe it’s a mistake to show users multiple designs and ask them which they think they would prefer to use. According to Daniel Gilbert, author of Stumbling on Happiness, “One of the most insidious things about side-by-side comparison is that it leads us to pay attention to any attribute that distinguishes the possibilities we are comparing”. In other words, “side-by-side comparisons will cause users to consider all the attributes on which the designs differ, and end up considering attributes that they don’t really care about but just so happen to distinguish one design from the other”.

In almost all cases it’s better to narrow down design alternatives to a single design, usability test and iterate.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.