Concept testing

Question

I'm currently working on a strategy project. The team conducted interviews with users on what kind of impressions they have about our product and other products like ours. They've told us what they like and what they don't like and we've narrowed down their list of what they link into a set of features.

We've built these features into our current site as a wireframe concept. Now, we have the opportunity to test these wireframes(which are now set us in InVision as a clickable prototype) out and get feedback from the people we originally interviewed.

So, we're looking for feedback on concepts, layout and a bit on functionality. I'm wondering if anyone has any tips on how to conduct this kind of test. Should I present the clickable wireframe and explain the flow/features or give the users situations and let them click through it themselves. Any tips for questions?

Really any advice or links to articles about testing a concept would be really helpful.

Thanks!

Answer 1

both approaches are OK, but it also depends on your project.

For example, if it's an informational site it will be different to an action oriented app, and if it's an action oriented app, it could be very different if it's something casual or an app for extreme users.

Also, it all depends on the features, if they're common or very technical, and so on.

To illustrate this: most people won't have many problems to use a search engine, or set up a date in a calendar, since these are apps most people are used to. However, it could be harder for them to use something like InVision or Atlassian. In the first case, I'd give your users the app "as is" and see what happens. IN the second case, you'll need to explain affordances to your users, and that's a given, unless you're using coachmarks or overlays of some sort

All in all, in general it's a good idea to sit them down and give them a "cold" approach to your project, and see how it works for them without any intervention.

Finally, if you created personas in order to research your app development, they will kind of tell you how to conduct the usability testing since you'll have some variables that are already defined, including level of expertise or similar

Answer 2

It sounds you want to determine what it is better to do: a more or less conducted usability exploration in your project.

I tipically choose a more 'relaxed' session with participants and no strict procedures when:

• I need general feedback of usefulness and usability of early ideas
• I know who the user is (users like, people from companies that share the office, neighbors, colleagues, any people not related to the project and not biased as well)
• I have less time to prepare, run the session and report (typically from agile/lean methodologies)
• My users are in remote locations or want to test asynchronously, so I prepare follow-up questionnaries to answer my questions, or call them later.

Recent examples in my life: early prototype of a mobile app with 3 mocked features on proto io, new mocks of a website

I tend to use a more 'focused' exploration sessions with scenarios and tasks when:

• I need to test a specific feature or flow with defined metrics (like, search filters) that seems to be fully finished (so users won't feel lost with things that do not work or make no sense)
• Users are co-located and will stay in the facilities for 40 minutes or more, or I have access to expensive remoting tools/skype (but it is not the same...)
• The task or flow is critical to the application/website/product

Recent examples in my life: alpha version of a new application

Bottomline, just try to pick the technique that gives you more valuable feedback with the least effort, and make sure your users are representative so you won't have a biased source.

Answer 3

I would personally just let them attempt a complete walkthrough as a user and see if they are able to do so. Because when you deploy the platform you will not be able to guide them through, so the best way to see where you fail in user experience development is allowing users to do it themselves. When we were doing similar testing on my platform. We allowed people to go through the full process and then had a little questionnaire they filled out and gave feedback as well. Doing so we found the absolute glitch and also allowed us to learn of features that we totally overlooked when building it out.

Answer 4

I'm afraid I don't have time to answer this in full right now, but I'll come back to it if I can.

I would suggest exploring tools such as usability hub and their offshoot 'peek' before doing a live test. They're pretty helpful.

Second I would underline how important it is to observe use as naturally as possible. Note 'observe'. For fundamental usability it's always more accurate to watch during than ask after.