User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
I would like to run few iterations of user testing on a checkout re-design. This project is still in the early stage so I would like to take full-advantage of the situation and run testing sessions on paper prototypes. The testing will be therefore focused on the checkout experience, filling forms and so on, which makes me wonder if is it a good idea to test paper prototypes on a so focused aspect of the website.
Do you think is still a good idea? Any tips on how to setup the tasks?
Testing paper prototypes is a great idea for testing information architecture and the discoverability of content. I've done it several times, and yielded useful results by doing so. Users are good at suspending their disbelief when using paper, and prototypes can be rolled out quickly. Win-win.
The problem is that they're not as good for testing webforms and interface behaviour, because of the interactivity element. Form design is a lot about validation of content, auto-defaulting and dynamic strategies like progressive disclosure, and that's quite hard to replicate on paper. Then again, these elements might not be the most important aspects of usability - users will muddle along with suboptimal forms if they're convinced what they want and know how to find it, so that's your biggest question, and one easy to answer with paper tests.
Another issue is that paper prototypes are very lo-fi, which makes it hard to test how the user perceives the trustworthiness and 'stability' of your service, which are critical when money changes hands. They also don't do a good job of branding the content, which can be important in consumer-facing situations.
I'm not saying don't use paper prototypes. Any data is better than no data, and the sooner you can resolve any design issues the better (and cheaper to fix). I'd just be aware of the limitations.
When selecting a testing method you should always first ask what the goal of the testing is, for no one method will suit all possible purposes. At the most basic level, different methods are good for discovery, while others are good for validation.
Paper-prototyping is a testing method for discovery and understanding, and is ideally used early in the design cycle.
You certainly can use paper-prototyping to test specific sections of websites, especially those which represent a work-flow. Paper-prototyping has the advantage of being quick and simple to test several alternative approaches, as well as supporting rapid redesign (possibly even while a user session is still in mid-train).
There are also some meta-benefits to the process.
Paper prototyping is very good for early stage testing of your interface workflows, to discover if you have any mismatches between your design model and the user's inherent mental model of the task. You can't (easily) get new users to arrive at your site with a changed mental model in mind, but you can always change your design model to suit. As much as possible you want to discover the user's mental model and preferred workflow, not simply refine an existing UI by papering over the incompatibilities.
Using lo-fi prototypes makes it easier for your users to suspend their disbelief and make it easier for them to think on a more abstract level (i.e. closer to their mental model of their problem, instead of adapting their understanding to the quirks of the interface before them).
Also, use a collage mix of rough line drawings, fine art sketches, cartoon images, and even some printouts of wire-frames. The (feigned) lack of respect for the honor and integrity of your "art" will give licence and permission to your participants to criticise the designs they are interacting with. (Sometimes test participants are too polite, and don't want to say you've made a design mistake, or they resort to giving socially acceptable answers).
Using lo-fi prototypes will also make it easier to kill your babies - this is not the time to have your designers fall in love with their creations. Prototypes are meant to be thrown away.
Paper prototyping is also good for testing workflows and basic UI interaction without the distractions of content, branding, styling, etc. It is actually a good thing to discover where in the process users will have trust issues, or get confused as to whether the content is from the website are is a third party thing. Once you know where these weak spots are you can ensure they are addressed, without wasting time and effort addressing these issues in places where they don't have an impact.
Some tips for when you do your paper-prototyping testing:
do paper prototyping very early in the design cycle, possibly before you've even progressed onto making wire-frames.
get your entire design team involved in making all the various parts - doing this early can serve to address the storming and norming phases of team building.
don't get your best sketcher to do all the sketches. It's OK to have a mix of rough line drawings with fine art sketches. (This collage effect communicates that you're not interested in execution details)
use lo-fi sketches where possible - you want to keep the users in the mode of exposing their mental model, and not getting hung up on execution glitches. You also don't want your designers getting tunnel vision and obsessing about execution details.
sketch up interface "blanks" and photocopy them, ready for quick re-design (eg. a variant of a drop-down menu would thus only need the menu item names added, no extra drawing needed).
use print outs of existing wire-frames for less critical pages (less critical for the testing objectives, that is). Go ahead and overlay lo-fi paper elements if you need to make a quick change.
ask the participants to use the thinking aloud protocol.
have at least two people involved on your side: an operator, and an observer.
We used paper-prototyping on a custom checkout flow of a service. This particular service had two co-dependent sections and we weren't certain whether to get the user to complete Module A before Module B, or vice versa. By testing, and getting the user to follow the think aloud protocol, we uncovered some latent mental model mismatches with one of the approaches and figured out the preferred workflow.
We also identified another redundant section earlier in the workflow and eliminated that step entirely (it was a product selection and preview step ... but at launch we'd only have the one product so it was rather pointless, even if it did present a pretty picture and a bullet list of product features).
You may be benefited by doing an interactive prototype testing rather than paper prototype.This could be done using Powerpoint or similar balsamiq kind of tools. This will help to address how the user thinks or either to do a cognitive walkthrough with the users user or stakeholders.
Paper prototype will not bring about to understand the users problem, since it will not completely make the user in the real-world scenario (real world in here is web world). It is good to streamline thoughts, but interactive prototyping will help to set up your task and test it efficiently.
On setting up the task, try to write cases for problems that people have been speaking around. You could even run a participatory design session in paper prototypes and check the task with the users, whether if the decisions have come out as intended!
Happy prototyping!
I am currently testing the checkout as well.
My opinion is: the more realistic the test the better.
1st iteration: real website, fake data — good to find the pitfalls of the current checkout solution to know what to improve.
2nd iteration: invision clickable prototype of the improved design; it was relatively easy to prepare and the users only had a few gesture-errors, other than that the experience for them was pretty realistic. Many useful findings.
I'd say paper prototypes are fine for exploring the ideas, test quickly during the design sprint if you have your users nearby. I would suggest go for as realistic as possible approach: if you can provide the full functioning prototype, go for it, if it's too demanding, go for the high fidelity clickable prototype.
