Should wireframes be tested?

Question

I've seen some mentioning in recent times that user testing wireframes is best practice.

Obviously testing as early as possible is something to pursue and a wire frame is a lot easier to make than a fleshed out design with realistic ui and visual elements.

However the argument being presented is not to test early but that even if you have a fully fleshed out design you should strip it all back to a black and white wire frame before testing.

The explanation for this is that it means users don't get distracted by by the ui and will focus purely on the ia and functionality.

However... I'm not so sure this is automatically the best thing to do. It strikes me that if in testing users constantly comment on visuals then it may come time to strip things back but if you have a hi res version available to you then you should at first test with that. Testing something as close to the finished product as possible gives you the most accurate results.

Has there been any decent research into the pros and cons of stripping everything back to pure wireframes?

Is this considered universal best practice now or are there only certain times it should be followed?

Answer 1

A rule of thumb you can consider is: Is the hifi mockup significantly more attractive than the product you're trying to improve?

If you're asking a user to compare a shiny hifi concept to a terrible 20-year-old interface, you very well might be introducing bias from the aesthetic usability effect. (This can happen when you're running usability tests with current-state users.) Testing lo-fi wireframes is better in this case.

If your design aesthetically matches what your users already know and expect, there's less chance for the bias, and I think you'll be fine showing the hifi concept.

One more thing to keep in mind: Some user testing participants don't understand wireframes and get hung up on "why the new UI looks so rough" or "Are you really turning everything gray?" Sometimes it's just easier to show a product that looks realistic.

Answer 2

In my view this depends a lot on

• What the maturity of your design is and
• Where the biggest uncertainties and risks remain in your design that you want to minimize through testing.

Generally, testing wireframes is best for figuring it out whether your IA and the underlying structure of your interface as well as any flows through it make sense to users and are easy to understand and consistent. If you need to verify these, having participants use a high-fidelity visual design is distracting, as they are tempted to comment on visuals instead of underlying structure, and it can be hard to distinguish between an issue being due to underlying structure or the visual representation thereof.

Any visual design that comes after that stage should already be based on a verified/tested structure. At that stage, it is imperative to test the actual visual design, because only this will allow you to figure out if the design that's actually going to be deployed does a good job of making the underlying principles understood and conveying information to users -- stripping things back would be counterproductive if the structure, IA, etc. are no longer what you have to test.

Coming back to the question of stripping a high-fidelity design back to wireframes, I would therefore argue that it makes sense only if you are unsure about the underlying structural aspects because you did not have an opportunity to verify them earlier in the design process. Ideally these should already be taken care of when the visual design gets fleshed out, but maybe issues come up later, your process didn't allow testing it, you've mocked things up in a high-fidelity way because you have a preexisting design system (though in that case the visual aspects should already be polished enough to not become an issue), etc.

Answer 3

I have never seen anywhere that striping it to BnW just to test is a best practice. I've never considered that option either because I'm already hiding so many crucial issues there by doing that.

If the user is complaining about the visuals in the usability test sessions, there could be different indications too.

1. The facilitator or the researcher couldn't take the control of the interview 100% and make the user concerned about the visuals.
2. Maybe the facilitator asked the participant's opinion rather than observing behavior. It happens in the test if the user is asked to give recommendations or opinions, they become wild in their opinions. It should be prohibited in the test to make the test as much unbiased as possible.
3. There might be some leading questions in the test which made the user give opinions on things that they were not supposed to make.
4. There might be issues in the visuals that were bothering the users, and need improvement there.

There can be more rationales too for this specific event.

If you have a full-fledged design already, you should(or must) test that design with the participants. Maybe you can make a Heuristic Analysis at first within the team, so that you can identify most of the problems beforehand and solve them before going to the test sessions. The result will be much better then.

If you get constant feedback on visuals, don't you think that you should consider that fact too? :)

Answer 4

There are different types of user testing, and it depends on what we're testing for.

Usability tests – Can users use what has been designed?
A high-fidelity design can be helpful, as colors and visuals can influence usability. It doesn't mean we can't use a wireframe, but the limitations should be considered when analyzing the results.

Value tests – Is the product/service useful, valuable, desirable?
For most value tests, the visual design component in not critical. We want to understand the perceived value of our product/service. An exception can be testing for desirability if the visual design would make a big difference.

Other testing In interviews where the role of a design is to illustrate an idea and enrich the conversation with a participant, wireframes will often be sufficient. Examples would be contextual inquiry for ethnographic research or sacrificial concepts.