Low-fidelty vs high-fidelity: Do they elicit different types of reactions?

Question

You often read claims that lower fidelity prototypes illicit different reactions than higher fidelity prototypes.

This came to mind again today because I came across it when reading a post on UIE (http://www.uie.com/articles/pitfalls_prototyping/):

A great designer, however, chooses the right fidelity for where they’re at with the design. Lower fidelity, like a pen-and-paper sketch, produces a different kind of reaction and critique than higher fidelity, like a jQuery rendition. (This is why tools like Balsamiq try hard to fake a lower fidelity.)

Now obviously the user can't really react to aesthetic things that don't exist in the paper prototype, so you have that much going for you. I am sure many of us have had a user who reacts more to colors than the functionality -- although both are important, early on this might not have as much value to the project.

But one thing I've heard fairly consistently (and I think is being implied in the UIE article) is that a user is more "honest" in their criticisms about a paper prototype. This seems to be why Balsamiq is designed the way it is too.

This thought assumes that the user is more likely to give their true opinion on something -- especially if it's negative -- on a paper prototype because the user perceives that less work was involved in creating it.

It also therefore assumes that they'll be more reserved in critiquing a higher-fidelity version because they feel like they're looking at an almost finished product -- where decisions are mostly final, a lot of work was done, etc.

While that makes sense to me (and I would say I've seen it happen), I've been unable to find any studies that significantly back this assumption up. That example is one I heard consistently in my past, but has anyone "officially" proved the accuracy of it?

I have found several studies about the efficacy of paper versus computers. For example at http://www.usability.gov/articles/newsletter/pubs/062005news.html:

The two types of prototypes produced essentially the same quantity and quality of critical user observations (there was no reliable difference). However, 92% of the test participants preferred working with the computer-based prototypes.

The others I have found generally back this finding up -- the quantity and quality of critical user observations remains the same in both cases.

In my work we do both low and high fidelity work and will continue to do so, but I find yself continuously curious about this concept and how much water it holds. To clarify -- I'm not asking about the value of paper prototyping (which I think is immense), but whether or not a user's impression of a low or high fidelity prototype actually changes the type of feedback they will give.

So what does all of this mean for the assumption that we're getting "different" types and approaches of criticism for low and high fidelity prototyping?

To clarify: I think what I'm trying to get answered is more psychological than it is specific to pixels or design. I definitely agree with the points on progressing from low to high, the various differences between the two, etc... I guess I'm just particularly curious about this one aspect of it. My interest on this is less about the TYPE of feedback (as in that pixel size is off) as it is the intent or context of that feedback.

Is there inherently something different such that someone will tell you about your overall design due to their impression of the amount of work they perceive went into the prototype (high vs low)? I suppose it's also a question of does one type cause people to be more reserved with their feedback than the other?

For example, if you showed the same design in low fidelity is that person more likely to talk about their general experience and expectations BECAUSE it is low fidelity? Would they be less likely to talk about that with a higher fidelity prototype they infer was more work and is less likely to be altered based upon their feedback?

And if this is the case (and I generally would agree that it seems to be), is there any formal study on this?

Answer 1

Yes, they do. Low fidelity tends to get people to focus on the higher level aspects of the mockups, like the overall layout and concept.

While with high fidelity mockups people tend to focus on the lower level details like "this should be two pixels to the left", or "this needs to be a different shade".

That is one of the reasons that I recommend starting with a low fidelity mockup (which is typically faster) to get the concepts right, and then moving onto a high fidelity mockup to work on the finer details.

Answer 2

I can only provide you with my experience having experimented with many different types of stimulus for users.

The original statement about getting the right fidelity is spot on. In general it is possible to get similair results from end users in most situations but if it's too far from the final result you run a risk of some users not really getting it.

The idea that users are more likely to comment on sketches because less work has been done doesn't match with anything I've seen.

With all these things there is no one right answer but a well rendered sketch is, in my view, on parity with a simple non sketchy wireframe in many situations. Having said that if the end experience is screen based then not presenting it on a screen distances it from the final experience. If a persons's sketching abilities mean they can produce good quality sketches, scan them inn and host spot them as fast as someone else can throw some grey boxes on screen using Axure then that's parity. In my view sketching is for idea generation and sharing and even extending to getting reactions from users about an overall concept.

So, in my experience, the closer you get to the end experience the more accurate the feedback you get. This needs to be balanced with what you need and how quickly you can produce the stimulus. Paper protoypes are kinda cool looking but don't offer any real advantage from what I've seen other than novelty value.

Answer 3

The purpose of mockups is to communicate ideas. Depending on the stage of development you select the appropriate fidelity.

During the initial phases you are debating different features and layout options in your design, a low fidelity prototype easily captures this (animations/transitions are not the kind of features I am talking about here). You can add colors (without worrying too much about hues). You can have basic click throughs and other forms of interactions here, balsamiq or paper based (yes paper prototypes can have interactions too!).

The added benefits of low fidelity benefits are obvious but, I will restate them:

• Quick and dirty production - can do this while discussing with the user.
• Easy to make changes
• Do not need to worry about minute details
• Get to throw them away when they have lived their purpose.
• Supposedly guilt free critiques are made possible

I think what your question is getting to is the collaborative aspect of low fidelity. Anyone can do it, your user, client, etc. It empowers them to think WITH you on a level playing field. It is a dialogue between their and your ideas.

---

Later on, once the goals are hashed out, you start working on the high fidelity stuff. How exactly should the stuff be laid out, how should the transitions happen, and other details. Things are taking shape here and you need your compositions to be as perfect as possible to hand them off to the developer or next person in chain. Every interaction should be labelled and all the functions should be defined in detail.

Answer 4

Here are a few research papers comparing the feedback provided in response to interacting with low-fidelity and high-fidelity prototypes. In these papers, 'fidelity' refers to visual design fidelity and not one of the other 4 dimensions useful for describing prototype fidelity.

This paper reports on the comparison of paper and computer prototypes for a ticket machine and a calendar system. They performed a usability test of the prototypes. The number of critiques and suggestions provided by test participants was not affected by the type of prototype nor were there differences in the categories of critiques or suggestions.

This is a comparison of low- and high-fidelity prototypes of a multi-touch UI. Here is a key result from this comparison

"...the low-fidelity prototyping method presented in this paper is an excellent method for idea generation...it makes it possible to examine an idea‟s qualities without building a copy of the final design."

"The low-fidelity prototype inspired new concepts such as ... This type of design ideas did not emerge during high-fidelity testing."

The authors of a comparison of 3 levels of fidelity of a mobile UI concluded

"...the major usability issues such as unclear meanings of labels, icon/symbol/graphical representation issues, locating appropriate interface elements, mental model mismatch, and appearance/look of the product, were identified by all the three types of prototypes..."

This is not an exhaustive list of the research on this topic. There are several papers on the effect of prototype fidelity on feedback regarding game design that I have not read. Check the NordCHI proceedings as a starting point.

A final reference is not a comparison of prototype fidelity but does show examples of low-fidelity prototypes and suggests that presenting many UI ideas is (more?) important than the choice of prototype fidelity in early stages of system design.

Answer 5

Two big factors to consider:

• Your decision-making latitude within the organization/team: What buy-off/input do you need from your audience and what are the sub-divisions of that audience?
• The abilities and understanding of the audience in question: Who can visualize and who gets get lost when the examples are loose?

When you identify the roles and responsibility, you can assess who needs to be involved and to what degree. Based on my experience, this is the process I recommend.

1. A data point outline and nothing more. At the early stages where I need conceptual agreement from the business owners and technical buy-in from the back-end team I don't even get into wireframes. We talk about the availability of data for the user. What needs to be available to the user, what needs to be available to make that available, and what is the hierarchy of importance?

2. Wireframing with a small group of experts. When it comes time to layout rough information, I work with a small group with excellent visualization skills. We start with a whiteboard or sketch pad and quickly determine our best take on the UX. That gets fleshed out in Balsamiq.

3. Build tight comps. I'm working mostly on brands where I've set up a fairly tight set of guidelines which equates to having clear UI standards that can be applied quickly. In rapid-development cycles (which seems to be most of them now) I skip the Balsamiq wires and throw an Illustrator mock-up together right on the big screen as our small group talks through the validity of our sketched ideas.

4. Prototype it. I quickly review the mock-ups with the execs and key development representatives to be sure we aren't off track completely, then jump into a proof-of-concept with a semi-functional prototype. This is where a larger group is brought in to review.

Most people don't have the experience to visualize what you're trying to indicate with wires or mockups. Because of this, they'll raise innumerable objections and suggestions that probably don't apply when presented with rough sketches or static mocks. Their value is at the data point stage where goals can be defined, not your progress in executing those goals. If you are the UX expert, you and your team are the ones to work out the details.

Answer 6

With regards to your question about why paper prototypes dont seem to be too effective, I believe its because with paper prototypes you have to ask the user to visualize the environment (e.g. if its an app,he has to visualize its not a piece of paper with a drawing but an app with which he can interact and similarly for a website).

To quote this article from uxmag

But unlike high-fidelity prototypes, paper prototypes lack the context of the physical device the product will ultimately be deployed to. For example, a paper prototype of a website can only be experienced on the piece of paper that it was created on, whereas an HTML/CSS-based prototype can be experienced in a web browser. Paper prototypes also depend on a facilitator to act “as the computer,” changing different states of the UI in response to a participant’s actions.

The lack of physical context of paper prototypes is particularly problematic when designing mobile experiences. Because mobile applications are inside mobile devices, they can be experienced in many possible positions and contexts. Showing prototypes in an unusual setting (such as a flat piece of paper on a table) doesn't give users a full and accurate experience of the design. Simple paper prototypes can test a product's UI and content, but leave much of the broader product experience untested.

If we could replicate more of the mobile user experience using low-fidelity prototypes, we could get more of the feedback that would otherwise be more difficult and take longer to obtain.

I looked at the link you had posted of paper prototypes being shown as effective by research and though that might have been true at the time of publishing those papers in 1998-2003 when you didnt have to worry much about trying to emulate physical artifacts such as form factors and the amount of interaction in websites was very limited (like links and popups), but now considering the number of different form factors and rich interaction available,it does take a stretch of imagination to try and visualize all of that in paper prototyping

With regards to your broader question of whether low fidelity prototypes work (referring to screen designs here designed in something like Balsamiq), here are my thoughts on why I find them effective

• They work in conveying a concept and ask the user to focus on the flow and the elements instead of being distracted by the colors
• They imply that this is just the skeleton and hence not the finished job and hence there is scope for improvement. This encourages users to provide inputs as opposed to when you have a highly polished design when users are left wondering about whether their design suggestion would fit into the design scheme

With regards to your question on whether they would elicit different responses, chances are yes because people can get caught up in the design and the colors and the flashy interactions and may not focus on the the primary usability issues. However if your low fidelity prototype and high fidelity prototype depict these interactions in a similar way and dont have too many distracting elements, by this research study it seems like both of them are just as effective in in detecting usability issues

Do low-fidelity prototypes tend to find different problems than high-fidelity prototypes? The question appears to be answered with a qualified, No. Both techniques uncovered the same usability problems for the most part, and at about the same level of sensitivity, as witnessed by the high positive correlations in the two studies between the proportion of low- and high-fidelity subjects finding particular problems. That is, when a problem was found by a high proportion of subjects in the high-fidelity group, it also tended to be found by a high proportion in the low-fidelity group. However, some problems that were found in the high-fidelity condition were not found in the low-fidelity condition, and vice versa. This is true for both studies. There are many reasons why this result might obtain, but inspection of the actual problems did not lead to any obvious conclusions. We believe, that except as noted earlier, this is probably a result of normal variability, given the small number of subjects, rather than a function of differences between the two prototypes, but we currently have no evidence to substantiate this belief.

I also recommend reading this excellent research paper on the difference in the way users interacted with a high fidelity prototype of an interactive table as opposed to the low fidelity model. This might be out of the the field you are looking at but it throws some interesting thoughts on a high fidelity design can influence users to react differently in a collaborative environment.