Expectation Management

Question

I'm struggling to convince the team of developers I work with to let me show users mockups to explore ideas and make sure we understand their requirements fully.

Although I swear that I will repeatedly tell users that nothing is guaranteed to actually appear in the final product, and that it may end up looking nothing like the mockups, the team are dubious. They say that similar sessions in the past have led to customers feeling let down when something they liked didn't (perhaps for technical reasons) appear in the final product.

How do others handle this - both with developers and with users?

Answer 1

It seems like you have two issues:

1. Verifying/understanding user requirements; and
2. Convincing your team that testing is a good idea.

Verifying/understanding user requirements

When trying to understand or verify user requirements, I try to stay from using any type of mockup or prototype. In this stage, I try to understand the problem not attempt a solution. It's also in this stage that I also test the feasibility of the requirement with the developers.

For example, users want to upload their own content in order create x. I'd relay this to the developers who would then give me any contraints.

Once I have the constraints, I design 1 or more solutions. This can be a individual activity or you can bring developers, users, stakeholders, etc. in to brainstorm with you. Once the design(s) are complete, I go back through these with the development team to spot any possible technical problems.

Convincing your team

If members of a team already a negative outcome will occur, I walk through the testing with them first, with them as a user. In my experience, some developers have had very different idea of what testing will be and once they go through the process, it makes more sense. Also, let the developers observe the sessions.

In addition, using paper prototypes or lo-fi designs will help address these possible expectations, but, in total honesty, I've never tested a design idea and then in later session have that user express disappointment or being let down by a change in the feature from the previous session.

In dealing with expectations, I think the best thing to do the frame the context of the session as well as possible by telling the user that this is an idea you're testing. People typically understand that.

Answer 2

I swear that I will repeatedly tell users that nothing is guaranteed to actually appear in the final product, and that it may end up looking nothing like the mockups

Tried that many times, it simply doesn't work that way.

Problem #1: Users will have false sense of progress - There is nothing you can say or do to explain to your customers that what you are showing them are in fact only mockups. They can SEE with their own eyes there's "only" some wiring under-the-hood left.

Problem #2: You'll get all kind of design feedback - Which you typically don't want at this point. Any discussion about fonts and colors and visual identity and whatnot is at least counter-productive (if not straight harmful) while you are still struggling to agree on basic structure and functionality of the system.

Problem #3: Your developers will confuse mockups for GUI spec - We can say what we want, but developer's psychology (and power of abstraction) isn't that different from ordinary user psychology, we are all homo sapiens after all. Which means you'll have a very hard time explaining to your developers (whatever they might say) that your mockups are just a vehicle to test different ideas and to test mutual understanding with the customer. Not to mention developers simply copying the mockups into actual screens without thinking for 30 seconds, and usually complaining that the person creating mockups should have better understanding of their tool / framework / whatever to create more accurate representation of the actual GUI.

So what do you do?

First, you don't do "nice" mockups, at least at the beginning. Use pen and paper. Or a drawing board. Or a mockup tool that have sketchy / black&white look and feel. This way you are communicating that mockups are only that - mockups.

Second, don't introduce some formal and complicated "process" with your mockups, that also invokes "spec" mentality easily. Simply draw some related screens, comment them with users in live workshops, draw some more screens etc.

Btw you'll want to do that before coding has even started. You'll want to start with several "main" application screens, and when you have those reasonably pinned down, only then proceed with some kind of scenarios (whatever methodology you use), because it's important to get the customers in their natural business context right away.

I was so frustrated with all this that I wrote my own tool almost a decade ago (MockupScreens, it actually became quite popular).

Oh, and here is the most comprehensive list of such tools I know of, both free and commercial: http://c2.com/cgi/wiki?GuiPrototypingTools

When choosing the tool, know in advance what do you want the tool for:

• High fidelity or low fidelity? Even prototyping right there in live workshops?
• Communicating via PDFs / Word documents, or just creating simple screenshots?
• Using the tool to produce something close to a spec or just use it for throw-away mockups?
• What's your target platform (the platform of the system you are developing): desktop, web, smartphone, etc?
• Populating screens with real-looking data to present real usage scenarios? (for this one, you can expect to have dozens of "main" screens but literally hundreds of their "children" with only difference being some on-screen data here and there
• What level of interaction do you need from the tool? (Most tools have "links" you can put on everything, e.g. to click on a button, but some tools are almost fully interactive)
• etc etc

Answer 3

A few random thoughts:

• Focus on value. On the convincing the developers side I would be talking more about the value of the feedback. How much will it cost us to build the wrong thing, or build the right thing badly?

• Is the cost worth while? The developers are right. Some of the people you show it too will think that feature X will be in the final release. Whatever you tell the user. Whatever you show the user. Accept that this will happen. The question then becomes - is the few annoyed users worth the value you get by making things better for everybody else.

• Use low-fi prototypes. I find that you get the problems you discuss much less often with paper prototypes than anything that looks "finished".

• Not the next release. When you're with the users don't say that this is something for the next release, or what you're working on now. Say that you're looking at options for a future release, or for the work that's after the next release. If you push the date for implementation into the future - people are less likely to expect it and complain.

• Are prototypes the right tool? Are prototypes what you need right now to understand the requirements? Would something less likely to raise user expectations be a better tool? Contextual interviews for example?

• Move when you do the testing If good ideas are being tested, but not built, that's a sign to me that you're testing too early (users generally won't complain about you dropping bad features they test ;-). Test more often and later in the development cycle. Get input on whether features are feasible for the next release before bothering the user. Again - if you're doing lots of prototype work on "good" features that don't get implemented, that can be a sign that prototyping isn't the right tool at that early stage.

Answer 4

Great answers from adrianh and brad. I struggled a bit with similar issues when I started in my current company - they had no experience of working with UX before, the UIs were designed by the CEO and devs prior to me.

My additions:

• Build trust with the devs, be transparent in everything that you do
• Include them in test sessions and planning, if possible. They will feel more ownership and control over the process - more likely to be "on board", they will learn about what you are doing, and very important aspect: they will understand the test results better than hearing about them second hand from you. You can rotate for instance one dev / session, however, need to be careful with them not interfering with your test if you do not have access to a proper usability lab.
• Consider running the first tests with internal test participants who are as closely as possible representing the real end users - this is usually faster and easier to organize, and the expectation management should not be so much of an issue.
• You can also organize "pilot" test sessions with people you wish to educate in the team so they understand how the test works, and you can hopefully gain some trust.
• If possible, try to establish yourself in a position where you have the authority to decide when and if usability testing/prototype testing is done
• Accept that it might take a bit time, in my case, it took a few months to get the point through, our CEO had similar concerns with creating unrealistic expectations or being able to get anything useful out of user research for our future products.

Answer 5

Don't mock. You're looking to test user experience! Have you tried mocking bird? https://gomockingbird.com/

There's also Balsamic http://www.balsamiq.com/products/mockups

It's quick to rapidly prototype experiences that are very clearly "theoretical mocks" so users won't so directly correlate these to your product. Also keep them simple. Break your application apart if it's complicated and test specific tasks a user might take. These will be seen as tests and not give testers expectations. You could even time users to do specific tasks. You can also give them multiple to see which they prefer.

You never ask a tester if they would like a feature. They will always say yes. You never ask what experience they'd prefer. Henry Ford is quoted "If I had asked customers what they wanted, they would have ask for a faster horse."

You know better. Don't ask. Test. Experience. Improve. Tell your company to stop making faster horses.