Determine scenario for task or let user decide?

Question

I've read material about user testing strategy and they all talk about determining a scenario for the task. Is this necessary?

Wouldn't it be more natural to let the user him/herself decide on the better, more realistic experience?

On the other hand, having a set scenario would mean results are more consistent. Have you any practice for this method? Do you see any problems with that?

Context: I am redesigning a website for renting rafts and I do not know if I should determine under what conditions (price, number of people, ...) should participants are ordering rafts

Answer 1

You need to think about workflows

When designing the solution, you have to work from a hypothesis about the user's workflow. What mental model do they bring to the problem? How are they going to break down the tasks to solve the problem? Your solution should attempt to answer those needs.

Users need to think about the problem

When you test your hypothesis, it's critical that you don't provide them with an outline that predisposes them to think like your solution. IOW, don't do this:

1. Find an orange raft for $200
2. Select the 8 person option
3. Reserve it for Friday ...

Being overly prescriptive will simply confirm that the user can follow your instructions. You want to present them with the real world problem and let them struggle (or not) through your features to find the solution.

To reach that end, I prefer to use something more narative like so:

• You're planning a rafting trip with 7 of your friends.
• $250 is budgeted for the raft rental.
• The group would like to be on the river early Friday morning.
• You'll be reserving your raft on RagingRiverRafts.com

That more closely aligns the story behind the experience with real life. The workflow is open ended. There's a little bit of stress about getting it right. This creates a mental state nearer the real life pressure of using your service and will help you find the errors in your assumptions.

---

I'd rather spend more time sifting through potentially inconsistent feedback than looking at highly normalized results that may be misleading.

Answer 2

Most tests are based on systems where the user has a number of choices in terms of the tasks they wish to perform - Even on a simple website the user can choose to find information about X, look at pictures of Y, contact the owners... etc.

Each of these tasks has it's own complexity - How does the user find out how to contact the site owners? Do they use a navigational link? Where does that take them? Is there a form to fill in? What information do they need to include?...

This is why tasks are often set before the test.

Reasons to test may be to find how to improve a particular problem for users - in which case you will want them to perform that task so that you can observe the problems they find.

On the flip side, you may have made an improvement and want to test it's effectiveness so you will want to create a task that ensures the user makes use of the improved interface.

Alternatively you might be running a competitor analysis or bench-marking study in which case you will want to focus specifically on task on competitors systems that match those in your systems.

I can think of lots of reasons to set tasks for user testing but the only reason I can think of not to set a task is that the system to be tested only allows the user to complete one specific task.

Answer 3

We usually do user testing in early stages using low/high fidelity prototypes/wireframes to find design issues earlier and avoid dev rework.

so it will take long time for us to create prototype which handles all real time scenarios.so we usually use some real time scenarios to do user testing..

Since you are redesigning, you will get analytics information to get common/priority scenario u need to handle in design or test scenarios which u felt difficulty when u reviewed existing websites.