Should we ask users to play a role when performing user testing?

Question

I'm actually working on a users-testing session, to test an expense report application.

My question is: When I ask the user to do a task, should I ask them to play a role?

Example: If we want users to declare a train ticket, should we say them:

"You are a traveling salesman/woman, today you'll meet a client, so you buy a train ticket from Paris to London. Declare this expense with your mobile app."

OR

"Buy a train ticket from Paris to London and declare this expense with the mobile app"

I like the idea of giving a story to follow, because it can help users to understand the sense of their tasks instead of just giving a tasks list. I'm not sure if that is the best approach though.

Answer 1

Yes, usage of onboarding stories are a good choice.

BUT - a user is only a useful testing subject if he is part of the target group. So asking him to "imagine you are a salesman" is not going to work: The results you will get will always be the answers of a person who is NOT a salesman and ultimately not part of the salesman-group you want to check.

The scenario has to be a real one, the user can follow and identify with.

Answer 2

Yes! as long as it is a scenario they will encounter when using your software.

Otherwise you are testing the wrong user and the wrong persona.

Absolutely don't get them to impersonate anyone else - even if it is a persona that will use your service - because they will make-up and invent behaviours that may not be accurate.

Answer 3

I'm going to go with no.

From experience, you normally get the most accurate and truthful user testing results by getting real users to do real tasks in relevant contexts. If you ask someone to want something they don't want, or behave in a way they don't normally behave, you get weird imagination interference where the user will try to emulate their idea of something rather than act based on the real desires.

It's the same reason why it can be hard to test with dummy content or in a lab. I think it's totally fine to ask someone to complete a task but in your example, I think the second example is much more effective as it doesn't dictate the context. Additionally, you're going to get better results when you show that stripped-back scenario to some users from different demographics because rather than declaring who they are, you can see how a range of different people go at achieving this task.

If you're asking that question with "you are an x" then you may find that you have only designed your app for x. If that's your intention, great, but remember that lots of different people use software, including users with accessibility requirements, users of all different ages and users who do and don't understand the basics of what your software does.

Don't box yourself in. Go broad, and you'll get real information from different people with different needs. That's the reality of how your software is going to be used, so don't be too specific and lose the important info that might come from widening your boundaries a bit.

Answer 4

I would say it would be better to avoid it.

I guess it depends on the user empathy: some users will be able to think as they are that kind of person, and some others won't.

But even though, playing a role is not accurate at all, and there will be a massive amount of context, experience and motivations you will be missing.