How to rescue a usability test whose participant is lacking confidence?

Question

Last night, I was conducting a usability test with a participant who was very uncertain of his actions. Before the test, I said all of the usual stuff about it being a test of the software and not him, asked him to imagine I wasn't there, that we were hoping he could help us find the software's weaknesses, and so on. The task he was given had also been successfully used with other participants already.

But it still went badly.

The participant became unsure of how to proceed, lost confidence in himself, and he clearly started feeling like it was more of a test of HIM rather than the software. The test dragged on and, despite yielding a good amount of useful feedback, it clearly hadn't been a pleasant experience for him.

How should I rescue such situations in the future? Is there anything I can do to get such sessions back on track, or should I just call an early end to tests where the user gets 'stuck' and stops enjoying it?

I'd welcome any advice...

Answer 1

It's not clear from your question if you have any insight into why the participant lacked confidence. You need to identify this because how you try to solve the problem depends on what's causing the anxiety.

Here are some causes I've found in the past:

• The participant is anxious because of the testing situation. He finds the recording equipment intrusive or he's worried that the test is some kind of elaborate scam.
• The software is very hard to use, making the participant feel stupid.
• The tasks you've set the participant aren't realistic -- he doesn't understand what he's expected to do.
• Something bad is going on in the participant's life (e.g. he had a row with his boss before coming in)
• The participant may have been poorly recruited and doesn't have the expertise necessary to take part.
• The moderator is anxious for some reason and this is being transferred to the participant.

Depending on what's causing it, here's some ideas that might help.

• Start off with an easily achievable task
• Ditch your canned tasks for a self defined task.
• Take a 10-minute break.
• Stop the problematic task and move on.
• Turn off any recording equipment and take written notes instead.
• Stop the usability test and do something else entirely -- for example, turn it into an interview instead.
• Be authentic and talk about the elephant in the room: "You seem quite anxious. Is there anything I can do to help?"

Answer 2

Did you get the tester to use the "think aloud" technique? Sometimes this helps them articulate how they are approaching the task. Maybe they are getting caught up in fine details, or its one specific label that throws them completely off the task at hand. Verbalising as they go is not a natural thing, so bear that in mind as well.

If I strike a similar problem, the user is completely lost, or you can tell they're really not sure, try some prompting.

"I can see you are hovering around the {x} feature, what's drawing your attention to that part of the screen?"

"It looks like your not sure which menu item to choose. Maybe you could take me through what you're thinking. How about the [1st|2nd|xth] menu item, if you clicked that what would you expect to get|where would you expect to go."

Another way I've approached this problem is to take the impact of the problem back. "Wow, it looks like we've really mucked up this design. Would you like to draw me a picture of you think we could do the layout better?"

Answer 3

I had one of these once.

The guy had decided after about 10 minutes that he thought the software was rubbish - and that he wasn't going to play ball (not couldn't - but wouldn't)

So as David Travis has already suggested I turned it into an interview.

And that produced some very interesting information on what the guy thought the software should have been like.