Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

Last night, I was conducting a usability test with a participant who was very uncertain of his actions. Before the test, I said all of the usual stuff about it being a test of the software and not him, asked him to imagine I wasn't there, that we were hoping he could help us find the software's weaknesses, and so on. The task he was given had also been successfully used with other participants already.

But it still went badly.

The participant became unsure of how to proceed, lost confidence in himself, and he clearly started feeling like it was more of a test of HIM rather than the software. The test dragged on and, despite yielding a good amount of useful feedback, it clearly hadn't been a pleasant experience for him.

How should I rescue such situations in the future? Is there anything I can do to get such sessions back on track, or should I just call an early end to tests where the user gets 'stuck' and stops enjoying it?

I'd welcome any advice...

share|improve this question

4 Answers 4

up vote 12 down vote accepted

It's not clear from your question if you have any insight into why the participant lacked confidence. You need to identify this because how you try to solve the problem depends on what's causing the anxiety.

Here are some causes I've found in the past:

  • The participant is anxious because of the testing situation. He finds the recording equipment intrusive or he's worried that the test is some kind of elaborate scam.
  • The software is very hard to use, making the participant feel stupid.
  • The tasks you've set the participant aren't realistic -- he doesn't understand what he's expected to do.
  • Something bad is going on in the participant's life (e.g. he had a row with his boss before coming in)
  • The participant may have been poorly recruited and doesn't have the expertise necessary to take part.
  • The moderator is anxious for some reason and this is being transferred to the participant.

Depending on what's causing it, here's some ideas that might help.

  • Start off with an easily achievable task
  • Ditch your canned tasks for a self defined task.
  • Take a 10-minute break.
  • Stop the problematic task and move on.
  • Turn off any recording equipment and take written notes instead.
  • Stop the usability test and do something else entirely -- for example, turn it into an interview instead.
  • Be authentic and talk about the elephant in the room: "You seem quite anxious. Is there anything I can do to help?"
share|improve this answer
    
Of the causes you've listed, it's probably a combination of the software's lack of clarity (not a huge amount, but if cumulative), my own inexperience coming across, and, perhaps most, the participant's lack of relevant experience (of the domain). Once I realised he had less experience than first thought, I probably could (and should) have paused to chat to him about it and fill in any gaps myself; it would've been fairly easy to work around, but I guess I was unsure whether I'd be (impolitely) teaching him to suck eggs (so to speak). Next time, I'll definitely clear it up though. –  Mal Ross Jul 6 '10 at 8:14
1  
The 'elephant in the room' is a good point. Don't ignore it. –  Nathan-W Jul 6 '10 at 8:15

Did you get the tester to use the "think aloud" technique? Sometimes this helps them articulate how they are approaching the task. Maybe they are getting caught up in fine details, or its one specific label that throws them completely off the task at hand. Verbalising as they go is not a natural thing, so bear that in mind as well.

If I strike a similar problem, the user is completely lost, or you can tell they're really not sure, try some prompting.

"I can see you are hovering around the {x} feature, what's drawing your attention to that part of the screen?"

"It looks like your not sure which menu item to choose. Maybe you could take me through what you're thinking. How about the [1st|2nd|xth] menu item, if you clicked that what would you expect to get|where would you expect to go."

Another way I've approached this problem is to take the impact of the problem back. "Wow, it looks like we've really mucked up this design. Would you like to draw me a picture of you think we could do the layout better?"

share|improve this answer
1  
Yeah, we used the thinking aloud stuff and, although he paused a lot half way through sentences, he did try to articulate what he was thinking. I probably left a little bit too much dead air, though. And perhaps I could've built his confidence more by telling him his suspicions about how to use the UI were correct, but I was wary of over-influencing the test. :-/ –  Mal Ross Jul 6 '10 at 8:19
    
At least you were able to recognise the test was going badly. A bad test can actually be a good thing - at least in the bigger view of things. Next time you might detect a similar problem earlier. –  Nathan-W Jul 6 '10 at 8:36

I used to work in IT for the engineering division, and I've had PLENTY of awkward studies. Personally, I've found that when users lose confidence in their actions, it's because they are just so completely lost and don't want to look like an idiot. It's like when you were in school taking a test you weren't prepared for, and the more you went on, the dumber you felt until it just felt like a completely lost cause until you would just give up. Feeling "dumb" is a painful feeling, and it's your job to minimize that, because if that one users feels that way during testing, then that means some people find the user experience painful too.

What I've found that helps is 4 main things;

  1. Reminding them the recordings are completely anonymous
  2. Saying that we made it hard on purpose.

    In the study preamble I tell the participant "Sometimes we throw curveballs at you to see how you will handle them, and what kind of workarounds you might come up with. This gives us really valuable feedback, and we don't expect you to know the right answer. We just want to see what you do." This way, when I remind them of this during the test, they don't feel like I'm just placating them.

  3. Telling them they're not the only ones who had problems.

    If it's something they seem embarrassed to not know how to do, I will say something like "You know, a lot of other users were stumped here too [even if that's not necessarily true]. Why do you think that is?"

  4. Blaming the design

    This kind of goes hand in hand with #3. "Since everybody seems to be stuck on this section, obviously we need to revise this section to make it clearer. How would you expect this to behave?"

  5. Reminding them this gives us better data

    If all else fails, I remind users that imperfect tests give us so much more data to work with. If they have problems, others will too, and we want to know what we can do to improve. If a power user blows through the test quickly, it doesn't really tell us what to fix in the application, just that that that particular participant can pick things up quickly.

It's always tough to deal with participants who aren't willing to play along, but most of the time encouragement can go a long way. And sometimes, you just have difficult people, like Phillip said. Sometimes you can adapt to them to get valuable information, but every once in a while, you can't. And that's okay.

Good luck!

share|improve this answer

I had one of these once.

The guy had decided after about 10 minutes that he thought the software was rubbish - and that he wasn't going to play ball (not couldn't - but wouldn't)

So as David Travis has already suggested I turned it into an interview.

And that produced some very interesting information on what the guy thought the software should have been like.

share|improve this answer
    
Very good thinking! I'll have to remember that one for next time :) –  Michael Lai Jul 21 at 0:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.