Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I've performed several usability tests, mainly with one participant at a time. However, I have never tried to perform a usability tests when you have e.g. two persons cooperating on a task.

  • What would be the pros and cons of having this setup? E.g. two participants helping each other during a usability testing instead of having two sessions, one per participant.
  • Is there any user research that exist that has been conducted for this area?
share|improve this question
    
I would imagine this setup is required for testing multi-player games, or any software platform where users interact in realtime. I'm not aware of any specific research in this area, but there must be some given the ubiquity of such platforms. –  Franchesca Jun 10 at 11:03

3 Answers 3

up vote 3 down vote accepted

Multiple user testing is generally utilized in collaborative projects. Multiplayer games, source control applications, social networking, etc.

Many times, it requires users to perform different roles simultaneous that affects the application working. There's this one music site (can't remember the name) that allowed musicians to take live requests from an online chat room. Naturally you would need a musician and a spectator simultaneously to verify the usefulness of your interface.

Projects which require long term involvement of the user can benefit from varied experiences of multiple users. If a single user were to participate in long-term usage, they would tend to stick to a particular method of making their way about the application. This is not really beneficial when you want to weed out potential problems in your interface.

References: http://www.nngroup.com/articles/multiple-user-simultaneous-testing/

share|improve this answer
    
Just what I was looking for! Thanks a lot. –  Andreas Johansson Jun 11 at 7:08

To me, it depends on the purpose of your app/site etc. If the goal is to achive something by yourself, testing should also be done with one participant at a time. Using more that one participant coorperating will give very different results, as theoretically with 2 participants only 50% need to know how something is done to ultimately get a 100% score.

Even worse: because they are cooperating they can feed of each others ideas and observations, both achieving more that they would indivually do. For example: examine this scenario:

Person 1 (p1): I have no idea what to do

Person 2 (p2): What if you click here?

p1: Hey that works!

p2: All right now I have no idea

p1: What is you slide this to the right?

p2: Yes that is the solution!

In this case, individually, neither would complete the test, as p1 would be stuck at the first step, while p2 would be stuck at the second step. That could lead to the conclusion that the product needs a lot of work. However if you just look at the results of the collaboration, your instinct would say that everything is working perfectly as they both achived the goal!

share|improve this answer
    
In my experience two achieving more than one is only a problem if the team is over-focussed on task completion during testing. Because you can still see the problems happen. In the scenario you described it's obvious that there is a problem - since the participants nicely articulated it for you ;-) –  adrianh Jun 11 at 7:37

The negatives I've experienced:

  • You have two people to observe. This takes more effort/skill.
  • You sometimes get one person dominating, which may mean you miss the input of the second person.
  • It's an even more unnatural setting in some contexts - especially if the people are strangers.
  • Participant recruiting costs sometimes dominate the budget for testing. So with more participants per test, you run fewer testing sessions. Since you generally get more value from more testing sessions this is a bad move.

The positives:

  • People sometimes expose more in a conversation than they do with think aloud. I think the example scenario that @user1261104 gave in their answer is actually a great example of this. By listening to the problem solving conversation between the participants it becomes very clear where problem spots are.

  • I've had a couple of experiences of it helping get more from sessions where the participants have status / safety issues with respect to the moderators. For example we did some testing a while back with teen and pre-teen women around a pretty sensitive topic. Testing with pairs of friends really helped produce a lot more useful info because they were much more at ease.

  • Especially when you're doing non-lab testing you'll discover that many situations that you nominally think of as being single-user tasks actually involve multiple people. For example I've seen several occasions where the person making the buying decisions on a web site is making the purchase in co-operation with a more technically savvy family member who drives the web site.

  • Some products are naturally multi-user (chat, games, etc.)

  • Very occasionally recruitment costs / participant availability / scheduling constraints means that having multiple participants only way you are going to be able to work with a set of participants.

For me the negatives generally outweigh the positives. Especially when having more participants per-test means fewer testing sessions.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.