2

For those of you who administer user tests, what do you do when testers don't keep their appointments? I'm currently in the midst of a round of user testing for one of my projects. The tests needed to be booked by appointment for this project, and using usability tester panels wasn't an option. The site in question is targeted for a general audience with a specific interest, and I am trying to get testers from that audience to test a possible new design. The prospective testers don't work for the client and aren't stakeholders.

I have been booking testing slots, but out of my first 7 testers only 2 have shown up for their appointment. One person did not show up, rescheduled, and did not show up to the second appointment either. I'm wondering whether or not a) I did the right thing in assuming good faith and booking the tester again, and b) it would be right to try to book this person a third time if they still express interest in testing this design.

I'm the only UX person on this project and working on this project as a consultant. So I would like to know, is there a standard way for UX practitioners to handle testers' missing (multiple) testing appointments? If there isn't a standard way to handle this situation, how have you handled it?

2 Answers 2

7

My rule of thumb is that I expect about 1/3 of participants to not show up (this doesn't count people who need to reschedule). Occasionally you get lucky and everyone shows up, occasionally you get unlucky and have a horrible no-show rate. There are some items that impact my 1/3 no-show rule-of-thumb:

  • The more difficult that the group of participants is to recruit, the no-show rate is generally higher. People who are more difficult to recruit are generally people who already have a lot of commitments otherwise.
  • If you are providing little or no renumeration to participants, then the no-show rate is generally higher.
  • If appointment times are scheduled to fit into your schedule instead of theirs, then the no-show rate is generally higher.
  • If your location is difficult to get to or has parking issues, your no-show rate is generally higher.
  • If participants are traveling a distance to participate in your research, they are less likely to show up.
  • If participants are not engaged with the research that was presented to them (that is, they're not really that interested in what you've told them you're studying), they are less likely to show up.

I use these strategies to reduce the number of no-shows:

  • Avoid scheduling them too far in advance, lest they forget or later schedule something else for the same time. I generally schedule participants about a week in advance of the appointment times that I'm planning to use.
  • Avoid scheduling near a public holiday or event that is of importance to your audience. For example, in my current role, I am mostly recruiting system administrators for my research. I keep track of the major conferences in the field to ensure that I'm not conflicting with them.
  • Be very flexible in your appointment times to maximize their ability to participate.
  • When recruiting, ensure that your communication with them is framed positively. Make sure that the research sounds interesting.
  • Communication with them should include statements about how valuable their feedback is, and how it will help you make a product that better meets their needs. Without giving away the goals of the study, talk about it in their own terminology and about what is important to them.
  • Email or call them the day before their appointment to verify their attendance. This serves as a good reminder about the appointment, and also gives them a chance to ask any questions that they might have (directions, where to park, length of the appointment, etc).

If a participant misses an appointment but is able to reschedule, I generally reschedule them unless I've already met my study quota. After all, I've made mistakes in scheduling something or gotten stuck in traffic or simply forgot, so I forgive others for human error too. If a participant misses two appointments, I don't try for a third time unless they have an excessively good reason for their previous two missed appointments or if I'm truly desperate to get someone scheduled. (Also, since I am a full-time researcher and my team maintains a database of potential participants, I flag a participant who has missed 2+ sessions as someone not to contact again for future research.)

Although this is somewhat outside the scope of your question, if I'm having difficulty with getting enough participants, I will ask those who have already participated in the study or who are signed up to participate in the study if they can recommend someone else who is like them to participate. If my level of difficulty is really high, I'll offer them a small incentive if the person is a good fit for the study and participates. The size of the incentive depends on how much trouble I'm having filling the study, I usually start with a $5 or $10 Starbucks or Amazon card (for US participants, or something on that order for international participants) to see if that gets me where I need to be.

2

The usual and simplest way is to take more testers than what you need, to compensate for the missing ones. Once you have all the ones you needed, cancel the others. Easy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.