3

I'm looking for user testers interested in geography, urban studies and digital storytelling. How do I suss this out? A yes/no question would be too leading, and an open ended question: "What are you interested in" seems unfocused. I'm leaning toward a multiple choice question with a lot of choices.

1
  • Welcome to UX.SE! Great question! Feb 12 '15 at 18:08
1

The site User Testing has an article talking about Recruiting Your Target Audience for UX Testing with Screener Questions. Within the introduction they point out:

If you’re in one of those circumstances, and you’re testing remotely, then you need to use screener questions—multiple-choice questions that can either eliminate users from taking part in your study or give them access to it.

So you are correct in leaning towards multiple choice questions. Multiple choice questions have multiple advantages but also multiple pitfalls. The article discusses some guidelines:

  • Always provide a “None of the above”, “I don’t know” or “Other” option just in case you’ve forgotten to include an answer that applies to the user or the user is confused by the question. This is especially important to include in screeners, because if users don’t have this option and pick an answer at random, they might end up in your study accidentally.

  • Provide clear and distinct answers that don’t overlap each other so that (for example) a size 6 doesn’t have to decide if they want to be grouped with the 0-6 sizes, or the 6-12 sizes.

  • Avoid asking leading questions or yes/no questions because users will be inclined to give you the answer they think you want instead of the one that really applies to them. We find that instead of asking direct questions, instructing users to select the option that most closely applies to them, followed by a list of statements, is the most neutral way to phrase most screeners. This method ensures that users will answer honestly because it’s less obvious what answer is desired.

The last point gets directly at your concern about leading questions. Asking "are you a fan of Geography" will tempt people to answer in a way they think you want them to answer.

The site goes on to describe 4 Guidelines for Writing Great Multiple Choice Questions

In generating a set of questions to get at the individuals hobbies, consider a series of questions that work of the previous responses. From the article above, they use the example of finding someone's job as a message therapist:

Obviously, massage therapy is a very specific profession, and it would be hard to come up with an exhaustive list of options inside of one screener question. But you also want to avoid asking a yes/no question, so you might start by listing broader professional categories, including Health (which would encompass massage therapy), and then in a follow-up screener, have users indicate the role they occupy within the Health industry. An example is below:

enter image description here

Which you can then lead into a secondary more specific question:

enter image description here

In your case, perhaps an initial question with something to the effect of:

  • Arts
  • Science
  • Sports
  • Collecting
  • Computers
  • Other

Which you can drive into more detail on under the "Science" category:

  • Biology
  • Geography
  • Cryptozoology
  • Ichthyology
  • Astrology
  • Other

If possible, don't think of a screening survey as a single piece of paper. If you can make it dynamic you can ask more-and-more specific questions along certain chains that avoid individuals simply answering a (potentially leading) yes/no question. If the answer "incorrectly" for this screener, maybe you found a good candidate for your next project!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.