How do you make sure you have a statistically representative and accurate sample of users from a population and what criteria would you base that sample on?

I want to make sure that the people we interview and observe are actually a fair sample of a population of approx. 250k.

Just wondering how others approach this problem.


2 Answers 2


I don't know if it applies to your situation but it's useful to remember that, from a statistical point of view, a (reasonably large) random sample is best, even if it is sometimes difficult to obtain in practice.

In fact, more complex techniques (quotas, screeners, etc.) have been developed to deal with specific problems (sampling smaller sub-populations, bias in the sampling process that make an actual random sample impossible, etc.) but if you have a full list of all potential users from which you can pick up names randomly (which also mean they have to be equally likely to accept to talk to you…) and your sample is large enough, you don't need to worry about representativeness.

One way to do it is to put your list in a table or database and order it or otherwise give each potential interviewee a number (any order will do as long as you don't change it during the procedure). Then go to http://www.random.org/integer-sets/ and generate one set with N (your desired sample size) integers between 1 and the total number in your list. You can then use the results as id number/position in the list to pick people.

  • That's a good idea, we're looking at a project for a library so we could pull all the members with a library card. Thanks!
    – MSaforrian
    Aug 26, 2011 at 14:03

It's hard to do this perfectly, but this is what people normally use screeners for. You establish the key dimensions along which your personas differ and then construct screener questionnaires to see which group an potential participant fits into and then accept a set amount of participants for each persona/group.

For example, say you had two personas: Steve (frequent, professional user - 70% of your user base) and Craig (sometime hobbyist and dreamer - 30% of users). The key dimensions along which they differ is whether they used an app professionally and how frequently they used it.

You could construct a screener:

  • Do you use example app for paid work?

  • How often did you use example app in the past week?

  • Do you expect to use example app in the coming week?

If you had 10 places for your research, you would take 7 Steves and 3 Craigs based on their answers.

  • I was wondering about this but how do you know the criteria and questions you generate aren't leading or are capturing the right traits.
    – MSaforrian
    Aug 26, 2011 at 14:04
  • What is a screener? Jan 21, 2012 at 7:26
  • 1
    @TonyBolero A questionnaire you ask potential candidates to fill out. I use them to filter out candidates (like those with web design experience) and organize the sample to be more representative. Jan 23, 2012 at 11:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.