Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

How do you make sure you have a statistically representative and accurate sample of users from a population and what criteria would you base that sample on?

I want to make sure that the people we interview and observe are actually a fair sample of a population of approx. 250k.

Just wondering how others approach this problem.

share|improve this question
add comment

2 Answers

I don't know if it applies to your situation but it's useful to remember that, from a statistical point of view, a (reasonably large) random sample is best, even if it is sometimes difficult to obtain in practice.

In fact, more complex techniques (quotas, screeners, etc.) have been developed to deal with specific problems (sampling smaller sub-populations, bias in the sampling process that make an actual random sample impossible, etc.) but if you have a full list of all potential users from which you can pick up names randomly (which also mean they have to be equally likely to accept to talk to you…) and your sample is large enough, you don't need to worry about representativeness.

One way to do it is to put your list in a table or database and order it or otherwise give each potential interviewee a number (any order will do as long as you don't change it during the procedure). Then go to http://www.random.org/integer-sets/ and generate one set with N (your desired sample size) integers between 1 and the total number in your list. You can then use the results as id number/position in the list to pick people.

share|improve this answer
    
That's a good idea, we're looking at a project for a library so we could pull all the members with a library card. Thanks! –  MatthewForr Aug 26 '11 at 14:03
add comment

It's hard to do this perfectly, but this is what people normally use screeners for. You establish the key dimensions along which your personas differ and then construct screener questionnaires to see which group an potential participant fits into and then accept a set amount of participants for each persona/group.

For example, say you had two personas: Steve (frequent, professional user - 70% of your user base) and Craig (sometime hobbyist and dreamer - 30% of users). The key dimensions along which they differ is whether they used an app professionally and how frequently they used it.

You could construct a screener:

  • Do you use example app for paid work?

  • How often did you use example app in the past week?

  • Do you expect to use example app in the coming week?

If you had 10 places for your research, you would take 7 Steves and 3 Craigs based on their answers.

share|improve this answer
    
I was wondering about this but how do you know the criteria and questions you generate aren't leading or are capturing the right traits. –  MatthewForr Aug 26 '11 at 14:04
    
What is a screener? –  Tony Bolero Jan 21 '12 at 7:26
1  
@TonyBolero A questionnaire you ask potential candidates to fill out. I use them to filter out candidates (like those with web design experience) and organize the sample to be more representative. –  cutwithflourish Jan 23 '12 at 11:01
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.