Identifying a fair and representative set of users for interviews and persona development

Question

How do you make sure you have a statistically representative and accurate sample of users from a population and what criteria would you base that sample on?

I want to make sure that the people we interview and observe are actually a fair sample of a population of approx. 250k.

Just wondering how others approach this problem.

Answer 1

I don't know if it applies to your situation but it's useful to remember that, from a statistical point of view, a (reasonably large) random sample is best, even if it is sometimes difficult to obtain in practice.

In fact, more complex techniques (quotas, screeners, etc.) have been developed to deal with specific problems (sampling smaller sub-populations, bias in the sampling process that make an actual random sample impossible, etc.) but if you have a full list of all potential users from which you can pick up names randomly (which also mean they have to be equally likely to accept to talk to you…) and your sample is large enough, you don't need to worry about representativeness.

One way to do it is to put your list in a table or database and order it or otherwise give each potential interviewee a number (any order will do as long as you don't change it during the procedure). Then go to http://www.random.org/integer-sets/ and generate one set with N (your desired sample size) integers between 1 and the total number in your list. You can then use the results as id number/position in the list to pick people.

Answer 2

It's hard to do this perfectly, but this is what people normally use screeners for. You establish the key dimensions along which your personas differ and then construct screener questionnaires to see which group an potential participant fits into and then accept a set amount of participants for each persona/group.

For example, say you had two personas: Steve (frequent, professional user - 70% of your user base) and Craig (sometime hobbyist and dreamer - 30% of users). The key dimensions along which they differ is whether they used an app professionally and how frequently they used it.

You could construct a screener:

• Do you use example app for paid work?

• How often did you use example app in the past week?

• Do you expect to use example app in the coming week?

If you had 10 places for your research, you would take 7 Steves and 3 Craigs based on their answers.