For my project, the first step of designing a service is design research. how many people should I interview to reach the point of synthesizing the interview results? is there any to go place to find out about these steps?
3 Answers
Academic studies tend to say between 8-12 is optimal for qualitative research. However if you are targeting discrete groups of users (e.g. teachers and students) you may need 8-12 of each discrete type. In my experience, the cost starts exceeding the benefit after about the 8th participant.
The following factors should also be taken into account:
- interview structure and content
- complexity of research
- level of certainty required for research outcomes
- cost of resources
- heterogeneity of sample
Sources: Mason (2010), Guest et al. (2006), Ryan and Bernard (2006)
A useful academic article can be found at http://www.qualitative-research.net/index.php/fqs/article/view/1428/3027
-
8-12 is optimal for user testing which is a fairly tightly defined procedure. However "qualitative research" is a much broader description.– PhillipWAug 5, 2014 at 9:40
While referring specifically to usability, the following article by Nielsen states that five are already enough to find out a lot of issues. This is close to the 8-12 rule mentioned by danimu:
http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
I also remember reading about a "rule of thumb" for qualitative research that goes like this: "Issue mentioned by one participant is an anecdote. Mentioned by two is a coinsidence. Mentioned by three is a trend worth to investigate." (sorry for lack of cite source)
So at least three participants would be required but due to high possibility of one missing something she wanted to mention, a size of 3x3 (==9) participants would be a more reliable size to go.
-
The Nielsen piece is about the numbers for usability testing — not the numbers for interviewing.– adrianhAug 6, 2014 at 7:53
-
Depends on how you see it. For me, usability testing with five participants is not very different from interviewing five participants about their software use. The difference is that in usability tests are very structured (with tasks to fulfill) while in an interview you may just start to talk, but can also be very structured... However the result is that you get qualitative problem statements either from a 5 men usability test or an 8 men interview run. That's what makes it comperable in my eyes. Aug 6, 2014 at 9:28
-
I think we have very different definitions of design research / interviewing ;) For me it's all about understand users, discovering needs, etc. It's all generative. Usability testing, on the other hand, is evaluative. It's about finding out whether something we already built is well or badly. The former is open. The latter is closed. Having done both they're very different methods to me, and have very different requirements on recruiting.– adrianhAug 9, 2014 at 9:43
-
@adriah Well i see it that in a usability test you test against a defined set of tasks, so basically it's closed. However, the findings you get are (at least for me) not any different to an interview. They are qualitative insights: "Subject 1, 2 and 5 clicked on red instead of green". Afterwards you test it quantitative and get "75% of users clicked red". So, for me it's no difference if I asked someone "what button would you click" or observing it a usability test with 5 subjects. I'm open for any comments if you think my point of view is kinda weird :-D Aug 11, 2014 at 10:52
I personally try to avoid going in with a fixed number in mind.
Instead I treat it as a continual process where we carry on interviewing while we're still learning new things (sometimes called the saturation approach). The point where we're start hearing the same stories repeatedly is the point where we move the focus from research to analysis and synthesis.
Sometimes that'll be after the fifth of sixth person we talk too. Sometimes we'll still be learning new things on person thirty.
The paper referenced in @danimu's answer talks about saturation) quite a bit and is well worth a read.
