3

I am looking for the best definition for qualitative and quantitative research. I wrote an article on UX Magazine in which I tried to address the need for doing research that tries to map what happens to why it happens.

I have seen articles use the term 'qualitative' and 'quantitative' research used to refer to these complementary types of research questions being asked. However, it seems that in the social sciences the terms refer to the type of methods (qualitative being associated with words and quantitative with numbers) rather the the type of question being asked (i.e. what happens as opposed to why it happens).

I wonder if UX practitioners have fully adopted the social science definitions combine both types of research methods in user experience studies. I think this is important because it is also possible to generate qualitative and quantitative data from qualitative or quantitative research methods.

My question is, what would be a UX practitioner's accepted definition of what qualitative and quantitative research is?

Does it necessarily have to fit with social science research, or can it be something different? Could we call it 'exploratory' and 'investigative' research instead?

3
  • 1
    The "why" and "how" of behavior, experience etc is exactly what qualitative research is concerned with in the social sciences. Some UX examples are contextual inquiry, task analysis, think-aloud techniques, cognitive walk-throughs, mental models. The data of qualitative research is typically text/images/sounds.
    – uxzapper
    Sep 30, 2013 at 7:50
  • Check out chapter 1 Qualitative Research. Defining and Designing, in Guest, Namey, Mitchell (2012) Collecting Qualitative Data: A Field Manual for Applied Research
    – uxzapper
    Sep 30, 2013 at 8:04
  • Can you just show the few easy examples of qunatitative and qualitative observations?
    – momna
    Feb 5, 2019 at 7:42

4 Answers 4

6

Definitions

While researchers still argue about the value of the qualitative and quantitative approaches, their definition is rather universal and agreed upon:

  • Quantitative - conclusions are derived by means of numerical analysis.
  • Qualitative - conclusions are derived by non-numerical analysis means.

There is also the mixed approach, which means utilising both approaches.

Indeed, there is some confusion, with many people attaching these approaches to research questions and to research methods. But only upon the occasion, this practice is correct.

The following article explains this in greater details.

On (Research) Questions

Certain research question call for a quantitative approach. A question such as 'how many people prefer design A', clearly hints that the conclusion must be based on some numerical analysis and numerical data collection.

A question such as 'why people prefer design A' suggests a more qualitative approach. Asking people why they prefer design A is considered qualitative, and the conclusion may be 'most subjects said that design A is better looking'.

Notice however, that many questions can be answered using both quantitative and qualitative approach - it really depends what is the focus of the research. For instance, in answering the latest question 'why people prefer design A', a researcher may employ a survey where subjects are asked to rank (1-5) different design criteria. The results may yield that 80% of the subjects chose aesthetics as the main reason for liking the design. So now we approach the question from a qualitative perspective.

On (Research) Methods

Quantitative, qualitative and mixed are often attached to the term 'research methodology'. As such it is easy to see why many people attach these to specific methods. What's more, it is harder to separate the approach from a method, but well possible!

The obvious example is interviews - most people will see these as a pure qualitative method. But this is incorrect: Given a sample large enough is taken, interviews can be analysed by numerical means (how many interviewees preferred design A). While this is possible, most researchers will claim that if quantitative data is what you are after, interviews are probably least suited for the task.

In surveys, open question are often part of qualitative research, where close questions are part of a quantitative one.

Even observations can be analysed by numerical means (83% of the subjects have smiled when landing on page X).

Given that, the term "qualitative or quantitative research methods" is flawed - the qualitative and quantitative terms are used to denote the data analysis approach, not the question or the methods employed.

However, does picking method X and applying a quantitative data analysis makes it a 'quantitative method'? Most people will live in peace with this definition, same as a 'red apple' means that this specific apple is red (but it can be green as well).

Conclusion

To answer your question directly:

A UX practitioner's accepted definition of what qualitative and quantitative research is that of the rest of the world, given above.

Also, it is important to note that in search for the 'why?', researchers are free to choose between the two approaches. 'Why?' can be equally explained by the quantitative approach as by the qualitative one.

On the question whether these definitions fit with social science research - absolutely yes. I think the real question is whether the quantitative appraoch is appropriate for social sciences. This question has been dealt with brilliantly in this excellent blog. It also mentions the terms 'exploratory' and 'confirmatory' (which I believe is what you call 'exploratory' and 'investigative'). From the blog:

The deductive–inductive dichotomy is one of several used to distinguish quantitative and qualitative approaches to research. Again in practice the distinction may not be so clear; for this reason some methodologists have proposed the terms exploratory and confirmatory: quantitative research can be exploratory and qualitative research confirmatory. Quantitative research is typically associated with theory testing, that is a theory-first research approach, whereas qualitative research is more commonly associated with an inductive or theory-generation approach. However, there is no necessary connection between purpose and approach. Quantitative research can be used for theory generation and qualitative research can be used for theory verification.

6
  • I like your thorough treatment of the question. I am still wondering about things like mouse click and heat map analysis which contains both qualitative and quantitative data (since it wouldn't make sense to record only one click or one snapshot of where the user is looking). From the literature I can only see the definition being tied to the data analysis method, so it seems like there should be something else describing the question or methods used.
    – Michael Lai
    Sep 28, 2013 at 12:30
  • With heat maps, the data being collected is of a high numerical nature (a percent of a quantity per a section of the screen). Your conclusion from these is based on numerical analysis. So I wonder how can you approach a heat map in a qualitative manner? How can you conclude something from a heat map that does not account to the numerical data the heat map is made of?
    – Izhaki
    Sep 28, 2013 at 13:00
  • And please see my revision of the conclusion. The link at the end of the exploratory/investigative sentence may well provide what you are after.
    – Izhaki
    Sep 28, 2013 at 13:37
  • @MichaelLai heat map and click data are numerical data. The type of data of qualitative research is typically text/images/sounds, and the questions to be answered are "how" and "why" questions. Heat map and click data is answering a "what" question.
    – uxzapper
    Sep 30, 2013 at 8:00
  • @uxzapper Just a quick clarification- qualitative research can also be used to answer "what" questions. In my experience, qual/quant can't be cleanly divided into what/how/why. Sep 30, 2013 at 16:17
3

However, it seems that in the social sciences the terms refer to the type of methods (qualitative being associated with words and quantitative with numbers)

Not just the social sciences. That would be my expectation in the UX field - or any other come to that. The words have meaning - they're in dictionaries and everything ;-) Redefining them will only cause confusion.

rather the the type of question being asked (i.e. what happens as opposed to why it happens). I wonder if UX practitioners have fully adopted the social science definitions combine both types of research methods in user experience studies.

Yes would be the short, and indeed long, answer in my opinion. UX folk use qual and quant the same way as everybody else does in my experience.

Could we call it 'exploratory' and 'investigative' research instead?

I've commonly come across the kind of separation that I think you're talking about being called "generative" (coming up with new insights) vs "evaluative" (figuring out whether something is any good). Which division cross-cuts the qual/quant divide. Nicely illustrated in this slide

3
  • My impression is that UX folks don't do as much user research as they should be, since we seem to be tied up with doing wireframes and mockups as the primary deliverable. I don't even see colleague refer to the terms qual or quant research, just that they have to do eye tracking or click mapping analysis or workshops with users. I have often wondered how agile practitioners convince managers to let them play planning poker when I am seldom allowed to do any serious user research or studies...
    – Michael Lai
    Sep 28, 2013 at 12:34
  • UX designers are employed to provide answers to very very specific problems mostly within the private sector (with typical budget constraints). The methods we use are nothing but the best compromise. Academic researchers, also in related fields like cognitive science or visual design, are the ones to tackle general problems in UX.
    – Izhaki
    Sep 28, 2013 at 14:13
  • I disagree. I encounter UX folk in industry doing user research all of the time. Indeed I've seen a positive resurgence of it in recent years. Just last week I was talking to a client about diary studies they had run. It very much depends on the organisation. Things like Lean Startup have brought classic qual techniques like interviewing to the table in many organisations.
    – adrianh
    Sep 28, 2013 at 16:19
2

I feel like the previous respondents covered your question pretty well, and I don't have much else to add on the question of quant vs qual. I did, however, read the article you linked to. I disagree that there is a need to label or identify research by whether it is discussing what is happening vs why it is happening. (If anything, your categorization of what vs why is sufficient!)

If you feel there is confusion regarding whether the research is investigating what vs why, this mostly suggests that the research question(s) haven't been defined well enough. Typically, the best method for the process should follow from the specific research questions and goals. The best method is and should be based on much more than whether your research question asks "what" or "why."

If you feel this is an aspect of UX research that needs improvement, I would suggest focusing your energy on educating people on the need to clearly define their research questions and goals, and help them figure out the most appropriate method for their SPECIFIC (not what vs why) question.

2
  • I definitely think you should always start with the specific question that the client wants to find out. I also see the importance of being able to clarify the research question so that people are communicating ideas in a consistent manner. I think my experience with social science has been that they have framed the research in terms of the methodology (and hence the quant/qual distinction), which I find difficult to use when communicating with product managers and the software developers.
    – Michael Lai
    Oct 1, 2013 at 11:10
  • I'm not sure what your background is, but I have an MA in social research methods, and based on my experience a good researcher should let the research question guide the methodology, not start with the method. I do agree that this can be difficult to convey to PMs and people less familiar with research. They get hung up on sexy terms like heatmapping and disregard the method's appropriateness. Oct 3, 2013 at 18:08
1

I agree with the definition because in quantitative research we will have to do with the statistical significant population but in the qualitative research we don't have to do it with statistical significant population. We just play with the words.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.