Transforming qualitative and quantitative data or methods

Question

I can see how a large number of interviews and video can be compiled and analysed numerically (i.e. turning qualitative research into quantitative research), but is it not possible to turn quantitative research into qualitative research as well? For example, if I have heatmap or clickmap data, can I not used the information to create questionnaires or usability study design?

Why has it been suggested that you can turn qualitative data into quantitative data but not the other way around? Or that you can turn qualitative research into quantitative research but not the other way around? If so, what is the reason for this?

For example, if you collect a lot of qualitative data from survey questions about what people think about a product (e.g. 1 for very good and 7 for very bad), you might not be able to work out exactly how good the product is, but you can work out what percentage of people think it is very good.

On the other hand, if you collected metrics from a website that shows completion rate for tasks or conversion rate for goals, to what extent can you say that users find the website easy to use?

Answer 1

The Principle

The principle that you can’t change quantitative into qualitative refers to a conversion of the same data, not, say, using the results of one study to feed the design the next. Qualitative analysis can follow quantitative analysis, with the qualitative analysis being either in a different study, or even in the same study.

What you can’t do is convert quantitative data into qualitative data. You have to use different data (collected before, after, or during the quantitative data collection) to do a follow-up qualitative analysis. The reason you can’t (generally?) convert quantitative to qualitative is because quantifying something necessarily means you’re abstracting it, and thus losing the details you need for qualitative analysis.

Example

In the case of your click map, you may notice a statistically high tendency for users to click on Products. You may wonder if this behavior results from the user culling other information from various parts of the web site. You can’t answer than with the click map data, because it’s only a user-by-link table of what each user clicked on (for which totals across users give the frequencies to represent in the click map). The click map data doesn’t tell you if users tended to click other links before clicking Products. That information was abstracted out in the process of preparing the user-by-link table for the click map.

You can, however, go back to the raw click logs (if you collected and kept them) and perform a qualitative analysis of some users to study their sequences of clicks. This, however, is not converting quantitative data to qualitative. This is going back to your original qualitative data from which you derived the quantitative data for the click map. You’re no longer using the user-by-link table.

Suppose you find that users click Products after reviewing About, Testimonials, Return Policy, or Privacy Policy, the exact page depending on the user (which explains why none of these individually were click as often as Products). This suggests that your users review the website to establish trust before doing business through the web site. To cement this observation, you can define and calculate a quantitative measure of each user's “Trust before Business” behavior (e.g., user gets a score of 1 if they click on About, Testimonials, Return Policy, Privacy Policy, and/or Sustainability Policy before products; 0 otherwise). Then you can determine the frequency users engage in “Trust before Business.” That’s converting qualitative data into quantitative data.

But you can’t convert your “Trust before Business” measure back into raw click logs to now see if some of these users also compared the site with the sites of other businesses. That information was abstracted out. You have to go back to the raw logs.

Answer 2

In general, quantitative data can only be transformed to qualitative data if there is a high correlation between the quantity and the quality. In reality there are many other factors (e.g. cost, season, advertising).

E.g. if number of likes implied how good something is, you could compare number of likes to calculate quality.

In your example, perhaps the heat-map and click-map, together with the final action the user performs could be used to determine how easy it was for the user to perform the action (e.g. how long it took the user to find it, how much of the page did this user need to scroll through to find it, how many clicks). The results probably won't be very accurate, because you can't tell what the user is looking at within the visible part of the page, or even if the user is looking at the screen or talking to someone in the room.

You would need to ask the users what there original intent was on completion of the action, to avoid skewing results with cases where the user originally intended to do one thing and the decided to do something else instead.

Also, you would need to factor in experience of user with your site and the current action. E.g. returning users should be able to do the same actions they have done in the past faster. You could confirm this is true (if your site has a really bad UX it might not be true) and hold a separate statistic for first time users.