7

You’ll find a taxonomy and descriptions of different ways of integrating qualitative and quantitative methods in the “Mixed Methods Procedures” chapter in John Creswell’s (2013) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (SAGE Publications). The chapter includes examples of actual studies. The book is intended for social ...


6

Definitions While researchers still argue about the value of the qualitative and quantitative approaches, their definition is rather universal and agreed upon: Quantitative - conclusions are derived by means of numerical analysis. Qualitative - conclusions are derived by non-numerical analysis means. There is also the mixed approach, which means utilising ...


6

A few possibilities come to mind: Heuristic analyses (requires 5-6 fellow HCI practitioners in order to be comprehensive) Validation against personas or behavioural goals (may not be reliable) GOMS / KLM analyses - basically your first bullet point (reliable, but doesn't uncover issues with IA or broken interface metaphors) Comparative assessment (...


4

I can only speak from personal experience, but I often run mixed-methods research that combines surveys and qualitative data. This gives you complimentary data such as "Quantitatively, users preferred Site A to Site B. The focus group explains some of the reasons why they preferred Site A."


4

Short answer is they compliment each other, but ACSI is not about usability, although the usability of something could influence the ACSI score, but I doubt anyone has found a correlation. There is clearly a small overlap between the ACSI approach and usability research approach, e.g. at a high level gathering qualitative and quantitative measures, but the ...


4

Measuring Usability has an article that briefly touches on sample size. TL;DR: 2 is the obvious minimum, 5 is a suitable starting point. From 10 Things To Know About The System Usability Scale (SUS): You can use SUS on small sample sizes: One common question I get when using the SUS (or when measuring usability in general) is about the lowest acceptable ...


3

Yes, there are modified Cooper-Harper Scales that measure operator workload. It involves some basic tweaking from "pilot" to "operator". Here is one such representation of the Modified Cooper-Harper Subjective scale: I used these modified scales many times to measure acceptable operator workload for multiple defense department and aerospace programs, which ...


3

Unless you have a large budget, alternatives to having a person (or group of people) parse the data are potentially complex and unreliable. Card Sort a Folksonomy Card sorting is a simple technique in user experience design where a group of subject experts or "users", however inexperienced with design, are guided to generate a category tree or ...


3

It's time to conduct a competitive heuristic evaluation. Your question has described most of the solution, so I'll talk about how you put those together. First, if you haven't yet, write up the main use cases for your IDE. If your use cases are too broad ("an experienced developer will maintain ~100k lines of Python code"), then you add in additional ...


3

However, it seems that in the social sciences the terms refer to the type of methods (qualitative being associated with words and quantitative with numbers) Not just the social sciences. That would be my expectation in the UX field - or any other come to that. The words have meaning - they're in dictionaries and everything ;-) Redefining them will only ...


3

There are 3 methods to take notes: 1. Hi-tech notes: using computer / tablet / mobile you mentioned it in your question. 2. Pen and Paper: Widely used but not recommended because you might miss some issues the user experienced while you are busy noting them down. 3. Record and analyze: keeps you focused, gives you an option of watching it n number of ...


2

I feel like the previous respondents covered your question pretty well, and I don't have much else to add on the question of quant vs qual. I did, however, read the article you linked to. I disagree that there is a need to label or identify research by whether it is discussing what is happening vs why it is happening. (If anything, your categorization of ...


2

Running qualitative and quantitative in parallel like you described above is an interesting idea. In my experience however the benefits from a quantitative study come fom the large sample sizes, tight margins of error, and generalizability. How might you account for trade off between large sample sizes in quantitative studies but the money and time it takes ...


2

No method is the best for what you want to accomplish. There are many different methods that you could choose, all of which will provide you with information that you can use to improve your user experience. None of the methods will provide you with perfect information. You could conduct 1:1 interviews with your current users to learn about how they use ...


2

The ten questions of SUS are not "quite a lot to ask" if you only administer the SUS questionnaire once at the end of all the tasks (which is how it's meant to be used). The thing with the SUS is that although there are 10 questions, they all fit the same basic format ("here is a statement, and a 5 point scale of strongly-agree to strongly-disagree"), and ...


2

The Principle The principle that you can’t change quantitative into qualitative refers to a conversion of the same data, not, say, using the results of one study to feed the design the next. Qualitative analysis can follow quantitative analysis, with the qualitative analysis being either in a different study, or even in the same study. What you can’t do ...


2

In general, quantitative data can only be transformed to qualitative data if there is a high correlation between the quantity and the quality. In reality there are many other factors (e.g. cost, season, advertising). E.g. if number of likes implied how good something is, you could compare number of likes to calculate quality. In your example, perhaps the ...


2

There is no standard, but there are standard approaches There are instances where you need to enumerate emotional reactions to a product. For these situations, designers have borrowed from well-known psychometric techniques to get the job done: Because words used to describe emotions/reactions are often subjective, researchers often use an intensity scale ...


2

I hope your question will be strictly answered from a UX position, although i'm a bit skeptical as technical debt is barely popular in the tech side. I have found a paper named An Analysis of Techniques and Methods for Technical Debt Management: A Reflection from the Architecture Perspective written by Carlos Fernández-Sánchez, et al. It enlists many ...


2

Based on the effort and process required in the design phases of UX projects, I would suggest that a simple way to estimate the amount of design debt can be calculated as follows: Human resources time taken to recruit (if done externally) the designer or allocate (if done internally) staff to the task time spent on the initial design tasks/activities by ...


2

Since you are basically testing for motivation and how the logic of the questions matches up with the mental models of the users, a basic paper form will still do the job. An higher fidelity of this approach will be to use a basic HTML form and then install Hotjar to do a form field analysis. The insights this will give you include "time spent on field", "...


2

Qualitative research is good to use at the beginning stages of product development (Interviews i.e.) because you want to gain insights in pain points from the user, that your product can ideally solve. Its also often used after prototyping or design part and prior to development, when you want to test overall design of the product. Quantiative on the other ...


1

If you want to measure the cognitive workload the most famous scale is NASA-TLX scale. SWAT is other one which is widely used. However they are multidimensional which means mental workload is calculated based on more than one scale, unlike Cooper-Harper. If you want to measure the usability you can use SUS scale, which is the widely approved and used for ...


1

Ethnography traditionally involves long-term observations of a culture, and Action Research is understood as when you want to invite research into your actual project. I think for your research environment setup, especially if you have a specific project, demographic study, focus group, etc., you are leaning towards Action Research instead of Ethnography.


1

Keeping Feedback: Probably Intercom - It's a great tool to manage all the feedback that comes in. Analysing the data: The problem with unstructured user feedback is that it's (mostly) qualitative, and the analysis of qualitative data is notoriously time-consuming. The most common method used for analyzing qualitative data is thematic analysis. It strives ...


1

For me, it depends. I used to store all of my feedback so I could look for patterns, which helped me avoid catering to a squeaky wheel. However, doing so means that the feedback can sit somewhere, untriaged. I think not using feedback in the near future is worse than risking listening to one point that not everyone shares. So now I tried to pull in some ...


1

On the UX side I typically have the following principles: don't listen to the feedback, look at the problems users share listen to solutions offered but make sure that whatever is used follows the principles of the product goals organizer user feedback so each piece tells a complete story so the analysis is comprehensive Putting my product management hat ...


1

Those are completely different methods of collecting data. In general, you cannot transform qualitative data into user-reported likert scale-type data. What you can do instead, if you want to report with numbers, is to create a grading scheme yourself, using elements you think are pertinent to whatever grade you give (very much like grading an assignment). ...


1

I think that providing them with a predefined list may be somewhat going against the idea of qualitative evaluation, it's already a step towards the quantitative approach. In qualitative research it's customary not to cage the subjects into a closed set of answers, but to prefer open-ended questions. A standard qualitative process that can help you ...


1

Well if you want to remove reader bias, you can try feeding all of the responses into a word cloud generator. That way keywords will stand out. And perhaps there's enough there for you to spot trends within your responses.


Only top voted, non community-wiki answers of a minimum length are eligible