Tag Info

Hot answers tagged

11

In addition to the HEART metrics (Happiness, Engagement, Adoption, Retention, Task Success) we also measure 1) new user adoption by whether or not training costs decreased and 2) existing user adoption by whether help calls decreased. I'll try to answer your follow up question and to do so you must first understand the personality of the user's you're ...


9

In Why You Only Need to Test with 5 Users Jakob Nielsen suggests: The best results come from testing no more than 5 users and running as many small tests as you can afford. However, rather than focusing on the number of users, it might be better to focus on the number and quality of the tasks: Usability test tasks are so critical that some people ...


6

Check out Skype's Screen Sharing feature (it's free). I'm planning on using Skype for this very purpose in a couple months. I'll be writing a "Skype Screen Sharing with Steve" blog post to try to encourage participation. I'm thinking the invitation should be up front about the fact that you're looking for feedback and are curious about uncovering ...


5

You’ll find a taxonomy and descriptions of different ways of integrating qualitative and quantitative methods in the “Mixed Methods Procedures” chapter in John Creswell’s (2013) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (SAGE Publications). The chapter includes examples of actual studies. The book is intended for social ...


5

A few possibilities come to mind: Heuristic analyses (requires 5-6 fellow HCI practitioners in order to be comprehensive) Validation against personas or behavioural goals (may not be reliable) GOMS / KLM analyses - basically your first bullet point (reliable, but doesn't uncover issues with IA or broken interface metaphors) Comparative assessment ...


5

I had this problem as well a short while ago. What I did was write out a list of tasks and have participants record a screencast of themselves doing them, while thinking out loud. Later on when you watch the screencast, if there are parts that you would want them to re-do, you could ask them to do a second round. What is the advantage for you in ...


4

There are no absolute 'fundamental' metrics. All metrics should be decided on a per-project basis, and be grounded in explicit business objectives. That way, designers will have explicit priorities and clients will understand the compromises we make to achieve them. So, for example, to look at some items of your list: What web browsers are most commonly ...


4

Definitions While researchers still argue about the value of the qualitative and quantitative approaches, their definition is rather universal and agreed upon: Quantitative - conclusions are derived by means of numerical analysis. Qualitative - conclusions are derived by non-numerical analysis means. There is also the mixed approach, which means ...


3

It's time to conduct a competitive heuristic evaluation. Your question has described most of the solution, so I'll talk about how you put those together. First, if you haven't yet, write up the main use cases for your IDE. If your use cases are too broad ("an experienced developer will maintain ~100k lines of Python code"), then you add in additional ...


3

Since a user rarely experiences just a page of your site by itself, I'd say a per-page evaluation is of questionable value. All that said, 'ux evaluation' can be done a number of ways. User testing, user surveys, heuristic evaluation, etc. For starters, I'd start with a heuristic evaluation.


3

It depends what your goals are, and how you assess the different risks in what you're testing. Some things to consider: How do your usage patterns break down? Are the majority of your users coming in on mobile devices? Then you should probably focus more of your testing resources on that breakpoint. You could keep the same number of participants - but run ...


2

I don't have a reference for this, but I would think that these numbers would depend on the size of the user base. For qualitative testing you need to have "typical" users. So if you have 3 roles your users can take you need at least 3 users - one for each role. In reality you'd want more than one, but this is your absolute minimum. For quantitative ...


2

I recently read an interview with Jonathan Ive (Head Apple designer) and a few paragraphs struck me as cutting to the heart of this issue: Q: Do consumers really care about good design? A: One of the things we’ve really learnt over the last 20 years is that while people would often struggle to articulate why they like something - as consumers we ...


2

You still can do it with the same number of participants as you would do with a single test -- it just takes longer to test each user because you're showing them more material (i.e. three breakpoints instead of just one). However, you should "shuffle" and randomize the order in which you test each breakpoint so you don't skew the results due to familiarity ...


2

It depends on your specific case. The key factors are how strongly qualified your incoming traffic is, what the risk to value proposition is, and what obstacles occur in your conversion funnel. Attrition rates upwards of 70% are probably normal - sometimes per step in your funnel. You're going to lose a bunch to gotchas which the user didn't see before ...


2

Something else to consider is where the result of your usability report is going. How much work can the people downstream from you fix - and what will the effect of those fixes be? Say I run a test with fifteen people. After the first three I've spotted problems A B C. By the end of the fifteen I've also spotted problems D E F G. The problem is that the ...


2

However, it seems that in the social sciences the terms refer to the type of methods (qualitative being associated with words and quantitative with numbers) Not just the social sciences. That would be my expectation in the UX field - or any other come to that. The words have meaning - they're in dictionaries and everything ;-) Redefining them will ...


2

I can only speak from personal experience, but I often run mixed-methods research that combines surveys and qualitative data. This gives you complimentary data such as "Quantitatively, users preferred Site A to Site B. The focus group explains some of the reasons why they preferred Site A."


2

Unless you have a large budget, alternatives to having a person (or group of people) parse the data are potentially complex and unreliable. Card Sort a Folksonomy Card sorting is a simple technique in user experience design where a group of subject experts or "users", however inexperienced with design, are guided to generate a category tree or ...


2

No method is the best for what you want to accomplish. There are many different methods that you could choose, all of which will provide you with information that you can use to improve your user experience. None of the methods will provide you with perfect information. You could conduct 1:1 interviews with your current users to learn about how they use ...


1

The Principle The principle that you can’t change quantitative into qualitative refers to a conversion of the same data, not, say, using the results of one study to feed the design the next. Qualitative analysis can follow quantitative analysis, with the qualitative analysis being either in a different study, or even in the same study. What you can’t do ...


1

In general, quantitative data can only be transformed to qualitative data if there is a high correlation between the quantity and the quality. In reality there are many other factors (e.g. cost, season, advertising). E.g. if number of likes implied how good something is, you could compare number of likes to calculate quality. In your example, perhaps the ...


1

I feel like the previous respondents covered your question pretty well, and I don't have much else to add on the question of quant vs qual. I did, however, read the article you linked to. I disagree that there is a need to label or identify research by whether it is discussing what is happening vs why it is happening. (If anything, your categorization of ...


1

Actually you are using humans for the evaluation of your novel UI, ain't you one? :-) I'm sure there are a few more humans with you that might be helpful for this task. The problem with you and the other humans around is that you know things about your UI that would slant the results of a naive user test. To overcome the IRB restrictions and the slant thing ...


1

Possibly how keyboard friendly it is. It is amazing how much time you spend removing your fingers from home row, moving the mouse, clicking then repositioning your fingers again.


1

Ethnography has been 're-calibrated' for Computer-Mediated Communication and related complex cultural phenomena, because anthropology, sociology and psychology (to name some) have never stop analysing them. Technologically mediated environments prevent researchers from directly observing research participants. The online environment requires ...


1

In academic (computer science) circles this is known as affective computing (link to Wikipedia article). From a computer science perspective, the challenge is first to characterise the user's behaviour/emotion while they are experiencing/using the system, and then classify this feature vector (a list of numbers describing the behaviour) to belong to a ...


1

My intuition says that you should have separate tests for the different break points. If we only consider two scenarios (small-mobile, large-desktop) then you are targeting two distinct input devices (touch vs mouse). You may find that your tests look different for touch devices, than for desktop devices (the tasks may differ, or priorities change depending ...


1

A few other question ideas: How frequently users come to the site (several times a day?). Once a week? What percentage of users contribute the content? Do contributors usually contribute to one topic or is there a lot of variety? (e.g. Jack adds comments only to posts related to environment, Kate comments to posts about cognitive science and gardening). ...


1

For quantitative testing, it's possible to be more explicit about the effect of the sample size on your results but the number of users you need depends on the particular tests or analyses you are considering (examples could be determining the proportion of participants successfully completing a task, estimating the average time-on-task, comparing two ...



Only top voted, non community-wiki answers of a minimum length are eligible