20 votes
Accepted

What to do if SUS scores contradict qualitative feedback?

How can this discrepancy be reconciled? You have divergent results because the number of participants is small and not representative. There is no randomization or blinding to prevent bias. You're ...
習約塔's user avatar
  • 1,885
11 votes

What to do if SUS scores contradict qualitative feedback?

How to reconcile the discrepancy? That I can't tell but here is why it might have happened. The "5 users will find everything wrong with your system" refers to usability problems test subjects will ...
locationunknown's user avatar
7 votes

What to do if SUS scores contradict qualitative feedback?

Look at your confidence intervals: the "real" score for v1 is somewhere between 58 and 88, while that score for v2 is somewhere between 51 and 79. Most notably, the confidence interval for each score ...
Mark's user avatar
  • 1,875
3 votes
Accepted

How to perform user interviews for a new product when there are IP restrictions

The question "Due to the actual cost of gas, would you like your car to run on a cheaper kind of energy rather than gas?" will not , on it's own, cause any kind of intellectual property issues. It ...
Roux Martin's user avatar
  • 14.2k
3 votes

Is asking 'balanced' leading questions the right way during user interviews?

Asking closed questions is not optimal, for the reasons pointed out by Kristian - they skew the results. I think Adrian has a point too, since getting feature-specific feedback might be a better fit ...
Sami Surakka's user avatar
3 votes

How do you uncover true motivations and goals during user research?

I think there's something here to be said for not asking 'why' and trying to understand their true rationale deeper. In a workplace context their entire rationale possible is that, and knowing why won'...
Susan's user avatar
  • 31
3 votes

UX Researcher Hiring Challenge Ideas

I would strongly recommend you pick a research problem that is relevant to your organization, perhaps one that you've already finished, and have the research candidate walk through it face-to-face in ...
Izquierdo's user avatar
  • 12k
2 votes

what kind of questions should i ask the user during the user interview to get insights for revamping website on-boarding?

UX pros mistrust surveys since survey answers are unreliable. People tend to mis-remember things and provide answers that are not-quite accurate. (Not that people lie on surveys. We all tell certain ...
Ken Mohnkern's user avatar
  • 9,034
2 votes

Test to evaluate a UX designer in product company

Often times it's not advised to have a design candidate solve an actual problem your company is trying to solve. This can be seen as spec work, which in the industry, is considered a no-no. Instead, ...
Aaron James's user avatar
2 votes

Interviewing a Single User About an Existing UI Improvements

One person's feedback will not be determinative, because this user can be an outlier. Depending on the complexity of the application you might need at least 5 people to uncover most problems. That ...
TRoh's user avatar
  • 71
2 votes

Is asking 'balanced' leading questions the right way during user interviews?

One approach that can be useful for getting a meaningful response is to ask the user to place their response on a sliding scale between two options that have the same level of "goodness" or "badness", ...
Adrian Long's user avatar
  • 1,245
2 votes

Is asking 'balanced' leading questions the right way during user interviews?

When you ask leading question you are biasing the responses because you "prime" the users to answer you in a particular direction. When you ask "Is clicking on this button difficult for you to ...
Kristiyan Lukanov's user avatar
2 votes
Accepted

UX Interview Questions

"How many long have you been using [the product]?" - what do you expect this question to reveal? "How would you rate your overall familiarity with [the product]?" - what do you expect this question ...
drabsv's user avatar
  • 372
2 votes
Accepted

Interview brief wording advice

Let's break this down. ...registration Call to Action "component"...that encourages registration ... [and] could be repurposed in the header, footer, middle of a page - wherever you propose "[...
Nate Green's user avatar
  • 2,473
2 votes
Accepted

App on-boarding via contextual imagery

May I know the goal of these questionaires? are you trying to personalize the feeds base on user's choices or? You have mentioned that there will be a simple animation prior to these questions, so I ...
Chiali Tsai's user avatar
2 votes

Interview methods for user needs phase

Your initial approach seems like a good start. Have a look at the following resources hopefully this will give you some guidance: Is asking 'balanced' leading questions the right way during user ...
ntnlbd's user avatar
  • 344
2 votes

What to do if SUS scores contradict qualitative feedback?

I think the most important thing is to realize that using a combination of qualitative and quantitative analysis methods is going to give you the most complete best picture of what the users are ...
Michael Lai's user avatar
  • 26.7k
2 votes
Accepted

How much incentives is too much?

The Nielson Norman Group is an excellent resource on a wide variety of UX topics. In this case they've actually conducted a study on how people incentivize recruits, among many other things. They ...
Benjamin Jex's user avatar
1 vote

Questions in Usability Testing without giving any lead

Use task based activities that require the use of the feature you want to test and base your questions around the problem that our feature sets out to solve rather than the feature itself. Lets say ...
Roux Martin's user avatar
  • 14.2k
1 vote

Questions in Usability Testing without giving any lead

When testing usability, think of which hypothesis you want to validate and which scenarios you have to work with. Based on those, create tasks that will help you measure how well you perform or solve ...
Ainhoa Ostolaza's user avatar
1 vote

2nd UX interview with Agile guy

I have only experienced working in an agile once and it wasn't fun. In user research we call this a small sample size. ...[they] did not give me a seat at the table. And no one ever will. You ...
Luke Smith's user avatar
  • 1,537
1 vote

UX Interview Questions

Instead of asking "Which of the below do you do most often? (select multiple if applicable)*" and "What are the most common tasks you do when using [the product]?", I would ask "How often do you ...
Janet's user avatar
  • 467
1 vote
Accepted

Should I tell interviewees remaining time for tests?

Interesting question and observation! I feel like the answer to this would be dependent on what you want to achieve from the test and how you want to achieve it. For OBSERVING the Psychology of the ...
Shreyas Tripathy's user avatar
1 vote

How many people should i interview for my project realization?

Is it possible to keep the system updated throughout the stores? If yes, launch with the current feedback level. Then you can improve the same based on further feedback. If the regular change is not ...
jitendragarg's user avatar
1 vote

Should I do a redesign as part of hiring process?

I don't know about the ethicalness, but I see the same red flags you do. They're asking you to do free work for them with no guarantee of them offering you the job, and it appears the job isn't even a ...
Ken Mohnkern's user avatar
  • 9,034
1 vote

Evaluate web application mockups

As you said yourself individual user testing would be better. Focus groups tend to be more about what the users want from the system and less about how users actually interact with the system. ...
Socrates Kolios's user avatar
1 vote

How to perform user interviews for a new product when there are IP restrictions

+1 on using an independent researcher to front for your company if you don't want participants to make assumptions about your product plans. Also, I realize your example interview question may not be ...
Ellen Francik's user avatar
1 vote

Interview brief wording advice

They're asking you to wireframe the site and show where you would put a registration link, whether that's a menu item or a tab that persists across the site but doesn't interfere with its use. Look ...
DarrylGodden's user avatar
  • 6,306
1 vote

How would identify the fake participant(s) when you conduct individual interviews?

Generally the agencies that recruit on your behalf would rely on you to provide the criteria for selecting/screening participants, so the onus is really to make sure that you provide tasks that will ...
Michael Lai's user avatar
  • 26.7k
1 vote

Hardware best practices for data entry while doing field work

This reminds me of a certain company in the medical field that introduced a way for patients to fill out their intake forms on a touch screen tablet while in the waiting room. It sounded like a good ...
ToddBFisher's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible