Avoiding negative or positive biases during interviews

Question

I'm going to be conducting user surveys and I want to ask questions like "What part of the software is good?" and "What part of the software is bad?" (more specific to the software in question of course). I'm going to make the interview conversational, but I don't want to come off sounding like "let's make fun of this old ugly software". Some parts of the old software is actually really good and I want to extract that information from the users.

Are there any techniques that I can use to prevent any negative or positive biases when interviewing?

Answer 1

First off, let's assess what you're trying to do.

The subject of this research is old (maybe obsolete) software. What you're looking for in this software is answers to "what patterns to drop", and "what patterns to keep".

Since users take good patterns for granted, as they should go unnoticed, you probably will want to focus on what the bad ones are.

Don't see this as "making fun of this old piece of software"! You'll have to make sure your users mold their negative feedback properly though. "Feature X sucks!" doesn't help. "Feature X didn't behave as I expected it would" however does help you.

It is imperative to set a challenge to your audience that allows them to focus their feedback (and if the software is exceptionally frustrating, keeps their spirit up as well!).

For example, you could precondition their experience with "This piece of software is rubbish. We know that. But we'd love you to have a look at it and define why certain things are bad. The why and what are more important than the fact that it's less-than-ideal."

TLDR: Acknowledge and address the inferiority of the product, and help your guinea pigs to focus on reporting useful feedback.

The form of the feedback (open questions, multiple choice, etc) depends on the size of the audience and the reporting tools at your disposal.

Answer 2

You have to explain in the beginning before the actual interview that the subject of the interview doesn't critisize or praise you as interviewer. You have no interest in whether or not it's good or bad. You just need a plain honest answer.

That way you make it obviously clear to the subject what this is all about. And don't forget that there are no wrong answers in interview. There are just answers from a specific user. No more, no less.

Answer 3

I am a fan of the how would you rate this experience 1 - 5. Good to bad that seems to be pretty neutral .

Also open ended questions are neutral.

Lastly make 2 versions of the survey one negative and one positive and see the difference in regards to any correlation you may observe .

Answer 4

They key is to remain neutral and do not ask any questions that generate bias. Here's an example that relies on attributes rather than Questionnaire bias or Interviewer's bias:

Incorrect way of asking: "Are you satisfied with this service?"

Correct way( Unbiased ): "With what aspects of this service are you satisfied?"

The latter one gives good account of information on the attributes, which are the essence of a survey.

In addition to this, I would strongly recommend using Likert scale for the survey. It is the most unbiased question form that you can create. Whenever the items are related you can use it to get results.

Tone down your question from "Do you prefer XYZ, such as YXZ, to be more efficient?" to "I would prefer XYZ, such as YXZ, to be more efficient" with radio buttons providing an options of "Strongly disagree", "Disagree"....., "Strongly agree".

Clifford Nass, HCI expert further describes how to creatively use and perform research with the help of computer in his book The man who lied to his laptop