2

I'm going to be conducting user surveys and I want to ask questions like "What part of the software is good?" and "What part of the software is bad?" (more specific to the software in question of course). I'm going to make the interview conversational, but I don't want to come off sounding like "let's make fun of this old ugly software". Some parts of the old software is actually really good and I want to extract that information from the users.

Are there any techniques that I can use to prevent any negative or positive biases when interviewing?

| improve this question | |
2

First off, let's assess what you're trying to do.

The subject of this research is old (maybe obsolete) software. What you're looking for in this software is answers to "what patterns to drop", and "what patterns to keep".

Since users take good patterns for granted, as they should go unnoticed, you probably will want to focus on what the bad ones are.

Don't see this as "making fun of this old piece of software"! You'll have to make sure your users mold their negative feedback properly though. "Feature X sucks!" doesn't help. "Feature X didn't behave as I expected it would" however does help you.

It is imperative to set a challenge to your audience that allows them to focus their feedback (and if the software is exceptionally frustrating, keeps their spirit up as well!).

For example, you could precondition their experience with "This piece of software is rubbish. We know that. But we'd love you to have a look at it and define why certain things are bad. The why and what are more important than the fact that it's less-than-ideal."

TLDR: Acknowledge and address the inferiority of the product, and help your guinea pigs to focus on reporting useful feedback.

The form of the feedback (open questions, multiple choice, etc) depends on the size of the audience and the reporting tools at your disposal.

| improve this answer | |
  • I agree with you on this. Getting feedback from fan-boys won't be of any help. – Pj_ Jan 13 '15 at 23:54
  • To agree, feel free to up-vote ;) Also, it's not necessarily fending off fanboyism. It's more about getting feedback that's actionable, regardless of positive or negative. Getting the bad patterns however is easier, since it triggers a reaction, where good patterns might not trigger anything at all. – Dirk v B Jan 14 '15 at 4:40
  • Actually, Fan-boys are often drive the wrong results. As from any good product development strategy, we can find illustrations where the first customer tide is often confused because of these fan boys. I feel the same can be replicated in this case. Upvoted ;-) – Pj_ Jan 14 '15 at 8:18
  • Perhaps "All software is rubbish, we're trying to improve things" rather than "This piece of software is rubbish, we're trying to improve it" - make it more general and people will think about other good/bad software they've used, which will give a frame of reference. – Jon Story Jan 14 '15 at 9:59
  • Note that those lines are an example and indeed need tailoring to the specific case and audience! – Dirk v B Jan 14 '15 at 21:16
1

You have to explain in the beginning before the actual interview that the subject of the interview doesn't critisize or praise you as interviewer. You have no interest in whether or not it's good or bad. You just need a plain honest answer.

That way you make it obviously clear to the subject what this is all about. And don't forget that there are no wrong answers in interview. There are just answers from a specific user. No more, no less.

| improve this answer | |
0

I am a fan of the how would you rate this experience 1 - 5. Good to bad that seems to be pretty neutral .

Also open ended questions are neutral.

Lastly make 2 versions of the survey one negative and one positive and see the difference in regards to any correlation you may observe .

| improve this answer | |
0

They key is to remain neutral and do not ask any questions that generate bias. Here's an example that relies on attributes rather than Questionnaire bias or Interviewer's bias:

Incorrect way of asking: "Are you satisfied with this service?"

Correct way( Unbiased ): "With what aspects of this service are you satisfied?"

The latter one gives good account of information on the attributes, which are the essence of a survey.

In addition to this, I would strongly recommend using Likert scale for the survey. It is the most unbiased question form that you can create. Whenever the items are related you can use it to get results.

Tone down your question from "Do you prefer XYZ, such as YXZ, to be more efficient?" to "I would prefer XYZ, such as YXZ, to be more efficient" with radio buttons providing an options of "Strongly disagree", "Disagree"....., "Strongly agree".

Clifford Nass, HCI expert further describes how to creatively use and perform research with the help of computer in his book The man who lied to his laptop

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.