18

My product team proposed a new feature for our mobile app and I'd like to get a sense of user interest for the feature.

I have the ability to run in-app surveys to our existing users but when running a survey of "would you want X feature?" I suspect there is a strong confirmation bias ("Of course I'd like an X!") that makes me suspicious of the survey results. What is a way to frame a feature survey differently or to analyze the results in a way that will mitigate this effect?

Improve this question
1
  • Users are notoriously bad at knowing what they actually need. Instead ask users about their experience in an attempt to find out which parts of your application need the most work. Also interview non-users to find out why they aren't using your application - this is where you can learn the most (meaning not from people who are already using your app). Great talk on the subject
    – Zach Saucier
    Sep 19, 2015 at 15:13

4 Answers 4

Reset to default
34

Instead of surveying for a particular feature, ask about the pain point the feature should solve.

A great question would be something like "What is the worst thing about our app?" And then give three options.

Remember, users are crap about giving good suggestions about solutions, but nobody knows their problems more than they do.

Improve this answer
0
31

You could always try asking your users to prioritize a list of 5 or 6 features - That way you're not asking "would you like to have X?" but "Which is more important to you U, V, W, X, Y, or Z?"

There's still a little of the confirmation bias there but you're also asking the user to trade off one feature for another and so reducing the effect of the bias.

Improve this answer
7
  • 25
    By randomising the order of the choices for each participant, the notorious "first option is prefered" bias can be easily removed.
    – ojdo
    Sep 17, 2015 at 17:59
  • @ojdo How does that help if each participant is more likely to choose the first option? You just spread the answers without gaining knowledge. Maybe rank top 3?
    – musiKk
    Sep 17, 2015 at 19:19
  • 8
    @musiKk spreading the answers as ojdo suggests flattens out any anomalous answers (like first option bias) and allows the values from the less lazy participants to show through
    – Roux Martin
    Sep 17, 2015 at 19:29
  • Make the first option a nonsense option that's never going to be implemented anyway.
    – Mr Lister
    Sep 18, 2015 at 16:27
  • @MrLister adding spurious options also has the potential to skew the result. How can you be sure that the false option isn't one that your users really DO want?
    – Roux Martin
    Sep 18, 2015 at 16:55
6

An approach I have used in surveys before is the open-ended question 'If you could make one change to the application, what would you do?'. This doesn't bias the user and has produced some really useful data about user's own concerns without unduly influencing them. For numbers of respondents up to a few hundred, the workload associated with analysing the data is reasonable. I tend to analyse by tagging and categorising the results.

Improve this answer
3
  • 1
    So those are great but as you noted, once you have more than a few hundred responses the cost for analysis is too high.
    – SAR622
    Sep 17, 2015 at 14:50
  • Well, maybe. You can then start using text processing tools to get a general sense, or just divide the work up among more people. I
    – Peter
    Sep 17, 2015 at 17:59
  • @SAR622 Wouldn't you first run a survey with just 100 respondents and then use a multi choice question in a second main survey (if you need more verification)?
    – David Mulder
    Sep 18, 2015 at 19:19
-3

Depending on the feature you could add a button for that feature (e.g. Share with a friend, if that's the new feature) in your navigation / interface and when people click the button you can show them a message that the feature is currently not implemented but soon will/might be. You can then gauge how many of your users actually click this button, and see this as a indication of how popular said feature will be. If the number of users clicking this button is below your pre-determined threshold, you can then decide not to build this feature.

Even better, you can then target these particular users that have clicked the button for more info about what they would expect from such a feature / what they were planning to do when they clicked the button. It's better to ask people about their current/past behaviour than ask them to predict their future behaviour.

Improve this answer
1
  • 2
    Offering your users a button for a function that doesn't exist will severely damage the user's perception of the site/brand. A button that simply tells users that the function they've tried to access doesn't exist is just going to frustrate them.
    – Roux Martin
    Sep 18, 2015 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.