We are building a fit finder for a client that enables users to find out whether or not there is a career that fits their competencies, their soft skills and hard skills. But we don't want to lose them with too many questions. Is there research that tells us at what point do users drop off when there is too many questions?
The little research I found has this to say
"Keep the survey short so that the user can finish it about 5-10 minutes", that said,there is no definite point at which all users will drop off but there is always an increase in the drop off rates as the number of survey questions increase. To quote this article from survey monkey
So what did we find?
As expected, the more questions per survey, the higher the respondent drop-off rate from start to finish. However, as can be seen in the graph below, the relationship between survey length and drop-off rate is not linear. Data suggests that if a respondent begins answering a survey, the sharpest increase in drop-off rate occurs with each additional question up to 15 questions. If a respondent is willing to answer 15 questions, our data suggests that the drop-off rates for each incremental question, up to 35 questions, is lower than for the first 15 questions added to a survey. For respondents willing to answer over 35 questions in a survey, our data suggests they may be indifferent to survey length, and are willing to complete a long survey
That said, this article on Mobile surveys suggests that 15 questions is the limit while asking users to perform surveys on a mobile device. To quote the article
15 questions is enough
While we advise to ask as few questions as possible (but as many as necessary) the recommendation is to stop at 15. To many this is a surprisingly high number of questions as the conventional thinking goes that people lose interest quicker on a smaller screen. Mobile surveys can be fairly long without seeing much drop-off in completes.
Every question beyond 15 cuts response rate by 5-10%.
Keeping your survey shorter, using routing to ask more questions only from a sub-set of the respondents is recommended. Add more questions only if you absolutely have to and your audience is large enough to compensate for the higher drop-off rate.
Use as few as possible - and provide convenient places to stop the survey. THEN ask the user if he would like to continue. You can get more involved by letting the user chose among a set of questions to answer (assuming you have a series). And you can get even more creative but the gist of it is: AS LITTLE AS POSSIBLE.
You need to let the user opt in again and again.
From what I've observed (in informal testing) people start dropping off at around 5 UNLESS they are keenly interested in the topic and providing answers. Granted the testing I've done revolved around commercial sites trying to find out more about the preferences of their customers.
This is a very domain specific question. People's attention will depend on how much they stand to gain from it. If this is a high school guidance counselor's app, the tolerance for more questions may be different than an app for an unemployment counselor.
I'd study your current outcomes. Can you rate the quality of a specific test, such as "75% of people were satisfied with the results"? If so, perhaps you can identify a subset of core questions that correlate to the most satisfied people. That could help lead you to divide your questionnaire into two sections: recommended and optional. People with more incentive to continue could get more refined or better tailored results, while those with less desire might still get adequate results.
Luke Wroblewski provides a good insight on this one. To summarize:
- Forms are awful and user hate filling them in. The fewer questions the better.
- These questions genuinely help the user experience, each is worth the user's time in the long run. The more the merrier.
The answer to this contradiction is not to find a trade-off, but to realize that the user views the website like it views a person. Too many questions up front and the user gets defensive. It's just like in a conversation between two people. For every question, you ask you say something about yourself, or add some 'content' (a joke, an insight, etc).
So the trick is to spread the form throughout the user's use of the website. Let them make a profile with no questions whatsoever (just an e-mail address is good enough, ask for their password on the confirmation page). Let them get some content, show that your site is useful, and add an option to let them give you information when they feel like it. Let them know it'll enrich their experience and show them if you can when a suggestion is a direct result of some questions they answered. This is a good use case for gamification.
This option gives you the best of both worlds. Zero bounce rate, and a potentially infinite number of questions. I don't know if it would work for your context, but it's worth considering.