I'm designing a web-based program for a university community. It's crucial that we get answers to some survey questions when people sign up, so we can evaluate the program's effectiveness. The more questions we can ask, the better -- but it's a voluntary program, and we don't want to ask so many questions that people quit during signup.
If the issue were making the questions clearer or easier, I could pilot test them with a small sample of people. But I don't know how to replicate, in a testing situation, the whole experience of wanting to sign up for a program vs. getting annoyed by too many questions. Is there an accepted way to do this?
Alternative options we've considered:
- A colleague recommended doing quick A/B testing during the actual signup period. Unfortunately, we're planning a big universitywide rollout with many of the signups occurring in the first day, so we can't gather preliminary data that way.
- We could give a subset of questions during signup and send the others in a second, optional survey. cons: this would reduce the response rate and be more trouble for both users and administrators.
- We could make the less critical questions optional. cons: I'm not sure if users will realize that they can skip them, or if they'll just quit anyway.
