I believe it is a good idea to conduct user interviews to understand experience of first users soon after they try out the site. What are some types of helpful question examples that might be relevant? What's a healthy # of users to try to conduct interviews with after BETA launch?
I usually kick off with around 12-15 users. In terms of what questions to ask really depends on the type of site you are building. For example, user interview questions for an eLearning app could include the following:
- How they learn a new concept online. What steps do they follow?
- Describe a typical day at school/work.
- What makes a good day in their current routine? How about a bad one?
- What activities waste their time?
- How do they approach problems and how do they solve them?
- What shortcuts do they use on web/apps?
- How they expect a course to be structured
- What expectations do they have from a learning app?
- What features do they expect from the app?
- What features they think would be great but don't necessarily expect?
- What do they enjoy about the experience?
- What issues do they face?
- When might they use the app?
- Where might they use it?
- Would they like to use it in a group experience?
Here are some brilliant resources that will help you plan the process: