Coming up with questions for user research

Question

The biggest problem for beginners at chess is remembering how each piece moves. I designed a set where the form of the piece itself provides a clue to the way it moves:

I would like to turn this into a chess app, and I'm researching my audience. I would like to interview people at an online and offline chess community, to establish need and gain insight into typical use cases.

What's the best way to come up with some questions for surveys and interviews for this type of user research? What type of questions would help me gain good qualitative data to establish need for such an app and define its audience?

Answer 1

Have you considered talking to chess instructors? They should be able to give you a top-10 list of common problems beginner chess players face (across different age groups). This way, you don't have to limit your app to only address the piece-movement problem. You can even expand it to include simple chess puzzles, or introduce the player to the other chess movement edge-cases such as en-passant and castling (neither of which are solved by improving the piece-design).

You will probably encounter different schools of thought with regards to teaching chess, but at the very least it should provide you with a clearer view of the needs of the beginner chess community.

Edit: One other option is to spend a lot of time at places that teach beginner chess, and see how much time they spend on teaching piece-movement, and how many players struggle with it. Observe how many questions they ask their trainers, and what they struggle with. This way you don't have to ask any questions. You see what questions other people ask about the problem domain.

Answer 2

Start with your objective. What do you need to learn to find success? Is it about whether people have a need for such a solution? Or for a solution, what they would value?

Once you identify what you need to learn, then you can formulate questions and the right methods for it.

Example: - "Who, if anyone, would use a new app that does X?"

• Define a hypothesis of who you think might and start with that persona, then you can expand to other personas as you learn more

• Avoid leading questions and use questions more like: How often do you do A? What do you use to do X today? What do you like about using X? What do you not like about using X? When doing A, what is most important to you? (Basically, formulate questions to understand how important this goal is for them, frequency, and satisfaction with what they use today. You're trying to ascertain if an unmet need exists that you can fulfill.)

• I'd start with qualitative interviews to hone your target personas as well as a list of needs you believe they have.

• I'd recommend doing a survey, if possible, where you list those needs and ask them how important they are and how satisfied they are with what they use today. The goal of using a survey is (a) larger sample size, depending how you administer it, and (b) quantitative data on the value of their needs.

• Alternatively, you can join the qualitative interview with a concept test where you ask the above open-ended questions to tease out customer needs then after that show them a representative concept of your solution then get their feedback. Just remember, reported behavior is different than actual behavior so many times customers will be biased to say they like your concept then in reality usage is low. That's why making sure there's a real unmet customer need is key.