User stories—how do I validate them with actual research?

Question

My PM has a list of user stories. We want to find out if they're "true"—basically, if these user stories have evidence to support them, or not.

We're already pulling usage data. I'm also going to talk to our customer support team to glean through customer support tickets/notes from customer support calls (to see if our customers have, in fact, expressed a need for our new feature).

Have any of you used interviews to validate (or refute) user stories? If you have, how have you conducted a good interview without asking leading questions? (or worse, the very unhelpful "Would you use this feature?")

Other qualitative methods to research user stories?

Ideas?

Answer 1

It is very common for Project Managers and Product Owners to have a list of User Stories beforehand, which they use to determine the direction of product development. These preconceived stories are assumptions.

Your task is to validate these assumptions.

Good to know that as a first step, you have a support team that can give you some customer feedback data. But you need to be a little careful because -

1. Customer feedback is not necessarily UX related feedback, so it will be very time consuming for you to skim through the data to find relevant findings
2. Customer feedback may be historic and outdated - and after a passage of time, the data may not be relevant anymore because user needs may change with time

However, you may find it more useful and fruitful to conduct your own user research. There are some methods you could use to conduct your research but it depends on time/budget that you have available -

1. Non-directed interviews - This is good when you don't want to directly ask them questions specific to any features. This will be useful for you since you want to test assumptions. For example, instead of asking 'Would you use this feature?', you can ask 'Can you talk a little bit about how you spend your day?'. Your main objective is to understand the user's behaviours, habits, expectations, environment, cultural background etc, all which really helps to understand the user's needs. Make it very clear to the participant that the interview is not to test their knowledge but to help you improve the product.

2. Widely distributed Survey - This helps when you have less time, very large user base and you need to validate your assumptions quickly. You can mix up the questions to be specific and non-specific and get quick insights from a lot of users. However, you may miss out on the human touch that interviews have to offer, and also it's difficult to follow up if you need further understanding of your user needs.

In a time constrained environment, you will find yourself do a bit of both, and that is fine too, as long as you have enough information from your users.

Answer 2

This approach appears to be a little backwards and is no doubt being directed to you by your PM. Generally, you want to use research to discover what user stories exist. In this case, I would sift through the research and find what user stories there are and their frequency. You will then have a fairly concise and useful list. You can then cross-check the actual stories list with what your PM is suggesting.

This would be a better approach then if you were just to scan the research to see if they exist. This is because if you find out that your PM's user stories do not exist in reality, then you have a list of which ones actually exist. If there were indeed matches and because you would have also marked the frequency, you could even tell your PM which stories are more common amongst your users in addition to informing them of ones that they weren't aware of.

Answer 3

I have found this model to be very effective:

• Executive teams set business goals and objectives. These are used to formulate a company and product strategy
• PM's help define that strategy by identifying and validating the problems to solve.
• UI/UX identifies and validates the solutions to those problems, from the users' side
• DEV identifies and validates the solutions to those problems, from a technical side

If your PM gives you requirements to solve problems that actually do not exist you will see signs. During user testing, users will say things along the line of, "that's not what I need," or "that's cool, but..."

To avoid designing for these "false" problems, I focus first on:

1. Defining the "Jobs to be Done"
2. Creating a "User Empathy Map"

If the jobs users are actually trying to accomplish do not match the problems I am solving, I seek clarity from the PM.

Last, if your PM is giving you solutions to the problems, then he may be out of his job scope. Solving this issue can be a challenge, but it can be done.