exploratory testing

Question

My team is currently testing a new feature that is still on the concept phase within our app for farmers. We are trying to have an exploratory test instead of a task oriented usability test and I am struggling with the idea of how open ended the test should be.

The reason for the test to be open ended is that I would like to understand what path a user will take after taking a picture of his sick crop instead of asking him to select a dealer where he can buy pesticides(the new feature. Has anyone experience with this kind of testing and ideas how is the best manner to phrase an open task? Thank you

Answer 1

If you are trying to validate a concept or determine a user's needs, you are in a generative research phase: you are trying to determine the right thing to do based on conditions your targeted users are dealing with. This is always a great place to start with something new and unvalidated, so you are in a great place to make sure you don't dive into developing something your customers don't actually want or need!

Interviews with customers that meet the persona that you've identified as the primary target of this new feature is definitely a solid way to go on this; in fact if you are trying to determine their path and actual needs, you are probably better off NOT showing something before talking to them.

There are a lot of methods for doing this kind of research - the one I'm most familiar with that seems to meet this criteria is stakeholder interviews. I've done many of them; in some ways they are easier than usability tests because there's less prep up front to some extent, and the conversations can be a little less formal and more free-ranging. On the other hand, they can be a little trickier because you want to know when to chase down fruitful rabbit holes with your participant and look for nuggets of gold in what they are saying, but you also need to keep them on task and get the answers to your well-formed questions; there is also more analysis work to be done on the back end to get the best insights out of them.

Here are two really good articles on conducting such interviews: https://www.testingtime.com/en/blog/stakeholder-interview-guide/ https://www.userinterviews.com/ux-research-field-guide-chapter/internal-stakeholder-interviews

I have just completed two rounds of this very thing for a new feature my company has been exploring, and we learned not only that what the engineers were planning to do is NOT what people want, we also learned a huge amount about what WILL benefit them. We did this work early enough in the process that the team can now pivot at minimal expense to creating something that will satisfy our users, rather than getting to the point of building something, doing usability testing, and THEN finding out no one needs it.

Good luck! I'm not a research methods guru by any stretch, but I'm happy to answer any questions I can.

The other thing you might consider doing, if you are trying to determine the path a user will take, is doing a customer journey map to learn exactly that.

Here's a great article on that, though there are many resources out there: https://www.nngroup.com/articles/journey-mapping-101/

Answer 2

I'm doing some similar customer interviews this week. It's not really a usability test -- there are no tasks -- but I will show customers an early-phase design and talk to them about their needs. So, it's more of a focus group interview, except one-on-one.

I was skeptical about this type of user research, but it has given us some good insights in the past.

Even if you don't have a prototype to show the customer, you can point to a similar UI on another site and ask about that. Let them explore the new interface, then ask:

1. What is this thing for; what does it do?
2. If you took X action, what would you expect to happen next?
3. Is this thing/flow/UI something that you would use? Why, or why not?

It's not quantitative, but it can help guide new development.