Usability testing: closed/open-ended tasks & actionable/verbal goals

Question

I am building an app which provides users information about stocks and allows users to follow stock development by adding them to their favourites. I am planning a usability test and have checked some resources: Link 1 and Link 2. I have two questions.

1. Closed or open-ended tasks? For example, I could have tasks like:
   • Search for a stock X and add it to your favourites (a closed task).
   • Search any stock from industry A and add it to your favourites (an open-ended task).

Which is better, or would it be good to include both types? I want for users to be able to explore the app a bit by themselves without too much limitations (open-ended), but I am worried if tasks then become too vague.

2. Actionable or verbal goals? For example, I could have something like:
   • Check the info of a stock X, what do you think is the current status of this stock?

Link 1 talks about always having actionable tasks. However, I also want to test how users understand and interpret displayed information. So the goal in these type of tasks would be verbal, but I am not sure is this a good idea.

Answer 1

Your questions immediately reminded me of the usability testing chapters from Steve Krug's book "Don't Make Me Think". To be honest, I think you should simply read that as it covers exactly what you're asking here (and I hate to just repeat experts who have been doing this for decades and "reaping the rewards").

To summarize (since I don't want to link you to Amazon or something like that) the "open-ended task" as you call it allows for more exploration and gives the user a chance to do what he wants (which is closer to a real world experience than ordering something arbitrary). This is shown perfectly in an episode of a remote testing webinar Steve Krug and Tomer Sharon did back in 2015. They actually do a full demo test run which you'll probably find very interesting (starts around minute 16, but the whole thing is very interesting).

https://www.youtube.com/watch?v=c7cRYX9BoDk

For your second question: Actionable goals are better and you should probably rethink what you expect users to provide information-wise. It's way more important to pay close attention to what they are doing, rather than what they are saying. In fact, many people will say something, but do the complete opposite. During the webinar I linked Steve and Tomer also talk about this and mention that they rarely pay attention to what participants actually say, but rather what they are doing (this is hard enough and matters more, according to them).