I am conducting my first usability test in our in-house testlab. I have asked my team (and the rest of the company) for hypotheses and aches regarding the service we are goint to test. Based on these (and pther things as well) I have made some tasks.
My question is: Who should I invite to comment or review the tasks?
I am thinking that the product owner should definitely look at them/have a say – and of course my UX colleagues. But what about the rest of the team (developers and testers)?
Mainly, you should invite someone who knows the job of your end user and understands the environment in which user will use your product, so that they can define goals of the end user. Usually that is done by business analysts or product management team. Around that usability test can be designed and given for you usability test team.
You business owners also need to specify their own goals as well, such as
The most important types of people to bring in are those who know the research for your end user very well, preferably people who can remain unbiased about the product and focus more on the user needs. It is good to let the some of the stakeholders know what you are doing and provide some feedback, but be careful not to give them too much control; stakeholders are attached to the product and are more likely to accidentally bias the testing process to achieve the results they are looking for.
The biggest piece of advice I would recommend that you should limit the amount of people who have a say in the process because of one common issue: the more people you have involved, the more subjective your research is likely to become. Each department has their own objectives which can quickly muddle what should be an unbiased UX process. If your research has been done well (which I'm sure it has) all you should need it some quick fact checking from no more than 3-5 people total.
You should invite anyone who is responsible for understanding the results from your research, implementing the results of your research, or building upon your research in some way.
If this is your first study with the team, this is an excellent opportunity for you to build your relationship with them. If this is the very first usability study that the team has ever had the opportunity to be involved with, it is imperative that you build a relationship with them. Unless you are the one making code changes or determining the relative priority of the work, one of your main jobs as a researcher is to influence the team (likely without any authority) to make unplanned changes that come about as a result of your work. Without this relationship, your chances of having an impact are significantly reduced.
Inviting feedback does not necessarily mean that you will immediately act upon all of their feedback. You might remind them that, as this is your first study, you are keeping the scope of the study narrowed to a relatively small list of user workflows, and you would love to discuss future research that addresses their question. You might tell them that their question is an excellent one, but it is best suited to be answered using another research methodology, and you would love to discuss how to answer it after you've completed this research. You might remind them to remember to use the users' terminology instead of company jargon. You might ask them to think about their question in terms of what the user actually wants to accomplish.
One of your goals in sharing your task list (and potentially whatever you are using to screen potential participants) is to ensure that whoever is responsible for acting upon the results of the research has the opportunity to understand what will happen during the research, what is (and is not) a goal of the research, and what outcomes they might expect. By giving them this opportunity, you greatly increase the chances that they will accept the results of your research and act upon them appropriately. In communicating with the team, you will learn about what is (and is not) important to them, which can help you both frame your research results in terms that they will understand, as well as shape future research to meet their needs.
I've always asked the product manager (hopefully a single person, not a committee), whom I understand to be the person who decides which qualities (like performance, usability, security, and what @ripu1581 mentions) are important to the product and who decides which features are developed. If that responsibility lies with somebody titled "Architect" or "Developer", that's fine as well. You need to synch with this person to ensure results from the test are implemented and thus, will become visible.
If you have knowledge in the business area and with previous version of the product, they are usually happy to consider proposals for tasks from you. If you are new to the team, you probably cannot press your own agenda of what should be changed, but need to follow their ideas of what is important. (That's not bad, since it's usually taken me a single usability test to surprise the team what users find intuitive and what not - so after the test results are out, your opinion will be appreciated much more.)
I would not extend the scope of the test across what is feasible to tackle in the near future. Preparing and conducting the test is effort for everybody, and the closer the connection between this effort and immediate changes, the clearer the value of a usability test becomes.
In addition to the tasks, you most likely also need a screener: A document describing your target group, which allows you to sort out people who your product is not addressing. Be careful when preparing this with the product manager! If anybody wants to question your results, a marginal fit between target group and test participants is their best argument.
To ensure everyone is informend, invite everyone from developer to manager to tester to watch the sessions - stories about users being stuck stay with the team and help you in subsequent priority settings.
