Research Project Planning - Time estimates for various research methods and phases

Question

Are there any suggested rough estimates for how long user research methods and phases of research planing, execution, and analysis take?

I'm trying to get better at project management and planning more methodically and accurately, which I've identified as a major opportunity for me increase my efficacy and impact. There are a few methods I know well enough through experience to estimate and I know how to break tasks down into subtasks for new methods I'm unfamiliar with, but I always get stuck on educated guesses on duration. This is a fundamental issue that has proved elusive to me, so I'm reaching out. I have looked online, in the archives of this list, and in several books about user research. Many practitioners talk about this issues and bring up bits and pieces, but nothing seems substantive enough in my opinion. In my research about conducting research, I'm running into the following hurdles:

• It seems like there is no consensus on this, as it can vary wildly due to project parameters and the unique needs and constraints at the time of execution.
• I know that with qualitative research, timing is very difficult to estimate since "information saturation" (inflection point where no new insights are discovered) is an unknown

Perhaps there are guidelines that I've somehow missed. If not, there are only two ways to get to estimates (generally speaking) that I can think of:

• Compile and average times for tasks and phases from a survey of practitioners - at the very least we can get ranges and relative timing differences.
• Start tracking my own projects and build historical averages - I assume others have done this internally but haven't shared it publicly.

If this info doesn't exist, ideally we as a community and cohort of researchers could combine those two methods in a publicly available and updatable format similar to that of salary ranges. I'm imagining a table of research methods and corresponding attributes that could be filtered and sorted with time estimates. I know that resources like this exist but the timing component is what seems to be missing. Perhaps for good reason, but I'd like to have some baseline to draw from.

Thoughts?

Answer 1

Overcoming the hurdles

It seems like there is no consensus on this, as it can vary wildly due to project parameters and the unique needs and constraints at the time of execution.

I think you need to approach it from looking at the factors (or context) that impact the amount of time required, which is probably easier to document compared to trying to find the 'magic' number. But in general you can try to break it down into the types of methods, the type of end user (or demographics), the nature of the information you want to collect. That should be relatively consistent enough across different researchers.

Of course, sometimes the method chosen is dictated by project constraints (e.g. time, budget, resources available, etc.) so you might not even get a true reflection of the actual amount of time needed since you are only getting a measure of the amount of time allotted.

I know that with qualitative research, timing is very difficult to estimate since "information saturation" (inflection point where no new insights are discovered) is an unknown

Again you can probably look at this in terms of the significance (how valid is the finding) and confidence (how applicable is this to the rest of the users) that the researcher wants to achieve with the research, which is more objective than just trying to estimate the point at which you reach information saturation. When you combine this with the type of information you want to uncover (e.g. critical usability problems versus minor ones) then you can come up with a rough estimate.

For example, it is often quoted that you don't need to test a large number of users to discover a critical usability issue, because something very major is likely to affect any user that you choose to do the testing on. Whereas something more subtle is going to take a lot more users for the researcher to determine the exact cause of the behaviour.

Methods versus phases

I think the if you can get a ballpark figure on the methods, then it is a matter of applying a factor on the phases since they will be dependent on the method chosen. For example, if you want to do most of the research manually (i.e. pen & paper in person) then the analysis is going to take more time. Doing card sorting on physical cards is fine as long as you are not dealing with hundreds of cards on many different walls because the processing and analysis time doesn't scale very well.

What is the purpose of the estimates?

I think if the ultimate goal is for you to have a resource that gives you a good baseline estimate of the effort required for research activities, you are better off compiling your own data because then at least many of the variations caused by factors that impact the accuracy and precision of the estimates will be reduced.

But I think if you put the same question to developers, they will also struggle to come up with an accurate answer. Basically the estimate is only going to be as accurate as the parameters used to produce the estimate, so estimating time to fix a bug versus time required to implement a feature using a given API and building on existing code will yield different degrees of accuracy (and this can also vary between individuals).

Agile UX?

It is great that you are doing research into this area, and this is the exact kind of problem that 'Agile' methodologies is supposed to help with when it comes to managing projects that can change over the lifetime of the product or service that you want to develop. So my advice is to just get as accurate as you can reasonably expect someone to do so and then adjust those baselines as you go.

Answer 2

Most definitely keep follow up notes on your own projects - how long did it actually take and why ?

When you've been doing it for a bit you'll then have a portfolio that you can refer to when quoting times and costs as you look for something similar you've already done.

Otherwise when starting from nothing, work out all the elements of the project, put time estimates in, and then double it (because things in life always take longer than you figure they should).