Summative user tests that allows you to compare against competitors?

Question

I want to understand when it would be appropriate to conduct summative users tests with the focus on evaluating the performance of our product against our competitors?

We have a finished live product which we know can be improved in many ways and want to think of a way to evaluate ours against others in the market. We've done some competitor analysis but want to see how people use our product and what they like/dislike. Would this kind of user testing work?

Answer 1

User testing competitor's products is quite standard and even recommended at times.

In your case I would perform three rounds of user testing, using same/similar tasks. One round on your product and two rounds for two competitor products.

This allows you to:

1. See the differences in flow between each product.

2. The actual impression your users get with each product.

3. Insights on what can be improved or what your competitors do really well.

These will potentially give you some deeper insights on how users behave, what they consider to work well and what doesn't. As well as give you insights on what to take on board if it's something that your competitors do really well.

NN/G As always have written an excellent article on the benefits and how-tos of user testing competitor's products.