Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I was reading up on best practices on usability testing for websites and a recommendation was that get your participants to do usability testing on your competitors website as well as that would help you better understand how they navigate sites like yours (e.g. an eCommerce site). Though I can see the merit in that, a lot of websites have different design patterns and user flows and I am not sure if there would be any real benefit in doing an usability test of the competitor website unless the site is very similar to your own site.

Has anyone done a similar exercise and has had results which show key takeaways even though the competitor site might not be a mirror image of your site. I would also love it if there are any references to published case studies or research articles.

share|improve this question
    
Interesting question. –  PhillipW Apr 12 '13 at 10:37
    
For research yes, not for validation for new work. Depends on how you define usability testing. I tend to use testing for the validation part of the UX process after something new has been put together and research for stuff pre concept/design. –  Stewart Dean Apr 13 '13 at 13:21

6 Answers 6

I can think of a number of things this may do for your test results.

The obvious: consider the situation where your product isn't yet available, or you're developing a new feature. Learning from mistakes others already made, or from what they did right, is immensely valuable.

It may also help your judgement of the problems the test reveals. I think we all sometimes fall into the trap of downplaying the shortcomings of something we created, while we have no trouble being quite critical of the work of others. In testing your competitor's products you may find shortcomings you were overlooking in your own product. You'll look at it with fresh eyes and it can provide a reality check looking at your own product (again) can't. Especially if your doing an expert/heuristics review this could be very valuable.

Users will have learned certain problem solving tactics that they'll apply without being conscious about doing so. They might be very handy doing something on a competitor's site, but if that tactic is not available in yours it will not appear in your test. They might tell you something about your product isn't very handy, but they probably won't be able to tell you what. So, looking at people using your competitor's product might reveal such tactics.

Same goes for features in a competitor's product you may not have thought were important. Users may not tell you they use them, but tests might reveal that they're important anyway. Note that this means that the fact that your product and that of the competitor are not the same could be valuable too.

Something else: you're inviting people to judge your product. Of course you'll prepare them carefully, but they may be holding back criticism. Looking at a competitor's product could solicit feedback you were not getting on your own product.

Although this does not strictly relate to usability testing, I've found that in context mapping sessions anecdotes and comments that happen around the actual session can be very informative. Unless your testing for a specific problem area, this to me means that breaking away from the core test can give you a lot of useful input for the (interaction) design of the product.

There's probably more.

share|improve this answer

No, these are two completely different activities. Usability test of your site is one thing where looking at competitors site doesn't do you any good in that context.

However, you should have scheduled activities to benchmark your site against competitors every quarter to find out where they are now and possibly where they are going. This is in this case a part of a bigger marketing plan where UX is one factor. Pricing, terms and conditions, shipping and bundlings are other factors to measure in a benchmarking session.

Keep them separate but keep on doing both, and you're on top of things.

share|improve this answer

There is value in doing (usually quick and dirty) usability testing on some competitor's websites, but this should not be a UX focus.

Doing this is usually more about market research than UX, and is done mainly to understand where you have potential improve your product over theirs and where to focus your differentiation strategy.

That said, you should regularly be using other websites and products to help get ideas for how you can improve the UX of your own products and keep up to date with how other people are solving similar UX problems.

share|improve this answer

Most usability practitioners agree with the ISO/IEC definition of usability, that states that it's a measure.
It is a measure but there is no defined unit like meter or pound or jakobs to express it in absolute terms.
This lets you with comparisons only, like whatever its value, the usability here is more, or less, than there.
So if you can charter a group of subjects to try in the competitive site a very similar task as the one you are testing on your site, you'll get a glimpse of the relative usability values.

share|improve this answer

Yes. I've done usability tests on multiple competitor sites on several occasions for a few different reasons.

  • As a way to judge the level relative ease of use on the various sites (using tools like SUS). If you are significantly better/worse than your competitors it can affect how you invest in future features.

  • Learning about features in competitors sites (Does feature X look good? Do users like it? Does it seem to affect behaviour in good/bad ways?)

  • To examine similar features (Does their checkout work better? Does their taxonomy make more sense to the user? etc.)

share|improve this answer
    
Can you define SUS? –  TotemFlare Apr 12 '13 at 22:48
    
See en.wikipedia.org/wiki/System_usability_scale ;) –  adrianh Apr 12 '13 at 23:57
    
Thanks much @adrianh –  TotemFlare Apr 13 '13 at 18:25

I have used others sites in user sessions a couple of times, although I would not call it usability testing as such. If you have the time and budget then this is a good user research activity. It is best done after the aims and scope of a project have been defined.

It's worth adding that the sites selected don't need to be competitors but need to be comparable in what they do. My aim has been to determine which sites do what they do well, what worked well for end users and how the existing site (if there is one) that we are redesigning compares.

The method was to use 10 sessions in which we picked 4-5 different sites with the clients site mixed in. After the initial warming up part of the session we would then go through a series of steps for each site with a number of possible routes to explore if a site drew up a dead end on one of them. For example when opening the site the users where asked to give their reactions to the site and then what do you think this site does and then if they where looking to do x where would they start.

Even in an hours section we could gather a lot of interesting information about what users where expecting and what they where not. This allowed us to create an informed set of 'best practice' recommendations as opposed to the 'expert' guides that I have often created.

One thing we learnt, for example, is that putting pictures of people on a website does not make the site warmer and more friendly, as is often said, but alienated people as they saw those people as not the kind of people they could relate to. People are very judgemental. Instead people preferred representations and close up shots of hands and tokens of wealth like pens and watches. In this case the client went on to redesign their printed material to take this into account.

So in summary - it's a great technique for quickly understanding what customers are looking for in a new site. You don't need to use competitors, just comparable sites and it's not a validation technique but a research technique and it is not the same as a customer audit - that happens prior to inform this activity.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.