User Experience Stack Exchange is a question and answer site for user experience researchers and experts. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We've designed a new navigation style for an up-n-running website. Now i want to user test if it is better than the current one.

But how do I avoid the bias that users will most likely be faster on the current solution since they might be used to it?

share|improve this question
Does that mean that you changed the navigation, although you didn't know of any existing problems with it? – Lovis Sep 18 '13 at 12:11
Do you have usability statistics from the time your site launched with the old navigation style? If yes, compare with those. If not, make an A/B test with new users only, and see which group gets better at orientation. Also, consider measuring not only the learnability, but also the ease of use for experts - let some new users get accustomed to the new navigation, and measure their results against the results of users accustomed to the old one. – Rumi P. Sep 18 '13 at 12:21
Better in what terms ? The speed of navigating within the navigation itself ? Or the ease of use ? Or the speed a user can get to a certain page ? Are you going to test it ? Or do you have a testing team ? Or a base of test users ? – happybuddha Sep 18 '13 at 20:05
Don: We changed it because it took up a lot of screen space that we could make better use of. Thus, I only need it to be at least as good as the current one. Good = Easy to understand how to get to the next page, easy to understand where on the site I am and, to some extent, speed. – user35678 Sep 19 '13 at 8:25
up vote 2 down vote accepted

Wouldn't it be best to test both the old and new navigation on new users instead? This would be the ideal testing condition for the website. You can also use the data from existing users for the old and new navigation to look at the impact of the change (especially if you have lots of existing users), but if you want a fair comparison, then try to eliminate as many of the variables as possible so that you are only comparing the navigation. However, I don't think it is very easy to isolate the impact of the navigation, because it will impact on other aspects of the website as well. You should really base your testing on the reasons for the redesign.

share|improve this answer
+1 for suggesting to test this with new users only – greenforest Oct 14 '13 at 10:13

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.