2

We've designed a new navigation style for an up-n-running website. Now i want to user test if it is better than the current one.

But how do I avoid the bias that users will most likely be faster on the current solution since they might be used to it?

Improve this question
4
  • 1
    Does that mean that you changed the navigation, although you didn't know of any existing problems with it?
    – Lovis
    Sep 18, 2013 at 12:11
  • 1
    Do you have usability statistics from the time your site launched with the old navigation style? If yes, compare with those. If not, make an A/B test with new users only, and see which group gets better at orientation. Also, consider measuring not only the learnability, but also the ease of use for experts - let some new users get accustomed to the new navigation, and measure their results against the results of users accustomed to the old one.
    – Rumi P.
    Sep 18, 2013 at 12:21
  • 1
    Better in what terms ? The speed of navigating within the navigation itself ? Or the ease of use ? Or the speed a user can get to a certain page ? Are you going to test it ? Or do you have a testing team ? Or a base of test users ?
    – happybuddha
    Sep 18, 2013 at 20:05
  • Don: We changed it because it took up a lot of screen space that we could make better use of. Thus, I only need it to be at least as good as the current one. Good = Easy to understand how to get to the next page, easy to understand where on the site I am and, to some extent, speed.
    – user35678
    Sep 19, 2013 at 8:25

1 Answer 1

Reset to default
2

Wouldn't it be best to test both the old and new navigation on new users instead? This would be the ideal testing condition for the website. You can also use the data from existing users for the old and new navigation to look at the impact of the change (especially if you have lots of existing users), but if you want a fair comparison, then try to eliminate as many of the variables as possible so that you are only comparing the navigation. However, I don't think it is very easy to isolate the impact of the navigation, because it will impact on other aspects of the website as well. You should really base your testing on the reasons for the redesign.

Improve this answer
1
  • +1 for suggesting to test this with new users only
    – greenforest
    Oct 14, 2013 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.