How do you test navigation engagement?

Question

Some coworkers and I are engaged in heated debate about changing the navigation arrangement from a left nav column to a top nav bar. Arguments in favor of a left nav are more visibility and more engagement, at the expense of horizontal screen real estate and wasted space (our nav only has 5 links). Arguments in favor of a top nav bar is for reclaiming horizontal screen space and a more aesthetically pleasing layout. It's not clear how much of an engagement hit a top nav bar could have (if at all). It's probably also impacted by how visually prominent it is.

Is it possible to run a study to test engagement without actually shipping code and looking at data? If so, how would you test this?

Answer 1

You don't need a working prototype to do usability testing.

Paper prototypes work fine for testing general ideas and interactions, including navigation schemes. There are plenty of examples on YouTube. (Customize your search to see paper prototyping for phone UIs and even watch UIs.)

You say you're interested in testing engagement, and I'm not sure what you mean by that. I'll assume you're testing your nav system's ease of use. Because engaging with the nav menus is probably not the point of your site, nor the goal of your users. Users are probably there to find information and do something with it, and testing lo-fi prototypes is a great way to look at that.

Answer 2

Do I understand you correctly - you want to know how to test your concept without having a prototype / developed code version?

So IF this is the question, my answer is: No, you can not test it if you have nothing to test, but have to rely on best practice instead, unless you get further data that pays in on a second version.

So assuming that you want this instead, I'd like to answer some of your points mentioned in your comment before:

• "25th percentile screen size is 1280px wide" - I would think this is quite a few percent of all your users. Personally, I go by the rule "an edge case is the lowest resolution that is used by less than 5% of the user base. Thus the smallest screen size that is used by more than 5% of the users is the design target." - while 5% often gets a 10%, depending on how technically advanced or "extraordinary" your product might be.

• Concerning the idea of using a hamburger menu, you might want to read this: https://lmjabreu.com/post/why-and-how-to-avoid-hamburger-menus/ or a more broad range of articles: http://www.smashingmagazine.com/smashing-newsletter-issue-110/#a6 old, but good and still valid.

• Left-side navigation - as you said - munches a lot of screen size. The question is: Do you NEED the space? Or will it be wasted and resulting in miles long vertical scrolling? If you need the space, do not waste it! Use a horizontal nav then.
• In case you use the horizontal nav on top: Do you need the navigation while scrolling on the page? If yes, you could consider having it sticky, so it is always available for the user.
• Our own research on our e-commerce page showed, that the comparison between using a vertical and a horizontal menu differs: The second item gets clicked ~10% less if vertical listed. The horizontal second position performs better. BUT, by any means, this might totally depending on the style and content of the page. To me the only general argument for this approach is european reading direction - top left to bottom right, line by line.

Maybe - just to make sure - you could have a clickdummie and run a heatmap OR watch people use the two different versions and see which one is more intuitive to them - thus a qualitative test with ~10 people.

Answer 3

Of course you'll need at least a prototype, that goes without saying. Research on the subject is quite extensive, but in the end, it is highly affected by the site itself, its content, its audience, devices and so on, so it's very difficult to have a "one size fits all" answer.

However, there's a very interesting article dealing with the exact same problem and discussions you're having with your colleagues, only that for mobile.

Either way, they mention the whole process, the problems, hypothesis, research, testing and what they finally did based on pure objective data, see article here.

Keep in mind you mention this:

Our 25th percentile screen size is 1280 wide

which leaves us out with the obvious question... what about the remaining 3/4 of your audience? This will affect your question greatly. For example, what if a big part of that remaining 75% uses huge screens? A menu on the side on a fixed width layout will be totally different to a side menu on a fluid full screen layout (you can test this if you want, you'll be shocked at the numbers). Let alone if a big part of that 75% navigates the site on mobile.

Finally, as a rule of thumb, and whenever possible, based in our own research, we tend to prefer a top nav by default, and even keep it as "sticky" or fixed. Then again, your case may be different, so you'll need to test and see how it works for your audience

If you're looking for more ideas besides these 2 options, take a look to How To Design Effective Navigation Menus which includes different types of navigation with examples and a short explanation about each of these options