Has user testing found that the "three horizontal bars" for main menu on mobile is commonly understood?

Question

"Three horizontal lines" aka "hamburger icon" seems to be becoming a convention for the "main menu" button, especially on mobile sites:

Has anyone done any user testing of this convention, or got any A/B testing results? (i.e. is there an increase in bounce rate compared to using the word "Menu"?)

I did some quick user testing with wireframes featuring this icon for the main menu. 3 out of 6 people got it and used it. The others did not.

One person in particular had a lot of trouble "getting back to the top" (i.e. restarting their navigation from the top level of the site) as they didn't know about the convention, or that the logo would link to the home page.

I guess it will be different for different demographics - users of Facebook on mobile probably know it.

Anyone got any findings they can share?

Answer 1

James Foster of Exis Web did an interesting A/B test on the hamburger icon:

Tests on mobile showed a difference, though not all that significant, when the icon was used with a border (so it looks like a button):

Perhaps more interestingly, the A/B test seemed to more clearly indicate that desktop users don't understand the icon:

I tested 4 variations of the content inside a blue button:

(Baseline) The word “MENU” The word “MORE” Hamburger icon + “MENU” Hamburger icon + “MORE” The results were far more significant.

Tests 2, 3, and 4 performed far worse (18%, 31%, and 43% worse respectively).

And interestingly, the icon + label performed worse than just the label (which is usually pretty easy to integrate into a desktop page/app). So it would appear for mobile the icon works serviceably, as long as it looks like a button, and on desktop it's best to not use it at all.

James Foster then conducted a second AB test, this time looking at how the hamburger menu tests when used in combination with the word MENU and with a button border and combinations thereof.

Based on this and my previous AB test, a flat hamburger icon may not be ideal on a responsive website (remember this is a website not an app). Using the word MENU (and making it look like a button) could be more helpful for visitors.

This does not mean that users do not understand the hamburger/sandwich – it could be that the word MENU draws more attention.

This also supports findings recently published by NNGroup

Clear labels help users make decisions faster: they give good information scent about what will come next. Labels should still be used for newer icons, such as the three-line menu icon (or “hamburger” icon)...

...Users are still unfamiliar with newer icons, including the three-line menu icon

Answer 2

From the A/B Testing (based on the article posted by keiwes) we can infer the following:

^{download bmml source – Wireframes created with Balsamiq Mockups}

There seems to be a correlation between correctly identifying the hamburguer icon functionality and the age of the users.

Hence from the usability perspective, you need to question yourself were is the majority of your users. You might find that the proportions are like this:

^{download bmml source}

It would be interesting to see whether this assumptions are correct based on your testing, by trying to identify that the users that struggled with the test were older than the others.

Another thing that comes to my mind is that since the hamburger icon is somewhat new (at least as compared to traditional menus), someone that knows the hamburger icon, will know menus, but not the other way around. Which means that the actual distribution of users should be something like this:

^{download bmml source}

So if you want to play it safe go for the menus; if you want to be trendy go for the hamburger (with the risk that older people might get confused). You'll be only guessing until you identify and segment your target audience.

*Although the aforementioned article suggests that older users are not familiar with the hamburger menu, it would also require further analysis to identify other factors such as technology knowledge or experience, early adopter or laggard, etc.

Answer 3

My company caters towards an older population, so we included text along with the hamburger icon. Here is an article that had similar results with the older population: http://mobile.smashingmagazine.com/2013/09/11/responsive-navigation-on-complex-websites/

Google defines a convention as "a way in which something is usually done, esp. within a particular area or activity." With this definition you can say that the Hamburger icon is the convention, but this does not mean it works in every case. Do your testers represent the users that will actually use your application? Only half of your participants understood it, which means it needs improvement.

Here is another question on UX.SE in which the top answer indicated the hamburger menu was the convention: How to best indicate navigation menu on mobile.

Some design guidelines that may aid the hamburger menu's usability: http://www.getelastic.com/dont-make-these-mobile-menu-mistakes/

Answer 4

I too would like to see some A/B testing results regarding the hamburger menu icon. One thing to note though is that both Google and Facebook utilize this icon and it is definitely becoming more and more common. As Google and Facebook are two of the biggest companies on the internet I find it difficult to believe they'd use the icon without any user testing. I think Keiwes is definitely on to something however. You really need to know your target demographic. And implementation really determines the impact/intuitiveness of using hamburger style navigation. It's important that a number of techniques (listed in one of Keiwes links) are followed to ensure the user knows the menu icon is clickable. But as more and more companies start to use this icon, and especially as the mobile world becomes a bigger and bigger deal I think we will see this icon become the commonly adapted norm (if it's not already). Google, I would argue is the most influential company in the tech world and they literally set tech standards. So if Google is using this icon to denote "menu," across multiple products nonetheless this is good enough for me.

Google mobile:

Gmail mobile:

Facebook mobile:

Facebook mobile

Answer 5

“@aj_ux: STOP PRESS: I solved the hamburger menu problem everyone pic.twitter.com/7aG2V1POa5” Everyone loves a compromise.

— Jonty Sharples (@Gringomoses) June 10, 2014

Answer 6

'Understood by 'regular' users but not everyone, a fair statement I think: https://medium.com/design-ux/bcc8e0257bc8. However no hard facts anywhere online right now sadly.

From the article:

This icon seems to be perfectly understood by regular users of such interfaces, but no one can tell if it is understood by absolutely everyone. Are we creating a standard that, like the restroom pictogram, would only be understandable after some explanation? I don’t have the answer and it would need to be thoroughly tested, like the ones used by the ISO standards but adapted for the web. If the tests have already done, it would be great to see them published.

Answer 7

Booking.com has now published new results from their own A/B testing concerning the hamburger menu.

They already use a hamburger menu and replaced this in a test with a new Icon explicitly stating [Menu].

We ran that experiment against our whole user base, and given the prominence and omnipresence of this UI element, it didn't take long for this change to be exposed to millions of our customers all over the world, in every supported language and across a multitude of devices.

Interestingly this changed didn't have much impact on the user behavior:

In our experiment, changing the icon to the word "menu" had no significant impact on the behaviour of our users. With our very large user base, we are able to state with a very high confidence that, specifically for Booking.com users, the hamburger icon performs just as well as the more descriptive version.

So in this specific environment the hamburger menu seems to be quite good understood.

Source: http://blog.booking.com/hamburger-menu.html

Answer 8

It's not just websites anymore that are solely using it, larger applications such as Facebook, Chrome, and many Google products are all using it for most of their work. The guys over at SmashingMagazine wrote a pretty good article about it, I suggest you give it a read.

The majority of responsive websites that use an icon to represent a hidden menu opt for the three horizontal stripes — these include some high-profile websites like Starbucks and also popular apps like Facebook and Path. Part of its power lies in its versatility, as the icon itself doesn’t clearly represent a precise object nor method, which means it can be applied to a variety of navigation-based design patterns without showing a preference to a particular pattern. Its vagueness in shape doesn’t detract from its message as the icon is becoming an emerging standard. Like a new term appearing in our everyday language, we know what it means. And with high-profile websites throwing their weight behind it, so will average users over time.

Quote from http://mobile.smashingmagazine.com/2012/10/08/the-semantic-responsive-design-navicon/

They go on to show various examples of big companies using the icon, and it being widely more adapted.

I'd also like to reference this other SE question: Is Apple's or Google's meaning of the three-horizontal-bar icon more popular?

This brought up the question nearly 2 years ago, and since then, even more websites and applications have thrown their weight behind it and used it all for the same thing (or similar).

I'd say for sure that people are accepting it as menu, but it doesn't hurt to also throw in text if your website is catered to older generations (who may not be used to using mobile applications and haven't seen the icon).

Answer 9

Finally somebody actually did some A/B testing on the hamburger icon vs. the word 'menu'. He is even planning to do some more extensive user testing, because he noticed his article being very popular.

http://exisweb.net/mobile-menu-icons

Also, here is a relevant Twitter timeline: https://twitter.com/mrjoe/timelines/428584785347231747

And here: http://www.exquisitetweets.com/collection/lukew/2919

Conclusion from the research above:
People over 35 years old don't seem to know the hamburger icon. Results are influenced by demographic values. Researcher recommend adding the word 'menu' to the hamburger icon.

Answer 10

In an icon busy environment I prefer using three dots rather than the three bar icon as it is easily distinguishable from the rest. Although I have never gone for any user testing for the 3-bar icon, I feel that with companies like Google using it, it is starting to become a convention.

Answer 11

The 3 bar is more and more common so will become a standard symbol much like @ can be for emails.

On your application the menu is on the right but the 3 bars are on the left for applications I use on Android with the three dots on the right for symbols (personally I preferred the cog)

Navigation has always, as a rule, been on the left so I think this will help you right away and worth retesting with this in mind.

There are some good articles out there including this site:

How to best indicate navigation menu on mobile

which having read through several others is the best articles to read.

It is worth noting is called a "Navigation Draw" on the Android site:

http://developer.android.com/training/implementing-navigation/nav-drawer.html

If you need any CSS for your icons you can look here:

http://css-tricks.com/three-line-menu-navicon/