# Has user testing found that the “three horizontal bars” for main menu on mobile is commonly understood?

"Three horizontal lines" aka "hamburger icon" seems to be becoming a convention for the "main menu" button, especially on mobile sites:

Has anyone done any user testing of this convention, or got any A/B testing results? (i.e. is there an increase in bounce rate compared to using the word "Menu"?)

I did some quick user testing with wireframes featuring this icon for the main menu. 3 out of 6 people got it and used it. The others did not.

One person in particular had a lot of trouble "getting back to the top" (i.e. restarting their navigation from the top level of the site) as they didn't know about the convention, or that the logo would link to the home page.

I guess it will be different for different demographics - users of Facebook on mobile probably know it.

Anyone got any findings they can share?

-
I have found this comment saying that they don't test well, but it's not really authoritative boagworld.com/usability/learning-from-mobile-navigation/… Hopefully there's more info out there. –  JonW Jan 8 '14 at 16:55
The hamburger is common convention, I think even if the user didn't know what it did, eventually they'd get curious and touch it. –  VoronoiPotato Jan 13 '14 at 16:51
We included user comprehension of the handburger icon as part of a broader set of formal usability tests on our mobile site. I haven't got anything to link you to, but we found no issues at all in comprehension across a range of ages and genders of users on a fashion ecommerce site. –  Racheet Jul 8 '14 at 9:56

Tests on mobile showed a difference, though not all that significant, when the icon was used with a border (so it looks like a button):

Perhaps more interestingly, the A/B test seemed to more clearly indicate that desktop users don't understand the icon:

I tested 4 variations of the content inside a blue button:

(Baseline) The word “MENU” The word “MORE” Hamburger icon + “MENU” Hamburger icon + “MORE” The results were far more significant.

Tests 2, 3, and 4 performed far worse (18%, 31%, and 43% worse respectively).

And interestingly, the icon + label performed worse than just the label (which is usually pretty easy to integrate into a desktop page/app). So it would appear for mobile the icon works serviceably, as long as it looks like a button, and on desktop it's best to not use it at all.

James Foster then conducted a second AB test, this time looking at how the hamburger menu tests when used in combination with the word MENU and with a button border and combinations thereof.

Based on this and my previous AB test, a flat hamburger icon may not be ideal on a responsive website (remember this is a website not an app). Using the word MENU (and making it look like a button) could be more helpful for visitors.

This does not mean that users do not understand the hamburger/sandwich – it could be that the word MENU draws more attention.

Clear labels help users make decisions faster: they give good information scent about what will come next. Labels should still be used for newer icons, such as the three-line menu icon (or “hamburger” icon)...

...Users are still unfamiliar with newer icons, including the three-line menu icon

-
I wish they'd tried out a combination of the two options - a border around the hamburger with menu text on it too. –  JonW Feb 13 '14 at 10:59
@JonW he said on Twitter he was interested in doing more testing, hopefully he will –  Ben Brocka Feb 13 '14 at 14:46
Yeah, the comments (and recent edit) in the article also say he's looking to do more work on it, so I'll keep my eyes on it! –  JonW Feb 13 '14 at 14:47
Thanks @JonW and James Foster! Pretty clear that "Menu" with border is better than anything with hamburger icon. Perhaps ought to call it the "humburger" icon as it makes people think (too much)! –  Dan Mar 19 '14 at 18:10

From the A/B Testing (based on the article posted by keiwes) we can infer the following:

There seems to be a correlation between correctly identifying the hamburguer icon functionality and the age of the users.

Hence from the usability perspective, you need to question yourself were is the majority of your users. You might find that the proportions are like this:

It would be interesting to see whether this assumptions are correct based on your testing, by trying to identify that the users that struggled with the test were older than the others.

Another thing that comes to my mind is that since the hamburger icon is somewhat new (at least as compared to traditional menus), someone that knows the hamburger icon, will know menus, but not the other way around. Which means that the actual distribution of users should be something like this:

So if you want to play it safe go for the menus; if you want to be trendy go for the hamburger (with the risk that older people might get confused). You'll be only guessing until you identify and segment your target audience.

*Although the aforementioned article suggests that older users are not familiar with the hamburger menu, it would also require further analysis to identify other factors such as technology knowledge or experience, early adopter or laggard, etc.

-
Balsamiq's hamburger icon has an extra patty. –  edgarator Jan 9 '14 at 1:17
It's not clear to me the auteur of that article actually did any AB testing. What I got out of the article is that he made an assumption based on the fact the hamburger icon is a somewhat new convention and therefor younger people are more likely to be familiar with it than older people are. –  Paul Jan 9 '14 at 8:35
Thats insane. Great post. –  Fresheyeball Jan 9 '14 at 22:20
@Paul, what I got from the article is that he made an observation based on user testing. However, since he was using both the burger and menu patterns I took it as a form of AB testing (although it wasn't). At the end, according to N/N Group, volume derived from AB testing doesn't add too much value when compared to direct, targeted and focused usability testing. Clinging into the details leave this question unanswered, interpreting results (or assuming as you would anyway for AB tests, is the only thing we can do) allow for this question to be answered. Insight & Refinement are what matters. –  edgarator Jan 12 '14 at 23:40
@user645715, I was just making a joke :) –  edgarator Jan 12 '14 at 23:46

My company caters towards an older population, so we included text along with the hamburger icon. Here is an article that had similar results with the older population: http://mobile.smashingmagazine.com/2013/09/11/responsive-navigation-on-complex-websites/

Google defines a convention as "a way in which something is usually done, esp. within a particular area or activity." With this definition you can say that the Hamburger icon is the convention, but this does not mean it works in every case. Do your testers represent the users that will actually use your application? Only half of your participants understood it, which means it needs improvement.

Here is another question on UX.SE in which the top answer indicated the hamburger menu was the convention: How to best indicate navigation menu on mobile.

-

I too would like to see some A/B testing results regarding the hamburger menu icon. One thing to note though is that both Google and Facebook utilize this icon and it is definitely becoming more and more common. As Google and Facebook are two of the biggest companies on the internet I find it difficult to believe they'd use the icon without any user testing. I think Keiwes is definitely on to something however. You really need to know your target demographic. And implementation really determines the impact/intuitiveness of using hamburger style navigation. It's important that a number of techniques (listed in one of Keiwes links) are followed to ensure the user knows the menu icon is clickable. But as more and more companies start to use this icon, and especially as the mobile world becomes a bigger and bigger deal I think we will see this icon become the commonly adapted norm (if it's not already). Google, I would argue is the most influential company in the tech world and they literally set tech standards. So if Google is using this icon to denote "menu," across multiple products nonetheless this is good enough for me.

Gmail mobile:

-
In all these cases the hamburger icon is not alone, it is near other icons and that is well understood as a "toolbar", users understand there are "commands", explore touching the icons and discover content. The same applies to the "little world" Facebook Icon, could you imagine it as clearly understandable if it was alone? –  FrancescoMM Jan 10 '14 at 14:31
The fact that FB and G use it, does not really imply people fully understand it, because people are strongly motivated to use FB and G, so if they get confused or pissed off they will try to understand, ask, learn and get familiar with the new icons. So G and FB can actually force habits on people, and people will follow, but the same does not probably apply to our app. If icon is not clear, they go. Of course after G and F use the same icon for long (and many others follow them) everybody gets used to it, and so it becomes a well established standard (like Youtube play button over videos). –  FrancescoMM Jan 10 '14 at 14:44
The issue I have with Google and Facebook is that they're not menu driven websites - the vast majority of the site activity is driven on the homepage itself. I'd wager many Google and Facebook users have never pressed that menu button as they just don't need to. –  JonW Jan 14 '14 at 11:41

@aj_ux: STOP PRESS: I solved the hamburger menu problem everyone pic.twitter.com/7aG2V1POa5” Everyone loves a compromise.

— Jonty Sharples (@Gringomoses) June 10, 2014

-
+1 for the creativity –  Adrien Be Dec 7 '14 at 11:40

'Understood by 'regular' users but not everyone, a fair statement I think: https://medium.com/design-ux/bcc8e0257bc8. However no hard facts anywhere online right now sadly.

From the article:

This icon seems to be perfectly understood by regular users of such interfaces, but no one can tell if it is understood by absolutely everyone. Are we creating a standard that, like the restroom pictogram, would only be understandable after some explanation? I don’t have the answer and it would need to be thoroughly tested, like the ones used by the ISO standards but adapted for the web. If the tests have already done, it would be great to see them published.

-
I think you're probably right here. It appears the answer to the question of whether anyone has actually tested and published the results of how usable these buttons are is: no, nobody has –  JonW Jan 15 '14 at 9:44

It's not just websites anymore that are solely using it, larger applications such as Facebook, Chrome, and many Google products are all using it for most of their work. The guys over at SmashingMagazine wrote a pretty good article about it, I suggest you give it a read.

The majority of responsive websites that use an icon to represent a hidden menu opt for the three horizontal stripes — these include some high-profile websites like Starbucks and also popular apps like Facebook and Path. Part of its power lies in its versatility, as the icon itself doesn’t clearly represent a precise object nor method, which means it can be applied to a variety of navigation-based design patterns without showing a preference to a particular pattern. Its vagueness in shape doesn’t detract from its message as the icon is becoming an emerging standard. Like a new term appearing in our everyday language, we know what it means. And with high-profile websites throwing their weight behind it, so will average users over time.

They go on to show various examples of big companies using the icon, and it being widely more adapted.

I'd also like to reference this other SE question: Is Apple's or Google's meaning of the three-horizontal-bar icon more popular?

This brought up the question nearly 2 years ago, and since then, even more websites and applications have thrown their weight behind it and used it all for the same thing (or similar).

I'd say for sure that people are accepting it as menu, but it doesn't hurt to also throw in text if your website is catered to older generations (who may not be used to using mobile applications and haven't seen the icon).

-

Finally somebody actually did some A/B testing on the hamburger icon vs. the word 'menu'. He is even planning to do some more extensive user testing, because he noticed his article being very popular.

Conclusion from the research above:
People over 35 years old don't seem to know the hamburger icon. Results are influenced by demographic values. Researcher recommend adding the word 'menu' to the hamburger icon.

-

In an icon busy environment I prefer using three dots rather than the three bar icon as it is easily distinguishable from the rest. Although I have never gone for any user testing for the 3-bar icon, I feel that with companies like Google using it, it is starting to become a convention.

-
The problem with that is that Google is now starting to use both for different elements in their new Android UI standards. The "hamburger" icon is now represented on the left as an app wide menu. Whereas the three dots on the right represent actions for that particular page. Since Android is the most widely used mobile OS around the globe, I think whatever Google does will be adopted as a "standard" even if Apple doesn't agree, or does the opposite to be different. –  Jeremy Miller Jan 10 '14 at 22:46

The 3 bar is more and more common so will become a standard symbol much like @ can be for emails.

On your application the menu is on the right but the 3 bars are on the left for applications I use on Android with the three dots on the right for symbols (personally I preferred the cog)

Navigation has always, as a rule, been on the left so I think this will help you right away and worth retesting with this in mind.

There are some good articles out there including this site:

which having read through several others is the best articles to read.

It is worth noting is called a "Navigation Draw" on the Android site:

If you need any CSS for your icons you can look here:

-

Booking.com has now published new results from their own A/B testing concerning the hamburger menu.

They already use a hamburger menu and replaced this in a test with a new Icon explicitly stating [Menu].

We ran that experiment against our whole user base, and given the prominence and omnipresence of this UI element, it didn't take long for this change to be exposed to millions of our customers all over the world, in every supported language and across a multitude of devices.

Interestingly this changed didn't have much impact on the user behavior:

In our experiment, changing the icon to the word "menu" had no significant impact on the behaviour of our users. With our very large user base, we are able to state with a very high confidence that, specifically for Booking.com users, the hamburger icon performs just as well as the more descriptive version.

So in this specific environment the hamburger menu seems to be quite good understood.

-