Looking for independent usability study of the Microsoft Fluent UI

Question

I have been trying to gather objective feedback/response on the design of the Fluent UI ('Ribbon') interface. Apart from one study by Forrester which was commissioned by Microsoft anyway, and various rants or opinions on every blog or website imaginable, I haven't really seen anything specific or objective about the design from a usability or long term productivity evaluation. This website gathered some survey on Excel 2007, but I couldn't get any meaningful usability analysis or whether it was just a short-term effect of getting used to the change.

Also, other software developers are also adopting the ribbon UI as a result of the Microsoft interface design change (e.g. Nitro PDF, Autodesk) and therefore now it is also a case of trying to conform to a known standard rather than purely based on UI research and analysis alone. Is there any objective research in this area that can be referenced at all that anyone knows of?

I feel like from a professional point of view, there still hasn't been any definitive study on this that allows an objective decision to be made about implementing the design, or any follow-up to gauge the impact of the change. Furthermore, comparison of productivity between new computer users starting from the ribbon interface versus people who started from the traditional menu interface is not available either even after such a long time. I think this shows the lack of objective research when it comes to this topic.

Answer 1

Martin Dostál conducted a study of the Ribbon interface focused on evaluating user acceptance of the new mechanism. He surveyed 117 participants of varying demographics and experience with computers and office software with the following hypothesis:

Our hypotheses were as follows:

H1 Ribbon user interface is received better by younger people.

H2 Less experienced users consider a switch-over to the Ribbon user interface as more difficult.

H3 More experienced users are less satisfied with the Ribbon user interface in comparison to less experienced ones.

H4 Intensive users of Word 2003, OpenOffice.org or another Word 2007 predecessor are less satisfied with the Ribbon user interface.

They found hypothesis H1 to be "partially supported", H2 to be unsupported, H3 to be supported, and H4 to be unsupported. In general, we could say that expert users did not happily accept the transition to the Ribbon interface.

In a review section, Dostál commented on how users criticized the "visual clutter" of the Ribbon, pointing out one participant's comment:

"I don’t like it at all. From a modern civilisation back to pictograms and hieroglyphs."

This is an interesting study, but please note that it was focused on user acceptance of the new interface in the context of transitioning out of the old Office interface, and as such, it is not a general study of how usable the Ribbon interface is in isolation of user prejudices, etc.

(1) Martin Dostál. 2010. User acceptance of the microsoft Ribbon user interface. In Proceedings of the 9th WSEAS international conference on Data networks, communications, computers (DNCOCO'10), Nikos E. Mastorakis and Valeri Mladenov (Eds.). World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA, 143-149.

Answer 2

Jensen Harris posted a number of articles explaining the design rationales behind the new (2007) Office experience, including the move to the Ribbon interface:

Why the new UI?

In episode 7, specifically, he discusses the data that went into the redesign. This isn't a usability test, specifically, but does give some insight into the thought process.

There's also the story of the ribbon, which gives a slightly different summary.

Answer 3

I don't have quantitative studies, but consider the following:

The Ribbon UI was created based on a huge quantitative research which came back with the result - if I remember correctly - that everyone uses a small amount of features, but everyone uses a different small set of those, with a pretty flat curve (if I remember well, all features were used by at least 10% of the participants)

Mind you, probably I am mistaken, if I am not, evidence should be in the Harris blogposts already linked.

So, this is one research we have.

The other research comes from Cooper, in About Face 3, where he states what a menu is used for (eg. it has educational purposes). I do hope he based it on research.

Also, we have this 3rd research, qualitatively good enough or not, it is you to decide.

The big question is, does your userbase demographics match the userbase of Office?

It has two parts:

• Are most of your users frequent Microsoft Office for Windows users?
• Do they have the same demographical distribution in terms of usage?

Demographical distribution is that "everyone uses a small amount of features, but they use a different set of it, and together, all features are used pretty often." If this is true for your application, well, after billions of sessions, MS should be correct.

However, if you have a very different demographics - eg. there are a top 5 things an average user uses, and the next 95 buttons are only pressed by the sysadmins if they are asked to help - then I guess you should stick to a traditional toolbar + menu.

Or, if you have a lot of mac and/or adobe users, perhaps toolbar + menu + inspector.

I believe, that H3 came up positively in the research because "expert users" use a large amount of features, they are likely more into other apps daily than Office and therefore they can't rely on their spatial memory to find those - this is the typical sysadmin: does not use Office on their own, but is asked on various hard topics.

For them, the educational purpose of the menu - where you can scan a list of items, decide which one or two fits your goals, check into those and repeat the process recursively - fits better, as a text-only menu is easy to scan.

For those, who don't exploit such educational stuff - because they only have to press 20 buttons repeatedly which they learn spatially where they are - this "more frequent equals bigger" fits the purpose.

I hope this helps.