How to Assess UI complexity / Intuitiveness

Question

Which tools / tests do you use to compare user interfaces in terms of complexity? So far, I applied the NASA-TLX.

Answer 1

The best way to assess a user interface is with actual users. There is no way around that simple fact. The challenge is getting the right users for your test. For example, if you are designing an interface for a pilot (NASA-TLX is designed for this space), you need pilots to evaluate your design, not Joe Random you found at the coffee shop this morning.

The best approach I've seen/used to assess intuitiveness is:

• Write up a small number of key tasks you want to test (no more than 5).
• Establish what you think is a reasonable time to complete each task.
• Get a number of users (between 3-5) in a room, each with their own computer.
• Tell them to jot down their first impressions.
• Give each of them the same task, but no instructions.
• Time how long it takes for each one to complete the task.
• Tell them to jot down their impressions, if they thought it was difficult, etc.
• After that instruct them on how you thought they would make it work. You can get some really interesting feedback here, from "That's cool" to "Why the F*** did you do that?"
• Repeat for each of the tasks.

What you'll find is that you might get a few more WTF moments at the beginning, but as the users start to learn how you are solving problems they can apply that to the newer tasks. The important things you are looking for are:

• Did anyone fail to perform any of the tasks?
• Were the times you saw very different from what you expected?
• How many clicks, or mis-steps did the user have when completing the tasks?
• Once the audience learned something about how the site works, did they improve on future tasks?

If the answer is yes to the first question, then you completely missed the boat for making the site intuitive for your audience. However, that is valuable information. When you are working with subject matter experts, you need to pick their brain for how they think about a problem.

If the times you saw were much longer than anticipated it warrants further investigation. It could simply be that you weren't reasonable in your expectations for someone who has never seen the app before. Of course, it could mean that what you thought was user friendly and intuitive didn't match the way your target audience thinks.

The number of mis-steps is an indication of "false leads". I.e. something that looked like it would be correct, but turned out not to be the case. The first task will inevitably have a few mis-steps, but the number should fall off steadily with the new tasks. If they remain constant or, worse, go up then this is an indication that you have added cognitive complexity that shouldn't be there.

There are two major types of complexity to worry about:

• Cognitive complexity: how hard it is to interpret and make use of the application
• Functional complexity: the number of steps needed to perform any action

It may be useful to introduce a little cognitive complexity at times to provide the user with new tools to do their jobs better. The important aspect is trying to design it so that once the user learns how to use the new tool, they can focus more on their job and less on the application.

Unfortunately, user experience is a very subjective topic with a number of factors that will give you different results with different people. It's important to select your UI evaluators from your target audience, or your testing will tell you the right story for the wrong group. The hardest part is creating a feedback loop to help you better match what your users need.

Check these resources out as well:

• http://www.alistapart.com/articles/usability-testing-demystified/
• http://www.alistapart.com/articles/can-you-say-that-in-english-explaining-ux-research-to-clients/
• http://www.alistapart.com/articles/quick-and-dirty-remote-user-testing/
• http://www.alistapart.com/articles/the-myth-of-usability-testing/