Counting Clicks vs. Path Deviations

Question

We're developing some metrics for upcoming usability testing. Our previous practice has been to count the number of clicks required to complete a task. This works relatively well but users that enjoy keyboard shortcuts can throw the stats for this off a cliff. Our testing covers electronic medical records so a user ordering, say, a medication can choose to use a pre-built drug string (e.g. 500 mg twice per day for 10 days) or enter that information manually. I don't see a way to prevent this from happening without prescribing (kind of proud of that pun) exactly what the user should and should not click on.

We've been debating a new method, comparing a user's navigation to an optimal path and recording deviations from that. We could take the ratio of those and use it similarly to our old click counting. Has anyone used this method before? How specific did you make your 'optimal path'? Was it just a glorified click this, click that breakdown or was it broader like "access this screen, input necessary values, submit"?

Answer 1

You should NOT be counting clicks as a metric for usability. Clicks are NOT correlated with user success OR satisfaction.

For more info refer to UX Myth #2: All pages should be accessible in 3 clicks and the empirical studies linked therein.

Path deviation sounds more useful, but what you should be doing is gathering qualitative data instead of quantitative data.

This is done through 1 on 1 usability tests where the facilitator sets a task for the user and observes. The value in this method is that not only do you observe tasks that the user fails, you can also observe difficulties users face even in tasks they complete successfully. Quantitative studies like yours are usually very narrow and don't highlight where the major issues in the interface are.

For more info on why quantitative studies aren't as useful see Risks of Quantitative Studies: Jakob Nielsen's Alertbox.