There are a few metrics that could be used to evaluate the effectiveness of a dashboard:
5 Second Test Responses
The 5 second test is a simple test to conduct. Simply place the dashboard in front of a user for 5 seconds, then remove the dashboard. After that you ask them a few questions about what they saw.
some questions that could be asked:
- What do you remember from the dashboard? (Evaluate what is most striking or memorable)
- What metric was performing poorly? (Ensure negative performance is easily recognized)
- Where on the screen do you think traffic source information could be found? (Evaluate Information scent)
Time to Insight
This is a variation of the typical task completion time. Basically, present the dashboard to a user and then ask them to find specific insights from the data in the dashboard.
With the example image, you could ask "What are your two top performing Traffic Channels?" and "What are your two worst performing Traffic Channels?". After asking the question, simply time the response.
Your metric would be the time to get a response and if the response is accurate.
Remember The time in isolation, doesn't mean much. There will be some nervousness of being observed and being put on the spot, plus the environment is simulated (a lab room is different from a noisy cubicle or crowded coffee shop). To use this metric effectively, you will need a sample of several participants and you will need a comparable metric. The comparable metric can come from a previous design or a competitor's dashboard.
Comparing Data Sets
The unique value that a dashboard provides above a single chart is the ability to compare various data sets against each other so a user can find insights into the environment or domain the data represents.
Through a one on one session, you could ask a user what conclusions they draw from the data. The goal would be to hear insights that show evaluation of various graphics on the dashboard. For example, "I see that in the past 3 weeks the traffic has grown, but also bounce rate has increased. It seems that we are attracting the wrong kind of visitor."
Click Count on Drill Down Links
Many dashboards provide high level trends, aggregated insights, or specific callouts of category performance. Given the snapshot of information provided, many dashboard visualizations will have links leading to a full reports.
Looking at the click counts and clicks per unique visitor on each drilldowns will show if the dashboard's visualizations are leading users to explore further.
Remember If the dashboard is live (not a mockup or prototype), then the underlying data will have some influence on this. Uneventful data will not entice a user to drill down (and rightfully so).
Underlying Goal of (Dashboard) UX Metrics
In the end, UX metrics will focus on if a user was able to quickly, effectively, and delightfully accomplish their goals. Any metric used to evaluate a dashboard would focus on if the dashboard quickly provides insight into the underlying datasets. It would also focus on if the insights provided are enhanced by the various data sets; essentially is the whole greater than the sum of its parts.