What metrics to track usability of a analytics dashboard? [closed]

Question

I understand that with something like Amazon, you can track how long it takes for a person to complete their goal (ie. buy a product). But for something like an analytics dashboard, for example:

What metrics would you use to see if this dashboard is useful? Is there even such a metric? I was thinking of length of time a user spends on specific pages. Maybe widget interactions (but this wouldn't work for widgets that don't have interactions). Is the only way to see if a dashboard is useful through user interviews?

thanks.

Answer 1

There are a few ways to track anything that you put up in your product. In your case the dashboard that you show. A few ways are here:

Analytics: Using something like Google Analytics you can track events (Understand how these work using google analytics resources online)

Try using heat maps: There are a lot of products that can give you heat maps which can help you decide what part of the dashboard seems more interesting to the majority of your users.

Try these to begin with and you’ll see you have a better understanding to take the next actions. Hope that gives you some direction.

Answer 2

This is an interesting question because I think often dashboards are not designed very well so users do not get the benefit of being able to take immediately action when viewing the information.

I think what you can do is try to follow the logic of how the dashboard is put together and track the trends in dashboard views to the pages that are relevant to it and see if there is change in user behaviour.

As mentioned before, there are alternative tracking tools like eye or mouse tracking that can show attention of the user on the various sections of the dashboard, but these still need to be combined with other information to give you more context about whether the user found it helpful or easy to user or not.

Better still, redesign the dashboard so that each widget or section leads to an actionable outcome so that after the user views the information there is a reason to perform an action (or a decision not to). If you make each widget specific to another page or pages, you will be able to track userflow from the dashboard to that page as well.

Answer 3

There are a few metrics that could be used to evaluate the effectiveness of a dashboard:

5 Second Test Responses

The 5 second test is a simple test to conduct. Simply place the dashboard in front of a user for 5 seconds, then remove the dashboard. After that you ask them a few questions about what they saw. some questions that could be asked:

• What do you remember from the dashboard? (Evaluate what is most striking or memorable)
• What metric was performing poorly? (Ensure negative performance is easily recognized)
• Where on the screen do you think traffic source information could be found? (Evaluate Information scent)

Time to Insight

This is a variation of the typical task completion time. Basically, present the dashboard to a user and then ask them to find specific insights from the data in the dashboard.

With the example image, you could ask "What are your two top performing Traffic Channels?" and "What are your two worst performing Traffic Channels?". After asking the question, simply time the response. Your metric would be the time to get a response and if the response is accurate.

Remember The time in isolation, doesn't mean much. There will be some nervousness of being observed and being put on the spot, plus the environment is simulated (a lab room is different from a noisy cubicle or crowded coffee shop). To use this metric effectively, you will need a sample of several participants and you will need a comparable metric. The comparable metric can come from a previous design or a competitor's dashboard.

Comparing Data Sets

The unique value that a dashboard provides above a single chart is the ability to compare various data sets against each other so a user can find insights into the environment or domain the data represents.

Through a one on one session, you could ask a user what conclusions they draw from the data. The goal would be to hear insights that show evaluation of various graphics on the dashboard. For example, "I see that in the past 3 weeks the traffic has grown, but also bounce rate has increased. It seems that we are attracting the wrong kind of visitor."

Click Count on Drill Down Links

Many dashboards provide high level trends, aggregated insights, or specific callouts of category performance. Given the snapshot of information provided, many dashboard visualizations will have links leading to a full reports.

Looking at the click counts and clicks per unique visitor on each drilldowns will show if the dashboard's visualizations are leading users to explore further.

Remember If the dashboard is live (not a mockup or prototype), then the underlying data will have some influence on this. Uneventful data will not entice a user to drill down (and rightfully so).

Underlying Goal of (Dashboard) UX Metrics

In the end, UX metrics will focus on if a user was able to quickly, effectively, and delightfully accomplish their goals. Any metric used to evaluate a dashboard would focus on if the dashboard quickly provides insight into the underlying datasets. It would also focus on if the insights provided are enhanced by the various data sets; essentially is the whole greater than the sum of its parts.