Measuring UX in a data visualization application

Question

We design and maintain a web app for projects monitoring and data visualization. The product is used in an intranet with 500+ users. The main feature of the product is using a dashboard with metrics of projects of the organization. Depending on roles, users can apply filters for the viewed information, search, and have a personal portfolio of projects they are involved or following.

We're interested in defining some UX metrics to inform design team with optimization and improvement of the product based on user data. As I see it the product doesn't have clear conversion goals like for example an eCommerce site, and has very few specifics tasks to measure its completion time. The app is more to explore different aspects of the portfolio and visualize data. Some metrics are already implemented in the app like: most used filters, number of users accessing the app or distribution of users per role.

What would be the best approach or methodology to define UX KPI's in this case?

Answer 1

KPIs are what you are trying to build here - These are loosely defined by the stakeholders and then refined by talking to the users and matching what they are doing with the business requirements solved by the product. They are essentially how you tell if a product is working well for both the business and the users. Literally: Key Performance Indicators - the key metrics that show how your product is performing.

The first thing you need to do is talk to your stakeholders - You need them to tell you what business requirements should be solved by having the dashboard.

Next, sit down with your users and understand where the dashboard fits in with their daily tasks, what they use it for, and what features they might like to see in future iterations. Get them to talk you through the way they use the dashboard either in person or on a recorded screen-share video if possible.

You could also build a quick and dirty NPS survey to find how your users 'feel' about the product.

Naturally, this gives you the KPIs, User Journeys, and a Roadmap/Backlog of feature requests which can all be used to benchmark and guide the future design and development of the product.

Answer 2

There are of course heuristics (Jakob Nielsen's 10 general principles for usability that you might want to apply or the 5 qualities of usability that you can measure and align with. But you can't go wrong with having a clear problem statement and definition then validating this with the users.

A good place to start with defining measures (not necessarily KPIs) for your product or service is to look at the original problem it is trying to solve and how effective it is at doing so.

It would seem like having a centralized place such as a dashboard to view this information is that they wouldn't have to go to many other places, so you should compare the views and actions performed on this site against analytics of the traffic on other sites or pages to see if you have reduced the amount of time people spend finding this type of information.

Also, a well designed dashboard can allow users to make decisions more effectively, so that they know exactly when they need to take certain actions. You can also design or track a flow from viewing the dashboard to where they go next to see if the dashboard is effective in providing them clear call-to-actions for what they need to do after viewing the information.

These are just some initial thoughts but if you have more details about the information and data displayed perhaps it will be easier to provide an answer as to what types of things will be useful to track so you can measure and improve the design.