UX Benchmarking: How much detail should be collected from a user within an enterprise product?

Question

There's a lot of value collecting benchmark data about a user's experience over time. The problem that we are running into is how much information should we ask from our user to help us make informed decisions from data collected. We work on an enterprise product with many different personas, within many different areas of the product so I am stuck with how much data beyond our handful of KPIs should we ask our users for when the "Are you interested in providing feedback?" survey modal shows up for the multiple different user types that use the many different areas in the system. Is it valuable to get a system-wide benchmark without capturing the area that the user is most frequently in or using given that we will still collect benchmark data at a more granular level within each feature of the product?

The concern is what do we do with the data collected if it is from a user we don't know much about - the user could use multiple areas of the system and we can pull some analytics but there would be a lot of assumptions put in place. Has anyone out there that works on and enterprise product collected benchmark data that could share some insight?

Answer 1

As you mentioned already, there are a handful of KPIs that you are monitoring across the system and for all the users. That means regardless of the type of information that you are collecting (and the level of granularity), it would ideally be relevant and linked to those KPIs.

Given that there are a number of different personas (or user groups) that you are tracking, and that there are many different features of the application, the data that you collect initially might provide some clues about which personas are closely related (and therefore can be grouped together), or which features of the application share common users (and therefore can be analyzed as a common behaviour/workflow.

Unfortunately you will have to figure out how to strike the right balance in terms of the number of groups and granularity of information required, but ultimately the high level information is more useful as a benchmark or baseline over a period of time while the more granular information is more useful to help you work out the areas to focus on with your research and testing if the performance is falling between your benchmark.

Keep in mind also that metric (and KPIs) should not necessarily be used as a true reflection of performance, but rather as an indicator of change that can be influenced by a number of different factors (that require research and testing).