How to document continuous usability testing for monitoring over time?

Question

Current situation

Our product is almost being released to the first group of customers after a successful pilot phase. We are regularly performing usability tests. The results are a snapshot in time of the current user experience and function as a basis for further development. I create an individual report with user feedback and related improvements and their priorities. After my team has seen the report and we defined related activities and user stories the report has no value anymore and can be disregarded. This works well for now.

Future need

However, I would also like to track the high level overall quality and user experience as a basis for strategic more high level decisions and to monitor performance over time. I couldn't find any examples let alone best practices of how to document this. I'm looking for a solution that is easy to access, update and analyse as well as being lightweight.

Can anyone point me in some direction?

Answer 1

I've seen SUS score tracked along with production deployments. Then you can track how new features and product changes affected usability over time. This helps to track the overall usability of the app/site over time.

Answer 2

Well. In production you can make good use of the Git and create deploys that are versioned and documented that you can always have a look at historically.

Answer 3

Versioning & analitycs can systemize progress over time:

Major | Minor | Patches

1.0.0

1.0.1

2.0.1

---

At the time of creation of the 10-20 version, you can easily place this data on the X axis in the chart.

The Y axis can represent anything (new users, views list, ratings etc. )

Answer 4

Including both of the answerer's solution but containing more than that, GitLab is doing it very well. More than this, they reveal their progress plans, hiring process, expectations to be done by the upcoming years, and every data related to UX roles in these positions presented via charts and graphs if you look further inside the link above.

So to say, keeping everything live on open-source is also my way of thinking what I'm trying to implement in my current situation but, incase you (or authorized people in the company) may prefer to keep it closed to everybody, it's still good idea to serve this information available to the employers is still valuable.