Automated testing WITH human factors to collect software metrics?

Question

reaching out to see if any other UXers have experience collecting automated data for software KPIs (did a quick search for related question, and didn't find anything. Apologies if this has been asked before).

Obviously, user testing is the best way to collect feedback on any product (and we do regularly perform user research). However, because of the time and resources involved in true UX testing, we are looking for other ways to validate the UX of our product in a more efficient, expedited way.

We are wanting to automate some task scripts to collect metrics such as success/failure rate, errors/ error messages triggered, time on task, etc. A huge bonus is being able to implement human factors in the script, such as familiarity with the system, age, accessibility variables, etc. (the exact data for these human factors would be based off a large sampling of our user base).

Does anyone have any tools or tests they have used to collect automated UX KPIs? Can anyone think of a way to embed human factors into these tests?

Thanks in advance for any thoughts!

Answer 1

You can embed Google Analytics into your solution. Some of the analytics will allow you to capture a few things related to human factors and the user experience.

However, Google Analytics will not answer any "why" questions (e.g. why did the user do that?) or qualitative questions (e.g. How do you feel about that form?), so Google Analytics should always be coupled with more traditional usability testing techniques to give you a more complete picture of the user experience.

I can understand the pressure to find a automated solution to gathering human factors KPI's, but it is important to communicate to your team that not everything can be determined programatically, especially when your are looking at KPI's related to human beings and their emotional experiences.

Answer 2

Try using VWO (Visual Website Optimizer) www.vwo.com. You might find what you are looking for.

Also, for a better understanding of things, you can go for a Machine Learning course by nanodegree (Udacity). Machine Learning is in trend these days. I'm sure this doesn't sound easy. But you can give it a try.