Validate user experience for expert users

Question

I am a computer scientist, and I have contributed in building a strongly technical solution (cloud-based, scalable) for the computation of large data volumes in short time. Such platform offers services for non-IT users (e.g., a software suite to be used by people working in a bank), but it obviously has to be used by IT users too (that have to install, configure and maintain it for the non-IT users).

Now I have to validate my platform, in several terms (e.g., efficiency, cost, UX).

While it is clear to me how to validate services for non-IT users (e.g., via learnability or usability or understandability etc), it is not clear to me how to validate them for IT users. The measurements that I would like to perform are about:

• is the user able to reconfigure the platform in a certain way that I suggest?
• is the user able to understand how to use the interface for the configuration of the platform?
• can the user understand the usage of each component of the platform?
• is it difficult for the user to use and manage all the components?

I would say that this is about perspicuity, efficiency, dependability (as define here). However, I feel I am missing something important here. Could you help me in understand what is missing?

Thank you for your help.

Answer 1

• General understanding (suitability for purpose, system requirements)
• Installation
• Configuration for purpose
• Monitoring & bug reporting/fixing/workarounds
• Version upgrade

...seem to be the functions to be covered. You can test for technical people the same way you do for non-technical: only the content of the tests is different. Whether you're using paper protos or live software, set the person being tested a problem to solve, then watch how easily she/he solves it and where the hangs and backtracks are for him/her.

Answer 2

Sometimes people don't know that your current users are not really the ideal users.

Think about it - if the solution was so easy a non-technical business person were able to do it, they would never need to involve a technical IT person. So maybe your solution is for the wrong person?

In any case your users are technical, but they are also humans, who may or may not know your solution, and will have specific needs and goals too. So validating them is no different to assessing anyone's needs. Just because they are technical doesn't mean the solution needs to be technical and/or difficult.

So you will be interested in a number of thing:

• Learn-ability (how quickly do they get up and running from cold?)
• Frequency of Task (do they do this all the time, or is this something they only do once a year, where they need to relearn each time?)
• Time on Task (are you making this better or worse than before?)
• Task Completion (are people failing to complete tasks? What do they do when they fail? Does it cost you when they fail?)

Remember that the ultimate goal of what you are doing is to appease some stakeholder who doesn't think like you - they think in terms of money, so everything you do should be converted into the money-saved or money-earned.

The data from monitoring these things and converting them to 'money saved' (or 'money earned') via time saved, can now be used to prove things are better than before, irrespective of what type of user is the focus.

Answer 3

IT user is yet another role you want your product to solve for. Because of that, apply the same UX/quality/performance metrics as for non-IT-Users.

The UEQ page you linked to, presents you with metrics. I feel that what you're missing is what to apply those metrics upon.

You have a good gut feeling of asking those questions like "is the user able to...". To understand what needs to get captured in terms of UX, start with creating a prioritized list of what the product really solves for.

One of the techniques you may find useful for that is Jobs To Be Done: https://blog.intercom.com/using-job-stories-design-features-ui-ux/