Is it valid to administer a SUS questionnaire after a system training session?

Question

We would like to do some tests to determine usability and learnability of a new system we have produced. We have a very short time period, so right now we are only looking at something quick and high-level.

We will be have a few training sessions with new users. In these training sessions they will be taught how to use the new software. (It is a very complex system requiring specific knowledge, and it's unlikely users would become adept through just the training sessions). They are required to do the training session, so I cannot really test on users who haven't done the training.

I am thinking about administering a SUS questionnaire at the end of the training sessions. Would this still produce valid results? My understanding is that it's generally used after usability tests with structured tasks (and not after being trained in the system).

Answer 1

As with any research or testing method, there are aspects that may influence results which you'd need to consider.

This is just an example of one of them, you could perhaps treat the specific answer to questions around learnability as "less valid" but at the same time, perhaps you were looking to validate your training program and a low score around whether the user needs training is a good thing?

Because the SUS questions are so generic (intentionally so) you can really read much into them regarding actionable results, it's more a marker in the sand, so with that in mind, I would argue they are still valid.

I would be sure to present any results with any assumptions, caveats you've considered to show you've considered them and present your assessment of what those assumptions and caveats might mean for the results.

Answer 2

how are you? I believe you can use some questions from SUS, eliminating the parts we asked "if the person would need some training". Moreover, questions focused on interface and usability, in my view, may be valid.