1

I'm currently investigating the probability of introducing System Usability Scale (SUS) as a means of quantifying the comparison of new vs. old systems. The system in-question is like-for-like, and as such SUS appears to be an ideal means of gathering such user satisfaction ratings.

However, to get buy in from the wider project team I've been asked to validate my proposal, which is where I become stuck.

My immediate thought being to pull together a list of organisation names that employ SUS as part of their processes. Not necessarily the actual data, but a means of demonstrating that organisations of all shapes and sizes have used (and indeed make use of) SUS for the purpose of validation among other things.

The fact that SUS has been around for 30+ years requires justification on my part I'm afraid. And whilst I appreciate similar questions have been asked in the past, I've Googled aplenty but unable to pull anything of relevance against what I'm ideally looking for.

If anybody could point me in any kind of direction it would be highly appreciated.

Improve this question
3
  • What are the team's concerns around using SUS? Are they already conducting quantifiable user research? The difficulty compiling a list of organizational names may be that different researchers use different methods... methodology isn't always standardized across a company or even a team since the best methodology can vary based on the data of interest.
    – Kimberley D
    Oct 5, 2015 at 23:24
  • At present it's more the justification of time (and cost) with respect to requesting X number of candidates in-order to run SUS. At present there is very little in the way of UX process, we're talking very tiny steps in the field, and SUS (from my perspective) has been touted as a means of validating the work undertaken to date. I say this as it's going to be used as a means of comparing new against old systems. I think given how quickly SUS can be ran, and the plethora of articles out there that validate it, then it shouldn't be too difficult for me to convince the powers that be... Maybe!
    – Mark Bateman
    Oct 6, 2015 at 9:22
  • 1
    I wouldn't say that articles using something make it valid. Most researchers prefer cheap methods. You need to decide if your approach is budget-based (industry) or standard of rigour / confidence - based (academia). If SUS tells you that a single change in an app produces different results between two identical populations, then it is possibly attributable to that change. It's still up to you to use statistics and determine how much variation you have across users in each tested group, though. To test two radically different systems, I'd use ethnographic methods.
    – Steve Dodier-Lazaro
    Dec 5, 2015 at 12:53

2 Answers 2

Reset to default
1

As far as I know there are a few SUS studies (some say about 500 and original Brooke, J. (1986) paper just the first of them) proving it is reliable and very convenient (you can acquire reliable results form a very small sample) method of measuring user satisfaction and recommendation likelihood. But at the same time some studies suggest it is not a good predictor of actual usability Task-Performance metrics http://www.measuringu.com/papers/Sauro_Lewis_CHI2009.pdf

Improve this answer
1
  • Thanks for the response Alexander, I'll take a look at the link you've suggested and apologies for the delayed response.
    – Mark Bateman
    Oct 29, 2015 at 11:21
0

SUS is a useful 'blunt tool' that can be adapted for many situations. Nobody actually validates it, they just use it and then eventually drop it. As some studies have shown, it has an odd factorial structure in that 2 of the items are really about Learnability; the other 8 are general impression type items. Added to that, most people score quite high on SUS so you need large sample sizes (precision of measurement and sample size are inversely proportional). The good thing about SUS is that it's totally free and totally in your hands.

A psychometrically validated questionnaire that gets a lot of traction is SUMI (sumi.uxp.ie). It has a standardisation base and some extra features that enable you to get a handle on what's really bothering the users. (Disclaimer: it's MINE!) The bad thing about SUMI is that it costs money but then, hey, a lot of work went into it :)#

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.