9

What major 'competitors' exist for the SUS survey?

Is there anything picking up a significant degree of acceptance as a standard to the extent that SUS has?

In particular...is there anything which produces somewhat similarly reliable results with fewer questions?

  • SUS? What does that mean? – Majo0od Feb 11 '15 at 13:45
  • System Usability Scale. I'll embed a link in the question for context for those unfamiliar with the term. OP - If you're using acronyms in questions you shouldn't assume everyone knows what you are referring to. – JonW Feb 11 '15 at 13:56
  • You are asking 3 separate questions here; I'm not sure which you want answered. – Code Maverick Feb 11 '15 at 15:22
  • The same question broken up I'd say. As one: Are there any accepted, reliable competitors to SUS, with particular interest paid to shorter surveys? – the other one Feb 12 '15 at 12:42
  • I really wonder how usable these scores are? – pzv Apr 23 '15 at 16:08
6

Jim Lewis wrote an article in 2012 for Measuring Usability

The article explains the direct correlation found between SUS scores and Net Promoter Scores (NPS). They found that it was possible to predict NPS scores based on SUS scores, which means the inverse is also true.

The Net Promoter Score is a widely used survey which consists of only one question. That's a lot less than the 10 questions in the SUS survey :)

The correlation between the two scores was drawn from over 2200 users across 81 different companies and products including rental car companies, financial applications and websites like Amazon.com.

The relationship published is:

LTR = 1.33 + 0.08(SUS)
3

There are the SUMI and WAMMI from University College Cork (http://www.ucc.ie/hfrg/questionnaires/index.html). They require a fee for use and are longer than the SUS.

3

In Germany these two questionaires are used (and I think known in usability circles):

  • AttrakDif
  • UEQ

I'll cite from my answer here: https://ux.stackexchange.com/a/72905/3311 :

AttrakDif

The AttrakDiffTM questionnaire by Hassenzahl et al. (2003), developed together with User Interface Design GmbH, measures subjective assessments concerning pragmatic and hedonic qualities and the attractiveness of interactive products.

The questionnaire has the format of a semantic differential and uses 28 seven-step items on 4 dimensions which poles are opposite adjectives (e.g. complicated - simple, cheap - premium, conservative - innovative, ugly - attractive). Source: http://www.qu.tu-berlin.de/menue/forschung/laufende_projekte/joyofuse/joy_of_use/joy_of_use/measurement_methods/attrakdiff/

UEQ - User Experience Questionaire

The User Experience Questionnaire (UEQ) allows a quick assessment of the user experience of interactive products. The format of the questionnaire supports users to immediately express feelings, impressions, and attitudes that arise when they use a product. Source: http://www.ueq-online.org/index.php/what-is-the-user-experience-questionnaire/

1

Regarding the question, "In particular...is there anything which produces somewhat similarly reliable results with fewer questions?"

You should take a look at the published research on the Usability Metric for User Experience, published by Finstad (2010). The UMUX has four items and has typically been found to have desirable psychometric properties and to have scores that correlate highly with the SUS and, in some studies, have also produced scores that track very closely with SUS scores in magnitude.

In 2013, Lewis, Utesch, and Maher took two of the UMUX items to produce the UMUX-LITE. According to the abstract of that CHI paper, "In this paper we present the UMUX-LITE, a two-item questionnaire based on the Usability Metric for User Experience (UMUX). The UMUX-LITE items are “This system’s capabilities meet my requirements” and “This system is easy to use.” Data from two independent surveys demonstrated adequate psychometric quality of the questionnaire. Estimates of reliability were .82 and .83 – excellent for a two-item instrument. Concurrent validity was also high, with significant correlation with the SUS (.81, .81) and with likelihood-to-recommend (LTR) scores (.74, .73). The scores were sensitive to respondents’ frequency-of-use. UMUX-LITE score means were slightly lower than those for the SUS, but easily adjusted using linear regression to match the SUS scores. Due to its parsimony (two items), reliability, validity, structural basis (usefulness and usability) and, after applying the corrective regression formula, its correspondence to SUS scores, the UMUX-LITE appears to be a promising alternative to the SUS when it is not desirable to use a 10-item instrument."

There have been additional studies of the UMUX-LITE since then. Here is a list of papers to read on the topic:

Borsci, S., Federici, S., Bacci, S., Gnaldi, M., & Bartolucci, F. (2015). Assessing user satisfaction in the era of user experience: Comparison of the SUS, UMUX and UMUX-LITE as a function of product experience. International Journal of Human-Computer Interaction, 31, 484-495.

Finstad, K. (2010). The usability metric for user experience. Interacting with Computers, 22, 323-327.

Finstad, K. (2013). Response to commentaries on "The Usability Metric for User Experience". Interacting with Computers, 25, 327-330.

Lewis, J. R. (2013). Critical review of "The Usability Metric for User Experience". Interacting with Computers, 25, 320-324.

Lewis, J. R., Utesch, B. S., & Maher, D. E. (2013). UMUX-LITE – When there’s no time for the SUS. In Proceedings of CHI 2013 (pp. 2099-2102). Paris, France: Association for Computing Machinery.

Lewis, J. R., Utesch, B. S., & Maher, D. E. (2015). Measuring perceived usability: The SUS, UMUX-LITE, and AltUsability. International Journal of Human-Computer Interaction, 31, 496-505.

0

Promoters − Detractors

I'm not sure how you define “competitors”, but ForeSee has developed a claimed successor to Net Promoter Score: WoMI > Word of Mouth Index. It’s just a trademark on a basic premise some researchers have been using for a while (ForeSee likes to bill for simple things).

WoMI alters the simplistic NPS formula with only one extra question on the 0–10 scale: likelihood to discourage (LTD) in addition to likelihood to recommend (LTR). You then factor together the strong responders from each (9s and 10s), to find your true score.

It’s not as detailed as SUS, but as you suggest, 10 questions isn’t always ideal. I like the fact that with a simple promotersdetractors calculation I can get the fundamental score I'm after. I find that it’s the lightest weight way to get a realistic indication of feature success.

0

Jeff Sauro published a paper in the Journal of Usability Studies on a new survey he has created. In the paper, he has a good summary table of the surveys currently in use. SUPR-Q: A Comprehensive Measure of the Quality of the Website User Experience

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.