Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

On SUS - the question "I think that I would like to use this system frequently" does not make sense for sites which are used infrequently, for example a mortgage payment site.

How does this question reflect the overall usability of the website? Ideas?

share|improve this question
    
Is there a specific reason you're using SUS or is it just because it's a common tool? It's an intentionally simple and blunt tool. –  Ben Brocka Nov 14 '11 at 22:35
    
And software's changed a bit since 1986... –  PhillipW Nov 15 '11 at 11:09
1  
If you haven't already, see my answer to SUS Scores and Doubts (ux.stackexchange.com/questions/5211/sus-scores-and-doubts/…). It both suggests how users can make sense of that particular item in the context of a rarely used site, and also describes the statistical procedure to test if that item "doesn't belong" with the others for your particular site and users. –  Michael Zuschlag Nov 15 '11 at 12:27
1  
@MichaelZuschlag And don't forget the answer posted by the author of the SUS himself (supposedly)! ux.stackexchange.com/questions/5211/sus-scores-and-doubts/… –  Ben Brocka Nov 15 '11 at 20:10
add comment

2 Answers 2

The System Usability Scale is mainly intended for a comparison between like systems. It's very general to allow it to compare systems, and it gives an easy reference point. It's generally assumed that the products you are comparing have similar use cases, which mitigates the problem of some questions not applying.

Comparing mortgage payment sites the question "I think that I would like to use this system frequently" is likely to be rated negatively for both similar systems. The scores presumably still be comparable for both systems because the question is equally poor for both products.

As with all cut and dried evaluations, the SUS is only a general tool and you must consider each question's validity to your use-case. You can't use it as the only analysis of usability; it's a great tool for comparison, but much too general for any single item.

If you're conducting your own usability analysis and not comparing two products, feel free to omit some questions if you feel they are confusing or irrelevant when applied to your product. Create a heuristic review that applies to your product. Here's a great Heuristic Evaluation Checklist with a great deal of questions. It's not a likert scale nor is it worded for an end user, but you can get a good idea of some questions you can ask to evaluate your system.

SUMI is a more comprehensive usability scale, practically the opposite of the "quick and dirty" SUS. SUMI can give you a lot more data but again, consider constructing a brief, personalized questionnaire for your product if you feel no questionnaires apply. SUMI is quite deep and a briefer survey is often easier to get people to complete (and complete honestly).

share|improve this answer
    
"You can't use it as the only analysis of usability; it's a great tool for comparison, but much too general for any single item." However SUS score goes hand in hand with usability, so you can use SUS to get quick idea about the usability of the system. It does not give specifics about what are the usability problems of a system. –  Illotus Aug 30 '12 at 5:20
add comment

In this type of questionnaires, the statements (called “items” in the technical literature) are not chosen mainly because they describe what usability is but because they are affected by it (i.e. the question is not “how does this affect usability” but “how does usability affect this”) and the ratings are correlated together. For the SUS, they were originally selected from a pool of 50 candidate statements based on a small study with two systems that could have been used frequently by the participants, which might explain why this question was retained.

If a statement does not seem to relate to the usability of the website for the participants, you would expect little variability, no differences between products regarding this particular statement and no correlation with the rest of the questionnaire. Technically speaking, it could limit the theoretical maximum score you could obtain with the questionnaire (i.e. bias) or contribute some random error to the total score. This would in turn slightly limit the power to detect differences but, as Ben noted, it would not bias a comparison between two websites in the same category (previous iteration of your design, competitors…)

It also conceivable that some respondents see through it and answer anyway based on their perception of the website itself, not the task it is designed to support. However, if you have several such irrelevant or awkwardly formulated statements in a questionnaire, the respondents might react negatively, complain about it or become less cooperative in general (I have seen it happen with the SUMI).

If you want to know more about the SUS, the original reference is

  • Brooke, J., (1996). SUS: A "quick and dirty" usability scale. In P.W. Jordan, B. Thomas, B.A. Weerdmeester, and A.L. McClelland (Eds.), Usability evaluation in industry (pp. 189-194). London: Taylor and Francis.

I also gave some other interesting references in this answer to another question about the SUS You can also check this question which raised similar issues and even includes an answer from the creator of the SUS.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.