3

My UX team has always utilized SUS Scoring after our usability tests.

We work with Student Information System Software and after testing a new feature, we follow up with a SUS Scoring survey.

The areas we target within the SUS scoring survey are:

  • Sufficiency
  • Ease of Use
  • Confidence
  • Performance
  • Reliability
  • Design Appeal
  • Overall Satisfaction

I just watched a video on (UIE's All You Can Learn) that mentions not to bulk up your tests with SUS Scoring, but there wasn't much detail to why. I tried to do some Google searches to see if I could find some quick answers of when to use them vs. not and I didn't find much.

Does anyone in this community have some good insight on this topic?

4
  • What did the video mean by 'bulk up your tests'? Any testing that you do are essentially data points to build confidence around the assumptions that you are making about users, and unless the value of the data is less than the effort required to conduct the testing then you should try to fit them in.
    – Michael Lai
    Mar 25, 2021 at 23:07
  • Can you reach out to the instructor on LinkedIn or Twitter to get more detail? I'm interested in the answer, as well.
    – MRL
    Mar 26, 2021 at 4:58
  • I tried connecting on LinkedIn, but nothing so far. I am going to make an assumption on why he said what he did. Let me know what you think... In his video he talked about getting feedback continuously. Testing should happen all the time using third party usability testing tools, such as UserZoom. The tests he was talking about are usually unmoderated and have 3-5 questions/goals. He mentions that as designers/analyzers "our time is valuable, so do not bulk up your tests with SUS scoring". With this information does it make sense to not add SUS scoring for quick, continuous testing?
    – L. Lemmer
    Apr 14, 2021 at 14:22
  • It's a 'use of time' thing. Could you be using the time to ask questions which specifically address your particular interface ?
    – PhillipW
    Jun 21, 2021 at 8:37

1 Answer 1

1

There are a number of possible interpretations on what 'bulking up' your usability tests with SUS scores might mean, so let's look at some of them.

Given that usability tests are generally much more in-depth and is generally in the context of specific goals or tasks, you can match the SUS score to add another data point to what your assumed user experience for this feature might be. If this SUS questionnaire is administered straight after the usability testing session as opposed to a pop-up that happens some time after the feature has been released, it provides a slightly different way of interpreting or comparing between the two results.

So the first interpretation of 'bulking up' the usability test is positive, which is to provide an additional data point to your overall picture about the user experience. And the second interpretation is weighing the number of responses you get from the SUS score to the number of usability tests and using that to influence your original usability test findings.

There is also this perception that the more tests you do, and the more data that you get, the more confidence you get become about the result (depending on the type of bias or biases you fall into), and I think this is probably the danger of trying to do more tests than it is necessary to come to a particular conclusion about a new feature. Often when trying to uncover serious design or usability issues, it should be easily identifiable when you have the right types of users, even with very small numbers. However, if you are trying to figure out some subtle detail about a feature, often the type of testing you do and who you put the tests in front of will determine the outcome.

But hopefully you can get a response from the person who made the comment originally to clarify. It is good to see that you are doing different types of testing, but just make sure that you are treating the results with the proper assumptions and interpreting them in the correct context when making design decisions.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.