User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
Learnability- a component of the System Usability Scale scored 13,45. Is that below or above average?
When broke into two groups of users (role-based) one group scored 6,88 and the second scored 15,58. What conclusion can we draw from that?
Any other comments on How to interpret SUS's learnability score are most welcome.
About SUS learnability component: http://www.measuringu.com/blog/10-things-SUS.php
A quick google search lead me to MeasuringU:
The average SUS Score is a 68: When looking at scores from 500 products, we found the average SUS score to be a 68. It's important to remember that SUS Scores are not percentages. Even though the SUS ranges from 0 to 100, these scaled scores are not a percentage. A score of a 68 is 68% of the maximum score, but it falls right at the 50th percentile. It's best to express the raw number as a score and, when wanting to express it as a percentage, convert the raw score to a percentile by comparing it to the database.
Based on that, the scores you gave seem rather low, regardless of scores being percentile-based or absolute numbers.
edit: apparently I misread the OP, and it only measures 2 questions, not the full SUS. Though I expected multiplication by 5 to fix the issue this wasn quite true. As seen in point 2 on MeasuringU, learnability tends to deviate above the average, about 10%. So this means that the average of all SUS-Learnability scores would be around 75 (68*1.1=74.8) per 10 questions, or 15 for two questions.
Based on that, 13.45 avg and 6.88 are still (somewhat) low, though 15.58 is just above average.
As for the difference in group scores, I'm guessing one group had already used the software (or similar software) before? Using something for a while changes your views of it, and this can have significante effects on SUS scores;
However...
If you're using a system but you don't know the fundamentals of it (e.g. grading system) then the results are probably not very reliable. How do you know you asked the right questions? Do you have context such as comparable scores and data about similar websites/apps? A music player can be MUCH simpler to learn than an client/invoice database.
Reading between the lines here, you seem to be looking for a simple system to show your work is good. That simply doesn't exist, simplicity always comes at the cost of detail and/or quality. MetaCritic has been criticized over this a lot; how can one number properly compare Toy Story with Schindler's List?
At best you'll probably be limited to comparing the usability of several iterations of your design, at worst you'll let yourself be distracted too much by this one number. Instead of looking at the SUS score which is extremely broad and generic, try to build a survey based more around actual use cases.
How easy was it to find a song in your app versus competitors like spotify and itunes ? How easy is it to compare 2 employees experience in a field? Such specific questions are much more valuable because they also indicate the direction you should work towards, instead of just a better/worse ranking.
Recent research has brought into question whether or not it is OK to break the SUS into Learnable and Usable subscales -- for the full article, see http://uxpajournal.org/revisit-factor-structure-system-usability-scale/ -- here's the abstract:
"In 2009, we published a paper in which we showed how three independent sources of data indicated that, rather than being a unidimensional measure of perceived usability, the System Usability Scale apparently had two factors: Usability (all items except 4 and 10) and Learnability (Items 4 and 10). In that paper, we called for other researchers to report attempts to replicate that finding. The published research since 2009 has consistently failed to replicate that factor structure. In this paper, we report an analysis of over 9,000 completed SUS questionnaires that shows that the SUS is indeed bidimensional, but not in any interesting or useful way. A comparison of the fit of three confirmatory factor analyses showed that a model in which the SUS’s positive-tone (odd-numbered) and negative-tone (even-numbered) were aligned with two factors had a better fit than a unidimensional model (all items on one factor) or the Usability/Learnability model we published in 2009. Because a distinction based on item tone is of little practical or theoretical interest, we recommend that user experience practitioners and researchers treat the SUS as a unidimensional measure of perceived usability, and no longer routinely compute Usability and Learnability subscales."
In other words, the same people who initially suggested the possibility of breaking it down into two separate dimensions, are now stating that a re-evaluation of the evidence indicates that it should not be done.
