# Statistics and Usability [closed]

I have an engineering background and I work as an interaction designer at a website company. I started to work 3 months ago. I am new in this field.

To interpret analytics data and usability tests, what statistics knowledge do I need to learn? What should I focus on?

Beyond that, to understand how severe a usability problem is, what should I know in terms of statistics?

https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/

• A good place to start is the blog at MeasuringU: measuringu.com/blog.php Commented Jan 27, 2016 at 8:19
• Your question is too broad as it includes multiple topics. If you were to split it into multiple narrow questions, you would get useful responses.
– JohnGB
Commented Jan 28, 2016 at 5:37

Why Statistics?

Once you get beyond graphs, averages, and percents, the bulk of statistics concerns answering the question, “Is this sample size big enough to convince me that what I see in the sample actually applies in general?” This branch of statistics is called “inferential statistics.”

It’s definitely worth knowing and doing inferential statistics so you can tell when you have convincing data that one UI alternative is better than the other, or what is the likely range of impact (best case to worse case) a problem could reasonably have. Using inferential statistics can tell you when you have a sufficient sample size to terminate a usability test or A-B test and go with what you have, for example. If you have limited control of the numbers of users in the test (maybe it’s restricted by the time available), it’s still helpful to know how much confidence to have in your resulting UI design choice or prioritization.

Even if you’re only doing qualitative usability testing (concern with what the usability problems are, not how frequent they are), a good understanding of the principles of inferential statistics and the issues of sample size is helpful for explaining to stakeholders why one user flailing in a usability test of five users is justification for changing the UI intended for many thousands of users.

What Statistics to Know?

I consider an introductory college-level course in statistics to be both necessary and adequate for an interaction designer. For interpreting usability test results and data analytics, the following inferential statistical procedures cover most situations:

• T-test, and associated confidence intervals
• Chi-square
• One-way ANOVA (once in a while)
• Fisher Z transform for the Pearson correlation coefficient (once in a while)

Usability data often have small sample sizes and/or very skewed results (e.g., small proportion of conversions), so you also should also be knowledgeable on

• Binomial, and associated confidence intervals
• Fisher’s Exact
• Mann-Whitney U

All the above are typically covered in an intro textbook (except maybe one-way ANOVAs, which some consider intermediate-level statistics).

What Statistics are Not Necessary?

Very rarely does usability work involve multi-way tests (e.g., all combinations of multiple UI variables), or true multivariate analyses (e.g., measuring the combined impact of a UI on multiple metrics). What some in UX call a “multivariate” analysis is really just a case of more than two UI alternatives being compared, which should be handled with a one-way ANOVA or multi-column Chi-square.

Advanced maximum likelihood methods like log-linear regression or Cox regression are helpful, and can be better alternatives to the above. However, unless you’re doing analyses every month, it may not be worth the effort to learn them.

It honestly depends on the type of work you are completing for your website.

You might be testing a new design out, just to see if your new design is usable. Or you could be completing A/B tests, which are tests that are conducted to compare two similar designs to see which is more successful. Another thing you might do is benchmark your existing website, to help expose pre-existing usability problems.

For usability testing, I would argue that you need to know very little statistical knowledge to understand and interpret usability measurements.

However, there are certain things you can and should quantify, such as:

1. Completion Rate
2. Number of Errors