I have a Master's degree in HCI. As a master's student, I had to pass some Stats courses at university. Within these courses, we were familiarized with tests like T-Tests and we were required to run them on datasets from participants (e.g. datasets showing users' performance on prototypes) to compare different prototypes.
My question is are these tests used in real-world projects which you have experienced in your career so far?
As you know, statistical analysis is one way of providing some level of quality assurance to the results from user testing, and these come with their own limitations and constraints (e.g. sample size affecting significance and confidence levels).
Jeff Sauro is someone who has spent a fair bit of time writing and analyzing many of the statistical analysis used in UX research, so you can read about some of his work over at his website MeasuringU
In real-world projects, unless you are working for a large company with lots of resources dedicated to user research, you are more likely than not to be working with less than perfect datasets (e.g. Adobe or Google Analytics that capture website traffic) or very small sample size user tests (and the method used to collect information may also vary).
Therefore, more often than not it is about applying the concepts in statistics to help understand the relevance or significance of the results from your analysis rather than being able to apply the statistical methods to make conclusions about the findings.
Personally, I think a sound grasp of statistics definitely helps you to educate people on data literacy and help them avoid some of the biases that we have when looking at information or data that is being presented.
