Defining usability: Is there a clear terminology structure for different methods?

Question

I am starting a small team project (as part of a placement) which involves the assessment of human machine interfaces through usability testing/assessment. We are currently producing a review of different usability testing/assessment methods and have been a little stumped by the variation in terminology.

Specifically, the terms 'Usability Assessment', 'Usability Testing', 'Usability Inspection' have caused some debate amongst the group. Although it may seem trivial or abstract, for the purpose of structuring the review, semantics is pertinent.

My understanding is as follows:

• Usability Assessment/Evaluation: a generalist term which encompasses the full body of methods which can be used assess/evaluate the usability of an interface

• Usability Testing: a body of usability assessment techniques which evaluates a product by testing it on end-users. Specific methods include: heuristic analysis with experts/users, collection of usability metrics, thinking aloud protocol

• Usability Inspection: a body of usability assessment techniques where an evaluator/designer inspects the user interface. Specific methods include: heuristic analysis, cognitive walk-through, ISO guideline evaluation.

• Other Usability Measures: surveys, interviews, focus groups.

How "correct" is my current definition structure? Is there a standardized procedure for defining forms of usability assessment? If so, what academic/concrete references outline this?

Answer 1

I think Interaction Design¹ offers quite a solid classification.

Evaluation strategies include:

• User testing - any method that requires users. This includes controlled lab experiments, interviews, questionnaires, user observations, field observations, remote testing, etc.
• Inspections - done by experts, largely by means of reason. Includes cognitive walkthroughs, Heuristics, etc.
• Analytics - done mostly by experts. Analysing data that was automatically gathered. Like Google Analytics analysis.
• Predictive - done mostly by experts. A systematic and quantitative analysis based on models. GOMS, etc.

Usability evaluation?

For me the real problem is the definition of usability, on which there is little agreement. I strongly support the following definition:

A quality of a design that yields a lower performance load — cognitive or physical — from those interacting with it.

which encapsulates most of the other definitions partials (learnability, errors, etc.).

But it does not capture the emotional response of users, which is often a subject in evaluations. And so really, the whole thing should be UX evaluation rather than usability evaluation.

_{¹Jenny Preece, Helen Sharp, Yvonne Rogers (2015) Interaction Design - Beyond Human-Computer Interaction, : John Wiley & Sons, Limited.}

Answer 2

I would think about it in this way:

1. Source of information: if you are looking for existing knowledge or information, or extending existing knowledge or information then it is a research activity. If you are generating new information or repeating existing procedures or protocols then it is a testing activity.
2. Person conducting or carrying out the activity: if it is done by internal people with knowledge of the system then it is a review, whereas if it is done by external people with little knowledge of the system then it is an independent evaluation.
3. Target(s) of the activity: if the research and testing is done internally or if it is one actual users
4. Type of information collected: usually a distinction needs to be made between qualitative and quantitative data because it determines how they should be processed and presented.

As you say, different terms may mean slightly different thing for people, but looking at the activity in terms of this type of breakdown or definition is less ambiguous and easier to explain.