I've been conducting a heuristic evaluation on a web application with roughly 15 evaluators, all of whom are single experts (i.e. familiar with the application but not usability experts). We're using NN/g's 10 heuristics.

In short, some of the violations they've reported are hard to categorize as a violation.

As an example, if an evaluator says that the 'X' icon inside a search input doesn't clear a user's inputted string, is that a violation?

Arguably, it could fall under:

  1. Heuristic 1 - Visibility of system status: for not providing feedback by clearing the input's string
  2. Heuristic 2 - Match between system and the real world: for not following the convention that most users expect when clearing an input
  3. Heuristic 4 - Consistency and standards: for simply not following other standards elsewhere in the app where the 'X' icon does clear an input's value.

I'm inclined to categorize these types of issues as mistakes in not adhering to acceptance criteria or poor testing. Otherwise, my concern is that introducing these kinds of submissions turns the heuristic evaluation into a bug list as opposed to an evaluation of an app's user experience.

What are your thoughts? Does anyone know of articles where NN/g talks about what criteria makes a usability issue a heuristic violation?

  • Not sure I understand your question. For me, something like your example would be like the definition of an heuristic evaluation fail. You're saying it falls under 3 heuristic criteria, then you are wondering if it's a violation of those heuristics. Why do you think it's NOT a violation? or maybe, what is your criteria for fail or success of those heuristics, or your definition of violation?
    – Devin
    Commented Jul 3, 2019 at 2:58
  • Sounds like your over complicating things. Is it important to squeeze a usability failure into a category or simply acknowledge that something isn't the best user experience and work around the problem? Commented Jul 3, 2019 at 8:06
  • My actual question here is: What is the X icon for then, in the input/search field if it doesn't clear the input? Commented Jul 3, 2019 at 9:56
  • @Devin IMO, there would be no usability or design issue if QA had been done properly. Or, said another way, the issue is not the result of poor design. It's the result of sloppy coding or QA, i.e. execution.
    – naturligt
    Commented Jul 3, 2019 at 15:54
  • 1
    @naturligt If bugs affect usability or create usability issues in any way then they are worth mentioning during your evaluation. If you want to "sanitise" your evaluation you can generate two lists, one strictly for usability issues and one for bugs which then you can pass on to QA/Dev. The fact remains that the heuristic evaluation is there to identify these issues no matter how they came to be. Commented Jul 3, 2019 at 16:12

1 Answer 1


I think the example with the X-icon you give is fairly straightforward; Why is it there if it doesn't clear the input afterall?

However, for aspects in your product where you are not sure (or cannot come to an agreement) if they qualify as a heuristic violation my approach would be this:

  1. Make a note of the potential heuristic violation
  2. Create a user task that includes the element you are not sure about (in this case, create a task that asks the user to use "search")
  3. Perform user testing, asking the user what they think of the element and what they expect to happen when they click the "X-icon".

Just keep in mind, heuristic violations can happen from UI inconsistencies, design mistakes, and even bugs. So yeah, it's quite normal for a heuristic violation to look like a bug report sometimes.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.