There are a variety of references and approaches to prioritizing usability problems, but the I have not identified any single approach that captures all the factors I feel are necessary to include in evaluating severity.

According to Jakob Nielsen, a severity rating should be determined by factoring the following:

  • Frequency - Is it common or rare? Does it occur on a red route?
  • Impact - Will it be easy or difficult for users to overcome?
  • Persistency - One-time problem a user can overcome once they know about it or will users repeatedly be bothered by the problem?

Other sources approach severity by measuring the three factors outlined by the ISO definition of usability which include:

  • Efficiency - Can users still complete tasks, goals? Can they do what they want to do? The resources expended in relation to the accuracy and completeness of goals achieved.
  • Effectiveness - How much effort is added per the problem? The accuracy and completeness with which specified uses can achieve specified goals.
  • Satisfaction - What do the users think about the products ease of use? The comfort and acceptability of the system.

Additionally, neither severity score factors listed above account for the impact the problem has to the business, e.g., direct impact to revenue, brand equity, profits, perceived value, etc.

Last, I have not seen any factors include economics, i.e., how much effort (cost) will it take to resolve the issue. Fixing a medium severity problem with marginal effort compared to a sever problem that's extremely costly can make good sense.

So I'm curious, has anyone implemented a process for evaluating and prioritizing usability problems that include all the above factors (or perhaps ones I'm not thinking) and if so, would you care to share your story?

Factors to be included;

  • Frequency
  • Impact
  • Persistency
  • Efficiency
  • Effectiveness
  • Satisfaction
  • Business profit/revenue
  • Brand equity
  • Cost to fix

On our team, we evaluate the user-based factors, brainstorm a couple of possible solutions, estimate the cost through an Agile development process (tech review and estimation meetings,) then present the viable solutions and their cost to the business decision-makers.

I know this will grate on some people, but I do think it's possible to overanalyze these things. I find that more granularity in analysis doesn't necessarily mean a better analysis. Since the analysis itself costs money, there are diminishing returns at some point.

Generally, I think of it as two core questions in comparison with other tickets/issues: how badly does it affect users, and how badly will it affect budgeting (of both time and money.) The specifics of what, exactly, you use to analyze each of those varies from project to project. (Which is probably why you get so many different opinions on it.)

  • What are the user-based factors you evaluate? I also find it interesting that you present produce viable solutions prior to presenting to business decision-makers. I agree with the overanalyzing but the intent is to generate some backing from business. The business feels that if it's not directly impact profit/revenue in a clear way, it will be reprioritized. Obviously a user-centric culture enables proper prioritization of user-experience problems but in the interim, trying to work on the 'overanalyzed' version of deliverables to help get some legs and build trust. Thanks! – user52330 Jan 11 '16 at 21:51
  • Totally agree on the second paragraph... As someone in academia I can relate to "overanalysing" and indeed there is a moment where it's completely out of place for businesses, provided they have enough experience / rigour to not make obviously wrong analyses... – Steve Dodier-Lazaro Jan 12 '16 at 1:02

I occasionally find it useful to place issues against these axis:


download bmml source – Wireframes created with Balsamiq Mockups

Which generally leads to prioritising fixes from top-left to bottom-right.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.