Under the current system I am using it is 2/1/-2 however I am beginning to doubt how fair this is; if something has 4 passes and 1 fail then surely that is 80% pass? Yet with this way of calculating points it would only be 60%.
Are fails so critical that they deserve to be punished in this way?
There is the consideration though that if a fail were to be worth 0 points then that would put a fail on a par with not relevant (e.g. a rule speaking about how to format tables on a site that has no tables). And it seems really unfair for a not relevant to be equal to a fail- on percentages this can be hidden (a 0 that counts towards the score vs. a 0 that is ignored) but in terms of brute points it will stand out.
As you can tell here I am really thinking a lot about a less important side of heuristic evaluations, the most important point is of course to find errors. but still, stake holders like numbers.
Are there any recommendations for scoring of heuristic evaluations?