0

Which WCAG 2.1 success criteria can be tested

  1. fully automatically,
  2. semi-automatically with intellectual validation,
  3. intellectually only?

It seems to me IT consulting firms exploit customers' ignorance on that topic to claim all success criteria can be tested automatically (expensively, of course), but that is certainly not true. Any experience on this?

Improve this question

2 Answers 2

Reset to default
2

I have never seen any IT company say that all success criteria can be automatically tested. Roughly 20-30% of WCAG is possible to be automated but that number might be misleading. I think that number might be too high although companies that make scanning tools will tell you it should be much higher, of course.

The reason the number might be misleading is that you can't say that certain criteria can always be automated. A scanning tool might be able to catch some 1.3.1 issues, for example, but miss lots of other 1.3.1 issues.

Some scanning tools are decent about catching color contrast for text, 1.4.3, but it's pretty easy to trick the tools and cause false positives. And it's easy for tools to miss contrast issues.

That being said, I think there is only 1 success criterion that can always be automated and should never cause false positives or be missed.

  • 3.1.1 Language of Page

A tool should be able to look at the main <html> element and see if it has a lang attribute. It's hard to screw that up.

A close one to automate is

  • 2.4.2 Page Titled

It's easy to check if there's a <title> element in the <head>, but 2.4.2 says there needs to be a title and it has to be descriptive. There's no way a tool can tell you if it's descriptive.

All the other success criteria are similar. Some failures can be found, some cannot, and some will be false positives.

Improve this answer
1

I would say that the best answer is “All and none”, similarly to “Which photographs can be tagged automatically?” and “Which languages can be translated automatically?”. Automated routines make mistakes, and human testers also make mistakes. Both can be trained to do better. And experts do not always agree on what conforms to a success criterion. The most reputable accessibility consultants use both, including the use of automatic filtering to direct human effort more efficiently.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.