How accurate our audits are
We measure our audit engine against public reference pages whose defects were documented by other people, and we publish the numbers here before we sell anything. Last updated 9 November 2026.
Two kinds of result
Findings come from rules and measurements: the axe-core rule set plus our own browser measurements and interaction probes (keyboard traces, focus tests, menus, pop-ups, forms). They are counted as failures. Likely issues to verify come from an AI review of the evidence (alt text that may not describe the image, a focus order that may not match the page, an error message that may not help). They carry their evidence and are never counted as failures. Advisory notes are good practice, not WCAG failures.
Results on public reference sets
| Reference set | What it is | Rules and measurements alone | With the AI review | False alarms on the fixed version |
|---|---|---|---|---|
| W3C WAI Before-and-After Demonstration | 5 pages of a 2012 demo site, 43 annotated defects | 19 of 37 criteria (51%) | 42 of 43 defects (98%) | none in scope (two placeholder links on a template page) |
| University of Washington "Accessible University" | 1 modern page: hover menu, carousel, modal dialog, CAPTCHA, validation, data table, video; 22 documented defects | 14 of 22 | 22 of 22 | none |
| UK Government Digital Service tool audit | 142 pages, one defect each; the standard yardstick for automated tools | 69 of 142 (49%); 14 of 19 form cases in full mode | not yet scored | not applicable |
| Our own fixture pair | one page with 18 planted defects and its properly built twin | all 18 | not applicable | 0 findings, 0 advisories |
Automated tools alone typically find a fifth to a half of real defects (published comparisons report 22% to 57%). Our rule layer sits at the top of that range because it measures behaviour in a real browser instead of only reading the markup; the AI review covers most of the rest on the reference sets, with every finding carrying evidence you can check.
Our rule for going live
An AI-observed check type is reported as a finding only once its precision on the reference sets is 90% or better; below that it stays a likely issue to verify. We have not yet declared that bar met for any AI check type, so today every AI observation is reported as a likely issue, never as a failure.
What we do not test
- Screen-reader output. We test the DOM, computed styles, keyboard behaviour and screenshots, not what a screen reader speaks.
- Pages behind a login or a payment step.
- The meaning of content: whether alt text describes the image well, whether an error message helps. These are reported as likely issues, with the evidence.
- A free scan never submits your forms or changes your controls; those tests run only in an audit you order for your own site.
How the numbers are made
Each reference page is scanned with the same engine customers get. The engine's own findings are compared, criterion by criterion or note by note, with the defects documented by the people who built the reference page. A finding on the fixed version of a page counts against us as a false alarm unless the criterion is outside the reference set's scope. The method, every number and its limits are kept in our calibration log, and the table above is updated whenever the engine changes.
This page describes measurements on reference pages. It is not a promise that every defect on your site will be found, and an audit is not a certification or legal advice.