A WCAG evaluation tells you whether a page's code matches a written standard. It does not tell you whether a screen-reader user can actually finish a checkout, whether a switch-access user gets stranded in a menu that never receives visible focus, or whether someone using voice control can select a link whose accessible name doesn't match what's on screen. That gap — between documented conformance and what happens when a disabled person actually tries to use the thing — is what testing with disabled users and assistive-technology users is for. It's a distinct layer of evidence, and in Ireland it's still rare: most accessibility offers stop at an automated scan or an expert walkthrough, and few name lived-experience testing as a separate, clearly scoped service in its own right.
Why an automated scan plus expert review isn't the full picture
Automated tooling is genuinely useful — it can sweep a large site quickly and flag missing alt text, contrast failures, and structural problems at a scale no person could match manually. But the W3C's own guidance on selecting evaluation tools is direct about the ceiling: tools "can not determine accessibility, they can only assist in doing so," tools cannot check every accessibility aspect automatically, human judgement is required, and evaluation tools can sometimes produce false or misleading results. An automated report on its own is a starting point, not an evaluation.
Adding manual, expert review closes a lot of that gap. A trained auditor working through keyboard operation, zoom and reflow, focus order, labelling and a range of assistive-technology checks will catch far more than a scan alone. But an expert review — however careful — is still one trained person checking a page against a written standard, drawing on their own experience of assistive technology and their knowledge of the success criteria. It doesn't draw on the range of impairments, assistive-technology setups, personal configurations and workarounds that real disabled users bring, because it isn't testing with them — it's testing on their behalf.
What lived-experience testing adds that standards-based evaluation can't
Sessions with disabled participants and assistive-technology users surface problems that pass a conformance check and still fail in practice. A form can have technically correct labels and still be confusing to navigate at the reading speed a screen-reader user actually uses. A control can be operable by keyboard in principle and still take more steps, or more time, than someone using switch access can manage before a session or a patience limit runs out. These are usability barriers sitting on top of, or alongside, conformance — and a standards-based evaluation isn't designed to catch them, because it's checking the code against a rule, not watching a person try to get something done.
It also surfaces the real diversity of assistive-technology use that a single auditor's checks can't fully represent: screen readers, switch access, voice control and speech-recognition software, and screen magnification are genuinely different ways of using a page, each with its own failure modes — the W3C's own overview of how disabled people use the web groups these under the general heading of assistive technology, "software and hardware that people with disabilities use to improve interaction with the web," precisely because no single one stands in for the others. Someone using a screen reader and someone using switch access can hit entirely different problems on the same page — testing with people who actually use these tools is the only way to see that difference directly.
The W3C's guidance on involving users in accessibility evaluation puts the same point plainly: user involvement should "combine with evaluating conformance to WCAG to ensure that accessibility is provided to users with a range of disabilities and situations," and it works best woven through a project rather than saved for a single event at the end. That's the case for treating it as a distinct, ongoing practice rather than a one-off add-on to a report.
Recruitment and access considerations specific to running this in Ireland
Running this well takes more planning than a standard usability-testing round, and some of it is Ireland-specific.
Consent and data protection. Information revealing that someone is disabled, or the nature of their disability, is capable of being special category data under Article 9 GDPR, which the Irish Data Protection Commission notes is subject to additional protection and, outside limited circumstances, is prohibited from processing at all. In practice that means explicit, clearly worded consent, a documented lawful basis for holding it, and a defined retention and deletion plan agreed before recruitment starts — not treated as paperwork to sort out afterwards.
Incentives. Participants who use assistive technology are often giving more time and more specialised expertise than a general-population usability session asks for — they are, in effect, demonstrating a skill. Incentive levels should reflect that and be agreed as part of the study scope; there is no universal figure that fits every study, and a supplier who quotes one standard rate for every recruitment type regardless of the ask is worth questioning.
Session format. Assistive-technology setups are deeply personal — a screen reader's verbosity settings, a switch scanning speed, a magnifier's zoom and colour settings are usually tuned over months or years to one person's needs. Wherever practical, sessions should run on the participant's own device and configured assistive technology rather than a generic test rig, because a stranger's default setup can introduce friction unrelated to what's being tested. Sessions may also need to run longer, allow more breaks, or use an alternative communication format — planned with participants, not decided for them.
Device and assistive-technology diversity. A study that only recruits screen-reader users tells you about screen-reader users. If switch access, voice control or screen magnification are relevant to how real users reach the service, the recruitment plan should say so and cover that spread deliberately, rather than letting one assistive-technology type stand in for "accessibility testing" generally. This mirrors the W3C's own caution against assuming that input from one person with a disability applies to all people with disabilities — the same caution applies across assistive-technology types, not only across individuals.
None of this changes the fundamentals of running a good study: the criteria should still be tied to the research question, and consent, incentives, recording, data handling and withdrawal should still be agreed with participants before sessions begin, the same way they should for any usability-testing engagement.
How it complements — not replaces — a WCAG evaluation
Lived-experience testing is additional evidence, not a substitute evaluation. A handful of sessions — however carefully run — cannot systematically check every success criterion across every page, state and process the way a scoped WCAG evaluation can, and the W3C is explicit that results from a small number of participants cannot be generalised to represent everyone with similar, or different, disabilities. If a client needs a documented conformance position against an agreed WCAG version, that still requires the standards-based evaluation: scope, sample, method and findings recorded against the criteria, as described on our accessibility evaluation service page.
What lived-experience testing does is sit alongside that evaluation and go further on the journeys that matter most commercially — showing not just where a page departs from a written rule, but where a real person, using their own assistive technology, gets stuck, gives up, or finds a workaround the evaluation wouldn't have flagged. Used together, the two methods cover different ground: the WCAG evaluation gives you the documented baseline; the testing gives you evidence of what that baseline feels like to actually use. Findings from either still need a route to action — content fixes some teams can make directly, and anything needing design or development work can go to the client's own developer or agency of choice, the same as with any other evaluation or testing engagement.
Read next
- Website accessibility evaluation — the standards-based service this page complements
- Usability testing in Ireland — how we scope and run moderated sessions, including with assistive-technology users
- 9 questions to ask a usability testing supplier in Ireland — question 9 asks any supplier whether they can test with assistive-technology users, not just run a scan
Standards and sources
- W3C guidance on involving users in accessibility evaluation
- W3C guidance on selecting accessibility evaluation tools
- W3C: assistive technologies people use to interact with the web
- W3C WCAG-EM evaluation methodology overview
- Irish Data Protection Commission: special category data
Ready to scope it?
Use the brief call to talk through which journeys matter most, whether a standards-based evaluation, lived-experience testing, or both fit the decision you need to make.