WAI-Tools Documentation of WCAG Interpretation and Test Rule
Preface
As a part of the WAI-Tools project, the Norwegian Digitalisation Agency (previously the Agency for Public Management and eGovernment – Difi) constitute the documentation of their internal interpretation of the Web Content Accessibility Guidelines (WCAG) and Accessibility Conformance Testing (ACT) Rules. This documentation is the part of deliverable “D2.4 Working Instance of Digdir Observatory” under Work Package 2 in the WAI-Tools project.
The purpose is to document the Agency’s interpretation of the WCAG success criteria and coverage of these success criteria by ACT Rules. In addition, the gap analysis between WCAG success criteria and ACT Rules has also been documented, such as aspects of the success criteria that are not covered by ACT Rules in WAI-Tools and aspects of the ACT Rules that go beyond the scope of the success criteria. This documentation is subdivided into one chapter per WCAG success criteria. Each chapter defines:
- specific details relevant to WCAG success criteria,
- purpose of success criteria,
- user accessibility needs (functional performance statements (fps)) (based on EN 301 549),
- interpretation and specification of the success criteria (Norwegian Digitalisation Agency),
- coverage of Success Criteria by ACT rules developed in WAI-Tools,
- gap analysis, and ACT Rules related to each WCAG success criterion.
The Norwegian Digitalisation Agency’s existing interpretation and specification of WCAG success criteria are limited to WCAG 2.0 with a brief overview of associated test procedures. The Norwegian Digitalisation Agency also has a fundamental interpretation and specification of the new WCAG 2.1 success criteria but does not have associated test procedures. Our contributions to the test rules development were based on our existing interpretation of the WCAG 2.0 and 2.1 success criteria.
Each chapter of this documentation has been written using a template specially designed for this documentation. This template contains two modules, one for WCAG success criteria and the other one for ACT Rule. To understand how these two modules have been filled out by the authors, the interested readers are referred to the template which can be found in Chapter 55 at the end of this documentation.
Background
This documentation has been written between 01 July 2020 and 31 October 2020 as one of the main results of deliverables in a European Commission (EC) co-funded project named Web Accessibility Initiative - Advanced Decision Support Tools for Scalable Web Accessibility Assessments (WAI-Tools) Project under Horizon 2020 Program (780057).
WAI-Tools Project
WAI-Tools is an Innovation Action project that has been active for 39 months between 1 November 2017 and 31 January 2021.
In WAI-Tools Work Package 1 the objective is “to build, implement, and validate a set of open test rules, developed through the international consensus process of W3C”. These test rules are developed by project staff through the W3C consensus process, implemented in open source engines developed and maintained by Deque, FCID (University of Lisbon), and Siteimprove.
The Norwegian Digitalisation Agency (previously the Agency for Public Management and eGovernment – Difi) is responsible for deliverable “Documentation of Digdir’s WCAG interpretation and ACT Rules” under deliverable “D2. Deployment of Test Rules” under Work Package 2. In this documentation, interpretation, and mapping of test rules developed under this project to WCAG 2.0/2.1 have been documented.
Web Accessibility Directive (WAD)
The Web Accessibility Directive (WAD), Directive (EU) 2016/2102, in force since 22 December 2016, aims to provide people with disabilities with better access to the public services websites and mobile applications. The Directive covers websites and applications of the public sector with a limited number of exceptions e.g. broadcasters, live streaming.
It is also required in WAD and its corresponding, that the member states are required to monitor the conformity of websites and mobile applications of public sector bodies with the accessibility requirements provided for in Article 4 of Directive (EU) 2016/2102 using a simplified monitoring method to detect non-compliance and an in-depth monitoring method to verify compliance.
Article 6 of Directive (EU) 2016/2102 refers to European standard EN 301 549. The content of websites and mobile applications meets harmonised standards and shall ensure at least a level of accessibility equivalent to that ensured by European standard EN 301 549, which in turn refers to WCAG 2.1 success criteria. In other words, it has been statutory throughout Europe for public sector bodies websites to be designed in accordance with WCAG 2.1.
In addition, we know from our experience and with reference to paragraph 56 of Directive (EU) 2016/2102 that there are different interpretations of WCAG success criteria across member states which requires harmonisation to monitor conformity. WAI-Tools is an initiative of a harmonized interpretation of WCAG, with associated test rules. This interpretation of WCAG is implicit in the test rules developed in WAI-Tools.
The terms compliance and non-compliance are not defined in Directive (EU) 2016/2102. The directive refers to European standard EN 301 549, where the determination of compliance is defined in the following way:
“Compliance is achieved either when the pre-condition is true and the corresponding test (in Annex C in EN 301 549) is passed, or when the pre-condition is false (i.e. the pre-condition is not met or not valid).”
For each of the requirements regarding web pages, the pre-conditions and tests are stated in the following way:
| Type of assessment | Inspection |
|---|---|
| Pre-conditions | 1. The ICT is a web page. |
| Procedure | 1. Check that the web page does not fail WCAG 2.1 Success Criteria X.Y.Z [Name of Success Criteria] |
| Result | Pass: Check 1 is true Fail: Check 1 is false |
Web Content Accessibility Guidelines (WCAG) 2.0/2.1)
The Web Content Accessibility Guidelines (WCAG) standard is developed through a W3C process in cooperation with individuals and organisations around the world. WCAG defines guidelines to make web content more accessible for people with disabilities. These accessibility guidelines cover a wide range of disabilities such as visual, auditory, physical, auditory, speech, cognitive, language, learning, and neurological disabilities.
WCAG 2.0 was published on 11 December 2008. WCAG 2.1 was published on 5 June 2018. There are 13 guidelines organised under 4 principles: perceivable, operable, understandable, and robust. For each guideline, there are testable success criteria, which are at three levels of conformance: A, AA, and AAA. All requirements (“success criteria”) from 2.0 are included in 2.1. The 2.0 success criteria are exactly the same (verbatim, word-for-word) in 2.1.
The accessibility requirements in WCAG 2.0/2.1 are mostly written in a technology-neutral language which requires interpretation of what exactly is required to comply with these requirements especially when it comes to specific technology.
Accessibility Conformance Testing (ACT) Rules
An ACT Rule is a plain language explanation of how to test a specific type of content for specific aspects of WCAG accessibility requirements (“success criteria”). An ACT Rule describes the expected results of accessibility testing tools and/or methodologies when running a conformance test against a specific part of the applicable test subject. In the context of WCAG, ACT Rules test for failures in satisfying Success Criteria. ACT rules are written for the testing process and usually more specific and limited to a single aspect of a WCAG accessibility requirement. Each ACT Rule contains a description of the type of content under test, the test to perform, and the expected results.
The ACT Rules were written and developed by the ACT Rules Community Group (ACT-R) using W3C ACT Rules Format. One of the objectives of this project was to establish an open community by bringing together key partners from industry, government, and research, who can develop authoritative resources on accessibility conformance testing (ACT).
The ACT Rules Community Group (ACT-R) is an open forum with the aim to document and harmonise the interpretation of W3C accessibility standards such as WCAG and WAI-ARIA. ACT-R is an initiative to keep working on the development of ACT Rules and harmonising the interpretation of W3C accessibility standards even though the WAI-Tools project has been ended. This long-term process of development of ACT Rules will continuously increase the coverage of WCAG success criteria requirements which later can be used not only in WAD simplified but in in-depth monitoring as well.
According to W3C ACT Rules Format, there are two types of ACT Rules:
- Atomic rules describe how to test a specific type of solution such as website content. It contains a precise definition of what elements, nodes or other “parts” of a test subject are to be tested, and when those test targets are considered to pass or fail the rule. Atomic rules are to be kept small and specific. These rules test a single “condition” and do so without using the outcomes from other rules.
- Composite rules describe how the outcomes of multiple atomic rules can be combined into a single outcome for each test target. In short, a composite rule can have multiple “conditions”, each of these tested in separate atomic rules.
The implementation of ACT Rules is based on test mode. The Test Mode of a rule describes how a test was carried out, in the case of ACT Rule, implementation can be automated, semi-automated, or manual. In an automated implementation, the test is carried out automatically means such as software tools and without any human intervention. In a manual implementation, the test is carried out by human evaluators, it not just includes actual test procedure carried out by the evaluators but also where the evaluators helped by instructions or guidance provided by software tools. An implementation is considered as semi-automated, where the test is carried out with the help of software tools, but human evaluation or judgment is still required to decide the outcome of the test.
The outcome of evaluation using ACT Rule on a test subject or one of its constituent test targets can be one of the three following types:
- Inapplicable: No part of the test subject matches the applicability of the rule
- Passed: A test target meets all expectations of the rule
- Failed: A test target does not meet all expectations of the rule
The outcome of an ACT Rule “Passed” does not mean that a test subject is compliance with all the applicable requirements of WCAG success criteria but only compliance with a specific aspect of WCAG success criteria covered by that ACT Rule. There are a few manual ACT Rules that possibly fully satisfy some of the WCAG success criteria. However, in general, “Passed” and “Inapplicable” outcomes of ACT Rules require further testing or manual checks.
ACT Rules are open source and can be used by member states and other organisations to implement these rules in the development of manual testing procedures, semi-automatic and/or automatic testing tools.
Summary of our findings
The WCAG success criteria are generic, complex, and require a wide range of various aspects and/or situations to be compliant. It is hard to cover all the aspects and/or situations required by WCAG success criteria, ACT Rules developed in the WAI-Tools project are very specific and atomic to a specific aspect and/situation that cover a small portion of WCAG success criteria. There can be several ACT Rules to cover all applicable aspects of WCAG success criteria accessibility requirements, which means that there could still be a gap in compliance with all aspects of a WCAG success criterion.
Currently, the ACT Rules developed in WAI-Tools can be used in and limited to testing contents created using web technologies, such as Hypertext Markup Language (HTML), Cascading Style Sheets (CSS-2018), Accessible Rich Internet Applications (WAI-ARIA), Scalable Vector Graphics (SVG2).
There have been 07 Composite ACT Rules (uses outcomes from 26 Input Rules that are atomic ACT Rules) and 49 Atomic ACT Rules developed until October 2020 in the WAI-Tools project. In total including both composite and atomic, there are 56 ACT Rules that have been documented in this documentation and covers compliance and/or non-compliance of certain accessibility requirements of 30 WCAG success criteria (SC). The documentation includes 4 atomic ACT Rules that are not related to any WCAG success criteria conformance.
The WAI-Tools project is moving forward, and the goal is to develop 70 ACT Rules by the end of January 2021. The overview of ACT Rules that have been documented in this documentation along with WCAG success criteria covered by these rules are shown in Table 1 below where N/A stands for not applicable:
NOTE: We documented each ACT Rule in this documentation when the rule has been completed after the intensive development process established by the project. There have been many changes along the way until the end of the project period. We may not update all the changes that were made after the ACT Rules are documented in this documentation. Therefore, the dates when the ACT Rule was last updated (completed) and documented in this documentation are also added in each ACT Rule module.
Note: Links in the "ACT-R ID" column of the following Table 1 navigates to the external webpage (ACT Rules website) and links in the "ACT Rules Name" column of the following Table 1 navigates to the internal section of contents (ACT Rules) of this document.
| Rule No. | ACT-R ID | ACT Rule Name | WCAG SC | Type |
|---|---|---|---|---|
| 1 | 5c01ea | ARIA state or property is permitted | N/A | Atomic |
| 2 | 73f2c2 | autocomplete attribute has valid value | 1.3.5 | Atomic |
| 3 | 97a4e1 | Button has non-empty accessible name | 4.1.2 | Atomic |
| 4 | e086e5 | Form field has non-empty accessible name | 4.1.2 | Atomic |
| 5 | 2779a5 | HTML page has non-empty title | 2.4.2 | Atomic |
| 6 | b5c3f8 | HTML has lang attribute | 3.1.1 | Atomic |
| 7 | 5b7ae0 | HTML page lang and xml:lang attributes have matching values | 3.1.1 | Atomic |
| 8 | 23a2a8 | Image has non-empty accessible name | 1.1.1 | Atomic |
| 9 | c487ae | Link has non-empty accessible name | 4.1.2, 2.4.4, 2.4.9 | Atomic |
| 10 | bc659a | meta element has no refresh delay | 2.2.1, 2.2.4, 3.2.5 | Atomic |
| 11 | 674b10 | Role attribute has valid value | 4.1.2 | Atomic |
| 12 | 4e8ab6 | Element with role attribute has required states and properties | 4.1.2 | Atomic |
| 13 | de46e4 | Element with lang attribute has valid language tag | 3.1.2 | Atomic |
| 14 | bf051a | HTML page lang attribute has valid language tag | 3.1.1 | Atomic |
| 15 | 6cfa84 | Element with aria-hidden has no focusable content | 1.3.1, 4.1.2 | Atomic |
| 16 | cae760 | iframe element has non-empty accessible name | 4.1.2 | Atomic |
| 18 | 3ea0c8 | id attribute value is unique | 4.1.1 | Atomic |
| 19 | 5f99a7 | aria-* attribute is defined in WAI-ARIA | N/A | Atomic |
| 20 | 6a7281 | ARIA state or property has valid value | N/A | Atomic |
| 21 | c3232f | video element visual-only content has accessible alternative | 1.2.1 | Composite |
| fd26cf | video element visual-only content is media alternative for text | N/A | Atomic | |
| ee13b5 | video element visual-only content has transcript | N/A | Atomic | |
| ac7dc6 | video element visual-only content has description track | N/A | Atomic | |
| d7ba54 | video element visual-only content has audio track alternative | N/A | Atomic | |
| 22 | e7aa44 | audio element content has text alternative | 1.2.1 | Composite |
| 2eb176 | audio element content has transcript | N/A | Atomic | |
| afb423 | audio element content is media alternative for text | N/A | Atomic | |
| 23 | eac66b | video element auditory content has accessible alternative | 1.2.2 | Composite |
| f51b46 | Video element auditory content has captions | N/A | Atomic | |
| ab4d13 | Video element content is media alternative for text | N/A | Atomic | |
| 24 | c5a4ea | video element visual content has accessible alternative | 1.2.3 | Composite |
| 1a02b0 | Video element visual content has transcript | 1.2.8 | Atomic | |
| 1ea59c | Video element visual content has audio description | N/A | Atomic | |
| ab4d13 | Video element content is media alternative for text | N/A | Atomic | |
| f196ce | Video element visual content has description track | N/A | Atomic | |
| 25 | 1ec09b | Video element visual content has strict accessible alternative | 1.2.5 | Composite |
| 1ea59c | Video element visual content has audio description | N/A | Atomic | |
| ab4d13 | Video element content is media alternative for text | N/A | Atomic | |
| f196ce | Video element visual content has description track | N/A | Atomic | |
| 26 | 80af7b | Focusable element has no keyboard trap | 2.1.2 | Composite |
| ebe86a | Focusable element has no keyboard trap via non-standard navigation | N/A | Atomic | |
| a1b64e | Focusable element has no keyboard trap via standard navigation | N/A | Atomic | |
| 27 | e6952f | Attribute is not duplicated | 4.1.1 | Atomic |
| 28 | c4a8a4 | HTML page title is descriptive | 2.4.2 | Atomic |
| 29 | cc0f0a | Form control label is descriptive | 2.4.6 | Atomic |
| 31 | 59796f | Image button has non-empty accessible name | 1.1.1, 4.1.2 | Atomic |
| 32 | ff89c9 | Aria required context role | 1.3.1 | Atomic |
| 33 | bc4a75 | Aria required owned element | 1.3.1 | Atomic |
| 34 | ffd0e9 | Heading has non-empty accessible name | 1.3.1, 2.4.6 | Atomic |
| 35 | 7d6734 | svg element with explicit role has non-empty accessible name | 1.1.1 | Atomic |
| 36 | 9eb3f6 | Image filename is accessible name for image | 1.1.1 | Atomic |
| 38 | b20e66 | Links with identical accessible names have equivalent purpose | 2.4.9 | Atomic |
| 39 | 4b1c6c | iframe elements with identical accessible names have equivalent purpose | 4.1.2 | Atomic |
| 40 | e88epe | Image not in the accessibility tree is decorative | 1.1.1 | Atomic |
| 41 | 80f0bf | audio or video avoids automatically playing audio | 1.4.2 | Composite |
| 4c31df | audio or video that plays automatically has a control mechanism | N/A | Atomic | |
| aaa1bf | audio or video that plays automatically has no audio that lasts more than 3 seconds | N/A | Atomic | |
| 42 | b33eff | Orientation of the page is not restricted using CSS transform property | 1.3.4 | Atomic |
| 44 | a25f45 | headers attribute specified on a cell refers to cells in the same table element | 1.3.1 | Atomic |
| 45 | d0f69e | Table header cell has assigned data cells | 1.3.1 | Atomic |
| 47 | b4f0c3 | meta viewport allows for zoom | 1.4.4 | Atomic |
| 48 | 36b590 | Error message describes invalid form field value | 3.3.1 | Atomic |
| 49 | fd3a94 | Links with identical accessible names and context serve equivalent purpose | 2.4.4 | Atomic |
| 50 | 59br37 | Zoomed text node is not clipped with CSS overflow | 1.4.4 | Atomic |
| 51 | 5effbb | Link in context is descriptive | 2.4.4, 2.4.9 |
Atomic |
| 53 | ffbc54 | No keyboard shortcut uses only printable characters | 2.1.4 | Atomic |
| 54 | c249d5 | Device motion based changes to the content can be disabled | 2.5.4 | Atomic |
| 55 | efbfc7 | Text content that changes automatically can be paused, stopped or hidden | 2.2.2 | Atomic |
| 57 | 46ca7f | Element marked as decorative is not exposed | N/A | Atomic |
| 58 | 0ssw9k | Scrollable element is keyboard accessible | 2.1.1 | Atomic |
| 62 | 8fc3b6 | Object element rendering non-text content has non-empty accessible name | 1.1.1 | Atomic |
| 65 | ucwvc8 | HTML page language subtag matches default language | 3.1.1 | Atomic |
| 67 | 7677a9 | Device motion based changes to the content can also be created from the user interface | 2.5.4 | Atomic |
| 68 | qt1vmo | Image accessible name is descriptive | 1.1.1 | Atomic |
Further use of this documentation
This section describes the value of this documentation and suggests how it can be used and further developed. After WAD was introduced and adopted by the member states, the accessibility of websites, applications, and ICT services is becoming one of the legal requirements around the member states in Europe.
According to WAD and the corresponding implementing act, all member states are required to perform monitoring of websites and mobile applications. This monitoring requirement of WAD made it more important for the member states to have a harmonized testing methodology to perform consistent monitoring.
ACT Rules can be used to monitor the degree of compliance with WAD accessibility requirements, which will make it possible for the organisations to know how to fulfil accessibility requirements required by the directive in a consistent manner. Monitoring bodies in member states can use this documentation of ACT Rules and interpretation of accessibility requirements to provide guidelines to businesses and perform controls, audits, and monitoring in their countries.
The Norwegian Agency tasks also include to perform a pilot for simplified and in-depth monitoring in line with Directive (EU) 2016/2102 on a limited set of websites under Work Package 2, to establish a “demonstrator”. The testing of websites was performed using testing tools and/or open-source engines developed by Deque, FCID (University of Lisbon), and Siteimprove that have adopted the test rules developed under the project. Note that the pilot monitoring documentation which includes a description of the sampling approach, testing, analysis, data model, and reporting results is delivered as a separate deliverable in the project.
These efforts provide valuable input and documentation on how a monitoring body enforces the Web Accessibility Directive through the use of the ACT Rules and deliverables from the WAI-Tools project. The complete documentation of WCAG interpretation and test rule is documented in a PDF document.
References
- W3C, Web Accessibility Initiative (2020). Web Accessibility Initiative - Advanced Decision Support Tools for Scalable Web Accessibility Assessments (WAI-Tools) Project. Retrieved 15.10.2020, from https://www.w3.org/WAI/about/projects/wai-tools/
- European Commission (2017). Grant Agreement number 780057 – WAI-Tools.
- How WAI Develops Accessibility Standards through the W3C Process: Milestones and Opportunities to Contribute. Retrieved 15.10.2020, from https://www.w3.org/WAI/standards-guidelines/w3c-process/
- Interpretation and overview of test procedures for WCAG 2.0 A and AA. Retrieved 15.10.2020, from https://www.uutilsynet.no/english/interpretation-and-overview-test-procedures-wcag-20-and-aa/138
- Accessibility Conformance Testing (ACT) Rules Format 1.0. Retrieved 15.10.2020, from https://www.w3.org/TR/act-rules-format/
- ACT Rules Community (2020). Retrieved 15.10.2020, from https://act-rules.github.io/pages/about
- ACT Rules Community (2020). Rules. Retrieved 15.10.2020, from https://act-rules.github.io/rules/
- W3C, Web Accessibility Initiative (2020). Evaluation and Report Language (EARL) 1.0 Schema. Retrieved 15.10.2020, from https://www.w3.org/TR/EARL10-Schema/
- European Commission (2016). Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies.
- European Commission (2018). Commission Implementing Decision (EU) 2018/1523 of 11 October 2018 establishing a model accessibility statement in accordance with Directive (EU) 2016/2102 of the European Parliament and of the Council on the accessibility of the websites and mobile applications of public sector bodies.
- European Commission (2018). Commission Implementing Decision (EU) 2018/1524 establishing a monitoring methodology and the arrangements for reporting by Member States in accordance with Directive (EU) 2016/2102 of the European Parliament and of the Council on the accessibility of the websites and mobile applications of public sector bodies.
- European Commission (2018). Commission Implementing Decision (EU) 2018/2048 of 20 December 2018 on the harmonised standard for websites and mobile applications drafted in support of Directive (EU) 2016/2102 of the European Parliament and of the Council.
- ETSI, CEN & CENELEC (2018). European standard EN 301 549 V2.V2.1.2 (2018-08) Accessibility requirements for ICT products and services.