Opened 5 weeks ago
Last modified 8 days ago
#66015 assigned defect (bug)
Introduce Axe-core accessibility automated testing for the admin pages
| Reported by: | afercia | Owned by: | afercia |
|---|---|---|---|
| Priority: | normal | Milestone: | 7.2 |
| Component: | Build/Test Tools | Version: | |
| Severity: | normal | Keywords: | has-patch has-unit-tests axe-core |
| Cc: | Focuses: | accessibility, javascript, coding-standards |
Description
Now that E2E tests in Core are in place, powered by PlayWright, I'd like to propose to introduce automated accessibility testing of the admin pages.
The implementation is not complex and would require:
- Add the
@axe-core/playwrightpackage. - Add an E2E test that runs the axe-core checks on a set of pages.
By default, axe-core will render the page in a browser and analyze the rendered DOM.
If any interaction with a specific page is necessary to test the page in a different state, E2E test have all the tools to do so.
To be determined:
- The set of pages to be checked.
- The actual architectural implementation.
Change History (15)
This ticket was mentioned in PR #13348 on WordPress/wordpress-develop by @afercia.
5 weeks ago
#1
- Keywords has-patch has-unit-tests added
This ticket was mentioned in Slack in #accessibility by joedolson. View the logs.
5 weeks ago
@afercia commented on PR #13348:
5 weeks ago
#4
At the moment, the tests fail because there are a few accessibility violations that need to be fixed. This is exactly what we want to achieve: automated accessibility testing to catch regressions or prevent new accessibility issues from being introduced.
@afercia commented on PR #13348:
4 weeks ago
#6
It is worth noting that in the Pages sceen, the em dashes — used to visually communicate the pages hierarchy are considered by Axe-core as 'surrounding text. As such, the page links are reported as violation of the link-in-text-block rule: Ensure links are distinguished from surrounding text in a way that does not rely on color`.
We can try to ignore this violation for now.
#7
@
4 weeks ago
- Owner set to
- Status new → assigned
Note that there are three pending issues (with PRs) in the Gutenberg repo that need to be fixed and included in Core for the Axe-core checks to pass:
https://github.com/WordPress/gutenberg/issues/82329 (merged)
https://github.com/WordPress/gutenberg/issues/82659
https://github.com/WordPress/gutenberg/issues/82826
@afercia commented on PR #13348:
4 weeks ago
#8
Note: once https://github.com/WordPress/wordpress-develop/pull/13368 is fixed, a new exception should be added to the ignores:
for the rule 'aria-allowed-role'
we should add a pattern selector '.attachments-browser .attachments-wrapper #__attachments-view-* li', to be excluded. It's another occurrence of the known false positive in axe-core, which reports as violation ARIA roles applied to <li> elements.
This ticket was mentioned in Slack in #accessibility by afercia. View the logs.
3 weeks ago
This ticket was mentioned in Slack in #accessibility by joedolson. View the logs.
3 weeks ago
@afercia commented on PR #13348:
2 weeks ago
#11
Note: pages that render React components may trigger false negatives / positives simply because when the Axe-core checks run the page doesn't fully render its content yet.
For example, without the waitInterval in the configuration, the Settings - Connectors page check reports one violation:
Found 1 accessibility violation(s): --- Violation #1 --- Rule ID: page-has-heading-one Impact: MODERATE Failure: Ensure that the page, or at least one of its frames contains a level-one heading Help: Page should contain a level-one heading Help link: https://dequeuniversity.com/rules/axe/4.13/page-has-heading-one?application=playwright Failing elements: - Target Selector: html - HTML Snippet: <html class="wp-toolbar" lang="en-US">
This is a false positive. The main H1 on the page hasn't rendered yet.
Vice-versa, there are two false negatives as in: with the waitInterval in place, two legitimate violations are reported.
Will try to find a better way to detecxt when the page is fully rendered other than a magic timeout number.
This ticket was mentioned in Slack in #accessibility by joedolson. View the logs.
2 weeks ago
This ticket was mentioned in PR #13682 on WordPress/wordpress-develop by @afercia.
2 weeks ago
#13
See https://core.trac.wordpress.org/ticket/66015
As promised to @joedolson , here's a quick PR to test the Axe-core checks on the Revisions page.
This PR isn't meant ot be merged. It is just aimed to validate an integrate accessibility issues findings on the Revisions page.
What it does:
- Creates a post with 3 revisions by adding 3 blocks and saving the post as draft each time.
- Accesses the Revisions and scans it for violations.
- Makes the page display the previous revision.
- Scans the page again.
I will post the reported violations in a following comment.
To run the test:
- Run
npm install. - Run
npm run test:e2e -- tests/e2e/specs/accessibility.spec.jsto get the violations report in your terminal. - Or append the
--uiflag to run the Playwright GUI:npm run test:e2e -- tests/e2e/specs/accessibility.spec.js --ui- Once the GUI starts, click the Play button highlighted in the screenshot below.
- Once the scan is complete, you can hover each step of the 2 tests to display the state of the UI at that time.
Use of AI Tools
@afercia commented on PR #13682:
2 weeks ago
#14
Reported violations:
First test: 1 violation: --- Violation #1 --- Rule ID: nested-interactive Impact: SERIOUS Failure: Ensure interactive controls are not nested as they are not always announced by screen readers or can cause focus problems for assistive technologies Help: Interactive controls must not be nested Help link: https://dequeuniversity.com/rules/axe/4.13/nested-interactive?application=playwright Failing elements: - Target Selector: html > body > #wpwrap > #wpcontent > #wpbody > #wpbody-content > .block-editor > #editor > ._0ebd6d9bf8b95794__wrapper > div > .components-snackbar-list > div > .components-snackbar-list__notice-container > .components-snackbar - HTML Snippet: <div class="components-snackbar" tabindex="0" role="button" aria-label="Dismiss this notice" data-testid="snackbar"> Second test: 4 violations: --- Violation #1 --- Rule ID: aria-allowed-role Impact: MINOR Failure: Ensure role attribute has an appropriate value for the element Help: ARIA role should be appropriate for the element Help link: https://dequeuniversity.com/rules/axe/4.13/aria-allowed-role?application=playwright Failing elements: - Target Selector: html > body > #wpwrap > #wpcontent > #wpbody > #wpbody-content > .block-editor > #editor > ._0ebd6d9bf8b95794__wrapper > div > .editor-editor-interface > .interface-interface-skeleton__editor > .interface-interface-skeleton__body > .interface-interface-skeleton__content > .editor-revisions-canvas__content > .editor-visual-editor > .components-resizable-box__container > div:nth-child(1) > .block-editor-iframe__container > .block-editor-iframe__scale-container > iframe, html > body > .editor-visual-editor__post-title-wrapper > h1 - HTML Snippet: <h1 contenteditable="false" class="wp-block wp-block-post-title block-editor-block-list__block editor-post-title editor-post-title__input rich-text" aria-label="Add title" role="textbox" aria-multiline="true" style="white-space: pre-wrap;"> --- Violation #2 --- Rule ID: aria-input-field-name Impact: SERIOUS Failure: Ensure every ARIA input field has an accessible name Help: ARIA input fields must have an accessible name Help link: https://dequeuniversity.com/rules/axe/4.13/aria-input-field-name?application=playwright Failing elements: - Target Selector: html > body > #wpwrap > #wpcontent > #wpbody > #wpbody-content > .block-editor > #editor > ._0ebd6d9bf8b95794__wrapper > div > .editor-editor-interface > .interface-interface-skeleton__editor > .interface-interface-skeleton__body > .interface-interface-skeleton__sidebar > .interface-complementary-area__fill > #edit-post\:document > .editor-sidebar__panel > #base-ui-\:r1h\: > .editor-post-revisions-timeline > .dataviews-picker-wrapper > .dataviews-layout__container > .dataviews-view-picker-activity - HTML Snippet: <div role="listbox" aria-multiselectable="false" aria-busy="false" class="dataviews-view-picker-activity" tabindex="0" aria-activedescendant=":r8u:"> --- Violation #3 --- Rule ID: color-contrast Impact: SERIOUS Failure: Ensure the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds Help: Elements must meet minimum color contrast ratio thresholds Help link: https://dequeuniversity.com/rules/axe/4.13/color-contrast?application=playwright Failing elements: - Target Selector: html > body > #wpwrap > #wpcontent > #wpbody > #wpbody-content > .block-editor > #editor > ._0ebd6d9bf8b95794__wrapper > div > .editor-editor-interface > .interface-interface-skeleton__editor > .interface-interface-skeleton__body > .interface-interface-skeleton__sidebar > .interface-complementary-area__fill > #edit-post\:document > .editor-sidebar__panel > #base-ui-\:r1h\: > .editor-post-revisions-timeline > .dataviews-picker-wrapper > .dataviews-layout__container > .dataviews-view-picker-activity > #\:r92\: > ._19ce0419607e1896__stack > .dataviews-view-picker-activity__item-content._19ce0419607e1896__stack > .dataviews-view-picker-activity__item-description > .editor-post-content-information > .components-truncate.css-tf28v4-Text-base-textStyles.components-text - HTML Snippet: <span data-wp-c16t="true" data-wp-component="Text" class="components-truncate css-tf28v4-Text-base-textStyles components-text">2 words, 1 minute read time.</span> - Target Selector: html > body > #wpwrap > #wpcontent > #wpbody > #wpbody-content > .block-editor > #editor > ._0ebd6d9bf8b95794__wrapper > div > .editor-editor-interface > .interface-interface-skeleton__editor > .interface-interface-skeleton__body > .interface-interface-skeleton__sidebar > .interface-complementary-area__fill > #edit-post\:document > .editor-sidebar__panel > #base-ui-\:r1h\: > .editor-post-revisions-timeline > .dataviews-picker-wrapper > .dataviews-layout__container > .dataviews-view-picker-activity > #\:r92\: > ._19ce0419607e1896__stack > .dataviews-view-picker-activity__item-content._19ce0419607e1896__stack > .dataviews-view-picker-activity__item-fields > .dataviews-view-picker-activity__item-field > .dataviews-view-picker-activity__item-field-value > ._3896336788892095__items-row.cc39999a5fb580d4__expanded-row[data-wp-component="HStack"] > .fields-controls__author-name - HTML Snippet: <span class="fields-controls__author-name">admin</span> --- Violation #4 --- Rule ID: nested-interactive Impact: SERIOUS Failure: Ensure interactive controls are not nested as they are not always announced by screen readers or can cause focus problems for assistive technologies Help: Interactive controls must not be nested Help link: https://dequeuniversity.com/rules/axe/4.13/nested-interactive?application=playwright Failing elements: - Target Selector: html > body > #wpwrap > #wpcontent > #wpbody > #wpbody-content > .block-editor > #editor > ._0ebd6d9bf8b95794__wrapper > div > .components-snackbar-list > div > .components-snackbar-list__notice-container > .components-snackbar - HTML Snippet: <div class="components-snackbar" tabindex="0" role="button" aria-label="Dismiss this notice" data-testid="snackbar">
![(please configure the [header_logo] section in trac.ini)](/chrome/site/your_project_logo.png)
Trac ticket: https://core.trac.wordpress.org/ticket/66015
Proof Of Concept (POF) to add axe-core accessibility tests for the admin pages.
For now, this POF only scans one page.
Re-introduces the bug fixed in https://core.trac.wordpress.org/changeset/62851 for testing purposes.
npm installand then build.npm run test:e2e -- tests/e2e/specs/accessibility.spec.js## Use of AI Tools
None