Using Speech Recognition Software with the Add Media Panel

[grahamarmfield]

Linked to #23560 this ticket specifically concerns the speech recognition user's accessibility experience of the Add Media functionality.

I'm using Dragon Naturally Speaking with IE9 on Windows 7 - a typical setup as Dragon works best with IE.

Within the Edit Post screen I can use Dragon to action the Add Media link successfully. The command in Dragon for such an action is "Click Add Media". This works, but then I run into the following problem:

• With the exception of Set Featured Image, none of the other links on the panel appear to be directly accessible with Dragon.
• If I select Set Featured Image I can't action any of the other links.

Dragon users can use voice commands to replicate pressing the tab key. The experience then mirrors that outlined in #23560 - but of course this cannot be used to select the images or other files.

It is possible for Dragon users to interact with screens using mouse commands but it is an incredibly laborious and time consuming process - used only as a last resort.

If one of the images is selected, the information panel for that images opens to the right. Unfortunately none of the input fields (for title, alt, etc) are directly available to Dragon. Ironically, the Insert Into Post button can be accessed directly with a voice command.

Some investigation needs to be done as to why most of the links and input fields cannot be directly accessed by Dragon when the panel is opened. This is a parallel with the situation on the Theme Customizer panel.

It is interesting that the Set Featured Image link and the Insert Into Post button can be directly accessed. What is different about them?

[tomauger]

Graham, the Media Modal has significantly changed in 3.9 can you re-run your tests and update this ticket?

[joedolson]

Graham - if you could re-test this, that would be great.

[joedolson]

We're going to have this checked over by the accessibility testing team to see where we stand on this question.

[afercia]

Definitely needs to be re-tested after all the relevant changes in the last months and before we can make a decision and a schedule for a future release.

[rianrietveld]

Test results on: ​https://make.wordpress.org/accessibility/2016/05/13/20th-wordpress-accessibility-test-add-media-with-speech-recognition-software/

[afercia]

Either we should make an action plan or close this ticket :) Moving it to Future release, as it is something that at the very least needs a decision.

[GrahamArmfield]

Replying to afercia:

Either we should make an action plan or close this ticket :) Moving it to Future release, as it is something that at the very least needs a decision.

I think there is a piece of work being done to improve media - I read about it on the core blog or WP Tavern (can't remember which).

Maybe efforts should be made to influence that in an accessible way rather than making specific changes to existing functionality which may get trashed in the next major release or one after.

[antpb]

Mentioned in the recent Media Component meeting, this looks like a good ticket for 5.8. Adding to the milestone.

[joedolson]

In order to start working on this, it's going to need some fairly extensive re-testing and documentation that's current. Both the media library and voice command software have changed significantly in the 8 years since this was reported.

Depending on the results of that testing, this may or may not stay in the 5.8 milestone; but we need to do the testing before the work to address this can be assessed. It's a pretty broad issue.

[joedolson]

This won't happen in 5.8, but I'm going to target it as a project to re-check for 5.9.

[audrasjb]

Should we reconsider (and re-test) the issue now that IE11 and below aren't supported anymore by WP Core? Also, the ticket was opened years ago and we're not sure the issue still occurs with the last version of this assistive technology.

Adding needs-testing keyword.

[joedolson]

This needs some time dedicated to testing before it can be considered for a release, so punting it to the future. Maybe in 6.0.

[joedolson]

Adding this to 7.0.

After 13 years, this is in dire need of a re-test; there is no reason to assume any of these issues are the same. While there may be issues, there are so many changes in all contexts (Dragon, browser APIs, and WordPress), that any prior research on this ticket is likely mooted.

[SirLouen]

@joedolson you asked for reporter-feedback back in the day.

I assume that you were missing testing information. But now you have re-milestoned this is because you have the test instructions?

[joedolson]

I know the test instructions; but they're basically quite specialized, since they require somebody who's comfortable with voice command tools. I've re-milestoned it because I'm going to test it myself, and break it down into whatever specific issues come out of that.

This is a very old ticket, and tons of accessibility improvements have been made in the media modal in the intervening time - but there hasn't been testing specifically using voice command.

[joedolson]

OK. Some of the issues found in retesting are obvious, others less so.

1) Opening the media library from the editor is quite difficult. When you insert an image block, you cannot click on the Media Library button in the place holder. It doesn't appear in the list of available buttons. This is because the button is hidden behind objects: the drop zone & an aria-hidden div which I'm uncertain of the intent behind. While screen readers ignore these layers, Dragon Naturally Speaking interprets the buttons as being hidden, and doesn't allow their selection or activation. This is likely true in every context where there are layers on top of buttons.

2) In the Media Library itself, it's quite difficult to select filters or select images. In both cases, I could do it easily if I used the inspector to find the name of the object - "Select [image name]" would easily select an image, and "Select [filter type]" would easily select the filter; but they were difficult to select otherwise. This is unsurprising; voice command is heavily tied to naming, and if you can't see what a control is named, it's difficult to select. This is particularly key for images, since the images are custom controls which are quite non-standard.

3) The issue cited by @GrahamArmfield above that related to select the input fields was not an issue, and I had no trouble with those.

As I see it, there are three key issues to resolve, two in the modal and one in the block editor:

1) Make labels visible for filters
2) Make labels visible for image thumbnails.

Block editor:

1) Masked controls need to be moved to the top layer. This simplest way I was able to handle this in testing was by adjusting their z-index so that they were above the mask layers. This was odd, however, as while it did work, buttons I adjusted that way still weren't selected when I used the 'Click button' command to highlight all buttons.

This pull request resolves the two major issues in the Media Library modal that block voice command users from being able to use the library, as described in the ticket.

Trac ticket: https://core.trac.wordpress.org/ticket/23562

Practically speaking, I'd like to have this extend to other list table views, but I think that's overreach for this PR.

[joedolson]

Media Grid view (after)

[joedolson]

Media modal view (after)

[joedolson]

Mobile viewport @412px (after). Note the scrolling of filters. The filters being hidden is an existing problem, this adds scroll so that they're at least *possible* to access.

[joedolson]

Mobile viewport @810px (after)

[joedolson]

Note: the issue with opening the media library from the image block does not occur using MacOS voice control.

The issue the current PR resolves is about being able to access controls using voice command, but voice command tools aren't required to test it. The issue is with information discovery: the controls have names, but they aren't exposed to the user, making them very difficult to use.

This PR exposes image names visually on images in the media library & shows filter labels.

Only applies to the modal and to grid view; there is a separate issue about making labels visible in list table views (#63728)

Testing info:

1) Open the media library in the media screen & in the media modal.
2) Upload at least one image.
3) Observe the visible label on the image
4) Observe the visible label on filters
5) Experiment with various viewports, options, to look for visual problems.

I'm open to design feedback on the title overlay. I restricted its height to avoid covering too much of the image, but it's otherwise the same design as the existing title overlay on documents.

[joedolson]

Tested with Windows speech recognition & confirmed that the issue with opening the media library from an image block is not a problem there, either. It appears to be uniquely a Dragon Naturally Speaking issue.

[ozgursar]

Patch Testing Report

Patch Tested: ​https://github.com/WordPress/wordpress-develop/pull/10832

Environment

• WordPress: 7.0-alpha-61215-src
• PHP: 8.2.29
• Server: nginx/1.29.4
• Database: mysqli (Server: 8.4.7 / Client: mysqlnd 8.2.29)
• Browser: Chrome 145.0.0.0
• OS: macOS
• Theme: Twenty Twenty-Five 1.4
• MU Plugins: None activated
• Plugins:
  • Test Reports 1.2.1

Software used for Voice Commands

I've used MacOS Voice Control for the following steps

Voice commands that worked for me

Switching between views in Media Library:

• Click grid view
• Click list view

In list view:

• Click add media file
• Click select files
• Select file_name
• Open document
• Click library (to go back)

In grid view:

• Click filter by type
• Click filter by date
• Click file_name (I can easily see the title after applying the patch and click them)

Voice commands that did NOT worked for me

In list view:

• Click file_name

In grid view:

• Click Select Files

✅ Patch is solving the problem

Expected result

• After applying the patch I can clearly see the image filenames in grid view and click them using voice commands. Without patch I have to go back to list view to see the image name.

Additional Notes

• To be able to upload images, I always need to go back to list view to be able to click Add media file button.

Screenshots/Screencast with results

Before:

After:

• I didn't remove needs-testing because it might be helpful if it's tested by multiple testers using different voice command software or devices.

[poojapadamad]

Test Report

Description

This report validates whether the indicated patch works as expected.

Patch tested: ​https://patch-diff.githubusercontent.com/raw/WordPress/wordpress-develop/pull/10832.diff

Environment

• WordPress: 7.0-alpha-20260216.164348
• PHP: 7.4.33
• Server: PHP.wasm
• Database: WP_SQLite_Driver (Server: 8.0.38 / Client: 3.51.0)
• Browser: Chrome 144.0.0.0
• OS: macOS
• Theme: Twenty Twenty-Five 1.4
• MU Plugins: None activated
• Plugins:
  • Test Reports 1.2.1

Actual Results

1. Image names are visibly displayed as an overlay on images in the Media Library (grid view)✅
2. Controls are easy to find and can be used directly with voice commands.❗️

Issue noticed:

Voice command to click on the file name did not work in list view.
Voice command to click on Select files did not work in grid view.
In grid view, the first options — “All media items” under Filter by type and “All dates” under Filter by date — are not fully visible.

===Screenshots are attached for reference:
https://core.trac.wordpress.org/attachment/ticket/23562/New%20Note.jpeg
https://core.trac.wordpress.org/attachment/ticket/23562/New%20Note.2.jpeg

[joedolson]

@poojapadamad Thanks for testing! This patch does not touch the list view; can you confirm whether this worked as expected in the grid view?

[poojapadamad]

@joedolson I was able to click on the file name with voice command on grid view but 'Select Files' did not work.
Other issue:
In grid view, the first options — “All media items” under Filter by type and “All dates” under Filter by date — are not fully visible.
Screenshot is attached for reference:
https://core.trac.wordpress.org/attachment/ticket/23562/New%20Note.3.jpeg

[joedolson]

@poojapadamad Thanks for reporting that!

Technically, the 'Select Files' is outside the scope of what this ticket is about. But it's pretty closely associated, and I believe I can see why it didn't work - the button uses aria-labelledby in a place where it should use aria-describedby.

[joedolson]

Added fixes to switch the button to use aria-describedby; tested this, and it now works using voice control. All the same data is available, it's just no longer part of the control name, and is instead added as the control description.

[joedolson]

After discussion in the accessibility team, we feel that this is a critical change for accessibility in the media library for users of voice command. It's essentially the difference between the media library being possible to use and not.

I'll review the various layouts and update screenshots.

Screenshots, after:

Notes:

• The misalignment of the menu button is unrelated; see core 64726.
• The overflowing filters being hidden behind the list of images in smaller viewports is a pre-existing problem which I did not choose to fix as part of this set of changes. I did adjust layouts so that the fact there is an overflow is visible, and made the overflow scrollable so it is possible to use.

[joedolson]

Marking for commit.

[alh0319]

I'm glad to see work being done on this, and the addition of accessible names over the images seems like the best solution to make the media library work for voice control.

Thanks for the work on this, @joedolson, and I agree that it is important to commit.

[joedolson]

In 61757:

Media: Make filters & image selection usable by voice control.

Add a visible label with the aria-label value for image selection to enable voice control users to select images. Add visible labels on filters to improve access to media filters.

Props grahamarmfield, tomauger, afercia, rianrietveld, antpb, audrasjb, SirLouen, ozgursar, poojapadamad, mukesh27, alh0319, joedolson.
Fixes #23562.

[arielmaidana]

Feedback received that the new visible Media Library grid labels can make the thumbnail view feel cluttered, especially for users who preferred the previous image-only grid view. A built-in option to hide the labels or reduce their visual prominence, while preserving accessibility for voice-control users, would be helpful.

[wildworks]

r61757 appears to display an unintended scrollbar. This is due to the overflow:hidden property being changed to auto.

.media-toolbar {
	overflow: auto;
}

I understand ​this change was intentional, but I didn't anticipate it affecting the desktop layout. This is particularly noticeable on Windows, where overflowed elements get scrollbars by default. Let's explore if there are any better approaches for the 7.0 release.

[wildworks]

Comparison of media headers in versions 6.9 and 7.0

[sabernhardt]

increases height by 12 pixels to reduce the need for scrollbars

See #23562.

Trac ticket: https://core.trac.wordpress.org/ticket/23562

## Screenshots

### Before

### After

## Use of AI Tools

None

[wildworks]

It looks like 23562.increase-height.diff and ​PR 11797 have the exact same changes.

[sabernhardt]

Ha! Then test PR 11797 :)

Height issue in Ipad screen

@mukeshpanchal27 This is a known issue. See ​https://github.com/WordPress/wordpress-develop/pull/10832#issuecomment-3961724891

This PR addresses an issue causing an overflow in the desktop layout. Resolving similar issues across all layouts would require additional effort.

[audrasjb]

This was reviewed during today's bug scrub.

Quoting @wildworks from Slack:

This is a very old ticket, but it has been reopened because a recent commit caused another regression. See https://core.trac.wordpress.org/ticket/23562#comment:64. The submitted PR needs review. However, it's not a critical issue, so it can be postponed.

Moving to 7.0.1 for further fixes/improvements.

Thanks for the PR. I’ve tested it and can confirm it resolves the issue.

However, the scrollbar seems to be caused by the spinner rendering at the bottom with visibility: hidden. In previous versions (and as expected), the spinner renders inline at the end of the filters. PR #11888 addresses this, and the scrollbar issue should be resolved as a side effect. I’d be interested to hear your thoughts on this.

[joedolson]

I took a look at these, and ​https://github.com/WordPress/wordpress-develop/pull/11888 also resolves the scrollbar, as well as fixing the media spinner positioning. I think overall that solution is a better solution for this issue. Barring disagreements, I'm inclined to commit that PR to resolve both this issue and #65275.

@yogeshbhutkar Thank you for letting me know! It seems your approach is more ideal, so let's close this PR.

[joedolson]

In 62480:

Media: Fix filter toolbar spinner alignment.

The spinner that appeared during filtering was positioned beneath the fields, causing a scrollbar in the filter wrapper container. While only visible during scrolling on MacOS, it was persistently visible on Windows.

Fixes the alignment of the toolbar to appear with predictable alignment to the select fields without generating a scrollbar.

Follow up to [61757].

Props luismulinari, yogeshbhutkar, dhruvang21, r1k0, sabernhardt, wildworks, audrasjb, joedolson.
Fixes #65275. See #23562.