WordPress.org

Make WordPress Core

Opened 3 years ago

Last modified 6 days ago

#37486 new task (blessed)

Make emojis accessible

Reported by: afercia Owned by:
Milestone: Priority: normal
Severity: normal Version: 4.2
Component: Emoji Keywords: has-screenshots a11y-task has-patch
Focuses: ui, accessibility, javascript Cc:

Description

Splitting this out from #37432.

Currently, the WordPress Emojis are not accessible. The alt text contains the emoji character (vs a HTML encoded entity, or the emoji description) and that's a deliberate implementation choice, see 37433#comment:3 intended to allow copy and pasting the emojis.

We've discussed a bit this issue in the accessibility weekly meeting and agreed there's room for improvements. At least, starting a discussion about emojis accessibility would be very welcome.

Quickly checking how screen readers announce the WordPress emojis, there's no accessible name or description they can use:

https://cldup.com/oioYLgdQ86.png

Note: VoiceOver reads out "group" because the image is a svg file.

Just adding an aria-label attribute (I've quickly edited in the browser console) gives screen readers an accessible name to announce:

https://cldup.com/xP0gzKlsnK.png

Comparing with what others do, for example Twitter because they usually have great solutions for accessibility, they use png images and both an aria-label and a title attribute, so VoiceOVer in this example reads out both. I guess the title attribute is used just to show the browser's "tooltip" with the emoji name:

https://cldup.com/MwMY0nHtoQ.png

<img class="Emoji Emoji--forText" src="https://abs.twimg.com/emoji/v2/72x72/1f60e.png" 
 draggable="false" alt="" title="Smiling face with sunglasses" aria-label="Emoji: Smiling face with sunglasses">

The aria-label solution is mentioned also on a related issue on the twemoji GitHub: https://github.com/twitter/twemoji/issues/41

Attachments (2)

37486.diff (502 bytes) - added by pento 3 months ago.
37486.2.diff (469 bytes) - added by pento 3 months ago.

Download all attachments as: .zip

Change History (20)

#1 @dd32
3 years ago

Loading descriptions client-side of all emoji's is unlikely to be a solution which we can use, simply due to the sheer size of emoji's and the lengths of their translations (which also then need to be translated).

Can we make it display the Emoji itself to screen readers instead of group? (Which was also an intention I believe) We'll then be throwing emoji descriptions into the ball court of screen readers though (many of which may not support the full emoji language for quite some time)

#2 @peterwilsoncc
3 years ago

For the SVG copies (used by 97% of web traffic) is there a server side solution we could implement by adding the descriptions to the SVGs on the CDN?

@afercia how do screen readers handle native emoji?

#3 @afercia
3 years ago

Screen readers usually read out what browsers expose to them through their accessibility APIs. Sometimes, in limited circumstances, they try to read directly the DOM to work around known browsers inabilities to correctly expose accessible properties. So it's mostly a matter of platform/browsers support for unicode/svg and how browser expose these informations to assistive technologies. However, WordPress is not using inline SVGs. These are just standard <img> elements with no textual alternative whatsoever.

As per the group thing, seems specific to Safari+VoiceOver and I think just adding a role="img" attribute could help the emojis to be correctly identified as images. Quick test available on this codepen: http://s.codepen.io/afercia/debug/wWXQao

https://cldup.com/GxRjqq-COb.png

Native support:
interestingly, VoiceOver announces the WP Emojis skin tone modifier as "Emoji modifier fitzpatrick type" and displays it in the Caption Panel:

https://cldup.com/e3qHBKD7Kz.png

hearing just "Emoji modifier fitzpatrick type" is a bit annoying and not useful at all, one more reason to provide a textual alternative in my opinion. This made me curious though so I've tested a bit more. Seems Safari+VoiceOver on OS X El Capitan can correctly announce the native emoji characters. I guess no surprise, since this platform has a very good support for Unicode. By the way, it seems they're often announced with a slightly different name, for example "Station" is announced as "Subway station" and this happens in many, many, other cases:

https://cldup.com/7ELA_klRB3.png

Unfortunately, on other platforms support greatly varies across browsers even for common Unicode characters, see the test for the "right-pointing double angle quotation mark" on the codepen. In the worst case, emojis are announced as "symbol" and then an alphanumeric string (often different from the character code).

Descriptions size/Translations:
yep, I can understand the technical challenge. I really have no idea about the current implementation details :) About translations, maybe there are already some translations available? See below:

https://cldup.com/1keQkjA04u.jpg

#4 @afercia
3 years ago

Update:
about supported/unsupported emoji versions, Safari+VoiceOver announce the supported one:
https://cldup.com/6TGfL6oYrs.png

the first one in this example is announced as "slightly smiling face". Setting VoiceOver in a different language, for example Italian, works and the emoji is announced as "Faccina leggermente sorridente". Nope for the unsupported version 9 fallback to image though.

This ticket was mentioned in Slack in #accessibility by afercia. View the logs.


3 years ago

#6 @rianrietveld
3 years ago

  • Milestone changed from Awaiting Review to Future Release

#7 @pento
3 years ago

I've been thinking about this a little bit, and I think I have a direction worth investigating.

  • First, change the emoji element from an img to an object. This allows the SVG to interact with the embedding window. (The CDN currently doesn't allow embedding this way due to X-Frame-Options, but that can be fixed.) In the mean time, here's an SVG with the below script added, and no X-Frame-Origins restrictions.
  • Modify all of the SVGs to add a little bit of JavaScript. Something like:
<script type="text/javascript">
    window.parent.postMessage( { description: 'Rainbow flag', emoji: '1f3f3-fe0f-200d-1f308.svg' }, '*' );
</script>
  • Listen for this message in wp-emoji.js:
window.addEventListener( 'message', descriptionListener, false ):
function descriptionListener( event ) {
    // Check that the message is coming from the CDN, and is correctly formatted.

    // Find all emoji elements that match event.data.emoji

    emojiEl.setAttribute( 'title', event.data.description );
    emojiEl.setAttribute( 'aria-label', 'Emoji: ' + event.data.description );
}

This idea has several weaknesses:

  • It only works for SVG embeds, not PNG embeds. That excludes IE 8 and earlier.
  • To be properly translatable, we'd need to have different files for each language. It's not practical to update a single SVG file each time a new translation is added, due to the way the CDN works. Effectively, we'd have 1f3f3-fe0f-200d-1f308.en_AU.svg, 1f3f3-fe0f-200d-1f308.it_IT.svg, etc. I'm not sure whether we'd load emoji by site language or user client language, but that's an easy argument to have. :-)
  • It updates the DOM twice - once to add the emoji image, once to add the aria-label. I don't know how this affects screen readers.
  • Adding the title attribute doesn't cause a hover tooltip on object elements. I don't know how the aria-label behaves.
  • I don't know if this would be accepted by the upstream Twemoji project. It's unlikely that we'd need to change twemoji.js, so the primary concern is additional maintenance when emoji need to be updated - we'd need a script to add the <script> to all of them. Also, Twitter may not share their translations, so we may need to add all of the strings to GlotPress.

Finally, here are some things that don't work, while I was experimenting:

  • Embedding as an img. The SVG can't send the message.
  • Accessing the SVG DOM from the parent window, so that we don't need to do the postMessage. Cross Domain Origin restrictions.

This ticket was mentioned in Slack in #accessibility by afercia. View the logs.


3 years ago

#9 @afercia
2 years ago

  • Keywords a11y-task added

This ticket was mentioned in Slack in #accessibility by afercia. View the logs.


19 months ago

This ticket was mentioned in Slack in #accessibility by afercia. View the logs.


11 months ago

This ticket was mentioned in Slack in #core by desrosj. View the logs.


7 months ago

@pento
3 months ago

#13 @pento
3 months ago

  • Keywords has-patch added

I don't think there's any sort of realistic method for us to maintain translations for all emoji descriptions. This would add ~20k words to the translation files, in addition to causing all sorts of issues trying to load them in a fast and bandwidth-efficient manner.

37486.diff adds an aria-label and role="img" to each emoji <img> element, which changes the behaviour slightly in Safari/VoiceOver, but otherwise doesn't seem like much of an improvement.

Unless there are any better ideas, I'm inclined to close this issue, as it's not something we can make progress on.

#14 @afercia
3 months ago

Thanks for the patch!

Testing on macOS Mojave 10.14.5
Safari 12.1.1
VoiceOver 9

http://cldup.com/YvIf3w1S8J.png

First thing to note, Safari and VoiceOver now announce the "carrot" emoji. Three years ago they didn't. Progress! :) This seems to confirm that with operating systems / browsers updates, new emoji versions are natively supported and recognized.

With the patch applied, VoiceOver announces "image" instead of "group", which is a welcomed improvement. The aria-label doesn't appear to make a difference. More on this later.

Worth noting the actual behavior varies depending on the command used to read content, for example:

Read All: Control + Option + A
reads the emojis, treating them as images

Read Paragraph: Control + Option + P
ignores the emojis and reads just the text: "Hello, this is a and this is a". Note: this is the standard behavior also with inline images.

Read Sentence: Control + Option + S
Same.

Read Line: Control + Option + L
Same.

Navigating content with Control + Option + Right (and Left) arrows:
reads the emojis, treating them as images

Testing on Windows 10 Pro – version 1803 build 17134.829
Chrome 75.0.3770.100
Firefox 67.0.4
Internet Explorer 11
JAWS 2018
NVDA 2019.1.1

Before the patch:

  • Firefox JAWS: "Hello, this is a and this is a"
  • Chrome JAWS: "Hello, this is a and this is a"
  • IE 11 JAWS: "Hello, this is a graphic slightly smiling face and this is a graphic" note: doesn't read automatically the whole paragraph though and requires to use arrows to navigate through content

So it appears JAWS 2018 doesn't announce the emojis with modern browsers and surprisingly announce at least the smile with an old browser like IE 11, probably because it fallbacks to scraping the DOM for IE 11?

  • Firefox NVDA: "Hello, this is graphic slightly smiling face and this is a graphic carrot"
  • Chrome NVDA: "Hello, this is graphic slightly smiling face and this is a graphic carrot"
  • IE11 NVDA: "Hello, this is graphic slightly smiling face and this is a graphic carrot"

NVDA works very well with all the tested browsers.

With the patch applied:

  • JAWS with all the 3 browsers: no difference
  • NVDA with all the 3 browsers: no difference

Looking at the Chrome dev tools > Accessibility tab helps to understand why the aria-label makes no difference:

Before the patch, the accessible name of the emoji come from the alt attribute (which by the way contains the emoji so it relies on the fact the emoji is recognized by the OS and browser):

http://cldup.com/kSsn_fzqxT.jpeg

After the patch, the accessible name comes from the aria-label which just overrides the alt attribute:

http://cldup.com/h134tOWlVU.jpeg

So it appears the aria-label is redundant and can be removed.

Technically, also the ARIA role is redundant because <img> element have a native role of... image. However, it helps VoiceOver to correctly announce "image" instead of "group" and I'd lean towards keeping it. HTML validators may warn this role is redundant but it doesn't harm anything.

Lastly, I do realize there are no realistic methods for further improvements, so far. However, I'd like to keep this ticket open to track the issue and future progress in OSes support. If no objections, I'd like to propose to make this ticket a blessed task.

Last edited 3 months ago by afercia (previous) (diff)

#15 @afercia
3 months ago

Update: I've just upgraded to latest Windows (Windows 10 Pro – version 1903 build 18362.207) and JAWS 2019. Thinks work far better and JAWS correctly reads the emojis on all browsers.

See the JAWS speech history in the screenshot below. Note: it displays the emojis as images but they're actually announced as "slightly smiling face" and "carrot":

http://cldup.com/wtzVBZduDB.png

while with JAWS 2018 they were completely ignored:

http://cldup.com/wBe4powcep.png

@pento
3 months ago

#16 @pento
3 months ago

  • Milestone Future Release deleted
  • Type changed from defect (bug) to task (blessed)

Thank you for the extensive testing, @afercia! 🙂

37486.2.diff removes the redundant aria-label.

I'm fine with changing this ticket to a task, if you'd like to use it for ongoing tracking of OS/browser/screen reader behaviour.

#17 @afercia
7 weeks ago

In 45726:

Accessibility: Improve the way emojis are announced by screen readers.

Adds an ARIA role="img" to the emojis markup in the front end. While the ARIA role is technically redundant, it helps Safari and VoiceOver to properly announce the emojis as "image". Without the role, they announce a generic "group" role.
Tested with Safari 12.1.1 / VoiceOver 9.

Props pento.
See #37486.

This ticket was mentioned in Slack in #accessibility by afercia. View the logs.


6 days ago

Note: See TracTickets for help on using tickets.