twemoji is always loaded on Windows devices

[Clorith]

The twemoji script seems to always be loaded and triggering, despite the requested page containing natively supported emoji.

Only the front-end seems to enqueue the twemoji script, the back-end editor still uses renders native emoji as expected.

By blocking the wp-emoji-release.min.js file, emoji are rendered as expected in the front-end.

Initially marking this as a minor issue, since technically we are loading an additional file that modifies the rendered content, but it also sort of ensures a consistent emoji experience regardless of devices for visitors which one could argue is a feature (😅?)

Versions:

• Windows 10
• Browsers:
  • Chrome 127.0.6533.72
  • Firefox 128.0.3

[dmsnell]

@Clorith do you see any browser console errors? the Twemoji detection should be checking if Emoji render a specific way and then determining whether to load the polyfill. It's always 100% client-side, and meant to detect very specific Emoji that come in through different updates to the Unicode standard.

For example, if the script detects that your operating system doesn't yet have support for Unicode 15.1.0 (released in September, 2023), then it will load the script even if the page doesn't have Emoji that were added in that release.

So I could imagine something is causing the script to fail, or the computer might not actually support the full Emoji set. Given that most Unicode releases introduce only a handful of Emoji, it would be generally unlikely that a page display wrong without the polyfill. On the other hand, a blog post with a shaking head (🙂‍↔️), a brown mushroom (🍄‍🟫), or a broken chain (⛓‍💥 - on my computer this displays as two emoji: a chain followed by an explosion) may not display correct without the polyfill.

[dmsnell]

cc: @westonruter in case you missed this

[Clorith]

So there are no JavaScript errors, it can be replicated on a svanilla Playground instance, so I'm confident it's not an issue with any plugins or theme.

Again though, the back-end seems to do things correctly, as it loads the system emoji within the block editor, its just the front-end render that pulls in the twemoji solution.

So either we're not enqueuing that in the back-end, which may or may not be a degraded experience for editors, or it's incorrectly triggering on the front-end, and thus causing additional requests to load up unnecessary JavaScript handling 🤔

As an example, I used the "👩🏻‍🦰" emoji in my test post (fair skinned, red haired woman), which I know is technically 2 (possibly 3?) combined, but renders just fine with system emoji, but still loads up. To be on the safe side and check if it's just weirdness, I did also load up the most basic emoji there is, the smileing face; "😀", which again renders using the system version in wp-admin, but front-end as a site visitor twemoji kicks in again.

[dmsnell]

@Clorith the Twemoji detection script runs on every page load in the front end, and needs to because the server is unable to know if a given combination of browser and operating system can display the Emoji properly. it does try to store some checks in local storage to avoid recomputing the detection when possible.

it's also not a property of the Emoji that are present in the post. the server could potentially avoid sending the script if there are no Emoji in the post, but then it would also need to ensure that new content can't come in with emoji after page load (for example, as might happen in the query block with enhanced pagination activated).

To be clear, are you seeing the script behave differently in Windows than on other platforms? Are you seeing it replace Emoji when you think it shouldn't, or are you merely noticing that it executes its detection code?

[Clorith]

I would expect the editor and front-end to behave and display the same emoji as a bare minimum (why else promote it as a WYSIWYG editor), which it currently does not. In addition, since the native vss twemoji versions are visually different to a degree (both in size representation, and some times things like accessibility related. Windows ones, for example, have a strong outline, where the twemoji ones have no outline and "meld" with the page at times).

I don't have any similar platform to compare "apples to apples", as it were (apple product pun intended), but on Chrome for Android, the same emoji is displayed in both use cases (appears to be the system emoji), so there's some form of discrepancy.

[dmsnell]

@Clorith just a note that the Twemoji script is there only for cases where it thinks that the combination of the browser and OS cannot properly display the Emoji that might appear on a page. In normative situations it shouldn't be replacing Emoji, but it will always^† run to determine if it should replace them.

I would expect the editor and front-end to behave and display the same emoji as a bare minimum

this is one place where it's not possible, unfortunately, because the text rendering support on the computer doing the editing may not match the text rendering support of the computer doing the viewing of the page.

since the native vss twemoji versions are visually different to a degree

this is another point to note, as Unicode does not standardize the appearance of the Emoji, but rather the semantics of their usage. notably, Android devices have long rendered Emoji differently, as does I think Facebook, who runs their own replacement code.

---

I think there are perhaps multiple points of discussion muddled in here. The goal of the script is to ensure that people whose systems cannot display the Emoji see a valid representation of the Emoji rather than � or a combination of characters like 🏴󠀇󠀆󠀇󠀆 (the rest are invisible) instead of 🏴󠁧󠁢󠁥󠁮󠁧󠁿, which is what they would see if the rendering support is missing.

It sounds like you are reporting that the script is erroneously detecting that it needs to replace the Emoji on the page when it doesn't need to. In that case, not only has the script run, but it has replaced the Emoji in the text nodes with an image tag <img> containing the Emoji-replacement glyph.

There are definite ways to improve the script, but its purpose isn't particularly controversial. Its mode of operation is to render text with some test Emoji and then detect graphical properties about the rendered snippet. If indeed it's always replacing Emoji on Windows, there may be a glitch we need to examine.

Test
What does 🐦‍⬛ look like on your system in the browser you're reporting from? Does it look like a blackbird or does it look like 🐦​⬛ a bird and then a black square? Can you also verify that these are being rendered as text and not replaced by anything else?

†: it tries to cache the computation so that it doesn't have to run the detection on every page load.

[Clorith]

The three emojis at the end of your message are indeed showing as a blackbird, then a red bird with a black square following it, but they're also all replaced by the img tags.

You are correct that it seems like it is incorrectly detecting that emojis can not be displayed, and that there are two issues reported here;

Incorrect detection of valid emoji support, leading to the triggering of the image conversion (which then loads assets from the WordPress CDN, s.w.org), adding additional networ ktraffic when it isn't actually needed.

And that the front-end and back-end of WordPress is not both enqueuing (and thus ensuring support for) emojis on the same basis, as the editor screen seems to intentionally dequeue it? This may very well be a twemoji issue in terms of Windows support though, but for the 2nd issue, it would ideally have the same emoji and visuals in the back-end and front-end. As you noted, perhaps this is relating to rendering vs editorial mode differences, although I was not aware that was a thing, but if it is a thing, then surely when a block is not in edit mode, it would be in display mode, and render the same as the front-end viewing? (this may be a deeper question for the block editor maintainers I guess?)

In an ideal scenario, the twemoji script would look up any emoji used on a page, and only replace the ones not supported (this may need reporting upstream to see if it's even viable?) So let's focus on that first and foremost, and then the editor side of it can be a separate ticket, to not muddy the waters too much :)

[dmsnell]

The three emojis at the end of your message are indeed showing as a blackbird, then a red bird with a black square following it, but they're also all replaced by the img tags.

Case in point 😄

No red bird over here. To be clear, though, @Clorith - are you seeing image replacement here on Trac? If you are, then maybe there's something in your browser? Because Trac involves no such emoji replacement. Maybe you copied and pasted these into a WordPress document? I was asking specifically about how the Emoji render in Trac, where no such script runs.

it would ideally have the same emoji and visuals in the back-end and front-end

This is a good point, though also complicated. I'm not sure of what impact or technical challenges there would be for replacing text in the editor with images. Presumably, an author would not be writing unsupported emoji on their computer.

surely when a block is not in edit mode, it would be in display mode

It's not so much edit vs display mode here. The Twemoji script is only intending on handling cases where a browser isn't able to render the Emoji. Normally it won't kick in for modern computers with modern software. Something on your system is telling it that it's failing to render.

In an ideal scenario, the twemoji script would look up any emoji used on a page, and only replace the ones not supported

Yeah this has been brought up, though each additional bit of complexity increases the burden on every page load. I think the compromise today (and this is something more that @westonruter can probably confirm) is that if a browser is lacking any of the given support, it should load Twemoji, otherwise don't burden the vast majority of page views by loading things that might not be necessary.

I'm not familiar with Twemoji or if it's possible to split it into different levels of support. That'd be a good avenue to explore, as it's quite common for browsers/OSs to lag support for the newest Unicode standard, when that might only involve a handful of new Emoji which are unlikely to be on the page itself.

Still, processing a page to find unsupported Emoji might be quite costly, and in some cases not possible. The emoji-loader script relies on some canary tests that only work because they are multi-character sequences, and those sequences don't combine when unsupported. Single-character Emoji may not provide any reasonable way to know "is this rendered properly?"

@westonruter - did you explore comparing the render of an Emoji against the replacement character �? I know that some systems don't use that, instead rendering the box with the code point inside it. For example:

---

My guess is that there's something affecting text rendering in your browser and it's causing the script to misdetect. Something that causes one string to have subtle differences: like even a pixel or sub-pixel difference between the real grapheme cluster and the detection cluster.

What do you see if you navigate to this test page I made? It runs the same tests that the emoji loader is running, but instead of making a decision, it visually displays the comparisons.

​https://s.xkq.io/emoji-detection.html

Here's what it looks like on my computer. It's not the most elegant tests, but if the "Drawn Text" matches the "Drawn canary" then it determines that the Emoji isn't supported.

[siliconforks]

Replying to dmsnell:

Because Trac involves no such emoji replacement. Maybe you copied and pasted these into a WordPress document? I was asking specifically about how the Emoji render in Trac, where no such script runs.

Trac does in fact load the wp-emoji-release.min.js script. I can see it in the HTML source code:

$ curl -s https://core.trac.wordpress.org/ | grep wp-emoji-release
window._wpemojiSettings = {"baseUrl":"https:\/\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/","ext":".png","svgUrl":"https:\/\/s.w.org\/images\/core\/emoji\/15.0.3\/svg\/","svgExt":".svg","source":{"concatemoji":"https:\/\/s.w.org\/wp-includes\/js\/wp-emoji-release.min.js?ver=6.6-alpha-58184-b976b651932bfd25b9ddb5b7693d88a7"}};

[dmsnell]

Wow! Thank you @siliconforks - I didn't realize this, but now that makes more sense.
The version I linked at ​https://s.xkq.io/emoji-detection.html definitely has no replacement because I wrote it all myself and it's just a single HTML page.

[pbearne]

I have been looking at this
It seems the flags test is failing

I am struggling to debug anybody upto working session to debug?

[siliconforks]

I think the problem is that Windows apparently does not support flags at all (with a small number of exceptions). See the discussion here under "Windows woes":

​https://nolanlawson.com/2022/04/08/the-struggle-of-using-native-emoji-on-the-web/

So it seems the twemoji script will always be loaded (by design) on Windows.

There is an additional complication: Firefox includes its own set of flags to work around this. So in Firefox the twemoji script may or may not get loaded, depending on whether other tests fail.