Global Emoji Support

[pento]

Once #21212 is complete, then WordPress will support storing emoji in the database.

The problem is, some browsers still don't support displaying emoji. We need to work around their lack of useful features.

Here's a plugin to get things rolling:

​https://github.com/pento/x1f4a9

[pento]

Note to self, for when it comes time to commit this: @batmoo, @joen and @mkaz are owed props for their work on WP.com's emoji support, upon which x1f4a9 is based.

[danielbachhuber]

We should also update the language of the setting. Emojis are different than emoticons.

Because emojis won't be properly displayed outside of the website context (e.g. search results, RSS feeds, email clients, etc.), should we have some compatibility shim to translate the data on the fly?

​https://dl.dropboxusercontent.com/s/0v81fuykb81q6oi/2015-02-06%20at%208.53%20AM%202x.png?dl=0

[pento]

Replying to danielbachhuber:

Because emojis won't be properly displayed outside of the website context (e.g. search results, RSS feeds, email clients, etc.), should we have some compatibility shim to translate the data on the fly?

We can definitely do something for RSS, and maybe for email. Not sure about search results, though. I don't know of any way to tell Google (or any search engine) to use an image in search results.

[pento]

x1f4a9 is now in the ​repo, and appears in the Beta Testing tab.

[afercia]

As per Slack conversation, uploading a screenshot: alt attributes need some love. See in the dev tools, Firefox and Chrome on Windows. Ideally, the alt attributes should be the emoji name, if not possible, better to have an empty alt attribute in lieu of a character that wouldn't be read out by screen readers or read out in a very weird way.

[hnle]

Replying to afercia:

Ideally, the alt attributes should be the emoji name, if not possible, better to have an empty alt attribute in lieu of a character that wouldn't be read out by screen readers or read out in a very weird way.

We often drag-select and Copy text&paste.
That time, images will copy as alt-text..

[kraftbj]

Replying to afercia:

As per Slack conversation, uploading a screenshot: alt attributes need some love. See in the dev tools, Firefox and Chrome on Windows. Ideally, the alt attributes should be the emoji name, if not possible, better to have an empty alt attribute in lieu of a character that wouldn't be read out by screen readers or read out in a very weird way.

See ​https://github.com/pento/x1f4a9/issues/6 . I reported it upstream to Twitter (as the issue is in the code we're taking from them) and they wontfix'd it.

[kraftbj]

Other issues opened/for discussion:
Emoji not rendered in form fields: ​https://github.com/pento/x1f4a9/issues/7
Emoji not rendered after ajax actions: ​https://github.com/pento/x1f4a9/issues/8
A couple of issues with emoji being dropped completely:
text widgets: ​https://github.com/pento/x1f4a9/issues/9
quick edit: ​https://github.com/pento/x1f4a9/issues/10
Emoji in terms will display blank slug: ​https://github.com/pento/x1f4a9/issues/11
Cursor position after TinyMCE emoji render: ​https://github.com/pento/x1f4a9/issues/12

[pento]

Well, 31242.diff​ sure is a thing.

Currently in need of a JS review, and unit tests.

[afercia]

Just noticed something bit weird in IE 8, see screenshot:

See the space below the admin bar. That space disappears as soon as I disable the Emoji plugin. Reproduced in trunk and 4.1 with Twenty Fifteen and just Twitters Emoji plugin active.
@azaozz pointed out that span is the standard MCE bookmark to keep the caret location in IE, I suspect it shouldn't be appended there.

[pento]

31242.2.diff​ fixes up all the smiley unit tests.

I need some sleep, I'll commit this in the morning, along with #31328.

[pento]

In 31733:

Add emoji support, with Twemoji fallback.

Replace exisiting smilies with equivalent emoji, or with shiny new smiley images where no emoji existed.

Props batmoo, joen and mkaz for the original plugin upon which this is based.

Props pento, iseulde, kraftbj and peterwilsoncc for making the internet's dreams come true.

See #31242

[nacin]

In 31744:

Emoji JS cleanup.

• Filename: emoji.js => wp-emoji.js
• Script handle: emoji => wp-emoji
• Object: WPEmoji => wp.emoji
• Script settings: EmojiSettings => _wpemojiSettings
• Setting key: base_url => baseUrl
• Remove executable bit from files

see #31242.

[pento]

In 31745:

Add new smiley images missed in [31733].

See #31242

[nacin]

In 31746:

The partial change to the script handle in [31744] was not actually intended.

see #31242.

[iseulde]

MutationObserver! :)

[iseulde]

This will take care of infinite scroll, theme modal, media modal etc.

[iseulde]

We also don't need to parse any more in other files.

[pento]

In 31750:

Emoji: Instead of having custom hooks for Ajax callbacks, use MutationObserver to re-parse any changed elements in the DOM.

Props iseulde

See #31242

[pento]

In 31752:

Emoji: There's a little tear in my eye as I remove DOMDocument from the Emoji staticizer. It was a beautiful dream, but it wasn't to be.

Instead, let's use the tried and trusted smiley replacement algorithm, which has stood the test of time.

See #31242

[adamsilverstein]

I was experiencing an Unresponsive script warning for twemoji until the last patches; testing with 31242.7.patch​ fixed the issue for me.

[iseulde]

Returning false in the callback for the image src is a bad idea. We'd need to adjust this in twemoji.

[johnbillion]

Replying to adamsilverstein:

I was experiencing an Unresponsive script warning for twemoji until the last patches

For the record, I was seeing the same thing too.

[iseulde]

This is what's causing the freezing. Twemoji fails to replace with an image, text node is replaced anyway (it found a match), observer detects change, twemoji parses again... infinite loop.

[iseulde]

Temporary fix for now.

[iseulde]

I sent a PR: ​https://github.com/twitter/twemoji/pull/52

[iseulde]

Discussed this with @azaozz and we'll patch twemoji in core. All twemoji tests pass. See PR.

[azaozz]

In 31242.12.patch: also some old IE fixes.

[azaozz]

In 31756:

Emoji:

• Patch twemoji.js to prevent infinite loop.
• Fix some undefined errors in old IE.

Props iseulde. See #31242.

[ocean90]

In 31758:

Update Twemoji to 1.3.0.

Includes a fix for Firefox: ​https://github.com/twitter/twemoji/commit/92d5bea1abf0183fd00d306511c541b55b414959
Since this fix breaks IE 11 we added modified = false; after subnode.parentNode.replaceChild(fragment, subnode).

props iseulde.
see #31242.

[ocean90]

In 31759:

grunt imageminfor [31745].

see #31242.

[azaozz]

In 31761:

TinyMCE: fix emoji parsing in IE. See #31242.

[azaozz]

In 31762:

TinyMCE: fix emoji parsing in IE typo. See #31242.

[iseulde]

Update twemoji to 1.3.1, but still exclude select. Actually, about that: ​http://www.w3.org/TR/html-markup/option.html.
option can only contain a text node.
This is actually a weird territory. Emoji work in Chrome in select tags because it is native UI, even though Chrome does not support emoji. :)

[iseulde]

Another TinyMCE bug: linking some text with an emoji is not possible. The emoji just disappears.

[boonebgorges]

[31733] broke Tests_Post::test_utf8mb3_post_saves_with_emoji() for some setups. Here's my local environment:

$ mysql --version
mysql  Ver 15.1 Distrib 10.0.17-MariaDB, for Linux (x86_64) using readline 5.1

At least in this specific case - where mariadb versioning differs from mysql - the problem traces in part to https://core.trac.wordpress.org/browser/trunk/src/wp-includes/wp-db.php?marks=2804#L2780. $wpdb->has_cap( 'utf8mb4' ) is failing (even though my DB does in fact support utf8mb4), and this causes the test not to be skipped as it should (for reasons I have not dug into, but pento probably knows off the top of his head). I'm not sure why [31733] in particular broke this - I think it's the 'utf8' charset check in wp_insert_post()?

In any case, it's happening on Travis as well: ​https://travis-ci.org/aaronjorbin/develop.wordpress/jobs/54178126

[azaozz]

In 31766:

Emoji: update Twemoji to 1.3.1. Props iseulde. See #31242.

[SergeyBiryukov]

31242.16.patch​ adds display: inline to Emoji styles, to avoid them being displayed as a block element (see the screenshot from the recent ​WordPress Core Weekly post).

[SergeyBiryukov]

In 31771:

Emoji and smiley images should be displayed inline.

see #31242.

[iseulde]

test is much faster than match.

[azaozz]

In 31772:

Emoji: better regex when testing for only char emoji. Props iseulde. See #31242.

[azaozz]

In 31773:

Emoji: yet another update for Twemoji, to 1.3.2. Props iseulde. See #31242.

[azaozz]

In 31242.19.patch:

• Move the TinyMCE plugin css to wp-content.css.
• Clean up both the plugin and wp-emoji.js, abstract and restructure a bit.

[azaozz]

In 31779:

Emoji:

• Move the TinyMCE plugin CSS to wp-content.css.
• Change the replacement images class to wp-emoji inside the editor.
• Clean up both the plugin and wp-emoji.js, abstract and restructure a bit.

See #31242.

[azaozz]

In 31780:

Emoji: always export the methods in wp-emoji.js, even when dependencies are missing and we cannot initialize.
See #31242.

[azaozz]

We probably should change the class for the replacement images from emoji to wp-emoji. It is more compatible and future-proof. Any reasons not to?

[pento]

No reason, that was on my to-do list for next week - feel free to take care of it while you're powering through emoji, though. :-)

[pento]

In 31781:

Smilies: The new smilies added in [31733] and [31745] are larger than the old smilies. While this is taken care of by the CSS on normal pages, it means they're disproportionally large when seen in RSS and email.

By adding a little bit of inline style to them, we get pleasingly sized smilies everywhere. :-)

See #31242

[azaozz]

Having some second thoughts about changing the replacement images class name. Not sure how many places use Twemoji but it seems the .emoji class name may become the "de-facto standard" for these images.

One user case for keeping that unchanged is when pasting in the editor. If there are image tags with .emoji, we can easily find them and convert them (add the other attributes). Of course we can check for .wp-emoji too, but using .emoji will probably help make it into something like a de-facto standard.

[azaozz]

In 31786:

Emoji:

• Add the styling for the replacement images to the admin CSS.
• Revert to using .emoji as replacement image class.
• When pasting in the editor, convert emoji images to our format so we can replace them with chars on saving.
• Some more clean up of both the plugin and wp-emoji.js.

See #31242.

[azaozz]

In 31787:

Emoji: fix few typos. See #31242.

[pento]

In 31788:

Emoji: Move a comment to match the restructure in [31779], and add an extra comment explaining how emoji rendering support is detected.

See #31242

[DrewAPicture]

In 31789:

Clean up some inline documentation for emoji functionality, including a missing @since for mail_emoji(), and a changelog entry for wp_insert_post().

See #31242.

[DrewAPicture]

@pento: Can we come up with more descriptive names for feed_emoji() and mail_emoji()? I don't feel like they really fit with core style in that it's not immediately obvious what purpose they serve. Perhspa something like convert_feed_emoji() and convert_mail_emoji()?

[SergeyBiryukov]

In 31790:

Fix typo in a comment in [31752].

see #31242.

[pento]

Replying to DrewAPicture:

@pento: Can we come up with more descriptive names for feed_emoji() and mail_emoji()? I don't feel like they really fit with core style in that it's not immediately obvious what purpose they serve. Perhspa something like convert_feed_emoji() and convert_mail_emoji()?

For you, @DrewAPicture, we can absolutely do that.

If we want to chat about purpose before deciding on a name, it's worth considering that they're really just single-purpose filters (to be thin wrappers that filter content through wp_staticize_emoji()), so maybe they're better as _-prefixed @ignore-d functions?

Perhaps _wp_staticize_emoji_for_email() and _wp_staticize_emoji_for_feeds()?

[DrewAPicture]

Replying to pento:

If we want to chat about purpose before deciding on a name, it's worth considering that they're really just single-purpose filters (to be thin wrappers that filter content through wp_staticize_emoji()), so maybe they're better as _-prefixed @ignore-d functions?

Yes. Yes.

Perhaps _wp_staticize_emoji_for_email() and _wp_staticize_emoji_for_feeds()?

Yes.

[pento]

In 31791:

Emoji: Rename the email and feed filter functions to be _ prefixed, and @ignore-d in the PHPDocs.

See #31242

[pento]

*ksssshhhhtttt* This is your Emoji Captain speaking, it's been a pleasure flying you to Emoji Island, all of us here at x1f4a9 Airlines hope your stay is a pleasant one.

If you have any feedback for us, please open a new ticket at your leisure.

[nacin]

In 32161:

Clean up wp_staticize_emoji() and friends.

• DOMDocument was removed in [31752] but not the check.
• wp_staticize_emoji() has never accepted a second arg; remove it from calls.
• Remove wp_staticize_emoji_for_feeds(), no need for it.
• Remove _ and @ignore from wp_staticize_emoji_for_email(), no need for it.

see #31242.