Twemoji: Explore alternatives

[kraftbj]

With the recent changes at Twitter, there is an uncertain path forward for Twemoji, the script and images we use for Core's emoji replacement support.

With this uncertainty in the air, WordPress should consider alternatives to this script and image library.

​https://github.com/twitter/twemoji/issues/570

[desrosj]

Thanks for opening this, @kraftbj!

I've been thinking about this a bit, but I don't really have a great recommendation yet.

It does look like there have been a few PRs merged this month, and a few of the contributors that were previously with the company are the ones interacting/merging these PRs.

[JeffPaul]

In a brief search I found ​noto-emoji though it’s licensed Apache 2.0/SIL Open Font 1.1 whereas Twemoji is MIT/CC-BY 4.0. Probably worth sitting on for now, but as we get deeper into WP 6.2 it would have been otherwise nice to have updated to ​Unicode/​Emoji 15.

[kraftbj]

I agree, it would be ideal to have Emoji 15 support in for 6.2. I don't want to rush to a decision if 1. Twitter starts releasing updates again or 2. Some other path for Twemoji to live on happens.

I don't think we should wait too long, though, so WP 6.3 should have Emoji 15 support somehow, one way or another.

If it is just an Twemoji update, I would support it in 6.2.x if the powers that be are okay with that. If it is moving to a different library, then 6.3 no doubt.

[azaozz]

In a brief search I found noto-emoji though it’s licensed Apache 2.0/SIL Open Font 1.1

Seems there are Noto Emoji (​https://github.com/fontsource/fontsource/tree/main/fonts/google/noto-emoji) and Noto Color Emoji (​https://github.com/fontsource/fontsource/tree/main/fonts/google/noto-color-emoji).

Afaik the fonts with SIL Open Font license can be used with GPLv2 code (they are not "merged" or part of the code, they are distributes alongside the code). Perhaps Noto Color Emoji would be a good alternative.

Also, related #52219.

[desrosj]

It looks like the previous maintainers (though no longer employed by Twitter) have ​created a fork with the full intention of supporting it going forwards.

From an issue asking for clarification on the purpose of the new repo:

" The original Twemoji was maintained by 2 Twitter employees, @sofodesign and me. With Elon Musk's acquisition of Twitter, we both no longer work there, nor do we have commit permissions to the original repo anymore. There's no indication at all from Twitter's side whether they intend to keep maintaining the original Twemoji; given Sofo and I did it in essentially our free time, we imagine it's highly unlikely and probably pretty low on the priority list of anyone there right now. We (and other past Twitter employees who worked on Twemoji and left well before the Musk acquisition) don't want Twemoji to go to the big Internet graveyard in the sky, so we've forked the original Twemoji here and plan to continue working on it (releasing new Unicode-approved emoji, maintenance, etc) despite no longer being employed by Twitter. I hope that clarifies things; if not, feel free to reply with any further questions."

There's also an ​issue dedicated to adding support for Unicode 15. The issue does say the hope is to have something in March (which has already passed), but it's made clear that the maintainers require some support in order to dedicate more time to maintaining the project.

I think this is likely our best option going forward.

[JeffPaul]

@desrosj I've been tracking this as well and concur with your assessment, though I would propose that we wait and see that the jdecked/twemoji fork actually gets a new Unicode 15 release out at some point before working on any changes here in core. I'm subscribed and following that fork and while they have some minor releases, I would prefer to wait and see that they get to v15 support before considering a switch.

[desrosj]

@JeffPaul Sure, I wasn't clear above. But that is also my suggestion. There is nothing worth including currently, but it is a compelling potential library to adopt in the future when there are significant enough modifications.

[kraftbj]

I concur with the above. Let's keep an eye on it and if it sticks, let's use it.

We can also consider and be aware of other options out there, but all things being equal, unless something else truly brings something different to the table, let's use the original maintainer's version.

[Hareesh Pillai]

I came here to reply about this fork, and it seems @desrosj beat me to it. :)

It looks like the previous maintainers (though no longer employed by Twitter) have ​created a fork with the full intention of supporting it going forwards.

This fork seems like a viable alternative as the license is kept intact.

I would like to add that the new fork has a release that contains the changes (made at the old Twemoji repository) to replace MaxCDN with jsDelivr. This was done as MaxCDN was retired by the company that acquired it.

PS: Should we update this ticket's component to External Libraries?

[desrosj]

I think leaving this in the Emoji component is fine. Historically, even though Twemoji is an external library, we've grouped related tickets into the Emoji component instead.

[desrosj]

Looks like the fork mentioned above has ​released version 15.0.2 with support for Unicode 15. 🎉

I have not yet confirmed license compatibility or whether we can continue to use npm to install. But an encouraging, unexpected development!

[kraftbj]

This is very encouraging. I'll put this on my list to test out. The license appears to be MIT and CC Attribution.

This switches the source for the Twemoji library from twitter/twemoji to jdecked/twemoji while also applying the 15.0.3 update.

Trac ticket: https://core.trac.wordpress.org/ticket/57600

This looks good to me, the test can probably be a blackbird as it uses a ZWJ ​https://emojipedia.org/black-bird

[desrosj]

@dd32 would you be able to get ​https://github.com/jdecked/twemoji/releases/tag/v15.0.3 committed to the .org CDN?

[dd32]

Replying to desrosj:

dd32 would you be able to get ​https://github.com/jdecked/twemoji/releases/tag/v15.0.3 committed to the .org CDN?

Done! Test image:

@desrosj In 63c1fa5b7eebbef0191394c0c22e1890d3bed4df I've pushed the update to the JavaScript tests.

On a system that doesn't support the new emoji, the canvas renders identically

Using the emoji in a post renders on a system without support renders the emoji correctly

On a system that does support the new emoji, the canvas renders differently

[peterwilsoncc]

So it's documented somewhere, this is the utility I used to get the correct codes for the new JavaScript test ​https://it-tools.tech/emoji-picker

Thanks @peterwilsoncc! I started to do some precommit testing on this earlier today before I helped get the update out, and I noticed that the grunt precommit:emoji script is broken. I tried several different forms of the URL, including changing it to a version specific one, but couldn't get it to work. I'll check in some more my tomorrow, but if you care to have a look during your day, feel free.

@desrosj Looks to be due to GitHub ​sunsetting SVN support. The grunt file is trying to use svn to get a list of the files without doing a clone.

​https://github.com/WordPress/wordpress-develop/blob/cf8b74de164f322a78ea51921b236213805e6f79/Gruntfile.js#L1056

In ​https://github.com/WordPress/wordpress-develop/pull/5804/commits/21cfb61661e405f8a422a5172937c520dfec17a0 I've pushed a fix to the precommit script that uses the GitHub CLI instead of SVN.

It appears to be working but I haven't pushed the changes to the formatting file as it would be good to get a logic check on that from you.

Precommit broken due to the number of files exceeding 1000.

I've tried using the graphql API and it works locally but the grunt.spawn command I've written is broken but should show what I am trying to do.

[desrosj]

In 57626:

Emoji: Replace twitter/twemoji with jdecked/twemoji.

After a chaotic change of ownership, the twitter/twemoji library is now considered abandoned.

After waiting for this moment to arise, a fork was created by several former employees who had maintained the library which lives at jdecked/twemoji on GitHub.

This switches out where the underlying source code comes from for the library, and applies the 15.0.3 update, which adheres to the Unicode 15 spec and adds support for all Emoji introduced in Emoji 15.0.

This does not update the underlying precommit:emoji Grunt script responsible for updating formatting.php. After GitHub recently sunset support for SVN, the current process needs to be replaced with a new one. This will be handled in #60520.

Let the masses rejoice for the 🐦‍⬛ singing in the dead of night, secure 🛜, aromatic 🫚, and some very silly 🪿🪿.

RIP Twemoji. Long live Twemoji!

Props peterwilsoncc, kraftbj, jeffpaul, azaozz, dd32, hareesh-pillai, jorbin.
Fixes #57600.

Committed in Core-57626. I've moved all build related changes to #6107 for further exploration.

[desrosj]

In 57758:

Build/Test Tools: Fix the precommit:emoji script.

GitHub recently sunset support for Subversion, causing the precommit:emoji Grunt script to break. Since there’s no direct replacement for svn ls in Git, this has been replaced with a query through the GitHub CLI.

This also adds a step in the workflow that tests the build process to run the precommit:emoji script to ensure no changes to built files are missed when updating the Twemoji library in the future.

Follow up to [57626].

Props kraftbj, peterwilsoncc, swissspidy.
Fixes #60520. See #57600.