Convert heic to a web safe image format.

[spacedmonkey]

​As of ios 11, Apple start using the HEIF format to save images. Sadly HEIF files can not be viewed in web browsers. For those taking images on their iPhone, this would mean to embed a raw iPhone photo, would require them to use a third party tool to convert that image into a web safe format like jpeg.

WordPress core should convert this image to a web safe format while keeping the original for reference.

[spacedmonkey]

Related: #50557 #42775

[adamsilverstein]

Great suggestion @spacedmonkey!

Questions: What happens when users upload an HEIF image currently? Are there other formats users might try to upload we should think about when building this? Do both GD and Imagick support HEIF?

[spacedmonkey]

Seems like Imagick ​does support handling HEIF but PHP GD ​does not. There seems to be a PHP library that may help here called ​Intervention Image. There is a javascript library called ​Heic2any.

I am not sure if this would have to be an opt in feature or not. Would it be okay if the feature worked only if you use imagemagick.

[adamsilverstein]

Looks like support was added to LibGD in v2.3 - ​https://github.com/libgd/libgd/issues/395.

Support will hopefully make it into the next PHP release via ​https://github.com/php/php-src/pull/5127 although that may have shifted to AVIF focus.

[spacedmonkey]

@adamsilverstein Good find. Seems like it there is support was added in ​2.3.2.

Meaning both image libraries do support it. Do we need to wait a while for this library versions to be more popular?

Can we add this as a projective enhancement like webp?

[adamsilverstein]

we should be able to treat this as a progressive enhancement. If the server supports HEIC, add the feature. We can start with a POC, I'd like to see if my local setup supports this.

[desrosj]

I have to admit that I am pretty hesitant to start converting uploaded images into different formats just because they are not web friendly.

I know that the ability to map mime types is new, but we haven't done this in the past with other formats. It's also perfectly reasonable to expect someone to want the original format of the image to persist.

On one hand, it would be great to upload any image they find and not have to worry about it being usable/visible to their users, but on the other hand there is value in the user being aware of the fact that the image they're uploading is not a widely supported format.

Also, where do we draw the line? TIFF is not supported by Firefox, JPEG 2000 is only supported by Safari. If we start building a list of mime types that we map, there's the potential to create a web that will be a maintenance headache.

To be clear, I think this is a great idea, but I am hesitant to add this to Core as the default without considering the implications of this more broadly.

[spacedmonkey]

I know that @swissspidy has been working on this as part of ​media-experiments. I wonder if he is interested in picking this up.

[swissspidy]

Indeed I have.

There are multiple ways to go about this, for example it can be opt-in, opt-out, or opt-in with a confirmation dialog the first time you upload such an image. So there are some UX considerations.

Right now in my project it's on by default (opt-out) but am ​considering changing it to be more flexible.

Either way, this project still a long way until this is ready for prime time, and that will have to go through Gutenberg anyway.

[markhowellsmead]

I like the idea of showing a message or making a choice when uploading an image file of a particular format for the first time, when the format is not usually used for inline display in the browser.

[noisysocks]

Thinking about a prompt some more, I believe the vast majority of users will not care what a HEIC file is ("I just want my image to appear!") and so we ought to make a JPEG or WEBP copy by default but allow this to be turned off if the user has atypical needs e.g. building a website exclusively for Safari users.

If the server doesn't support HEIC (e.g. if Imagick is disabled or outdated), then we can upload the HEIC and educate the user that it likely won't work well and they should probably convert the image manually. WordPress already does this, but the UX could be improved.

Support for other non web safe formats (TIFF, JPEG2000, etc.) is interesting to think about but, practically speaking, it's HEIC that is prolific and causing endless frustration, so let's focus on just that.

I think better to explore a server-side Imagick/GD solution first since ​there's complexity and licensing issues with doing the conversion client side.

I'm keen to pick this up for WP 6.7 unless someone wants it more.

[swissspidy]

Regardless of any work on client-side conversion in GB, adding such support on the server makes total sense. We should really do both. Just beware that server support will be quite limited (probably even more than AVIF support for example).

While I try to push forward the client-side work, I'd be happy to support any server-side efforts.

​Performance Lab's Modern Image Formats plugin should be a great starting point for this, as it already does server-side format conversion during upload. The only difference is that it also keeps the original file, but I'm not sure we'd want/need that for HEIC.

[markhowellsmead]

​Performance Lab's Modern Image Formats plugin should be a great starting point for this, as it already does server-side format conversion during upload. The only difference is that it also keeps the original file, but I'm not sure we'd want/need that for HEIC.

I would recommend that all image formats are handled in the same way, otherwise it would get complicated trying to work out if and why some original files are deleted, when others aren't. Some users may want to offer full-resolution downloads of their original images, and (in this example) link to the HEIC file.

[noisysocks]

I'd appreciate a designer's opinion on some of the UX details we're discussing.

[Joen]

Hey, hopefully I can provide some design feedback. What UI pieces are we considering? It seems the gist is: "I want my images to work", and the questions related to that is how WordPress makes that happen. Is it:

• A modal that appears on upload: "Want to convert this to JPG?"
• A notice that appears after upload: "We converted this to JPG."
• Just doing it, without asking.
• Just doing it, provided you've not opted-out through a Settings > Media checkbox.
• Only doing it if you've opted-in through as Settings > Media checkbox.
• Converting automatically, but not doing anything at all. It just becomes a JPG.

In most cases, images are already run through a parser to create thumbnail sizes in jpeg format. Is there anything we can do there, so if users insert a HEIC image and chooses "Full size", it's essentially converted to JPG for display at that time as well?

Tricky questions, but perhaps one path forward is:

• Allow HEIC uploads, but don't store them. Convert them to JPGs and store the JPG.
• Show a notice upon successful upload, that it was converted to JPG because HIEC only works in Apple browsers.

Let me know if that was useful.

[swissspidy]

Aren't these two the same thing?

• Just doing it, without asking.
• Converting automatically, but not doing anything at all. It just becomes a JPG.

Interesting suggestion with showing something after upload.

Originally I thought about a one-time modal before upload, and then storing the preference for later. That works well in the editor (where we have a preferences view and can easily display such notices), but not in the old media library.

Also, I feel like a (before-upload) notice works best when we can do the conversion in the browser and not on the server.

So for a server-side solution like here, my vote actually goes to "just do it, don't tell" - do not show any notice. There could be PHP filters for changing that behavior, as I don't think this warrants a checkbox on the settings page though (decisions, not options).

Is there anything we can do there, so if users insert a HEIC image and chooses "Full size", it's essentially converted to JPG for display at that time as well?

Well it would happen even before you choose a size. You drop the file into the browser and on the server we would immediately convert.

[markhowellsmead]

If there is a site which wants to offer HEIC downloads, then the decision to automatically convert the files to JPG and delete the originals would stop that from being possible.

HIEC only works in Apple browsers

…which is currently 20% of the internet, so that shouldn't be a dismissive statement.

There could be PHP filters for changing that behavior,

That's a reasonable suggestion, but one which would mean that a site owner would need to add a plugin—or have one developed—to allow HEIC on their site, where all* other image formats are allowed.

(* I believe…)

[swissspidy]

I think it would be helpful to show the status quo first. So here's the current behavior when uploading HEIC images to WordPress:

In the editor:

The image is uploaded but cannot be displayed (unless you are using Safari). No sub-sizes are generated (because the server doesn't support it).

In the media library:

The image is uploaded but cannot be displayed (unless you are using Safari). No sub-sizes are generated (because the server doesn't support it). Additionally, an error message is displayed that reads as follows:

This image cannot be displayed in a web browser. For best results convert it to JPEG before uploading.

If there is a site which wants to offer HEIC downloads

That sounds like an edge case to me, not a majority use case. Plus, we can still store the original anyway so no data is lost.

[peterwilsoncc]

My inclination is to auto-convert if possible and reject the upload if it's not possible, per Pascal's comment to just do it, don't tell.

The clear intent of a user uploading an image is to display it on the web. If displaying the image fails 80% of the time, then the user's expectations are not being met. My view is that conversion meets ​WordPress philosophy to design for the majority.

I understand the argument about TIFF and other image formats but it seems like a red-herring as the dominant issue is the uploading of HEIC images.

[noisysocks]

I'm going to write a patch implementing the "just do it, don't tell" approach with a filter to disable the auto conversion.

At first I was keen to add a setting or filter that allowed users to keep the original HEIC but I've come around to the suggestion to make this entirely invisible, because this is what Apple themselves do. For example, drag a HEIC photo from Photos on macOS to your desktop and it will create a JPEG with no warning or messaging. Same story if you upload a photo to WordPress or any website in Safari on your iPhone.

Users who wish to keep the original HEIC file or covert to WebP instead of JPEG can install a plugin like ​HEIC Support which already works very well.

[spacedmonkey]

@noisysocks I would recommending looking into the ​modern-images-wp plugin by @adamsilverstein. This uses the image_editor_output_format filter added in [54416].

[adamsilverstein]

Looks like Imagick supports HEIC, as well as ​libGD (but not enabled by default in GD built into PHP 8.0+ as far as I can tell)...

we can add support for uploading/conversion/subsizing as long as the image format support support is present (ie. with Imagick). This would be very similar to adding AVIF support with the one addition of automatically converting the subsized images to JPEGs so they are actually usable on the web.

[adamsilverstein]

attached a couple of screenshots showing failed uploads in Chrome of heic images (MIT licensed test images curtesy of ​https://github.com/dsoprea/heic-exif-samples)

Related Gutenberg issue for preventing uploading of images that aren't supported: ​https://github.com/WordPress/gutenberg/issues/25130

[adamsilverstein]

Note: HEIC is supported in LibGD, but not in the PHP implementation, see ​https://bugs.php.net/bug.php?id=80828

First pass, excluding tests

Trac ticket: https://core.trac.wordpress.org/ticket/53645

[adamsilverstein]

Created an (untested) PR: ​https://github.com/WordPress/wordpress-develop/pull/7034

Automatically converts HEIC images to JPEG on upload. See ticket for details and motivation.

Implementation is based on the GPL2 ​[HEIC Support](wp_handle_upload_prefilter) plugin written by @csalzano. Attribution is included in the doc comment.

Unlike the HEIC Support plugin, HEIC images are _always_ converted to JPEG and the original file is deleted. This is what the majority of users will want and aligns with what Apple does in its apps that support HEIC. Website admins can customise the behaviour by installing a plugin (e.g. HEIC Support) or by removing the hook:

remove_filter( 'wp_handle_upload_prefilter', 'wp_auto_convert_heic_images' );

The ​[existing error message](​https://github.com/WordPress/wordpress-develop/blob/5b0006ca9b5408aa73b6dc71fb5b3ebf85f497ec/src/wp-includes/script-loader.php#L1006) that appears when you upload a HEIC image that encourages the user to convert the image and try again is suppressed when auto-conversion is enabled.

[noisysocks]

Ah, this is awkward 😅, @adamsilverstein beat me to opening a PR. I'll open one anyway containing the patch that I worked on yesterday: ​https://github.com/WordPress/wordpress-develop/pull/7035. It's a different approach, but perhaps there's value in comparing the two.

I'll review ​https://github.com/WordPress/wordpress-develop/pull/7034.

Thanks for the PR @adamsilverstein. It doesn't seem to work when I test it, though.

If I drop a .heic into the gallery I see an error, the image cannot be previewed, and the file type is still image/heic.

​https://github.com/user-attachments/assets/6a11403b-c794-439c-8cb6-59ebf104817f

[adamsilverstein]

Ah, this is awkward 😅, @adamsilverstein beat me to opening a PR. I'll open one anyway containing the patch that I worked on yesterday: ​https://github.com/WordPress/wordpress-develop/pull/7035. It's a different approach, but perhaps there's value in comparing the two.

Ooops! I should have said something about starting to work on this... thanks for working on that PR! It is validating to see both PRs rely on the same underlying methodology of using Imagick directly.

Thanks for the PR @adamsilverstein. It doesn't seem to work when I test it, though.

@noisysocks some parts were missing, I had a little trouble debugging, but finally got it working. Can you give it another test?

Here is a screencast of uploading an HEIC image via drag and drop onto the editor:

​https://github.com/user-attachments/assets/600766e8-bb07-4fa8-ad88-3dfad79139e1

on the right side you can see all of the files being generated in jpeg format.

Using the "upload" selector in Gutenberg fails because HEIC is still not a recognized type there:

This probably needs fixing in Gutenberg.

Note the preview functionality still isn't working correctly.

Yep we'll need to fix those two issues in the Gutenberg repo.

Also, using the "upload" selector in Gutenberg fails because HEIC is still not a recognized type there:

This seems to work for me. Might be browser dependent. We can investigate on the Gutenberg side.

I'm seeing a different error about the image size now when I test with a picture I took the other day.

​https://github.com/user-attachments/assets/1643092f-4c6c-40b4-b8d9-a8469cbe68ee

Wiki says .heic, .heif, image/heic and image/heif are all valid. Should we be liberal in what we accept?

​https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format

OK, it's working well for me with this patch applied:

src/wp-includes/compat.php

@@ -514 +514 @@
 }
 
 // IMAGETYPE_HEIF constant is not yet defined in PHP as of PHP 8.3.
-if ( ! defined( 'IMAGETYPE_HEIF' ) ) {
-        define( 'IMAGETYPE_HEIF', 99 );
+if ( ! defined( 'IMAGETYPE_HEIC' ) ) {
+        define( 'IMAGETYPE_HEIC', 99 );
 }

I tested a bunch of different upload flows. Here's how it looks:

​https://github.com/user-attachments/assets/f1c3d19b-fd5d-46e4-a2fc-d35945653a9c

I like that this approach means that you can download the original HEIC image via the attachment page. Very nice.

👍 LGTM once that define() is fixed.

Here's a fix for the Image block: ​https://github.com/WordPress/gutenberg/pull/63643

Interestingly, in my project I just saw some differences in the browser tests where the browser-determined mime type was once image/heic and once image/heif. Strangely that was for the same image 🤔

I don't know how the mime type detection works in PHP, but it might be worth testing this with a few different HEIC images to see if the image/heif mime type also needs to be checked (in addition to just image/heic).

HEIC images have a different FourCC code depending on the type:

'mif1', // .heic / image/heif
'msf1', // .heic / image/heif-sequence
'heic', // .heic / image/heic
'heix', // .heic / image/heic
'hevc', // .heic / image/heic-sequence
'hevx', // .heic / image/heic-sequence

Looks like media.php references IMAGETYPE_HEIF but combat.php defines IMAGETYPE_HEIC.

Ah, right. sticking with HEIC for now, fixed in a65c937c5e6cb92de8ca76ca36d0d20a44c9430e

Interestingly, in my project I just saw some differences in the browser tests where the browser-determined mime type was once image/heic and once image/heif. Strangely that was for the same image 🤔

I don't know how the mime type detection works in PHP, but it might be worth testing this with a few different HEIC images to see if the image/heif mime type also needs to be checked (in addition to just image/heic).

HEIC images have a different FourCC code depending on the type:

'mif1', // .heic / image/heif
'msf1', // .heic / image/heif-sequence
'heic', // .heic / image/heic
'heix', // .heic / image/heic
'hevc', // .heic / image/heic-sequence
'hevx', // .heic / image/heic-sequence

Currently I'm using ​this bit of code which I haven't really verified works beyond a single test image. This will be important to test throughly because PHP won't properly detect the mime type.

I plan to add tests next and can bundle several test images to make sure they are identified correctly (and hopefully that we can get the sizes correctly with Imagick). I'll be looking for test images that have a compatible license and are relatively small, contributions welcome!

Here's a fix for the Image block: ​WordPress/gutenberg#63643

Amazing, thank you!

Code looks good imho. Wondering if it may be good to inform the user that an image they uploaded was converted? Or perhaps treat the conversions as "standard behavior" and inform them when it cannot be done? Perhaps in the second case WP can advise them to convert it to JPEG and upload it again?

Also, by default WP attempts to keep uploaded images as-is and treat them as "backups". Wondering if this behavior should continue when converting .heic/.heif images too?

This works well in my testing. I really like that you get to the original image from the attachment screen.

Worth noting that I tried uploading heif from ​https://filesampleshub.com/format/image/heif. I was unable to upload and I get this error message.

This works well in my testing. I really like that you get to the original image from the attachment screen.

yum!

Worth noting that I tried uploading heif from ​filesampleshub.com/format/image/heif. I was unable to upload and I get this error message.

Thanks for the link to the sample files, that is helpful.

I could try to extend full support to .heif files. Do we know how common these are or where/how they are created? I double checked and the files my iphone was creating before I switched it off were .heic files.

As an experiment I took a .heic image from my iPhone and renamed it to .heif. Regardless of whether the file is .heic or .heif it works the same in Preview and other Apple apps. In both cases the MIME type that file -I spits out is image/heic.

$ file -I playground-1.HEIF
playground-1.HEIF: image/heic; charset=binary
$ file -I playground-2.HEIC
playground-2.HEIC: image/heic; charset=binary

I think it makes sense to have WordPress do the same. Support both .heic and .heif but map them to image/heic in both cases.

Also, by default WP attempts to keep uploaded images as-is and treat them as "backups". Wondering if this behavior should continue when converting .heic/.heif images too?

Yeah this is what the alternative solution ​https://github.com/WordPress/wordpress-develop/pull/7034 does. I prefer it. Will close this PR.

Other sample images

​https://filesampleshub.com/format/image/heic
​https://github.com/tigranbs/test-heic-images/tree/master
​https://www.learningcontainer.com/sample-heic-files-download/#
​https://github.com/dsoprea/heic-exif-samples

[noisysocks]

@adamsilverstein: How are you going with adding tests? Do you need any help?

[adamsilverstein]

@adamsilverstein: How are you going with adding tests? Do you need any help?

I haven't had a chance to work on it, do you want to give it a pass @noisysocks?

Added some tests @adamsilverstein.

I think coverage is okay, though I struggled to write one for the changes to wp_generate_attachment_metadata. Ended up deciding it's not worth the effort since it more or less is already covered by test_resize_heic.

Something strange I noticed is that our PHP 8.3 container in GitHub Actions doesn't have the Imagick extension enabled but the other containers do 🤷‍♂️

This still tests really well for me. Here's how things look now that ​https://github.com/WordPress/gutenberg/pull/63643 is merged.

​https://github.com/user-attachments/assets/6cfcda0c-d04b-445f-9884-677f539aac2a

I'll commit on Monday if there's no objections. Keen to get something in early for 6.7 so that there's plenty of time for feedback and iteration.

[noisysocks]

In 58849:

Media: Automatically convert HEIC images to JPEG

Automatically create a JPEG version of uploaded HEIC images if the server has
a version of Imagick that supports HEIC. Conversion is done silently through
the existing WP_Image_Editor infrastructure that creates multiple sizes of
uploaded images.

This allows users to view HEIC images in WP Admin and use them in their posts
and pages regardless of whether their browser supports HEIC. Browser support
for HEIC is relatively low (only Safari) while the occurrence of HEIC images is
relatively common. The original HEIC image can be downloaded via a link on
the attachment page.

Props adamsilverstein, noisysocks, swissspidy, spacedmonkey, peterwilsoncc.
Fixes #53645.

Committed in r58849.

[adamsilverstein]

Amazing, thanks for landing this @noisysocks!

Something strange I noticed is that our PHP 8.3 container in GitHub Actions doesn't have the Imagick extension enabled but the other containers do 🤷‍♂️

This is due to an upstream compatibility issue between PHP 8.3 and Imagick, ​issue

[spacedmonkey]

@adamsilverstein @noisysocks I strongly recommend writing and publishing a dev note for this change ASAP. I would like users and web hosts alike should test this change. Specially with different versions of PHP, imagik and GD.

[adamsilverstein]

Replying to spacedmonkey:

@adamsilverstein @noisysocks I strongly recommend writing and publishing a dev note for this change ASAP. I would like users and web hosts alike should test this change. Specially with different versions of PHP, imagik and GD.

+1. Also, note that we added support for Imagick but not GD (yet). I haven't found a test environmrnt with GD supporting HEIC/AVIF so its hard to work on.

[adamsilverstein]

I verified that while HEIC support was added to libgd, it has not been added to the GD fork that is shipped with PHP 8.1+. I left a ​comment on the related tickets describing our use case.

[noisysocks]

Dev note: ​https://make.wordpress.org/core/2024/08/15/automatic-conversion-of-heic-images-to-jpeg-in-wordpress-6-7/

[swissspidy]

In 58942:

Docs: Fix typo in wp_show_heic_upload_error docblock.

Follow-up to [58849].
See #53645.

[adamsilverstein]

Note: Follow up to add HEIC server support telemetry to upgrader data: #61981