Opened 23 months ago
Last modified 20 months ago
#57238 assigned feature request
Criteria for Responsive Images of WebP: Preserve Alpha + Preserve Lossless + Smaller in size than original
Reported by: | abitofmind | Owned by: | adamsilverstein |
---|---|---|---|
Milestone: | Future Release | Priority: | normal |
Severity: | normal | Version: | |
Component: | Media | Keywords: | |
Focuses: | performance | Cc: |
Description
On WordPress 6.1.1 I uploaded different WebPs (lossless/lossy + with/without alpha channel) and the Responsive Images that WordPress creates from them should fulfill these criteria
- ✅ Preserve the alpha channel → To maintain layout/design capabilities
- ✅ Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)
- ✅ File size of smaller sized versions should never be larger than the original → Else the purpose of Responsive Images to conserve bandwidth is moot.
To my shock in one case ALL 3 criteria were violated, in other cases some criteria were violated.
Input: WebP, lossless, with alpha channel
Original: Axure-Prototyping-2-dynamic-title-scaled.webp • 2880 × 1750 • 21.77 KB • 1:926 (compression factor)
- ❌ Without alpha channel → Looses me layout/design capabilities.
- Found no ticket "Responsive Images of WebP should preserve alpha channel" - Should we file this?
- ❌ Lossy versions from lossless original → Looses me quality against my intention.
- ❌ Larger in file size, even if smaller in dimensions → Contrary to the intended goal of Responsive Images.
- Axure-Prototyping-2-dynamic-title-scaled.webp • 2560 × 1556 • 29.67 KB • 1:536
- Axure-Prototyping-2-dynamic-title-2048x1244.webp • 2048 × 1244 • 23.89 KB • 1:426
- Axure-Prototyping-2-dynamic-title-1536x933.webp • 1536 × 933 • 16.20 KB • 1:353
- Is the first one to be smaller in file size than the original
- Issue #53653 fixed larger file sizes by avoiding lossless versions from lossy original.
- But here we have the reverse, but nevertheless an increase in file size! Also need to avoid that.
Input: WebP, lossless, without alpha channel
Original: Confluence-Absence-Inspector-in-Create-Mode.webp • 2880 × 1620 • 672.73 KB • 1:27
- N/A No alpha channel present in the original. So no alpha channel can be lost either.
- ❌ Lossy versions from lossless original → Undesired!
- ✅ File size of smaller sized versions are all smaller than the original:
- Confluence-Absence-Inspector-in-Create-Mode-scaled.webp • 2560 × 1440 • 136.88 KB • 1:107
- Already the first smaller version is already smaller in file size
- Thanks to compression ofc. Note: When the version would remain lossless (see point 2) I still would have that expectation ofc.
- Already the first smaller version is already smaller in file size
- Confluence-Absence-Inspector-in-Create-Mode-scaled.webp • 2560 × 1440 • 136.88 KB • 1:107
Further notes
- The WebP files were created from PNGs with GraphicConverter 11.7.1 (beta build 5672) with these metadata options:
- Without any color profile information (the raw pixel values are sRGB compliant, so the default fallback sRGB applies)
- With a minimal Exif chunk which only stores resolution info (I tag my Retina images as 144 PPI, something which I would like WordPress to detect as a @2x Retina image, but that's another issue)
- Btw I was concerned that that Exif chunk is at the end of the file (at the end of the linked post), but the developer of GraphicConverter said, I just use the public code from Google to create the WebP and can't influence the order, that's how the WebP comes out.
- Wanted to mention it here for completeness, maybe relevant.
- Btw I was concerned that that Exif chunk is at the end of the file (at the end of the linked post), but the developer of GraphicConverter said, I just use the public code from Google to create the WebP and can't influence the order, that's how the WebP comes out.
- With webpinfo I confirmed the existence of alpha channel and lossless/lossy status of WebP for all mentioned files (originals + responsive image versions by WordPress) .
Attachments (4)
Change History (14)
#1
@
22 months ago
- Milestone changed from Awaiting Review to Future Release
- Type changed from defect (bug) to feature request
- Version 6.1.1 deleted
#2
@
22 months ago
First an erratum
- Original file name is
Axure-Prototyping-2-dynamic-title.webp
without the-scaled
suffix! - But the image dimension, file size, compression factor were reported correctly.
(Offtopic: Really hate that TRAC does not allow editing an initial issue post, makes communication really inefficient)
Discussion regarding the criteria, factoring in your answers/infos:
1) Preserve the alpha channel
- Both lossy and lossless WebP support alpha → Great, no need for complicated decision logic here.
- If both graphic libraries (ImageMagick, GD) are confirmed to support it → We would need no decision logic at all!
- Sadly GD's imagesavealpha documentation is not mentioning whether that applies to WebP too, only mentions PNG explicitly. I inquired to please amend the documentation in regards of WebP support and a mention that this would help the WordPress team to make decisions, with a backlink to here.
- Does anyone know from trial'n'error or other sources?
- Sadly GD's imagesavealpha documentation is not mentioning whether that applies to WebP too, only mentions PNG explicitly. I inquired to please amend the documentation in regards of WebP support and a mention that this would help the WordPress team to make decisions, with a backlink to here.
2) Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)
- Is it safe to assume that if an original is provided as lossless, that this is intentional?
- I would assume that WebP files which typical consumer software produces as output will by default be lossy. And lossless to be a pro option only.
- Or do we need to fear that non tech savvy have unintentional lossless WebP files which could greatly benefit from lossy compression at little visual degradation?
- To follow the WordPress philosophy "Options are bad" an Opt-in (as a general preference or on a per image basis in the Media Library) for preservation of lossless WebP certainly would be acceptable for Pro users, if no safe assumptions (see b) can be made that ensure things are going alright automatically for the average user.
- Specifically: My real world example file
Axure-Prototyping-2-dynamic-title.webp
showed there are lossless files which can be considered web ready, because its creator knew what to do. - In these cases (large monochromatic areas, or a lot of void in images with alpha) lossless compression can be really "good enough" to be web ready. And as it seems even better as the lossy variant, which astonished me.
- Or where the "extra in file size" is worth the guaranteed lossless image (logos, diagrams, screen designs or screenshots "which should shine") for the author.
- Specifically: My real world example file
3) File size of smaller sized versions should never be larger than the original
- What is the concrete reason here why the lossless original with alpha
Axure-Prototyping-2-dynamic-title.webp
is smaller in file size than its next 2 smaller sub-sized lossy siblings?
This ticket was mentioned in Slack in #core by abhanonstopnews. View the logs.
22 months ago
#4
@
21 months ago
Again an example where the smaller sized lossy version is larger in file size than original WebP lossless with alpha:
836 × 426 | original | lossless | 24.598 bytes |
768 x 391 | thumbnail | lossy | 35.324 bytes (!) |
300 × 153 | thumbnail | lossy | 9.056 bytes |
150 × 76 | thumbnail | lossy | 3.346 bytes |
I compared them with Exiftool:
- They are all: 24 Bit Color, 8 Bit Alpha
- But although the sized versions use compression, their alpha channel is saved uncompressed!
- I cannot explain why this is larger than the original which supposedly has both composite and alpha channel uncompressed, but I do not know the details of the WebP format.
But hopefully this gives you an idea what is going on and how to improve it.
- Attached is a screenshot of the image file comparison + ZIP archive containing original and 3 sized versions
- Below you find the relevant ExifTool output (irrelevant stuff like file permissions, dates, etc removed)
ExifTool Version Number : 12.50 ======== Style-Guide.webp File Name : Style-Guide.webp File Size : 25 kB File Type : Extended WEBP File Type Extension : webp MIME Type : image/webp WebP Flags : Alpha Image Width : 836 Image Height : 426 Image Size : 836x426 Megapixels : 0.356 ======== Style-Guide-768x391.webp File Name : Style-Guide-768x391.webp File Size : 35 kB File Type : Extended WEBP File Type Extension : webp MIME Type : image/webp WebP Flags : Alpha Image Width : 768 Image Height : 391 Alpha Preprocessing : Level Reduction Alpha Filtering : Horizontal Alpha Compression : Lossless VP8 Version : 0 (bicubic reconstruction, normal loop) Horizontal Scale : 0 Vertical Scale : 0 Image Size : 768x391 Megapixels : 0.300 ======== Style-Guide-300x153.webp File Name : Style-Guide-300x153.webp File Size : 9.1 kB File Type : Extended WEBP File Type Extension : webp MIME Type : image/webp WebP Flags : Alpha Image Width : 300 Image Height : 153 Alpha Preprocessing : Level Reduction Alpha Filtering : Horizontal Alpha Compression : Lossless VP8 Version : 0 (bicubic reconstruction, normal loop) Horizontal Scale : 0 Vertical Scale : 0 Image Size : 300x153 Megapixels : 0.046 ======== Style-Guide-150x76.webp File Name : Style-Guide-150x76.webp File Size : 3.3 kB File Type : Extended WEBP File Type Extension : webp MIME Type : image/webp WebP Flags : Alpha Image Width : 150 Image Height : 76 Alpha Preprocessing : Level Reduction Alpha Filtering : Horizontal Alpha Compression : Lossless VP8 Version : 0 (bicubic reconstruction, normal loop) Horizontal Scale : 0 Vertical Scale : 0 Image Size : 150x76 Megapixels : 0.011
#7
@
20 months ago
Thanks for opening this @abitofmind!
I'm curious about the environment you are testing in because format support may depend on the underlying GD or Imagick library your web server has. You can find this information under Tools->Site Health->Info->Media Handling.
✅ Preserve the alpha channel → To maintain layout/design capabilities
✅ Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)
✅ File size of smaller sized versions should never be larger than the original → Else the purpose of Responsive Images to conserve bandwidth is moot.
I think you are raising three separate issues:
- Alpha channel preservation - this should already work with both GD and Imagick, I will test to verify
- Lossless format -> output should be lossless. This is a little tricky - first, we do support with Imagick which supports lossless (with a quality setting of 100); GD requires PHP 8.1, we can probably add support for this maybe by checking for the new
IMG_WEBP_LOSSLESS
constant (see https://github.com/php/php-src/pull/7348. I can work on a patch for that! - File size: we don't currently check file sizes at all and that isn't really the purpose of the responsive images entirely. Is this directly related to WebP?
#8
@
20 months ago
- Environment info attached.
- Thanks for looking into 1 and 2.
File Size: The HTML output provides a srcset
and the web browser tries to get the smallest possible variant to saturate its needs. With the implicit assumption that a smaller image dimension is by tendency also smaller in file size. To fulfill the end goal: Consume as little network bandwidth as necessary for a fast loading time. There is no filesize stated per variant in the srcset b/c of that implicit assumption. Otherwise the srcset standard would probably have provided a means to indicate filesize-in-bytes per each img variant. Needing a little less of client-side image scaling (and hence RAM/GPU) may be a small benefit for low end devices, but negligible. The main goal of Responsive Images is definitely asset loading speed. How to achieve that sized images are smaller in file size too?
- Sophisticated: By in depth analysis of perceptive quality in the input and then creating the output similarly too, but never unnecessarily higher.
- ❌ Certainly not in Core. Too sophisticated. There are plugins which send the images via API to external specialized SaaS, e.g. https://wordpress.org/plugins/wp-smushit/
- By best practises: Which ensure that the sized versions are by tendency never larger than the input and have a certain agreed-upon quality level.
- ✅ This is what I meant.
- And if users reports an example where this is violated, then adapt the algorithm accordingly:
- e.g. my discovery in comment:4
- Or other example: When the input has a limited color set (color bits per pixel), also use that in the output, or else you will blow up the sized versions unnecessarily with no information gain.
- Avoid adding unnecessary extra metadata (color profile, etc) or other chunks which are not present in the source.
- And similar best practises.
#9
@
20 months ago
By best practises: Which ensure that the sized versions are by tendency never larger than the input and have a certain agreed-upon quality level.
Ah, I see your point - when the generated images are larger that the original, it may not make sense to include them.
This is something we could do when building the srcset then - we have all the file sizes at that point. If we see this we could skip those images in the srcset. I 'm not sure we could handle it when generating the images though.
I think this deserves a separate trac ticket, it is a pretty distinct issue from the handling of Lossless and Alpha transparency on upload and I believe has more to do with te srcset we generate.
The main goal of Responsive Images is definitely asset loading speed
Yes... that is one goal and it is also about serving the right sized image for the viewer (device). For example, screen size but also screen density - part of this might be sending larger assets to HighDPI devices for example, which are larger images and would load slower, but look much better.
Avoid adding unnecessary extra metadata (color profile, etc) or other chunks which are not present in the source.
WordPress already removes meta data on uploads, for speed and privacy reasons.
And if users reports an example where this is violated, then adapt the algorithm accordingly:
We may not be able to predict which images would be larger, but we could avoid serving them. Would this only happen with the large or full sized image?
#10
@
20 months ago
1) Goal setting of Responsive Images
The main goal of Responsive Images is definitely asset loading speed
Yes... that is one goal and it is also about serving the right sized image for the viewer (device).
The usability goals are to get optimal: 1) quality, 2) speed and to some degree 3) compatibility (multiple file formats).
- Goal 1 could technically be achieved by just providing a single high definiton image, and each devices scale down as needed. For this alone no Responsive Image solution like SRCSET would be necessary.
- Responsive Images solutions only exists mostly to facilitate goal 2!
- Seeing our end goals clearly can be helpful in the decision process how to fulfill the criteria (stated in issue title).
2) When the generated images are larger than the original, it indeed makes no sense to include them in the SRCSET.
- a) In theory this should not happen if the sizing algorithm gets further optimized now (and continually gets improved whenever new edge cases get reported).
- Please investigate with the source material and findings I provided to you.
- b) I agree with the idea of a "last safety net" and filed a separate enhancement ticket:
- #57781 Exclude sized variant from SRCSET if larger in file size than the original image
- c) "For which variants does this happens?" → My assumptions from the observations:
- c1) Each sized variant created by WP Core Media will always be smaller in file size then the next higher sized variant. I can truly not imagine any anomaly with these.
- c2) But the first X (ca. 1-2) sized variants may be larger in file size than the provided original if that original was very heavily optimized (as my provided samples prove, I'm a pro).
- c2a) For lossless originals the only remaining possible explanation that I see is that the sized versions create an alpha channel with unnecessarily more information density (bit-depth, compression, etc) than in the original. Please investigate from my provided samples/findings. Once thats fixed I cannot foresee this to happen.
- c2b) Metadata/byte-padding/etc you confirmed to not being happening for the sized variants.
- c2c) For lossy originals with strong compression, the first Y (ca. 1-3) lossy sized variants may be larger, as long as WP Media Core has no perceptual analytics and adaptive target quality but simply a fixed target compression a la "works well on average". For these anomalies the fallback solution "exclude from SRCSET" could kick in.
- c2d) Perceptual analysis and quality targetting is out of scope for WP Media Core.
- Who needs this must use a specialized 3rd party service IMHO.
- Implementing the simple ruleset "lossless in → lossless out" and "lossy in → lossy out" which was implicit in PNG and JPEG, now also for WEBP (which can be both!) will solves the majority of anomalies, I'm sure.
- Because competent content creators create line art images as lossless (and have better quality and in many cases ALSO smaller file size than with average lossy most of the times) and will save photos as lossy.
- And mere "image users" (editors, bloggers) anyhow mostly get the images from content creators (directly or via stock image services) who know what they are doing.
- Even amateur image creators will be safe with most WEBP software which defaults to lossy and offers lossless only in advanced settings. On a very few occasions they may manage to save a photo as lossless unintentionally. This very unlikely unhappy edge case would result in "larger than nesessary" reponsive image files. I think this is is better than punishing all pros who knowingly create line art as lossless files which WEBP compresses very reasonable anyhow (contiguous areas, color palette reduction, etc).
- c2e) If the collective expert conclusion here is to not preserve lossless WEBPs but compress all WEBPs by default, then offer a way how a WEBP file can signal to WordPress "I'm a lossless original and I demand lossless sized variants"
- c2e1) with a very particular filename suffix such as "--lossless" which gets removed after upload (for nice filenames / SEO)
- c2e2) marker in the metadata
- c2e3) "serve lossless" checkbox in the media library / upload dialog. (greyed out and unchecked for lossy WEBPs)
Yes, handling/post-processing of WebP images needs improvements.
Yes, definitely. Note that WebP support alpha transparency with both lossless and lossy compression. As far as I see ImageMagick can do that. Not sure about GD, probably can.
Makes sense but perhaps only for images with slammer file size? The purpose of image post-processing in WP is to make images "web ready". A 2MB lossless image is not. If lossy compression can be used to significantly reduce the file size without noticeable reduction in quality, it should be used imho.
On the other hand it is pretty rare to see such images. I actually had problems finding "real" lossless WebP images to test with (real as in specifically crafted that way for use on the web). So not sure if implementing this is a priority.
There are some examples of lossless and lossy compression with alpha transparency: https://developers.google.com/speed/webp/gallery2.
Yes, of course. This usually applies to images with lossless compression. In some cases the sub-sizes of the image may not be created with the same settings. For example a sub-size may be created with full colors while the original has only 8 bit, etc.
Related #53663. Also pinging @adamsilverstein who may be interested.