Make WordPress Core

Opened 16 months ago

Last modified 13 months ago

#57238 assigned feature request

Criteria for Responsive Images of WebP: Preserve Alpha + Preserve Lossless + Smaller in size than original

Reported by: abitofmind's profile abitofmind Owned by: adamsilverstein's profile adamsilverstein
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: Media Keywords:
Focuses: performance Cc:

Description

On WordPress 6.1.1 I uploaded different WebPs (lossless/lossy + with/without alpha channel) and the Responsive Images that WordPress creates from them should fulfill these criteria

  1. ✅ Preserve the alpha channel → To maintain layout/design capabilities
  2. ✅ Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)
  3. ✅ File size of smaller sized versions should never be larger than the original → Else the purpose of Responsive Images to conserve bandwidth is moot.

To my shock in one case ALL 3 criteria were violated, in other cases some criteria were violated.

Input: WebP, lossless, with alpha channel

Original: Axure-Prototyping-2-dynamic-title-scaled.webp • 2880 × 1750 • 21.77 KB • 1:926 (compression factor)

  1. ❌ Without alpha channel → Looses me layout/design capabilities.
    • Found no ticket "Responsive Images of WebP should preserve alpha channel" - Should we file this?
  2. ❌ Lossy versions from lossless original → Looses me quality against my intention.
    • Possibly related to one of those:
      • #53663 another example for a failed feature detection?
      • #53669 maybe related?
  3. ❌ Larger in file size, even if smaller in dimensions → Contrary to the intended goal of Responsive Images.
    • Axure-Prototyping-2-dynamic-title-scaled.webp • 2560 × 1556 • 29.67 KB • 1:536
    • Axure-Prototyping-2-dynamic-title-2048x1244.webp • 2048 × 1244 • 23.89 KB • 1:426
    • Axure-Prototyping-2-dynamic-title-1536x933.webp • 1536 × 933 • 16.20 KB • 1:353
      • Is the first one to be smaller in file size than the original
    • Issue #53653 fixed larger file sizes by avoiding lossless versions from lossy original.
      • But here we have the reverse, but nevertheless an increase in file size! Also need to avoid that.

Input: WebP, lossless, without alpha channel

Original: Confluence-Absence-Inspector-in-Create-Mode.webp • 2880 × 1620 • 672.73 KB • 1:27

  1. N/A No alpha channel present in the original. So no alpha channel can be lost either.
  2. ❌ Lossy versions from lossless original → Undesired!
  3. ✅ File size of smaller sized versions are all smaller than the original:
    • Confluence-Absence-Inspector-in-Create-Mode-scaled.webp • 2560 × 1440 • 136.88 KB • 1:107
      • Already the first smaller version is already smaller in file size
        • Thanks to compression ofc. Note: When the version would remain lossless (see point 2) I still would have that expectation ofc.

Further notes

  • The WebP files were created from PNGs with GraphicConverter 11.7.1 (beta build 5672) with these metadata options:
  • With webpinfo I confirmed the existence of alpha channel and lossless/lossy status of WebP for all mentioned files (originals + responsive image versions by WordPress) .

Change History (14)

#1 @azaozz
16 months ago

  • Milestone changed from Awaiting Review to Future Release
  • Type changed from defect (bug) to feature request
  • Version 6.1.1 deleted

Yes, handling/post-processing of WebP images needs improvements.

Preserve the alpha channel

Yes, definitely. Note that WebP support alpha transparency with both lossless and lossy compression. As far as I see ImageMagick can do that. Not sure about GD, probably can.

Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)

Makes sense but perhaps only for images with slammer file size? The purpose of image post-processing in WP is to make images "web ready". A 2MB lossless image is not. If lossy compression can be used to significantly reduce the file size without noticeable reduction in quality, it should be used imho.

On the other hand it is pretty rare to see such images. I actually had problems finding "real" lossless WebP images to test with (real as in specifically crafted that way for use on the web). So not sure if implementing this is a priority.

There are some examples of lossless and lossy compression with alpha transparency: https://developers.google.com/speed/webp/gallery2.

File size of smaller sized versions should never be larger than the original

Yes, of course. This usually applies to images with lossless compression. In some cases the sub-sizes of the image may not be created with the same settings. For example a sub-size may be created with full colors while the original has only 8 bit, etc.

Related #53663. Also pinging @adamsilverstein who may be interested.

#2 @abitofmind
16 months ago

First an erratum

  • Original file name is Axure-Prototyping-2-dynamic-title.webp without the -scaled suffix!
  • But the image dimension, file size, compression factor were reported correctly.

(Offtopic: Really hate that TRAC does not allow editing an initial issue post, makes communication really inefficient)

Discussion regarding the criteria, factoring in your answers/infos:

1) Preserve the alpha channel

  • Both lossy and lossless WebP support alpha → Great, no need for complicated decision logic here.
  • If both graphic libraries (ImageMagick, GD) are confirmed to support it → We would need no decision logic at all!

2) Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)

  • Is it safe to assume that if an original is provided as lossless, that this is intentional?
    • I would assume that WebP files which typical consumer software produces as output will by default be lossy. And lossless to be a pro option only.
    • Or do we need to fear that non tech savvy have unintentional lossless WebP files which could greatly benefit from lossy compression at little visual degradation?
  • To follow the WordPress philosophy "Options are bad" an Opt-in (as a general preference or on a per image basis in the Media Library) for preservation of lossless WebP certainly would be acceptable for Pro users, if no safe assumptions (see b) can be made that ensure things are going alright automatically for the average user.
    • Specifically: My real world example file Axure-Prototyping-2-dynamic-title.webp showed there are lossless files which can be considered web ready, because its creator knew what to do.
    • In these cases (large monochromatic areas, or a lot of void in images with alpha) lossless compression can be really "good enough" to be web ready. And as it seems even better as the lossy variant, which astonished me.
    • Or where the "extra in file size" is worth the guaranteed lossless image (logos, diagrams, screen designs or screenshots "which should shine") for the author.

3) File size of smaller sized versions should never be larger than the original

  • What is the concrete reason here why the lossless original with alpha Axure-Prototyping-2-dynamic-title.webp is smaller in file size than its next 2 smaller sub-sized lossy siblings?

This ticket was mentioned in Slack in #core by abhanonstopnews. View the logs.


16 months ago

#4 @abitofmind
15 months ago

Again an example where the smaller sized lossy version is larger in file size than original WebP lossless with alpha:

836 × 426 original lossless 24.598 bytes
768 x 391 thumbnail lossy 35.324 bytes (!)
300  ×  153 thumbnail lossy 9.056 bytes
150 ×  76 thumbnail lossy 3.346 bytes

I compared them with Exiftool:

  • They are all: 24 Bit Color, 8 Bit Alpha
  • But although the sized versions use compression, their alpha channel is saved uncompressed!
  • I cannot explain why this is larger than the original which supposedly has both composite and alpha channel uncompressed, but I do not know the details of the WebP format.

But hopefully this gives you an idea what is going on and how to improve it.

  • Attached is a screenshot of the image file comparison + ZIP archive containing original and 3 sized versions
  • Below you find the relevant ExifTool output (irrelevant stuff like file permissions, dates, etc removed)
ExifTool Version Number         : 12.50

======== Style-Guide.webp
File Name                       : Style-Guide.webp
File Size                       : 25 kB
File Type                       : Extended WEBP
File Type Extension             : webp
MIME Type                       : image/webp
WebP Flags                      : Alpha
Image Width                     : 836
Image Height                    : 426
Image Size                      : 836x426
Megapixels                      : 0.356

======== Style-Guide-768x391.webp
File Name                       : Style-Guide-768x391.webp
File Size                       : 35 kB
File Type                       : Extended WEBP
File Type Extension             : webp
MIME Type                       : image/webp
WebP Flags                      : Alpha
Image Width                     : 768
Image Height                    : 391
Alpha Preprocessing             : Level Reduction
Alpha Filtering                 : Horizontal
Alpha Compression               : Lossless
VP8 Version                     : 0 (bicubic reconstruction, normal loop)
Horizontal Scale                : 0
Vertical Scale                  : 0
Image Size                      : 768x391
Megapixels                      : 0.300

======== Style-Guide-300x153.webp
File Name                       : Style-Guide-300x153.webp
File Size                       : 9.1 kB
File Type                       : Extended WEBP
File Type Extension             : webp
MIME Type                       : image/webp
WebP Flags                      : Alpha
Image Width                     : 300
Image Height                    : 153
Alpha Preprocessing             : Level Reduction
Alpha Filtering                 : Horizontal
Alpha Compression               : Lossless
VP8 Version                     : 0 (bicubic reconstruction, normal loop)
Horizontal Scale                : 0
Vertical Scale                  : 0
Image Size                      : 300x153
Megapixels                      : 0.046


======== Style-Guide-150x76.webp
File Name                       : Style-Guide-150x76.webp
File Size                       : 3.3 kB
File Type                       : Extended WEBP
File Type Extension             : webp
MIME Type                       : image/webp
WebP Flags                      : Alpha
Image Width                     : 150
Image Height                    : 76
Alpha Preprocessing             : Level Reduction
Alpha Filtering                 : Horizontal
Alpha Compression               : Lossless
VP8 Version                     : 0 (bicubic reconstruction, normal loop)
Horizontal Scale                : 0
Vertical Scale                  : 0
Image Size                      : 150x76
Megapixels                      : 0.011

#5 @joemcgill
14 months ago

  • Focuses performance added

#6 @adamsilverstein
14 months ago

  • Owner set to adamsilverstein
  • Status changed from new to assigned

#7 @adamsilverstein
14 months ago

Thanks for opening this @abitofmind!

I'm curious about the environment you are testing in because format support may depend on the underlying GD or Imagick library your web server has. You can find this information under Tools->Site Health->Info->Media Handling.

✅ Preserve the alpha channel → To maintain layout/design capabilities
✅ Lossless original should result in lossless sized versions → To preserve full quality (e.g. screen designs, diagrams, etc)
✅ File size of smaller sized versions should never be larger than the original → Else the purpose of Responsive Images to conserve bandwidth is moot.

I think you are raising three separate issues:

  1. Alpha channel preservation - this should already work with both GD and Imagick, I will test to verify
  2. Lossless format -> output should be lossless. This is a little tricky - first, we do support with Imagick which supports lossless (with a quality setting of 100); GD requires PHP 8.1, we can probably add support for this maybe by checking for the new IMG_WEBP_LOSSLESS constant (see https://github.com/php/php-src/pull/7348. I can work on a patch for that!
  3. File size: we don't currently check file sizes at all and that isn't really the purpose of the responsive images entirely. Is this directly related to WebP?

#8 @abitofmind
14 months ago

  • Environment info attached.
  • Thanks for looking into 1 and 2.

File Size: The HTML output provides a srcset and the web browser tries to get the smallest possible variant to saturate its needs. With the implicit assumption that a smaller image dimension is by tendency also smaller in file size. To fulfill the end goal: Consume as little network bandwidth as necessary for a fast loading time. There is no filesize stated per variant in the srcset b/c of that implicit assumption. Otherwise the srcset standard would probably have provided a means to indicate filesize-in-bytes per each img variant. Needing a little less of client-side image scaling (and hence RAM/GPU) may be a small benefit for low end devices, but negligible. The main goal of Responsive Images is definitely asset loading speed. How to achieve that sized images are smaller in file size too?

  1. Sophisticated: By in depth analysis of perceptive quality in the input and then creating the output similarly too, but never unnecessarily higher.
  2. By best practises: Which ensure that the sized versions are by tendency never larger than the input and have a certain agreed-upon quality level.
    • ✅ This is what I meant.
    • And if users reports an example where this is violated, then adapt the algorithm accordingly:
    • e.g. my discovery in comment:4
    • Or other example: When the input has a limited color set (color bits per pixel), also use that in the output, or else you will blow up the sized versions unnecessarily with no information gain.
    • Avoid adding unnecessary extra metadata (color profile, etc) or other chunks which are not present in the source.
    • And similar best practises.
Last edited 14 months ago by abitofmind (previous) (diff)

#9 @adamsilverstein
14 months ago

By best practises: Which ensure that the sized versions are by tendency never larger than the input and have a certain agreed-upon quality level.

Ah, I see your point - when the generated images are larger that the original, it may not make sense to include them.

This is something we could do when building the srcset then - we have all the file sizes at that point. If we see this we could skip those images in the srcset. I 'm not sure we could handle it when generating the images though.

I think this deserves a separate trac ticket, it is a pretty distinct issue from the handling of Lossless and Alpha transparency on upload and I believe has more to do with te srcset we generate.

The main goal of Responsive Images is definitely asset loading speed

Yes... that is one goal and it is also about serving the right sized image for the viewer (device). For example, screen size but also screen density - part of this might be sending larger assets to HighDPI devices for example, which are larger images and would load slower, but look much better.

Avoid adding unnecessary extra metadata (color profile, etc) or other chunks which are not present in the source.

WordPress already removes meta data on uploads, for speed and privacy reasons.

And if users reports an example where this is violated, then adapt the algorithm accordingly:

We may not be able to predict which images would be larger, but we could avoid serving them. Would this only happen with the large or full sized image?

#10 @abitofmind
13 months ago

1) Goal setting of Responsive Images

The main goal of Responsive Images is definitely asset loading speed

Yes... that is one goal and it is also about serving the right sized image for the viewer (device).

The usability goals are to get optimal: 1) quality, 2) speed and to some degree 3) compatibility (multiple file formats).

  • Goal 1 could technically be achieved by just providing a single high definiton image, and each devices scale down as needed. For this alone no Responsive Image solution like SRCSET would be necessary.
  • Responsive Images solutions only exists mostly to facilitate goal 2!
  • Seeing our end goals clearly can be helpful in the decision process how to fulfill the criteria (stated in issue title).

2) When the generated images are larger than the original, it indeed makes no sense to include them in the SRCSET.

  • a) In theory this should not happen if the sizing algorithm gets further optimized now (and continually gets improved whenever new edge cases get reported). Please investigate with the source material and findings I provided to you.
  • b) I agree that as a "last safety net" it is clever to exclude the smaller sized variants if larger in file size. I will create a separate ticket and cross-reference them.
  • c) You asked for which variants this happens: My assumptions from the observations:
    • c1) Each sized variant created by WP Core Media will always be smaller in file size then the next higher sized variant. I can truly not imagine any anomaly with these.
    • c2) But the first X (ca. 1-2) sized variants may be larger in file size than the provided original if that original was very heavily optimized (as my provided samples prove, I'm a pro).
      • c2a) For lossless originals the only remaining possible explanation that I see is that the sized versions create an alpha channel with unnecessarily more information density (bit-depth, compression, etc) than in the original. Please investigate from my provided samples/findings. Once thats fixed I cannot foresee this to happen.
      • c2b) Metadata/byte-padding/etc you confirmed to not being happening for the sized variants.
      • c2c) For lossy originals with strong compression, the first Y (ca. 1-3) lossy sized variants may be larger, as long as WP Media Core has no perceptual analytics and adaptive target quality but simply a fixed target compression a la "works well on average". For these anomalies the fallback solution "exclude from SRCSET" could kick in.
      • c2d) Perceptual analysis and quality targetting is out of scope for WP Media Core.
        • Who needs this must use a specialized 3rd party service IMHO.
        • Implementing the simple ruleset "lossless in → lossless out" and "lossy in → lossy out" which was implicit in PNG and JPEG, now also for WEBP (which can be both!) will solves the majority of anomalies, I'm sure.
        • Because competent content creators create line art images as lossless (and have better quality and in many cases ALSO smaller file size than with average lossy most of the times) and will save photos as lossy.
        • And mere "image users" (editors, bloggers) anyhow mostly get the images from content creators (directly or via stock image services) who know what they are doing.
        • Even amateur image creators will be safe with most WEBP software which defaults to lossy and offers lossless only in advanced settings. On a very few occasions they may manage to save a photo as lossless unintentionally. This very unlikely unhappy edge case would result in "larger than nesessary" reponsive image files. I think this is is better than punishing all pros who knowingly create line art as lossless files which WEBP compresses very reasonable anyhow (contiguous areas, color palette reduction, etc).
      • c2e) If the collective expert conclusion here is to not preserve lossless WEBPs but compress all WEBPs by default, then offer a way how a WEBP file can signal to WordPress "I'm a lossless original and I demand lossless sized variants"
        • c2e1) with a very particular filename suffix such as "--lossless" which gets removed after upload (for nice filenames / SEO)
        • c2e2) marker in the metadata
        • c2e3) "serve lossless" checkbox in the media library / upload dialog. (greyed out and unchecked for lossy WEBPs)
Version 0, edited 13 months ago by abitofmind (next)
Note: See TracTickets for help on using tickets.