Make WordPress Core

Opened 12 hours ago

Last modified 9 minutes ago

#66763 assigned defect (bug)

Client side media: the upload stays on disk after permanent delete for big image when the output format is filtered

Reported by: b1ink0 Owned by: adamsilverstein
Priority: normal Milestone: Awaiting Review
Component: Media Version: 7.1
Severity: normal Keywords: has-patch has-unit-tests
Cc: Focuses: rest-api, performance

Description

This happens with client side media processing when an image_editor_output_format filter changes the output format, for example JPEG to WebP. Without such a filter, these uploads work as expected.

The browser uploads the file with generate_sub_sizes set to false. Since [63014], WP_REST_Attachments_Controller::create_item() also turns off big_image_size_threshold for these uploads, so that the upload is stored untouched. With an output format filter it isn't: wp_create_image_subsizes() treats the image as small and converts the main file, so photo.jpg is saved as photo.webp, with photo.jpg as original_image.

For an image over the big image size threshold (2560 by default), the browser then sideloads its own scaled image, which replaces the main file again. sideload_item() records the file it replaces as original_image, and that file is now photo.webp, not the upload. Nothing points at photo.jpg anymore, so it stays in the uploads folder after the image is deleted permanently. wp_get_original_image_path() also returns photo.webp. The full size photo.webp the server encoded is also thrown away right after it's written.

The block editor uses this pipeline, and now Media Library uploads also uses it with 7.2.

Steps to reproduce

Add this as a must-use plugin:

<?php
add_filter(
        'image_editor_output_format',
        function ( $formats ) {
                $formats['image/jpeg'] = 'image/webp';
                return $formats;
        }
);
  1. In a browser that supports client side media processing, upload a JPEG larger than 2560 pixels (for example 3000x2000) in the block editor or the Media Library.
  2. Look in the uploads folder. There is photo.jpg, photo.webp and photo-scaled.webp, and the attachment's original_image is photo.webp.
  3. Delete the image permanently from the Media Library.
  4. photo.jpg is still in the uploads folder.

Change History (10)

#1 @b1ink0
12 hours ago

EXIF rotated images hit the same problem at any size, not only over the big image size threshold. For a JPEG with an EXIF orientation other than 1, the browser sideloads the rotated copy with image_size set to original, which since [62805] replaces the main file like a scaled sideload. So photo.jpg is left behind the same way, and the main file ends up as the rotated JPEG instead of WebP. Uploading a JPEG under 2560 pixels with an EXIF orientation of 6 shows it with the must-use plugin from the description.

Images that are both big and EXIF rotated also leave photo.webp behind, since they get both sideloads (​Gutenberg #83757).

This ticket was mentioned in ​PR #16119 on ​WordPress/wordpress-develop by ​@b1ink0.


11 hours ago
#2

  • Keywords has-patch has-unit-tests added

With client side media processing and an image_editor_output_format filter, the server converted the main file of the upload, for example photo.jpg to photo.webp. The browser then replaced the main file with its own scaled or rotated image, and the sideload recorded photo.webp as original_image. Nothing pointed at photo.jpg anymore, so it stayed on disk after "Delete permanently".

When generate_sub_sizes is false, create_item() now leaves the main file unconverted, using the same filter as convert_format: false. The browser supplies the converted main file through the sideload endpoint, so only one place writes the main file. The filter is added after the upload is stored, so the upload's name still reserves the names of the converted files. If another attachment already has photo.webp, the upload is saved as photo-1.jpg and its converted copy as photo-1.webp.

The new test uploads a JPEG with a JPEG to WebP filter and checks that the upload stays the attached file, that no WebP copy is written, and that the response still asks the client for WebP.

This needs the client change from ​https://github.com/WordPress/gutenberg/pull/87135 in the same release. Without it, images under the big image size threshold keep a main file in the uploaded format.

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5.5
Used for: Tracing the upload flow, a script that replays the client's REST requests, and drafts of the fix. I reviewed, edited and tested the final changes.

---
This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See ​GitHub Pull Requests for Code Review in the Core Handbook for more details.

🤖 Generated with ​Claude Code

#3 @b1ink0
11 hours ago

  • Focuses rest-api added

#4 @adamsilverstein
84 minutes ago

  • Owner set to adamsilverstein
  • Status new → assigned

#5 @adamsilverstein
76 minutes ago

Thanks for the ticket @b1ink0.

We can probably handle the EXIF rotation part separately.

Following your steps to reporoduce:

  • In a browser that supports client side media processing, upload a JPEG larger than 2560 pixels (for example 3000x2000) in the block editor or the Media Library.
  • Look in the uploads folder. There is photo.jpg, photo.webp and photo-scaled.webp, and the attachment's original_image is photo.webp.

So far, so good. I expect the actual original image to still be uploaded and saved.

Here i expect original_image to point to the webp since thats what most places serve, then I expect an additional meta value "source_image" that points to the true original: the uploaded jpg.

  • Delete the image permanently from the Media Library.
  • photo.jpg is still in the uploads folder.

This is the actual bug, photo.jpg should also be deleted.

​@adamsilverstein commented on ​PR #16119:


72 minutes ago
#6

This needs the client change from ​https://github.com/WordPress/gutenberg/pull/87135 in the same release. Without it, images under the big image size threshold keep a main file in the uploaded format.

I believe thats expected behavior - _the original upload is always saved_. we may called the -scaled version "original_image" in meta so it gets served that way, but under the hood the original upload is a different image preserved under a different key, likely source_image.

#7 @adamsilverstein
49 minutes ago

With an output format filter it isn't: wp_create_image_subsizes() treats the image as small and converts the main file, so photo.jpg is saved as photo.webp, with photo.jpg as original_image.

we don't really want the server to do the conversion though. when a file is uploaded with generate_subsizes false, it should only store the original image. the rest of the required images - sub sizes, scaled, new format output, exif rotated etc. should all be created on the client side. that way it works for formats like avif the server may not support

#8 @b1ink0
29 minutes ago

Replying to adamsilverstein:

we don't really want the server to do the conversion though. when a file is uploaded with generate_subsizes false, it should only store the original image. the rest of the required images - sub sizes, scaled, new format output, exif rotated etc. should all be created on the client side.

Agreed, and that's what the two PRs do together:

  • ​Gutenberg #87135: the browser creates the converted main file for every image that needs a new format, and sends it with a sideload like the sub sizes. Today it only sends a main file when the image needs scaling or rotating.
  • ​wordpress-develop #16119: when generate_sub_sizes is false, the server stores the upload as it is and no longer converts it.

Both are needed. For an image that doesn't need scaling or rotating, the server's conversion is currently the only thing that turns the served file into photo.webp, so the server can only stop converting once the browser takes over that job.

Deleting the leftover photo.jpg afterwards would also stop the leak, but the server would keep encoding a full size copy of every upload only for the browser to replace it. Fixing it where the conversion happens removes both the extra work and the leftover file.

Replying to adamsilverstein:

Here i expect original_image to point to the webp since thats what most places serve, then I expect an additional meta value "source_image" that points to the true original: the uploaded jpg.

The served file is the attached file, not original_image. With both PRs, uploading a 3000x2000 photo.jpg gives:

  • served file: photo-scaled.webp
  • original_image: photo.jpg, the untouched upload

That is also what core does when it processes the same upload on the server. I checked it with client side processing turned off: photo-scaled.webp is served and photo.jpg is original_image. Regenerating sub sizes reads original_image, so it starts from the real upload.

source_image is for uploads the browser can't show, such as HEIC, where a JPEG becomes the attachment and the HEIC is kept next to it. A JPEG upload doesn't need it.

This is the actual bug, photo.jpg should also be deleted.

Yes. With both PRs nothing is left in the uploads folder after a permanent delete.

We can probably handle the EXIF rotation part separately.

Sure. The PRs already cover it, since the browser now rotates and converts the main file in one step, but I can move it to its own ticket if you prefer.

One correction to the ticket description: the full size photo.webp the server writes today isn't thrown away. It stays on disk as original_image, but nothing serves it.

​@b1ink0 commented on ​PR #16119:


26 minutes ago
#9

I believe thats expected behavior - _the original upload is always saved_. we may called the -scaled version "original_image" in meta so it gets served that way, but under the hood the original upload is a different image preserved under a different key, likely source_image.

Agreed that the upload is always saved, and with both PRs it is. The sentence you quoted was unclear, sorry. By "main file" I meant the file the attachment serves, not the upload. I've reworded it in the description.

One thing about the keys: the served file is the attached file, and original_image is the upload. With both PRs, a 3000x2000 photo.jpg is served as photo-scaled.webp and kept as original_image, the same as when core processes it on the server.

More detail on the ticket: https://core.trac.wordpress.org/ticket/66763#comment:8

​@adamsilverstein commented on ​PR #16119:


9 minutes ago
#10

Thanks for the detailed replies @b1ink0, that clears up my confusion about the keys - the attached file is what gets served and original_image is the upload. Agreed the server shouldn't be converting here. I wanted to double check that we wind up with the same files as the server path, so I had Claude test that.

_Claude ran the uploads on each path and compared the results:_

Tested in the block editor (Chromium) with the JPEG to WebP filter from the ticket, using a 3000x2000 and a 2000x1332 JPEG. "Server path" is the same upload with wp_client_side_media_processing_enabled filtered to false.

3000x2000 (over the big image threshold)

| Path | Attached (served) file | original_image | Full size files on disk | Left after delete |
|---|---|---|---|---|
| Server path | photo-scaled.webp | photo.jpg | photo.jpg, photo-scaled.webp | nothing |
| Client side, trunk | photo-scaled.webp | photo.webp | photo.jpg, photo.webp, photo-scaled.webp | photo.jpg |
| Client side, this PR | photo-scaled.webp | photo.jpg | photo.jpg, photo-scaled.webp | nothing |

2000x1332 (under the threshold)

| Path | Attached (served) file | original_image | Full size files on disk | Left after delete |
|---|---|---|---|---|
| Server path | photo.webp | photo.jpg | photo.jpg, photo.webp | nothing |
| Client side, trunk | photo.webp | photo.jpg | photo.jpg, photo.webp | nothing |
| Client side, this PR with trunk's client | photo.jpg | not set | photo.jpg | nothing |
| This PR plus the requests from Gutenberg #87135 | photo.webp | photo.jpg | photo.jpg, photo.webp | nothing |

In every case the sub-sizes are WebP, source_image is not set and post_mime_type stays image/jpeg, the same as the server path.

So for the big image this PR matches the server path exactly and fixes the leftover file. For the smaller image it only matches once the client sends the converted main file: with trunk's client the full size stays a JPEG, where trunk and the server path both serve photo.webp.

Note the last row was not tested with a build of ​https://github.com/WordPress/gutenberg/pull/87135. It replays the requests that PR sends (upload, an original sideload of the full size WebP with convert_format false, then finalize) against this branch, so it confirms the server side of the pairing only.

On the server path a full size WebP at the upload's dimensions only exists for images under the threshold. Over the threshold the full size is photo-scaled.webp and there is no unscaled WebP, which this PR also matches. The upload itself is kept as original_image in both cases.

Suggestions:

  • This should not be committed ahead of the Gutenberg change reaching core, since on its own it changes the full size format for images under the threshold. Committing it together with the gutenberg.sha bump that includes #87135 would avoid that window.
  • The new test covers create_item() only. A flow test with the filter active (upload, scaled sideload, finalize, permanent delete) asserting original_image is the JPEG and no files remain would cover the bug from the ticket, ideally with an original sideload case for the smaller image as well.
  • generate_sub_sizes false now also means the main file is not converted. Worth saying so in the parameter description and the @since note on create_item().
  • Adding the filter after the upload is stored looks like the right place, since wp_unique_filename() still reserves the .webp name. The inline comment could say that, it is the non-obvious part.

Unrelated to this PR: for the 2000x1332 image the client names medium_large 767x511 where the server path produces 768x511. Probably worth its own issue.

Before, on trunk (photo.jpg is left after "Delete permanently"):

​https://github.com/user-attachments/assets/bfb4ffdc-e8b1-4cae-9c7c-058278da5798

After, with this PR:

​https://github.com/user-attachments/assets/51b13775-72c3-4cbf-bb59-da52b99cb5e0

Does the plan to land both together work for you? I'd rather not have the core change go in first.

Note: See TracTickets for help on using tickets.