Make WordPress Core

Opened 15 months ago

Last modified 11 days ago

#40439 new enhancement

Save progress of intermediate image creation after upload

Reported by: mikeschroder Owned by:
Milestone: 5.0 Priority: normal
Severity: normal Version:
Component: Media Keywords: has-patch needs-testing needs-unit-tests
Focuses: ui Cc:


When uploads fail with an HTTP Error, a user currently has no recourse other than to re-upload media and hope that it makes it the next time.

Currently, WordPress only saves data about intermediate sizes in meta after all of the sizes have been completed on upload. This means that every time a user attempts to re-upload, WordPress is required to resize *every* intermediate size necessary over again during this retry.

WordPress should save the progress of each successful image resize/intermediate so that it can be resumed on retry.

See #39647 for the global ticket on solving HTTP Error issues on upload.

Attachments (2)

40439.patch (21.1 KB) - added by azaozz 3 weeks ago.
missing-files.png (18.2 KB) - added by azaozz 3 weeks ago.

Download all attachments as: .zip

Change History (14)

This ticket was mentioned in Slack in #core-media by mike. View the logs.

14 months ago

#2 @enshrined
14 months ago

If it's of any interest I had a play with this issue as a plugin here https://github.com/darylldoyle/WP-Background-Image-Processing.

It stops creating image sizes on upload and then uses the background processing library by deliciousbrains to create intermediate sizes in the background.

#3 @nosilver4u
14 months ago

Be careful with WPBIP if you have any batch image import processes on your site. I use the background lib from delicious brains in EWWW Image Optimizer, and it has a rather large issue with dispatching the queue. It doesn't reset the $data variable after saving it to the db, so you can end up with something like this: queue 1: 1 queue 2: 1,2 queue 3: 1,2,3 queue 4: 1,2,3,4 queue 5: 1,2,3,4,5 ... queue 10: 1,2,3,4,5,6,7,8,9,10 and so on. Import a couple hundred images, and you'll have a couple hundred queues with 1-200 items in them. There were some other issues with batch imports that I had to fix as well. You might be able to drop-in the background processing libs with the ones from EWWW IO: https://github.com/nosilver4u/ewww-image-optimizer/tree/master/vendor

This ticket was mentioned in Slack in #core-media by mike. View the logs.

13 months ago

This ticket was mentioned in Slack in #core-media by joemcgill. View the logs.

12 months ago

This ticket was mentioned in Slack in #core-media by joemcgill. View the logs.

6 months ago

#7 @blobfolio
6 months ago

An alternative to a queue- or batch-type approach would be to alter image_get_intermediate_size() (and probably a few related functions) so that it consults the global $_wp_additional_image_sizes to see if a size exists (independently of the stored image meta), and run a basic file_exists()-type check to see if the file does indeed exist on the system, and if not, build it then and there.

The above would help WP recover from upload timeouts, and as an added bonus, help old media transition to new themes.

The main downside is that file_exists() lookup times can add up, particularly on Windows hosts. Another factor to consider is that this would potentially push thumbnail generation overhead to the frontend, which could lead to slow first loads (if i.e. a lot of thumbnails were missing).

To clarify, the Core wouldn't really be changing its default upload behaviors with the above approach. These changes would serve as more of a failsafe, catching and creating images that should exist but don't for whatever reason. :)

Last edited 6 months ago by blobfolio (previous) (diff)

This ticket was mentioned in Slack in #core-media by mike. View the logs.

4 months ago

#9 @azaozz
3 weeks ago

  • Milestone changed from Future Release to 5.0

This seems simple enough to implement: instead of generating all image sizes and saving them first, then updating the image_meta in the db, we should be updating the meta after each size is created. That way if a size creation fails for some reason, we will be able to re-run that loop and create any missing sizes (and add the meta for them).

It's true, this will write to the db a few times more, but that is negligible overhead compared to timeouts of 30 seconds :)

The main downside is that file_exists() lookup times can add up

True, but still negligible (compared with 30 sec. running). Also, not sure we should do file_exists() for each newly created sub-size. If the file was saved successfully, it does exist at that point :)

Thinking we should try this now. After this is implemented we can think of a good UI/UX to expose the ability to regenerate image sizes when some are missing, see #43525.

#10 @azaozz
3 weeks ago

  • Focuses ui added
  • Keywords has-patch needs-testing needs-unit-tests added; needs-patch removed

In 40439.patch:

  • Abstract/expose creating of an image sub-size in class-wp-image-editor-imagick.php and class-wp-image-editor-gd.php. Note that I didn't change the signature of Class_WP_Image_Editor as "something" may be extending it.
  • Refactor wp_generate_attachment_metadata() a bit, move the creation of image sub-sizes to a new function: _wp_create_image_subsizes().
  • Add couple more helper functions for creating and checking of image sub-sizes.
  • Add wp_update_image_attachment_sizes() that tries to create all missing sub-sized of an image.
  • Add wp_check_image_attachment_sizes() that is used to check whether all sub-sizes were created successfully.
  • Use the above function in the Media Library list-table to check for missing image sizes and add a button if any are found, so the user can create them.
  • Add AJAX for the above.

This started as a "quick test" to see exactly how difficult it may be to add that functionality (I know, it's Sunday, but...) :)

TODO: needs UI/UX review and perhaps some design for how to add the messages and the button to the Media Library grid view (and the media modal?). Also probably needs some cleanup, and of course: testing!

3 weeks ago

3 weeks ago

This ticket was mentioned in Slack in #core-media by azaozz. View the logs.

3 weeks ago

This ticket was mentioned in Slack in #core-media by mike. View the logs.

11 days ago

Note: See TracTickets for help on using tickets.