Add a method to generate the image subsizes with multiple requests

[azaozz]

One of the most common reason image uploading fails is when the server doesn't have enough resources to generate all image subsizes "in one go". Then the server runs out of memory or times out, several image subsizes may be generated but the attachment meta in not updated so we don't know about them. The result is uploading has failed and there are several "orphaned" image subsizes in the uploads directory.

[azaozz]

There are (at least) two ways we can fix this:

• Generate only one subsize on image upload, add meta for it and return "success". Then do another request (from PHP) and generate another. Continue until all subsizes are created.
• On uploading first create the attachment post, then update the attachment meta after each image subsize is created. If the upload fails to create all subsizes, send another (AJAX) request from the browser to continue creating subsizes (we will need some UI for that).

[enshrined]

I was actually playing with something similar a while ago in order to stop the issue with the upload failing.

It was only a quick plugin knocked up for personal use but in case it's of any inspiration here it is: ​https://github.com/darylldoyle/WP-Background-Image-Processing

I'd love to see this addressed though, it can be a really frustrating issue, especially when themes/plugins define lots of image sizes.

[azaozz]

Thanks @enshrined this looks pretty good.

it can be a really frustrating issue, especially when themes/plugins define lots of image sizes.

Yeah, exactly. Most shared servers would have enough resources to generate WPs default image subsizes, but when themes and plugins start adding more and more it becomes pretty intensive.

Thinking we'll probably do wp_cron in combination with "self requests" to trigger it right after an image is uploaded (some smaller or cached sites may not get the necessary "hits" soon enough). Of course there will be some sensible limit on this wp_cron loop. Also if there is an error we will have to set it on the attachment post (meta) and display in the UI so the user can retry.

There should also be a way for the user to retry creating just the missing sizes. We may actually be able to fix this by getting the user more involved in the process. Adding a check if all files have been created, outputting an error when not, and a button to try to create the rest.

[azaozz]

Possibly related #41973. Seems some of the reported errors there were fixed by increasing memory limit and server timeouts.

[kirasong]

Is this a duplicate of #40439?

Related task/umbrella ticket: #39647

[azaozz]

Replying to mikeschroder:

Is this a duplicate of #40439?

Yeah, similar. Would be good to fix #40439 first.

This ticket also tries to explore/implement additional actions the user can take after an upload has failed during creation of subsizes, or wp_cron tasks (plus additional requests) that will ensure the creation of the rest of the subsizes.

[joemcgill]

I agree that this is a duplicate of #40439. Let's close this and ensure @azaozz thoughts about implementation are included in discussion for that ticket.