Make WordPress Core

Opened 4 years ago

Closed 4 years ago

Last modified 22 months ago

#50587 closed enhancement (fixed)

Provide short-circuit filter for scandir() in wp_unique_filename()

Reported by: joehoyle's profile joehoyle Owned by: sergeybiryukov's profile SergeyBiryukov
Milestone: 5.5 Priority: normal
Severity: normal Version: 5.3
Component: Upload Keywords: has-patch dev-feedback has-dev-note
Focuses: performance Cc:

Description (last modified by SergeyBiryukov)

In #42437, WordPress introduced an enhancement to how unique filenames are calculated. The tl;dr of that change was to run scandir() on the directory that a file will be placed in, and iterate over all files, to check if there is any files that can potentially conflict with the newly added file (or future resize variants).

In situations where you have many thousands of files in your monthly uploads directories, or where the upload DIR path is particularly slow for access (such as remote file systems) this scandir() can cause a large performance overhead.

I propose we add a "pre filter" to scandir() to allow plugins to override the file fetching behaviour. The current implementation is quite wasteful, as it will do a list on the whole directory, but then call preg_match() on each file with ^$filename*.... In situations where plugins can get file listings by other mechanisms, a file list for $dir/$filename* would return many fewer results but provide the same file list needed for _wp_check_existing_file_names() for example.

Change History (8)

This ticket was mentioned in PR #393 on WordPress/wordpress-develop by joehoyle.

4 years ago

  • Keywords has-patch added

#2 @joehoyle
4 years ago

  • Keywords dev-feedback added

#3 @SergeyBiryukov
4 years ago

  • Description modified (diff)

#4 @SergeyBiryukov
4 years ago

  • Focuses performance added
  • Milestone changed from Awaiting Review to 5.5

#5 @SergeyBiryukov
4 years ago

  • Owner set to SergeyBiryukov
  • Resolution set to fixed
  • Status changed from new to closed

In 48369:

Upload: Introduce pre_wp_unique_filename_file_list filter to allow for short-circuiting the scandir() call in wp_unique_filename().

This allows plugins to override the file fetching behavior to provide performance improvements for large directories.

Props joehoyle.
Fixes #50587.

#6 @desrosj
4 years ago

  • Keywords has-dev-note added

This received a call out in the Miscellaneous Developer Changes in 5.5 dev note:

This ticket was mentioned in Slack in #core-media by joedolson. View the logs.

23 months ago

#8 @antpb
22 months ago

#54609 was marked as a duplicate.

Note: See TracTickets for help on using tickets.