Make WordPress Core

Opened 12 months ago

Closed 12 months ago

Last modified 11 months ago

#50587 closed enhancement (fixed)

Provide short-circuit filter for scandir() in wp_unique_filename()

Reported by: joehoyle Owned by: SergeyBiryukov
Milestone: 5.5 Priority: normal
Severity: normal Version: 5.3
Component: Upload Keywords: has-patch dev-feedback has-dev-note
Focuses: performance Cc:

Description (last modified by SergeyBiryukov)

In #42437, WordPress introduced an enhancement to how unique filenames are calculated. The tl;dr of that change was to run scandir() on the directory that a file will be placed in, and iterate over all files, to check if there is any files that can potentially conflict with the newly added file (or future resize variants).

In situations where you have many thousands of files in your monthly uploads directories, or where the upload DIR path is particularly slow for access (such as remote file systems) this scandir() can cause a large performance overhead.

I propose we add a "pre filter" to scandir() to allow plugins to override the file fetching behaviour. The current implementation is quite wasteful, as it will do a list on the whole directory, but then call preg_match() on each file with ^$filename*.... In situations where plugins can get file listings by other mechanisms, a file list for $dir/$filename* would return many fewer results but provide the same file list needed for _wp_check_existing_file_names() for example.

Change History (6)

This ticket was mentioned in PR #393 on WordPress/wordpress-develop by joehoyle.

12 months ago

  • Keywords has-patch added

#2 @joehoyle
12 months ago

  • Keywords dev-feedback added

#3 @SergeyBiryukov
12 months ago

  • Description modified (diff)

#4 @SergeyBiryukov
12 months ago

  • Focuses performance added
  • Milestone changed from Awaiting Review to 5.5

#5 @SergeyBiryukov
12 months ago

  • Owner set to SergeyBiryukov
  • Resolution set to fixed
  • Status changed from new to closed

In 48369:

Upload: Introduce pre_wp_unique_filename_file_list filter to allow for short-circuiting the scandir() call in wp_unique_filename().

This allows plugins to override the file fetching behavior to provide performance improvements for large directories.

Props joehoyle.
Fixes #50587.

#6 @desrosj
11 months ago

  • Keywords has-dev-note added

This received a call out in the Miscellaneous Developer Changes in 5.5 dev note: https://make.wordpress.org/core/2020/07/29/miscellaneous-developer-focused-changes-in-wordpress-5-5/.

Note: See TracTickets for help on using tickets.