Audit and preload current "from" version files into memory before upgrading filesystem to the "to" version

[hellofromTonya]

Follow-up to #54546.

Some terms for clarity:

• current "from" version: the current version of WordPress on the site
• new "to" version: the version of WordPress that the site will be upgraded to after the upgrade process is complete

The Problem

In the wp-admin/includes/update-core.php file, the filesystem is upgraded to the new "to" version before all of the upgrader process is completed.

What happens if the rest of the upgrader process doesn't yet have its code loaded into memory?

• Fatal error, for example #54546
• Unexpected behavior due to intermingling of "from" and "to" version of code

A fatal error happened in #54546 when cURL timed out and attempted to throw an exception when its file hadn't yet been loaded into memory. The exception class was changed, which would be handled by the "to" version autoloader but was not recognized by the "from" version autoloader in memory.

While #54546 is a specific instance, it revealed a fundamental problem in Core's upgrader.

The Goal

Ensure all files from the current ("from") version of WordPress that may be needed during the upgrade process are loaded into memory before upgrading the filesystem and database to the "to" version of WordPress.

Audit and Preload current "from" version files into memory

An audit will be needed to identify all of the files that are or could be used during the upgrade process, including alternative paths such as errors which may have dependencies.

A preloader mechanism will be needed capable of loading nested directories (such as loading an entire library or API into memory such as Requests) and individual files.

Consideration will be needed for:

• differences in current "from" versions of files and code
• possibly for downgrades from "to" version back to the "from" version
• case-insensitive filesystems as identified in #54582

Props to @jrf, @pbiron, @costdev, @schlessera, @sergeybiryukov, and @mkaz for extensive investigation and testing.

This PR implements a preloader for the upgrade process.

• This is set to preload the wp-includes/Requests directory.
• This is set to skip preloading wp-includes/Requests/Autoload.php on WP < 5.9.
• These are in place as examples but should be part of the overall audit.

Source

• [x] Allow preloading files.
• [x] Allow recursively preloading directories.
• [x] Allow skipping files from preloading.
• [ ] Identify additional functionality that may be needed. Please identify in a review for discussion.
• [ ] Identify additional functionality that may be helpful. Please suggest in a review for discussion.
• [ ] Audit additional files needed by the upgrader.
• [ ] Add the additional paths to the preloader, skipping individual files where necessary.

Tests

• [x] Add initial tests to achieve 100% line/branch coverage.
• [ ] If necessary, review the sample files implementation for an alternative.
• [ ] Add more tests.
• [ ] Manual testing.
• [ ] Review documentation.

Trac ticket: https://core.trac.wordpress.org/ticket/54589

[costdev]

Per the discussion on the bug scrub, I'm changing this ticket to an enhancement, as it proposes a general improvement as a follow-up to #54546 and changing the Version to 2.7.

While the WP_Upgrader and Core_Upgrader classes were introduced in 2.8, wp-admin/includes/update-core.php was introduced in 2.7, and the copying of Core files via _copy_dir() - initially, copy_dir() - was introduced in [8595]. From this point onwards, there was no mechanism to ensure required resources were preloaded.

[peterwilsoncc]

I'm moving this off the milestone for now, it looks like it still has a bit of work before it's ready for merging.

The tests seem inadequate.

If class-b.php, e.g.

class B extends A {}

is require_once'd after class-a.php has been, then it will fail.

I'm not familiar with the WordPress build process itself, but how I would approach this is:

Use ​Composer\Autoload\ClassMapGenerator::createMap() to build a classmap from the autoload key in the library.
Use either regex (or maybe Rector related tool) to read each PHP file and know which rely on none and which ones subclass/implement/use others.
Build a new "ordered" classmap that can safely be used with require_once.
This new classmap can be used in a regular autoloader too.

I worked on a library for prefixing namespaces for WordPress plugins to avoid conflicts: ​https://github.com/BrianHenryIE/strauss

(my point was to show how I've thought of this in terms of WordPress already. But it's honest to say: The same code could be used to rename namespaces, i.e. the WpOrg WPorg WordPress Wordpress discussion that was earlier.)

Again, I'm not familiar with the WordPress build process and how it could use Composer libraries, but I do see how Strauss's functionality could be split into another library that helps this issue – ultimately listing all classes in a library and ordering them based on what they require (extend/implement/use).

I put together a proof of concept here: ​https://github.com/BrianHenryIE/composer-require-once

It's really just one function in one class, with a test running against Requests.

Requires ​composer/composer and ​nikic/php-parser.