Make WordPress Core

Opened 2 years ago

Last modified 18 months ago

#54589 new enhancement

Audit and preload current "from" version files into memory before upgrading filesystem to the "to" version

Reported by: hellofromtonya's profile hellofromTonya Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version: 2.7
Component: Upgrade/Install Keywords: early has-patch has-unit-tests
Focuses: Cc:

Description

Follow-up to #54546.

Some terms for clarity:

  • current "from" version: the current version of WordPress on the site
  • new "to" version: the version of WordPress that the site will be upgraded to after the upgrade process is complete

The Problem

In the wp-admin/includes/update-core.php file, the filesystem is upgraded to the new "to" version before all of the upgrader process is completed.

What happens if the rest of the upgrader process doesn't yet have its code loaded into memory?

  • Fatal error, for example #54546
  • Unexpected behavior due to intermingling of "from" and "to" version of code

A fatal error happened in #54546 when cURL timed out and attempted to throw an exception when its file hadn't yet been loaded into memory. The exception class was changed, which would be handled by the "to" version autoloader but was not recognized by the "from" version autoloader in memory.

While #54546 is a specific instance, it revealed a fundamental problem in Core's upgrader.

The Goal

Ensure all files from the current ("from") version of WordPress that may be needed during the upgrade process are loaded into memory before upgrading the filesystem and database to the "to" version of WordPress.

Audit and Preload current "from" version files into memory

An audit will be needed to identify all of the files that are or could be used during the upgrade process, including alternative paths such as errors which may have dependencies.

A preloader mechanism will be needed capable of loading nested directories (such as loading an entire library or API into memory such as Requests) and individual files.

Consideration will be needed for:

  • differences in current "from" versions of files and code
  • possibly for downgrades from "to" version back to the "from" version
  • case-insensitive filesystems as identified in #54582

Props to @jrf, @pbiron, @costdev, @schlessera, @sergeybiryukov, and @mkaz for extensive investigation and testing.

Change History (13)

This ticket was mentioned in Slack in #core by hellofromtonya. View the logs.


2 years ago

This ticket was mentioned in PR #2015 on WordPress/wordpress-develop by costdev.


2 years ago
#2

  • Keywords has-patch has-unit-tests added

This PR implements a preloader for the upgrade process.

  • This is set to preload the wp-includes/Requests directory.
  • This is set to skip preloading wp-includes/Requests/Autoload.php on WP < 5.9.
  • These are in place as examples but should be part of the overall audit.

Source

  • [x] Allow preloading files.
  • [x] Allow recursively preloading directories.
  • [x] Allow skipping files from preloading.
  • [ ] Identify additional functionality that may be needed. Please identify in a review for discussion.
  • [ ] Identify additional functionality that may be helpful. Please suggest in a review for discussion.
  • [ ] Audit additional files needed by the upgrader.
  • [ ] Add the additional paths to the preloader, skipping individual files where necessary.

Tests

  • [x] Add initial tests to achieve 100% line/branch coverage.
  • [ ] If necessary, review the sample files implementation for an alternative.
  • [ ] Add more tests.
  • [ ] Manual testing.
  • [ ] Review documentation.

Trac ticket: https://core.trac.wordpress.org/ticket/54589

This ticket was mentioned in Slack in #core by jrf. View the logs.


2 years ago

This ticket was mentioned in Slack in #core by costdev. View the logs.


21 months ago

This ticket was mentioned in Slack in #core by costdev. View the logs.


21 months ago

#6 @costdev
21 months ago

  • Type changed from defect (bug) to enhancement
  • Version set to 2.7

Per the discussion on the bug scrub, I'm changing this ticket to an enhancement, as it proposes a general improvement as a follow-up to #54546 and changing the Version to 2.7.

While the WP_Upgrader and Core_Upgrader classes were introduced in 2.8, wp-admin/includes/update-core.php was introduced in 2.7, and the copying of Core files via _copy_dir() - initially, copy_dir() - was introduced in [8595]. From this point onwards, there was no mechanism to ensure required resources were preloaded.

This ticket was mentioned in Slack in #core-auto-updates by afragen. View the logs.


21 months ago

This ticket was mentioned in Slack in #core by chaion07. View the logs.


21 months ago

#9 @peterwilsoncc
21 months ago

  • Milestone changed from 6.0 to Future Release

I'm moving this off the milestone for now, it looks like it still has a bit of work before it's ready for merging.

This ticket was mentioned in Slack in #core-auto-updates by costdev. View the logs.


20 months ago

This ticket was mentioned in Slack in #core by costdev. View the logs.


18 months ago

BrianHenryIE commented on PR #2015:


18 months ago
#12

The tests seem inadequate.

If class-b.php, e.g.

class B extends A {}

is require_once'd after class-a.php has been, then it will fail.

I'm not familiar with the WordPress build process itself, but how I would approach this is:

Use Composer\Autoload\ClassMapGenerator::createMap() to build a classmap from the autoload key in the library.
Use either regex (or maybe Rector related tool) to read each PHP file and know which rely on none and which ones subclass/implement/use others.
Build a new "ordered" classmap that can safely be used with require_once.
This new classmap can be used in a regular autoloader too.

I worked on a library for prefixing namespaces for WordPress plugins to avoid conflicts: https://github.com/BrianHenryIE/strauss

(my point was to show how I've thought of this in terms of WordPress already. But it's honest to say: The same code could be used to rename namespaces, i.e. the WpOrg WPorg WordPress Wordpress discussion that was earlier.)

Again, I'm not familiar with the WordPress build process and how it could use Composer libraries, but I do see how Strauss's functionality could be split into another library that helps this issue – ultimately listing all classes in a library and ordering them based on what they require (extend/implement/use).

BrianHenryIE commented on PR #2015:


18 months ago
#13

I put together a proof of concept here: https://github.com/BrianHenryIE/composer-require-once

It's really just one function in one class, with a test running against Requests.

Requires composer/composer and nikic/php-parser.

Note: See TracTickets for help on using tickets.