Use composer to install and update external PHP libraries

[spacedmonkey]

As of WordPress 5.2, the minimum version of PHP requires was changed to 5.6. This opens up some options around how PHP dependencies can be managed. ​Composer is a popular PHP dependency manager similar to npm or yarn for javascript. WordPress core has support already got ​support for composer since 4.7. Since 5.0, many of the javascript dependancies are now loaded in using npm, php should have a similar build step.

Here are a list of dependancies that could be loaded in using composer.

• ​ID3
• ​Requests
• ​Pomo
• ​SimplePie
• ​sodium_compat
• ​random_compat
• ​Text_Diff

If there are modification to these library in core, these libraries should be forked and maintained outside of core. Composer has a custom install path feature, that may allow us to maintain the current file structure. However, it may mean creating a new vendor directory (wp_vendor) and use autoloading.

Loading composer these via composer will remove a lot of the bloat of the code case, will make it easier for the core to manage updates in the future.

[spacedmonkey]

As a talking point, I create the first patch 47256.diff. The patch is completely untested. But it does maintain the directory / file location using a package called mnsami/composer-custom-directory-installer.

[swissspidy]

As of WordPress 5.2, the minimum version of PHP requires was changed to 5.6. This opens up some options around how PHP dependencies can be managed.

Since this is only meant for during the build process, the recent PHP version bump should be irrelevant for that, no?

If we want to have autoloading as well, I'd suggest continuing work on #36335.

If there are modification to these library in core, these libraries should be forked and maintained outside of core.

If only minor modifications are needed, the most common approach is to ​use patches.

But it does maintain the directory / file location using a package called mnsami/composer-custom-directory-installer

Hmm that package seems to be quite popular, but not very actively maintained. Might this become an issue?

[johnbillion]

There's also oomphinc/composer-installers-extender which is much more popular, even if it too hasn't been updated recently. We use this at Human Made for client projects.

[giuseppe.mazzapica]

Would not make sense to write a custom installer?

The installer code would be quite simple, according to the current patch, it could be something around the lines of (completely untested):

<?php
use Composer\Config;
use Composer\Installer\LibraryInstaller;
use Composer\Package\PackageInterface;

class WordPressLibInstaller extends LibraryInstaller
{
    const SRC_BASE = 'src/wp-includes/';

    const PACKAGES = [
        'paragonie/sodium_compat' => 'sodium_compat',
        'paragonie/random_compat' => 'random_compat',
        'pomo/pomo' => 'pomo',
        'simplepie/simplepie' => 'SimplePie',
        'rmccue/requests' => 'Requests',
        'pear/text_diff' => 'Text',
        'james-heinrich/getid3' => 'ID3',
    ];

    private $vendor;

    public function getInstallPath(PackageInterface $package)
    {
        $name = $package->getPrettyName();
        if (empty(self::PACKAGES[$name])) {
            return $this->defaultVendor();
        }

        return self::SRC_BASE . self::PACKAGES[$name] . '/';
    }

    public function supports($packageType)
    {
        return $packageType === 'library';
    }

    public function defaultVendor()
    {
        if (!$this->vendor) {
            $this->vendor = $this->composer->getConfig()
                ->get('vendor-dir', Config::RELATIVE_PATHS) ?: 'vendor';
        }

        return $this->vendor;
    }
}

Which has basically the same maintenance burden of the config in the installer-paths array in composer.json.

[netweb]

Switching the the component to Build/Test Tools which is where other tickets discussing Composer reside.

[pierlo]

Copying the library using a custom installer would also include additional files bundled with the library that is not relevant to the project. Adding the libraries to composer.json then using a script to copy the needed files (such as the Webpack copy plugin) would be the best option for now until autoloading is possible.

[pierlo]

47256.1.diff shows an example of getting the dependencies through Composer then using the Webpack copy plugin to update the libraries in src/wp-includes.

This introduces a custom Composer installer plugin in order to facilitate installing some third party libraries via Composer. This is needed because none of the existing Composer installer libraries support features such as ignoring files or installing just a subdirectory of the package, both of which are needed for WordPress.

Trac ticket: https://core.trac.wordpress.org/ticket/47256

## Done

• PHPMailer
• sodium_compat

## In progress

• Requests
  • Need to handle the fact that Requests/library/Requests.php is not actually the file from Requests
  • Need to address the fact that security hardening was applied in core and never backported to Requests
• SimplePie
  • Need to investigate why there is a difference in src/IRI.php

## Todo

• ID3
• Pomo
• random_compat
• Text_Diff

[sajib1223]

The patch does not apply cleanly against trunk 7.0-alpha-61215-src.

Running "patch:47256" (patch) task
? Please select a patch to apply 47256.1.diff​ (2.7 KB) - added by pierlo 6 years ago.
patching file composer.json
Hunk #1 FAILED at 9.
1 out of 1 hunk FAILED -- saving rejects to file composer.json.rej
can't find file to patch at input line 42
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git tools/webpack/packages.js tools/webpack/packages.js
|index d040f0701f..5f6a393a4b 100644
|--- tools/webpack/packages.js
|+++ tools/webpack/packages.js
--------------------------
File to patch: tools/webpack/packages.js
tools/webpack/packages.js: No such file or directory
Skip this patch? [y]
Skipping patch.
2 out of 2 hunks ignored

3\. Check and commit the changed files

Is this necessary? Couldn't those files now become ignored from version control? This is what is being done for npm package dependencies, at least increasingly. For example, the long-overdue update to CodeMirror now uses webpack to build the scripts from the npm package: Core-61539.

@westonruter Yes we could go this route but it would require anyone running a checkout of the repo to install dependencies using Composer. This would affect sites such as wordpress.org AFAIK.

Doable yes, but not sure I want to propose it.

@johnbillion Wouldn't they need to also have npm installed and run a build anyway in that case? I'd be surprised if dotorg is running WordPress/develop-develop. Wouldn't it instead be running WordPress/WordPress so that all built files are present?

Dunno. @dd32 what say you?

[johnbillion]

Punting pending further discussion.

Related: whether the php-api-client could be pulled in via Composer instead of committing the source directly to SVN. On Slack: ​https://wordpress.slack.com/archives/C08TJ8BPULS/p1772583774524569?thread_ts=1772583774.524569&cid=C08TJ8BPULS

[mmorris8]

Pointed to this ticket by @westonruter

Parallel to the core is how composer's availablity affects plugins. Currently plugins that make use of composer libraries have to ship the code with their distribution, which is not idea.

Worse, to prevent conflicts with other plugins that may use the same libraries plugins must "monkey type" those libraries using tools like Strauss ( ​https://github.com/BrianHenryIE/strauss ) which is again not ideal as such programmatic namespace reassignment can introduce bugs.

Third problem, two plugins wanting to use different incompatible versions of the same library. A policy will need to be in place when such a conflict arises as both the plugins cannot be active at the same time. There's also the tricky situation where both the plugins need to be updated together so that they both are using the same dependency.

Fourth - Plugins that are shipped as composer libraries are going to live in the composer vendor library, not in the current plugin directory. This could cause trouble for custom setups that shuffle plugins around from the default location. Having plugin code in three locations (including mu-plugins) instead of two is a consideration. Maybe not a problem, but it is a change worth nothing.

A plugin that is packaged as a composer library likely has little to no need for the traditional root file except for writing hooks. Those, strictly speaking, only need the hooks to be invoked and that could be moved into their package.json file. Composer will ignore attributes in the package.json it doesn't recognize, so we could extend that definition to include hooks like this

{
  "hooks": [
    { "activation": {"ExamplePlugin\\Installer", "activate"} },
    { "deactivation": {"ExamplePlugin\\Installer", "deactivate"} },
    { "uninstall": {"ExamplePlugin\\Installer", "uninstall"} }
  ]
}

That isn't going to be executable, but executable PHP code can be generated from it. The install process could maintain such a file.

I was around for the testing of the Drupal integration of composer back in the day, and the problems faced there will surface here. Some of the same solutions will apply, and some will require new solutions. I'm willing to help with this and think I do have enough experience for this.

Before writing further implementation though I think a discussion on goals and scope is warranted. Plocking in composer to handle some core libraries without giving any thought to how that will affect the theme and plugin ecosystem is a patently bad idea. If composer is brought in there will need to be core code that handles dependency management issues beyond those composer can handle.

In short, if this is carefully thought out and done it will be a massive win. Mishandle this and it's going to be a disaster.

[mmorris8]

#65305 was marked as a duplicate.

[spacedmonkey]

Following up on a discussion with @johnbillion at WordCamp EU regarding external vendor management in WordPress core:

As core relies more on third-party libraries, we need a standardized strategy for handling vendor dependencies. Third-party packages have varying degrees of upstream maintenance, core customization requirements, and patch longevity, meaning a single, rigid approach for all vendor packages causes friction during core updates.

To resolve this, we can categorize third-party dependencies into three clear tiers and apply a tailored workflow for each:

1. Standard / Unmodified Dependencies

Criteria: Upstream packages that work out of the box without core-specific changes.

Workflow: Installed via Composer, committed directly to trunk as-is, and updated through standard Composer version bumps.

2. Minor Customizations & Bug Fixes

Criteria: Upstream packages that need minor tweaks, hotfixes, or small adaptations for WordPress core compatibility.

Workflow: Installed via Composer using the composer-patches plugin. Patch files are version-controlled and stored in a top-level /patches directory within core.

Rationale: This isolates custom modifications into explicit diff files. When upgrading a vendor dependency, Composer automatically re-applies the patch files. In most cases, minor version updates upstream will patch cleanly without manual intervention, making dependency upgrades fully repeatable and transparent.

3. Unmaintained, Orphaned, or Heavily Custom Dependencies

Criteria: Upstream packages requiring heavy structural alterations, or packages whose original maintainers have abandoned the project.

Workflow: WordPress.org officially forks the repository under the @wordpress namespace (e.g., @wordpress/package-name). Core-specific patches and ongoing maintenance are handled in that fork, which is then pulled into core via Composer using the Tier 1 workflow.

Rationale: Attempting to patch unmaintained code via huge diff files leads to fragile build steps. Forking under the official @wordpress organizational umbrella provides a clean, predictable fallback for dead upstream projects while keeping trunk free of unstructured vendor edits.

Long-Term Sustainability & Ecosystem Collaboration
Beyond solving our immediate build and maintenance hurdles, established official forks under the @wordpress namespace provide a strategic path forward for abandoned libraries:

Future-Proofing Core: When critical upstream dependencies go unmaintained in the future, core isn't stuck on a dead branch or forced to execute rushed refactors. We have an established blueprint to step in and stabilize the dependency immediately.

Cross-Ecosystem Collaboration: Once an official fork exists, we can invite other open-source projects and communities who rely on the same legacy packages to collaborate on the fork. Sharing the maintenance burden with external contributors outside the immediate WordPress core committer pool keeps these libraries secure and active without placing the full load solely on core maintainers.

Summary of Benefits
Auditability: Storing explicit patch files in /patches makes it instantly obvious how core diverges from upstream libraries.

Maintainability: Significantly reduces the manual effort required during release cycles when updating vendor packages.

Sustainability: Gives the core team a clear governance model and safety net for orphaned libraries while creating opportunities for broader community co-maintenance.

[swissspidy]

I think it makes a lot of sense from a maintainability and auditability standpoint.

Workflow: Installed via Composer using the composer-patches plugin. Patch files are version-controlled and stored in a top-level /patches directory within core.

Downside: when someone requires wordpress/wordpress-develop via Composer, they then also need this plugin. IIRC there can be issues with using a patches directory, which is why ​https://github.com/ampproject/amp-wp/ for example uses remote patches (@westonruter correct me if I'm wrong).

Of course one would typically require wordpress/wordpress via Composer, not the development project, but people are definitely doing it.

Bottom line, this will require some testing.

[mmorris8]

Any thoughts on the implications of the plugin and theme system? If core supports composer for its own purposes and there's no support extended to plugins that becomes... Awkward.

[westonruter]

Replying to swissspidy:

IIRC there can be issues with using a patches directory, which is why ​https://github.com/ampproject/amp-wp/ for example uses remote patches

I recall this was just for the sake of maintainability. I wanted to keep the patches pulled in from PR URLs so that fixing merge conflicts could be done closer to the source of truth. Reasoning is found in ​https://github.com/ampproject/amp-wp/pull/5556

I do see it reference this issue with local patches though: ​https://github.com/cweagans/composer-patches/issues/315

This doesn't really make sense to me. If anything, a local patch should be more reliable. In any case, the issue was closed as completed, so probably something which isn't an issue anymore.

Of course one would typically require wordpress/wordpress via Composer, not the development project, but people are definitely doing it.

For wordpress/wordpress, this is a build artifact, no? In the same way it doesn't require JS building or npm install, it shouldn't require any composer install either. All composer dependencies should be pulled in and committed as they are presently, similar to how Gutenberg is pulled in as well.

[swissspidy]

For wordpress/wordpress, this is a build artifact, no

Yes, nothing would change for that. What I am saying is that people install wordpress-develop via Composer and the story is a bit different there. Not that it matters much though :)

[spacedmonkey]

As far as I know, WordPress/WordPress does not support installation as a composer package. That is an area the community has stepped in with projects like ​roots/wordpress-full or ​johnpbloch/wordpress-core-installer. If anyone is using wordpress/wordpress as a composer package, they are working in an unsupported way. But on the off chance they are doing this, a dev note would be useful IMO.

I am unclear on your feedback @mmorris8. To be clear, I don't think that this would effect installing themes and plugins in your own projects, unless, I have missed something.

[mmorris8]

Replying to spacedmonkey:

I am unclear on your feedback @mmorris8. To be clear, I don't think that this would effect installing themes and plugins in your own projects, unless, I have missed something.

Themes and plugings cannot use composer packages as intended without running the risk of a conflict with another theme or plugin. There is no central coordination available in WordPress as there is in Drupal.

Core using composer wouldn't affect themes and plugins, but it would be a massive missed opportunity to improve the application. And any future change that allows plugins and themes to use composer as intended would have to deal with whatever decisions are being made here.

[johnbillion]

This discussion is muddying the waters of this ticket. The purpose of this ticket is to facilitate switching some of the third party libraries in WordPress core to use Composer for their updates during development of WordPress itself. None of these changes will affect end users, plugins, or themes. There are other tickets for that topic.

[johnbillion]

My PR at ​https://github.com/WordPress/wordpress-develop/pull/10477 is in a pretty good place by the way. It just needs someone to pick it up and continue with it, as I don't have time at the moment.

[mmorris8]

Replying to johnbillion:

This discussion is muddying the waters of this ticket. The purpose of this ticket is to facilitate switching some of the third party libraries in WordPress core to use Composer for their updates during development of WordPress itself. None of these changes will affect end users, plugins, or themes. There are other tickets for that topic.

I started such a thread. I was pointed here and that thread was closed. So where do I post on this?