Make WordPress Core

Opened 8 months ago

Last modified 3 weeks ago

#59486 new enhancement

Make test suite compatible with PHPUnit 10 / 11

Reported by: dingo_d's profile dingo_d Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version: 6.4
Component: Build/Test Tools Keywords: needs-patch
Focuses: Cc:

Description

PHPUnit 10 was released back in February of 2023, and included many breaking changes - they completely rewrote the internal functionality of PHPUnit (mainly the event system for extending the test runner). Besides that, they upped the minimum PHP version to >=8.1, so for WordPress, it could only be used on PHP versions of >=8.1.

While this is not a huge and critical update, since the tests are running fine on PHP versions all the way up to version 8.3, it would be a good idea to start preparing for PHPUnit 10 compatibility.

Besides being up to date, it will allow external testing library maintainers (such as wp-browser or wp-pest), to use the latest versions of both PHPUnit, or dependencies that are using PHPUnit 10 (such as PestPHP v2 which depends on PHPUnit 10).

I understand that this is a huge undertaking, but it's okay to have a ticket open for this (I didn't find one opened while searching, could be wrong tho), so that we can keep track of changes, and maybe create a roadmap or a to-do list of what needs to be fixed in order to make WordPress test suite compatible with PHPUnit 10.

Change History (10)

#1 @desrosj
5 months ago

  • Keywords needs-patch added; dev-feedback needs-testing removed
  • Milestone changed from Awaiting Review to Future Release

I went through and made a preliminary list by looking at the removed section within the 10.0 changelog and searching the test directory for occurrences of removed elements. The following have been removed and are found in the test directory:

  • withConsecutive() (details)
  • MockBuilder::setMethods() and MockBuilder::setMethodsExcept() (details)
  • expectDeprecation(), expectDeprecationMessage(), and expectDeprecationMessageMatches() methods
  • expectError(), expectErrorMessage(), and expectErrorMessageMatches() methods
  • expectNotice(), expectNoticeMessage(), and expectNoticeMessageMatches() methods
  • expectWarning(), expectWarningMessage(), and expectWarningMessageMatches() methods

This is by no means an all inclusive list. But wanted to provide a starting point if someone wanted to try and tackle this.

To run the test suite, the yoast/phpunit-polyfills version constraint needs to be updated to ^2.0.0 in the composer.json file.

#2 @ironprogrammer
4 months ago

To provide more detail to the list from comment:1, here are the deprecated expect*() warnings to increase discoverability of this ticket (details):

Expecting E_DEPRECATED and E_USER_DEPRECATED is deprecated and will no longer be possible in PHPUnit 10.
Expecting E_ERROR and E_USER_ERROR is deprecated and will no longer be possible in PHPUnit 10.
Expecting E_STRICT, E_NOTICE, and E_USER_NOTICE is deprecated and will no longer be possible in PHPUnit 10.
Expecting E_WARNING and E_USER_WARNING is deprecated and will no longer be possible in PHPUnit 10.

Props @azaozz for pointing this out.

#3 @desrosj
4 months ago

Related: #60305.

#4 @swissspidy
3 months ago

  • Summary changed from Make test suite compatible with PHPUnit 10 to Make test suite compatible with PHPUnit 10 / 11

#5 @swissspidy
3 months ago

#60551 was marked as a duplicate.

#6 follow-up: @jujuguru
3 months ago

Adding my experience here with a workaround.

WP_UnitTestCase_Base::expectDeprecated makes a call to \PHPUnit\Util\Test::parseTestMethodAnnotations during test setup and renders testing useless.

My simple workaround
I've commented the call to parseTestMethodAnnotations out in the file abstract-testcase.php and it has allowed test to complete.

I will caveat that I'm only early doors with testing so other issues may arise. If they do and I figure out workarounds I'll share.

Current set up:
PHP 8.2.15
PHPUnit 11.0.3
WordPress 6.4.3
Yoast Polyfills ^2.0

#7 @jrf
3 months ago

Just found this ticket. There is much bigger problem we have to deal with before this could be contemplated (complete reorganization of the test suite). I don't think adding my analysis in a comment in this ticket is the right way forward, so I propose I open a new ticket with a detailed analysis and roadmap to replace this ticket.

#8 in reply to: ↑ 6 @chesio
3 months ago

Replying to jujuguru:

My simple workaround
I've commented the call to parseTestMethodAnnotations out in the file abstract-testcase.php and it has allowed test to complete.

Thanks for the hint, I hit the issue too while running integration tests of one of my plugins. I use a slightly more flexible workaround:

<?php

namespace My\Plugin\Tests\Integration;

/**
 * Base class for all integration tests
 */
abstract class TestCase extends \WP_UnitTestCase
{
    /**
     * @internal Workaround to allow the tests to run on PHPUnit 10.
     *
     * @link https://core.trac.wordpress.org/ticket/59486
     */
    public function expectDeprecated(): void
    {
        return;
    }
}

This way WordPress test suite can be kept up to date without further edits.

Last edited 3 months ago by chesio (previous) (diff)

This ticket was mentioned in Slack in #core-test by ironprogrammer. View the logs.


3 weeks ago

This ticket was mentioned in Slack in #hosting by javier. View the logs.


3 weeks ago

Note: See TracTickets for help on using tickets.