Make WordPress Core

Opened 22 hours ago

Last modified 13 hours ago

#65736 reopened task (blessed)

Trim the PHPUnit test matrix to boundary PHP versions, with full coverage on the weekly run

Reported by: lancewillett Owned by: lancewillett
Priority: normal Milestone: 7.1
Component: Build/Test Tools Version:
Severity: normal Keywords: has-patch
Cc: Focuses:

Description (last modified by lancewillett)

#64083 trimmed the GitHub Actions workflows on older branches, reducing the PHPUnit matrix to the highest and lowest supported version of each major PHP branch. Its title noted "(and possibly trunk)". This completes that work: it applies the boundary trim to trunk and pull requests, and to the current release branch.

The matrix drops from seven PHP minors to three boundary versions (7.4, 8.0, 8.5) across the three test jobs. Version-specific failures cluster at the boundaries of each major, so the intermediate minors add job count without adding coverage. Base PHPUnit combinations drop from ~168 to ~72 per run.

To keep full coverage, the existing weekly scheduled run (Sundays, 00:00 UTC) tests every supported PHP version. Push and pull request events test the boundary versions. Scheduled runs fire from the default branch, so the weekly full sweep covers trunk.

The immediate benefit is at release time. When many backports land on the current release branch together, the full matrix floods the shared runner pool. Tests get cancelled, or the release ships before they finish.

A follow-up could trim the database axis on the same principle. That would be filed separately.

Change History (11)

This ticket was mentioned in PR #12719 on WordPress/wordpress-develop by @lancewillett.


22 hours ago
#1

## What

Trims the PHPUnit test matrix on the active branch to boundary PHP versions: the highest and lowest supported version of each major. 7.4, 8.0, and 8.5 replace the full seven-minor list, in test-with-mysql, test-with-mariadb, and test-innovation-releases.

## Why

This mirrors the trim #64083 applied to older branches, whose title named "(and possibly trunk)". Version-specific failures cluster at the boundaries of each major, so the intermediate minors add job count without adding coverage. At release time the full matrix floods the shared runner pool, forcing tests to be cancelled or the release to ship early.

## Impact

Base PHPUnit combinations drop from ~168 to ~72, roughly a 55% cut, before the unchanged include: jobs.

## Scope

PHP axis only. Database versions, multisite, memcached, and the single-representative include: jobs are untouched. Highest 8.x is 8.5; happy to use 8.4 if reviewers prefer current-stable. A database-axis trim on the same principle is a possible follow-up, filed separately.

Trac ticket: Core-65736

## Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Opus 4.8
Used for: Identifying the boundary-trim approach from the existing older-branch precedent, drafting the matrix change and ticket text. Verified manually.

#2 @lancewillett
22 hours ago

  • Description modified (diff)
  • Summary Trim the active branch GitHub Actions test matrix to boundary PHP versionsTrim the PHPUnit test matrix to boundary PHP versions, with full coverage on the weekly run

Broadened the scope to cover trunk and pull requests, with the existing weekly scheduled run switched to the full matrix so coverage is preserved. The current release branch gets the same trim in a separate change.

This ticket was mentioned in PR #12720 on WordPress/wordpress-develop by @lancewillett.


22 hours ago
#3

## What

Trims the PHPUnit test matrix on the 7.0 branch to boundary PHP versions: 7.4, 8.0, and 8.5 in place of the full seven-minor list, across test-with-mysql, test-with-mariadb, and test-innovation-releases.

## Why

This applies to the 7.0 branch the same boundary trim #64083 used on older branches (Trac #65736 tracks extending it to the current branches). The immediate benefit is at release time. When many 7.0.x backports land together, the full matrix floods the shared runner pool. Tests get cancelled, or the release ships before they finish.

## Impact

Base PHPUnit combinations drop from ~168 to ~72 per run, roughly a 55% cut, before the unchanged include: jobs.

## Scope

PHP axis only. The 7.0 branch does not run the weekly scheduled full matrix. Scheduled workflows fire only from the default branch. So this is a plain trim, not the conditional used on trunk. Database versions, multisite, and memcached are untouched.

Trac ticket: Core-65736

## Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Opus 4.8
Used for: Applying the boundary-trim approach from the existing older-branch precedent. Verified manually.

@lancewillett commented on PR #12719:


22 hours ago
#4

@johnbillion Agree on the tradeoff. If we could make all runs faster, maybe we can keep the full matrix.

Another idea is to allow a PR tag full-ci to kick off the full, ad-hoc.

Apologies, I made a mistake in the first commit. Can you re-review 2538d740ae65f9439c269a91244d75a470276fcd ? Adds the weekly back.

#5 @johnbillion
17 hours ago

In 62856:

Build/Test Tools: Trim the PHPUnit matrix to boundary PHP versions in the 7.0 branch.

This reduces the number of jobs that get triggered for each push or PR to the branch while retaining testing on the highest and lowest supported versions of PHP.

Developed in https://github.com/WordPress/wordpress-develop/pull/12720

Props lancewillett, adrianmoldovanwp

See #65736, #64083

#7 @johnbillion
17 hours ago

  • Milestone Awaiting Review7.1
  • Type enhancementtask (blessed)

#8 @lancewillett
17 hours ago

  • Owner set to lancewillett
  • Resolutionfixed
  • Status newclosed

In 62858:

Build/Test Tools: Trim the PHPUnit matrix to boundary PHP versions.

Test only the highest and lowest supported PHP version of each major (7.4, 8.0, 8.5) on push and pull request events; the weekly scheduled run still exercises every supported version.

Version-specific failures cluster at the boundaries of each major, so the intermediate minors add job count without adding coverage. Cuts base PHPUnit combinations from roughly 168 to 72 per CI run for both time and cost savings, and a much better developer experience in core GitHub repos.

Developed in: https://github.com/WordPress/wordpress-develop/pull/12719

Props johnbillion, adrianmoldovanwp
Fixes #65736, see #64083

#9 @lancewillett
14 hours ago

  • Resolution fixed
  • Status closedreopened

#10 @lancewillett
14 hours ago

In 62863:

Build/Test Tools: Trim the PHPUnit matrix to boundary PHP versions in the 6.8 branch.

The 6.8 PHPUnit matrix runs the full PHP axis (7.2 through 8.4) across three job groups, about 198 matrix jobs, while trunk and 7.0 were trimmed to boundary versions. During a security release several branches run at once, and 6.8's full matrix is the single largest contributor to the concurrent-job peak. Trim the PHP axis to the highest and lowest of each major (7.2, 7.4, 8.0, 8.4), matching trunk and 7.0. The intermediate versions still run on trunk and the weekly scheduled full matrix, so coverage holds.

Developed in: https://github.com/WordPress/wordpress-develop/pull/12726

See #65736, see #64083

@lancewillett commented on PR #12720:


13 hours ago
#11

See also: https://github.com/WordPress/wordpress-develop/pull/12726

6.8 runs were still peaking at ~198, this takes that version's runs down to ~99.

Note: See TracTickets for help on using tickets.