Make WordPress Core

Opened 22 hours ago

Last modified 22 hours ago

#65862 new defect (bug)

Build/Test Tools: Upgrade testing workflow spawns downgrade jobs when dispatched with an older version

Reported by: adrianmoldovanwp Owned by:
Priority: normal Milestone: Awaiting Review
Component: Build/Test Tools Version:
Severity: normal Keywords: has-patch
Cc: Focuses:

Description

The Upgrade Tests workflow (.github/workflows/upgrade-testing.yml) takes a new-version input on workflow_dispatch, but the starting versions in its five job matrices are static.
Dispatching an older version therefore spawns jobs that start from a newer one. With the current matrices, dispatching 6.6.4 spawns "6.9 to 6.6.4", "7.0 to 6.6.4" and "7.1-RC2 to 6.6.4".

Proposed fix: build each matrix's list of starting versions from the version being tested, dropping every version that is not older than it, and skip a job whose list ends up empty.

  • A major target (6.4 or 6.4.0) drops its own branch, since no upgrade occurs.
  • A minor target (6.4.4) keeps its own branch, since 6.4 to 6.4.4 is real.
  • latest and nightly drop nothing, so scheduled and CI runs are unchanged.

Change History (1)

This ticket was mentioned in PR #12972 on WordPress/wordpress-develop by @adrianmoldovanwp.


22 hours ago
#1

  • Keywords has-patch added

Dispatching this workflow with a specific version spawns jobs that start from a *newer* version. Dispatch 6.6.4 today and it spawns "6.9 to 6.6.4", "7.0 to 6.6.4" and "7.1-RC2 to 6.6.4".

wp core update declines to move backwards without --force, so those jobs pass having tested nothing.

Each matrix now takes its starting versions from a build-test-matrix job, which drops every version that is not older than the target. A job whose list ends up empty is skipped.

Dispatched Starting versions still tested
latest, nightly all of them, unchanged
7.1 all of them, including 7.1-RC2
7.0.1 up to 7.0, dropping the 7.1-RC2 pre-release
6.9.1 up to and including 6.9, since 6.9 to 6.9.1 is a real upgrade
6.9 up to 6.5, since a major target drops its own branch
6.6.4 up to 6.5

## Known limitation, unchanged by this PR

A minor target keeps its own branch, because 6.4 to 6.4.4 is a real upgrade. The called workflow runs wp core update --minor before the final upgrade step, so the minor update performs that upgrade and the final step is a no-op. When the target is not the newest release on its branch, the minor update overshoots the target and the target is never installed. Now documented in the workflow.

## Testing Instructions

CI on this PR covers the latest path, where nothing is dropped: it spawns 292 jobs across all 15 starting versions, matching trunk exactly.

The filtering itself only runs on workflow_dispatch. Dispatch Upgrade Tests from this branch and check the spawned jobs:

  1. latest — every job spawns, same as trunk today.
  2. 6.6.4 — "6.9 to 6.6.4", "7.0 to 6.6.4" and "7.1-RC2 to 6.6.4" are gone; Build Test Matrix reports recent-releases=[] and upgrade-tests-recent-releases skips.
  3. 6.9.1 — "6.9 to 6.9.1" still spawns, "7.0 to 6.9.1" does not.
  4. 7.0.1 — "7.1-RC2 to 7.0.1" does not spawn; "7.0 to 7.0.1" does.
  5. 4.6 — all five jobs skip, since every list empties.

Cases 2 and 5 have been run on a fork, with the github.repository gates pointed at it and the reusable callee swapped for a stub so each spawned job is trivial:

  • Case 2, target `6.6.4` — 125 succeeded, 1 skipped. Spawned exactly the 124 expected combinations from the four remaining matrices, across starting versions 4.7 through 6.5; upgrade-tests-recent-releases skipped.
  • Case 5, target `4.6` — all five jobs skipped, run concluded successfully.

Neither produced Matrix vector 'wp' does not contain any values, which confirms a job's if is evaluated before its strategy.matrix is expanded — the assumption the != '[]' guards rest on.

## Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5
Used for: Implementation, inline documentation, and iterative code review of the change. The approach — generating the version lists rather than generating matrix exclusions — was settled in discussion. I have reviewed the result and take responsibility for it.

Note: See TracTickets for help on using tickets.