Make WordPress Core

Opened 5 weeks ago

Closed 11 days ago

Last modified 11 days ago

#65862 closed defect (bug) (fixed)

Build/Test Tools: Upgrade testing workflow spawns downgrade jobs when dispatched with an older version

Reported by: adrianmoldovanwp Owned by: lancewillett
Priority: normal Milestone: 7.2
Component: Build/Test Tools Version:
Severity: normal Keywords: has-patch
Cc: Focuses:

Description

The Upgrade Tests workflow (.github/workflows/upgrade-testing.yml) takes a new-version input on workflow_dispatch, but the starting versions in its five job matrices are static.
Dispatching an older version therefore spawns jobs that start from a newer one. With the current matrices, dispatching 6.6.4 spawns "6.9 to 6.6.4", "7.0 to 6.6.4" and "7.1-RC2 to 6.6.4".

Proposed fix: build each matrix's list of starting versions from the version being tested, dropping every version that is not older than it, and skip a job whose list ends up empty.

  • A major target (6.4 or 6.4.0) drops its own branch, since no upgrade occurs.
  • A minor target (6.4.4) keeps its own branch, since 6.4 to 6.4.4 is real.
  • latest and nightly drop nothing, so scheduled and CI runs are unchanged.

Change History (4)

This ticket was mentioned in PR #12972 on WordPress/wordpress-develop by @adrianmoldovanwp.


5 weeks ago
#1

  • Keywords has-patch added

Dispatching this workflow with a specific version spawns jobs that start from a *newer* version. Dispatch 6.6.4 today and it spawns "6.9 to 6.6.4", "7.0 to 6.6.4" and "7.1-RC2 to 6.6.4".

wp core update declines to move backwards without --force, so those jobs pass having tested nothing.

Each matrix now takes its starting versions from a build-test-matrix job, which drops every version that is not older than the target. A job whose list ends up empty is skipped.

Dispatched Starting versions still tested
latest, nightly all of them, unchanged
7.1 all of them, including 7.1-RC2
7.0.1 up to 7.0, dropping the 7.1-RC2 pre-release
6.9.1 up to and including 6.9, since 6.9 to 6.9.1 is a real upgrade
6.9 up to 6.5, since a major target drops its own branch
6.6.4 up to 6.5

## Known limitation, unchanged by this PR

A minor target keeps its own branch, because 6.4 to 6.4.4 is a real upgrade. The called workflow runs wp core update --minor before the final upgrade step, so the minor update performs that upgrade and the final step is a no-op. When the target is not the newest release on its branch, the minor update overshoots the target and the target is never installed. Now documented in the workflow.

## Testing Instructions

CI on this PR covers the latest path, where nothing is dropped: it spawns 292 jobs across all 15 starting versions, matching trunk exactly.

The filtering itself only runs on workflow_dispatch. Dispatch Upgrade Tests from this branch and check the spawned jobs:

  1. latest — every job spawns, same as trunk today.
  2. 6.6.4 — "6.9 to 6.6.4", "7.0 to 6.6.4" and "7.1-RC2 to 6.6.4" are gone; Build Test Matrix reports recent-releases=[] and upgrade-tests-recent-releases skips.
  3. 6.9.1 — "6.9 to 6.9.1" still spawns, "7.0 to 6.9.1" does not.
  4. 7.0.1 — "7.1-RC2 to 7.0.1" does not spawn; "7.0 to 7.0.1" does.
  5. 4.6 — all five jobs skip, since every list empties.

Cases 2 and 5 have been run on a fork, with the github.repository gates pointed at it and the reusable callee swapped for a stub so each spawned job is trivial:

  • Case 2, target `6.6.4` — 125 succeeded, 1 skipped. Spawned exactly the 124 expected combinations from the four remaining matrices, across starting versions 4.7 through 6.5; upgrade-tests-recent-releases skipped.
  • Case 5, target `4.6` — all five jobs skipped, run concluded successfully.

Neither produced Matrix vector 'wp' does not contain any values, which confirms a job's if is evaluated before its strategy.matrix is expanded — the assumption the != '[]' guards rest on.

## Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5
Used for: Implementation, inline documentation, and iterative code review of the change. The approach — generating the version lists rather than generating matrix exclusions — was settled in discussion. I have reviewed the result and take responsibility for it.

#2 @lancewillett
11 days ago

  • Owner set to lancewillett
  • Status newaccepted

#3 @lancewillett
11 days ago

  • Resolutionfixed
  • Status acceptedclosed

In 63437:

Build/Test Tools: Stop spawning downgrade test jobs.

Manually dispatching the upgrade testing workflow with an older target can spawn jobs that start from newer WordPress versions. WP-CLI rejects those downgrades, so the jobs pass without testing an upgrade.

Build each matrix from starting versions older than the target, and skip jobs whose matrix becomes empty. Scheduled and pull request runs remain unchanged.

Developed in: https://github.com/WordPress/wordpress-develop/pull/12972

Props adrianmoldovanwp.
Fixes #65862.

#4 @lancewillett
11 days ago

  • Milestone Awaiting Review7.2
Note: See TracTickets for help on using tickets.