Make WordPress Core

Opened 25 hours ago

Last modified 21 hours ago

#66744 new defect (bug)

Build/Test Tools: Release-day Installation and Upgrade Tests fail before the new zip reaches every download server

Reported by: adrianmoldovanwp Owned by:
Priority: normal Milestone: Awaiting Review
Component: Build/Test Tools Version:
Severity: normal Keywords: has-patch
Cc: Focuses: tests

Description

WordPress.org dispatches the Installation Tests and Upgrade Tests workflows seconds after it uploads a release zip. Every job downloads the zip itself, and the jobs fail:

Downloading update from https://wordpress.org/wordpress-5.0.32.zip...
Error: Download failed.

On 2026-11-18:

  • 5.6.24: Upgrade Tests failed 60 of 75 jobs (​run), and Installation Tests failed 42 jobs in each of two attempts (​run).
  • 5.0.32: Upgrade Tests failed attempts 1–4 (​run), and Installation Tests failed attempts 1–3 (​run). Both passed about 21 minutes after the upload.

Installation Tests already retry the download, but for only 30 seconds. The Upgrade Tests step does not retry. The error does not show the HTTP status.

  • Proposed fix: in each workflow, one job downloads the zip with WP-CLI, retries for up to 45 minutes, requires a verified md5, and gives the zip to every job. Each run downloads once instead of up to 75 times, and WP-CLI shows the HTTP status of each failed try. Runs for latest, nightly and develop do not change.

Other options considered:

  • Longer retry in each job: up to 75 jobs per run each wait up to 20 minutes.
  • Delay the dispatch, or add a job that only waits for the zip: one good download does not mean every server has the zip.
  • One job downloads with curl and shares the zip.
  • One job fills the WP-CLI cache and shares it: this depends on WP-CLI's internal cache layout, and a cache miss fails silently.

Change History (1)

This ticket was mentioned in ​PR #16106 on ​WordPress/wordpress-develop by ​@adrianmoldovanwp.


21 hours ago
#1

  • Keywords has-patch added

WordPress.org starts the Installation Tests and Upgrade Tests runs right after it uploads a release. For about 20 minutes, some download servers do not have the new zip yet, and every job that downloads it can fail.

Now one job in each workflow downloads the zip with WP-CLI. It tries every 60 seconds for up to 45 minutes and accepts the zip only when WP-CLI verifies its md5 hash. The test jobs then take the zip from that job instead of downloading it again. WP-CLI does not stop a downgrade when it updates from a zip, so Upgrade Tests skip the update when wp core update --minor already reached or passed the target. If the download job fails, the test jobs are skipped and the run reports a failure.

Runs for latest, nightly, trunk and develop keep the current steps. The start versions and wp core update --minor in Upgrade Tests also keep their current download.

Testing Instructions

These jobs only run in WordPress/wordpress-develop. To test on a fork, change the github.repository checks to the fork's name.

  1. Run Installation Tests and Upgrade Tests with a released version, for example 5.0.32. One job logs md5 hash verified, and all test jobs pass.
  2. Run them with a version that does not exist, for example 9.9.9. The download job fails after 45 minutes, and the test jobs are skipped.
  3. Run Installation Tests with latest and nightly, and Upgrade Tests with latest. No zip is shared, and the test jobs download WordPress as before.
  4. Run Upgrade Tests with an older patch, for example 6.4.3. The 6.4 jobs log is not older than 6.4.3, so there is nothing to upgrade.
  5. Run Installation Tests with 5.9 and 6.0. The post-install smoke check passes.

Test runs on a fork

Workflow Version What it proves Run
Installation Tests 5.0.32 One job downloads the zip and verifies its md5 hash. Every test job installs from that zip. ​see run
Installation Tests 9.9.9 A failed download skips the test jobs and fails the run. ​see run
Installation Tests latest No zip is shared. The test jobs download WordPress as before. ​see run
Installation Tests nightly No zip is shared. The test jobs download the nightly build as before. ​see run
Installation Tests 5.9 Long Twenty Twenty-Two file names survive unzip, so the smoke check passes (#66403). ​see run
Installation Tests 6.0 The same as 5.9. ​see run
Upgrade Tests 5.0.32 Every upgrade job updates from the shared zip. ​see run
Upgrade Tests 9.9.9 A failed download skips all upgrade jobs and fails the run. ​see run
Upgrade Tests latest No zip is shared. wp core update downloads WordPress as before. ​see run
Upgrade Tests 6.4.3 The jobs do not downgrade when wp core update --minor already passed the target. ​see run
Upgrade Develop Version Tests develop Callers that pass no zip, such as release branches, keep the current steps. ​see run

The 9.9.9 runs used a 2-minute wait instead of 45 minutes.

Use of AI Tools

AI assistance: Yes

Note: See TracTickets for help on using tickets.