Make WordPress Core

Opened 3 weeks ago

Closed 12 days ago

#62830 closed enhancement (fixed)

Add a retry mechanism for tests that perform external HTTP requests

Reported by: johnbillion's profile johnbillion Owned by: johnbillion's profile johnbillion
Milestone: 6.8 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch has-unit-tests
Focuses: Cc:

Description

PHPUnit tests that perform an external HTTP request are the worst. They rely on network connectivity and the behaviour of an external resource remaining stable. Although such tests mostly get skipped now if the request times out, other factors can still cause them to fail occasionally.

While it would be great to eliminate most external HTTP requests in the PHPUnit tests, that's potentially quite a bit of work. I am proposing a simple wrapper for HTTP API requests in the tests which retries the request up to three times with a 5 second sleep between them. This reduces the chance that a temporary network connectivity problem will cause an otherwise passing test suite to fail.

This is not meant to be a comprehensive solution to HTTP request retries, and I don't want to propose adding such behaviour to core (at the moment anyway). Rather this is a way of increasing the reliability of the test suite.

Change History (2)

This ticket was mentioned in PR #8146 on WordPress/wordpress-develop by @johnbillion.


3 weeks ago
#1

  • Keywords has-patch has-unit-tests added

#2 @johnbillion
12 days ago

  • Owner set to johnbillion
  • Resolution set to fixed
  • Status changed from new to closed

In 59729:

Build/Test Tools: Add a retry mechanism for tests that perform external HTTP requests.

While the skipTestOnTimeout() method will catch a timeout and prevent it from causing a test to fail, other errors such as a failed DNS lookup or HTTPS handshake can still cause a test to unnecessarily fail. This introduces a simple retry mechanism that will hopefully further reduce the flakiness of tests that perform HTTP API requests.

Fixes #62830

Note: See TracTickets for help on using tickets.