wp_remote_post not respecting timeout & blocking params

[mihai200]

Tried on an fresh WordPress 6.7.2 install following to prove my point and i get 0.4-0.6 seconds:

$url = "https://ipify.org";

$args = [
      'timeout' => 0.1,
      'blocking' => false,
      'sslverify' => false,
];

$start = microtime(true);
$response = wp_remote_post($url, $args);
$end = microtime(true);
$exec_time = $end - $start;

I ve made a simple script to test the issue :
​https://gist.github.com/misu200/7bc269af5dbf8b755059ad5822a7a970

[TimothyBlynJacobs]

Dupe of ​https://github.com/WordPress/Requests/issues/826

[westonruter]

I'm re-opening this for us to track on the Performance team.

Performance could be negatively impacted because ​WP Cron is spawned with these args:

'timeout'   => 0.01,
'blocking'  => false,

Since wp_cron() runs at the init action, which then schedules _wp_cron() to run at the wp_loaded action, meaning that if the loopback request takes longer than expected, every page load that spawns crawn will have a bad TTFB.

In particular, this ​comment in wp-cron.php file header is false:

The HTTP request to this file will not slow down the visitor who happens to visit when a scheduled cron event runs.

---

What follows is what I had drafted before @TimothyBlynJacobs replied noting the existing issue in Requests.

I created a ​Sleepy Responses plugin to help test this, where a response can be slowed down by 5 seconds by adding ?sleep=5 to the URL.

I adapted your WP-CLI test script as follows (e.g. try-http-api.php):

<?php
$args = array(
        'timeout'   => 0.01,
        'blocking'  => false,
        'sslverify' => false,
);

$url = site_url( '/wp-cron.php?doing_wp_cron' );
global $argv;
if ( isset( $argv[3] ) ) {
        if ( str_starts_with( $argv[3], 'http://' ) || str_starts_with( $argv[3], 'https://' ) ) {
                $url = $argv[3];
        } elseif ( str_starts_with( $argv[3], '/' ) ) {
                $url = site_url( $argv[3] );
        } else {
                WP_CLI::error( sprintf( 'Bad URL: %s', $argv[3] ) );
        }
}

$start     = microtime( true );
$response  = wp_remote_post( $url, $args );
$end       = microtime( true );
$exec_time = $end - $start;

WP_CLI::log( sprintf( 'Requested URL: %s', $url ) );
WP_CLI::log( sprintf( 'Execution time: %s seconds', round( $exec_time, 4 ) ) );
if ( is_wp_error( $response ) ) {
        WP_CLI::error( sprintf( '%s (%s)', $response->get_error_message(), $response->get_error_code() ) );
} elseif ( wp_remote_retrieve_response_code( $response ) ) {
        WP_CLI::error( sprintf( 'Expected non-blocking request to return empty response code but got %s', wp_remote_retrieve_response_code( $response ) ) );
} elseif ( $end - $start > 0.1 ) {
        WP_CLI::error( 'Response took unexpected amount of time for non-blocking request.' );
} else {
        WP_CLI::log( 'OK ✅' );
}

Here's what I'm seeing:

$ wp eval-file try-http-api.php '/?sleep=3' 
Requested URL: http://localhost:8000/?sleep=3
Execution time: 1.0174 seconds
Error: Response took unexpected amount of time for non-blocking request.

$ wp eval-file try-http-api.php 'https://wordpress.org/' 
Requested URL: https://wordpress.org/
Execution time: 0.5602 seconds
Error: Response took unexpected amount of time for non-blocking request.

$ wp eval-file try-http-api.php '/wp-cron.php?doing_wp_cron&sleep=3' 
Requested URL: http://localhost:8000/wp-cron.php?doing_wp_cron
Execution time: 1.0125 seconds
Error: Response took unexpected amount of time for non-blocking request.

[westonruter]

If I patch wp-cron.php to add sleep( 3 ); at the top, and then I schedule an immediate task via WP-CLI:

wp cron event schedule foo

Then if I load the the site in the browser with Query Monitor running, I can see that this Non-Blocking request took over 1 second. What's more is that it did so even though it has a 0.01 second timeout. This caused the response time to be 1.21 seconds instead of 192 ms when no cron is spawned.

[westonruter]

I tried creating a plugin that sets the CURLOPT_NOSIGNAL option (as referenced in ​Requests#826) on cURL for requests to spawn WP Cron:

<?php
/**
 * Plugin Name: WP Cron Perf Fix
 */

namespace WPCronPerfFix;

use CurlHandle;

/**
 * Sets NOSIGNAL option on requests to spawn WP Cron.
 *
 * @param CurlHandle|resource $handle      The cURL handle returned by curl_init() (passed by reference).
 * @param array               $parsed_args The HTTP request arguments.
 * @param string              $url         The request URL.
 */
function add_nosignal_opt( $handle, array $parsed_args, string $url ) {
        $parsed_url = wp_parse_url( $url );
        if (
                isset( $parsed_url['host'], $parsed_url['path'], $parsed_args['blocking'] ) &&
                wp_parse_url( site_url(), PHP_URL_HOST ) === $parsed_url['host'] &&
                'wp-cron.php' === basename( $parsed_url['path'] ) &&
                ! $parsed_args['blocking']
        ) {
                curl_setopt( $handle, CURLOPT_NOSIGNAL, 1 );
        }
}

add_action( 'http_api_curl', __NAMESPACE__ . '\add_nosignal_opt',  10, 3 );

However, it is not improving the response time.

To test, I've activated that plugin and I use WP-CLI to schedule a task and then immediately time the request the frontend:

npm run env:cli cron event schedule foo$(date +%s.%N) && time curl http://localhost:8889/ > /dev/null

The result is 0m1.177s.

When I try making a request without the cron scheduling, the transpired time is 0m0.150s.

So spawning WP Cron is still adding an entire second to the response.

[westonruter]

Another workaround could be to spawn WP Cron at shutdown instead of at wp_loaded. I'm not sure why wp_loaded was chosen to begin with:

<?php
/**
 * Plugin Name: WP Cron Perf Fix
 */

namespace WPCronPerfFix;

use CurlHandle;

// Facilitate testing the performance impact by disabling with a query parameter.
if ( isset( $_GET['disable_wp_cron_perf_fix'] ) || defined( 'WP_CLI' ) ) {
        return;
}

/**
 * Spawns WP Cron at shutdown.
 */
function spawn_wp_cron_at_shutdown() {
        // Make sure response is flushed so the TTFB hopefully will get marked here even if the following finish requests aren't available.
        flush();

        // End the response, as is done in wp-cron.php.
        if ( function_exists( 'fastcgi_finish_request' ) ) {
                fastcgi_finish_request();
        } elseif ( function_exists( 'litespeed_finish_request' ) ) {
                litespeed_finish_request();
        }

        _wp_cron();
}

// Move WP Cron from being spawned at wp_loaded to happen at shutdown instead.
if ( has_action( 'init', 'wp_cron' ) ) {
        remove_action( 'init', 'wp_cron' );
        add_action( 'shutdown', __NAMESPACE__ . '\spawn_wp_cron_at_shutdown' );
}

[dkarfa]

Welcome to WordPress Core, @mihai200! 🎉

Thank you for submitting your first ticket to WordPress Core! I'm excited to see you join our community of contributors who help make WordPress better for millions of users worldwide.

Your contribution is valuable, and I appreciate the time and effort you've put into identifying and documenting this issue.

First tickets are always special milestones; yours shows excellent attention to detail.

What happens next?

The community team will review the ticket. Please don't hesitate to ask questions if you need clarification on anything during the process. The WordPress community is here to help and guide you through your contribution journey.

Keep contributing!

I hope this is just the beginning of your involvement with WordPress Core. Whether it's reporting bugs, suggesting enhancements, testing patches, or writing code, every contribution helps make WordPress stronger.

Welcome aboard, and thank you for helping improve WordPress!

[siliconforks]

It seems like the issue here is that the timeout is not allowed to be less than one second - if the requested timeout is less than that, it is increased to one second anyway:

​https://github.com/WordPress/wordpress-develop/blob/7d5fb7b4a98a4ac02ef0d7b354f11b1396363d51/src/wp-includes/Requests/src/Transport/Curl.php#L427

From the preceding comment it appears that this is done to work around a problem with cURL when using the system DNS resolver. However, it also says that "There's no way to detect which DNS resolver is being used" - but this does not seem to be true any more since PHP 7.3 was released with the ​CURL_VERSION_ASYNCHDNS constant. So I think it would be possible to allow timeouts of less than one second for systems with CURL_VERSION_ASYNCHDNS.

Someone actually suggested this a long time ago - ​https://github.com/WordPress/Requests/issues/264 - but then it appears that the conversation wandered off into other topics and the original suggestion was never implemented...

[b1ink0]

diff --git a/src/wp-includes/Requests/src/Transport/Curl.php b/src/wp-includes/Requests/src/Transport/Curl.php
index 7316987b5f..1537014d91 100644
--- a/src/wp-includes/Requests/src/Transport/Curl.php
+++ b/src/wp-includes/Requests/src/Transport/Curl.php
@@ -424,7 +424,7 @@ final class Curl implements Transport {
 		// end, so we need to round up regardless of the supplied timeout.
 		//
 		// https://github.com/curl/curl/blob/4f45240bc84a9aa648c8f7243be7b79e9f9323a5/lib/hostip.c#L606-L609
-		$timeout = max($options['timeout'], 1);
+		$timeout = max($options['timeout'], defined( 'CURL_VERSION_ASYNCHDNS' ) ? 0 : 1 );
 
 		if (is_int($timeout) || $this->version < self::CURL_7_16_2) {
 			curl_setopt($this->handle, CURLOPT_TIMEOUT, ceil($timeout));

After trying @siliconforks' suggestion about using ​https://php.watch/codex/CURL_VERSION_ASYNCHDNS, it seems to fix the issue. Here is the result of running the test script from https://core.trac.wordpress.org/ticket/63547#comment:3.

$ npm run env:cli -- eval-file src/wp-content/plugins/hello.php '/?sleep=3'

> WordPress@6.9.0 env:cli
> node ./tools/local-env/scripts/docker.js run --rm cli eval-file src/wp-content/plugins/hello.php /?sleep=3

[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                               0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                               0.0s 
Requested URL: http://localhost:8889/?sleep=3
Execution time: 0.0316 seconds
OK ✅

[siliconforks]

Replying to b1ink0:

After trying @siliconforks' suggestion about using ​https://php.watch/codex/CURL_VERSION_ASYNCHDNS, it seems to fix the issue. Here is the result of running the test script from https://core.trac.wordpress.org/ticket/63547#comment:3.

Note that the actual fix would need to be a bit more complicated than that, since you would need to call curl_version() and check whether it contains the CURL_VERSION_ASYNCHDNS value.

I've proposed a patch to the upstream project here: ​https://github.com/WordPress/Requests/pull/932