wp_remote_get() cannot retrieve webcal URIs

[johnjamesjacoby]

In #31666, webcal was added to the list of allowed protocols. Unfortunately this does not bubble up into the HTTP API for remote requests, and wp_remote_get() on a webcal:// URI will fail with:

object(WP_Error)[532]
  public 'errors' => 
    array (size=1)
      'http_request_failed' => 
        array (size=1)
          0 => string 'A valid URL was not provided.' (length=29)
  public 'error_data' => 
    array (size=0)
      empty

Here is my proof-of-concept to show off the failure:

add_action( 'plugins_loaded', function() {

	// Public iCloud calendar I created
	$uri = 'webcal://p41-caldav.icloud.com/published/2/AAAAAAAAAAAAAAAAAAAAAF-eqSypTVlehAPwNTiPeHHBkTEvCi1qK6G4LDcU1Fr6AKLM-yaJrbRrhSSGMrjSbAxJZJ6TibzOCKLh0xBSpKI';

	// Regular remote get call
	$get = wp_remote_get( $uri );

	// Dump results
	var_dump( $get ); die;
} );

[johnjamesjacoby]

This bug exists because PHPs parse_url() function apparently does not consider webcal a valid scheme, even though WordPress does.

Inside WP_Http::request(), parse_url() is used on the webcal:// url, which does not return a scheme:

array (size=2)
  'host' => string 'p41-caldav.icloud.com' (length=21)
  'path' => string '/published/2/AAAAAAAAAAAAAAAAAAAAAF-eqSypTVlehAPwNTiPeHHBkTEvCi1qK6G4LDcU1Fr6AKLM-yaJrbRrhSSGMrjSbAxJZJ6TibzOCKLh0xBSpKI' (length=120)

$arrURL['scheme'] ends up being empty, and a WP_Error() is returned.

---

Oddly, if you attempt to use parse_url() with the PHP_URL_SCHEME flag, it will correctly identify the webcal scheme:

add_action( 'plugins_loaded', function() {

	// Public iCloud calendar I created
	$uri = 'webcal://p41-caldav.icloud.com/published/2/AAAAAAAAAAAAAAAAAAAAAF-eqSypTVlehAPwNTiPeHHBkTEvCi1qK6G4LDcU1Fr6AKLM-yaJrbRrhSSGMrjSbAxJZJ6TibzOCKLh0xBSpKI';

	// Regular remote get call
	$scheme = parse_url( $uri, PHP_URL_SCHEME );

	// Dump results
	var_dump( $scheme ); die;
} );

No doubt this is an error/oddity in PHP's implementation of parse_url(), but because WordPress made the decision to explicitly support it, I believe there is an obligation to follow through with that where PHP itself may be failing it.

[johnjamesjacoby]

For context & clarity, webcal schemes being supported inside of the_content is in no way directly connected to the HTTP API itself.

All that ticket 31666 exhibits is a willingness to explicitly support them in the WordPress project. I hope that will help justify further code changes to accommodate developers who want to use the recommended APIs to interact with remote webcal:// URIs.

[johnjamesjacoby]

If WordPress wanted to connect the HTTP API to its allowed protocols, the code would look something like:

		$scheme = parse_url( $url, PHP_URL_SCHEME );
		$allowed_protocols = wp_allowed_protocols();

		if ( empty( $url ) || ! in_array( $scheme, $allowed_protocols, true ) ) {

If it simply wanted to maintain essentially any scheme, it would look something like:

		$scheme = parse_url( $url, PHP_URL_SCHEME );

		if ( empty( $url ) || empty( $scheme ) ) {

---

PHP.net ​says:

This function is intended specifically for the purpose of parsing URLs and not URIs. However, to comply with PHP's backwards compatibility requirements it makes an exception for the ​file:// scheme where triple slashes (​file:///...) are allowed. For any other scheme this is invalid.

So... whether WordPress considers webcal:// a URL or a URI scheme may also be up for discussion.

I believe for the purposes laid out here it is a URI scheme that is intended to be allowed, making my first code change recommendation the most accurate one I can imagine at this time.

---

I don't have a core development checkout on this computer right now to make the patches myself, but I'll try to remember to circle back here once I do.

[johnjamesjacoby]

I just read the second section of the PHP docs for WP_Http::request() which says:

	 * Send an HTTP request to a URI.
	 *
	 * Please note: The only URI that are supported in the HTTP Transport implementation
	 * are the HTTP and HTTPS protocols.

So, even though it works for URIs, it only works for HTTP and HTTPS, which really stinks.

---

Perhaps there is an opportunity to introduce a filter here, allowing plugins to use this API with their own unsupported protocols and at their own risk.