WordPress.org

Make WordPress Core

Opened 11 months ago

Last modified 9 months ago

#37820 new defect (bug)

wp_remote_get referrer not being sent correctly

Reported by: schrapel Owned by:
Milestone: Awaiting Review Priority: normal
Severity: normal Version: 4.6
Component: HTTP API Keywords: has-patch upstream
Focuses: Cc:

Description

We use an API that would check the referrer url. In the process of upgrading to 4.6 their API now responds telling us an invalid referrer was provided. If I roll back to 4.5.3 and run wp_remote_get it works without problems.

Attachments (4)

patch_37820.diff (662 bytes) - added by tristangemus 10 months ago.
Screen Shot 2016-10-03 at 3.32.05 PM.png (41.0 KB) - added by tristangemus 10 months ago.
Screen Shot 2016-10-03 at 3.30.41 PM.png (47.4 KB) - added by tristangemus 10 months ago.
referer.png (14.0 KB) - added by differentthemes 10 months ago.
Referer header

Download all attachments as: .zip

Change History (18)

#1 @swissspidy
11 months ago

Hey there, thanks for your report!

What kind of arguments are you passing to wp_remote_get()? Some sample code to look at would help greatly.

Looking at the code from 4.5 and 4.6, no referer was ever set by WordPress in 4.5. Now with 4.6, Requests seems to set the referer to the URL being requested:

curl_setopt($this->handle, CURLOPT_URL, $url);
curl_setopt($this->handle, CURLOPT_REFERER, $url);

Citing RFC 2616:

The Referer[sic] request-header field allows the client to specify, for the server's benefit, the address (URI) of the resource from which the Request-URI was obtained

Seems legit in that case.

#2 @schrapel
11 months ago

Why would the CURLOPT_URL be the same as the CURLOPT_REFERER? If I manually set CURLOPT_REFERER to the URL of the domain I am sending from then I get the response I desire.

The code is below but I obviously removed our API key. I'm going to try get another API key for you that I can share here

<?php
wp_remote_get('http://api.embed.ly/1/feature?feature=oembed&key=***')

#3 @swissspidy
11 months ago

Why would the CURLOPT_URL be the same as the CURLOPT_REFERER?

I don't know why it was set that way. It has been like this in Requests ever since, see https://github.com/rmccue/Requests/commit/78d4f3c117642445cf729a7eaaaa87d8d3897fd6.

There's an open issue on GitHub, so maybe @rmccue can chime in.

#4 @schrapel
11 months ago

I had a look at Guzzle and they don't seem to do this. Hopefully @rmccue can offer his opinion on this

#5 @jeremyfelt
11 months ago

#37888 was marked as a duplicate.

#6 @differentthemes
11 months ago

Any news here? Will it be fixed in next updates as a additional argument?

#7 @swissspidy
11 months ago

@differentthemes As Requests is an external library, this requires some upstream changes first. You can follow https://github.com/rmccue/Requests/issues/232 for updates on that.

#8 @differentthemes
10 months ago

A month has gone and still don't see any changes here :(

#9 follow-up: @tristangemus
10 months ago

  • Keywords has-patch added

Wordpress 4.5.4 did not include the referrer at all by default. The referrer should be the root URL of the referring website, site_url should be an adequate solution to this. See patch.

#10 @dd32
10 months ago

  • Keywords upstream added

Small note, You can send a Referer header along with your wp_remote_get() which will cause cURL not to send the Referer.
Unfortunately, the Streams handler will send a Referer: header anyway.

This should be fixed upstream at https://github.com/rmccue/Requests/issues/232 - inputs and patches would be welcome there.

#11 in reply to: ↑ 9 ; follow-up: @differentthemes
10 months ago

Replying to tristangemus:

Wordpress 4.5.4 did not include the referrer at all by default. The referrer should be the root URL of the referring website, site_url should be an adequate solution to this. See patch.

Thanks! That helps, but only we can't now force all our customers to edit they wp files. This really should be fixed ASAP at least as a new parameter, if not even by default.

#12 in reply to: ↑ 11 @tristangemus
10 months ago

Replying to differentthemes:

Replying to tristangemus:

Wordpress 4.5.4 did not include the referrer at all by default. The referrer should be the root URL of the referring website, site_url should be an adequate solution to this. See patch.

Thanks! That helps, but only we can't now force all our customers to edit they wp files. This really should be fixed ASAP at least as a new parameter, if not even by default.

Considering this was selected as a solution, it could be part of an upcoming Wordpress release. Hoping to get some support on this or another solution as the referrer being set as the destination URL does not make any sense at all.

Another solution is to not include the referrer and make it an option at all as it is optional in HTTP.

I'd also look into this solution - https://core.trac.wordpress.org/ticket/37820#comment:10

Last edited 10 months ago by tristangemus (previous) (diff)

@differentthemes
10 months ago

Referer header

#13 @differentthemes
10 months ago

Not quite sure if I get it right, but like this it doesn't work.

Edited: Sorry missed this part

$referer = site_url();
Last edited 10 months ago by differentthemes (previous) (diff)

#14 @differentthemes
9 months ago

I have checked WordPress 4.7 BETA 2, but seems like the changes wan't been made there..

Note: See TracTickets for help on using tickets.