#37820 closed defect (bug) (reported-upstream)
wp_remote_get referrer not being sent correctly
Reported by: | schrapel | Owned by: | |
---|---|---|---|
Milestone: | Priority: | normal | |
Severity: | normal | Version: | 4.6 |
Component: | HTTP API | Keywords: | has-patch |
Focuses: | Cc: |
Description
We use an API that would check the referrer url. In the process of upgrading to 4.6 their API now responds telling us an invalid referrer was provided. If I roll back to 4.5.3 and run wp_remote_get it works without problems.
Attachments (4)
Change History (20)
#2
@
8 years ago
Why would the CURLOPT_URL be the same as the CURLOPT_REFERER? If I manually set CURLOPT_REFERER to the URL of the domain I am sending from then I get the response I desire.
The code is below but I obviously removed our API key. I'm going to try get another API key for you that I can share here
<?php wp_remote_get('http://api.embed.ly/1/feature?feature=oembed&key=***')
#3
@
8 years ago
Why would the
CURLOPT_URL
be the same as theCURLOPT_REFERER
?
I don't know why it was set that way. It has been like this in Requests ever since, see https://github.com/rmccue/Requests/commit/78d4f3c117642445cf729a7eaaaa87d8d3897fd6.
There's an open issue on GitHub, so maybe @rmccue can chime in.
#4
@
8 years ago
I had a look at Guzzle and they don't seem to do this. Hopefully @rmccue can offer his opinion on this
#7
@
8 years ago
@differentthemes As Requests is an external library, this requires some upstream changes first. You can follow https://github.com/rmccue/Requests/issues/232 for updates on that.
#9
follow-up:
↓ 11
@
8 years ago
- Keywords has-patch added
Wordpress 4.5.4 did not include the referrer at all by default. The referrer should be the root URL of the referring website, site_url should be an adequate solution to this. See patch.
#10
@
8 years ago
- Keywords upstream added
Small note, You can send a Referer
header along with your wp_remote_get()
which will cause cURL not to send the Referer.
Unfortunately, the Streams handler will send a Referer:
header anyway.
This should be fixed upstream at https://github.com/rmccue/Requests/issues/232 - inputs and patches would be welcome there.
#11
in reply to:
↑ 9
;
follow-up:
↓ 12
@
8 years ago
Replying to tristangemus:
Wordpress 4.5.4 did not include the referrer at all by default. The referrer should be the root URL of the referring website, site_url should be an adequate solution to this. See patch.
Thanks! That helps, but only we can't now force all our customers to edit they wp files. This really should be fixed ASAP at least as a new parameter, if not even by default.
#12
in reply to:
↑ 11
@
8 years ago
Replying to differentthemes:
Replying to tristangemus:
Wordpress 4.5.4 did not include the referrer at all by default. The referrer should be the root URL of the referring website, site_url should be an adequate solution to this. See patch.
Thanks! That helps, but only we can't now force all our customers to edit they wp files. This really should be fixed ASAP at least as a new parameter, if not even by default.
Considering this was selected as a solution, it could be part of an upcoming Wordpress release. Hoping to get some support on this or another solution as the referrer being set as the destination URL does not make any sense at all.
Another solution is to not include the referrer and make it an option at all as it is optional in HTTP.
I'd also look into this solution - https://core.trac.wordpress.org/ticket/37820#comment:10
#13
@
8 years ago
Not quite sure if I get it right, but like this it doesn't work.
Edited: Sorry missed this part
$referer = site_url();
#14
@
8 years ago
I have checked WordPress 4.7 BETA 2, but seems like the changes wan't been made there..
#15
@
5 years ago
- Keywords upstream removed
- Milestone Awaiting Review deleted
- Resolution set to reported-upstream
- Status changed from new to closed
I am going to close this one out as reported upstream, since action on the GitHub repository is required for this to be merged into WordPress Core.
Please center discussion on Requests issue 232.
#16
@
5 years ago
I am confused about this. I have not digged though the source but looking at the patch it seems the URL is hard-coded in. However, if I got this right, people say we can set the referer with in the $argsheaders? the screenshot above shows code like this and the codex says the arrays should contains a array with header lines. However I have seen a lot of code that uses key-value pairs in the array so my questions are.
- Can I set the referer with the $args on wp_remote_get/post.
- If yes, do both of this ways work?
- Any easy way to test this?
<?php $referer = site_url(); $wp_remote_get_args = array( 'headers' => array( "Referer: $referer\r\n" ); );
<?php $wp_remote_get_args = array( 'headers' => array( 'Referer' => site_url() ) ); $response = wp_remote_get( $api_url, $wp_remote_get_args );
Edit
Ok I just tested it on https://webhook.site awesome site for testing requests. So turns out the code in the Screenshot from @differentthemes above is actually wrong. It end with with header 0 so not sure where this keyless array with newline is coming from. So only my 2nd example will work.
After thinking about this I think its fine if it stays this way or of WP sends no referer at all for privacy reasons as long as you can manually set it if needed. And having it to the same URL as requested seems kind if the same as setting none all all.
Hey there, thanks for your report!
What kind of arguments are you passing to
wp_remote_get()
? Some sample code to look at would help greatly.Looking at the code from 4.5 and 4.6, no referer was ever set by WordPress in 4.5. Now with 4.6, Requests seems to set the referer to the URL being requested:
curl_setopt($this->handle, CURLOPT_URL, $url); curl_setopt($this->handle, CURLOPT_REFERER, $url);
Citing RFC 2616:
Seems legit in that case.