HTTP API: Chunked fails, Redirects fail, and warnings

[DD32]

It appears that not all hosts which set the transfer-encoding to chunked, actually send the output in chunked format.

An example site of this is ​http://www.spiritblog.net/ (Purely random selection)

The issue is that WP_Http::chunkTransferDecode() will return false on this event.

I see 2 solutions:

• Have chunkTransferDecode() return the content back if its deterined not to be chunked
• have the transports check that the return result of chunkTransferDecode() is true before assigning to the body.

Option 1 seems a good route honestly, Decode it if its determined to be chunked, else just pass it back. Patch will be attached for this route shortly.

[DD32]

A Thought just occured to me, Its using the CURL transport on that server, Could it be that the Server might be returning chunked format, But the curl library is decoding it instead?

[DD32]

Also related;

The HTTP Extension chunked decode may fail with an error if the timeout occured before the request fully finished:

Warning: http_request() [function.http-request]: a timeout was reached; Operation timed out after 5 seconds with 32097 bytes received (http://whosit.wamble.net/) in /test/wp-includes/http.php on line 861

( ! ) Warning: http_chunked_decode() [function.http-chunked-decode]: Truncated message: chunk size 71343 exceeds remaining data size 32097 at pos 7 of 32104 in //test/wp-includes/http.php on line 873

[jacobsantos]

Replying to DD32:

A Thought just occured to me, Its using the CURL transport on that server, Could it be that the Server might be returning chunked format, But the curl library is decoding it instead?

I've noticed that also, however I kept it there in case I was incorrect.

[Otto42]

Yes, this is definitely a curl thing. If curl receives a chunked message, then it passes back a single non-chunked response to PHP, but it also leaves the Transfer-Encoding: chunked header.

Probably should make the curl module always assume the message is not chunked, since curl deals with it for you.

[jacobsantos]

So the solutions are remove the chunked decode from cURL and well, there is not yet a good solution for the warnings.

[DD32]

Changed the title of this ticket to reflect the contents of the latest patch

• General: Move body processing functions after redirect checks
• Curl: Remove chunked transfer check
• Curl: When it redirects, it returns the headers for each request, use the last set of headers from the list
• Fopen / Streams: Implement redirects
• Ext HTTP: Handle error conditions, silence warnings

[DD32]

extra redirect code wasnt needed.

[jacobsantos]

7770.3.diff looks good to me.

[ryan]

(In [9091]) Chunked encoding fixes from DD32. fixes #7770