WP_HTTP can cause PHP Warnings during attempted decompression

[dd32]

WARNING: wp-includes/class-http.php:1656 - gzinflate(): data error

WP_Http_Encoding can cause PHP Warnings when it attempts to decompress data using gzinflate() which has been encoded in any way.
We currently work around this this in a few ways, but we still take a "try it and see" method instead of detecting the compressed contents signature and handling it appropriately.

Attached is a first-run patch at detecting Huffman coding, which is what we currently use @gzinflate( substr( $gzData, 2 ) ) for (and hey, who doesn't like making magic numbers clearer?)

I have been running a similar patch on WordPress.com and gathering data on how the myriad of different Web Servers out there respond, and so far this causes it to correctly identify the vast majority of responses.

It appears that we may also be attempting to decompress compressed files retrieved through WP_HTTP on some poorly configured servers, but this is something I haven't yet traced properly.

I have also considered that we're using gzinflate() completely wrong, for example:

• gzencode() == gzip
• gzcompress() == zlib (aka. HTTP deflate)
• gzdeflate() == *raw* deflate encoding

Currently we use gzinflate() (the raw DEFLATE standard) to decompress the data created by compressors which add their compression header/footer wrappers.
gzuncompress() for example handles Huffman encoding internally, as it's designed for uncompressing HTTP "deflated" content.
gzdecode() on the other hand is designed for gzip encoded files (which uses DEFLATE internally as the compression method) which has it's own headers (as it's designed for multiple files stored within the archive) - I believe this is the appropriate function to decompress data for the block above the changes in WP_Http_Encoding::compatible_gzinflate() which strips the full zlib headers

Ultimately, we've been bitten by poor implementations from other web servers and software many times, which has lead me to disregarding the above and continuing forward and detecting it ourselves and striping the Huffman headers which gzinflate() can't handle. The changes here are 100% backwards compatible so far.

This still has the possibility of generating warnings (which are silenced by all those @'s) but significantly reduces the number of configurations which might cause one.

I asked about this on wp-hackers at ​the end of November, but from what @dd32 said, I understood it to be something on my end.

However, I never managed to overcome it so +1 for fixing it.