WordPress.org

Make WordPress Core

Opened 4 years ago

Last modified 3 months ago

#22952 new defect (bug)

WP_HTTP can cause PHP Warnings during attempted decompression

Reported by: dd32 Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version: 3.3
Component: HTTP API Keywords: has-patch needs-testing
Focuses: Cc:

Description (last modified by SergeyBiryukov)

WARNING: wp-includes/class-http.php:1656 - gzinflate(): data error

WP_Http_Encoding can cause PHP Warnings when it attempts to decompress data using gzinflate() which has been encoded in any way.
We currently work around this this in a few ways, but we still take a "try it and see" method instead of detecting the compressed contents signature and handling it appropriately.

Attached is a first-run patch at detecting Huffman coding, which is what we currently use @gzinflate( substr( $gzData, 2 ) ) for (and hey, who doesn't like making magic numbers clearer?)

I have been running a similar patch on WordPress.com and gathering data on how the myriad of different Web Servers out there respond, and so far this causes it to correctly identify the vast majority of responses.

It appears that we may also be attempting to decompress compressed files retrieved through WP_HTTP on some poorly configured servers, but this is something I haven't yet traced properly.

Attachments (3)

22952.diff (2.2 KB) - added by dd32 4 years ago.
22952.2.diff (2.2 KB) - added by dd32 4 years ago.
22952.3.diff (4.6 KB) - added by dd32 23 months ago.

Download all attachments as: .zip

Change History (40)

@dd32
4 years ago

#1 @SergeyBiryukov
4 years ago

  • Description modified (diff)

#2 @dd32
4 years ago

I have also considered that we're using gzinflate() completely wrong, for example:

  • gzencode() == gzip
  • gzcompress() == zlib (aka. HTTP deflate)
  • gzdeflate() == *raw* deflate encoding

Currently we use gzinflate() (the raw DEFLATE standard) to decompress the data created by compressors which add their compression header/footer wrappers.
gzuncompress() for example handles Huffman encoding internally, as it's designed for uncompressing HTTP "deflated" content.
gzdecode() on the other hand is designed for gzip encoded files (which uses DEFLATE internally as the compression method) which has it's own headers (as it's designed for multiple files stored within the archive) - I believe this is the appropriate function to decompress data for the block above the changes in WP_Http_Encoding::compatible_gzinflate() which strips the full zlib headers

Ultimately, we've been bitten by poor implementations from other web servers and software many times, which has lead me to disregarding the above and continuing forward and detecting it ourselves and striping the Huffman headers which gzinflate() can't handle. The changes here are 100% backwards compatible so far.

This still has the possibility of generating warnings (which are silenced by all those @'s) but significantly reduces the number of configurations which might cause one.

@dd32
4 years ago

#3 @DrewAPicture
4 years ago

  • Cc xoodrew@… added

I asked about this on wp-hackers at the end of November, but from what @dd32 said, I understood it to be something on my end.

However, I never managed to overcome it so +1 for fixing it.

#4 @tollmanz
4 years ago

  • Cc tollmanz@… added

#5 @ocean90
3 years ago

  • Cc ocean90 added

#6 @dd32
3 years ago

  • Milestone changed from 3.6 to Future Release

#7 @WraithKenny
3 years ago

  • Cc Ken@… added

#8 @m_uysl
3 years ago

  • Cc m_uysl@… added

#9 follow-up: @nico23
3 years ago

+1

I see this error regularly since I have debug bar ( and the extender) installed. Running Ubuntu 13.04 Apache localhost server. I liked to see it go away, did apply the 2nd patch from above and it seems to be working fine, error is gone.

#10 in reply to: ↑ 9 @jorgeorpinel
3 years ago

I see this error regularly since I have debug bar ( and the extender) installed. Running Ubuntu 13.04 Apache localhost server. I liked to see it go away.

-nico23

Same here. I'm running wp 3.7.1 and its still there.

#11 @jmh
3 years ago

Same for 3.8 - I see this error on the debug bar only when trying to work with the Twitter API.

#12 @thongta
2 years ago

  • Type changed from enhancement to defect (bug)
  • Version set to 3.9

I see this issue in 3.9, it appears randomly on the debug bar on both frontend & backend.

#13 follow-up: @master412160
2 years ago

It get this notice in debug bar too, is this issue patches in next version?

Also get this sometimes:
NOTICE: wp-includes/post-template.php:29 - Trying to get property of non-object

#14 @SergeyBiryukov
2 years ago

  • Version changed from 3.9 to 3.3

Introduced in [18718].

#15 in reply to: ↑ 13 ; follow-up: @SergeyBiryukov
2 years ago

Replying to master412160:

Also get this sometimes:
NOTICE: wp-includes/post-template.php:29 - Trying to get property of non-object

See #17034.

#16 in reply to: ↑ 15 ; follow-up: @master412160
2 years ago

Replying to SergeyBiryukov:

Replying to master412160:

Also get this sometimes:
NOTICE: wp-includes/post-template.php:29 - Trying to get property of non-object

See #17034.

I see so this is a plugin calling a non-object?

#17 in reply to: ↑ 16 @SergeyBiryukov
2 years ago

Replying to master412160:

I see so this is a plugin calling a non-object?

A plugin or theme calling get_the_ID() outside of the loop.

#18 @naomicbush
2 years ago

I see this issue in 3.9 when receiving a JSON response from the MailChimp API v2.0

#19 @GregLone
2 years ago

I'm still having the issue with WordPress 4.0, but this one is new for me:

WARNING: wp-includes/class-http.php:2015 - gzinflate() [function.gzinflate]: data error
require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), include('/absolute-path/to/wp-content/themes/mash/index.php'), get_template_part, locate_template, load_template, require('/themes/golf/content/content-page.php'), the_content, apply_filters('the_content'), call_user_func_array, WP_Embed->autoembed, preg_replace_callback, WP_Embed->autoembed_callback, WP_Embed->shortcode, wp_oembed_get, WP_oEmbed->get_html, WP_oEmbed->fetch, WP_oEmbed->_fetch_with_format, wp_safe_remote_get, WP_Http->get, WP_Http->request, WP_Http->_dispatch_request, WP_Http_Curl->request, WP_Http_Encoding::decompress, gzinflate

Which leads to WP_Http_Encoding::decompress().

This whole thing is triggered if I have an embedded tweet in my content, and if I cleanup the embeds cache ($GLOBALS['wp_embed']->delete_oembed_caches( $post_ID );). Embedded videos are fine.

#20 @swissspidy
23 months ago

Just ran into this issue while retrieving images from https://maps.googleapis.com/maps/api/staticmap using download_url().

I'll try to update the patch. Does this need unit tests?

@dd32
23 months ago

#21 @dd32
23 months ago

22952.3.diff is an updated patch of what's running on WordPress.com after I attempted to fix most of the remote fetching failures we saw (Pingbacks, etc). The code is pretty crazy, but I haven't heard of any false failures.

The basic rule is that there's so many servers out there compressing data in so many different ways, using so many different standards, and all calling it either deflate or gzip that it's not funny..

I never pushed these changes back to core primarily as I never 100% finished tracking down examples of failures (I'd love to unit test it, but need a bunch of obscurely compressed pages that we can use), but secondly because the code looks so crazy that I question if we should be doing so much work here in PHP.. :)

#22 @lkraav
17 months ago

Since this lights up Debug Bar red on the regular, it'd be nice to somehow improve the situation. A comprehensive core fix here seems to be far and away, perhaps Debug Bar can somehow be made to ignore this particular error?

#23 @ocean90
13 months ago

#33949 was marked as a duplicate.

#24 @kanenas
12 months ago

It get this notice in debug bar with WordPress 4.3.1, it appears randomly on backend. See a couple examples below:

Dashboard > Home

WARNING: wp-includes/class-http.php:2136 - gzinflate(): data error
require_once('wp-admin/admin.php'), do_action('admin_init'), call_user_func_array, _maybe_update_plugins, wp_update_plugins, set_site_transient, apply_filters('pre_set_site_transient_update_plugins'), call_user_func_array, KM_UpdatesV3->set_update_transient, KM_UpdatesV3->_check_updates, KM_UpdatesV3->sendApiRequest, wp_remote_post, WP_Http->post, WP_Http->request, WP_Http->_dispatch_request, WP_Http_Curl->request, WP_Http_Encoding::decompress, gzinflate

Dashboard > Updates

WARNING: wp-includes/class-http.php:2136 - gzinflate(): data error
require_once('wp-admin/admin.php'), do_action('load-update-core.php'), call_user_func_array, wp_update_plugins, set_site_transient, apply_filters('pre_set_site_transient_update_plugins'), call_user_func_array, KM_UpdatesV3->set_update_transient, KM_UpdatesV3->_check_updates, KM_UpdatesV3->sendApiRequest, wp_remote_post, WP_Http->post, WP_Http->request, WP_Http->_dispatch_request, WP_Http_Curl->request, WP_Http_Encoding::decompress, gzinflate

How do I check if gzip decompression is disabled on my server? I am on shared hosting environment.

#25 @dd32
12 months ago

That warning is expected, and what this ticket aims to fix. There's nothing wrong with your server.

#26 @travisnorthcutt
11 months ago

Just another data point, I see this issue in 4.3.1 when receiving a JSON response from the Vimeo API.

#27 @l3rady
11 months ago

I'm starting to see this error on 4.3.1 when getting data from GitHub and SendGrid APIs. If those services have changed the way they are compressing data I suspect this is going to be a more common problem for people.

#28 @jcmaci03
8 months ago

+1 Seeing this on WP4.4.2 / php 5.6.15 (shared hosting). Here's my variation.

WARNING: wp-includes/class-wp-http-encoding.php:58 - gzinflate(): data error
require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), include('/themes/mytheme/index.php'), get_footer, locate_template, load_template, require_once('/themes/mytheme/footer.php'), wp_footer, do_action('wp_footer'), call_user_func_array, wp_admin_bar_render, do_action_ref_array, call_user_func_array, wp_admin_bar_updates_menu, wp_get_update_data, get_site_transient, apply_filters('site_transient_update_plugins'), call_user_func_array, GFForms::check_update, GFCommon::check_update, GFCommon::get_version_info, GFCommon::post_to_manager, wp_remote_post, WP_Http->post, WP_Http->request, WP_Http->_dispatch_request, WP_Http_Curl->request, WP_Http_Encoding::decompress, gzinflate

@dd32, what can the community do to help move this ticket forward? There is obviously interest in seeing this resolved. :)

#29 @vlad.olaru
7 months ago

Yep. Same here on WP 4.4.2 and latest PHP 5.6.x

+1

#30 follow-up: @swissspidy
7 months ago

I can imagine that this could be fixed by #33055. Perhaps @rmccue could tell us a bit about how Requests handles compressed data.

#31 in reply to: ↑ 30 ; follow-up: @rmccue
7 months ago

Replying to swissspidy:

I can imagine that this could be fixed by #33055. Perhaps @rmccue could tell us a bit about how Requests handles compressed data.

Requests' compressed data handling (specifically, compatible_gzinflate) is based off WP_Http's (albeit technically the same code independently contributed by dd32 for licensing reasons), so no real changes here. Any issues WP_Http suffers from, Requests will likely too.

#32 @lumpysimon
6 months ago

I've started seeing this a lot recently (possibly only since 4.4.2 upgrade? Although I'm not certain of that), across several sites on different servers, all WordPress 4.4.2, either PHP 5.6 or 7.0.1, both live and on localhost.

Here are a couple of example outputs from Query Monitor in case it's helpful. The first one is retrieving data from twitter.com, the second from maps.googleapis.com

Warning:
gzinflate(): data error

Location:
wp-includes/class-wp-http-encoding.php:58	

Call stack:

gzinflate()
 wp-includes/class-wp-http-encoding.php:58
WP_Http_Encoding::decompress()
 wp-includes/class-wp-http-curl.php:293
WP_Http_Curl->request()
 wp-includes/class-http.php:367
WP_Http->_dispatch_request()
 wp-includes/class-http.php:277
WP_Http->request()
 wp-includes/class-http.php:412
WP_Http->post()
 wp-includes/http.php:187
wp_remote_post()
 wp-content/themes/Avada/widgets/tweets-widget.php:67
Tweets_Widget->widget()
 wp-includes/class-wp-widget.php:347
WP_Widget->display_callback()
 wp-includes/widgets.php:729
dynamic_sidebar('avada-footer-widget-1')
 wp-content/themes/Avada/footer.php:32
load_template('~/wp-content/themes/Avada/footer.php')
 wp-includes/template.php:521
locate_template()
 wp-includes/general-template.php:85
get_footer()
 wp-content/themes/Avada/page.php:188
Warning:
gzinflate(): data error

Location:
wp-includes/class-wp-http-encoding.php:58	

Call stack:

gzinflate()
 wp-includes/class-wp-http-encoding.php:58
WP_Http_Encoding::decompress()
 wp-includes/class-wp-http-curl.php:293
WP_Http_Curl->request()
 wp-includes/class-http.php:367
WP_Http->_dispatch_request()
 wp-includes/class-http.php:277
WP_Http->request()
 wp-includes/class-http.php:430
WP_Http->get()
 wp-includes/http.php:170
wp_remote_get()
 wp-content/plugins/nfa/inc/shortcodes.php:266
nfa_shortcodes->location_lookup()
 wp-content/plugins/nfa/inc/shortcodes.php:317
nfa_shortcodes->map()
 wp-includes/shortcodes.php:326
do_shortcode_tag()
 Unknown location
preg_replace_callback()
 wp-includes/shortcodes.php:223
do_shortcode()
 wp-includes/plugin.php:235
apply_filters('the_content')
 wp-content/themes/nfa/sidebar.php:6
load_template('~/wp-content/themes/nfa/sidebar.php')
 wp-includes/template.php:521
locate_template()
 wp-includes/general-template.php:125
get_sidebar('page')
 wp-content/themes/nfa/page.php:28

#33 @ocean90
6 months ago

#36560 was marked as a duplicate.

#34 @pkostadinov
5 months ago

The error is still present in WP 4.5:

Warning: gzinflate(): data error in /wp-includes/class-wp-http-encoding.php on line 58

Here is my trace:

----------------------------------------------------------------
 # Date ::: 2016-04-25 11:42:11
----------------------------------------------------------------
Array
(
    [0] => Warning: gzinflate(): data error in /wp-includes/class-wp-http-encoding.php on line 58
    [1] => Array
        (
            [file] => 
            [line] => 
            [function] => crb_ignore_plugins_errors2
        )

    [2] => Array
        (
            [file] => \wp-includes\class-wp-http-encoding.php
            [line] => 58
            [function] => gzinflate
        )

    [3] => Array
        (
            [file] => \wp-includes\class-wp-http-curl.php
            [line] => 293
            [function] => decompress
        )

    [4] => Array
        (
            [file] => \wp-includes\class-http.php
            [line] => 433
            [function] => request
        )

    [5] => Array
        (
            [file] => \wp-includes\class-http.php
            [line] => 342
            [function] => _dispatch_request
        )

    [6] => Array
        (
            [file] => \wp-includes\http.php
            [line] => 153
            [function] => request
        )

    [7] => Array
        (
            [file] => \wp-content\plugins\jetpack\class.jetpack-client.php
            [line] => 162
            [function] => wp_remote_request
        )

    [8] => Array
        (
            [file] => \wp-content\plugins\jetpack\class.jetpack-client.php
            [line] => 118
            [function] => _wp_remote_request
        )

    [9] => Array
        (
            [file] => \wp-content\plugins\jetpack\class.jetpack-ixr-client.php
            [line] => 34
            [function] => remote_request
        )

    [10] => Array
        (
            [file] => \wp-content\plugins\jetpack\class.jetpack.php
            [line] => 5819
            [function] => query
        )

    [11] => Array
        (
            [file] => \wp-content\plugins\jetpack\class.jetpack.php
            [line] => 6072
            [function] => get_cloud_site_options
        )

    [12] => Array
        (
            [file] => \wp-content\plugins\jetpack\class.jetpack-sync.php
            [line] => 173
            [function] => is_staging_site
        )

    [13] => Array
        (
            [file] => 
            [line] => 
            [function] => sync
        )

    [14] => Array
        (
            [file] => \wp-includes\plugin.php
            [line] => 525
            [function] => call_user_func_array
        )

    [15] => Array
        (
            [file] => \wp-includes\load.php
            [line] => 638
            [function] => do_action
        )

    [16] => Array
        (
            [file] => 
            [line] => 
            [function] => shutdown_action_hook
        )

)

#35 @jfarsen
4 months ago

I am getting the same error as @pkostadinov on a Siteground shared hosting account using PHP 7.0x

#36 in reply to: ↑ 31 @dd32
4 months ago

This should be fixed in trunk/4.6 as Requests was added in #33055 which contains more-or-less the same as 22952.3.diff already.

#37 @ReneHermi
3 months ago

I vote for fixing and merging it. This issue became disturbing since the latest core updates (4.4 / 4.5) and is visible on nearly every wordpress site i have inspected so far.

https://monosnap.com/file/C9IeQkLXD489qA9XxHA3TAE3s1D8jd.png

Last edited 3 months ago by ReneHermi (previous) (diff)
Note: See TracTickets for help on using tickets.