Gzip compression should be offered by default in Wordpress

[caesarsgrunt]

Output should be Gzipped when possible - I think the used to be an option, but it was removed in 2.5. Can we please have it back?

[dd32]

Gzip'd output is much better handled on the server level, or with a Caching plugin.

that being said.. a simple ob_start("ob_gzhandler"); in a plugin, or in your themes functions.php would enable application-level Gzip compression.

[Denis-de-Bernardy]

or in wp-config.php

[westi]

And it was removed because it caused more trouble than benefit.

We people enabling it even when the server was already doing it and things getting double zipped.

The correct solution is at the web server level as dd32 suggests.

[micasuh]

I agree with ceasarsgrunt on this and believe this feature should be added back into future releases with default access being turned off. Let the developer choose whether he or she wants to use built in Gzip compression. Not only does gzipping promote good practice but it provides additional competition against major CMSes which already include this by default.

​Gzipping generally reduces the response size by about 70%. Approximately 90% of today's Internet traffic travels through browsers that claim to support gzip.

To satisfy westi's comment about double zipping, it should be rather easy for Wordpress to detect whether the server is already using another form of compression on its output.

I agree gziped output should live at the server level, but there's no reason Wordpress can't give a smart option to inform developers of this option and help enable it. Additionally, since the vast majority of Wordpress installations are on shared servers, it's unlikely that gzipped compression is enabled by default. Web hosts might offer and provide this component on the server, but it's very unlikely that it is enabled unless the developer specifically knows what he or she is doing.

Given the opportunity, developers should use this without another plugin. Plugins cause additional and unneeded requests that slow down rendering time for site to load into browser. Having this functionality intelligently built into Wordpress would actually benefit the Internet in general because of Wordpress' universal popularity and widspread use.

[Denis-de-Bernardy]

Still, it causes tremendous amounts of problems on some servers, and this is really something you want to deal with from apache. In your htaccess file, add:

<IfModule mod_deflate.c>
# Insert filters
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/x-httpd-php
AddOutputFilterByType DEFLATE application/x-httpd-fastphp
AddOutputFilterByType DEFLATE image/svg+xml

# Drop problematic browsers
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</IfModule>

Doing so is a lot less resource intensive that php's ob_gzhandler, it's much faster, and it avoids potential double compression problems.

[caesarsgrunt]

Yse, but some hosts don't support it.

[novasource]

It's interesting contrasting this to how warmly the Drupal community embraced the same request. See ​http://drupal.org/node/290280.

[micasuh]

I don't disagree that apache should handle gzip, but I also believe the final decision should ultimately be left up to the developer. Wordpress should provide the option in the first place, as it used to offer.

There's no reason why Wordpress can't provide this option, disabled by default or after detecting whether gzip is already enabled to prevent double compression problems, and a link to a page that describes the different gzip compression methods to let the developer decide.

We can't assume that all hosts will provide the same options but we should educate and inform developers to use best practices and encourage gzipping their output.

[westi]

Closing again as Wontfix

I am not convinced this should become a core feature again it was very difficult for it to work 100% reliably and there is little use in something like this being included but disabled by default.

It would be much more suitable as a plugin IMHO.

[caesarsgrunt]

Westi, there's a discussion going on here. Not everybody seems to have the same opinion as yourself. Perhaps other opinions should also be taken into account?
Reopening for continued discussion.

[Denis-de-Bernardy]

Replying to micasuh:

We can't assume that all hosts will provide the same options but we should educate and inform developers to use best practices and encourage gzipping their output.

I strongly disagree. We should educate users to switch to better hosts, of the kind that enable this by default.

[Denis-de-Bernardy]

Replying to caesarsgrunt:

Westi, there's a discussion going on here. Not everybody seems to have the same opinion as yourself. Perhaps other opinions should also be taken into account?
Reopening for continued discussion.

I'm all for closing once and for all, myself. I've suffered enough, from a support standpoint, because of this option. I even added a filter so it was *always* disabled no matter what the user would set it to, when it was still around.

This is totally plugin material. And it is an architecturally incorrect way to do things. Apache should deal with this as indicated above.

What *would* get my favor is if the above-mentioned htaccess statements are included by default in the htaccess file. That won't hurt any installation, and it avoids double-compression problems.

[caesarsgrunt]

Replying to Denis-de-Bernardy:

We should educate users to switch to better hosts, of the kind that enable this by default.

In principle I agree with that, but the fact is that some hosts which are good in most/all other respects don't have mod_deflate for some reason, whilst some which do have it are rubbish in other respects.
Also, not all users know how to switch hosts or can afford to.

[caesarsgrunt]

Replying to Denis-de-Bernardy:

This is totally plugin material.

Thing is, the point of this is to speed things up. Adding another plugin goes against that concept, really, even if it only has a very small effect in practice.

And it is an architecturally incorrect way to do things. Apache should deal with this as indicated above.

Maybe, but it doesn't without modules installed which not all hosts have enabled. And most/many other equivalent projects to WP have this option.

What *would* get my favor is if the above-mentioned htaccess statements are included by default in the htaccess file. That won't hurt any installation, and it avoids double-compression problems.

That's a good concept, but still doesn't help in the case where mod_deflate isn't available.

[caesarsgrunt]

We could make it an option when installing - either use mod_deflate, or ob_gzhandler, or neither.

Alternatively, detect whether mod_deflate is enabled, and if not do it through php.

[Denis-de-Bernardy]

no no, mod_deflate, else nada, is the way to go.

it's just too much hassle to deal with ob_gzhandler(). it might have been enabled already by a plugin, it doesn't work on all browsers, it might... it just not worth the trouble.

[Denis-de-Bernardy]

Replying to caesarsgrunt:
I'm not hostile at the idea of gzip'ing things in core if it's done properly (i.e. mod_deflate). I'm just extremely hostile to the idea of reallowing ob_gzhandler'ing things in core.

Also, not all users know how to switch hosts or can afford to.

Then they should switch to ​http://hub.org. It's run by one of the postgresql founders, the owner is the very best system admin I can name, they have one-click WP installs (I maintain the script myself) and they migrate sites for free.

That's a good concept, but still doesn't help in the case where mod_deflate isn't available.

Best I'm aware and best I've seen, mod_deflate is available on apache 2 hosts. FWIW, it was installed by default when I ran port install apache2. How many apache 2 hosts don't offer mod_deflate. Personally, I don't know any.

If some hosts are still stuck in apache 1.3/prehistory, it's shouldn't be our problem. Especially not if doing so creates boat loads of problems on our end.

[caesarsgrunt]

Replying to Denis-de-Bernardy:

How many apache 2 hosts don't offer mod_deflate. Personally, I don't know any.

How about Site5.com? A very good host in all other respects.

If some hosts are still stuck in apache 1.3/prehistory, it's shouldn't be our problem. Especially not if doing so creates boat loads of problems on our end.

I don't think it does create many problems if you only enable it when mod_deflate is disabled.

Anyhow, I agree with you that adding the mod_deflate rules above to the default htaccess is a good start.

[Denis-de-Bernardy]

Anyhow, I agree with you that adding the mod_deflate rules above to the default htaccess is a good start.

Cool. Then let's get the patch tested on a couple of hosts with mod_deflate. It certainly won't hurt performance on sites that use it. And imo, it's enough to close the ticket without even bothering with any ob_gzhandler().

[caesarsgrunt]

Hmmm... I beg to differ! :-)

But anyway, yes, let's get on with testing your patch, and we can discuss ob_gzhandler later...

[petervanderdoes]

Replying to novasource:

It's interesting contrasting this to how warmly the Drupal community embraced the same request. See ​http://drupal.org/node/290280.

The Drupal request is not the same request. The Drupal request has the javascript and CSS files zipped and have rules in .htaccess to serve those files.
I have this setup on my WordPress site, manually done.

The request here is about serving the produced HTML page compressed as well.

[micasuh]

Replying to Denis-de-Bernardy:

Also, not all users know how to switch hosts or can afford to.

Then they should switch to ​http://hub.org. It's run by one of the postgresql founders, the owner is the very best system admin I can name, they have one-click WP installs (I maintain the script myself) and they migrate sites for free.

This argument sounds like Windows vs. Mac or Ford vs. Chevy. It's not Wordpress community's job to make developers choose a host. There are MANY hosts who support no or only one gzipping method.

I've seen certain hosts ​lock down module requests for security or performance reasons, but that does not mean this host is bad. You have to take into account those people who maintain their own VPS or physical server as well as people who don't have a clue what they are doing. Those on a shared server environment might not be able to choose which method or any method at all.

How many apache 2 hosts don't offer mod_deflate. Personally, I don't know any.

How about Site5.com? A very good host in all other respects.

Denis-de-Bernardy: We should educate users to switch to better hosts, of the kind that enable this by default.

User education to go with hosts that support mod_deflate or gzip compression is a noble cause but not practical. I'm sure there are countless other hosts which don't offer mod_deflate like Site5.com. Are old Apache 1.3 servers completely out of date? I guess this is a similar debate for which PHP versions Wordpress should support, and obviously anything before PHP 4.3 is not compatible and with good reason.

Replying to ceasarsgrunt:

We could make it an option when installing - either use mod_deflate, or ob_gzhandler, or neither.

Alternatively, detect whether mod_deflate is enabled, and if not do it through php.

This is exactly what I think. Is there any reason why something like this cannot be implemented into Wordpress? It just makes no sense not to do this.

The reason I resurrected this ticket is because of increasing noise from Google (specifically Matt Cutts) about Google's next generation search engine build called Caffeine which is coming out after the holidays. Matt said specifically that ​site speed is now a factor for page ranking. While this isn't the main reason for gzip implementation, Google's influence and new position on site speed gives this issue new light.

Finally, and please excuse my ignorance, but are mod_deflate and ob_gzhandler the only two server side methods for gzip compression on Apache 2?

[westi]

From memory one of the issues with the current patch that will still cause double compression is that if someone has configure php to always compress the output you will get double compression again.

php has a number of ini file settings to enable compression for all pages - e.g. setting an output_handler or zlib.output_compression

[Denis-de-Bernardy]

@westi: best I'm aware, apache would conditionally compress the output if the mime fits. if a plugin compresses a js file and doesn't set the headers correctly, it's another issue.

[caesarsgrunt]

@Denis:
When does the function save_mod_rewrite_rules which you've added the code into actually get run?

Also, why not compress all content except images? We can use the sample code at ​http://httpd.apache.org/docs/2.0/mod/mod_deflate.html (with some of the comments stripped out...) :

<IfModule mod_deflate.c>
  SetOutputFilter DEFLATE
  
  # Drop problematic browsers
  BrowserMatch ^Mozilla/4 gzip-only-text/html
  BrowserMatch ^Mozilla/4\.0[678] no-gzip
  BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
  
  # Don't compress images
  SetEnvIfNoCase Request_URI \
  \.(?:gif|jpe?g|png)$ no-gzip dont-vary
  
  # Make sure proxies don't deliver the wrong content
  Header append Vary User-Agent env=!dont-vary
</IfModule>

[Denis-de-Bernardy]

Replying to caesarsgrunt:

@Denis:
When does the function save_mod_rewrite_rules which you've added the code into actually get run?

when you save permalinks.

Also, why not compress all content except images? We can use the sample code at ​http://httpd.apache.org/docs/2.0/mod/mod_deflate.html (with some of the comments stripped out...) :

yeah, my suggest is based on theirs. But their doc's example is actually crappy. it mindlessly compresses audio files, video files, pdfs, etc. those are already compressed, and compressing them prevents some user agents to process them correctly. with the one I suggest, we only compress html/xml/css/js files.

[caesarsgrunt]

Replying to Denis-de-Bernardy:

when you save permalinks.

Yeah, I thought so. Isn't there a better/more relevant place to insert this?

yeah, my suggest is based on theirs. But their doc's example is actually crappy. it mindlessly compresses audio files, video files, pdfs, etc. those are already compressed, and compressing them prevents some user agents to process them correctly. with the one I suggest, we only compress html/xml/css/js files.

Can't pdfs be gzipped? But I take your point about compressing things which shouldn't be.

[novasource]

Replying to petervanderdoes:

Replying to novasource:

It's interesting contrasting this to how warmly the Drupal community embraced the same request. See ​http://drupal.org/node/290280.

The Drupal request is not the same request. The Drupal request has the javascript and CSS files zipped and have rules in .htaccess to serve those files.
I have this setup on my WordPress site, manually done.

The request here is about serving the produced HTML page compressed as well.

Sorry, you're correct, and additionally that is just for one Drupal module. But Drupal does have gzip compression for cached pages built into core, configurable at admin/settings/performance.

[dd32]

A Personal IMO: This should be built and offered as part of a plugin, Specifically, a Caching plugin. Should have the options of different levels:

1. No Caching
2. Compressed output
3. Dynamic Caching (Compressed)
4. Static Caching (Compressed)

Or similar. I was against the idea of "Core/canonical Plugins" but things like this it really suits better to, A plugin which controls the WordPress HTML output in the way the user wishes. - A plugin has the benefit that it is seperate and can be updated more easily as browser/server compatibility issues pop up.

[micasuh]

Replying to dd32:

I was against the idea of "Core/canonical Plugins" but things like this it really suits better to, A plugin which controls the WordPress HTML output in the way the user wishes. - A plugin has the benefit that it is seperate and can be updated more easily as browser/server compatibility issues pop up.

I understand where you're coming from but there's some fundamental reasons why this should be built into Wordpress rather than become a plugin.

1. Compression is forward thinking. Wordpress is currently one of the only major Blogging/CMS platforms which doesn't inherently offer gzip compression built in. Why does this matter? A platform so popular and so widely used should promote good web practices such as compression and similar bandwidth saving techniques.
2. Fostering compression usage promotes efficiency and cuts costs. It reduces request amounts, speeds up site loading, and is less of a strain on the server. Wordpress could even claim it's going "Green"!
3. Compression becomes a standard. Arguably, the world is watching what Wordpress does and how it develops. In some sense, Wordpress IS the innovator and should push the envelope by adopting best practices such as compression. This kind of innovation becomes the envy and standard for other platform developers and it helps everyone.

To address compatibility issues, Wordpress is on an aggressive updating schedule that competes with the schedule that mainstream browser manufacturers release updated versions of popular browsers.

If compatibility issues do arise, there's no reason a patch can't be applied within hours or days of a new browser version. On top of this, especially with Firefox development, it would amaze me that there's not better communication of potential problems in compatibility between these two extremely popular platforms.

I argue that this is a dire necessity to Wordpress' core and shouldn't be dismissed as a plugin. This would bring a certain badge of honor to the Wordpress community to show that Wordpress' developers are serious about speed, performance and efficiency.

[hakre]

As long as there is no exact point where the output starts nor a defined, binding, strict and explicit point where it stops, you are technically not able to solve it within PHP code if you do not edit the first line of code which is not an option for the core code.

Ignoring this simple fact will only lead us to shoot in our foot. First of all there should be a definition of the workflow the output undergoes before discussing condiderations of why this feature is good or bad. Such a feature will always find it's pro/counter-speakers based on wether or not a blog admin is able to enable output compression properly when wanted or vice-versa.

Therefore it's pretty much adviseable to provide this by a plugin because a plugin can be written for a certain server configuration. The core (should) not (I know there are exceptions to this, but not here).

Even better would be some documentation that just plainly talks about what is going on regarding "compression". It took me longer to read this dicussion here then to create a codex page that just helps everybody right away:

​WordPress Codex: Output Compression

Feel free to add your knowledge to that page so that it is accessible by everybody. I have not searched for already existing plugins or the like, so this might be something valuable to add. Thanks.

[dalmaatieur]

I placed the IfModule mod_deflate.c code in my ​website hosting blog. It's just a little bit faster. The notorious Wordpress speed is gonna have big problems with the 2010 Google algorythms that require high website loading speeds. We should act before it's too late!

[hakre]

Replying to dalmaatieur:

The notorious Wordpress speed is gonna have big problems with the 2010 Google algorythms that require high website loading speeds. We should act before it's too late!

This is not the topic of this ticket. For general optimization, please test first why your blog takes the time it takes, ​Codex has some hints as well for this. Feel free to add there.

[caesarsgrunt]

Replying to dalmaatieur:
I'm one of the biggest proponents of restoring gzip functionality (I opened the ticket), but the fact is that it can hardly be described as critical, and that as yet it has not been agreed that it should have a high priority. Also, that's not really how we use keywords on the Trac here - they mostly refer to the status of the ticket rather than the subject of it, which is covered by the Component field.

[Denis-de-Bernardy]

Replying to dalmaatieur:

I placed the IfModule mod_deflate.c code in my ​website hosting blog. It's just a little bit faster.

it's only a bit faster because the asset files are not concatenated.

[hakre]

Reviewed the patch, apache_mod_loaded() is broken: #11604

[nacin]

Why should this not be reserved for a plugin?