WP increasing the memory above other server limit

[priitsalumaa]

Hi, I discovered a following problem:

I'm trying out the WP 3.0RC2 (no additional plugins or themes) and the admin dashboard breaks. Namely, the following menu items display a blank page under wp-admin:

• Links->Links
• Links->Link Categories
• Appearance->Menus
• Users

It seems to be caused by problems with heap overflow (and maybe WP increasing the memory above the limit set for PHP (it's 64MB)).

Examples from Appache error log:

[Fri Jun 11 13:11:49 2010] [error] [client 90.190.xxx.xxx] ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker '90.190.xxx.xxx', file '/var/www/www.xxxxx.ee/wp-admin/admin.php', line 96), referer: ​http://www.xxxxx.ee/wp-admin/
[Fri Jun 11 13:11:49 2010] [error] [client 90.190.xxx.xxx] ALERT - canary mismatch on efree() - heap overflow detected (attacker '90.190.xxx.xxx', file '/var/www/www.xxxxx.ee/wp-admin/includes/template.php', line 3557), referer: ​http://www.xxxxx.ee/wp-admin/

[Fri Jun 11 13:12:16 2010] [error] [client 90.190.xxx.xxx] ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker '90.190.xxx.xxx', file '/var/www/www.xxxxx.ee/wp-admin/admin.php', line 96), referer: ​http://www.xxxxx.ee/wp-admin/link-manager.php
[Fri Jun 11 13:12:16 2010] [error] [client 90.190.xxx.xxx] ALERT - canary mismatch on efree() - heap overflow detected (attacker '90.190.xxx.xxx', file '/var/www/www.xxxxx.ee/wp-admin/includes/template.php', line 3557), referer: ​http://www.xxxxx.ee/wp-admin/link-manager.php

[Fri Jun 11 13:12:23 2010] [error] [client 90.190.xxx.xxx] ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker '90.190.xxx.xxx', file '/var/www/www.xxxxx.ee/wp-admin/admin.php', line 96), referer: ​http://www.xxxxx.ee/wp-admin/index.php

The same errors are reported when I use the nightly build.

If the memory limit is increased to 256M for PHP the "ALERT - script tried to increase memory_limit" errors dissapear from the log, but the "ALERT - canary mismatch on efree() - heap overflow detected" remain. The menus are still broken in wp-admin environment.

My questions hereby are:

• Is there possibly a bug in WP 3.0 code causing memory corruption (read the: ​http://www.suspekt.org/2008/10/12/suhosin-canary-mismatch-on-efree-heap-overflow-detected/)?

• For memory increase related error messages - is there any solution to this, which does not require increasing the memory limit of the server?

Cheers,
Priit Salumaa

[wpmuguru]

The issue is WP attempting to increase the memory limit above another server imposed limit:

"script tried to increase memory_limit to 268435456 bytes which is above the allowed value"

[nacin]

We increase the memory limit in the admin area to 256M now. This is error-suppressed and also has a filter you can adjust: admin_memory_limit.

We bumped to 256M in other places previously, when we specifically needed it, so this is not without precedent.

[westi]

For the memory limit messages you can use the filter to stop us trying to raise it that high.

For the suhosin errors these hi-light a bug in the php interpreter or a php extension you have installed not a bug in WordPress itself.

I guess you may be running with incompatible php extensions or esoteric ones.

Are you using self built php or a distribution built one?
Which host operating system and what version of php/suhosin?

[priitsalumaa]

Memory limit messages are just annoying not a real problem - increasing the PHP memory took care of them anyway.

The heap problem is an issue, though.

The WP 2.9.2 runs without any problems on the server.

Since I'm not the admin of the server and can't reach the guy during weekend I can't answer that question with certainty: "Are you using self built php or a distribution built one?" Neither can I prove or dismiss this suggestion: "I guess you may be running with incompatible php extensions or esoteric ones." - although I don't think neiter is the case.

I could find out the following things, though, maybe this helps:

$ uname
Linux

$ uname -r
2.6.27.45-server-1mnb

$ php -v
PHP 5.2.6 with Suhosin-Patch 0.9.6.2 (cli) (built: Jan 15 2010 11:32:10)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies

with Suhosin v0.9.27, Copyright (c) 2007, by SektionEins GmbH

$ mysql -v
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is
Server version: 5.0.89 Mandriva Linux - MySQL Standard Edition (GPL)

Cheers,
Priit

Replying to westi:

For the memory limit messages you can use the filter to stop us trying to raise it that high.

For the suhosin errors these hi-light a bug in the php interpreter or a php extension you have installed not a bug in WordPress itself.

I guess you may be running with incompatible php extensions or esoteric ones.

Are you using self built php or a distribution built one?
Which host operating system and what version of php/suhosin?

[westi]

PHP 5.2.6 was released on 01-May-2008 so in all likelihood you are using a self compiled php.

Moving out of 3.0 into 3.1 to wait for more reproductive information and confirmation that the error being triggered here is not caused by incompatible php modules in your install.

[priitsalumaa]

I'm not sure if I will find the time to debug it further - it was easier to downgrade to 2.9.2 and to modify the theme accordingly.

[layotte]

For what it's worth, I just noticed this same thing on one of my VPS's...

Jun 26 18:19:53 teleread suhosin[12017]: ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker '71.251.209.17', file '/var/www/teleread/wp-admin/admin.php', line 96)

Server version: Apache/2.2.14 (Ubuntu)
PHP 5.3.2-1ubuntu4 with Suhosin-Patch (cli) (built: Apr 9 2010 08:23:39)
Suhosins 0.9.29

I'm not confident that this is a WP issue, as much as it is just suhosin being suhosin.

Lew

[layotte]

ugh, stupid formatting... these lines should have been separated:

Server version: Apache/2.2.14 (Ubuntu)

PHP 5.3.2-1ubuntu4 with Suhosin-Patch (cli) (built: Apr 9 2010 08:23:39)

Suhosins 0.9.29

[layotte]

Also, I noticed this because I've been having an issue with this server, about once a week or two it keeps going nuts and I've been needing to force reboot it.

The only things running are apache and mysql w/ a single WP3.0 install. So it might be suhosin freaking out over this too. But I still sense that it is more suhosin than it is WP.

[hakre]

I ​reviewed the code and in the end I tend to patch this against a constant to better deal with the 256MB magic number.

[hakre]

Use Constant instead of Magic Number

[hakre]

Introducing WP_MAX_MEMORY_LIMIT replacing the current 256MB Magic Number.

Name: WP_MAX_MEMORY_LIMIT
Value: Integer, String

Description:
Set WP_MAX_MEMORY_LIMIT to configure the maximum available memory for wordpress (the PHP memory limit).

When an integer is used, the value is measured in bytes. Shorthand notation, as described in ​this FAQ, may also be used.

[Denis-de-Bernardy]

Isn't there a suhosin ini setting that allows to pull the max memory limit directly? I suspect the current patch ought to be changed in such a way that, when present, it gets used instead of a flat 256MB, along with a warning to the end user in a way or another where it's useful (i.e. upgrades and file uploads).

[hakre]

Replying to Denis-de-Bernardy:

Isn't there a suhosin ini setting that allows to pull the max memory limit directly? I suspect the current patch ought to be changed in such a way that, when present, it gets used instead of a flat 256MB, along with a warning to the end user in a way or another where it's useful (i.e. upgrades and file uploads).

Sure, my first patch is only adding a constant instead of using the magic numbers. So it address only the furthermost problem that stands in the way to enable a user to deal with the problem w/o modifying core-code.

The standard PHP Memory Setting can be read by using the ini_get() function (I assume that by proxy this will be gathering the same value suhosin is checking against) and if that set to the maximum available memory would certainly solve the problem but if so, there would be no need at all to use ini_set() all over the place because it's then assumed that the server has been already configured.

But wordpress is assuming that the server is not properly configured and therefore assumes it's the better admin by just setting this up high.

As you can see it's not trivial to find the right solution so I started by removing that flaw that is solveable right away: reducing the magic numbers to a single constant.

---

Related: #14425

[hakre]

Memory Related: #14245

[hakre]

Related: ​http://unit-tests.trac.wordpress.org/ticket/6

[hakre]

Memory Related: #14889

[hakre]

I would love to see some traction in the one or other direction with this one. For the moment I would favorable something that at least offers some working configuration.

[gionn]

Please include the proposed patch, an hardcoded 256MB value is a waste of memory on small VPS using php-cgi, I would like to configure a small memory limit in my wp-config.php.

[hakre]

Related: #15966

[prettyboymp]

Would love to see this get some movement. I've often run into issues where I need to boost the memory limit while running imports and the media handling constantly drops it back down to 256MB.

[wpmuguru]

apply the admin_memory_limit filter throughout wp-admin

[hakre]

Replying to wpmuguru:

Thanks for providing a patch, it's missing wp-includes/media.php, right? Any ideas how to deal with it?

---

Related: #16789

[wpmuguru]

Replying to hakre:

Replying to wpmuguru:

Thanks for providing a patch, it's missing wp-includes/media.php, right? Any ideas how to deal with it?

I did the patch for the wp-admin/* area. I would not expect an admin_area_ filter to be used in any of the files in wp-includes. If there is something you feel needs to be patched in wp-includes, a second patch to files in wp-includes won't interfere with the one I've uploaded above.

[hakre]

Replying to wpmuguru:

Replying to hakre:

Replying to wpmuguru:

Thanks for providing a patch, it's missing wp-includes/media.php, right? Any ideas how to deal with it?

I did the patch for the wp-admin/* area. I would not expect an admin_area_ filter to be used in any of the files in wp-includes. If there is something you feel needs to be patched in wp-includes, a second patch to files in wp-includes won't interfere with the one I've uploaded above.

I have one patch provided already, probably it's usefull to combine the constants with the filter. So it's possible to configure the maximum memory allowed via wp-config.php and give plugins some flexibility as well. What do you think?

[wpmuguru]

Replying to hakre:

What do you think?

On constants in general, as far as WP is concerned, less is better. I'm primarily a plugin developer and feel that the strength of WP as a platform comes from the hook and filter system which allow for essentially an infinite number of unique installations/applications of the core codebase.

The only instances where I'm in favor of constants are

• when they are necessary - WP_CACHE allows various types of caching to be loaded instead of the built in cache and the object cache is a prerequisite to the hook & filter system
• when they turn on a component - WP_ALLOW_MULTISITE turns on the network screen

[hakre]

13847.patch refreshed against current trunk

[hakre]

with admin_memory_filter in all admin includes and constants, refreshed against trunk

[hakre]

Missing filter for media.php - image_memory_limit

[hakre]

Same patch with the magic number '256M' but w/o the WP_MAX_MEMORY_LIMIT constant.

[hakre]

Informative: ​Replace Magic Number with Symbolic Constant.

[hakre]

Fixed 13847.5.patch

[hakre]

Overview of patches:

The following patches are working against current trunk and are covering all memory_limit locations with the '256M' value:

• 13847.2.patch​ - '256M' replaced with WP_MAX_MEMORY_LIMIT constant
• 13847.6.patch​ - '256M' fully filtered through 'admin_memory_limit' and 'image_memory_limit'
• 13847.4.patch​ - '256M' replaced with constant AND fully filtered (so two in one)

Using a constant will make the memory limit configure-able.

Using a filter will make the memory limit control-able by plugins.

[markjaquith]

This sounds like a reasonable idea. I'd go with the .4 patch.

[lloydbudd]

+1 to .4 patch

Confirmed issue still present in trunk r17731 (3.2-bleeding) when trying to do a large import today with memory set to larger than 256M.

PHP is sure a special language when it makes it interesting to figure out whether the code is increasing or decreasing the memory.

[westi]

Agreed .4 looks like the way to go.

[westi]

(In [17749]) Introduce WP_MAX_MEMORY_LIMIT constant for the high memory limit we set when image processing and unzipping.
Ensure it is always filterable by plugins as well as configurable in wp-config
Fixes #13847 props hakre