Sporadic timeout /wp-includes/gettext.php

[WhitewolfFox]

When accessing any site in WP, the following message occurs sporadically:

==============================
Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in /home/Whitewolf_Fox/public_html/wordpress/wp-includes/gettext.php on line 85

Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in /home/Whitewolf_Fox/public_html/wordpress/wp-includes/gettext.php on line 85

Fatal error: Maximum execution time of 30 seconds exceeded in /home/Whitewolf_Fox/public_html/wordpress/wp-includes/gettext.php on line 159
==============================

I'm running Wordpress with de_DE language Pack. Version is the latest 2.3.2 .

I've found some blogs which describe the same Problems. A few provide a list of what they have tried to fix the error, but all seem to had only temporary success.

When I restart my MySQL Server and Apache2 Webserver, everything works well again. But that can't be the final resolution ;)

[darkdragon]

1. Disable your plugins. What happens?
2. Disable your language pack. What happens?
3. Switch to a new language pack. What happens?
4. Switch back to your language pack. What happens?

[WhitewolfFox]

1. Done.
2. Done.

Because it's a very sporadic error, I have to wait a few days.

I'll give Feedback then.

[darkdragon]

Well, this actually belongs on the forums, so don't be surprised if this is closed as invalid.

It is too bad that there isn't a way to profile your setup to see where it is halting. Well, actually there is. If you download your site, download AMP system, I recommend WAMP 2.0 for Windows. You can get the XDebug extension and follow the directions to turn the profiler on. You can then download another (free) application that will take you step by step. Or you can just upload the results of the profiler and someone else can figure it out.

The rest is just some general troubleshooting in case the above four don't help.

---

Have you checked with your host on what sort of file system you are on? There have been reports that some file systems have a longer file load time, which might be what is causing the delay.

Also, the issue is not with pack or gettext (most likely) but some other process before it. In the case of your web site, I would think you would immediately know that if your web site doesn't load in less than a second then something is wrong and it might time out.

Also, the reason it might be sporadic is, if you are on a shared host, if other processes on that host might be taking more CPU time than they should causing your processes (PHP) to take longer to perform.

[darkdragon]

I'm setting the priority and severity back to normal since not everyone is having this problem.

[WhitewolfFox]

I ran WP since 01/07/08 now in English without any plugin loaded. I hadn't faced those errors anymore.
I activated ALL the Plugins but keept English language a few Days ago, but still didn't face any error.
I installed xdebug today and reactivated the German language for Wordpress.

I'll wait a little more days if the error still occours. I hope I can give some more informations then.

[WhitewolfFox]

... ok, that didn't take log.

Ca. 3 hours after I switched to german language, the error occours. I archived the xdebug site and put that to: ​http://www.zoosau.de/wp_error/ .

I switched back tu English allready.

Can somebody help me with this please? I don't understand what I see there.

[nbachiyski]

I have a couple of questions too:

1. What operating system/architecture is your host using?
2. Could you give us your mo file?

It looks like the gettext library expects some more bytes, which aren't there either because of a system issue or erroneous mo file.

[darkdragon]

The system issue is that there is a timeout and it is taking to long. It is unlikely that the first two errors mean anything other than they are what PHP was executing when it halted execution of the script.

This is starting to appear to me like you have disk issues, meaning you are on a host that has slower disk times because of NFS or some other (mis)configuration. The reason the English is running fine is that there is no translation file being pulled in.

To confirm my theory, you would need to use another translation file about the same size as the German translation and see if it is causing a timeout. If it is, then it is a disk/host issue and not a WordPress issue.

If my theory is correct, then please close this was invalid, if not, then further feedback will be needed.

[WhitewolfFox]

@nbachiyski:

I'm running a Debian lenny (4.0r1 Testing) with PHP 5.2.4-2 and apache 2.2.6-1 .
My mo File is here: ​http://www.zoosau.de/wp_error/de_DE.mo

@darkdragon:

Sorry, that doesn't fit for my System :)
It's not an filesystem issue. Here's a quick performance proof:

root@h1322702:~ # dd if=/dev/urandom of=randomdata bs=1024 count=100000
100000+0 Datensätze ein
100000+0 Datensätze aus
102400000 Bytes (102 MB) kopiert, 44,14 Sekunden, 2,3 MB/s
root@h1322702:~ # time cp randomdata randomdata2

real 0m1.275s
user 0m0.040s
sys 0m1.072s
root@h1322702:~ #

Should be enough for pulling in a 174K sized .mo file .

[darkdragon]

Replying to WhitewolfFox:

@darkdragon:

Sorry, that doesn't fit for my System :)
It's not an filesystem issue. Here's a quick performance proof:

Then the problem is outside the scope of my knowledge and patience. Good luck to you and nbachiyski (who I believe is fairly competent in localization issues).

The only other recommendation I would give is to profile. Which can be accomplished by doing the following:

Add the following to php.ini file:

xdebug.profiler_enable = 1
xdebug.profiler_output_dir = /Full/Directory/Path

Then use ​kcachegrind to see where bottlenecks are. If you can't find anything, then just do the same thing as the .mo file and I or someone else will see where it is.

[codestyling]

class StringReader {

var $_pos;
var $_str;

function StringReader($str=) {

$this->_str = $str;
$this->_pos = 0;

echo "StringReader(init)<br />";

}

function read($bytes) {

var_dump($this->_pos);

var_dump($this->_str);
exit;

results only in output prior to exit:

NULL string(226673) "ޕ� ........

I think, the class CachedFileReader which extends StringReader for *.mo file handling, does not initialize the $this->_pos member! At stress cases with optimized bytecode caching and heavy amount of requests simultanously, the member may be set to random value from requests beside and the page generation call shows above behavoir.
I only use cgi versions without optimizer, but i'm afraid, that mod_php in combination with zend or other may force this described behavoir

[westi]

(In [8117]) Ensure we initialise the position variable. See #5599 props codestyling.

[westi]

Maybe that will fix this heisenbug.

Please try that fix - you can happily take the new streams.php and use it with 2.5.1 and let us know how you get on.

[codestyling]

The problem has been detected and is very strange: The PHP function substr() returns sometimes a string, that has an additional offset of 1 byte!
I have traced this at an affected system, where this randomly occures and wrote an extra logfile and detection code. Here comes the issue:

de_DE.mo (Filestart, hex):
DE 12 04 95 00 00 00 00 5E 08 00 00 1C 00 00 00 

int: -1794895138     -> hex: FFFFFFFF950412DE  (byte order mark, ok)
int: 1577058304      -> hex: 5E000000 (revision, failed! 1 byte additional offset)
int: 469762056       -> hex: 1C000008 (string count, failed!)
int: 201326592       -> hex: 0C000000 (offset original table, failed!)
int: 687865923       -> hex:29000043 (offset translation table, failed!)

This leads to this bad behavoir, because all integers are wrong and too big for file size.

I have written a fix inside streams.php and did replace substr() call to unpack() call and now it works at the tested machine as expected without any further occurances.

streams.php line: 84

old code:

  function read($bytes) {
    $data = substr($this->_str, $this->_pos, $bytes);
    $this->_pos += $bytes;
    if (strlen($this->_str)<$this->_pos)
      $this->_pos = strlen($this->_str);

    return $data;
  }

new code:

  function read($bytes) {
    $data = array_shift(unpack('@'.$this->_pos.'/A'.$bytes, $this->_str));
    $this->_pos += $bytes;
    if (strlen($this->_str)<$this->_pos)
      $this->_pos = strlen($this->_str);

    return $data;
  }

I think, this should be also reported to PHP core team for further investigation.
I have tested this patch successful using de_DE.mo / ar.mo (arabic for rtl purpose) and ja.mo (japanees for extra unicode space).

[Jnet]

I got similar problems for several of my clients using a .mo-file. I tried the fix suggested by ​codestyling and ​#3780, both without success. System/Wordpress configs:

° Wordpress 2.2.1 and 2.5
° PHP 5.2.5 and PHP 5.2.6
° Apache 2.0.59
° Intel Xeon 2.80GHz (i386 architecture), Intel Xeon 3.00GHz Dual Core (i386 architecture) and Intel Xeon 2.50GHz Quad Core (x86_64 architecture)
° CentOS 4

At this moment, disabling the language setting with

define ('WPLANG', '');

is the only thing that prevents the timeout.

Anyone?

[codestyling]

Replying to Jnet:

I got similar problems for several of my clients using a .mo-file. I tried the fix suggested by ​codestyling and ​#3780, both without success.

What does similar meen?
Did you get unpack() errors anyway or only max runtime exceeded ?

btw: This kind of issue have been tracked down over several blogs and i found similarities:

• seems to require Apache 2.x Server (1.x not affected)
• seems to require PHP 5 Version (PHP 4 not affected)
• seems to require mod_php (no occurance at cgi versions)
• seems to require 64Bit Architecture (even if running 32Bit Unix/Linux)
• seems to require dedicated hoster (massively occured at hoster all-inkl.com)
• seems to require more traffic as normal to happen

The tested system affected is hosted at all-inkl.com and runs at

• Linux dd10606 2.6.24.4-nmm2 SMP Tue Apr 15 18:13:27 CEST 2008 i686
• PHP 5.2.5
• Apache APi Version: 20051115
• PHP API 20041225
• PHP Extension 20060613
• Zend Extension 220060519
• protected with the Suhosin Patch 0.9.6.2
• additional modules: ionCube Loader / sysvsem / sysvshm

At this server the patch above does work, but it's not the full solution i'm afraid.
It's not clear, why sporadically this substr() bug occures. Mostly this server runs well and uses correct the *.mo-files but under pressure it happens but very seldom.
I also read reports, that server may fails at every attempt to read any *.mo-file, so they have only the chance to disable usage and fall back to english.
I assume, that this could also be related to SMP system (multi core's) that are not qualified supported at PHP ?

[Jnet]

Replying to codestyling:

Replying to Jnet:

I got similar problems for several of my clients using a .mo-file. I tried the fix suggested by ​codestyling and ​#3780, both without success.

What does similar meen?
Did you get unpack() errors anyway or only max runtime exceeded ?

The unpack() errors. It is indeed only affecting servers running PHP5 (we got the troubles since we upgraded from PHP4).

[codestyling]

I have changed version from WP 2.3.2 to WP 2.5.1 because it's massively happens there now (last weeks, while hoster upgrade their PHP4 to PHP5 versions). Some environments has been reported, that won't run in any case with mo-files and have to disable usage of translation files!

[codestyling]

new error occurance:
At german WordPress Forum ​http://forum.wordpress-deutschland.org/plugins-und-widgets/36304-nach-update-auf-akismet-2-1-6-fehler.html#post175030 a user has reported, that this error occures immediately after upgrade to Akismet 2.1.6! After deactivation of Akismet all works well, if Akismet will be activated the well know error dump occures:

Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in .../news_blog/wp-includes/gettext.php on line 91
Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in .../news_blog/wp-includes/gettext.php on line 91
Fatal error: Maximum execution time of 30 seconds exceeded in .../news_blog/wp-includes/gettext.php on line 163

I gave the user the advice to my patch suggested above and he replaced his streams.php with my downloadable patched one for WP2.5.1. After using the patch, the error doesn't occure any longer.
The thing is: Why forces upgrade of Akismet such a behavoir too ?
I think, there must be a big problem inside some PHP5 versions implementation of substr(). Should this be for sure prevented at WP using unpack instead of substr() ?

[codestyling]

bugfix for streams.php file (mbstrings.func_overload issue)

[codestyling]

I have found the reason behind this bug (sporadically or permanent occurance). The detailed description can be found at my article: ​http://www.code-styling.de/english/wordpress-error-during-translation-file-usage-the-reason

short conclusion: The crash will be force by usage of php.ini value mbstring.func_overload. The streams.php is not able to handle this. I have attached the patched complete file also can be downloaded at my page. Has been successfully tested at WP 2.5.x and WP 2.6.0

If you have read my article and inspected the patch, you hopefully agree, that this must be fixed as soon as possible.

[westi]

I think the style of this diff could be improved by removing code duplication.

New patch imcoming

[westi]

Rejigged patch

[ryan]

Tested 5599.alternate.diff with mbstring.func_overload unset and with it set to 2. I tested with a few LTR and RTL languages. Looks good. Also confirmed that without the patch mbstring.func_overload = 2 results in the fatal error.

[ryan]

(In [8420]) Work around fatal error caused by mbstring.func_overload = 2. Props codestyling . fixes #5599 for trunk

[ryan]

(In [8421]) Work around fatal error caused by mbstring.func_overload = 2. Props codestyling . fixes #5599 for 2.6

[codestyling]

This bug is back but on an other reason.
I tested the PHP5 XAMPP Installation (PHP 5.1.4 and higher) and set the php.ini value

; Magic quotes for runtime-generated data, e.g. data from SQL, from exec(), etc.
magic_quotes_runtime = On

to "On" instead of normally "Off". After restart of Server, i got immediatly the descibed bug too, again 30 seconds limit reached and crash at line 91.
After inspectation of whole WP 2.7 code i found only import / mailer and pclzip do respect this by override the value.
After testing the additional code line

set_magic_quotes_runtime(0);

at wp-config.php the issue was gone. I suggest to include this as first global instruction at settings.php to avoid this crash at the future. (At PHP6 this value has been trashed and can't generate any more trouble)

Furthermore, if the german 'de_DE' WPLANG setting (or any other language file i tested like he_IL or it_IT) has been set empty, it steps over but the install won't work nervertheless, because the database requested values at capabilities.php gots damaged too:

PHP Fatal error: Cannot use string offset as an array in C:\xampp\root\wordpress\wp-includes\capabilities.php on line 116, referrer: \wordpress\wp-login.php

The database role data has been damaged and produces this "white page during install problem"! All this can be solved by adding one line of code switching this value "off".

[nbachiyski]

Can you please check if the issue is still occuring, in the light of ​changeset 10286?

[codestyling]

The patch set contains the nessessary set_magic_quotes_runtime(0); at line 18 newly introduced an will be executed prior to any attempt to load a language file. I have tested it at my system and it works as expected (related to language files).
I assume, that the secondary issue i found has been fixed with the bugfix the patchset stands for.
This can be closed from prospective of language files.