Make WordPress Core

Opened 16 years ago

Closed 15 years ago

Last modified 9 years ago

#5599 closed defect (bug) (worksforme)

Sporadic timeout /wp-includes/gettext.php

Reported by: whitewolffox's profile WhitewolfFox Owned by: westi's profile westi
Milestone: Priority: highest omg bbq
Severity: blocker Version: 2.5.1
Component: General Keywords: gettext unpack() timeout crash install
Focuses: Cc:

Description

When accessing any site in WP, the following message occurs sporadically:

==============================
Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in /home/Whitewolf_Fox/public_html/wordpress/wp-includes/gettext.php on line 85

Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in /home/Whitewolf_Fox/public_html/wordpress/wp-includes/gettext.php on line 85

Fatal error: Maximum execution time of 30 seconds exceeded in /home/Whitewolf_Fox/public_html/wordpress/wp-includes/gettext.php on line 159
==============================

I'm running Wordpress with de_DE language Pack. Version is the latest 2.3.2 .

I've found some blogs which describe the same Problems. A few provide a list of what they have tried to fix the error, but all seem to had only temporary success.

When I restart my MySQL Server and Apache2 Webserver, everything works well again. But that can't be the final resolution ;)

Attachments (4)

streams.php-WP2.5.x_2.6.0.zip (2.0 KB) - added by codestyling 16 years ago.
bugfix for streams.php file (mbstrings.func_overload issue)
5599.diff (3.5 KB) - added by ryan 16 years ago.
5599.2.diff (2.3 KB) - added by ryan 16 years ago.
5599.alternate.diff (2.2 KB) - added by westi 16 years ago.
Rejigged patch

Download all attachments as: .zip

Change History (32)

#1 @darkdragon
16 years ago

  1. Disable your plugins. What happens?
  2. Disable your language pack. What happens?
  3. Switch to a new language pack. What happens?
  4. Switch back to your language pack. What happens?

#2 @WhitewolfFox
16 years ago

  1. Done.
  2. Done.

Because it's a very sporadic error, I have to wait a few days.

I'll give Feedback then.

#3 @darkdragon
16 years ago

Well, this actually belongs on the forums, so don't be surprised if this is closed as invalid.

It is too bad that there isn't a way to profile your setup to see where it is halting. Well, actually there is. If you download your site, download AMP system, I recommend WAMP 2.0 for Windows. You can get the XDebug extension and follow the directions to turn the profiler on. You can then download another (free) application that will take you step by step. Or you can just upload the results of the profiler and someone else can figure it out.

The rest is just some general troubleshooting in case the above four don't help.


Have you checked with your host on what sort of file system you are on? There have been reports that some file systems have a longer file load time, which might be what is causing the delay.

Also, the issue is not with pack or gettext (most likely) but some other process before it. In the case of your web site, I would think you would immediately know that if your web site doesn't load in less than a second then something is wrong and it might time out.

Also, the reason it might be sporadic is, if you are on a shared host, if other processes on that host might be taking more CPU time than they should causing your processes (PHP) to take longer to perform.

#4 @darkdragon
16 years ago

  • Priority changed from high to normal
  • Severity changed from blocker to normal

I'm setting the priority and severity back to normal since not everyone is having this problem.

#5 @WhitewolfFox
16 years ago

I ran WP since 01/07/08 now in English without any plugin loaded. I hadn't faced those errors anymore.
I activated ALL the Plugins but keept English language a few Days ago, but still didn't face any error.
I installed xdebug today and reactivated the German language for Wordpress.

I'll wait a little more days if the error still occours. I hope I can give some more informations then.

#6 @WhitewolfFox
16 years ago

... ok, that didn't take log.

Ca. 3 hours after I switched to german language, the error occours. I archived the xdebug site and put that to: http://www.zoosau.de/wp_error/ .

I switched back tu English allready.

Can somebody help me with this please? I don't understand what I see there.

#7 @nbachiyski
16 years ago

I have a couple of questions too:

  1. What operating system/architecture is your host using?
  2. Could you give us your mo file?

It looks like the gettext library expects some more bytes, which aren't there either because of a system issue or erroneous mo file.

#8 @darkdragon
16 years ago

The system issue is that there is a timeout and it is taking to long. It is unlikely that the first two errors mean anything other than they are what PHP was executing when it halted execution of the script.

This is starting to appear to me like you have disk issues, meaning you are on a host that has slower disk times because of NFS or some other (mis)configuration. The reason the English is running fine is that there is no translation file being pulled in.

To confirm my theory, you would need to use another translation file about the same size as the German translation and see if it is causing a timeout. If it is, then it is a disk/host issue and not a WordPress issue.

If my theory is correct, then please close this was invalid, if not, then further feedback will be needed.

#9 follow-up: @WhitewolfFox
16 years ago

@nbachiyski:

I'm running a Debian lenny (4.0r1 Testing) with PHP 5.2.4-2 and apache 2.2.6-1 .
My mo File is here: http://www.zoosau.de/wp_error/de_DE.mo

@darkdragon:

Sorry, that doesn't fit for my System :)
It's not an filesystem issue. Here's a quick performance proof:

root@h1322702:~ # dd if=/dev/urandom of=randomdata bs=1024 count=100000
100000+0 Datensätze ein
100000+0 Datensätze aus
102400000 Bytes (102 MB) kopiert, 44,14 Sekunden, 2,3 MB/s
root@h1322702:~ # time cp randomdata randomdata2

real 0m1.275s
user 0m0.040s
sys 0m1.072s
root@h1322702:~ #

Should be enough for pulling in a 174K sized .mo file .

#10 in reply to: ↑ 9 @darkdragon
16 years ago

Replying to WhitewolfFox:

@darkdragon:

Sorry, that doesn't fit for my System :)
It's not an filesystem issue. Here's a quick performance proof:

Then the problem is outside the scope of my knowledge and patience. Good luck to you and nbachiyski (who I believe is fairly competent in localization issues).

The only other recommendation I would give is to profile. Which can be accomplished by doing the following:

Add the following to php.ini file:

xdebug.profiler_enable = 1
xdebug.profiler_output_dir = /Full/Directory/Path

Then use kcachegrind to see where bottlenecks are. If you can't find anything, then just do the same thing as the .mo file and I or someone else will see where it is.

#11 @codestyling
16 years ago

  • Priority changed from normal to highest omg bbq

class StringReader {

var $_pos;
var $_str;

function StringReader($str=) {

$this->_str = $str;
$this->_pos = 0;

echo "StringReader(init)<br />";

}

function read($bytes) {

var_dump($this->_pos);

var_dump($this->_str);
exit;

results only in output prior to exit:

NULL string(226673) "ޕ� ........

I think, the class CachedFileReader which extends StringReader for *.mo file handling, does not initialize the $this->_pos member! At stress cases with optimized bytecode caching and heavy amount of requests simultanously, the member may be set to random value from requests beside and the page generation call shows above behavoir.
I only use cgi versions without optimizer, but i'm afraid, that mod_php in combination with zend or other may force this described behavoir

#12 @westi
16 years ago

(In [8117]) Ensure we initialise the position variable. See #5599 props codestyling.

#13 @westi
16 years ago

  • Owner changed from anonymous to westi
  • Status changed from new to assigned

Maybe that will fix this heisenbug.

Please try that fix - you can happily take the new streams.php and use it with 2.5.1 and let us know how you get on.

#14 @codestyling
16 years ago

  • Severity changed from normal to critical

The problem has been detected and is very strange: The PHP function substr() returns sometimes a string, that has an additional offset of 1 byte!
I have traced this at an affected system, where this randomly occures and wrote an extra logfile and detection code. Here comes the issue:

de_DE.mo (Filestart, hex):
DE 12 04 95 00 00 00 00 5E 08 00 00 1C 00 00 00 

int: -1794895138     -> hex: FFFFFFFF950412DE  (byte order mark, ok)
int: 1577058304      -> hex: 5E000000 (revision, failed! 1 byte additional offset)
int: 469762056       -> hex: 1C000008 (string count, failed!)
int: 201326592       -> hex: 0C000000 (offset original table, failed!)
int: 687865923       -> hex:29000043 (offset translation table, failed!)

This leads to this bad behavoir, because all integers are wrong and too big for file size.

I have written a fix inside streams.php and did replace substr() call to unpack() call and now it works at the tested machine as expected without any further occurances.

streams.php line: 84

old code:

  function read($bytes) {
    $data = substr($this->_str, $this->_pos, $bytes);
    $this->_pos += $bytes;
    if (strlen($this->_str)<$this->_pos)
      $this->_pos = strlen($this->_str);

    return $data;
  }

new code:

  function read($bytes) {
    $data = array_shift(unpack('@'.$this->_pos.'/A'.$bytes, $this->_str));
    $this->_pos += $bytes;
    if (strlen($this->_str)<$this->_pos)
      $this->_pos = strlen($this->_str);

    return $data;
  }

I think, this should be also reported to PHP core team for further investigation.
I have tested this patch successful using de_DE.mo / ar.mo (arabic for rtl purpose) and ja.mo (japanees for extra unicode space).

#15 follow-up: @Jnet
16 years ago

I got similar problems for several of my clients using a .mo-file. I tried the fix suggested by codestyling and #3780, both without success. System/Wordpress configs:

° Wordpress 2.2.1 and 2.5
° PHP 5.2.5 and PHP 5.2.6
° Apache 2.0.59
° Intel Xeon 2.80GHz (i386 architecture), Intel Xeon 3.00GHz Dual Core (i386 architecture) and Intel Xeon 2.50GHz Quad Core (x86_64 architecture)
° CentOS 4

At this moment, disabling the language setting with

define ('WPLANG', '');

is the only thing that prevents the timeout.

Anyone?

#16 in reply to: ↑ 15 ; follow-up: @codestyling
16 years ago

Replying to Jnet:

I got similar problems for several of my clients using a .mo-file. I tried the fix suggested by codestyling and #3780, both without success.

What does similar meen?
Did you get unpack() errors anyway or only max runtime exceeded ?

btw: This kind of issue have been tracked down over several blogs and i found similarities:

  • seems to require Apache 2.x Server (1.x not affected)
  • seems to require PHP 5 Version (PHP 4 not affected)
  • seems to require mod_php (no occurance at cgi versions)
  • seems to require 64Bit Architecture (even if running 32Bit Unix/Linux)
  • seems to require dedicated hoster (massively occured at hoster all-inkl.com)
  • seems to require more traffic as normal to happen

The tested system affected is hosted at all-inkl.com and runs at

  • Linux dd10606 2.6.24.4-nmm2 SMP Tue Apr 15 18:13:27 CEST 2008 i686
  • PHP 5.2.5
  • Apache APi Version: 20051115
  • PHP API 20041225
  • PHP Extension 20060613
  • Zend Extension 220060519
  • protected with the Suhosin Patch 0.9.6.2
  • additional modules: ionCube Loader / sysvsem / sysvshm

At this server the patch above does work, but it's not the full solution i'm afraid.
It's not clear, why sporadically this substr() bug occures. Mostly this server runs well and uses correct the *.mo-files but under pressure it happens but very seldom.
I also read reports, that server may fails at every attempt to read any *.mo-file, so they have only the chance to disable usage and fall back to english.
I assume, that this could also be related to SMP system (multi core's) that are not qualified supported at PHP ?

#17 in reply to: ↑ 16 @Jnet
16 years ago

Replying to codestyling:

Replying to Jnet:

I got similar problems for several of my clients using a .mo-file. I tried the fix suggested by codestyling and #3780, both without success.

What does similar meen?
Did you get unpack() errors anyway or only max runtime exceeded ?

The unpack() errors. It is indeed only affecting servers running PHP5 (we got the troubles since we upgraded from PHP4).

#18 @codestyling
16 years ago

  • Version changed from 2.3.2 to 2.5.1

I have changed version from WP 2.3.2 to WP 2.5.1 because it's massively happens there now (last weeks, while hoster upgrade their PHP4 to PHP5 versions). Some environments has been reported, that won't run in any case with mo-files and have to disable usage of translation files!

#19 @codestyling
16 years ago

new error occurance:
At german WordPress Forum http://forum.wordpress-deutschland.org/plugins-und-widgets/36304-nach-update-auf-akismet-2-1-6-fehler.html#post175030 a user has reported, that this error occures immediately after upgrade to Akismet 2.1.6! After deactivation of Akismet all works well, if Akismet will be activated the well know error dump occures:

Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in .../news_blog/wp-includes/gettext.php on line 91
Warning: unpack() [function.unpack]: Type V: not enough input, need 4, have 0 in .../news_blog/wp-includes/gettext.php on line 91
Fatal error: Maximum execution time of 30 seconds exceeded in .../news_blog/wp-includes/gettext.php on line 163

I gave the user the advice to my patch suggested above and he replaced his streams.php with my downloadable patched one for WP2.5.1. After using the patch, the error doesn't occure any longer.
The thing is: Why forces upgrade of Akismet such a behavoir too ?
I think, there must be a big problem inside some PHP5 versions implementation of substr(). Should this be for sure prevented at WP using unpack instead of substr() ?

@codestyling
16 years ago

bugfix for streams.php file (mbstrings.func_overload issue)

#20 @codestyling
16 years ago

  • Milestone changed from 2.9 to 2.6.1

I have found the reason behind this bug (sporadically or permanent occurance). The detailed description can be found at my article: http://www.code-styling.de/english/wordpress-error-during-translation-file-usage-the-reason

short conclusion: The crash will be force by usage of php.ini value mbstring.func_overload. The streams.php is not able to handle this. I have attached the patched complete file also can be downloaded at my page. Has been successfully tested at WP 2.5.x and WP 2.6.0

If you have read my article and inspected the patch, you hopefully agree, that this must be fixed as soon as possible.

@ryan
16 years ago

@ryan
16 years ago

#21 @westi
16 years ago

I think the style of this diff could be improved by removing code duplication.

New patch imcoming

@westi
16 years ago

Rejigged patch

#22 @ryan
16 years ago

Tested 5599.alternate.diff with mbstring.func_overload unset and with it set to 2. I tested with a few LTR and RTL languages. Looks good. Also confirmed that without the patch mbstring.func_overload = 2 results in the fatal error.

#23 @ryan
16 years ago

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [8420]) Work around fatal error caused by mbstring.func_overload = 2. Props codestyling . fixes #5599 for trunk

#24 @ryan
16 years ago

(In [8421]) Work around fatal error caused by mbstring.func_overload = 2. Props codestyling . fixes #5599 for 2.6

#25 @codestyling
15 years ago

  • Cc info@… added; whitewolf_fox@… removed
  • Keywords crash install added
  • Milestone changed from 2.6.1 to 2.7.1
  • Resolution fixed deleted
  • Severity changed from critical to blocker
  • Status changed from closed to reopened

This bug is back but on an other reason.
I tested the PHP5 XAMPP Installation (PHP 5.1.4 and higher) and set the php.ini value

; Magic quotes for runtime-generated data, e.g. data from SQL, from exec(), etc.
magic_quotes_runtime = On

to "On" instead of normally "Off". After restart of Server, i got immediatly the descibed bug too, again 30 seconds limit reached and crash at line 91.
After inspectation of whole WP 2.7 code i found only import / mailer and pclzip do respect this by override the value.
After testing the additional code line

set_magic_quotes_runtime(0);

at wp-config.php the issue was gone. I suggest to include this as first global instruction at settings.php to avoid this crash at the future. (At PHP6 this value has been trashed and can't generate any more trouble)

Furthermore, if the german 'de_DE' WPLANG setting (or any other language file i tested like he_IL or it_IT) has been set empty, it steps over but the install won't work nervertheless, because the database requested values at capabilities.php gots damaged too:

PHP Fatal error: Cannot use string offset as an array in C:\xampp\root\wordpress\wp-includes\capabilities.php on line 116, referrer: \wordpress\wp-login.php

The database role data has been damaged and produces this "white page during install problem"! All this can be solved by adding one line of code switching this value "off".

#26 @nbachiyski
15 years ago

Can you please check if the issue is still occuring, in the light of changeset 10286?

#27 @codestyling
15 years ago

  • Resolution set to worksforme
  • Status changed from reopened to closed

The patch set contains the nessessary set_magic_quotes_runtime(0); at line 18 newly introduced an will be executed prior to any attempt to load a language file. I have tested it at my system and it works as expected (related to language files).
I assume, that the secondary issue i found has been fixed with the bugfix the patchset stands for.
This can be closed from prospective of language files.

#28 @DrewAPicture
9 years ago

  • Milestone 2.7.1 deleted
Note: See TracTickets for help on using tickets.