gettext fails to determine byteorder on 64bit systems with php5.2.1

[abtime]

After upgrading to php5.2.1 on FreeBSD6.2-amd64 the function in wp-includes/gettext.php fails to determine the correct byteorder, so the language (.mo) file is not loaded.

The problem possibly is around line 114 of ​gettext.php

[abtime]

Replying to abtime:

After upgrading to php5.2.1 on FreeBSD6.2-amd64 the function in wp-includes/gettext.php fails to determine the correct byteorder, so the language (.mo) file is not loaded.

The problem possibly is around line 114 of ​gettext.php

This seems to be a working fix (works for me): ​http://wordpress.org/support/topic/63038

[rob1n]

Added a patch from the fix detailed in ​http://wordpress.org/support/topic/63038

Testing needed, and a dev to check over the code, as all I did was adapt the fix's code.

[ryan]

(In [4917]) gettext byteorder fix for 64 bit archs. fixes #3780

[JanRei]

This fix should be in Wordpress 2.2 now, but I have still the same problem. However, it works if I apply the changes suggested in ​http://wordpress.org/support/topic/63038.

[marcinkk]

I have the same problem. FreeBSD 6.2 on AMD64 with PHP5.2.1, WordPress localization with 2.2 original code doesn't work :(

Known code from gettext.php in WordPress 2.2:

// $MAGIC1 = (int)0x950412de; //bug in PHP 5.0.2, see https://savannah.nongnu.org/bugs/?func=detailitem&item_id=10565
$MAGIC1 = (int) - 1794895138;
// $MAGIC2 = (int)0xde120495; //bug
$MAGIC2 = (int) - 569244523;
// 64-bit fix
$MAGIC3 = (int) 2500072158;

$magic = $this->readint();

Results on my system:

echo $magic;				Result:	-1794895138
echo $MAGIC1;					-1794895138
echo $MAGIC2;					-569244523
echo $MAGIC3;					2500072158
echo $MAGIC1 & 0xFFFFFFFF;			2500072158
echo $MAGIC2 & 0xFFFFFFFF;			3725722773
echo $MAGIC3 & 0xFFFFFFFF;			2500072158
echo (int)0x950412de;				2500072158
echo (int)0xde120495;				3725722773

So the results on all comparisons was 0, so I modified functions:

if ($magic == ($MAGIC1 & 0xFFFFFFFF) || ($magic == ($MAGIC3 & 0xFFFFFFFF)) { // to make sure it works for 64-bit platforms
    $this->BYTEORDER = 0;
} elseif ($magic == ($MAGIC2 & 0xFFFFFFFF)) {
    $this->BYTEORDER = 1;
} else {
    $this->BYTEORDER = 0;
    //$this->error = 1; // not MO file
    //return false;
}

And localization worked properly. In above code magic value is not checked but I was sure that BYTEORDER should be 0. So next I made:

if (($magic & 0xFFFFFFFF) == ($MAGIC1 & 0xFFFFFFFF) || ($magic & 0xFFFFFFFF) == ($MAGIC3 & 0xFFFFFFFF)) { // to make sure it works for 64-bit platforms
    $this->BYTEORDER = 0;
} elseif (($magic & 0xFFFFFFFF) == ($MAGIC2 & 0xFFFFFFFF)) {
    $this->BYTEORDER = 1;
} else {
    $this->error = 1; // not MO file
    return false;
}

It works too and something is checked ... but I don'y know what this magic numbers do, so somebody better than me should check the results...

[hakre]

Just two questions:

1. How did you guys compiled PHP, does it support 64 Bit integers?
2. Maybe this helps? ​http://www.gerd-riesselmann.net/archives/2006/01/a-64-bit-integer-for-php

[nbachiyski]

Do we really need the bitwise operators? All the numbers are 4-byte ints anyway.

I have tested the patch gettext-64-without-bitwise-ops.diff and it works on:

• Linux x86_64, PHP 5.2.3
• Linux i686, PHP 5.2.3
• Linux i686, PHP 4.3.10
• FreeBSD i386, PHP 4.4.0

[westi]

Replying to nbachiyski:

Do we really need the bitwise operators? All the numbers are 4-byte ints anyway.

The problem is that on a 64bit platform with a PHP compiled for 64bit (not a 32bit PHP running on a 64bit platform) the php int is 8 bytes instead on 4 bytes.

The bitwise operators are there to mask off the top of the 8-byte int on 64bit platforms.

I have tested the patch gettext-64-without-bitwise-ops.diff and it works on:

• Linux x86_64, PHP 5.2.3
• Linux i686, PHP 5.2.3
• Linux i686, PHP 4.3.10
• FreeBSD i386, PHP 4.4.0

what it the value of PHP_INT_SIZE on the x86_64 PHP install - I expect it is 4 which why all your tests worked.

[nbachiyski]

Nope, westi, PHP_INT_SIZE is 8. And in that case $magic is equal to $MAGIC1, whihc is strange. However because of the 64-bit zero offsets we we have $MAGIC3, so we shouldn't care about removing the zeroes.

Maybe we should find more testers.

[ophy]

(on the behalf of my friend lapo@…)

On my 64 bit server, with release 2.3:

magic=ffffffff950412de
MAGIC1=ffffffff950412de
MAGIC2=ffffffffde120495
MAGIC3=950412de

So the fix with MAGIC3 doesn't work and MAGIC1 would work if it was not truncated to 32bit itself.
I've made it working with the following patch:

--- wp-includes/gettext.php.orig        Mon Oct 22 22:35:56 2007
+++ wp-includes/gettext.php     Mon Oct 22 22:50:38 2007
@@ -113,7 +113,7 @@
                $MAGIC3 = (int) 2500072158;

                $this->STREAM = $Reader;
-               $magic = $this->readint();
+               $magic = $this->readint() & 0xFFFFFFFF;

                if ($magic == ($MAGIC1 & 0xFFFFFFFF) || $magic == ($MAGIC3 & 0xFFFFFFFF)) { // to make sure it works for 64-bit platforms
                        $this->BYTEORDER = 0;

[thenlich]

This bug is still present in Wordpress 2.3 on Linux/amd64/PHP 5.2.3

It worked for me to remove the & 0xFFFFFFFF's. Solution is described here: ​http://faq.wordpress-deutschland.org/wieso-zeigt-wordpress-trotz-sprachdatei-englischen-text-an/

[thenlich]

Addition: Bug still present in 2.3.1 also.

[nbachiyski]

Could everybody, please, try the both patches and report if any is working.

[qb]

Replying to nbachiyski:

Could everybody, please, try the both patches and report if any is working.

Both patches not working for me on Wordpress 2.3.1 / Linux 64 / PHP 5.2.4

[nyuwec]

Replying to nbachiyski:

Could everybody, please, try the both patches and report if any is working.

I have a gentoo linux on a 64 bit AMD processor, described in: #5356, fighting with the above described problem. PHP using PHP_INT_SIZE=8.

Apply 3780.diff: FAILS.
Apply gettext-64-without-bitwise-ops.diff: WORKS!

As I can see the gettext-64-without-bitwise-ops.diff patches the gettext to the format described in: ​http://wordpress.org/support/topic/63038 That patch also works on both 64bit and on 32 bit systems I can test!

If only the 3780.diff applied to a 32 bit system it still works, applying the gettext-64-without-bitwise-ops.diff: still works!

So this code in the gettext.php (starting at line 105) works for me on both 32 and 64 bit systems:

		// Caching can be turned off
		$this->enable_cache = $enable_cache;

		// $MAGIC1 = (int)0x950412de; //bug in PHP 5.0.2, see https://savannah.nongnu.org/bugs/?func=detailitem&item_id=10565
		$MAGIC1 = (int) - 1794895138;
		// $MAGIC2 = (int)0xde120495; //bug
		$MAGIC2 = (int) - 569244523;
		// 64-bit fix
		$MAGIC3 = (int) 2500072158;

		$this->STREAM = $Reader;
		$magic = $this->readint();
		if ($magic == $MAGIC1 || $magic == $MAGIC3) { // to make sure it works for 64-bit platforms
			$this->BYTEORDER = 0;
		} elseif ($magic == ($MAGIC2 & 0xFFFFFFFF)) {
			$this->BYTEORDER = 1;
		} else {
			$this->error = 1; // not MO file
			return false;
		}

[nyuwec]

To my previous comment, here is a working version of the patched gettext.php on a 64bit AMD processor powered Gentoo Linux (jam 2.6.18-gentoo-r6) server with Apache2 and PHP 5.2.4:
​http://koles.hu/turakraft/

It uses the hungarian language file, so it is mostly unreadable to the most of the world population. :)

[madsjensen]

This works !!! thanks ... I have tested many other hacks but only this seems to works.

Apache version 2.0.61
PHP version 5.2.4
MySQL version 5.0.45
Architecture amd64
Operating system: FreeBSD

Danish localization

cheers mads

Replying to nyuwec:

Replying to nbachiyski:

Could everybody, please, try the both patches and report if any is working.

I have a gentoo linux on a 64 bit AMD processor, described in: #5356, fighting with the above described problem. PHP using PHP_INT_SIZE=8.

Apply 3780.diff: FAILS.
Apply gettext-64-without-bitwise-ops.diff: WORKS!

As I can see the gettext-64-without-bitwise-ops.diff patches the gettext to the format described in: ​http://wordpress.org/support/topic/63038 That patch also works on both 64bit and on 32 bit systems I can test!

If only the 3780.diff applied to a 32 bit system it still works, applying the gettext-64-without-bitwise-ops.diff: still works!

So this code in the gettext.php (starting at line 105) works for me on both 32 and 64 bit systems:

		// Caching can be turned off
		$this->enable_cache = $enable_cache;

		// $MAGIC1 = (int)0x950412de; //bug in PHP 5.0.2, see https://savannah.nongnu.org/bugs/?func=detailitem&item_id=10565
		$MAGIC1 = (int) - 1794895138;
		// $MAGIC2 = (int)0xde120495; //bug
		$MAGIC2 = (int) - 569244523;
		// 64-bit fix
		$MAGIC3 = (int) 2500072158;

		$this->STREAM = $Reader;
		$magic = $this->readint();
		if ($magic == $MAGIC1 || $magic == $MAGIC3) { // to make sure it works for 64-bit platforms
			$this->BYTEORDER = 0;
		} elseif ($magic == ($MAGIC2 & 0xFFFFFFFF)) {
			$this->BYTEORDER = 1;
		} else {
			$this->error = 1; // not MO file
			return false;
		}

[nbachiyski]

I got feedback from more than 10 Bulgarian users on 64-bit servers (Linux and various BSDs), experiencing the same problem. For all of them the second patch worked well, so I guess it is quite safe to be committed.

[westi]

(In [6488]) Avoid the 64bit dragons. Fixes #3780 props nbachiyski.

[MichaelH]

Confirming that [6488] fixes Version 2.3.2 installations on Bluehost.com servers when using a .mo language file.

If there is a 2.3.3, might consider this for that release.

From the phpinfo:

`System Linux box288.bluehost.com 2.6.22-9_1.BHsmp #1 SMP Fri Sep 28 23:36:16 MDT 2007 x86_64

Build Date Jan 10 2008 03:11:57
`

[ryan]

(In [6708]) Avoid the 64bit dragons. Fixes #3780 for 2.3 props nbachiyski.

[swissspidy]

In 57763:

I18N: Cast magic MO marker number to integer.

In gettext, 0x950412de is used to signal GNU MO files. In WP_Translation_File_MO this magic marker is used to detect whether a file uses little endian or big endian.

On 32 bit systems, this number will be interpreted by PHP as a float rather than an integer. This change adds extra casting to force an integer.

A similar change was done in the pomo library in the past, see #3780.

Props tmatsuur, swissspidy.
Fixes #60678.