WordPress.org

Make WordPress Core

Opened 6 years ago

Closed 6 years ago

Last modified 5 years ago

#10165 closed enhancement (fixed)

New implementation of MO file reader

Reported by: soletan Owned by: nbachiyski
Milestone: 2.9 Priority: normal
Severity: normal Version: 2.8
Component: I18N Keywords:
Focuses: Cc:

Description

On moving from a Debian-driven virtual server to one of my dedicated root server running Gentoo Linux I had to encounter poor performance in Wordpress on every click. Other sites hosted on same server respond within a second at most while Wordpress 2.8 took about 10-15 seconds on each click.

I started to upgrade packages, tried to performance-tune Apache and MySQL etc. without any remarkable success. And so I tried to locate the part of code in WP wasting so much time. Finally I stopped on method import_from_file() of class MO in wp-includes/pmo/mo.php.

I'm running WP with German localization available from de.wordpress.org. de_DE.mo takes 300KB disk spaces and about 8 seconds to load with your current class MO here. I did some investigations on the used code and saw potential to improve its performance, thus started to implement my own MO file reader omitting all that POMOReader stuff etc. Finally i got a linear method reading (at least my) MO file up to four times faster.
While server is emerging and compiling in background (e.g. compiling gcc and glibc) benchmarking existing MO file reader and my implementation resulted in a 8s vs. 2s runtime. BTW: other sites hosted on same server didn't remarkably slow down due to background compilation processes.

In addition to reading MO files faster the class tries to utilize wp_cache_* interface (which seems to be useless at the moment, isn't it) and it tries to completely work without instances of class Translation_Entry, which I consider to take much more memory than a normal hash (As I tried to produce instances on successfully parsing MO file performance decreased remarkably by estimated 10%.)

I'd like to provide my class here as-is since I didn't find the time to test it with other MO files or check if it's properly implementing API of class Translation.

Attachments (3)

mo.php (8.1 KB) - added by soletan 6 years ago.
Drop-In replacement for wp-includes/pomo/mo.php
mo.2.php (10.7 KB) - added by soletan 6 years ago.
Updated re-implementation, using internal cache, supporting multi-byte overloading
mo.3.php (9.2 KB) - added by soletan 6 years ago.
Updated implementation using wp-cache API again, using different way of parsing MO-file header …

Download all attachments as: .zip

Change History (14)

@soletan6 years ago

Drop-In replacement for wp-includes/pomo/mo.php

comment:1 @ShaneF6 years ago

  • Keywords needs-testing needs-unit-tests added
  • Milestone changed from Unassigned to Future Release

Moved this to 2.9 release just in-case, but we need some test cases for speed I would guess to see how this compares to the current system.

comment:2 @ShaneF6 years ago

  • Milestone changed from Future Release to 2.9

comment:5 @soletan6 years ago

I tried to understand issues reported in comments on #10236. I'm not quite familiar with trac and can't follow the timeline and relations of posts and comments that clearly.

Nevertheless I changed my implementation a bit. I removed any PHP5-specific stuff and added support for the reported multibyte overloading issue. Next I dropped senseless integration of non-persistent WP object cache and added three simple methods to manage cached translation maps in wp-content/cache/mo internally. I tested my previous code on whether its working with plugins' translations as well. Here I'm having wpcf contact form and nextgen gallery both getting translation files successfully loaded by my class.

I don't know current state of things and thus provide my updated post as-is again. Hope you'll find it useful ...

Best Regards,
Thomas Urban

@soletan6 years ago

Updated re-implementation, using internal cache, supporting multi-byte overloading

comment:6 follow-up: @ryan6 years ago

The cache API should be used. It is not useless if someone installs a persistent caching backend. Writing directly to files breaks the cache abstraction and will not work nicely in multi-server MU environments.

comment:7 in reply to: ↑ 6 @soletan6 years ago

Replying to ryan:

The cache API should be used. It is not useless if someone installs a persistent caching backend. Writing directly to files breaks the cache abstraction and will not work nicely in multi-server MU environments.

Even though I consider this cache API including some limitations on automatically testing for cache-external updates to cached data requiring flushing the latter I agree with you in the fact that breaking APIs isn't acceptable.

So here is another version of my implementation using WPs cache API again. I didn't test that but it should work nevertheless.

This version includes further improvements regarding support for MO files not containing hashes maybe fixing previously reported issues on failing to read some plugins' translation catalogues. This was done with the support of H. Rabe from codestyling.de ...

@soletan6 years ago

Updated implementation using wp-cache API again, using different way of parsing MO-file header ...

comment:8 @soletan6 years ago

BTW: Did you ever consider to commit yourself on using gettext including dgettext() etc. to support plugin-specific translation catalogues and thus stopping to parse MO-files in PHP yourself? Why isn't code doing it the other way around: if a user doesn't have gettext extension the currently available API is used as a fallback.

Is there a thread I can read to get eligible arguments for not doing this? Or did I miss some runtime configuration option to switch?

comment:9 @nbachiyski6 years ago

I managed to get your ideas for parsing into a new patch, which almost retains the speed of mo3.php, but is still object-oriented. Several plugin authors really liked the objects idea and are already using it in their plugins.

See the patch here: http://trac.glotpress.org/changeset/293

On caching: on WordPress.com we have tried caching the read MO file and it didn't work out so well. Since this usually involves transmitting the data over a network it both filled the bandwith with translations and wasn't that much faster.

On using php gettext extension: we haven't revisited it in a long time. It used do depend on server locales and too few servers had it installed, that's why we haven't incorporated it. Also, there were problems with old gettext versions.

comment:10 @ryan6 years ago

  • Resolution set to fixed
  • Status changed from new to closed

(In [12174]) pomo performance improvements. Props nbachiyski. fixes #10165

comment:11 @nacin5 years ago

  • Keywords l10n MO performance needs-testing needs-unit-tests removed
Note: See TracTickets for help on using tickets.