Make WordPress Core

Opened 5 years ago

Last modified 8 months ago

#17128 new enhancement

POMO library performance considerations

Reported by: wet Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: I18N Keywords: has-patch
Focuses: Cc:


Profiling a localized WordPress instance with WPLANG defined as 'de_DE' on my testbed shows that loading the .PO files for WP core and TwentyTen increases the frontend runtime by about 45 percent and the backend runtime by about 30 percent.

This is the profiling environment:

  • WordPress 3.2-bleeding r17633
  • TwentyTen 1.1
  • No active Plugins
  • Idling webserver on localhost
  • Apache 2.2.9
  • PHP 5.2.6
  • MySQL 5.0.67
  • Xdebug 2.1.1
  • webgrind 1.1
  • Windows 7 SP1

Without WPLANG defined, runtimes for various code paths are as follows:

  • Front page with 10 simple posts: 2300 ms
  • Admin Dashboard: 3000 ms
  • RSS XML feed output: 1130 ms

With the de_DE l10n in effect, runtimes increase as such:

  • Front page with 10 simple posts: 4300 ms
  • Admin Dashboard: 4280 ms
  • RSS XML feed output: 2950 ms

I wonder whether anyone is able to reproduce this grave performance degradation. A sample output from webgrind showing various POMO functions as the top offenders is attached.

Attachments (3)

frontend-de_DE-issue-17128.png (41.5 KB) - added by wet 5 years ago.
Webgrind profiling report of site frontpage
pomo-entry-17637.patch (683 bytes) - added by wet 5 years ago.
Replace is_null() with NULL comparison. A low hanging fruit, but 1.5 percent performance increase is maybe better that a sharp stick in the eye. See http://hakre.wordpress.com/2011/03/29/php-is_null-vs-null/.
17128.patch (685 bytes) - added by SergeyBiryukov 4 years ago.

Download all attachments as: .zip

Change History (32)

5 years ago

Webgrind profiling report of site frontpage

#1 @wet
5 years ago

  • Cc r.wetzlmayr@… added

#2 @pavelevap
5 years ago

  • Cc pavelevap@… added

5 years ago

Replace is_null() with NULL comparison. A low hanging fruit, but 1.5 percent performance increase is maybe better that a sharp stick in the eye. See http://hakre.wordpress.com/2011/03/29/php-is_null-vs-null/.

#3 @wet
5 years ago

  • Keywords has-patch needs-testing added

#4 @jottlieb
5 years ago

  • Cc jottlieb@… added

#5 @scribu
5 years ago

  • Milestone changed from Awaiting Review to 3.2

Looks like a fine candidate for 3.2.

#6 @linushoppe
5 years ago

  • Cc linus.hoppe@… added

Also see #17268, there is a patch which reduces the used php-memory for translation about 75%.

#7 @Philipp15b
5 years ago

  • Cc hebipp1@… added

#8 @nacin
5 years ago

  • Keywords needs-patch added; has-patch needs-testing removed
  • Milestone changed from 3.2 to Future Release

Nothing here yet, punting.

#9 @SergeyBiryukov
4 years ago

  • Keywords has-patch added; needs-patch removed

pomo-entry-17637.patch seems like a valid patch.

#10 @MHagemeister
4 years ago

  • Cc MHagemeister added

#11 @knutsp
4 years ago

  • Cc knut@… added

#12 @ocean90
4 years ago

  • Keywords 3.4-early added

#13 @SergeyBiryukov
4 years ago

Refreshed as per coding standards.

#14 @ocean90
4 years ago

  • Keywords 3.4-early removed
  • Milestone changed from Future Release to 3.4

#15 @Chouby
4 years ago

  • Cc Chouby added

I observed the same issue.

Please correct me if I am wrong, but it looks like every internationalized strings are loaded on every page. Loading the complete mo file on every page could explain such huge difference of performance between English and localized versions. Moreover, as WorPress is growing, the mo file is growing too (for de_DE.mo, it was 296 ko for 2.8, 386 ko for 3.0 and 435 ko now), so this performance issue is growing too.

So if my assumption is good, the most optimized solution could be to load only the strings needed for the current page. I am not a specialist of the pomo structure, but I imagine that in the worst case it could mean splitting the current mo file in many small files. Obviously it would be difficult to do and maintain.

However, it may be a good solution to split the current mo file in two. One small file for the frontend and the big file for the admin side (it seems to me that there are not a lot of core strings displayed on the frontend). Maybe it would be necessary to load both files on the admin side. I am not sure.

This way, of course, we do not improve the time running on the admin side but we should improve a lot the time running on the frontend. Since, inn most cases, the frontend is loaded much more times than the admin, the server load decrease should be quite perceptible...

Version 0, edited 4 years ago by Chouby (next)

#16 @scribu
4 years ago

@Chouby: Splitting the translation files in two is an idea worth exploring, but you should open a new ticket for it.

#17 follow-up: @pavelevap
4 years ago

There are also many strings which can be seen only in multisite version. Maybe they should get right textdomain? Or is there any reason why these strings are not inluded in existing separate multisite .mo file?

#18 in reply to: ↑ 17 @SergeyBiryukov
4 years ago

Replying to pavelevap:

Or is there any reason why these strings are not inluded in existing separate multisite .mo file?

I guess this also should be investigated in a new ticket.

#19 @pavelevap
4 years ago

OK, there are 2 new tickets: #19831 #19832

Last edited 4 years ago by scribu (previous) (diff)

#20 @noah.williamsson
4 years ago

  • Cc noah.williamsson added

FWIW, I've experienced similar performance issues as outlined in the original report on an Ubuntu 10.04 LTS (PHP 5.3.2, APC 3.1.3) system running Wordpress 3.3.1.

Switching from Wordpress' own gettext implementation to PHP's native gettext libary (using a patch from #17268) improved page response times by 43% compared to a stock Wordpress 3.3.1 installation where WPLANG is used.

Feature Response time Performance hit
Stock 3.3.1, WPLANG not set 139ms 0%
WPLANG set to 'sv_SE' 255ms 45% slower
WPLANG set + patch from #17268 145ms 4% slower

#21 @ryan
4 years ago

  • Milestone changed from 3.4 to Future Release

#23 @rulatir
3 years ago

IMO the whole l10n infrastructure should be refactored to use on-demand loading. Rationale: AJAX! My ajax requests that go through wp-includes/wp-admin.php load all translated strings, only to return a JSON array with a few rows from the DB that contains no translated strings whatsoever.

#24 @scribu
3 years ago

We could do on-demand loading of strings on a per-textdomain basis, but that would need some profiling, since each call to translate() would now need an if clause to see if a textdomain load has already been attempted.

#25 @rulatir
3 years ago

The if can be avoided the same way we currently avoid checking if the .mo file was there in the first place for the given domain: if translations for a domain are not available, the framework creates an instance of NOOP_Translations, rather than Gettext_Translations, see l10n.php:499. We could use the same pattern for on-demand loading: we could write Textdomain_Loader, another implementation of the Translations() interface, and instantiate it where we now instantiate Gettext_Translations. Textdomain_Loader::translate() would do the actual loading and change the global reference to point to the newly crated MO object.

#26 @rulatir
3 years ago

I am investigating the feasibility of doing this with a plugin using override_load_textdomain filter.

#27 @mbijon
3 years ago

  • Cc mike@… added

#28 @greencp
22 months ago

I just published a plugin that implements on-demand loading of mo files and on-demand translating of strings: WP Performance Pack.
It uses override_load_textdomain and a new Translations implementation. It already improves performance and memory usage but I'm working on some more improvements by reducing unnecessary translations, of wich there are quite some in WordPress core (e.g. #26746). I also plan to implement usage of native gettext if available.

Last edited 8 months ago by SergeyBiryukov (previous) (diff)

#29 @ccismaru
8 months ago

There are a ton of calls to chr and is_null. Checking the code, there can be easily replaced with string equivalent or with language constructs, which will reduce the overhead of a function call as well as other code ran in the internal function bodies.

chr(0) could be replaced with "\0x0", chr(4) with "\0x04". is_null(exp) with exp === NULL. Have tested this and about 22K calls were saved because of these minor changes. I'm aware that the code is more readable keeping them in place, as they are now, but in a shared hosting environment could make a big difference in saving CPU cycles.

Note: See TracTickets for help on using tickets.