Make WordPress Core

Opened 14 years ago

Closed 6 years ago

Last modified 6 years ago

#17128 closed enhancement (fixed)

POMO library performance considerations

Reported by: wet's profile wet Owned by: ocean90's profile ocean90
Milestone: 5.1 Priority: normal
Severity: normal Version:
Component: I18N Keywords: has-patch
Focuses: Cc:

Description

Profiling a localized WordPress instance with WPLANG defined as 'de_DE' on my testbed shows that loading the .PO files for WP core and TwentyTen increases the frontend runtime by about 45 percent and the backend runtime by about 30 percent.

This is the profiling environment:

  • WordPress 3.2-bleeding r17633
  • TwentyTen 1.1
  • No active Plugins
  • Idling webserver on localhost
  • Apache 2.2.9
  • PHP 5.2.6
  • MySQL 5.0.67
  • Xdebug 2.1.1
  • webgrind 1.1
  • Windows 7 SP1

Without WPLANG defined, runtimes for various code paths are as follows:

  • Front page with 10 simple posts: 2300 ms
  • Admin Dashboard: 3000 ms
  • RSS XML feed output: 1130 ms

With the de_DE l10n in effect, runtimes increase as such:

  • Front page with 10 simple posts: 4300 ms
  • Admin Dashboard: 4280 ms
  • RSS XML feed output: 2950 ms

I wonder whether anyone is able to reproduce this grave performance degradation. A sample output from webgrind showing various POMO functions as the top offenders is attached.

Attachments (4)

frontend-de_DE-issue-17128.png (41.5 KB) - added by wet 14 years ago.
Webgrind profiling report of site frontpage
pomo-entry-17637.patch (683 bytes) - added by wet 14 years ago.
Replace is_null() with NULL comparison. A low hanging fruit, but 1.5 percent performance increase is maybe better that a sharp stick in the eye. See http://hakre.wordpress.com/2011/03/29/php-is_null-vs-null/.
17128.patch (685 bytes) - added by SergeyBiryukov 13 years ago.
17128.diff (3.6 KB) - added by ocean90 6 years ago.

Download all attachments as: .zip

Change History (36)

@wet
14 years ago

Webgrind profiling report of site frontpage

#1 @wet
14 years ago

  • Cc r.wetzlmayr@… added

#2 @pavelevap
14 years ago

  • Cc pavelevap@… added

@wet
14 years ago

Replace is_null() with NULL comparison. A low hanging fruit, but 1.5 percent performance increase is maybe better that a sharp stick in the eye. See http://hakre.wordpress.com/2011/03/29/php-is_null-vs-null/.

#3 @wet
14 years ago

  • Keywords has-patch needs-testing added

#4 @jottlieb
14 years ago

  • Cc jottlieb@… added

#5 @scribu
14 years ago

  • Milestone changed from Awaiting Review to 3.2

Looks like a fine candidate for 3.2.

#6 @linushoppe
14 years ago

  • Cc linus.hoppe@… added

Also see #17268, there is a patch which reduces the used php-memory for translation about 75%.

#7 @Philipp15b
14 years ago

  • Cc hebipp1@… added

#8 @nacin
14 years ago

  • Keywords needs-patch added; has-patch needs-testing removed
  • Milestone changed from 3.2 to Future Release

Nothing here yet, punting.

#9 @SergeyBiryukov
13 years ago

  • Keywords has-patch added; needs-patch removed

pomo-entry-17637.patch seems like a valid patch.

#10 @MHagemeister
13 years ago

  • Cc MHagemeister added

#11 @knutsp
13 years ago

  • Cc knut@… added

#12 @ocean90
13 years ago

  • Keywords 3.4-early added

#13 @SergeyBiryukov
13 years ago

Refreshed as per coding standards.

#14 @ocean90
13 years ago

  • Keywords 3.4-early removed
  • Milestone changed from Future Release to 3.4

#15 @Chouby
13 years ago

  • Cc Chouby added

I observed the same issue.

Please correct me if I am wrong, but it looks like every internationalized strings are loaded on every page. Loading the complete mo file on every page could explain such huge difference of performance between English and localized versions. Moreover, as WordPress is growing, the mo file is growing too (for de_DE.mo, it was 296 ko for 2.8, 386 ko for 3.0 and 435 ko now), so this performance issue is growing too.

So if my assumption is good, the most optimized solution could be to load only the strings needed for the current page. I am not a specialist of the pomo structure, but I imagine that in the worst case it could mean splitting the current mo file in many small files. Obviously it would be difficult to do and maintain.

However, it may be a good solution to split the current mo file in two. One small file for the frontend and the big file for the admin side (it seems to me that there are not a lot of core strings displayed on the frontend). Maybe it would be necessary to load both files on the admin side. I am not sure.

This way, of course, we do not improve the time running on the admin side but we should improve a lot the time running on the frontend. Since, inn most cases, the frontend is loaded much more times than the admin, the server load decrease should be quite perceptible...

Last edited 13 years ago by Chouby (previous) (diff)

#16 @scribu
13 years ago

@Chouby: Splitting the translation files in two is an idea worth exploring, but you should open a new ticket for it.

#17 follow-up: @pavelevap
13 years ago

There are also many strings which can be seen only in multisite version. Maybe they should get right textdomain? Or is there any reason why these strings are not inluded in existing separate multisite .mo file?

#18 in reply to: ↑ 17 @SergeyBiryukov
13 years ago

Replying to pavelevap:

Or is there any reason why these strings are not inluded in existing separate multisite .mo file?

I guess this also should be investigated in a new ticket.

#19 @pavelevap
13 years ago

OK, there are 2 new tickets: #19831 #19832

Last edited 13 years ago by scribu (previous) (diff)

#20 @noah.williamsson
13 years ago

  • Cc noah.williamsson added

FWIW, I've experienced similar performance issues as outlined in the original report on an Ubuntu 10.04 LTS (PHP 5.3.2, APC 3.1.3) system running Wordpress 3.3.1.

Switching from Wordpress' own gettext implementation to PHP's native gettext libary (using a patch from #17268) improved page response times by 43% compared to a stock Wordpress 3.3.1 installation where WPLANG is used.

Feature Response time Performance hit
Stock 3.3.1, WPLANG not set 139ms 0%
WPLANG set to 'sv_SE' 255ms 45% slower
WPLANG set + patch from #17268 145ms 4% slower

#21 @ryan
13 years ago

  • Milestone changed from 3.4 to Future Release

#23 @rulatir
12 years ago

IMO the whole l10n infrastructure should be refactored to use on-demand loading. Rationale: AJAX! My ajax requests that go through wp-includes/wp-admin.php load all translated strings, only to return a JSON array with a few rows from the DB that contains no translated strings whatsoever.

#24 @scribu
12 years ago

We could do on-demand loading of strings on a per-textdomain basis, but that would need some profiling, since each call to translate() would now need an if clause to see if a textdomain load has already been attempted.

#25 @rulatir
12 years ago

The if can be avoided the same way we currently avoid checking if the .mo file was there in the first place for the given domain: if translations for a domain are not available, the framework creates an instance of NOOP_Translations, rather than Gettext_Translations, see l10n.php:499. We could use the same pattern for on-demand loading: we could write Textdomain_Loader, another implementation of the Translations() interface, and instantiate it where we now instantiate Gettext_Translations. Textdomain_Loader::translate() would do the actual loading and change the global reference to point to the newly crated MO object.

#26 @rulatir
12 years ago

I am investigating the feasibility of doing this with a plugin using override_load_textdomain filter.

#27 @mbijon
12 years ago

  • Cc mike@… added

#28 @greencp
11 years ago

I just published a plugin that implements on-demand loading of mo files and on-demand translating of strings: WP Performance Pack.
It uses override_load_textdomain and a new Translations implementation. It already improves performance and memory usage but I'm working on some more improvements by reducing unnecessary translations, of wich there are quite some in WordPress core (e.g. #26746). I also plan to implement usage of native gettext if available.

Last edited 10 years ago by SergeyBiryukov (previous) (diff)

#29 @ccismaru
10 years ago

There are a ton of calls to chr and is_null. Checking the code, there can be easily replaced with string equivalent or with language constructs, which will reduce the overhead of a function call as well as other code ran in the internal function bodies.

chr(0) could be replaced with "\0x0", chr(4) with "\0x04". is_null(exp) with exp === NULL. Have tested this and about 22K calls were saved because of these minor changes. I'm aware that the code is more readable keeping them in place, as they are now, but in a shared hosting environment could make a big difference in saving CPU cycles.

@ocean90
6 years ago

#30 @ocean90
6 years ago

  • Milestone changed from Future Release to 5.0

17128.patch was applied in [35714].

17128.diff replaces the chr() calls mentioned in comment:29.

#31 @ocean90
6 years ago

  • Owner set to ocean90
  • Resolution set to fixed
  • Status changed from new to closed

In 43635:

I18N: In the POMO library, replace chr() calls for static values with their string representation.

Props ccismaru, ocean90.
Fixes #17128.

#32 @johnbillion
6 years ago

  • Milestone changed from 5.0 to 5.1
Note: See TracTickets for help on using tickets.