WordPress.org

Make WordPress Core

Opened 3 years ago

Last modified 3 months ago

#17128 new enhancement

POMO library performance considerations

Reported by: wet Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: I18N Keywords: has-patch
Focuses: Cc:

Description

Profiling a localized WordPress instance with WPLANG defined as 'de_DE' on my testbed shows that loading the .PO files for WP core and TwentyTen increases the frontend runtime by about 45 percent and the backend runtime by about 30 percent.

This is the profiling environment:

  • WordPress 3.2-bleeding r17633
  • TwentyTen 1.1
  • No active Plugins
  • Idling webserver on localhost
  • Apache 2.2.9
  • PHP 5.2.6
  • MySQL 5.0.67
  • Xdebug 2.1.1
  • webgrind 1.1
  • Windows 7 SP1

Without WPLANG defined, runtimes for various code paths are as follows:

  • Front page with 10 simple posts: 2300 ms
  • Admin Dashboard: 3000 ms
  • RSS XML feed output: 1130 ms

With the de_DE l10n in effect, runtimes increase as such:

  • Front page with 10 simple posts: 4300 ms
  • Admin Dashboard: 4280 ms
  • RSS XML feed output: 2950 ms

I wonder whether anyone is able to reproduce this grave performance degradation. A sample output from webgrind showing various POMO functions as the top offenders is attached.

Attachments (3)

frontend-de_DE-issue-17128.png (41.5 KB) - added by wet 3 years ago.
Webgrind profiling report of site frontpage
pomo-entry-17637.patch (683 bytes) - added by wet 3 years ago.
Replace is_null() with NULL comparison. A low hanging fruit, but 1.5 percent performance increase is maybe better that a sharp stick in the eye. See http://hakre.wordpress.com/2011/03/29/php-is_null-vs-null/.
17128.patch (685 bytes) - added by SergeyBiryukov 2 years ago.

Download all attachments as: .zip

Change History (31)

wet3 years ago

Webgrind profiling report of site frontpage

comment:1 wet3 years ago

  • Cc r.wetzlmayr@… added

comment:2 pavelevap3 years ago

  • Cc pavelevap@… added

wet3 years ago

Replace is_null() with NULL comparison. A low hanging fruit, but 1.5 percent performance increase is maybe better that a sharp stick in the eye. See http://hakre.wordpress.com/2011/03/29/php-is_null-vs-null/.

comment:3 wet3 years ago

  • Keywords has-patch needs-testing added

comment:4 jottlieb3 years ago

  • Cc jottlieb@… added

comment:5 scribu3 years ago

  • Milestone changed from Awaiting Review to 3.2

Looks like a fine candidate for 3.2.

comment:6 linushoppe3 years ago

  • Cc linus.hoppe@… added

Also see #17268, there is a patch which reduces the used php-memory for translation about 75%.

comment:7 Philipp15b3 years ago

  • Cc hebipp1@… added

comment:8 nacin3 years ago

  • Keywords needs-patch added; has-patch needs-testing removed
  • Milestone changed from 3.2 to Future Release

Nothing here yet, punting.

comment:9 SergeyBiryukov3 years ago

  • Keywords has-patch added; needs-patch removed

pomo-entry-17637.patch seems like a valid patch.

comment:10 MHagemeister2 years ago

  • Cc MHagemeister added

comment:11 knutsp2 years ago

  • Cc knut@… added

comment:12 ocean902 years ago

  • Keywords 3.4-early added

SergeyBiryukov2 years ago

comment:13 SergeyBiryukov2 years ago

Refreshed as per coding standards.

comment:14 ocean902 years ago

  • Keywords 3.4-early removed
  • Milestone changed from Future Release to 3.4

comment:15 Chouby2 years ago

  • Cc Chouby added

I observed the same issue.

Please correct me if I am wrong, but it looks like every internationalized strings are loaded on every page. Loading the complete mo file on every page could explain such huge difference of performance between English and localized versions. Moreover, as WordPress is growing, the mo file is growing too (for de_DE.mo, it was 296 ko for 2.8, 386 ko for 3.0 and 435 ko now), so this performance issue is growing too.

So if my assumption is good, the most optimized solution could be to load only the strings needed for the current page. I am not a specialist of the pomo structure, but I imagine that in the worst case it could mean splitting the current mo file in many small files. Obviously it would be difficult to do and maintain.

However, it may be a good solution to split the current mo file in two. One small file for the frontend and the big file for the admin side (it seems to me that there are not a lot of core strings displayed on the frontend). Maybe it would be necessary to load both files on the admin side. I am not sure.

This way, of course, we do not improve the time running on the admin side but we should improve a lot the time running on the frontend. Since, inn most cases, the frontend is loaded much more times than the admin, the server load decrease should be quite perceptible...

Last edited 2 years ago by Chouby (previous) (diff)

comment:16 scribu2 years ago

@Chouby: Splitting the translation files in two is an idea worth exploring, but you should open a new ticket for it.

comment:17 follow-up: pavelevap2 years ago

There are also many strings which can be seen only in multisite version. Maybe they should get right textdomain? Or is there any reason why these strings are not inluded in existing separate multisite .mo file?

comment:18 in reply to: ↑ 17 SergeyBiryukov2 years ago

Replying to pavelevap:

Or is there any reason why these strings are not inluded in existing separate multisite .mo file?

I guess this also should be investigated in a new ticket.

comment:19 pavelevap2 years ago

OK, there are 2 new tickets: #19831 #19832

Last edited 2 years ago by scribu (previous) (diff)

comment:20 noah.williamsson2 years ago

  • Cc noah.williamsson added

FWIW, I've experienced similar performance issues as outlined in the original report on an Ubuntu 10.04 LTS (PHP 5.3.2, APC 3.1.3) system running Wordpress 3.3.1.

Switching from Wordpress' own gettext implementation to PHP's native gettext libary (using a patch from #17268) improved page response times by 43% compared to a stock Wordpress 3.3.1 installation where WPLANG is used.

Feature Response time Performance hit
Stock 3.3.1, WPLANG not set 139ms 0%
WPLANG set to 'sv_SE' 255ms 45% slower
WPLANG set + patch from #17268 145ms 4% slower

comment:21 ryan2 years ago

  • Milestone changed from 3.4 to Future Release

comment:23 rulatir20 months ago

IMO the whole l10n infrastructure should be refactored to use on-demand loading. Rationale: AJAX! My ajax requests that go through wp-includes/wp-admin.php load all translated strings, only to return a JSON array with a few rows from the DB that contains no translated strings whatsoever.

comment:24 scribu20 months ago

We could do on-demand loading of strings on a per-textdomain basis, but that would need some profiling, since each call to translate() would now need an if clause to see if a textdomain load has already been attempted.

comment:25 rulatir20 months ago

The if can be avoided the same way we currently avoid checking if the .mo file was there in the first place for the given domain: if translations for a domain are not available, the framework creates an instance of NOOP_Translations, rather than Gettext_Translations, see l10n.php:499. We could use the same pattern for on-demand loading: we could write Textdomain_Loader, another implementation of the Translations() interface, and instantiate it where we now instantiate Gettext_Translations. Textdomain_Loader::translate() would do the actual loading and change the global reference to point to the newly crated MO object.

comment:26 rulatir20 months ago

I am investigating the feasibility of doing this with a plugin using override_load_textdomain filter.

comment:27 mbijon17 months ago

  • Cc mike@… added

comment:28 greencp3 months ago

I just published a plugin that implements on-demand loading of mo files and on-demand translating of strings: WP Performance Pack.
It uses override_load_textdomain and a new Translations implementation. It already improves performance and memory usage but I'm working on some more improvements by reducing unnecessary translations, of wich there are quite some in WordPress core (e.g. 26746). I also plan to implement usage of native gettext if available.

Note: See TracTickets for help on using tickets.