Translation is not applied in load_theme_textdomain() function

[wildworks]

Originally reported in the Gutenberg repository: ​https://github.com/WordPress/gutenberg/issues/66547

Themes not hosted on wordpress.org need to have translations applied in the following way:

<?php
function translate_test_load_theme_textdomain() {
  load_theme_textdomain( 'translate-test', get_template_directory() . '/languages' );
}
add_action( 'init', 'translate_test_load_theme_textdomain' );

However, in 6.7 RC2, it seems that the translations are not applied at all.

I created a minimal repository to test this issue: ​https://github.com/t-hamano/translate-test

[swissspidy]

Thanks, I‘ll look into it tomorrow

[swissspidy]

This is a consequence of [59157]. If calling load_*_textdomain() _after_ WordPress has already attempted just-in-time loading, nothing happens. For themes, just-in-time loading is triggered by _register_theme_block_patterns (which is hooked to init at prio 10).

While a look into this, a quick fix is to change init to after_setup_theme in your code snippet above.

Tentatively moving to 6.7.1 milestone instead of 6.7, given the release schedule and minor severity of the issue.

[twvania]

Actually, it doesn't work on after_setup_theme either.

​https://github.com/WordPress/gutenberg/issues/66547

[swissspidy]

It does, I tested it with the theme above. But of course if you do something like directly adding echo esc_attr_x( 'Page not found', '404 error page', 'your-theme-name' ); to your functions.php file, that will trigger translation loading way too early (and would result in a doing_it_wrong warning).

Anyhow, this was just a first quick feedback while I am looking into it.

This is a follow-up to https://core.trac.wordpress.org/changeset/59157.

I18N: Retry just-in-time translation when setting custom path.

This is a follow-up to [59127] and [59157], which changed the `load_*_textdomain()` functions to reuse the existing just-in-time translation loading functionality. While this works great for cases where those functions are called too early, it broke cases where they are called after the just-in-time logic already ran.

To solve this, cached entries from the just-in-time logic are reset, so that the loading will be simply reattempted the next time a translation is attempted (e.g. by calling `__()`).

Props ...
Fixes #62337.
See #44937

Trac ticket: https://core.trac.wordpress.org/ticket/62337

[swissspidy]

There's now a patch at ​https://github.com/WordPress/wordpress-develop/pull/7713 that can be tested.

@peterwilsoncc let me know if you think this could even go into 6.7 still.

[peterwilsoncc]

@swissspidy In the dev-chat in a few hours, there's an agenda item to discuss whether the 6.7 release will need an RC4. If so, then I think this can be included.

[twvania]

Hi @swissspidy,

In RC4, the translation already works for the standard pattern in the Site Editor, but the issue remains on the front-end.

Here’s how you can see the bug:

1) Create a pattern called hidden-404.php with the following code:

<?php
/**
 * Title: 404
 * Slug: theme-name/hidden-404
 * Inserter: no
 */
?>
<!-- wp:heading {"level":1} -->
<h1 class="wp-block-heading"><?php echo esc_attr_x( 'Page not found', '404 error page', 'theme-name' ); ?></h1>
<!-- /wp:heading -->

2) Create a 404.html template with the following code:

<!-- wp:pattern {"slug":"theme-name/hidden-404"} /-->

3) If you navigate to the 404 page on the front-end, the translation doesn’t work.
4) If you open this page in the Site Editor, it works (after RC4 fixes), but to make it work on the front-end, you need to save this template to the database.

Either I’m doing something wrong, or the translation should work for templates pulled from the code rather than the database.

P.S. load_theme_textdomain trying on init/after_setup_theme action.

Thank you in advance for your time,
Regards

[swissspidy]

Have you tested the PR at ​https://github.com/WordPress/wordpress-develop/pull/7713? Does it still happen with that PR applied?

I'm asking because that's exactly the scenario this has been tested with successfully.

[twvania]

Yes, this works on PR ​https://github.com/WordPress/wordpress-develop/pull/7713. Thank you..))

[looswebstudio]

Hello.

I am developing a Japanese theme. (I am not registered with wordpress.org.)

Most of my users are Japanese, so instead of setting the base language to English and loading ja.mo, I have prepared en_US.mo for Japanese in the languages ​​directory.

In 6.7 RC-4, it is no longer loaded.

It seems that other language files are loaded. Only en_US.mo is not loaded.

[swissspidy]

@looswebstudio Thanks for your report! I was able to reproduce this & fix this as part of ​https://github.com/WordPress/wordpress-develop/pull/7713. Could you also try that PR to see if it resolves the issue for you? Thanks!

[tigriweb]

Hi @swissspidy version 6.7 has been released, but it does not include pull/7713 code. Sites that updated automatically are not working correctly. When is this code expected to be released?

Regards

[swissspidy]

As you can see, the milestone is set to 6.7.1. There are also workarounds mentioned here thst you can use in the meantime

[stimul]

Hi,

It appears that the proposed workaround isn’t effective. We were already using the after_setup_theme hook, but we’re still experiencing this issue across all our websites. Testing with the init hook and adjusting the priority didn’t resolve it either.

Do you have any other possible workarounds while we wait for version 6.7.1?

[mattialerda]

Hi @swissspidy, we have implemented the PR code on some of our sites. The fix works on most of them, but on sites where we have set the same textdomain for theme and plugin, it doesn’t work. Previously, it worked in these cases as well. I’m not sure if this requires a core adjustment or if it’s an error on our part.

[swissspidy]

Oh that‘s interesting edge case, I‘ll need to think about that a bit

[stimul]

We found a workaround, maybe not the best solution but it seems to be working fine. Maybe this can help debug the issue.

Before:

<?php
add_action('after_setup_theme', function() {
  load_theme_textdomain('our_theme_name_domain', get_template_directory() . '/languages');
});

After:

<?php
add_action('init', function() {
  global $l10n, $wp_textdomain_registry;
  $domain = 'our_theme_name_domain';
  $locale = get_locale();
  $wp_textdomain_registry->set($domain, $locale, get_template_directory() . '/languages');
  if ( isset( $l10n[ $domain ] )) {
    unset( $l10n[ $domain ] );
  }
  load_theme_textdomain($domain, get_template_directory() . '/languages');
});

One interesting thing is that the workaround only works on "init" and not with the "after_setup_theme" callback.

edit: We confirm that the pull request 7731 fixed the issue, so we will have to wait for 6.7.1

[audrasjb]

#62438 was marked as a duplicate.

[finntown]

Replying to audrasjb:

#62438 was marked as a duplicate.

Thank you, waiting for fix. :)

[swissspidy]

@finntown Please try the attached pull request to see if it solves your issue. I take it your plugin is a custom one that‘s not in the repo?

[finntown]

Tested the PR in my staging area, it seems to fix the plugin languages (3rd party plugins from the Plugin Directory). But it does not fix my custom Elementor theme language, not entirely sure why yet, the translation files are there as they always were. Looking into this.

[bluantinoo]

@stimul fix works for me, both on theme and child theme.
I tested on Hello Elementor child theme.

Will this be the new "standard" or it's just a temporary workaround?

[asadkn]

Can't believe this is not a high priority. Tens of thousands of sites (that I know of) had issues with translation due to this. All requiring fixes from premium theme / plugin authors.

[Adrian2k7]

Replying to asadkn:

Can't believe this is not a high priority. Tens of thousands of sites (that I know of) had issues with translation due to this. All requiring fixes from premium theme / plugin authors.

Should have been resolved before the final release ... already had a couple of support requests and needed to implement workarounds, just because of this

Also, why is there not just an explicit hook for registering localisation ...
? all this confusion with init/after_theme_setup could be easily solved with a "init_localisations" action

[luehrsen]

We are also experiencing this issue with many of our clients and would deeply appreciate a prompt fix. The workaround provided is helpful temporarily, but it is not an elegant or ideal solution.

Addressing this with higher priority would be greatly beneficial to streamline workflows and maintain functionality. Thank you!

[timwhitlock]

It's taken me a few days to fully appreciate the impact of this.

I'm advising my plugin users to roll back to 6.6 and wait for 6.7.1. In the mean time, any components that use translations too early need to be identified and fixed

​https://wordpress.org/support/topic/wordpress-6-7-major-issue-loading-custom-translation-files/

It would have been helpful if the early loading notice (which is great) had been introduced some versions prior to the sudden change in load_*_textdomain, but that's the benefit of hindsight. The relationship between premature JIT loads, and explicit loading of custom translation paths was probably not obvious.

[albigdd]

Your patch works for my custom theme which provides its localized files in /languages. So this code work's again:

load_theme_textdomain( 'MYTEXTDOMAIN', get_template_directory() . '/languages' );

Looking forward to see this patch in 6.7.1 soon.

[timwhitlock]

It looks like the patch only unloads the text domain if it's NOOP_Translations. That means authors still can't override GlotPress translations that have been JIT loaded. You may think they shouldn't need to, but why prevent them doing so?

[swissspidy]

What‘s your use case for that and why do unload_textdomain & load_textdomain() not work for you in that case?

[michaelwp85]

Will there be any discussion to prevent a reoccurrence of a similar situation in the future?

This isn't the first time I have seen a minor release with breaking changes on top of that, this looks like it was identified before the release of 6.7. This should not have been released if it was known.

We have over 400 sites that are affected by this release. Needless to say, I'm reverting.

[timwhitlock]

Replying to swissspidy:

What‘s your use case for that and why do unload_textdomain & load_textdomain() not work for you in that case?

My plugin lets users override GlotPress translations with their own custom translation files. It does this by listening on the load_textdomain action hook. When the original MO is requested, a custom file is loaded if one exists. If both exist, they are merged.

You can see the code if you like here: ​https://github.com/loco/wp-loco/blob/master/src/hooks/LoadHelper.php

WP 6.7 breaks this function for the reasons well noted in this thread. Premature JIT invocation is a pervasive problem, and one out of my control. With 1+ million installs, you can imagine my inbox is full of some quite angry people.

So to answer your question - I can't call unload_textdomain, because my code won't run until hooks fire. Which they now don't. Only the text domain provider can make that fix. My users will be lucky if that happens fast enough, or at all.

In order to restore previous functionality myself, I would have to forcefully unload all text domains when my plugin starts up. I don't want to do that, partly for performance. But if 6.7.1 doesn't rectify the issue I may have to.

It's worth noting that unloading NOOP translations does indeed cover a lot of the most common cases, but not all of them. I don't see a good reason to be so selective, other than to protect the performance gains for which this change was [I assume] primarily made. I respect the need for performance, but not at the expense of broken websites.

Ideally I'd like to see an action hook that gets fired for load_theme_textdomain and load_plugin_textdomain. This would allow people to fix the problem as they see fit without compromising performance for everyone else.

Sorry for such a long answer, but I've lived and breathed this problem for the past week.
Thanks for your consideration!

[swissspidy]

Thanks, I‘ll take a closer look at your code later this week when I‘m back from vacation. Curious to learn more about why the other existing hooks won‘t work for that use case.

@michaelwp85 Note that 6.7 is a major release, not a minor one. WP doesn‘t use semver.

[timwhitlock]

Replying to swissspidy:

Thanks, I‘ll take a closer look at your code later this week when I‘m back from vacation. Curious to learn more about why the other existing hooks won‘t work for that use case.

I appreciate the offer of your time, but the reason is very simple - There is nothing to hook into. If JIT loading has already been done, then no hooks fire at all when calling load_plugin_textdomain.

[michaelwp85]

Replying to swissspidy:

@michaelwp85 Note that 6.7 is a major release, not a minor one. WP doesn‘t use semver.

Oops, I thought it did. Well, that changes things a little bit.

[swissspidy]

Replying to timwhitlock:

Replying to swissspidy:

Thanks, I‘ll take a closer look at your code later this week when I‘m back from vacation. Curious to learn more about why the other existing hooks won‘t work for that use case.

I appreciate the offer of your time, but the reason is very simple - There is nothing to hook into. If JIT loading has already been done, then no hooks fire at all when calling load_plugin_textdomain.

Because then the function doesn‘t have to do anything anymore, since a translation was already loaded. That‘s at least the idea behind it. Amd for that first load hooks are fired.
But let me take a closer look when I am back.

[desrosj]

There are a handful of edge cases being addressed here, and the current PR solves the majority of them. The only one that does not currently have a working solution is My plugin lets users override GlotPress translations with their own custom translation files..

With 6.7.1 RC1 targeted in less than 24 hours, I am going to suggest that we move forward with the current pull request to fix the majority of the issues described here for now. If another Core team member with time and deep i18n knowledge is able to come up with a patch that also fixes that use case, we can consider it for 6.7.1.

I'm marking this as commit and will return in a few hours to test and review once more before committing.

[desrosj]

In 59430:

i18n: Account for load_*_textdomain() after JIT loading.

When load_*_textdomain() functions are called after WordPress has already attempted just-in-time loading of translations, nothing happens.

This updates the related logic to retry translation loading when a custom path is set to ensure all translations are available.

Additionally, this also fixes cases where an en_US.mo file is provided with non-English strings to override the default language.

Follow up to [59157].

Props swissspidy, peterwilsoncc, desrosj, apermo, sergeybiryukov, wildworks, tigriweb, twvania, looswebstudio, stimul, audrasjb, finntown, bluantinoo, timwhitlock, albigdd.
See #62337.

[desrosj]

Marking for a second committer's review before backporting.

Merged in https://core.trac.wordpress.org/changeset/59430.

[webdados]

Thanks for the fix. It's still not included on the latest nightly (6.7.1-alpha-59423), is it?

(Not exactly sure what's the nightly schedule and time zone.)

[SergeyBiryukov]

[59430] looks good to backport.

[desrosj]

In 59433:

i18n: Account for load_*_textdomain() after JIT loading.

When load_*_textdomain() functions are called after WordPress has already attempted just-in-time loading of translations, nothing happens.

This updates the related logic to retry translation loading when a custom path is set to ensure all translations are available.

Additionally, this also fixes cases where an en_US.mo file is provided with non-English strings to override the default language.

Follow up to [59157].

Reviewed by SergeyBiryukov.
Merges [59430] to the 6.7 branch.

Props swissspidy, peterwilsoncc, desrosj, apermo, sergeybiryukov, wildworks, tigriweb, twvania, looswebstudio, stimul, audrasjb, finntown, bluantinoo, timwhitlock, albigdd.
See #62337.

[desrosj]

Since there are changes associated with this ticket that will ship in 6.7.1, I'm closing this out as fixed. I've opened #62487 to continue discussing the final problem that has been reported in this ticket in 6.7.2.

[luehrsen]

Confirmed that WordPress 6.7.1 resolves the issue with missing translations for themes not hosted on WordPress.org.

Thank you for addressing this!

[adc01684]

Finally how can we solve this problem ? I have the latest WP _ Elementor + Hello Theme and I got 2 notices

[swissspidy]

@adc01684 If a plugin or theme of yours is triggering this notice, ensure you update them first, and then reach out to the developer and point them to ​https://make.wordpress.org/core/2024/10/21/i18n-improvements-6-7/ to fix them.