It is not possible to install multiple variants of the same language

[caesarsgrunt]

Because of the way mu_dropdown_languages() and format_code_lang() work, it is impossible to have multiple variants of the same language installed. The list of languages returned by mu_dropdown_languages() only includes one variant of each language (the one with the country code which comes last when sorted alphabetically).

For example, if .mo files for es_ES and es_PE are both installed, then only es_PE will be shown.

Furthermore, whatever variant of the language is installed, the name will be the same, not indicating the correct language variant. In the example above, the language name will be "Spanish; Castillian", even for the es_PE localisation. This is highly misleading.

A final problem is that the langauge names are not localised themselves, with the exception of "English".

The cause of these problems is the fact that format_code_lang() ignores the country code and only considers the language code when choosing the name to return. Furthermore, it returns hardcoded language names rather than translated names.

Since mu_dropdown_languages() uses the return value of format_code_lang() as the key for its internal $output array, this means that later language variants overwrite earlier ones.

[nacin]

I believe nikolay has suggested that there's a better alternative to mu_dropdown_languages() somewhere in GlotPress.

[vanillalounge]

Replying to nacin:

I believe nikolay has suggested that there's a better alternative to mu_dropdown_languages() somewhere in GlotPress.

nbachiyski suggested this in #13069

[iandunn]

A workaround for this is to visit http://example.org/wp-admin/options.php and manually set the WPLANG value to pt_BR (or whatever). I'll work on a proper patch, but it'll probably be a few weeks before I have time.

Related #19760

[iandunn]

I've attached a first attempt at this, using locales.php from GlotPress to define the locales. I've got it running on WordCamp.org and it seems to be doing fine. The only issue is going to be backcompat, since the structure of the array has changed some.

e.g., $lang_codes['pt'] is now split into $lang_codes['pt_PT'] and $lang_codes['pt_BR']. There are some similar changes with other languages as well.

That could bite a plugin that's calling format_code_lang() directly, or using the lang_codes filter, since the input/output wouldn't always match what they're expecting anymore.

One inelegant solution would be to go through all the codes one by one, determine which ones have changed, and add duplicate entries for them in the array. That would be kind of lame, though, so I'm hoping someone has a better idea.

As far as updating the locales goes, it should be fairly easy to diff our version of the file against the original to bring in any updates.

I didn't internationalize the locale names since there's already a separate ticket for that (#19760), but it should be trivial to wrap them in __() in locales.php. Maybe even do it upstream to make maintenance easier.

If this is committed, it'd also fix #13069.

[iandunn]

refreshed for 3.7

[iandunn]

refresh

[iandunn]

Fall back to iso code or slug if wp locale not present

[iandunn]

ran against 4.diff

[iandunn]

15677-lang-codes.2.diff compares the $lang_codes array before and after the patch, to help assess the backcompat impact. A lot of the differences are just in ordering, but there's still a significant amount of changes from codes like pt to pt_BR and pt_PT.

The idea I had earlier about manually adding the codes that are missing (or changed) in GlotPress won't work, because then that language would show up in the dropdown multiple times.

Any ideas? Maybe we need to approach this from a different angle?

Or maybe there just isn't a good way to do this while maintaining backcompat? If that's the case, then maybe the best thing to do would be to just take this code and make it into a plugin that replaces the dropdown with a new one via the mu_dropdown_languages filter?

[nacin]

mu_dropdown_languages() is terrible. But there are actually deeper issues. There are two different pt_PT translations, one formal and one informal. Same goes for German. These situations will grow as more decide to do them.

I've been planning to pick this up starting in 3.9. Basically, locales.php from GlotPress is a major component, but we need to go further than that.

(Long-term goal is for language choosing to be able to just happen on Settings > General and for language pack files to then be downloaded into place.)

[iandunn]

Sounds good, thanks for the update.

[SergeyBiryukov]

Related: #28303

[nacin]

In 29007:

Introduce wp_dropdown_languages() and use it on general settings.

Early rough cut. Obviously not fully implemented.

see #15677.

[ocean90]

In 29514:

Add a label to site language setting.

see #15677.

[ocean90]

15677.patch​ is an idea to quickly switch to a language which isn't even installed yet. On saving the translation packs should be downloaded automatically and when you get the screen back, it's already translated. What do you think?

Edit: Patch currently doesn't save the value, because sanitize_option() allows only one of get_available_languages().

[johnbillion]

I like that idea. Here are a few brain dumps:

• We should use two <optgroup>s instead of the separator made of dashes. One group for installed languages and one for available languages.
• We should add a wrapper function for wp_get_available_translations_from_api() which uses a short term transient to avoid hitting the API every time the General Settings screen is viewed.
• We should add a filter to allow the installation of new languages to be disabled.
• Needs to handle API request failures.

[ocean90]

Good points working on a new patch.

[SergeyBiryukov]

Related: #28831

[pavelevap]

I am using cs_CZ locale (WPLANG). In Settings - General is in the first place en_US (should be currently used language). I tried to save it and swith to original English version and nothing happened.

[ocean90]

Replying to pavelevap:
Don't use the WPLANG constant, it's just a fallback now. When you set the setting to en_US the option will be empty and so WPLANG is used.

[pavelevap]

ocean90: OK, thank you. But what about older installations (updated from 3.9.x)? All will have WPLANG defined and they could switch language to English original (and back). There should be any Help tooltip available or en_US option unavailable for them.

[knutsp]

Core can detect WPLANG and en_US selection, and then display a message that WPLANG needs to be removed to make en_US work correctly.

[ocean90]

15677.2.patch​ is a current state, looks like this: ​https://cloudup.com/cxEoda9nxh6

[netweb]

Using 15677.2.patch:

Installed r29573 and installed language to Korean, changed to English (UK) and went to WordPress updates to download the English (UK) translations.

Edit: Scrap the 'date' issues, unless you change your timezone and date format of course this will occur:
​https://cloudup.com/csCGjDOfHJL

On the dashboard Popular plugin and Install remain in Korean ​https://cloudup.com/cuU-QR27j0e

And the category title Uncategorized, remains in Korean, though I think this particular item will be wontfix and as it is 'content' end user should rename any 'created' content.

[ocean90]

show spinner while downloading language files

[jesin]

Fix undefined notices and HTTP url fallback

[jesin]

Changes in patch 15677.4.patch:

• Undefined variable $action in filters translations_api and translations_api_result changed to $type.
• WP.org API http: URL assigned to the $http_url variable for fallback to work.
• Variable $args in line 55 of translation-install.php changed to $options.

Bugs:

• Switching back from any language to "English" (default) shows Settings saved. admin notice in the previous language. This is because the default "English" has an empty value attribute.
• If there is problem communicating with WP.org and transients is empty the "Available Languages" list shows blank values except for English.

[ocean90]

15677.5.patch​:

• Fixes bugs mentioned by jesin
• translations_api():
  • $type = core: Pass WP version as $args['version'] to get core translations
  • $type = themes|plugins: Pass version as $args['version'] and theme/plugin slug as $args['slug'] to get theme/plugin translations
• Only allow super admins to install languages when is_multisite()

ToDo:

• Better handling of the default selected language. How should WPLANG constant and get_site_option('WPLANG') be handled, can we use get_locale() instead?

[ocean90]

Adds language chooser to network settings too

[ocean90]

Ditch WLANG constant

[DrewAPicture]

Docs adjustments

[ocean90]

15677.7.patch​:

• Ditches WPLANG constant
• On upgrade set WPLANG option based on the WPLANG constant
• An empty WPLANG option means the site is in English
• Adjust get_locale() so that an empty WPLANG option overrides the WPLANG constant

[ocean90]

Merges .7 (based on .6) and .8

[ocean90]

Add $wp_local_package to get_locale(). $wp_local_package can override WPLANG.

[ocean90]

There is currently one issue with wp-admin/upgrade.php: When WPLANG and $wp_local_package isn't set, get_locale() falls back to en_US because it doesn't load the options when WP_INSTALLING is set. This was added in [12900] (changed in [19059]) and [28774].

[jesin]

When installing WordPress with this patch, the "Information needed" screen shows the following two database errors:

WordPress database error: [Table 'wordpress.wp_options' doesn't exist]
INSERT INTO `wp_options` (`option_name`, `option_value`, `autoload`) VALUES ('_site_transient_timeout_available_translations', '1408896686', 'yes') ON DUPLICATE KEY UPDATE `option_name` = VALUES(`option_name`), `option_value` = VALUES(`option_value`), `autoload` = VALUES(`autoload`)
WordPress database error: [Table 'wordpress.wp_options' doesn't exist]
INSERT INTO `wp_options` (`option_name`, `option_value`, `autoload`) VALUES ('_site_transient_available_translations', 'a:41:{s:2:\"ar\";a:8:{s:8:\"language\";s:2:\"ar\";s:7:\"version\";s:9:\"4.0-alpha\";s:7:\"updated\";s:19:\"2014-01-01 00:00:00[...]

This is because wp-admin/install.php calls wp_get_available_translations() which tries to store the language data as transients.

[ocean90]

Don't set transients when WP_INSTALLING

[ocean90]

In load_default_textdomain() load WP_LANG_DIR . "/admin-$locale.mo" when WP_INSTALLING

[ocean90]

Patches .12 to .14 are going back to the main issue: Allow to install multiple variants of the same language, for example en_US, en_GB and en_AU, see also #28730.

To solve this we use wp_dropdown_languages() which will be populated by translations_api(). This will also fix #13069 and #19760 (probably wontfix).

15677.14.patch​ includes:

• Add optional $locale argument to load_default_textdomain(), used in wp_switch_default_textdomain()
• Remove !defined('WP_INSTALLING') from get_locale() for get_option( 'WPLANG' )
• Add $wp_local_package to get_locale()
• Allow an empty get_option( 'WPLANG' ) value to override WPLANG constant
• Rename wp_install_download_language_pack() to wp_download_language_pack()
• Rename wp_get_available_translations_from_api() to wp_get_available_translations(), API part extracted into translations_api()
• Rename wp_install_load_language() to wp_switch_default_textdomain()
• Remove define('WPLANG', ''); from wp-config-sample.php and populates WPLANG option on upgrade. That's not directly related to this issue, but makes some things easier to handle. Also I think 4.0 sounds like a good version number to get rid of this constant.

[ocean90]

populate WPLANG option on upgrade again

[ocean90]

In 29630:

Language packs: No WPLANG anymore.

• The WPLANG constant is no longer needed. Remove define('WPLANG', ); from wp-config-sample.php. Populate WPLANG option based on the WPLANG constant. When get_option('WPLANG') is an empty string it will override WPLANG.
• Introduce translations_api() which is available to communicate with the translation API. Move translation install related functions to a new file.
• Replace mu_dropdown_languages() with wp_dropdown_languages(). wp_dropdown_languages() is now populated by the translation API.
• Remove wp_install_load_language() and allow load_default_textdomain() to switch a core translation.

fixes #13069, #15677, #19760, #28730, #29281.

[pavelevap]

Nice, great work! Only small notice, "Note" is visible only when "English" language is selected. When I select "čeština" there is no "Note". Maybe it should be visible for all languages, it is a big change for localized versions and users should be notified that it is deprecated...

[nacin]

Replying to pavelevap:

Nice, great work! Only small notice, "Note" is visible only when "English" language is selected. When I select "čeština" there is no "Note". Maybe it should be visible for all languages, it is a big change for localized versions and users should be notified that it is deprecated...

This is by design, yep. Your WPLANG is set to pl_PL and so is your option, so there's no reason to nag you about it. This message is simply to cut down on support requests for people who change WPLANG and don't understand why it's not taking effect.

It does also issue a deprecated notice which will get picked up by Log Deprecated Notices, Debug Bar, etc.

[pavelevap]

OK, makes sense even if "Note" could be there in all cases.

I noticed another problem after 29630: During installation, "čeština" (please change it to "Čeština", my eyes are bleeding) was moved to the second place in dropdown (maybe locating my browser settings?), but there is still "Continue" and not localized version "Pokračovat" (maybe string changes?). Clicking on "Deutsch" works, there is localized "Fortfahren". But getting back to "čeština", there is still deutsch "Fortfahren" preserved. See attached screenshot.

[ocean90]

In 29633:

Language Form: The language option for $wp_local_package should get the same attributes as the other languages.

see #15677.

[ocean90]

Replying to pavelevap:

Thanks, fixed in [29633].

[netweb]

Replying to pavelevap:

During installation, "čeština" (please change it to "Čeština", my eyes are bleeding)...

Fixed ​https://glotpress.trac.wordpress.org/changeset/947

[pavelevap]

ocean90: Nice, works well now!

netweb: OK, but I can still see only "čeština" during installation? Maybe it is cached somehow somewhere?

Anyway, WPLANG is deprecated now and how can I change language through administration?

Example (very common): Initial installation is created in English and users want to switch to Czech. It could be easily done changing WPLANG, but what is the recommended approach now? I did not find anything in Settings - General and WPLANG is deprecated... Or I can use Czech version and want to switch to Slovak (very similar language). How can I do it? Should I create new ticket?

[ocean90]

Replying to pavelevap:

To switch the language you have to copy the translation files to wp-content/languages and then select it in the dropdown. So instead of touching the wp-config.php, you can now set it via the language dropdown.

Core will not install new languages. That was included in my patches until .13 but we decided do move this out of 4.0. I will create a new ticket for this feature and it will be a goal for 4.1.

[pavelevap]

ocean90: OK, thank you. It is a little bit of regression, I guess. Now I could add only cs_CZ to wp-config.php and everything worked. In 4.0 I will have to download localization pack (it will be available on cs.wordpress.org?), extract it on local computer, find localization files, copy them through FTP client in the right place and in the end change language in dropdown. Too many steps for common users, I guess. Maybe do not deprecate WPLANG yet? I like the idea to not touching wp-config.php by users, because there are many possible problems (Fatal error, BOM, etc), but current workflow for changing language is an overkill. Please, do not deprecate WPLANG until it will be possible to switch languages easier...

[markoheijnen]

Ok wow. That decision is quite disappointed for me and I'm sure a lot of others because to me that was one of the main things I expected from 4.0.

[ocean90]

Replying to pavelevap:

Just setting the WPLANG constant doesn't provide you the language files in 3.9, so there is no difference.

[pavelevap]

ocean90: That is not true, simply setting WPLANG and going to Dashboard - Updates to automatically install localized version works well in 3.9.x. So it is a regression and should be solved somehow... And it was also planned for 4.0.

Another related ticket (other problem with dropdown functionality): #29297

[johnbillion]

If we're installing new language files during installation, what's stopping us installing new language files when the WPLANG option is changed in the admin area? Anything I can help with?

[pavelevap]

I am not sure about the best solution, but now it is a mess. WPLANG is deprecated now, users have to manually copy language files through FTP, because adding languages through administration does not work. And when they copy only .mo files, they can switch to new language, but automatic language packs will not work (#29297).

So, generally we are telling to users: In 3.9.x you could simply do it with WPLANG, in 4.0 you have to copy files (and you have to learn what to do and how) and everything will be changed in 4.1 once again? Not good, long Codex page needed...

[ocean90]

Replying to ocean90:

Core will not install new languages. That was included in my patches until .13 but we decided do move this out of 4.0. I will create a new ticket for this feature and it will be a goal for 4.1.

See #29395.

[pavelevap]

ocean90: OK, but you should reconsider including this feature into 4.0. You are presenting really bad user experience for adding languages in 4.0. Better postponing release for some days than annoying users with current functionality, I guess...

[pavelevap]

Is it OK to have 2 strings for 1 sentence? "Note:" and "The %s constant in your %s file is no longer needed."? What about RTL languages?

[pavelevap]

Current behaviour is regression and there are other posts about this issue, see: ​http://make.wordpress.org/polyglots/2014/08/29/let-me-share-some-issues-about-4-0-language/

It will be really postponed and not included in 4.0?

[knutsp]

I agree that current behaviour, as described, is a regression. I 3.9 you can just change the locale in wp-config.php and update WordPress to a new version, as 3.9.2 nb_NO to 3.9.2 sv_SE.

Over the years I have advised several clients that somehow got the wrong language version installed, to correct this by just editing wp-config.php. So, if this happens again and the current functionality will be what's released, I will have a hard time advising these clients to help themselves through a simple file edit. We are back to the situation many years ago, when, in these cases, you had to get the latest translation files downloaded and put them in the right place, and change WPLANG.

So this either isn't ready to go, or a decision must be made to openly accept regression and make apologies for that. 4.0 should be great when it comes to localization.

[ocean90]

#28736 was marked as a duplicate.