Use of localized calendar system and translating digits in entire WordPress

[kambiz.k]

Current implementation of WordPress (v2.6.2) do not offer a mechanism to globally convert Gregorian Calendar System to Localized Calendar System, and to translate Latin Digits to Localized Digits.

Localization of Calendar System

To convert Gregorian Calendar System to Localized Calendar System (like Jewish, Hijri, Jalali, and so on) some users use the_post and get_comment_date filters. However this is just a workaround and has some problems:

1. These filters have to ignore output of their respective WordPress functions, and recompute the new output from scratch. This is not only a performance drawback, but also breaks consistency of code.

2. Every single filter should have its own callback function because each callback can manage only one object (e.g. post or comment). If user needs to localize dates in other places, should write new callback functions.

5. In some cases, there is no way to use filters for changing calendar syste (for example dates shown by plugins or dates shown in admin area).

The solution is quite simple. WordPress as well as plugins use either mysql2date or date_i18n to format date and time. What we need is a filter in these functions to apply on the unixtimestamp and return the converted date/time.

Actually, we need the filter only in date_i18n function. When translate parameter of mysql2date function is true, it should call date_i18n but in the current implementation has duplicated code).

Localization of Numbers

As same as the calendar system, some users get benefit of available filters in conjunction with heavy regular expressions to translate digits of numbers to their language locale. Of course there is no filter for every single item and users have to leave some numbers as Latin.

WordPress has already a function named number_format_i18n that eveybody calls it to format numbers. If we apply a filter to output of this function, we can translate digits of numbers all in one place, and of course with a better performance.

By applying a filter on output of date_i18n function, we can translate digits in date and time too.

RECOMMENDED CODE

Here is the code that I suggest for mysql2date, date_i18n, and number_format_i18n functions (attached as a patch and tested with v2.6.2).

function mysql2date( $dateformatstring, $mysqlstring, $translate = true ) {
	global $wp_locale;
	$m = $mysqlstring;
	if ( empty( $m ) )
		return false;

	if( 'G' == $dateformatstring ) {
		return gmmktime(
			(int) substr( $m, 11, 2 ), (int) substr( $m, 14, 2 ), (int) substr( $m, 17, 2 ),
			(int) substr( $m, 5, 2 ), (int) substr( $m, 8, 2 ), (int) substr( $m, 0, 4 )
		);
	}

	$i = mktime(
		(int) substr( $m, 11, 2 ), (int) substr( $m, 14, 2 ), (int) substr( $m, 17, 2 ),
		(int) substr( $m, 5, 2 ), (int) substr( $m, 8, 2 ), (int) substr( $m, 0, 4 )
	);

	if( 'U' == $dateformatstring )
		return $i;

	if ( -1 == $i || false == $i )
		$i = 0;

	if ( $translate ) // localizing names or calendar system
		$j = date_i18n( $dateformatstring, $i );
	else
		$j = @date( $dateformatstring, $i );

	/*
	if ( !$j ) // for debug purposes
		echo $i." ".$mysqlstring;
	*/

	return $j;
}

function date_i18n( $dateformatstring, $unixtimestamp ) {
	global $wp_locale;
	$i = $unixtimestamp;
	// Sanity check for PHP 5.1.0-
	if ( -1 == $i )
		$i = false;
  
 	// Let the user convert date from Gregorian to localized calendar system
	$j = apply_filters( 'pre_date_i18n', $dateformatstring, $i ); 
	if ($j !== $dateformatstring)
		return $j;

	if ( ( !empty( $wp_locale->month ) ) && ( !empty( $wp_locale->weekday ) ) ) {
		$datemonth = $wp_locale->get_month( date( 'm', $i ) );
		$datemonth_abbrev = $wp_locale->get_month_abbrev( $datemonth );
		$dateweekday = $wp_locale->get_weekday( date( 'w', $i ) );
		$dateweekday_abbrev = $wp_locale->get_weekday_abbrev( $dateweekday );
		$datemeridiem = $wp_locale->get_meridiem( date( 'a', $i ) );
		$datemeridiem_capital = $wp_locale->get_meridiem( date( 'A', $i ) );
		$dateformatstring = ' '.$dateformatstring;
		$dateformatstring = preg_replace( "/([^\\\])D/", "\\1" . backslashit( $dateweekday_abbrev ), $dateformatstring );
		$dateformatstring = preg_replace( "/([^\\\])F/", "\\1" . backslashit( $datemonth ), $dateformatstring );
		$dateformatstring = preg_replace( "/([^\\\])l/", "\\1" . backslashit( $dateweekday ), $dateformatstring );
		$dateformatstring = preg_replace( "/([^\\\])M/", "\\1" . backslashit( $datemonth_abbrev ), $dateformatstring );
		$dateformatstring = preg_replace( "/([^\\\])a/", "\\1" . backslashit( $datemeridiem ), $dateformatstring );
		$dateformatstring = preg_replace( "/([^\\\])A/", "\\1" . backslashit( $datemeridiem_capital ), $dateformatstring );

		$dateformatstring = substr( $dateformatstring, 1, strlen( $dateformatstring ) -1 );
	}
	$j = @date( $dateformatstring, $i );       

	// Let the user translate digits from latin to localized language
	return apply_filters( 'date_i18n', $j);
}

function number_format_i18n( $number, $decimals = null ) {
	global $wp_locale;
	// let the user override the precision only
	$decimals = ( is_null( $decimals ) ) ? $wp_locale->number_format['decimals'] : intval( $decimals );

	$num = number_format( $number, $decimals, $wp_locale->number_format['decimal_point'], $wp_locale->number_format['thousands_sep'] );
  
	// let the user translate digits from latin to localized language
	return apply_filters( 'number_format_i18n', $num ); 
}

Example of Usage

We need two functions as core of our localization.

// Converts Latin digits to Persian ones
function latin_to_persian($number) {
  $latin = array('0', '1', '2', '3', '4', '5', '6', '7', '8', '9'); 
  $persian = array('۰', '۱', '۲', '۳', '۴', '۵', '۶', '۷', '۸', '۹');
  return str_replace($latin, $persian, $number);
}

// Represents a Unix timestamp as Jalali date/time
function gregorian_to_jalali($formatstring, $unixtimestamp) {
  // do the work and calculate $the_jalali_date
  return latin_to_persian($the_jalali_date);
}

By use of the new filters we localize date and numbers in entire WordPress with no pain.

// Changes date to the new calendar system
add_filter('pre_date_i18n', 'gregorian_to_jalali', 10, 2);
// Translates digits of numbers
add_filter('number_format_i18n', 'latin_to_persian', 10, 1);

That's all!

[kambiz.k]

Patch to enhance localization system, tested with v2.6.2

[kambiz.k]

Sorry, I missed to mention that the patch is for /wp-includes/functions.php

[ryan]

Moving enhancements to 2.8.

[kambiz.k]

Update Patches for WordPress 2.7

[Denis-de-Bernardy]

broken patch. please include diff files next time, it helps when you're automatically testing patches

[kambiz.k]

What was wrong with the patches?!

[Denis-de-Bernardy]

Replying to kambiz.k:

What was wrong with the patches?!

see #9724

DB:~/Sites/wp $ wptest http://core.trac.wordpress.org/attachment/ticket/7753/wp-includes_functions.zip
patch unexpectedly ends in middle of line
patch: **** Only garbage was found in the patch input.

and once wp-includes_functions.zip is unzipped, they don't apply cleanly against trunk.

please refresh them: the committers are going through the commit list since yesterday.

[kambiz.k]

I updated patches for SVN revision 11200.

Some of modifications were already applied on the latest revision. I think because of that the older patches were broken.

Thank you.

[Denis-de-Bernardy]

the stuff in Patch.zip, in a single diff format

[Denis-de-Bernardy]

won't the changes to get_inline_data() potentially adversely affect other languages?

[Denis-de-Bernardy]

I'm closing this as invalid, but please understand it as fixed. a few trac versions back, we added two lines to date_i18n(), as follows:

	// allow plugins to redo this entirely for languages with untypical grammars
	$j = apply_filters('date_i18n', $j, $req_format, $unixtimestamp, $gmt);
	return $j;

the use-case was slightly different (it was for a polish translator), but it does the exact same job as what you're doing with the pre_date_i18n, and it seems to me that this'll allow to achieve what you're looking for without a hiccup.

[Denis-de-Bernardy]

doh, sorry, I hadn't noticed the number_format_i18n.

[Denis-de-Bernardy]

fix number_format only

[kambiz.k]

Patch files for rev. 11200 (Updated)

[Denis-de-Bernardy]

the number format only patch is against today's trunk. by using the date_i18n hook in place of your own, you should be able to make it works as you need it to.

[kambiz.k]

Yes, the new date_i18n hook does the trick however changes in wp-admin/includes/template.php are required. My old patch for this file was incorrect and I updated it now.

In touch_time function of wp-admin/includes/template.php file, mysql2date function is used to get numeric values of year, month, and day. This values should be always in Gregorian calendar system without any translation. However, translate parameter of mysql2date function is set to true by default. You should set value of translate parameter to false for mysql2date calls in touch_time function.

[Denis-de-Bernardy]

Ya, I think I'm understanding what you're meaning. Took me a good hour of playing around to make sense of it, too.

My understanding of the code in that area and of your suggested change to the mysql2date call in touch_time() is that, it may prevent other locale (that use the standard calendar) from working properly. Trying to rephrase what you're suggesting:

• You'd like to be able to output numbers in a non-latin format -- patch 7753.2.diff lets you do just that, but I'll open a separate ticket else it'll get lost in this one.
• You'd like the WP front end, and the back-end, to display dates in your localized calendar.

By the latter point, you're not merely meaning changing the number format. You're also meaning that it should be able work with other calendars such as jalali, jewish, maya, julian, french revolutionary, and so on... These calendars don't all 7 days a week, or 365 days per year (some are lunar based).

Making that happen would involve huge workflow changes in the WP code base: your suggested change to touch_time() would also be relevant to scores of other template tags, and the number_i18n filter is missing on the likes of pagination template tags. And I dare not even imagine how this translates into DB queries. How does one group posts by month in a non-gregorian calendar? :D

The get_calendar() function outputs a Gregorian calendar in that format. It lets one change the start date, and that's about it... To make the existing code work with a non-gregorian calendar, we'd want to look into:

• the number of days per week which isn't always 7 (assuming there is any kind of week concept to start with)
• something to make special days work (in some calendars, some days don't belong in any month)
• lunar months, and scores of other oddities, rather than gregorian months
• non-decimal number support (some cultures count in base-5 and base-20)

Personally, I'd advise to write a plugin...

[Denis-de-Bernardy]

Moving this back to Future, pending dev feedback.

[Denis-de-Bernardy]

See also: #7837

[kambiz.k]

You got half of the issue. I am taking only about displaying dates in another culture (or calendar system) only in pages. It is very difficult to input dates in other calendar systems. In the other hand, it is worthless because the author can easily enter the date as Gregorian. I try to describe the issue better.

In touch_time(), WP uses mysql2date function to get fields of the publish date/time. User may change these fields, but anyway with or without modification later these fields will be used to create a timestamp for storing in DB.

If somebody write a plugin and use the new date_i18n hook to override calendar system or to localize date/time digits, it will corrupt the fields of publish date. While editing a post, the fields of its publish date/time should kept in Gregorian calendar with Latin digits to prevent such problem.

To get rid of problem you only need to pass false for translate parameter of mysql2date calls in touch_time(), otherwise nobody can use date_i18n hook. Be sure, it doesn't affect other parts of code.

[Denis-de-Bernardy]

Ahh, that makes a lot more sense now. I had miserably failed to install a corrupt fr_FR file earlier, to try to make things break. :-)

So there is genuine potential for bugs as it is now, due to the new hook. Let's open a separate ticket for that.

On the front end:

• The calendar widget has a callback method that can be overridden. I take it that writing a separate calendar function should get the job done.
• Ticket #7837 had an early discussion on number formats.

[Denis-de-Bernardy]

New ticket related to the hook is at #9730

[Denis-de-Bernardy]

7753.2.diff should be safe to apply now.

[Denis-de-Bernardy]

the check that is needed is to verify that thing that get processed through number_format_i18n() never get played around with javascripts.

[ryan]

(In [11407]) Add number_format_i18n filter. Props kambiz.k. fixes #7753

[chriscct7]

#7837 was marked as a duplicate.