non-latin category names not alpha sorted (broken in 2.5.1)

[jeremyclarke]

This is a wierd one, but the revamping of the category code to use the Walker class in 2.5.1 broke the alphabetical sorting of categories for my arabic editor (seperate site in arabic with translations of english content). We have more than 200 categories in a hierarchy, so it is vital that authors be able to scan quickly to find categories without reading each one.

In all previous versions it worked fine, but at 2.5.1 there was no discernible sorting of category names. I tracked it down to the removal of the sort_cats() function, which was used with usort() to sort the categories array alphabetically inside the get_nested_categories() function. It uses the strcasecmp() php function to compare the names which seems to work in all cases at sorting Arabic (my Bangla editor confirmed that Bangla is also affected, there must also be others).

Right now the get_terms() function is supposed to deal with the sorting using the 'orderby' argument and feeding that to the SQL so that the results are already sorted (~line 595 wp-includes/taxonomy.php).

Unfortunately, for my setup (which worked before), Arabic category names just don't get sorted by name. I am using mysql version 5.0.22 and the collation of the whole DB is utf8_unicode_ci, it's possible that a different setup would fix the problem, but I think that having it work on a standard setup by default it a very good idea (tested it on a centos private server and on Dreamhost with same results).

Now, I can write a quick bit of plugin code to hook into

apply_filters('get_terms', $terms, $taxonomies, $args);

and run the sorting function myself on the categories, but it seems to me that it would be a lot more user-friendly to make this part of core so that users (especially Arabic users, who are way less likely to be able to wade through forums and trac to find out why their categories are randomly sorted) don't even have to think about it, it should just work.

As a point of consideration, I think it would be foolish to let a plugin called "Make Non-Latin Category Names be Sorted on Write Screen" even exist.

My patch creates the _usort_terms_by_name() function to do the sorting. It seems to be exactly what is needed. I set the actual sorting to happen near the end of &get_terms() before the $terms array is reset but after empty terms are removed, obviously it only does the sorting if the $argsorderby? value is 'name'. If you guys think it should happen elsewhere no beef by me, though it seems that somewhere in get_terms() is the best idea so that other times when it's called the same effect is felt (like a sidebar list, which should also be alphabetical if that was requested). If the position of the sort in get_terms() seems unelegant we could instead do an add_filter('get_terms') around line 595 when we're already going through the $orderby value, but the effect will be the exact same (let me know if you'd like another patch in that format). Here's the basic code:

Resort by name in PHP in case SQL didn't (for non-latin alphabets)
if ( 'name' == $argsorderby? )

usort( $terms, '_usort_terms_by_name' );

Any thougts, scoldings welcome. Sorry to reintroduce bloat, but I think it would be valuable for all the international WP users.

(it would be great if this could go into 2.5.2 as it's pretty non-destructive and basically reverts a behavior from <2.3.3

[jeremyclarke]

add php alpha sorting to get_terms() in taxonomy.php

[jeremyclarke]

oops, two corrections.

1: I didn't create the _usort_terms_by_name() function, just used it.

2: $argsorderby? is an unfortunate victim of the wikiformatting, it should be

	// Resort by name in PHP in case SQL didn't (for non-latin alphabets)
	if ( 'name' == $args['orderby'] ) 
		usort( $terms, '_usort_terms_by_name' );

[jeremyclarke]

Any thoughts on this? Is it going to be denied?

[ryan]

Milestone 2.5.2 deleted

[jeremyclarke]

updated patch to fix sorting

[ryan]

The PHP sorting just does a strcmp and is not sensitive to character set and collation. MySQL should be, however. Maybe we need to make do with a filter and a plugin while figuring out the underlying problem. I don't want to stomp on someone's working db collation with strcmp.

[jeremyclarke]

In case someone finds this and wants the plugin code solution, this works for me:

// --------------------------------------------
// Refilter fetched terms alphabetically in PHP
// --------------------------------------------
//
// Needed to ensure ar/bn etc have alpha sorted
// cats despite some failure in mysql or something
//
// jer wrote a patch to wp but it was denied
//
// the filter: 	$terms = apply_filters('get_terms', $terms, $taxonomies, $args);
// $terms is the fetched terms, $taxonomies is like category or tags, passed into get_terms, 
// $args is the mixed arguments passed into get_terms or else defaults from the function
//

function gv_filter_get_terms($terms, $taxonomies, $args) {
	// name is default order, but check in case we are running custom checks
	if ( 'name' == $args['orderby'] ) 
		usort( $terms, '_usort_terms_by_name' ); // usort func is part of wp core
	reset ( $terms );
	return $terms;
} // gv_filter_get_terms()

add_filter('get_terms', 'gv_filter_get_terms', 10, 3);

[Denis-de-Bernardy]

the collation is what defines how things get alpha sorted. suggesting invalid/wontfix.