sanitize_title_with_dashes() has issues with non breaking space (  and  )

[michelski]

This is a follow-up to #13623. Maybe related to #31499 and #19292.

If you have a   oder   (non breaking space) in between the words in your title, the slug won't get converted into a dash, instead the words before and after the nbsp are merged.

So, if you have the title

Die Crux mit Blogsystem und Wordpress

or

Die Crux mit Blogsystem und Wordpress

the permalink is created to

die-crux-mit-blogsystem-undwordpress

and not

die-crux-mit-blogsystem-und-wordpress

If you just put a " " in the title, it works fine.

I have struggle with this since I imported some postings from a Textpattern blog (some years ago, thinking of 2009) which had a function named no_widow turned on. This had put the   in the titles.

[swissspidy]

This bug still exists in trunk.

The problem is this line in sanitize_title_with_dashes():

$title = preg_replace('/&.+?;/', '', $title); // kill entities

A bit further down, we'll see:

// Convert nbsp, ndash and mdash to hyphens
$title = str_replace( array( '%c2%a0', '%e2%80%93', '%e2%80%94' ), '-', $title );

This should be extended to replace the characters mentioned here.

[polevaultweb]

@swissspidy would you mind having a look at the patch when you get a second? Cheers :)

[swissspidy]

Replying to polevaultweb:

@swissspidy would you mind having a look at the patch when you get a second? Cheers :)

The replacement should probably only be done if 'save' === $context, just like further down the function. That's what you're testing, after all. With your current patch you're doing it on display as well.

Regarding the test, I'd personally stick to the "1 assertion per test" rule used in most of the other functions — and use non-German strings for testing :). Example (taken from another test):

<?php
function test_replaces_ndash_entities() {
        $this->assertEquals("do-the-dash", sanitize_title_with_dashes("Do&ndash;the Dash", '', 'save'));
        $this->assertEquals("do-the-dash", sanitize_title_with_dashes("Do the&#8211;Dash", '', 'save'));
}

Oh, and you should add the @ticket 31790 annotation for every new test method you introduce, not only the first one.

[SergeyBiryukov]

In 36775:

Formatting: In sanitize_title_with_dashes(), convert &nbsp, &ndash, and &mdash HTML entities to hyphens on save.

Props polevaultweb for initial patch.
Fixes #31790.

[westonruter]

In 61061:

Formatting: Replace non-breaking hyphens (U+2011) with hyphens in sanitize_title_with_dashes().

Developed in ​https://github.com/WordPress/wordpress-develop/pull/10204

Follow-up to [18705], [36775].

Props patpiwo, westonruter, dmsnell.
See #31790, #10797.
Fixes #64089.

## Problem

sanitize_title_with_dashes() only converted non-breaking spaces (%c2%a0,  ,  ) to hyphens in the save context. In the default display context:

• the   /   HTML entity was later stripped as generic HTML, merging the surrounding words (foo bar → foobar);
• the UTF-8 form was left as the literal foo%c2%a0bar.

This surfaces on content imported from platforms that insert non-breaking spaces into titles (e.g. "no widow" helpers).

## Change

Move the non-breaking space replacement above the context branch so it runs in all contexts, before the blanket entity removal. –, —, and the non-breaking hyphen remain save-only (unchanged).

## Testing

Added test_replaces_nbsp_in_display_context covering the UTF-8 char and both entities in the default context. Full sanitizeTitleWithDashes suite passes (95 tests), plus sanitizeTitle (2 tests).

Before:

nbsp entity   display:[foobar]        save:[foo-bar]
utf8 nbsp     display:[foo%c2%a0bar]  save:[foo-bar]

After:

nbsp entity   display:[foo-bar]  save:[foo-bar]
utf8 nbsp     display:[foo-bar]  save:[foo-bar]