curly quotes not stripped out slugs

[alxndr]

Curly single- and double-quotes are percent-encoded instead of stripped out when creating slugs for post titles, categories, or tags.

To recreate, add a new post with the title: Procol Harum’s “A Whiter Shade of Pale”
expected slug: procol-harums-a-whiter-shade-of-pale
actual slug: procol-harum%e2%80%99s-%e2%80%9ca-whiter-shade-of-pale%e2%80%9d

patch to sanitize_title_with_dashes() in formatting.php, replaces curly single- and double- quotes and en- and em-dashes with empty string

I'm all for the fix, but it should be str_replace instead of preg_replace, no?

No version or svn revision info included, please always set and include in description as well for bugs.
​http://codex.wordpress.org/Reporting_Bugs#Reporting_a_Bug

This is not a newly introduced issue (regression), updating milestone to 3.0 as too late for existing, lower impact issues.

Please see #9591, which I believe this strongly relates to -- better to work on a more cohesive solution, then one offs.

#9591

Closed #16036 as duplicate.

I guess sanitize_title_with_dashes() is a more appropriate function for such replacements than remove_accents(), which handles i18n-related improvements.

So I suggest to reopen that ticket.

Refreshed the patch for 3.3.

This falls under "Finally fix the issues relating to special characters in permalinks using an upgrade routine" from 3.3 scope.

Not sure if an upgrade routine is necessary here. Old permalinks (with curly quotes) still work after the patch.

Moved sanitization to sanitize_title(), as per ​today's dev chat.

Actually, the best place would be remove_accents().

I guess remove_accents() should only handle international characters.

Yeah, you're right, but the rest of the special chars are handled elsewhere and hooked into 'sanitize_title'.

I'm not sure why nacin said we shouldn't rely on the filter.

Oh yeah. Because it wouldn't have the 'save' context check.

Replying to scribu:

Yeah, you're right, but the rest of the special chars are handled elsewhere and hooked into 'sanitize_title'.

I'm not sure why nacin said we shouldn't rely on the filter.

I was referring to something else.

This logic probably makes the most sense in sanitize_title_with_dashes. We can alter the add_filter() call to let the context arg be passed in.

Replying to nacin:

This logic probably makes the most sense in sanitize_title_with_dashes. We can alter the add_filter() call to let the context arg be passed in.

Done in 10797.3.patch​.

Looks good. Let's get some unit tests.

Based on the tickets linked from #9591, 10797.5.patch​ fixes:

• #3206: ¡ and ¿ should get stripped
• #8765: Strip ° characters from permalink
• ticket:9591:30: Guillemets
• #12956: © and ™ not stripped from sanitize_title

• #15768: Unencoded non-breaking spaces in titles don't get sanitized properly

unit tests

Add unit tests for the latest patch

In [18705]:

Strip a number of special characters in sanitize_title_with_dashes on save. Includes quotes (curly, angle), dashes, marks, etc. props SergeyBiryukov. props ampt for the unit tests in ​[UT438]. see #10797.

:-) Thanks ampt! Things look great.

Before patch: Tests: 14, Assertions: 15, Failures: 6.

After patch: OK (14 tests, 22 assertions)

​http://unit-tests.trac.wordpress.org/changeset/438

Think this one is done.