Abbreviated year followed by punctuation or markup doesn't texturize

[pah2]

An abbreviated year followed by punctuation or markup doesn't texturize properly.

e.g. (Bruce Sterling, '97) is texturized as (Bruce Sterling, ‘97) when the apostrophe should be texturized as ’

e.g. <li>Casino Royale '06</li> is texturized as <li>Casino Royale, &#8216;06</li> when the apostrophe should be texturized as &#8217;

Note that the patch I attached also includes the patch I created for ​Ticket #1258.

I tried applying this against trunk 10289 but it didn't seem to fix the issue for me.

my mistake, I was testing against the wrong site. D'oh.

The attached fixes the curly quotes and s and the issue above.

wp-include formatting.php

Refreshed patch

broken patch

Added Test cases for this to wordpress-tests.

Moving to Future Release until it has a working patch

#1258 and #11275 closed as duplicates.

Diffed against r14075.

Here's another go ready for testing. This patch was created against r14075.

Replying to jmstacey:

Here's another go ready for testing. This patch was created against r14075.

Thank You!

I ran the unit tests against the patch (​http://svn.automattic.com/wordpress-tests/wp-testcase/test_includes_formatting.php) with the following results:

• TestWPTexturize::test_quotes_before_s - these tests now pass
• TestWPTexturize::test_quotes - still fail
• TestWPTexturize::test_quotes_before_numbers - simple case passes quoted case doesn't.

So we are heading in the right direction but still need some more work.

I'll commit the current patch but leave the ticket open for improvements.

(In [14095]) Improve behaviour of wptexturize with respect to single quotes. See #4539 props jmstacey.

Going to punt this along for continued work.

Working on a patch that makes all failing test cases pass.

I was able to make 2 of the failing test cases pass by reorganizing the order of patterns in r16195.

Needs another patch for test_quotes() with HTML tags and tag attributes in it.

Technically related to #15241.

Almost done with the simplified patch that resolves all issues.

Submitted a patch above with regexps rewritten from scratch that makes all existing test cases pass, enables disabled test cases to run, including skipped tests and I also added several new cases that all pass.

The only edge case it does not handle is when you want to combine units of degrees, specifically minutes and seconds with the same class of quotes, e.g. ' with ' and " with ". I don't think that can correctly be fixed.

The patch also adds support for quotes spanning multiple html tags and/or shortcodes.

It also fixes #15241.

(​[UT317]) Extra tests for wptexturize from ​#WP4539 props norbertm.

I switched the escaped slashes back to using a double quoted string for the string with single quotes in in test_other_html so as to follow the WP Coding Standards.

(In [16280]) Improved RegEx for quote matching in wptexturize. Fixes #4539 and #15241 props norbertm.

See my comment in #15241.

(In [16409]) Revert changes to wptexturize() until test cases pass. Reverts [16280], [16378]. see #4539 and #15241.

Added snapshot of progress. Everything passes including #15241 except #15444. Working on that and test coverage.

Added new snapshot.

Includes fix for #15444.
Follows WP coding standards.
Improved test coverage.

Cross-referencing #10606 as another one of those texturize bugs.

Updated patch and included further tests. Fixes #10606 as well.

Aiming to improve performance by getting rid of the two preg_replaces() within the while()s with no success so far (we need the count parameter that str_replace() does not have). Is this a performance sensitive function or cached?

Replying to norbertm:

Is this a performance sensitive function or cached?

The output of this isn't cached

Let me see if I can further optimize it.

Any other tickets/concerns related to this?

Replying to norbertm:

Let me see if I can further optimize it.

Any other tickets/concerns related to this?

A quick search identified the following open texturize-related tickets: #4539 #4116 #6969 #8912 #10033 #15241 #15444.

Another: #14491.

Thanks for collecting related tickets. Here is another patch.

• No change required for #4116. Added tests.
• Added tests for #6969 (incomplete).
• Fixes HTML comments (#8912, #10033). Tests included.
• #15241, #15444 already covered.

Waiting for feedback on #14491.

Optimized code for performance by replacing two preg_replace()s to str_replace()s (and while()s to if()s).

WPTexturize test results: OK (21 tests, 130 assertions)

I think it's getting mature. Will post a followup once I'm done.

Any other thoughts?

Working in #14491 which needs some rethinking.

Next dev snapshot. Adds test cases and fix for #14491.

There is one issue left. How do you convert

'Class of '99'

into

&8216;Class of &8217;99&8217;

and not

&8216;Class of &8216;99&8217;

if you still want to convert '99' into &8216;99&8217;?

Obviously it depends on the starting quote but that's a leftover quote to be evaluated later to allow quote state to persist across tags and shortcodes. Still thinking about this.

Another snapshot.

Fixes #4116, #4539, #8912, #10606, #14491, #15241, #15444.

Partially covers #6969 and #10033.

Includes a bunch of new unit tests all of which pass now (23 tests, 162 assertions in total).

I'm not totally confident with the current patterns and test coverage. Still reviewing those.

Completed the review. I think it can't become much better than this. It's pretty well covered with tests too. If you have suggestions please post it, otherwise I think the patch could be applied and the fully covered tickets mentioned in the second post above can be closed together with this.

How's the performance?

Closed #15241, #10606, #15444 all as duplicates. Should be confirmed as fixed before this ticket is ultimately closed.

Then I'll paste my ​test-post from #10606, simply to present that some mundane punctuation uses still just don't work (only for closing quotes):

• text "[link]".
• text "[link]", text.
• text "[link]"; text.
• text "[link]"- text.
• text "text [link]".
• text "text [link]"...
• text "text [link]"text.

All these have the opening quote applied instead of the closing quote (and then the locale is applied, the closing quote in fr-FR being "« ").

• text "text number".
• text "text number" text.

These last two have the correct closing quote applied, but the locale is not taken into account.

I did provide test cases for all of these, a year ago now. If they're wrong, let me know about it, I'll update them.

Test blog is running WordPress 3.1-beta1-16732.

@xibe - It hasn't been committed to SVN yet. The patch is available above. It makes your test cases submitted in #10606 pass. To make sure I've included new tests for the simplified strings that you just submitted. All pass now.

@nacin - As for performance, it's hard to tell because it highly depends on the strings that are passed to it. The original test suite did not pass and many tests were skipped so there's nothing to compare against. What would you suggest for benchmarking this? Please keep in mind that the original function returns incorrect results so we may be comparing code that generates incorrect vs correct output.

As for optimization, I was able to replace a preg_replace() to an str_replace() and a while() loop to a single if() by realizing the same could be accomplished with these better performing functions/constructs.

@norbertm: Thanks for all your hard work on this.

I think it is best left for implementation in 3.2 so we get a good deal of time for the changes to soak in trunk.

#9306 should be reviewed when the changes here are reviewed and committed.

See #8912 re: commented HTML in the post content.

Hello!

May I ask for renewed interest in this ticket? Is that patch from norbertm 1 year ago still applicable (somehow I doubt it)? May I help in any way (not much of a coder, but I can add more tests to my batch from #11099)?

Thanks all!

"like" 42" patch.

Tiny patch for those situation where a quote ends with digits.

I still have the issue with the wrong quotes in the case

text "[link]", text.

Related: #22971