Make WordPress Core

Opened 3 years ago

Closed 2 years ago

#29608 closed defect (bug) (duplicate)

wptexturize messes up shortcode with parameter meta_compare=">="

Reported by: bobbingwide Owned by: miqrogroove
Milestone: Priority: normal
Severity: normal Version: 4.0
Component: Formatting Keywords: needs-patch wptexturize
Focuses: Cc:

Description (last modified by SergeyBiryukov)

This is a direct follow on to #28564.

I have a number of shortcodes where I pass args to get_posts() almost directly from the shortcode.

In one particular example I use meta_compare="<" to display posts which are earlier than a certain date.
In another I use meta_compare=">=" to show posts on or greater than the date.

In WordPress 4.0 the ">=" meta_compare has stopped working.

Expected result: WordPress 3.9.1

[shortcode meta_compare=">=" ]

will end up passing a meta_compare value of >= to get_posts(). i.e.

[meta_compare] = >=

Unexpected result: WordPress 4.0

the array field value passed to get_posts() becomes

[meta_compare] => &#8217;>=&#8217;

This problem is caused by wptexturize(), which converts a double quote to &#8221; or a single quote to &#8217;

Pragmatic workaround:

 remove_filter( "the_content", "wptexturize" );

Alternatively, I could introduce a specific test on the meta_compare parameter value and strip unexpected entities.
I may have to do this anyway since I've seen a situation where the visual editor has converted > to &gt; which has caused a similar failure.

Change History (21)

#1 @SergeyBiryukov
3 years ago

  • Description modified (diff)
  • Milestone changed from Awaiting Review to 4.0.1

#2 @miqrogroove
3 years ago

There is a possible compromise here. Since it looks like we may have to import the registered shortcode names for #29557 anyway, it may not be necessary to filter the contents of the shortcodes at all. The only test that fails in this case is the "Let's get crazy" test from test_tag_avoidance(). That particular test allowed recursive nesting of square braces within a hyperlink element within a shortcode. I'm guessing that case is less valuable than the one described above in the ticket.

#3 @miqrogroove
3 years ago

It looks like we can use this patch if there are no other problems reported:


#4 @miqrogroove
3 years ago

  • Resolution set to duplicate
  • Status changed from new to closed

Duplicate of #29557.

Going to wrap these all into one ticket because they have the same patch.

#5 @miqrogroove
3 years ago

See #29661 for discussion about future versions.

#6 @nacin
3 years ago

  • Milestone 4.0.1 deleted

#7 @bobbingwide
3 years ago

Just to confirm that the patch has fixed the problem that I reported.

#8 @bobbingwide
3 years ago

  • Resolution duplicate deleted
  • Status changed from closed to reopened

The fix for #29557 in 4.0.1 has not fixed this problem.

Actually the problem has got worse.

It no longer accepts
[shortcode meta_compare="<"]

My pragmatic workaround is still needed.
I am reopening this defect since it actually occurs in a live site.

Last edited 3 years ago by bobbingwide (previous) (diff)

#9 @miqrogroove
3 years ago

  • Keywords needs-patch wptexturize 4.2-early added

#10 @miqrogroove
3 years ago

Yes we are aware of the problem. It is documented here:


I've done some brainstorming here:


The fixes in 4.0.1 were a time-consuming effort and since 4.1 beta is out already we are looking at 4.2 for a full solution.

#11 @miqrogroove
3 years ago

I should also caution that there might not be a solution to using meta_compare=">=". We might end up fixing #15694 to provide better escaping functions and then conclude the raw angle braces are a wontfix idea. It depends heavily on design and backward compatibility decisions being made in other areas. To fix one thing we have to break another, etc.

#12 @SergeyBiryukov
3 years ago

  • Milestone set to Awaiting Review

#13 @bobbingwide
3 years ago

OK, I have reviewed miqrogroove's blog and appended my comments, which were to agree on implementing the filters using different priorities.

I have developed a local patch that

  1. implements the priority change
  2. supports ‘no_texturize_shortcodes’
  3. and replaces wptexturize with a new filter function that

a) detects no texturize tags created around shortcodes which don’t like having their output texturized AND
b) calls a stripped down version that doesn’t do any regex processing on shortcodes…

It is currently a combination of one of my plugins and some changes in formatting.php

Changes are:

do_shortcode_tag() now wraps the result of a 'no_texturize_shortcodes' shortcode with
<!--notext:--> and <!--dotext:-->

the replacement for wptexturize() loops through the output finding notext:'s and dotext:'s

  • stuff before a notext: is passed to the stripped down texturize function
  • everything between notext: and the next dotext: is left as is
  • then the loop continues after the dotext:

In my solution I also reprioritize wpautop() - invoking it so that it doesn't convert newlines to <br />.
I don't call shortcode_unautop() either.

I'm now in the process of moving the plugin code into core so that I can run unit tests.
But I'm happy with what I see. My shortcodes work as they did in 3.9 and wptexturize logic works as well.

Another option that I discussed ( with johnbillion ) was to implement a solution that supports condition expressions. LT, LE, EQ, GE, GT etc... I can develop this solution for my own code, OR it could be implemented in WP_Query. Comments?

#14 @miqrogroove
3 years ago

It's something that might get traction during the 4.2 cycle or might not. I will help work on this where I can, but after the 4.1 release at the earliest.

#15 @boonebgorges
3 years ago

Another option that I discussed ( with johnbillion ) was to implement a solution that supports condition expressions. LT, LE, EQ, GE, GT etc... I can develop this solution for my own code, OR it could be implemented in WP_Query. Comments?

I'd be happy to talk about this for 4.2 or beyond. A separate enhancemet ticket might be appropriate.

#16 @bobbingwide
3 years ago

OK, will raise separate enhancement ticket for condition expressions.

#17 @miqrogroove
3 years ago

#29658 was marked as a duplicate.

#18 @miqrogroove
3 years ago

Can we just make this a duplicate of #15694? At this point they are both solved by proper escaping.

#19 @obenland
2 years ago

  • Owner set to miqrogroove
  • Status changed from reopened to assigned

#20 @obenland
2 years ago

  • Keywords 4.2-early removed
  • Milestone changed from Awaiting Review to 4.3

#21 @miqrogroove
2 years ago

  • Keywords changed from needs-patch, wptexturize to needs-patch wptexturize
  • Milestone 4.3 deleted
  • Resolution set to duplicate
  • Status changed from assigned to closed

Duplicate of #15694.

I've updated the Summary on the other ticket to include "<" and similar characters. Based on extensive testing and feedback from several core committers, I am confident that we cannot support raw angle braces inside of shortcodes in future versions and the best solution will be to implement escaping functions for all shortcode input. This includes administrators who have unfiltered_html capability. We are generally in agreement that past versions unfortunately created the impression that shortcodes were magical in the HTML context, when actually they were not and their attributes need to be HTML encoded for bugs-free operation. While this might create some minor inconveniences for those people editing raw HTML, the larger consideration is that this direction will fix several problems for users of shortcode GUI such as image galleries, image captions, and other media.

Note: See TracTickets for help on using tickets.