Curly quotes injected into short code attributes when special characters present

[msimpson]

WP 4.0, Theme: Twenty Fourteen 1.2

Given this short code, that simply displays the short code attributes passed in:

function showattributes($atts, $content = null) {
    echo '<code>';
    foreach ($atts as $name => $value) {
        echo "$name=$value\n";
    }
    echo '</code>';
}
add_shortcode('quotebug', 'showattributes');

(1) Case that works as expected

1. Create a WP Post, Editor in Text mode (not Visual)
2. Put in this short code and preview

[quotebug a="x_0" b="x_0" c="hello"]

3. Yields expected output

a=x_0 b=x_0 c=hello

(2) Bug: Case that injects curly quotes (not expected)

Same as previous scenario, but we will have special characters like ampersand and greater-than in the short code attributes

1. Create a WP Post, Editor in Text mode (not Visual)
2. Put in this short code and preview

[quotebug a="x>0" b="x&0" d="hello"]

3. Yields output where all the values have curly-quotes injected in them. This confounds the short code which is not expecting quotes in the values it gets passed in.

a=”x>0″ b=”x&0″ d=”hello”

(3) Switching to Visual mode changes the behavior

1. Given the previous example (2), simply switch the editor to Visual mode
2. Preview the post. It yields the expected output (no quotes) like the first example (1).
3. Toggle back to Text mode. You see that the special characters have been changed to their html-entity codes.
4. Previewing again yields the expected (no quotes) output (1).

Impact

As a plugin author with short codes, a number of my users are complaining that their short codes no longer work. Presumably they entered the short code in Text edit mode. They don't want to find all the short codes that they (or their clients) have used across various posts and web sites to toggle to Visual and re-save. I'm forced to put code in my plugin's short codes to try to detect and strip out curly quotes. However if the quotes were intended by the user, then my code will incorrectly strip them out. It creates an ambiguity.

[msimpson]

(bad formatting in ticket) Case 2 short code is:

[quotebug a="x>0" b="x&0" d="hello"]

[msimpson]

Short code attributes incorrectly parsed when space present

Additional effect of the bug. Given this short code entered in Text mode in the editor

[quotebug filter="submit_time>-6 days"]

It parses it incorrectly, breaking at the space inside of the quoted values, producing an extra "0" attribute with value "days".

filter=”submit_time>-6 
0=days”

Also notice that "filter" has the injected begin curly quote and "0" has the injected end curly quote.

whereas if I toggled to Visual mode and preview again, I get the expected:

filter=submit_time>-6 days

[miqrogroove]

We do have a patch for this at #29557. The ">" character, and HTML elements in general have not been described as allowed by the Shortcode API documentation, but we are able to accommodate that for version 4.0.1. And perhaps the documentation should be updated to state this one way or the other.

[miqrogroove]

Toggle back to Text mode. You see that the special characters have been changed to their html-entity codes.

I'm fairly certain this is how it always works. Did you see a change from a previous version? And if so, what is the exact difference?

[miqrogroove]

Added ticket #29661 to discuss the documentation and unit issues.

[msimpson]

Replying to miqrogroove:

Toggle back to Text mode. You see that the special characters have been changed to their html-entity codes.

I'm fairly certain this is how it always works. Did you see a change from a previous version? And if so, what is the exact difference?

HTML entity codes is not the issue. The issue is the injection of quotes into the attribute string value and incorrect parsing of the attributes in the case I demonstrated my additional comment.

It has always been true (past few years AFAIK) that if you put special characters in a short code attribute then you get the exact characters if you entered the short code in the editor's Text mode but HTML-entity versions of them if you entered the short code in Visual mode. As a plugin writer, I have accounted for that years ago.

What's different now, is if you put special characters in a short code attribute in Text mode, WP now inject quotes into the attribute strings. And, as in the case I describe above, WP can process the attributes wrong (case with the space above).

Within the past week or so, some users of my short code complained the short codes no longer work. Given that, my best guess it that the quoting bug was introduced in WP 4.0 and users started seeing the issue once they upgrade.

[miqrogroove]

Replying to msimpson:

Within the past week or so, some users of my short code complained the short codes no longer work. Given that, my best guess it that the quoting bug was introduced in WP 4.0 and users started seeing the issue once they upgrade.

The quoting change was intentional to prevent corruption of HTML in some cases. There were no tests for special characters appearing in shortcode attributes, so we will have to look into testing that from now on.

[DrewAPicture]

Duplicate of #29557.

[sareiodata]

I don't see how this is a duplicate of #29557

Tested it with WordPress 4.0.1 and having a shortcode like: [quotebug val1="test" val2="test" operator=">"] the attributes return as:

array (size=3)

'val1' => string '”test”' (length=18)
'val2' => string '”test”' (length=18)
'operator' => string '”>”' (length=15)

Basically it's adding asci encoded quotes in the attributes.

[miqrogroove]

Duplicate of #29608.

Reference ​http://codex.wordpress.org/Shortcode_API#HTML