Right Arrow ">" At Beginning of Line (For Markdown Quote) is Replaced when Switching Editors

[Kelderic]

When writing in Markdown, a right arrow, >, at the beginning of a line is used to create a blockquote. However, when switching editors back and forth the arrow is changed to >.

Steps to reproduce:

1. Add New Page
2. Select Text Editor
3. Type: > This is my block quote
4. Switch to Visual Editor
5. Switch to Text Editor

[johnbillion]

Markdown isn't support in core. Are you using a Markdown plugin?

[Kelderic]

I'm using Jetpack's Markdown module. Even without Markdown though, I sometime use right arrows at the beginning of lines, mostly as an alternative visual for unordered lists.

[iseulde]

You'll need to use the Jetpack support forums for this. You can only use HTML in the editor's text mode.

[Kelderic]

This doesn't really have anything to do with Jetpack. The problem is that the editor is escaping the arrow because it thinks it's HTML.

Use Case: Using a > to begin a list. If I want a list to look like:

> List 1
> List 2
> List 3

I can put that in the text editor. When I switch to visual though, and then back, I suddenly have:

> List 1
> List 2
> List 3

[SergeyBiryukov]

Replying to Kelderic:

I can put that in the text editor. When I switch to visual though, and then back, I suddenly have:

It should still look correctly on the front-end, just tested on a clean install.

[iseulde]

But in visual you still have:

> List 1
> List 2
> List 3

So what's the problem?

[Kelderic]

It's a problem because it's changing what is saved. While the visually it appears the same on the front end, it appears as more code in the text editor which can be unsettling to non coders. Plus anything that is using the arrow specifically as a character breaks. It shouldn't be getting changed in the first place. I know we are just shy of 4.1 release and this is probably a minor issue, but the root of the problem is in the code which fires when changing editors.

I'd be happy to dive in and try writing a patch if someone could point me to which file handles the changes.

[iseulde]

The problem is that the editor is escaping the arrow because it thinks it's HTML.

That's because it *is* HTML. The text editor is a simple HTML editor where you're not required to wrap your paragraphs in <p>'s
Switching between modes will encode the character's you've added. Same happens for & for example.

I'm not sure if it would be better to preserve the user input and encode before outputting on the front-end.

[azaozz]

This doesn't really have anything to do with Jetpack. The problem is that the editor is escaping the arrow because it thinks it's HTML.

In both the editor and the saved post_content the "context" is HTML. Having non-encoded HTML special chars is invalid there.

It's a problem because it's changing what is saved. While the visually it appears the same on the front end, it appears as more code in the text editor which can be unsettling to non coders. Plus anything that is using the arrow specifically as a character breaks. It shouldn't be getting changed in the first place.

When using the Text editor you can save invalid HTML. That editor is targeted at users who prefer to type/edit HTML by hand. On the other hand the Visual editor is for users who don't want to code HTML by hand. It is not acceptable to produce invalid HTML from there.

[Kelderic]

Replying to azaozz:

When using the Text editor you can save invalid HTML. That editor is targeted at users who prefer to type/edit HTML by hand. On the other hand the Visual editor is for users who don't want to code HTML by hand. It is not acceptable to produce invalid HTML from there.

A right arrow at the beginning of a new line, or with a space in front of it, isn't invalid HTML. It's not HTML at all. If we stand by this, we are saying we don't support using arrow characters at all for anything other than HTML.

I am working on a patch to fix this. It adds 2 new lines to editor.js (or one if we only want to support arrows at the beginning of a new line, and not arrows that have spaces in front). I'm in the process of setting up to submit patches but I should have it ready to go this afternoon.

[johnbillion]

Replying to Kelderic:

A right arrow at the beginning of a new line, or with a space in front of it, isn't invalid HTML. It's not HTML at all.

Of course it is. While most browsers handle unencoded angle brackets fine, it's still a reserved character in HTML and needs to be encoded.

[Kelderic]

Patch to Reset Right Arrows To Right Arrow Character

[Kelderic]

I've attached a patch which should fix this. It un-converts right arrows that are at the beginning of a new line and that have spaces in front.

If a right arrow is the first character of a line, it can't be part of an HTML element, and the same could be said of right arrows with spaces in front. With the latter I'm not 100% if that is always true, so I'm attaching a second patch that only apples this on the right arrows that begin a line.

[Kelderic]

Patch to Reset Right Arrows To Right Arrow Character, Only Beginning of New Lie

[nacin]

I believe kses would end up converting this on save anyway, if it runs (it wouldn't for administrators).

[Kelderic]

With the second patch, I've tested as an Administrator. I've tried saving in both editors, and then switching back and forth, and the arrow at beginning of line isn't converted.

Edit: Misunderstood your meaning, Nacin. You are right that kses is a problem that needs to be addressed. I'll try to address it with a new patch. Thanks for the feedback!

[Kelderic]

Screen capture of testing Patch 2

[Kelderic]

Screen capture of testing Patch 2 from a non-administrator. Shows patch 2 doesn't fully work.

[azaozz]

Replying to Kelderic:

If a right arrow is the first character of a line, it can't be part of an HTML element, and the same could be said of right arrows with spaces in front.

​Space characters (including new line U+000A and space U+0020) are allowed before > in start and end HTML tags: ​http://www.w3.org/TR/html-markup/syntax.html#syntax-start-tags and ​http://www.w3.org/TR/html5/syntax.html#syntax-start-tag.

These are all valid HTML:

<p
id="some"
>

<p id="some" >

</p
>

It's true "stray" > are allowed in text content, however they will trip many other functions that don't expect them. If it's any indication, when using contentEditable all browsers encode all > chars.

If you're trying to implement Markdown support in TinyMCE, perhaps have a look at ​https://github.com/tinymce/tinymce/blob/master/js/tinymce/plugins/textpattern/plugin.js which converts some of the Markdown syntax to HTML on the fly.

If you're implementing Markdown from PHP, I'd suggest replacing \n> with \n> before processing.

[Kelderic]

Patch to have kses only encode right arrows that aren't at the beginning of new lines.

[Kelderic]

Patch to have kses only encode right arrows that aren't at the beginning of new lines by tweaking regex and adding multiline flag.

[Kelderic]

Screen capture of testing both patches (ver2) together, from a non admin user. Appears to work completely; preserving right arrows at the beginning of new lines.

[miqrogroove]

Ticket author has reopened this issue twice without a valid reason. The KSES and Editor features do not support plain text editing. They are working as intended.