WordPress' TinyMCE automatically remove Invisible Space (\u200B)

[so_vichet]

Invisible Space is being used in some language (Khmer Unicode,...). TinyMCE 3.4.5 (which is included in WordPress 3.3.2) had no problem with invisible space, but TinyMCE 3.4.9 which is included in WordPress 3.4 has a problem with invisible space. That version of TinyMCE will remove invisible space automatic when click publish.

[so_vichet]

Replace in /wp-includes/js/tinymce , then clear cache in browser.

[nacin]

Can you provide an actual patch, rather than the entire file? I can't tell what changed.

It may be necessary to make a patch against TinyMCE itself: ​http://github.com/tinymce/tinymce

[azaozz]

TinyMCE uses "zero-width non-breaking space" characters to place the caret in DOM nodes and some other places. In version 3.4.5 (WP 3.3) only \uFEFF was used, however in version 3.4.9 (WP 3.4) \u200B is used instead of \uFEFF in Gecko. Unfortunately both are stripped from the content when the DOM is serialized.

This code exists in TinyMCE latest too: ​https://github.com/tinymce/tinymce/blob/master/jscripts/tiny_mce/classes/dom/Serializer.js#L346

The only thing we could do is to pass this upstream.

[nacin]

Hmm. This is unfortunate. We should see what we can do to help TinyMCE on this one. Reading some of the history there, it appears that they have no idea what causes these markers to even show up. \uFEEF is a BOM but \u200B is not. They should revert rather than break valid uses of that character.

[azaozz]

As far as I understand, the problem here is not that \u200B is used as "placeholder", but that this char is stripped from the content when TinyMCE does cleanup.

When used as placeholder \u200B seems to be at the beginning or the end of the text string (or the only char there). So maybe the stripping can be changed to:

args.content = args.content.replace(/^[\uFEFF\u200B]+/, '').replace(/[\uFEFF\u200B]+$/, ''); 

To test this: make sure SCRIPT_DEBUG is set to true, download the TinyMCE 3.4.9 development package from ​http://www.tinymce.com/download/download.php, then rename tiny_mce_dev.js to tiny_mce.js and replace it in wp-includes/js/tinymce.

Edit the non-minified tiny_mce.js, change line 8621 to the above example.

Alternatively line 16492 can be changed to:

invisibleChar = INVISIBLE_CHAR;

Then take out stripping of \u200B completely on line 8621.

This will stop TinyMCE using \u200B as placeholder in Firefox but won't stop some browsers inserting it on paste, etc. so there would probably be some \u200B left in the content without being typed by the user.

[azaozz]

Oops, wrong ticket.

[nacin]

Replying to azaozz:

If we are going to continue supporting "bad" plugins, wouldn't it be better to abstract this

did_action( 'init' ) && $scripts->localize( 'handle', 'handleL10n', array(...) );

into another method? So add localise_on_init() or similar to WP_Scripts and call that from script-loader.

I imagine that was meant for #20971. To answer: Maybe. I was mainly concerned with keeping the diff small and logic simple for 3.4.1.

[azaozz]

Right, sorry, you replied before I could correct it :)

Yes, makes sense for 3.4.1. Perhaps can revisit later for 3.5.

[bi0xid]

Just FYI quick access from keyboard in actual TinyMCE is not working properly in Chrome 20.0.1132.47
When you use CMD+i to set italic, it deletes the emphasis tag when you deactivate it.

[psophy]

Hi azaozz,

I try a few days but I can't fix it. This problem effect to Khmer blog more than 200 blogs that use WordPress.com Service and hundred of blogs use WordPress CMS.

I hope this problem will sold next version of WP.

[azaozz]

Replying to psophy:

I try a few days but I can't fix it.

Could you download the tiny_mce.zip, unzip it and replace it in WP. Then test if that fixes it (it includes the above changes).

Also it would be helpful if you can attach an example text file here (UTF-8) to test with.

[azaozz]

Should be fixed for trunk in [21228]. Please reopen if the problem still exists.

[psophy]

Replying to azaozz:

Should be fixed for trunk in [21228]. Please reopen if the problem still exists.

Hi Azaozz on WordPress.com it work now but I not yet test on WordPress.org. I am waiting release version 3.4.2

Thanks

[psophy]

Replying to nacin:

On WordPress.com works correctly but on Version 3.4.2 not yet work it still remove zero space. I hope on version 3.5 it will work.

[markoheijnen]

Replying to psophy:

Replying to nacin:

On WordPress.com works correctly but on Version 3.4.2 not yet work it still remove zero space. I hope on version 3.5 it will work.

You shouldn't test on 3.4.2 but on trunk.

[dd32]

Re-closing as fixed, If the problem still exists in trunk (3.5) then please re-open.

[psophy]

Replying to dd32:

Re-closing as fixed, If the problem still exists in trunk (3.5) then please re-open.

Thank you all!!! It work correctly on trunk(3.5).

Cheer Khmer Language lives again on the WordPress!

[nacin]

Unfortunately this missed the boat for 3.4.2. It would be nice if we could monkey-patch TinyMCE to make this work in the 3.4 branch.

[psophy]

Replying to dd32:

Re-closing as fixed, If the problem still exists in trunk (3.5) then please re-open.

Could you upload your patch for 3.4 branch?

[azaozz]

Replying to psophy:

Could you upload your patch for 3.4 branch?

The whole TinyMCE has to be updated to 3.5.6 (this is currently in trunk).

[nacin]

I manually applied ​https://github.com/tinymce/tinymce/commit/1805a661ebf4836e57456df3c5e3eaa685ed9586 and things appear to work as expected.

[nacin]

In 22372:

Patch TinyMCE to not strip the zero-width u200B space, which the Khmer language uses. for the 3.4 branch. fixes #21025.

[nacin]

As [22372] has the potential to introduce some issues in Firefox (Gecko) without additional changes in TinyMCE, I've reverted it in [22374]. This can be considered as fixed against 3.5.