plugin editor content empty when source contains an invalid character

[bobbingwide]

I happened to create a plugin source file which contained a pound sterling character (£) copied and pasted from a web page, and which therefore appeared in my Windows text editor as lower case u acute ( hex A3, ascii 163 ).

esc_textarea() makes a call to htmlspecialchars() which returns a null value for safe text.

$safe_text = htmlspecialchars( $text, ENT_QUOTES, get_option( 'blog_charset' ) );

Note: blog_charset is UTF-8

So the plugin editor displayed nothing at all for the source.

Question: Is this really the expected behaviour?

The documentation for htmlspecialchars says

If the input string contains an invalid code unit sequence within the given encoding an empty string will be returned, unless either the ENT_IGNORE or ENT_SUBSTITUTE flags are set.

Shouldn't the plugin editor pass ENT_IGNORE OR otherwise issue a message to the user at least advising not to save the empty file when the safe content is nothing like the original.

[SergeyBiryukov]

Related: #20368

[bobbingwide]

Sample plugin file demonstrating the problem

[bobbingwide]

A simple fix to this problem is to remove the call to esc_textarea() in plugin-editor.php
The invalid character then shows as U+FFFD - question mark in black diamond.

[Ov3rfly]

Same problem in theme editor, wp-admin/theme-editor.php line 121, WP 4.0

+Component: Themes

[DrewAPicture]

@pento @dd32 Any suggestions on how we could mitigate this issue without removing the esc_textarea() call (for obvious reasons)?

[jipmoors]

The problem lies in the implicit character coding assumption.
The esc_textarea assumes the supplied content has been written inside the CMS and thus conforms to the charset selected.

A quick test with a duplicate of the function without the implicit encoding shows the file as it is saved on disk.

[jipmoors]

No implicit encoding for htmlentities in plugin source code editor