KSES: Re-implement using HTML API

[dmsnell]

Since its initial commit, WordPress has relied on the KSES library to sanitize and normalize and transform content from untrusted sources. Unfortunately, the parsing inside of wp_kses() has never been spec-compliant, leading to numerous challenges in safely, efficiently, and reliably fulfilling its purpose.

The HTML API provides a mechanism for structurally interacting with HTML in an efficient manner, and wp_kses() should be rewritten to lean on that existing capability.

Resolves

• #25851 Large attribute values may crash PCRE patterns and cause content loss.
• #37698 Global pollution in wp_kses_split() calls.
• #48873 CSS contents are corrupted by wp_kses().
• #51482 wp_kses() turns SCRIPT and STYLE content into renderable text.
• #52333 wp_kses() and HTML disagree on the set of named character references.
• #58377 Block names with consecutive hyphens are corrupted.
• #58921 Valid tag names are rejected from the allow-list.
• #59310 parse_blocks() called unnecessarily.
• #61246 wp_kses() un-comments HTML comments.
• #62024 wp_kses_post() incorrectly escapes “<” in attribute values.

Potentially resolves

• #51093 CSS > corruption.
• #56118 Slashes are improperly stripped when exporting data.

Related

• #63881 Deprecate wp_kses_stripslashes().
• #65599 HTML sanitization API implementation.

Trac ticket: Core-66208
Trac ticket: Core-65984

Replaces #6577

Description

Rewrites wp_kses() to rely on the HTML API for structural and reliable application of sanitization rules, normalizing the output for improved downstream parsing.

Notables

• A new filter wp_kses_force_legacy_parser provides the choice of whether to use this new parser or stick with the legacy code.

Fixes

• Core-61246 wp_kses() un-comments HTML comments.
• Core-62024 wp_kses_post() incorrectly escapes "<" attributes values.

Todo

~Merge after #13273, which accounts for three of the failing tests.~

• [x] self-closing non-HTML elements
• [~] remove opening tag when required attributes are missing, and closing tag
  • while this would be a nice enhancement it’s going to be left out of this work to preserve existing behaviors. with the HTML Processor powering wp_kses(), it’s possible to simply wait until an opened element is closed based on depth, and skip that closing element if it exists.
• [x] replace C0 controls
  • [x] replace C0 controls with their escapes, rather than stripping them away
  • [x] replace C0 controls in attribute values?
  • [ ] original commit _removing C0_ controls is c7fd8c7973
  • [ ] C0 controls are left in-place to prevent problems with creating new syntax through their removal
• [~] handle incomplete parsing, including closing all open elements
  • plenty of existing code in Core calls wp_kses() with intentionally-incomplete input, for example, a wrapper opening tag with part of the content, separately from the closer. closing open elements does a good job of isolating content, but legacy behaviors depend too much on the more procedural use of wp_kses() so isolation cannot be reasonably added without mangling websites.
• [x] if SVG or MATH are not allowed, the entire element should disappear
• [x] remove default pre_kses filters but then call pre_kses

Notes

• Branch tip with HTML Processor ​bdbae3a

[dmsnell]

#61246 was marked as a duplicate.

[dmsnell]

#62024 was marked as a duplicate.

[dmsnell]

#25851 was marked as a duplicate.

[dmsnell]

#52333 was marked as a duplicate.

[dmsnell]

#58377 was marked as a duplicate.

[dmsnell]

#48873 was marked as a duplicate.

[dmsnell]

#51482 was marked as a duplicate.