Changeset 58281 for trunk/src/wp-includes/class-wp-token-map.php

Timestamp:

06/02/2024 03:14:35 PM (2 years ago)

Author:

dmsnell

Message:

HTML API: Add custom text decoder.

Provides a custom decoder for strings coming from HTML attributes and
markup. This custom decoder is necessary because of deficiencies in
PHP's html_entity_decode() function:

• It isn't aware of 720 of the possible named character references in HTML, leaving many out that should be translated.

• It isn't aware of the ambiguous ampersand rule, which allows conversion of character references in certain contexts when they are missing their closing ;.

• It doesn't draw a distinction for the ambiguous ampersand rule when decoding attribute values instead of markup values.

• Use of html_entity_decode() requires manually passing non-default paramter values to ensure it decodes properly.

This decoder also provides some conveniences, such as making a
single-pass and interruptable decode operation possible. This will
provide a number of opportunities to optimize detection and decoding
of things like value prefixes, and whether a value contains a given
substring.

Developed in ​https://github.com/WordPress/wordpress-develop/pull/6387
Discussed in https://core.trac.wordpress.org/ticket/61072

Props dmsnell, gziolo, jonsurrell, jorbin, westonruter, zieladam.
Fixes #61072.

File:

• trunk/src/wp-includes/class-wp-token-map.php (modified) (7 diffs)

trunk/src/wp-includes/class-wp-token-map.php

@@ -436 +436 @@
          * @since 6.6.0
          *
-         * @param string  $word             Determine if this word is a lookup key in the map.
-         * @param ?string $case_sensitivity 'ascii-case-insensitive' to ignore ASCII case or default of 'case-sensitive'.
+         * @param string $word             Determine if this word is a lookup key in the map.
+         * @param string $case_sensitivity Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching. Default 'case-sensitive'.
          * @return bool Whether there's an entry for the given word in the map.
          */
@@ -522 +522 @@
          *
          * @param string  $text                       String in which to search for a lookup key.
-         * @param ?int    $offset                     How many bytes into the string where the lookup key ought to start.
-         * @param ?int    &$matched_token_byte_length Holds byte-length of found token matched, otherwise not set.
-         * @param ?string $case_sensitivity           'ascii-case-insensitive' to ignore ASCII case or default of 'case-sensitive'.
-         * @return string|false Mapped value of lookup key if found, otherwise `false`.
+         * @param int     $offset                     Optional. How many bytes into the string where the lookup key ought to start. Default 0.
+         * @param ?int    &$matched_token_byte_length Optional. Holds byte-length of found token matched, otherwise not set. Default null.
+         * @param string  $case_sensitivity           Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching. Default 'case-sensitive'.
+         * @return string|null Mapped value of lookup key if found, otherwise `null`.
          */
         public function read_token( $text, $offset = 0, &$matched_token_byte_length = null, $case_sensitivity = 'case-sensitive' ) {
@@ -540 +540 @@
                                 return strlen( $this->small_words ) > 0
                                         ? $this->read_small_token( $text, $offset, $matched_token_byte_length, $case_sensitivity )
-                                        : false;
+                                        : null;
                         }
 
@@ -565 +565 @@
                 return strlen( $this->small_words ) > 0
                         ? $this->read_small_token( $text, $offset, $matched_token_byte_length, $case_sensitivity )
-                        : false;
+                        : null;
         }
 
@@ -573 +573 @@
          * @since 6.6.0.
          *
-         * @param string  $text                       String in which to search for a lookup key.
-         * @param ?int    $offset                     How many bytes into the string where the lookup key ought to start.
-         * @param ?int    &$matched_token_byte_length Holds byte-length of found lookup key if matched, otherwise not set.
-         * @param ?string $case_sensitivity           'ascii-case-insensitive' to ignore ASCII case or default of 'case-sensitive'.
-         * @return string|false Mapped value of lookup key if found, otherwise `false`.
+         * @param string $text                       String in which to search for a lookup key.
+         * @param int    $offset                     Optional. How many bytes into the string where the lookup key ought to start. Default 0.
+         * @param ?int   &$matched_token_byte_length Optional. Holds byte-length of found lookup key if matched, otherwise not set. Default null.
+         * @param string $case_sensitivity           Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching. Default 'case-sensitive'.
+         * @return string|null Mapped value of lookup key if found, otherwise `null`.
          */
         private function read_small_token( $text, $offset, &$matched_token_byte_length, $case_sensitivity = 'case-sensitive' ) {
@@ -617 +617 @@
                 }
 
-                return false;
+                return null;
         }
 
@@ -693 +693 @@
          * @since 6.6.0
          *
-         * @param ?string $indent Use this string for indentation, or rely on the default horizontal tab character.
+         * @param string $indent Optional. Use this string for indentation, or rely on the default horizontal tab character. Default "\t".
          * @return string Value which can be pasted into a PHP source file for quick loading of table.
          */