Changeset 60949 for trunk/tests/phpunit/tests/compat/mbStrlen.php

Timestamp:

10/16/2025 08:58:26 PM (12 months ago)

Author:

dmsnell

Message:

Charset: Rely on new UTF-8 pipeline for mb_strlen() fallback.

The existing polyfill for mb_strlen() contains a number of issues leaving plenty of opportunity for improvement. Specifically, the following are all deficiencies: it relies on Unicode PCRE support, assumes input strings are valid UTF-8, splits input strings into an array of character to count them (1,000 at a time, iterating until complete), and entirely gives up when the Unicode support is missing.

This patch provides an updated polyfill which will reliably count code points in a UTF-8 string, even in the presence of sequences of invalid bytes. It scans through the input with zero allocations. Additionally, the underlying fallback extends the behavior of mb_strlen() to provide character counts for substrings within a larger input without extracting the substring (it can counts characters within a byte offset and length of a larger string).

This change improves the reliability of UTF-8 string length calculations and removes behavioral variability based on the runtime system.

Developed in ​https://github.com/WordPress/wordpress-develop/pull/9828
Discussed in https://core.trac.wordpress.org/ticket/63863

See #63863.

File:

• trunk/tests/phpunit/tests/compat/mbStrlen.php (modified) (2 diffs)

trunk/tests/phpunit/tests/compat/mbStrlen.php

@@ -11 +11 @@
 
         /**
-         * Test that mb_strlen() is always available (either from PHP or WP).
+         * Test that the native mb_strlen() is available.
          */
         public function test_mb_strlen_availability() {
-                $this->assertTrue( function_exists( 'mb_strlen' ) );
+                $this->assertTrue(
+                        in_array( 'mb_strlen', get_defined_functions()['internal'], true ),
+                        'Test runner should have `mbstring` extension active but doesn’t.'
+                );
         }
 
         /**
-         * @dataProvider data_utf8_string_lengths
+         * @dataProvider data_utf8_strings
          */
-        public function test_mb_strlen( $input_string, $expected_character_length ) {
-                $this->assertSame( $expected_character_length, _mb_strlen( $input_string, 'UTF-8' ) );
+        public function test_mb_strlen( $input_string ) {
+                $this->assertSame(
+                        mb_strlen( $input_string, 'UTF-8' ),
+                        _mb_strlen( $input_string, 'UTF-8' )
+                );
         }
 
         /**
-         * @dataProvider data_utf8_string_lengths
+         * @dataProvider data_utf8_strings
          */
-        public function test_mb_strlen_via_regex( $input_string, $expected_character_length ) {
-                _wp_can_use_pcre_u( false );
-                $this->assertSame( $expected_character_length, _mb_strlen( $input_string, 'UTF-8' ) );
-                _wp_can_use_pcre_u( 'reset' );
+        public function test_mb_strlen_via_regex( $input_string ) {
+                $this->assertSame(
+                        mb_strlen( $input_string, 'UTF-8' ),
+                        _mb_strlen( $input_string, 'UTF-8' )
+                );
         }
 
         /**
-         * @dataProvider data_utf8_string_lengths
+         * @dataProvider data_utf8_strings
          */
-        public function test_8bit_mb_strlen( $input_string, $expected_character_length, $expected_byte_length ) {
-                $this->assertSame( $expected_byte_length, _mb_strlen( $input_string, '8bit' ) );
+        public function test_8bit_mb_strlen( $input_string ) {
+                $this->assertSame(
+                        mb_strlen( $input_string, '8bit' ),
+                        _mb_strlen( $input_string, '8bit' )
+                );
         }
 
@@ -45 +55 @@
          * @return array
          */
-        public function data_utf8_string_lengths() {
+        public function data_utf8_strings() {
                 return array(
-                        array(
-                                'input_string'              => 'баба',
-                                'expected_character_length' => 4,
-                                'expected_byte_length'      => 8,
-                        ),
-                        array(
-                                'input_string'              => 'баб',
-                                'expected_character_length' => 3,
-                                'expected_byte_length'      => 6,
-                        ),
-                        array(
-                                'input_string'              => 'I am your б',
-                                'expected_character_length' => 11,
-                                'expected_byte_length'      => 12,
-                        ),
-                        array(
-                                'input_string'              => '1111111111',
-                                'expected_character_length' => 10,
-                                'expected_byte_length'      => 10,
-                        ),
-                        array(
-                                'input_string'              => '²²²²²²²²²²',
-                                'expected_character_length' => 10,
-                                'expected_byte_length'      => 20,
-                        ),
-                        array(
-                                'input_string'              => '３３３３３３３３３３',
-                                'expected_character_length' => 10,
-                                'expected_byte_length'      => 30,
-                        ),
-                        array(
-                                'input_string'              => '𝟜𝟜𝟜𝟜𝟜𝟜𝟜𝟜𝟜𝟜',
-                                'expected_character_length' => 10,
-                                'expected_byte_length'      => 40,
-                        ),
-                        array(
-                                'input_string'              => '1²３𝟜1²３𝟜1²３𝟜',
-                                'expected_character_length' => 12,
-                                'expected_byte_length'      => 30,
-                        ),
+                        array( 'баба' ),
+                        array( 'баб' ),
+                        array( 'I am your б' ),
+                        array( '1111111111' ),
+                        array( '²²²²²²²²²²' ),
+                        array( '３３３３３３３３３３' ),
+                        array( '𝟜𝟜𝟜𝟜𝟜𝟜𝟜𝟜𝟜𝟜' ),
+                        array( '1²３𝟜1²３𝟜1²３𝟜' ),
                 );
         }