Changeset 60630 for trunk/tests/phpunit/tests/formatting/seemsUtf8.php

Timestamp:

08/12/2025 06:13:48 PM (9 months ago)

Author:

dmsnell

Message:

Add wp_is_valid_utf8() for normalizing UTF-8 checks.

There are several existing mechanisms in Core to determine if a given string contains valid UTF-8 bytes or not. These are spread out and depend on which extensions are installed on the running system and what is set for blog_charset. The seems_utf8() function is one of these mechanisms.

seems_utf8() does not properly validate UTF-8, unfortunately, and is slow, and the purpose of the function is veiled behind its name and historic legacy.

This patch deprecates seems_utf() and introduces wp_is_valid_utf8(); a new, spec-compliant, efficient, and focused UTF-8 validator. This new validator defers to mb_check_encoding() where present, otherwise validating with a pure-PHP implementation. This makes the spec-compliant validator available on all systems regardless of their runtime environment.

Developed in ​https://github.com/WordPress/wordpress-develop/pull/9317
Discussed in https://core.trac.wordpress.org/ticket/38044

Props dmsnell, jonsurrell, jorbin.
Fixes #38044.

File:

• trunk/tests/phpunit/tests/formatting/seemsUtf8.php (modified) (1 diff)

trunk/tests/phpunit/tests/formatting/seemsUtf8.php

@@ -1 @@
-<?php
-
-/**
- * @group formatting
- *
- * @covers ::seems_utf8
- */
-class Tests_Formatting_SeemsUtf8 extends WP_UnitTestCase {
-
-    /**
-     * `seems_utf8` returns true for utf-8 strings, false otherwise.
-     *
-     * @dataProvider data_seems_utf8_returns_true_for_utf8_strings
-     */
-    public function test_seems_utf8_returns_true_for_utf8_strings( $utf8_string ) {
-        // From http://www.i18nguy.com/unicode-example.html
-        $this->assertTrue( seems_utf8( $utf8_string ) );
-    }
-
-    public function data_seems_utf8_returns_true_for_utf8_strings() {
-        $utf8_strings = file( DIR_TESTDATA . '/formatting/utf-8/utf-8.txt' );
-        foreach ( $utf8_strings as &$string ) {
-            $string = (array) trim( $string );
-        }
-        unset( $string );
-        return $utf8_strings;
-    }
-
-    /**
-     * @dataProvider data_seems_utf8_returns_false_for_non_utf8_strings
-     */
-    public function test_seems_utf8_returns_false_for_non_utf8_strings( $big5_string ) {
-        $this->assertFalse( seems_utf8( $big5_string ) );
-    }
-
-    public function data_seems_utf8_returns_false_for_non_utf8_strings() {
-        // Get data from formatting/big5.txt.
-        $big5_strings = file( DIR_TESTDATA . '/formatting/big5.txt' );
-        foreach ( $big5_strings as &$string ) {
-            $string = (array) trim( $string );
-        }
-        unset( $string );
-        return $big5_strings;
-    }
-}