Improve UTF-8 scanning performance with codepoint limits

[jonsurrell]

_wp_scan_utf8() includes a fast-path for scanning past ASCII bytes. The function also exposes optional byte and codepoint limits.

The ASCII fast-path relies on strspn() with a length limit in case a byte range has been provided. It does not limit the scan length if a codepoint limit has been provided but no byte limit. In this case strspn() will scan through to the end of the string even if a single codepoint remains to be matched which could be satisfied by checking a single byte for ASCII.

The missing codepoint limit on the ASCII fast-path is likely the cause of the issue mentioned in this comment.

See #63863 and [60768].

When _wp_scan_utf8() is invoked with a maximum number of codepoints, the ASCII fast-path will still scan the entire input string. In the case of long ASCII-only strings searching for a limited number of codepoints, this is wasteful and can be optimized by only searching up to remaining number of desired codepoints.

ASCII are all 1-byte UTF-8 codepoints, so the maximum number of ASCII bytes can be limited by the maximum number of remaining codepoints.

See the demonstration below.

​https://github.com/WordPress/wordpress-develop/blob/c710ca6b53db970d1b59b4e34ba130b25458ec1b/src/wp-includes/compat-utf8.php#L56-L69

<details>
<summary>Demonstration</summary>

php \
    -d memory_limit=8G \
    -d opcache.enable_cli=1 \
    -d opcache.enable=1 \
    -d opcache.jit_buffer_size=128M \
    -d opcache.jit=tracing \
test.php

// test.php
<?php
require_once 'src/wp-includes/compat-utf8.php';

                   //"Hello, 🌎!"
$s                 = "Hello, \xF0\x9F\x8C\x8E!" .
                     str_repeat( 'abc123', 512 * 1024 * 1024 ) .
                   //Invalid UTF-8 byte
                     "\xF1";
$at                = 0;
$invalid_length    = 0;
$max_bytes         = null;
$max_codepoints    = null;
$has_noncharacters = false;
$dur               = -hrtime( true );
$scanned           = _wp_scan_utf8( $s, $at, $invalid_length, $max_bytes, $max_codepoints, $has_noncharacters );
$dur              += hrtime( true );

echo <<<"TEXT"
Found {$invalid_length} invalid UTF-8 bytes up to position {$scanned} in {$dur}ms.
Max bytes: {$max_bytes}.
Max codepoints: {$max_codepoints}.
===

TEXT;

$at                = 0;
$invalid_length    = 0;
$max_bytes         = 12;
$max_codepoints    = null;
$has_noncharacters = false;
$dur               = -hrtime( true );
$scanned           = _wp_scan_utf8( $s, $at, $invalid_length, $max_bytes, $max_codepoints, $has_noncharacters );
$dur              += hrtime( true );

echo <<<"TEXT"
Found {$invalid_length} invalid UTF-8 bytes up to position {$scanned} in {$dur}ms.
Max bytes: {$max_bytes}.
Max codepoints: {$max_codepoints}.
===

TEXT;

$at                = 0;
$invalid_length    = 0;
$max_bytes         = null;
$max_codepoints    = 9;
$has_noncharacters = false;
$dur               = -hrtime( true );
$scanned           = _wp_scan_utf8( $s, $at, $invalid_length, $max_bytes, $max_codepoints, $has_noncharacters );
$dur              += hrtime( true );

echo <<<"TEXT"
Found {$invalid_length} invalid UTF-8 bytes up to position {$scanned} in {$dur}ms.
Max bytes: {$max_bytes}.
Max codepoints: {$max_codepoints}.
===

TEXT;

$dur   = -hrtime( true );
$count = _wp_utf8_codepoint_count( $s, 0, 9 );
$span  = _wp_utf8_codepoint_span( $s, 0, 9 );
$dur  += hrtime( true );
$text  = substr( $s, 0, $span );

echo <<<"TEXT"
Counted {$count} codepoints in {$span} bytes in {$dur}ms:
{$text}
TEXT;

Output diff:

• trunk@ c710ca6b53db970d1b59b4e34ba130b25458ec1b
• this branch@ 5829e041927202bcf421c318f5177a01c303a4b5

.txt

@@ -1 @@
-Found 1 invalid UTF-8 bytes up to position 3221225481 in 1453752292ms.
+Found 1 invalid UTF-8 bytes up to position 3221225481 in 1436277917ms.
 Max bytes: .
 Max codepoints: .
 ===
-Found 0 invalid UTF-8 bytes up to position 9 in 4417ms.
+Found 0 invalid UTF-8 bytes up to position 9 in 8208ms.
 Max bytes: 12.
 Max codepoints: .
 ===
-Found 0 invalid UTF-8 bytes up to position 9 in 1005872250ms.
+Found 0 invalid UTF-8 bytes up to position 9 in 833ms.
 Max bytes: .
 Max codepoints: 9.
 ===
-Counted 8 codepoints in 12 bytes in 978311000ms:
+Counted 8 codepoints in 12 bytes in 8084ms:
 Hello, 🌎!

</details>

This seems to address a performance issue inspecting a 10MB XML file. The file is mostly ASCII (>98%), and at each codepoint boundary the rest of the document was scanned to find the ASCII code point count, often reading very large chunks of the document repeatedly. This is fixed by recognizing the max codepoint limit in the ASCII fast path.

<details><summary>performance benchmark results</summary>

### Before (trunk@ c710ca6b53db970d1b59b4e34ba130b25458ec1b )

Timed: 8.93s user 0.02s system 99% cpu 8.982 total

Parsing a 10MB XML file with next_codepoint_wp_scan_utf8...
Starting at: 1781785757.8389
Parsed 100000 codepoints in 0.56971597671509 seconds
Parsed 200000 codepoints in 2.5511300563812 seconds
Parsed 300000 codepoints in 4.1061329841614 seconds
Parsed 400000 codepoints in 4.4027769565582 seconds
Parsed 500000 codepoints in 4.5246770381927 seconds
Parsed 600000 codepoints in 4.7350499629974 seconds
Parsed 700000 codepoints in 5.7863998413086 seconds
Parsed 800000 codepoints in 7.3096778392792 seconds
Parsed 900000 codepoints in 8.7035548686981 seconds
Parsed 1000000 codepoints in 8.9111640453339 seconds

### After (branch@ 5829e041927202bcf421c318f5177a01c303a4b5 )

Timed: 0.19s user 0.01s system 96% cpu 0.214 total

Parsing a 10MB XML file with next_codepoint_wp_scan_utf8...
Starting at: 1781785739.3894
Parsed 100000 codepoints in 0.016802072525024 seconds
Parsed 200000 codepoints in 0.032394170761108 seconds
Parsed 300000 codepoints in 0.047827005386353 seconds
Parsed 400000 codepoints in 0.063305139541626 seconds
Parsed 500000 codepoints in 0.078748941421509 seconds
Parsed 600000 codepoints in 0.094329118728638 seconds
Parsed 700000 codepoints in 0.10992097854614 seconds
Parsed 800000 codepoints in 0.12537217140198 seconds
Parsed 900000 codepoints in 0.14099717140198 seconds
Parsed 1000000 codepoints in 0.15646696090698 seconds

</details>

Follow-up to: r60768
Trac ticket: https://core.trac.wordpress.org/ticket/65483

## Use of AI Tools

AI assistance: Yes
Tool(s): Claude (Fable 5, Opus 4.8, etc.), Codex (GPT 5.5)
Used for: Fuzz testing and discovery, implementation, testing..

[jonsurrell]

In 62523:

Charset: Limit _wp_scan_utf8() ASCII scan to remaining code points.

The ASCII fast-path in _wp_scan_utf8() uses strspn() to skip past ASCII bytes. When a code point limit was provided without a byte limit, the scan would include the rest of the input even when there was a code point limit. Because ASCII characters are single-byte code points, the fast-path scan length can be bounded by the number of remaining code points. This improves performance when working with some large documents.

Developed in ​https://github.com/WordPress/wordpress-develop/pull/12214.

Follow-up to [60768].

Props jonsurrell, dmsnell, zieladam.
Fixes #65483. See #63863.