Opened 5 months ago
Last modified 3 days ago
#65466 reviewing defect (bug)
HTML API: Introduce helper function for parsing CSS class names from a (decoded) attribute value
| Reported by: | westonruter | Owned by: | jonsurrell |
|---|---|---|---|
| Priority: | normal | Milestone: | 7.2 |
| Component: | HTML API | Version: | 6.4 |
| Severity: | normal | Keywords: | has-patch has-unit-tests commit |
| Cc: | Focuses: |
Description
The HTML API has the WP_HTML_Tag_Processor::class_list() method which can be used to tokenize the class names from a class attribute according to the HTML spec. However, to use it, you have to first construct an HTML tag with the class attribute value encoded as HTML. For example (PHP Playground):
<?php /** * Parses class names from a class list string. * * @return Generator<int, non-empty-string> */ function classnames_to_class_list( string $class_names ): Generator { $processor = new WP_HTML_Tag_Processor( sprintf( '<div class="%s"></div>', esc_attr( $class_names ) ) ); $processor->next_tag(); foreach ( $processor->class_list() as $class_name ) { yield $class_name; } } print_r( iterator_to_array( classnames_to_class_list( ' foo bar baz ' ) ) );
Output:
Array
(
[0] => foo
[1] => bar
[2] => baz
)
We can avoid the overhead of encoding and parsing the HTML.
In fact, this was done recently in:
- r62475 for
wp_render_elements_class_name()(gutenberg_render_elements_class_name()) - r62359 for
wp_render_custom_css_class_name()(gutenberg_render_custom_css_class_name())
The two functions are duplicating the same underlying logic for parsing the class list according to the HTML spec, as identified in a PR comment.
There additional cases which could make use of this helper:
wp_render_block_style_variation_class_name()(gutenberg_render_block_style_variation_class_name())wp_get_block_style_variation_name_from_registered_style()(gutenberg_get_block_style_variation_name_from_registered_style())gutenberg_get_block_style_variation_name_from_class()WP_Duotone_Gutenberg::restore_image_outer_container()
Both of these functions have bugs in parsing class names, either by not tokenizing correctly at a token boundary (e.g. by using the \b word boundary in regex) or by incorrectly splitting by spaces, when other whitespace characters may separate tokens.
An initial implementation has been sketched out by @jonsurrell in a PR comment.
Change History (15)
#2
@
5 months ago
There does seem to be utility in a function like this. I do share some concerns that there's potential for confusion and mis-use. The fact that different implementations of this functionality exist in WordPress Core right now (" ", \S, \b separators) suggests there's remove for improvement.
it could even be better that we use
strok( $decoded_class_list, ' ' );
I think I understand where this is coming from. The proposed function deals with a narrow part of HTML (whitespace separated lists) while ignoring some other parts (HTML character reference decoding or some DOMTokenList set behaviors like deduplication).
As long as this function is useful and its scope is clear, it's worth considering. Let's survey the examples already mentioned:
wp_render_elements_class_nameandwp_render_custom_css_class_namewere both updated to use HTML whitespace splitting (on one of" \t\f\r\n") to find matching class names in a stringwp_render_block_style_variation_class_nameuses `\b` and `\S` to match a class name.wp_get_block_style_variation_name_from_registered_styleexplodes on " " to find a matching class name.gutenberg_get_block_style_variation_name_from_classuses `\b` and `\S` to match a class name.WP_Duotone_Gutenberg::restore_image_outer_containeruses the Tag Processor to process block content, but explodes the attribute value on `" "` to work with individual class names. (The HTML API class helpers should be used in this case, it's not a candidate for the proposed method and should be updated).
All of these cases revolve around the className block attribute. On one hand, we should be able to normalize this block attribute to a " " separated list instead of HTML-whitespace separated. On the other hand, it does seem clear that this is a plaintext representation of the HTML attribute value, where the HTML whitespace separators " \t\f\r\n" should work fine. Perhaps both would be good, both _recognizing and respecting_ the HTML separators, but also normalizing to " " when storing or parsing this attribute.
For reference, this is the implementation I shared:
<?php /** * Splits a string containing multiple class names yielding individual values. * * This operates on _plaintext_. DO NOT USE WITH HTML ENCODED VALUES. It will * not perform any decoding. Only use on plaintext inputs! * * Named for classnames for discoverability as most common use case, but suitable * for any HTML whitespace delimited list (DOMTokenList). * * For HTML encoded values, use {@see WP_HTML_Tag_Processor}. * * Splits on HTML ASCII whitespace. This matches the behavior of DOMTokenList * and is suitable for class lists (…add more things that use DOMTokenList). * * @todo add examples to documentation… * * @param string $classnames Plaintext space-delimited list. * @return Generator<string> Individual items. */ function classnames_to_class_list( $classnames ): Generator { if ( ! is_string( $classnames ) ) { // _doing_it_wrong? return; } $delimiters = " \t\n\f\r"; $length = strlen( $classnames ); $offset = 0; while ( $offset < $length ) { $offset += strspn( $classnames, $delimiters, $offset ); if ( $offset >= $length ) { break; } $token_length = strcspn( $classnames, $delimiters, $offset ); yield substr( $classnames, $offset, $token_length ); $offset += $token_length; } }
#3
@
5 months ago
WP_Duotone_Gutenberg::restore_image_outer_containeruses the Tag Processor to process block content, but explodes the attribute value on" "to work with individual class names. (The HTML API class helpers should be used in this case, it's not a candidate for the proposed method and should be updated).
#4
@
5 months ago
the thought struck me while looking through the CSS spec that we could have some luck using the terminology for the semantic values rather than the lexical values. we’re dealing with various input sources and meanings, but this is a task whose goal is mostly to split a set of space-separated tokens
a shift to this layer in the abstraction could redeem the parsing issues without introducing the conflation of “a JSON string value whose space-separated names are CSS class names” and “an HTML class attribute”
that leaves some things feeling awkwardly-named, but we have a lot of awkwardly named things.
#5
@
3 months ago
- Milestone 7.1 → 7.2
Since we are already in the 7.1 Beta phase, let's release the new API in 7.2.
This ticket was mentioned in PR #13917 on WordPress/wordpress-develop by @sabbir345.
5 weeks ago
#6
- Keywords has-patch has-unit-tests added
Introduces wp_split_space_separated_tokens(), a helper that splits a plain-text string on ASCII whitespace (" \t\n\f\r"), as a browser does for a set of space-separated tokens such as a list of class names. It then uses the helper to fix class name parsing in block supports.
Notes
- Following the discussion on the ticket, the helper is a lexical operation on plain text, not HTML. It does not decode character references, deduplicate tokens, or normalize case, and it is named for space-separated tokens rather than class lists to avoid confusion with parsing an HTML
classattribute. For HTML,WP_HTML_Tag_Processor::class_list()remains the right tool. - The helper lives in
functions.phpnext towp_parse_list(), which is not suitable here because it also splits on commas. wp_render_elements_class_name()andwp_render_custom_css_class_name()contained the samestrtok()loop. Both now use the helper, with no change in behavior.- The following functions are fixed:
wp_render_block_style_variation_class_name()used/\bis-style-(\S+?--\d+)\b/. Because\bmatches after a hyphen,custom-is-style-fancy--1andis-style-fancy--1-extraboth causedis-style-fancy--1to be added to the block markup.wp_get_block_style_variation_name_from_class()used\bthe same way, so it matchedis-style-*inside another class name. This is the core counterpart ofgutenberg_get_block_style_variation_name_from_class()listed in the ticket.wp_get_block_style_variation_name_from_registered_style()usedexplode( ' ', … ), so a variation separated by a tab or newline was missed.
- Existing behavior is otherwise kept. For example,
wp_get_block_style_variation_name_from_class()still skips any variation whose name starts withdefault, as the(?!default)lookahead did. WP_Duotone::restore_image_outer_container()is not changed. It was already updated to use the HTML API class helpers in r62627, as noted in comment:3.- This takes a different approach from #12499, which adds a
class_list_from_string()method toWP_HTML_Tag_Processorwithout updating the call sites. - The Gutenberg counterparts (
gutenberg_*) will need the same change.
Testing Instructions
- Run the new and updated tests:
npm run test:php -- --filter 'wpSplitSpaceSeparatedTokens|WpGetBlockStyleVariationNameFromClass|Tests_Block_Supports_Layout|Tests_Block_Supports_BlockStyleVariations|WpRenderElementsSupport|WpRenderCustomCssClassName' - To confirm the regression tests, check out
trunkversions of the changed files insrc/wp-includes/block-supports/and run the tests again. Five cases fail:test_block_style_variation_class_namewith "prefix inside another class name" and "instance number followed by a suffix"test_get_block_style_variation_name_from_registered_stylewith "variation separated by a tab or newline"test_block_style_variation_name_extractionwith "prefix inside another class name" and "variation ending in a non-word character"
- Run the wider groups:
npm run test:php -- --group block-supports,functions,blocks.
Use of AI Tools
AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5.5
Used for: Verifying the ticket against trunk, implementing the helper and call-site changes, writing the tests, and drafting this description.
@algorithmsunlocks commented on PR #13917:
5 weeks ago
#7
Test report
Environment — WordPress 7.2-alpha (trunk 2bf0714531), PHP 8.3, single site, local
wordpress-develop env. PR head 7658328747.
The helper matches the spec
wp_split_space_separated_tokens() splits on exactly the five ASCII whitespace characters and
nothing else:
| Input | Splits? | |
|---|---|---|
| space, TAB, LF, FF, CR | yes | correct |
vertical tab U+000B | no | correct — not ASCII whitespace |
NUL, NBSP U+00A0 | no | correct |
'' and " \t\n " | 0 tokens | correct |
Behaviour changes at the call sites
I ran 16 edge cases through wp_get_block_style_variation_name_from_class() on trunk and on
this branch. Four differ:
| Input | trunk | this PR | |
|---|---|---|---|
foo-is-style-blue | {blue} | {} | correct — \b let the old regex match mid-token; covered by your prefix inside another class name test
|
foo-is-style-blue--12-bar | {blue--12-bar} | {} | same cause |
is-style-blue, | {blue} | {blue,} | correct — the token really is blue,; covered by your variation ending in a non-word character test
|
is-style-a + U+000B + is-style-b | {a, b} | {ais-style-b} | correct per spec, and the only one of the four with no test. PCRE \S treats vertical tab as a separator; the HTML spec's ASCII whitespace does not. Might be worth a data-set row to pin it
|
Automated
--group functions,block-supports,formatting: 3823 tests, 0 failures. The 32 warnings are the pre-existing PHPUnit-10E_STRICTdeprecation notices, present on trunk too.- PHPCS on all five changed source files: 0 errors. The one warning
(
functions.php:1858,DESCRIBE $table) is pre-existing on trunk — I verified it there.
The failing CI job looks unrelated
PHP 8.3 / MariaDB 11.8 with memcached has exactly one failure:
1) Tests_Formatting_ConvertSmilies::test_convert_standard_smilies with data set #1 -'... We got fun n games! 🙂 ...' +'... We got fun n games! :) ...'
There is no call path from that test to wp_split_space_separated_tokens() or to any file this
PR touches, and the test passes locally on this branch inside the formatting group.
A likely mechanism, which I could not reproduce locally — I had no way to enable memcached
in this environment, so treat it as a hypothesis rather than a finding:
convert_smilies() bails when get_option( 'use_smilies' ) is falsy, and
test_convert_standard_smilies() sets use_smilies to 0 inside the test body, relying on
the transaction rollback to put it back. A rollback restores the DB row but not a persistent
object cache, so under memcached data set #1 can read a stale 0 — which is exactly where it
failed, right after data set #0 set it to 0. That would make this an instance of #65893
rather than anything to do with this patch; adding a test file is enough to shift ordering and
expose it.
Suggested next step: re-run just that job. Green means ordering; red again points at
convertSmilies.php, not at this PR.
Not tested
The memcached configuration itself (reason above), multisite, the full suite locally, and the
edge cases of wp_render_block_style_variation_class_name() beyond the two rows in the table —
its lazy-to-greedy change on inputs like is-style-a--1--2 is worth a look.
One note
This is 26 commits behind trunk, so it will want a refresh before commit. And since it adopts
dmsnell's "set of space-separated tokens" framing from comment:4 on the ticket rather than the
earlier class-list naming, a committer nod on the name would probably unblock it — there has
been no review on it yet.
---
AI assistance: Yes (Claude Code), used for the edge-case comparison, running the suites, triaging
the CI failure and drafting this report; reviewed and tested by me.
#8
@
3 weeks ago
I think https://github.com/WordPress/wordpress-develop/pull/13917 could move forward, however I'd make this a generator instead of returning an array of tokens.
@jonsurrell commented on PR #12499:
2 weeks ago
#11
Thanks for this. I'm going to close it in favor of #13917 which is farther along, but we'll make sure this work is credited when something lands.
@jonsurrell commented on PR #13917:
2 weeks ago
#12
@Debarghya-Banerjee should receive props via #12499 when this lands.
@jonsurrell commented on PR #13917:
2 weeks ago
#13
I dismissed my approval. That's not a statement about the work, which is good and deserves to land. I do want to pause and figure out _where it should go_ because that part is less clear to me.
I don't like introducing yet another global function, one that's fairly niche.
It's useful for the HTML API internals for the class_list loop. It _could_ be a static method on that class, it clearly associates it with HTML. However, for many of the places it's used in this PR it seems strange to call a tag processor static method.
I'd like to know what other folks think is the best place for this type of utility, or whether the new function with it's refined name is best.
@dmsnell commented on PR #13917:
12 days ago
#14
I don't like introducing yet another global function, one that's fairly niche.
generally I concur, but I don’t mind in this case because it’s now a specifically-named function with a verifiable behavior. as long as the function does what it says and comprehensively so, it seems fine. we can move it around later and eventually deprecate it, delegating to some newer version if we want.
we could move these into src/wp-includes/html-api/microsyntaxes.php, which would also be a good place to continually add more, such as wp_html_microsyntax_unordered_set_of_unique_space_separated_tokens() and wp_html_microsyntax_comma_separated_tokens() and wp_html_microsyntax_signed_integer() etc.
as lower-level functions in that directory they could be shielded a bit against appearing like the obvious functions to reuse in places they don’t belong. we could also start by making it @access private
@adamsilverstein’s work in #14811 includes a srcset parser, which might also fit here as wp_html_microsyntax_srcset(), though it’s unique enough that maybe that wouldn’t helpful. for these collections and definitive microsyntaxes, I like the idea. that leaves the semantic parsing still unaddressed, but supplies the primitives to iterating over attribute values which are various kinds of collections of values.
@peterwilsoncc commented on PR #13917:
3 days ago
#15
@propsbot add Debarghya-Banerjee
![(please configure the [header_logo] section in trac.ini)](/chrome/site/your_project_logo.png)
this part worries me, particularly because we know from the examples that we are not working with HTML data, but some implicit data type in JSON which is a list of class names.
to that end, we definitely don’t want to decode HTML character references, and we may want to be careful about assuming whether or not to treat case-variations of the same class name as duplicates or distinct.
if the JSON semantic is "space separates class names" then it could even be better that we use
strok( $decoded_class_list, ' ' );(against my own previous advice, I’m sure).and maybe this is roughly another perspective on needing something like a string set, which deduplicates on byte-identical inputs.
it would seem important to me to make sure that the appearance of this function does not lead to confusion between the semantic act of parsing class names from HTML strings and adding class names from some custom implicit and undocumented but already decoded setting.