Make WordPress Core

Opened 6 weeks ago

Last modified 8 days ago

#65757 new defect (bug)

Proposal: Make the Interactivity API extensible

Reported by: nickchomey Owned by:
Priority: normal Milestone: Future Release
Component: Interactivity API Version:
Severity: normal Keywords: has-patch needs-testing 2nd-opinion
Cc: Focuses:

Description (last modified by nickchomey)

Make the Interactivity API extensible

Component: Interactivity API
Keywords: has-patch needs-testing

Summary

WP_Interactivity_API is final and all of its methods are private, so plugins cannot extend or customize server-side directive processing without duplicating the entire class tree (~2,400 lines). A few small visibility changes (removing final, changing private to protected) would enable subclassing, following the pattern WP_Image_Editor already uses.

Motivation

The Interactivity API is WordPress's recommended approach for frontend interactivity, and for good reason. But its server-side processing class is locked down:

  • final class, so it cannot be extended.
  • All directive processor methods are private. Even the evaluation engine (evaluate()) and the processor registry ($directive_processors) are inaccessible from subclasses.

Anyone who needs custom server-side behavior, such as full-expression evaluation in directives, custom directive processors, or integration with third-party templating, has to either patch core directly or maintain a standalone re-implementation that silently diverges from upstream.

A tangible example is with regards to my effort to add support for writing full JS expressions within inline directives https://github.com/WordPress/gutenberg/issues/79765. It was ultimately rejected, but the appropriate solution going forward seems to me to be to make it a plugin. However, this would be cumbersome with the currently-locked class.

Likewise, if we create new custom directives, a companion server-side processor cannot currently be implemented.

Proposed Changes

Following the pattern used by WP_Image_Editor and WP_Image_Editor_Imagick, visibility should be organized into three tiers:

Tier Visibility Purpose Example
API contract public Methods any caller may invoke load(), save(), process_directives()
Extension contract protected Override points for subclasses thumbnail_image(), evaluate()
Internal private Implementation detail, no extension value write_image(), merge_style_property()

1. Remove final from the class declaration

-final class WP_Interactivity_API {
+class WP_Interactivity_API {

This is the sole blocker for subclassing. The final keyword was presumably added to discourage abuse, or to let the feature mature before anyone relied on extending it. Either way, legitimate extensions must fork the entire class.

2. Reclassify methods into appropriate visibility tiers

Methods that should remain public (the API surface, already correct):

  • state(), config(), get_context(), get_element(): data access
  • process_directives(): main entry point
  • add_hooks(): registration
  • filter_script_module_interactivity_data(), filter_script_module_interactivity_router_data(): hook callbacks
  • add_load_on_client_navigation_attribute_to_script_modules(), add_client_navigation_support_to_script_module(): script module support
  • print_router_markup(): output
  • Deprecated methods kept as-is for back-compat

Methods that could become protected (extension points):

Method Rationale
evaluate() Primary extension point: override to support full expressions, alternate path resolution, or custom state lookups. All directive processors automatically use the overridden version.
_process_directives() The tree walk itself: override to alter how tags are traversed while keeping the HTML processor.
get_directive_entries() Override to filter or modify which directives apply to an element.
extract_directive_value() Override to support custom namespace/value parsing.
parse_directive_name() Override to support custom directive naming conventions.
data_wp_interactive_processor() through data_wp_each_processor() (8 methods) Override to replace a single directive processor without touching the registry.

Methods that should remain private (internal helpers, no extension value):

  • merge_style_property(): pure string manipulation
  • get_router_animation_styles(): CSS string literal
  • kebab_to_camel_case(): utility

3. Reclassify properties into appropriate visibility tiers

Properties that could become protected:

Property Rationale
$state_data An overridden evaluate() must read state per namespace.
$namespace_stack An overridden evaluate() must resolve the active namespace.
$context_stack An overridden evaluate() must resolve context values.
$derived_state_closures An overridden evaluate() that calls derived-state getters must track accessed paths for client hydration.
$directive_processors A subclass must register its own processor methods or the overridden ones won't be called.

Properties that should remain private:

  • $config_data, set only externally via the public config() method
  • $has_processed_router_region, an internal flag with no extension value
  • $script_modules_that_can_load_on_client_navigation, registered via the existing public add_client_navigation_support_to_script_module()
  • $current_element, set and cleared during processing and read via the existing public get_element()

4. Filter for registering custom directive processors

In addition to the visibility changes above, a filter would allow any plugin to register custom server-side directive processors without subclassing:

self::$directive_processors = apply_filters(
    'wp_interactivity_directive_processors',
    self::$directive_processors
);

This covers the common case of adding new directives (e.g. data-wp-on, data-wp-show), while the protected visibility covers replacing existing behavior. It mirrors how WP_Image_Editor uses the wp_image_editors filter to register new editor implementations.

Backward compatibility

The changes are strictly additive. Removing final and widening private to protected only unlocks new capabilities; it does not affect existing code.

Use cases enabled

  1. Custom directive value evaluation. Override evaluate() to support full JS expressions, template syntax, or alternate namespace resolution. All directive processors (bind, class, style, text, each, context) automatically use the overridden method.
  1. Custom directive processors. With a protected method override or the proposed filter, plugins can add server-side handling for custom directives or override the existing handlers.

Implementation

I'll submit a pull request on GitHub for discussion.

Change History (26)

#1 follow-up: @jorbin
6 weeks ago

Hi @nickchomey can you add a human written proposal with real use cases and not a bunch of AI slop, please.

This ticket was mentioned in PR #12758 on WordPress/wordpress-develop by @nickchomey.


6 weeks ago
#2

Trac ticket: 65757

Remove final from WP_Interactivity_API and change method/property visibilities from private to protected where appropriate, enabling plugins to extend or customize server-side directive processing without forking the entire class.

Changes:

  • Remove final from class declaration
  • Change evaluate(), _process_directives(), get_directive_entries(), extract_directive_value(), parse_directive_name() to protected
  • Change all 8 data_wp_*_processor() methods to protected
  • Change $state_data, $namespace_stack, $context_stack, $derived_state_closures, $directive_processors to protected
  • Add wp_interactivity_directive_processors filter

## Use of AI Tools

#3 in reply to: ↑ 1 @nickchomey
6 weeks ago

Replying to jorbin:

Hi @nickchomey can you add a human written proposal with real use cases and not a bunch of AI slop, please.

I directed the AI on what to write and reviewed it. Beyond em dashes (which are a perfectly valid and useful grammatical construct), I don't see anything that is obviously "AI slop". The use cases seem quite valid to me (though I just removed two of them which are less likely). Would you mind pointing out what you are taking exception to? Would you like for it to be just less verbose/well(overly?)-formatted?

You might also find it useful to read the link I provided in the Motivation section (https://github.com/WordPress/gutenberg/issues/79765) where @luisherranz and I discussed the original proposal, which is what this idea grew out of.

Last edited 6 weeks ago by nickchomey (previous) (diff)

#4 @nickchomey
6 weeks ago

  • Description modified (diff)

#5 @nickchomey
6 weeks ago

  • Description modified (diff)

@nickchomey commented on PR #12758:


6 weeks ago
#6

I suspect that you'll want me to separate the directive filter stuff and associated tests to a separate PR, to keep this tightly scoped to removing final and changing private to protected. Let me know if that's the case and I'll be happy to do so.

#7 @nickchomey
6 weeks ago

  • Description modified (diff)

#8 @nickchomey
4 weeks ago

  • Description modified (diff)

#9 @nickchomey
3 weeks ago

  • Description modified (diff)

This ticket was mentioned in Slack in #core-interactivity-api by nickchomey. View the logs.


3 weeks ago

#11 @luisherranz
3 weeks ago

  • Milestone Awaiting ReviewFuture Release

Hi @nickchomey, thanks for taking the time to write this up, and for the PR.

After careful consideration, I think we should postpone this decision. The internals of this class are temporary. The server-side processing runs on WP_Interactivity_API_Directives_Processor, a subclass of the Tag Processor that exists only because WP_HTML_Processor wasn't complete when the Interactivity API merged in 6.5. That's also the reason behind the current limitations, like bailing on unbalanced tags, SVG and MathML. The plan has always been to migrate the implementation to the finalized WP_HTML_Processor once it exposes the required functionality and that migration will change these methods, their signatures, the properties and how the tree traversal works.

If we change these members to protected now, their current shape becomes a public contract under WordPress' backward compatibility policy, and we would have to preserve it through that migration. Basically, we would be freezing the temporary implementation as the permanent API.

So in my opinion the right moment to evaluate which extension points make sense is after the class runs on the final parser, not before. Once that happens, I'll be happy to revisit this with you.

I'm moving the milestone to Future Release to reflect that. Thanks again for pushing on this.

This ticket was mentioned in Slack in #core-interactivity-api by luisherranz. View the logs.


3 weeks ago

This ticket was mentioned in PR #13178 on WordPress/wordpress-develop by @nickchomey.


3 weeks ago
#13

Trac ticket: 65757

Remove final from WP_Interactivity_API to allow for minimal extensibility of the server-side iAPI processor

## Use of AI Tools

No

#14 @nickchomey
3 weeks ago

Thanks for the reply. I had assumed the all-private implementation was primarily because it was still in flux, but I wasn't aware that the server-side processor was expected to be replaced so substantially. As such, it certainly makes sense for those internals to remain private.

As it turns out, I have found a (hacky but acceptable) workaround for my goals, which only requires removing final from WP_Interactivity_API.

It seems to me that this avoids the concern about freezing the current implementation as an API, while still providing a useful extension point.

Would that small change be something that could be made for 7.2? I just made a PR for it.

@nickchomey commented on PR #12758:


3 weeks ago
#15

Closing in favour of #13178 13178

#16 @luisherranz
2 weeks ago

Thanks for iterating on this.

The problem is that removing final only has an effect through one mechanism: a subclass never runs unless its instance replaces the one that wp_interactivity() holds, so the workaround has to be some variation of

<?php
global $wp_interactivity;
$wp_interactivity = new My_Subclass();

executed before core's first call, so that the instanceof check keeps your instance.

Once plugins in the wild rely on that, it becomes a public contract even though we never designed one. We'd have to preserve the instantiation strategy (a single global instance created lazily in wp_interactivity()), how and when core invokes the public methods, because overrides depend on intercepting those exact calls, and even behavioral details like what process_directives() currently bails on.

All of that is precisely what the migration to WP_HTML_Processor needs to change, so in practice we'd be blocking it. If instance replacement proves to be a good extension point, I think it deserves a designed mechanism, like a filter on instantiation, and the right moment to design that is still after the migration.

Happy to look at your concrete use case, though. There may be a supported way to do it today.

#17 @nickchomey
2 weeks ago

Yes, the solution I found was precisely what you presented above - replace global $wp_interactivity and its hooks very early. It is simple to do.

My subclass is almost exactly the same as the parent, and I intend to keep it that way by repatching on each new version of Core. It is really just a small patch that modifies/extends the evaluate() method to support the server-side processing of the inline JS expressions mechanism that I worked on in https://github.com/WordPress/wordpress-develop/pull/12383.

If things change in the official parent class, I'll adapt my patch, hooks, etc. as needed. To the extent that I'll need to do other hacky things to support multiple versions of Core (eg. separate subclasses for each supported version of Core), so be it.

I can certainly appreciate the prudence of not wanting to impose any sort of burden on Core or users when the class and its associated mechanisms inevitably change. But I hope that we can find an acceptable way to accommodate this userland innovation prior to the eventual HTML Processor rewrite. I really do want this functionality quite badly, and I have to figure that I'm not alone, given the popularity of Alpine.js and Datastar. Also, doing something like this in a plugin could help inform the implementation of an eventual limited js expression DSL in core.

What if, in addition to removing final, we just add a comment to the class saying something to the effect of "This class is no longer final, but should still be considered internal and will be subject to breaking changes. Any extensions of the class will carry the risk/burden of adapting to them"?

Again, I'm perfectly comfortable with doing whatever I need to in order to accommodate future changes.

The only alternative I can think of would be perhaps adding some sort of filter within evaluate(), but I have to figure that's even less palatable to you than removing final.

Does any of this seem possible? Or do I simply have to accept that the iAPI simply is what it is, take it or leave it?

P.S. Is there a ticket anywhere for the HTML Processor rewrite, even if just a placeholder/tracking ticket while waiting for the HTML Processor to evolve?

#18 @luisherranz
12 days ago

The problem is that a comment (or your personal commitment) only binds you. Once final is removed, the mechanism is available to every plugin in the directory, and core ends up inheriting the least careful consumer, not the most careful one.

The only mechanism I can think of that would open extension without creating a backward compatibility contract is something in the spirit of the @wordpress/private-apis package in Gutenberg (https://github.com/WordPress/gutenberg/tree/trunk/packages/private-apis). There, access requires programmatically passing an explicit consent string ("I acknowledge private features are not for use in themes or plugins and doing so will break in the next version of WordPress."). That makes the consent active instead of a passive comment, so breaking changes are part of the deal by construction. Something equivalent on the PHP side, like an unlock function gating the extensibility of this class, could work here.

To be honest, I don't know if a mechanism like this would be accepted on the PHP side of core, or whether there are precedents. If you want to pursue it, it would need research and buy-in from other committers, probably starting with a discussion in #core-php. What I can tell you is that I'd be willing to accept something in that style.

#19 @nickchomey
12 days ago

Thanks, that's a reasonable idea! I'll ask in core-php.

In the meantime, how about something like the following (I'm on my phone so can't provide a sample PR)

  1. Remove final from class-wp-interactivity-api.php
  1. interactivity-api.php change the wp_interactivity() function to something like
<?php
function wp_interactivity(): WP_Interactivity_API {
        global $wp_interactivity;

        if ( ! ( $wp_interactivity instanceof WP_Interactivity_API ) ) {
                $wp_interactivity = new WP_Interactivity_API();

                if (
                        defined( 'WP_INTERACTIVITY_ALLOW_PRIVATE_API' )
                        && WP_INTERACTIVITY_ALLOW_PRIVATE_API
                ) {
                        $instance = apply_filters(
                                'wp_interactivity_instance',
                                $wp_interactivity
                        );

                        if ( $instance instanceof WP_Interactivity_API ) {
                                $wp_interactivity = $instance;
                        }
                }
        }

        return $wp_interactivity;
}
  1. wp-settings.php change the hook instantiate to this, so that it lazy loads from whatever class ends up being used
    <?php
    add_action(
            'after_setup_theme',
            function () {
                    wp_interactivity()->add_hooks();
            }
    );
    
  1. Then a plugin does something like
    <?php
    
    define( 'WP_INTERACTIVITY_ALLOW_PRIVATE_API', true );
    
    add_filter(
            'wp_interactivity_instance',
            function () {
                    return new My_Interactivity_API();
            }
    );
    
    class My_Interactivity_API extends WP_Interactivity_API {
            // Customized implementation.
    }
    

It seems reasonably clean and the default behavior is the exact same. Any alternatives or feedback are welcomed.

Though, I suppose this explicit opt-in brings up the originally-rejected proposal once again: could methods and properties be changed from private to protected? That would make it easy to extend those rather than need to copy/fork the entire class. Of course, if that would make these changes less likely to happen, I'd be happy with only what i shared above in this comment.

Last edited 9 days ago by nickchomey (previous) (diff)

This ticket was mentioned in PR #13343 on WordPress/wordpress-develop by @nickchomey.


8 days ago
#20

Trac ticket: https://core.trac.wordpress.org/ticket/65757#comment:19

As per the discussion in the trac ticet, this PR aims to make the Interactivity API's WP_Interactivity_API class - used for server side processing of directives - minimally extensible.

Removing final from the class is the most minimal change that needs to be made to achieve this goal, without changing the private and protected status of the various properties and methods. But the concern still remains that even if someone wrote their entirely own class rather than override the methods/properties, it would still lock the API into a public contract.

So, this PR implements a pattern that is used in Gutenberg, and in the interactivity package in particular: an explicit developer opt-in when they are using an unstable/internal/private API. This would be achieved by defining a WP_INTERACTIVITY_ALLOW_PRIVATE_API const and also using the wp_interactivity_instance filter to replace the class instance with one that extends it.

Feedback on the possibility of this approach would be quite welcomed.

## Use of AI Tools

AI assistance: No

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

This ticket was mentioned in Slack in #core-php by nickchomey. View the logs.


8 days ago

#22 @dmsnell
8 days ago

bailing on unbalanced tags, SVG and MathML

@luisherranz you may find this interesting, but I explored a new path to using the Tag Processor for SVG and MathML. specifically, it allows us to process many documents, perhaps even most, as long as they don’t encounter tags which could potentially escape the parsing rules inside these elements (called “foreign content”).

You can see this work in PR#13271. Basically once we enter SVG or MATH elements, we proceed as usual, but if we encounter one of the special tags in the spec which requires processing with the more capable rules, we just give up and return only the HTML which was handled before the foreign element opened (before the SVG or MATH tags).

Anyway, just a side thought that allows us to safely process a bit more using the Tag Processor.

#23 follow-up: @dmsnell
8 days ago

adding some sort of filter within evaluate()

This seems more likely to fit within the framework, if at all.

As it is, there is strong reason to keep the class final due to the security-sensitivity of what it does, as well as how sensitively it must harmonize with the JS and HTML.

Attempting to evaluate JavaScript code from within PHP is likely to be a non-starter, not only because of the scope of needing to parse the JavaScript, but also because of the security domain it opens up.


This ticket is a good candidate for wontfix, but have you explored pre-processing the content which the Interactivity API processes to replace the expressions with their evaluation? If you can get there first then you wouldn’t have to modify the Interactivity API classes.

#24 in reply to: ↑ 23 @nickchomey
8 days ago

Replying to dmsnell:

adding some sort of filter within evaluate()

This seems more likely to fit within the framework, if at all.

As it is, there is strong reason to keep the class final due to the security-sensitivity of what it does, as well as how sensitively it must harmonize with the JS and HTML.

Would you mind elaborating on what security concerns you foresee with allowing a developer to explicitly opt-in to extend the class?

Attempting to evaluate JavaScript code from within PHP is likely to be a non-starter, not only because of the scope of needing to parse the JavaScript, but also because of the security domain it opens up.

To be clear, inline js expressions (with server side processing) is only one of the things that I'd like to be able to extend/modify. I also want to add other directives and associated processors. And there might be other modifications that I'll want to make as I explore further. Removing final and having the opt-in would enable all of that.

(also, please correct me if I'm wrong, but it seems to me that it isn't really WP Core's concern if anyone wants to do imprudent things in plugins - Core provides a solid foundation and functions to help make plugins secure, but can't and shouldn't try to enforce that. Not that I consider anything that I'm trying to do to actually be imprudent... I'm going to great lengths to make it secure and robust)

This ticket is a good candidate for wontfix, but have you explored pre-processing the content which the Interactivity API processes to replace the expressions with their evaluation? If you can get there first then you wouldn’t have to modify the Interactivity API classes.

I haven't explored that. Would you mind elaborating a bit more on the idea? Though, again, for something like adding a new directive processor, it seems that extending the class is quite clearly the appropriate approach.

Thanks for the feedback!

Last edited 8 days ago by nickchomey (previous) (diff)

#25 @dmsnell
8 days ago

Would you mind elaborating on what security concerns you foresee with allowing a developer to explicitly opt-in to extend the class?

All sub-classing is opt-in, so nothing is different here. However, extending the Core class invites changes to how values are secured in the output, even by accident. Whereas with a final class we can ensure that the important security boundaries are preserved.

(also, please correct me if I'm wrong, but it seems to me that it isn't really WP Core's concern if anyone wants to do imprudent things in plugins - Core provides a solid foundation and functions to help make plugins secure, but can't and shouldn't try to enforce that)

You are right that there are scopes where Core cannot prevent plugins from doing arbitrary work. However, it is very much a concern to design WordPress in such a way that it discourages accidental or malicious misuse. While there is certainly no perfect record of doing this, where able, it’s intended that Core should try and secure the page for site admins and visitors where possible.

Would you mind elaborating a bit more on the idea? Though, again, for something like adding a new directive processor, it seems that extending the class is quite clearly the appropriate approach.

While it may seem clear from one perspective, that doesn’t mean it’s clear to everyone. The Interactivity API examines HTML attributes whose names start with data-wp- — you can iterate through these and change what they have before the Interactivity API sees them.

E.g. if you want to create an “an ad hoc, informally-specified, bug-ridden, slow implementation of half of [JavaScript]”† you could turn things like data-wp-bind--post-date="Date.now()/1000" into data-wp-bind--post-date="1788215906550", so that the Interactivity API would never see the JS expression.

† Just a little humor from Greenspun’s Tenth Rule of programming.

#26 @nickchomey
8 days ago

I certainly greatly appreciate how much work goes into making WordPress secure and robust. But any plugin can cause catastrophic damage to any site - both in PHP and JS code (which has zero oversights/restrictions to what can be enqueued). Core even actively enables it by allowing any plugin to replace any of the highly sensitive functions within pluggable.php!

And putting aside malintent, it could even be as simple as forgetting to use wp_kses() and the like, which would allow user input to inject malicious script. It is up to things like the WP plugin repo, Plugin Check, and external reviewers/scanners to identify which ones are reliable.

But, I think we're getting somewhat derailed by my tangential explorations into implementing inline JS expressions which, in all likelihood, will remain a custom thing that I just use for myself. So, I'd like to refocus the discussion on the general case of how we might be able to provide extensibility to the iAPI directive processor.

If we just focus on the straightforward case of wanting to add an extra directive processor, ultimately if a plugin calls wp_interactivity_process_directives() directly without providing a filter to modify the content pre or post, then there's no way to intercept it in a granular way (we could use, for example, the wp_finalized_template_enhancement_output_buffer hook, but it would require walking the entire document rather than specific fragments, as is done in class-wp-blocks on a per-block basis)

As an alternative to removing final, I suppose that perhaps adding a filter within the iAPI (eg within wp_interactivity_process_directives() or process_directives() or _process_directives() themselves, might allow for sufficiently-modifying the string without extending the class. But it would be likely be convoluted and require implementing a lot of the same things that are already provided to other directives processors via the class. Most importantly, it might be tricky to make use of the state and context that was already registered.

Being able to explicitly opt-in to extending the class would be significantly simpler, and is also literally the same mechanism as is needed in order to add the actual custom directive to the iAPI JS, via privateApis.

I am certainly open to any creative suggestions, but it is very difficult to see how any alternative would be comparable in utility to removing final and making overriding opt-in in one way or another. I hope that you folks will be able to help refine my proposed PR into something that would satisfy all parties!

Last edited 8 days ago by nickchomey (previous) (diff)
Note: See TracTickets for help on using tickets.