Add speculative loading support

[flixos90]

With the ​Speculative Loading plugin being successfully used on over 40,000 WordPress sites today, this ticket is about introducing the feature in WordPress Core.

As outlined in the ​speculative loading announcement post, speculative loading using the Speculation Rules API “can lead to near-instant page load times”. As such, it is a crucial feature in improving load time performance and (in terms of performance metrics) LCP. Please refer to the post for additional context on what the feature does and how it works. Feel free to review the ​Chrome documentation about the Speculation Rules API as well.

There are several signals that indicate that the feature is ready for WordPress Core:

• The ​Speculative Loading plugin has amassed 40,000+ active installs and has been used for over 6 months without major issues, while receiving exclusively 5-star ratings.
• Based on data queried from ​HTTP Archive and ​CrUX datasets over all 6+ months since the plugin launched, the sites that enabled speculative loading improved their LCP passing rate by ~1.9% at the median which, while it may seem a small number, is a huge boost for one specific feature, considering that 100s of sites with various performance implications contribute to the data.
• Based on ​Chrome stats on Speculation Rules API usage, speculative loading is already used on over 8% of Chrome page loads.
• ​Cloudflare has enabled speculative loading at large scale, providing assurance that even at a large scale like WordPress Core’s the feature would function without issues and lead to great results. In the linked post, they share data that their enabling of the feature led to a “reduction in LCP of 45% on successful prefetches”.
• Based on data from a joint experiment between Chrome and Shopify, conservative prefetch on Shopify sites gave a 50-60 ms P75 LCP improvement for successful prefetches depending on the platform (for an overall 30-50 ms P75 LCP improvement across the entire origins).

The proposed approach for implementing speculative loading in WordPress Core is to port over most of the code from the ​Speculative Loading plugin, with some key differences:

• There should not be any options or UI controls to customize the behavior of the feature. Instead a filter should be introduced for that purpose.
• The default behavior should “prefetch” URLs instead of “prerender” them, and the eagerness should be “conservative” instead of “moderate”. Basically we would be starting with the safest possible default, which should still give reasonable performance wins per the data above.
  • The plugin would continue to use the more performant yet riskier configuration of “prerender” with “moderate” eagerness, to further assess the implications of that configuration in the long term.
  • Once speculative loading would be merged into Core, the plugin would be modified to rely on Core’s implementation, but alter the default Core configuration as outlined.
• Rather than explicitly defining “prefetch” and “conservative” as the default values for the filter, a custom value “auto” should be used for the default, with the value signifying that WordPress Core makes the decision on how to speculatively load URLs. This will allow us to refine the default behavior for those sites in the future to achieve even bigger performance wins, e.g. if support of certain speculative loading behaviors across WordPress plugins and/or the web as a whole improves or additional research data suggests a better default.

The differences were informed by the plugin usage as well as the aforementioned Cloudflare’s large-scale enabling of speculative loading. They have rolled out conservative prefetching for now, but ​anticipate being able to iterate on it in the future.

This PR adds speculative loading support to WordPress Core, as outlined in the Trac ticket:

• Introduce function wp_get_speculation_rules_configuration() to get the current speculation rules configuration (consisting of "mode" and "eagerness").
  • The default values for mode and eagerness are auto, which later is evaluated into whatever the current Core default is. For the initial implementation at least, that default is to "prefetch" with "conservative" eagerness.
  • A new filter wp_speculation_rules_configuration can be used to override the behavior, either to change it to something more eager, or to hard-code it to the current default to force-control it, even if Core's default should change in the future.
• Introduce function wp_get_speculation_rules() to compute the full data for speculation Rules JSON output, based on the given configuration (return value from wp_get_speculation_rules_configuration()).
  • By default, the configuration will lead to speculative loading of all frontend URLs, while excluding patterns like /wp-admin/*, /wp-login.php, etc which should never be speculatively loaded.
  • A new filter wp_speculation_rules_href_exclude_paths can be used to add further URL patterns to exclude, relevant for plugins that may want to customize this. This filter receives the mode (prefetch or prerender) as context, which thus allows to add exclusions depending on which mode is currently used.
  • While the filter can be used to add exclusions, it cannot remove any of the default exclusions. This is because they are simply not safe to prefetch / prerender because of their dynamic nature.
  • The default configuration also excludes prefetching links that use a no-prefetch class on their anchor, as well as excludes prerendering links that use a no-prerender class.
• Introduce function wp_print_speculation_rules() which is hooked into wp_footer to print the default speculation rules JSON.

Trac ticket: https://core.trac.wordpress.org/ticket/62503

[adamsilverstein]

Thanks for opening this ticket @flixos90!

Some additional data from usage of the the Speculative Loading plugin collected from CrUX/httparchive is available in ​this colab. The data shows improvements across LCP, INP and CLS scores when the plugin is installed, and leads to approximately 6% of navigations getting "prerendered".

[westonruter]

In an upcoming version of Chromium, page responses with response that disable caching headers will no longer be bypassed for prefetch. I believe this may be a critical point to address as part of any core merge and as part of the Speculative Loading plugin. I've filed more details in the Performance repo: ​https://github.com/WordPress/performance/issues/1709

[westonruter]

Replying to westonruter:

In an upcoming version of Chromium, page responses with response that disable caching headers will no longer be bypassed for prefetch. I believe this may be a critical point to address as part of any core merge and as part of the Speculative Loading plugin. I've filed more details in the Performance repo: ​https://github.com/WordPress/performance/issues/1709

Thanks to @tunetheweb for confirming this does not impact Speculation Rules: ​https://github.com/WordPress/performance/issues/1709#issuecomment-2518254334

[swissspidy]

#62687 was marked as a duplicate.

[flixos90]

I have updated the PR ​https://github.com/WordPress/wordpress-develop/pull/7860 based on the latest additions to the Speculative Loading plugin:

• It now excludes URLs with query parameters, as custom query parameters may be used by plugins for "action links" that modify state and therefore should not be speculatively loaded.
• It also disables speculative loading entirely for logged-in users.

Both of these decisions strengthen the implementation, yet they can be bypassed via the filter in case the site owner is confident that those considerations aren't an issue for their site.

@swissspidy Feedback applied!

[flixos90]

I would appreciate additional reviews on this PR, since this is a new performance feature for WordPress Core.

Any chance some of you could take a look @joemcgill @mukesh27 @desrosj @SergeyBiryukov?