Need the ability to clear WP_Query cache partially and ability to set expiry for queries.

[thekt12]

Background

During the audit of an enterprise client's website, it was observed that the core caching mechanism in WordPress had been disabled, and a custom caching mechanism was built around WP_Query. This custom implementation was introduced to enable selective cache flushing for specific queries after data synchronization from a third-party system.

The primary motivation for this change was to avoid the "cache stampede" issue that occurred when flushing the entire WP_Query cache on high-traffic days. However, the custom solution, while addressing this specific need, lacked the robustness and performance benefits of WordPress's core caching mechanism.

Current Limitation in WP_Query Caching

At present, WordPress does not provide a way to selectively remove cached entries for WP_Query. The current implementation of core allows either full caching or none at all. This limitation necessitated the custom implementation, but it also highlighted opportunities for improving the core functionality of WP_Query caching.

Proposed Improvements

1. Selective Cache Flushing via Salted Keys

Introduce an action after cache key generation code that takes $cache_key and $args as parameters.
​$cache_key = $this->generate_cache_key( $q, $new_request );

$cache_key   = $this->generate_cache_key( $q, $new_request );
do_action( 'wp_query_cache_key_generated', $cache_key, $q ); 
// $q is query args, and the developer can pass a salt, which can be used to identify a group of queries.

Developers can utilize this action to maintain a record of cache entries associated with a specific set, enabling targeted cache clearing when needed.

2. Ability to set custom groups via query args

While the exact implementation needs further exploration, introducing the ability to assign a custom group name to specific queries via query arguments would be highly beneficial. This feature would enable developers to group related queries under a custom group name, allowing for targeted cache invalidation. Additionally, it would facilitate better segregation and management of cached entries.

Additionally add support for cache_expiry in query args.

Add support for a cache_expiry argument within WP_Query. This feature would allow developers to set a custom expiration time for cache entries. For example, data that frequently updates from the backend could have a shorter expiry time, reducing the need for manual cache-clearing mechanisms.

[mukesh27]

Here’s a polished version of your statement:

This ticket was discussed during the Performance bug scrub.

Caching the WP_Query is a critical change and needs to be approached with careful consideration.

Additional thoughts from @joemcgill: Implementing this change via an action is a viable approach, but it's also worth considering the ability to selectively purge specific queries from the cache or modify the expiry time.

This is something we can address in a Future Release.

Props to @joemcgill

[vineet2003]

I’ve been following the conversation around WP_Query caching, and I appreciate the feedback and insights shared by @joemcgill and others during the Performance bug scrub.

The primary challenge here is that WordPress currently lacks flexibility when it comes to caching WP_Query results. This limitation makes it difficult to selectively clear cached data and manage cache expiration times effectively, which can cause performance bottlenecks, especially on high-traffic sites.
The goal is to introduce a more granular caching system that allows developers to:

1. Selectively clear cache entries for specific queries or groups of queries.
2. Set custom cache expiration times for different queries.
3. Ensure performance is improved without compromising cache integrity.

To address these needs, I propose the following enhancements to WP_Query caching:

1. Selective Cache Flushing via Action Hooks: Introduce an action like wp_query_cache_key_generated to allow developers to hook into the cache key generation process, enabling targeted cache invalidation for specific queries or groups.

2. Custom Cache Groups: Allow developers to assign custom cache group names via query arguments (e.g., 'cache_group' => 'category_news') to group related queries together for easier cache management and invalidation.
3. Custom Cache Expiry: Introduce a cache_expiry argument for WP_Query, which will allow developers to set custom expiration times for cached results, reducing the need for manual cache clearing and ensuring data freshness.

I’d be happy to take on the implementation of this solution and contribute a patch for review.
If there are any additional concerns or suggestions, I’m open to collaborating further to ensure that this solution meets the requirements and is aligned with the best practices of WordPress development.