Delete Cache by Group

[filosofo]

Currently, you have to specify the id of the cached item in order to delete it via wp_cache_delete, or you have to flush the entire cache via wp_cache_flush.

Instead, it would be helpful if one were able to delete an entire group without having to flush all of the cache.

For example, term-related plugins could expect their particular caches to be deleted whenever changes to any terms were made, instead of having to use callbacks for every term-related action.

[ryan]

Cache has been gutted, so this is no longer relevant.

[sc0ttkclark]

Looks like the reason this was closed is no longer the case. What's the plan here? I want to delete all items for a specific group.

For now, I'll just work with the object itself, but there's a definite need for a function like wp_cache_delete_group or the patch as attached.

[scribu]

The reason this is not part of the API is that most persistent caching backend such as memcache don't support deleting an entire group.

What's done instead is increment the group prefix and let memcache's garbage collector remove the items from the old group, which are never accessed anymore.

[sc0ttkclark]

Okay, sorry for my n00bness as I'm diving deeper into WP caching than ever before, but how would I increment the group prefix? Feel free to reply with a *slap*

[scribu]

Actually, you don't change the prefix of the group, but add a prefix to each individual key:

$ns_key = wp_cache_get( 'foo_namespace_key' );

// if not set, initialize it
if ( $ns_key === false )
  wp_cache_set( 'foo_namespace_key', 1 );

$my_key = "foo_".$ns_key."_12345";
$my_value = wp_cache_get( $my_key, 'some_group' );

To clear the namespace do:

wp_cache_incr( 'foo_namespace_key' );

Source: ​https://groups.google.com/forum/?fromgroups#!topic/memcached/Izov0cFjBXI

[Ste_95]

Why hasn't this received any more attention? I still think it's relevant. The invalidation method posted by @scribu is good with memcached, but not if cached data goes to the database as a transient:

It is important to note that if you are not careful about how you are using this design pattern, you could accidentally consume all of your cache storage space. This strategy relies on changing the key that is used to access the cached object; it never removes the previously cached versions of the objects. It leaves it in the cache's storage. If your caching backend relies on a caching technology that does not automatically handle cache evictions, you may completely fill that storage. I tend to work with memcached on large projects that need efficient caching. Memcached implements an aggressive (sometimes way too aggressive) eviction policy whereby cached objects are frequently removed from cache. You can change the keys for the cached objects and the older objects will be removed from the cache, making room for newer versions of the objects. As such, the incrementor invalidation strategy is an effective tool in a memcached environment.

​https://www.tollmanz.com/invalidation-schemes/

Why not implement the patch from @filosofo? Or at least provide a method to access the $cache class var, so that we could iterate through the category array and manually delete each key?

[Ste_95]

Any news on thys guy? @SergeyBiryukov tagging you as you seem to be the only one with recent activity.

[dd32]

Replying to Ste_95:

Why hasn't this received any more attention? I still think it's relevant.

This was made irrelevant when WordPress removed persistent caching.
File-based Persistent caching was removed in WordPress 2.5 and switched to a non-persistent per-request cache instead, which removed the need to clear the cache like this.

The cache is designed to be replaced by a persistent in-memory cache such as memcache, redis, or any other number of cache backends. These do not generally allow deletion of an entire cache group, adding support to WordPress for it is therefor not exactly useful.

Why not implement the patch from @filosofo? Or at least provide a method to access the $cache class var, so that we could iterate through the category array and manually delete each key?

While we could enable this, it would cause your code to be not-very-portable, as soon as you used it on any site that had an external Object Caching dropin enabled, you'd see the code completely fail to operate as expected.

The invalidation method posted by scribu is good with memcached, but not if cached data goes to the database as a transient:

Transients are completely different to the wp_cache_*() functions, although transients do utilise them if an external cache is in use.
If you've got wp_cache_*() setup through a drop-in to use transients, then that's probably a bad move - it's really not designed to be used like that, and you'd be better off with that disabled in many scenario's I would assume.

You may want to look at how you're using transients if that's the case, and reconsider their usage. It sounds like you're using them in a way they weren't designed, or something similar. #20316 may interest you.

I'm re-closing this ticket as wontfix, you may comment here while the ticket is closed, but for the reasons outlined above, we won't be adding a Delete Cache by Group functionality right now, and probably not anytime soon.

[dhilditch]

Instead of adding 'delete by group', to achieve the same desired functionality - namely, the ability to 'delete' chunks from the cache without wiping the entire cache - how about this approach:

1. Add an extra parameter to wp_cache_get and wp_cache_set called 'namespace'
2. When the namespace parameter exists, use the code given by @scribu to append a number to the $key param
3. Add a new function wp_cache_clear_namespace with the namespace parameter - this function increments the namespace number so - in-effect - all items for this namespace are 'deleted'.

Otherwise EVERY developer has to keep coding their own versions of the code from @scribu both for getting items from the cache and for setting items in the cache, and most won't bother and instead will just flush the entire cache when their settings pages change which is entirely unnecessary and detrimental to the performance of large websites.

[dougal]

I have a current use-case where deleting by group would be useful -- a long-running report job where I want to track whether I've already processed certain items in the current queue run, then delete the cached hits after the report is done.

But yeah, since not all external caching mechanisms provide a mechanism for that functionality, and the dropins would have to be updated with the new method, I can see why it's wontfix. :-/

[lucasbustamante]

It would be nice if WordPress could enforce an object-cache API that the drop-ins could follow so we could use it reliably in plugin development.

For instance, Pantheon's Redis drop-in supports deleting a group of cache keys with wp_cache_delete_group( $group ):

​https://github.com/pantheon-systems/wp-redis/blob/master/object-cache.php#L98
​https://github.com/pantheon-systems/wp-redis/blob/master/object-cache.php#L547

While this other Redis drop-in with 60k+ installations does not:
​https://wordpress.org/plugins/redis-cache/ (​https://github.com/devgeniem/wp-redis-object-cache-dropin/blob/master/object-cache.php)

Automattic WP Memcached doesn't support groups, either: ​https://github.com/Automattic/wp-memcached/blob/master/object-cache.php

The solution I propose is that WordPress enforces an interface on cache drop-ins, so we can be sure that they will allow deleting groups. If Memcached needs some namespace hacks to work with groups, then that's handled internally by the drop-in.

[SergeyBiryukov]

Just noting that wp_cache_get_multiple() was added to wp-includes/cache-compat.php in [47939] / #20875 as a compat function for drop-ins that don't support it natively.

It seems like wp_cache_delete_group() could be added in the same way too.

[dg12345]

The main reason I can think to do this is that, currently to delete targeted cache you must use the "last_changed" method which essentially just adjust the keys used but that would also leave the old keys in the cache, so your redis or memcache will slowly get bigger.

The only other way is for you to keep track of all the keys used and write a script to delete each one when you want to delete the cache.

I agree with @lucasbustamante, simply putting it in as an official core function you will force the hand of the plugin writers to implement it in the main redis and memcache plugins.

I would think its a five minute job to add a delete by group function, the core doesn't even need to use it (although give the cache size issue it would be nice if it did).

​https://github.com/WordPress/performance/issues/38

Trac ticket: https://core.trac.wordpress.org/ticket/4476

[spacedmonkey]

I understand the desire for this ticket to go into core. But apart from the redis examples seen in this ticket, how would this work in other object cache drop-ins. How would this work for a memcache implementation? If someone could link to or provide a code snippet for a memcache or other object cache implement, that would be helpful.

If we can reliably have a way for keys in a group to be in core, then sadly, this function can not live core. The worry is that someone calls wp_cache_delete_group and it simply does not work. If this function existing in core, we could need some way to backfill / pollyfil, like done in wp_cache_get_multiple patch. Again, a can't think of a good way to do this without massively refactoring the internals of the object cache.

If this to live in core there needs to be a compat function found in core in ​cache-compat.php. This function would need to work for ALL object cache implementation, including and not limited to memcached.

I personally think this is going be nearly impossible, as it is hard to know the implementation details of every object cache. But without that compatible layer, this can not live in core 😞

This function would need to work for ALL object cache implementation, including and not limited to memcached.

It's a FAQ and the suggested approch is ​using prefixes, which I believe ​`wp-memcached` is already using heavily.

Please use remove test for wp_cache_get_linked_meta and revert changes to comments and spacing that are not required for this change.

If this is going to make it into core, then we are going to need some constant, like WP_ALLOW_CACHE_GROUPS. Drop-in providers, will have to define this. That way, developers / core can work out if we can even use this function.

Thoughts?

If this is going to make it into core, then we are going to need some constant, like WP_ALLOW_CACHE_GROUPS. Drop-in providers, will have to define this. That way, developers / core can work out if we can even use this function.

Reminds me of get_theme_support() and add_theme_support().

If we default to flushing the entire cache (in the compatibility method), can you give an example _how and where_ WP_ALLOW_CACHE_GROUPS would be used?

I am not sure we do need the global WP_ALLOW_CACHE_GROUPS as I test for a flush_group function
method_exists( $wp_object_cache, 'flush_group' )
and return false if not available so the calling code and handle it

[spacedmonkey]

Even through the PR at ​#2368 looks promising, it is clear from the backwards and forwards on the code review, that this ticket needs a little more consideration.

I am hopeful this can be resolve and make it into a future release.

There seems to be unit tests failing now and coding standards need fixing.

Hello,

What is the next step ?

Do you need help for something ?

@aksld The memcache object cache used by the tests needs support for this feature. Likely by using prefixes that increment.

Replacement for #2368.

Resolves ​WordPress/performance#38.

Trac ticket: https://core.trac.wordpress.org/ticket/4476

This is already looking good. Added some early feedback. The unit test failure, you may want to look into that.

[tillkruess]

I've opened a refreshed PR for this. Feedback greatly appreciated.

​https://github.com/WordPress/wordpress-develop/pull/2969

closed by ​https://github.com/WordPress/wordpress-develop/pull/2969

@spacedmonkey @tillkruss

Is this too silly to even consider? I suspect it might be.

WIP -- needs some docs updated, etc.

Nevermind, this won't work.

The wp_cache_* functions are effectively pluggable and it is expected the drop-ins will define them. A persistent drop-in that doesn't support group flushing would need to be updated to match hacking the groups with the last_changed key.

I'll continue investigating using an action on the last_changed key to allow drop-ins to hook in to that to trigger a group flush.

{{{php
add_action(

'wp_cache_set_last_changed',
function( $group ) {

global $object_cache;
$object_cache->flush_group( $group );

}

);
}}}

Marking a ready to commit.

Committed

Committed

Committed.

[SergeyBiryukov]

Fixed in [53763].

[SergeyBiryukov]

In 53767:

Cache API: Make the placement of wp_cache_flush_group() more consistent.

Includes:

• Placing WP_Object_Cache::flush_group() next to ::flush().
• Placing wp_cache_supports_group_flush() next to wp_cache_flush_group().
• Placing the wp_cache_flush_group() unit test next to the ::flush() method test.
• Removing test name from assertion messages, as it is already mentioned directly above in case of failure.
• Adjusting function descriptions per the documentation standards.

Follow-up to [52706], [53763].

See #55647, #4476.

[desrosj]

There are a lot of caching API related changes in 6.1 and a dev note should be written collecting them all. Marking this for inclusion.