Garbage collect transients

[nacin]

Per an IRC discussion and long-considered changes:

• In wp_scheduled_delete(), look for expired transients and purge them.

• On DB upgrade, purge all transients, regardless of expiration. (This should probably happen on auto-update as well, but let's let that part slide for now, as DB upgrade will run on all major releases these days.)

We should only do this if ! $_wp_using_ext_object_cache.

I would love to chase this one but I am rather new and could use some guidance.

Would it be better to set this up as a filter like rarst did here: ​http://wordpress.stackexchange.com/a/6652/11310 or just as a function call or even inline?

+1 for this.

I just checked the transients on one of my sites with my ​plugin and noticed that there are about 400 browser_ transients which are expired since 250 days and still exists.

On DB upgrade, purge all transients, regardless of expiration.

I'm not sure about this. This could result in an heavy first page load, depends on how the transients are used.

Should transients also be purged when an object-cache.php file is dropped in? Or at least should a task be scheduled which could do so? This way when someone eventually adds a custom object cache they are not stuck with hundreds of transient records in the database (even non-expired but now no longer used ones).

Or is this grounds for a separate ticket (or just a non-issue)?

20316.patch​ is an attempt to address both points of the description.

I'm also not sure about the possible performance impact of the second one though.

Meanwhile, I've ​released a plugin that will perform the proposed functionality.

Additionally, and based on a suggestion I read elsewhere, I've added in an optimisation of the table after the removals take place.

Closed #24206 as a duplicate.

Is there anything we can do to help progress on this issue? It just bit me - I found my options table had bloated to 250,000 entries, after just 2 months of the site being live.

David

Replying to DavidAnderson:

Is there anything we can do to help progress on this issue? It just bit me - I found my options table had bloated to 250,000 entries, after just 2 months of the site being live.

I can't answer as to when it will be looked at, but can I assume you've tried my plugin in the meantime (link above)? This will, for the time being, perform the same processing and keep the entries cleared.

I'm looking to implement a further option to report on which plugins are causing it. Technically they're not doing anything wrong but there may be an advantage to them re-thinking their caching strategy (as I've been doing with my own).

Yes - I'd seen the plugin and was about to deploy it. The code using transients was my own! I've used transients as expiring access-tokens on many and various sites. I thought it was great that I *didn't* then have to spend any time thinking about cacheing strategy since WordPress had core functionality to do that for me.

Deploying a plugin is little effort, of course. But having this in core seems like a no-brainer - is there any good reason to *not* garbage collect expired transients?

Replying to DavidAnderson:

Yes - I'd seen the plugin and was about to deploy it. The code using transients was my own! I've used transients as expiring access-tokens on many and various sites. I thought it was great that I *didn't* then have to spend any time thinking about cacheing strategy since WordPress had core functionality to do that for me.

Deploying a plugin is little effort, of course. But having this in core seems like a no-brainer - is there any good reason to *not* garbage collect expired transients?

Not that I'm aware of.

I'm not a Core developer so not in a position to do it myself. However, I know Andrew (Nacin) is keen to get this included - I guess it's just a case of time and priorities.

Replying to DavidAnderson:

Is there anything we can do to help progress on this issue? It just bit me - I found my options table had bloated to 250,000 entries, after just 2 months of the site being live.

David

It's been marked 3.7-early, so hopefully we can get this in in the next version.

Is there anything we can do to help progress on this issue?

1. Use Memcached
2. Manually delete them, then use Memcached

The ​WP e-Commerce plugin includes a use case for unique transient keys.

The Australia Post API responses are cached using transients, and the cache key(s) depend on the volume/dimensions/destination of each customer's cart contents, which means there can be quite a few different transient cache keys on each store.

Garbage collection of expired transients would be a fantastic addition I think.

Expired transients should probably be removed using either delete_option() — akin to how get_transient() removes an expired transient, rather than announcing it with delete_transient()'s hooks — or since we are already doing straight database calls, with a DELETE. I don't think there is a need to do this noisily. It's kind of like something silently vanishing from a cache backend.

+1 for early 3.7, btw.

Replying to nacin:

Expired transients should probably be removed using either delete_option() — akin to how get_transient() removes an expired transient, rather than announcing it with delete_transient()'s hooks — or since we are already doing straight database calls, with a DELETE. I don't think there is a need to do this noisily. It's kind of like something silently vanishing from a cache backend.

I totally agree. I was using a simple selection of the transient by SQL in my plugin but using delete_transient to then remove them - amongst other things this isn't very efficient at removing large numbers of un-housekeep transients. I'm coding a change this week to replace it with a single SQL to achieve it.

Having worked on the aforementioned plugin to provide this functionality can I highlight two concerns that need to be considered?

1. As a user has mentioned above some people have a LARGE number of expired transients. Can I suggest that whichever package this is implemented in should perform a total wipe of transient data even if a database upgrade is not due? Although it will only affect the first run, performance does need to be considered - a single SQL statement (or as few as possible) if at all possible.
2. I have also experienced issues with orphaned transients - where one of the transient record pairs has either gone missing or, due to truncation of the name, are now mis-matched. This will mean that the regular housekeeping is likely to miss these but I would suggest that the database upgrade 'clear all' should handle these. In my case I've, basically, performed a delete on the options table for all '_transient_' records - it's rough and basic but is needed to ensure that all transients truly are removed.