Many WP Dashboard-only transients for 'browser version' set to autoload=yes in wp_options

[archon810]

I've been recently optimizing WP for speed and for various reasons (mostly because it's buggy as hell), we're forced to run without W3TC's object cache on a busy site. Because of that, when wp_options gets bloated, especially with values that have autoload=yes, and every thread ends up transferring that much more data, overall performance decreases, and db traffic goes up.

Transients are typically (with few exceptions) added as autoload=no. This keeps performance in check.

However, there's one notable exception, and on our server, it's 120 rows all dedicated to a single transient, with over 30KB of data in total that gets transferred for no reason.

That transient is "browser_" . $key, and it's only used by wp_check_browser_version() in wp-admin/includes/dashboard.php. There's no reason at all for it to use autoload=yes.

[dd32]

As a recap of the rules we currently have

set_transient():

• autoload = no, if it never expires
• autoload = yes, if it expires (as is the case of browser_* cached for a week)

set_site_transient():

• multisite: never autoloaded (it uses sitemeta, which only autoloads specific options, the theme_roots transient being one of them)
• non-multisite; See set_transient()

It's also worth noting that most users should only ever really have one or two browser_* rows, as most peoples browser UA doesn't change often. If you have many different people accessing wp-admin with different browsers though, it'll certainly add up.

[archon810]

Correct me if I'm wrong but:

• The key here is user agent, which does change as you keep updating browsers (browser versions update pretty often these days). If you add a dozen writers and Firefox/Chrome variations, you end up with a lot.
• Transients are only cleaned up upon access, which means old browser entries are going to stick around forever.

I think this shows a larger problem with options, which is the lack of namespace if you will, so that wp-admin options load in threads used by all users, as well as lack of transient cleanup, which I think is a larger issue.

Finally, I'm not sure that a generic rule like autoload = yes if there's an expiration date without exceptions is good practice. In this case, for example, it should be set to autoload=no as it's not used in 99.999% of queries.

[dd32]

I incorrectly thought we were doing weekly garbage collection of transients, although it appears we're only doing it upon update #20316 - which definitely causes these transients to pile up for the reasons you mention (although browsers update far less than weekly on average, Chrome Devel does weekly updates for example).

To be clear - I wasn't saying that this is an ideal situation, I was merely recording our current logic behind transients and autoload (since most people probably assume transients are always non-autoloaded).

Additionally, if you're using a object-cache such as Memcache, these transients will never be stored in the DB and never auto-loaded.

---

One option to fix this, could be to define a list of admin-only transients, which would go hand-in-hand with a garbage collection.

[archon810]

Replying to dd32:

I incorrectly thought we were doing weekly garbage collection of transients, although it appears we're only doing it upon update #20316 - which definitely causes these transients to pile up for the reasons you mention (although browsers update far less than weekly on average, Chrome Devel does weekly updates for example).

To be clear - I wasn't saying that this is an ideal situation, I was merely recording our current logic behind transients and autoload (since most people probably assume transients are always non-autoloaded).

Additionally, if you're using a object-cache such as Memcache, these transients will never be stored in the DB and never auto-loaded.

---

One option to fix this, could be to define a list of admin-only transients, which would go hand-in-hand with a garbage collection.

Re: object caching. It's horribly broken in W3TC, probably the most popular caching plugin. I disabled object caching because of that - it's inconsistent and unpredictable. What's the suggested object caching plugin to use nowadays, if any? I'm concerned with data loss since Memcached loses data on restart, and I know I can rely on MySQL to not do that (unless the object caching plugin offers write-through caching).

Re: fixing. wp-admin transients sound good. In fact, a more generic support for grouping/namespacing sounds even better.

[dd32]

Re: object caching. ... What's the suggested object caching plugin to use nowadays, if any? I'm concerned with data loss since Memcached loses data on restart, and I know I can rely on MySQL to not do that

WordPress's object caching support offloads content from the database to memory, however all write operations still hit the database (Except transients, they never hit the database in this scenario - they're temporary data only).
It reduces processing load on the database and offers significant performance benefits, ​http://wptavern.com/persistent-object-caching and ​http://scotty-t.com/2012/01/20/wordpress-memcached/ offer insights on it.

[lukecavanagh]

@archon810

I would look at both
​https://wordpress.org/plugins/wp-redis/
​https://wordpress.org/plugins/memcached-redux/

[pbearne]

Looking at the code this look like it is fixed, and now

set_site_transient( 'browser_' . $key, $response, WEEK_IN_SECONDS );

All site_transient use update_network_option which sets the autoload to no