Suspend and resume object cache

[ryan]

We sometimes need to suspend object cache operations, notably during installation and importing. During install we want to know we are working with the raw DB values. During import we want to avoid set/get trips to the cache (especially when using the memcached backend) for a potentially huge number of new objects -- objects that probably won't be requested again. We also want to avoid the overhead of caching imported objects in memory. With big imports we can bust the php memory limit.

In some places, we check if WP_IMPORTING or WP_INSTALLING is set and disable cache sets and gets. Not all cache backends do this however. Instead of referencing global defines, let's formalize this with two function calls, wp_suspend_cache() and wp_resume_cache(). When the cache is suspended, all cache gets return false and all sets/adds return true without actually setting anything. Deletes should probably still be processed. It is up to the caller to make sure it doesn't do things that would make the cache inconsistent with the DB while the cache is suspended. Given our current model where we usually delete on change and add on read, this shouldn't be a problem.

[markjaquith]

Cool by me.

[ryan]

Didn't really help import performance so not doing this.

[nacin]

Per duck_, for 10k posts, peak memory went from ~200 MB to 120 MB using the WordPress importer.

Closing #15812 as a duplicate and re-opening.

[ryan]

The old patch of mine suspended the cache completely, including gets. The new one just suspends adding new stuff. I think I prefer the new patch. Advantage of having the suspend function outside of cache.php is that you know it will always exist. It means a new global though.

[Denis-de-Bernardy]

See also #11405

[duck_]

Link the the "new patch"​ and to ​paste of my figures for reference, from #15812.

[duck_]

Was thinking about this again and two ideas for slightly different approaches.

Could use a filter instead of a global: if ( apply_filters( 'suspend_caching', false .... Downside being more overhead in a filter than another check.

Or take Ryan's old patch, move the functions out of cache.php and still allow gets. Therefore not introducing a new global, reusing $wp_object_cache instead. I prefer this approach.

Will re-test and upload patch later.

[joostdevalk]

I would be very much in favor of getting this feature in ASAP. My particular problem being with get_permalink, as that throws the post you're getting the permalink for into cache. This causes a huge memory usage when you're trying to generate a 1,000 post sitemap, and all you really need is the URL...

[SergeyBiryukov]

Replying to duck_:

Or take Ryan's old patch, move the functions out of cache.php and still allow gets.

Implemented in 5389.patch​.

[duck_]

Or it could be done with a static variable, see 5389.static.diff.

[ryan]

5389.static.2.diff is fine by me.

[nacin]

Let's go with wp_cache_add_suspend or wp_cache_suspend_adds?

[duck_]

In [18681]:

Introduce wp_suspend_cache_addition() to allow reduced memory usage when cache additions aren't useful. Fixes #5389.

[SergeyBiryukov]

Shouldn't we now use wp_suspend_cache_addition() during installation and importing, as suggested in the ticket description and in cache_suspend.diff​? Or is that already done elsewhere?