Locks API for atomic data manipulation

[lpghatguy]

I originally opened a ticket at #34304 for locking methods specifically for post metadata. @rmccue made a note that a full-scale locking API might be more useful than a locking API specifically for post or userdata.

I won't fully iterate over why locking is beneficial, but in short, it prevents data races that can have bad results.

So now I'm here to propose a locking API for all sorts of things:

• Posts
• Users
• Post/User Meta
• Options

Initiailly, I proposed an API using methods like get_and_update_post_meta, taking in the normal parameters for get_post_meta and the like, but also taking a callback method. During the callback, the data would be locked and only accessible from the callback itself.

I provided a prototype implementation using MySQL's GET_LOCK and similar methods; it is attached here. To scale to the full system, it would require lock checking code in all queries reading any data, which might be a large undertaking. I haven't researched the potential performance implications of this, or how it would be applied to more complicated data.

Some questions I'd like to look into:

• What would a full-scope locking API look like?
• Is this feature useful for a regular WordPress site?
• Is this a problem other users have encountered?
• Does this solution scale?

[lpghatguy]

A proof of concept of the original locking proposal

[pento]

A locking API would be an interesting undertaking. :-)

I have a handful of thoughts around implementation:

• MySQL's GET_LOCK is really only useful for a site that has a single DB server - as soon as the site starts sharding and using HyperDB, it won't work. This method also wouldn't work for locks that need to last across multiple page loads - MySQL locks are implicitly released when the connection terminates.
• The way around that problem is to have a plugin to use memcached as the lock store, but that increases the complexity of scaling the site, if the site owner isn't ready to introduce memcached.
• There is already a locking pattern used successfully in a couple of places: WP_Upgrader::run() and _wp_batch_split_terms() are good examples. I'm not sure how well this would scale to a full locking API, though. This could potentially be the "core" method for locking, with a memcached-based plugin available for those who need something more scaleable.

From a user perspective, you mentioned that the implementation would need lock checking added to all queries, which is a large undertaking. I think it would be larger that that, however. For example, how do you indicate to the end user that they can't edit something, because it's currently being held by a long-lived lock? We have some special UI for showing when a post is currently being edited, something like that would need to be added everywhere. I'd also be concerned about the behaviour of plugins that don't take this into account - they update some data, and assume that it has been updated.

To take your example from #34304, I'm not sure that locking would be the correct solution for that scenario. When this happens:

R1: R
R2: R
R1: W
R2: W

I think it would be a better experience for R2 to notice that the data it's trying to change has been changed since it was read, and either magically handle it, or prompt the user for the correct course. I'm not sure that there's a nice way to implement this kind of the experience in a generic fashion - by nature, it's a very specialised response. But, that's part of the fun of implementing user interfaces that delight the user - it needs special handling for each edge case. :-)

I'm not adverse to the idea of us including a locking API in Core, but I would like to see some strong use cases before we start getting too far into the implementation.

[desrosj]

The milestone for this one was lost along the way. Re-adding.

I'm also going to add a close suggestion. It's been 7 years, and no strong use cases have been presented, so thinking a maybelater would be appropriate here.

I'll leave this open for a few more months to see if anyone wants to weigh in with any details or interest and I'll return to close it out if nothing is added.