Framework for storing revisions of Post Meta

[alexkingorg]

There are a couple of tickets that would seem to benefit from storing revisions of post meta (#11049, #20299). We had a need for this feature in our Carrington Build and RAMP products and built it a few years back.

It handles the storing of post meta along with revisions, as well as restoring those revisions when someone restores an older version of a post(/page/etc.). There is an API for registering the post meta keys you want to track revisions of, it does not track all post meta keys by default.

There are two versions of the code. The bare-bones revisions framework code is here:

​https://github.com/crowdfavorite/wp-revision-manager
​https://github.com/crowdfavorite/wp-revision-manager/blob/master/cf-revision-manager.php

while I started adding features to a version that would be more user-friendly plugin, allowing the users to select which post meta keys they want to track revisions for:

​http://plugins.svn.wordpress.org/revision-manager/
​http://plugins.svn.wordpress.org/revision-manager/trunk/cf-revision-manager.php

If this would be a valuable addition to core, I'd be happy to help polish this up in whatever manner is most helpful. I'd recommend breaking out the code I started on for the admin form and having that be the extent of the "revision manager" plugin - basically it becomes a nicer UI for selecting additional post meta keys to track revisions for.

+1

This and #20299 would block #23314.

This and #20299 also potentially block the no-JS fallback for #19570, although MarkJaquith mentioned not allowing a post to switch formats after the fact if you have no JS.

Replying to nacin:

This and #20299 would block #23314.

will bring this back into scope, have you reviewed the code attached? if it looks reasonable i can work to apply.

Replying to adamsilverstein:

Replying to nacin:

This and #20299 would block #23314.

will bring this back into scope, have you reviewed the code attached? if it looks reasonable i can work to apply.

It's still outside the scope that the revisions team will primarily focus on.

From johnbillion, a very simple take: ​https://gist.github.com/johnbillion/5225514

Replying to markjaquith:

From johnbillion, a very simple take: ​https://gist.github.com/johnbillion/5225514

i have a patch brewing for this ticket and will post for your review monday.

for testing

20564.diff​

• adds new wp_save_post_revision_meta() - called in _wp_put_post_revision
• adds wp_restore_post_revision_meta() - added in restore action
• also stores post format as revision meta (and restores on revision restore), seemed to be best way to store post_format
• requires http://core.trac.wordpress.org/attachment/ticket/16215/16215.9.diff to work correctly (stores meta on copy of current revision, only present after patch)
• wp_revisions_keep_meta filter to override storing of meta
• initial testing verified post format and related meta stored and restored correctly

20564+16215.diff​​ combined patch for testing

updated against current trunk

update against current

We need the exact opposite approach of this patch. :-)

We should have a list of meta keys that we WILL revision, not a list of ones we WON'T. There should be no filter to opt out of revisioning of post format postmeta.

Replying to markjaquith:

We need the exact opposite approach of this patch. :-)

We should have a list of meta keys that we WILL revision, not a list of ones we WON'T. There should be no filter to opt out of revisioning of post format postmeta.

ok, this was the method suggested in #20299 so i went with it. a possible advantage of this method is it will catch new meta options as they are added or used, but i could also see how it could catch fields you didn't need.

i can change the method and include all current used metas, maybe add a filter for plugin authors to register new metas for revisioning?

I think we just need a few new wrapper functions (like add_versioned_meta(), update_versioned_meta(), and delete_versioned_meta() ) that don't check against wp_is_post_revision(). Using these would "opt in" to metadata on revisions. (of course you could currently just use *_metadata('post'...) functions, no?)

then on 'wp_restore_post_revision' (with 2 args)

$meta = get_post_meta( $revision_id, $key );
if ( empty( $meta ) )
    delete_post_meta( $post_id, $key );
else
    update_post_meta( $post_id, $key, $meta );
// or whatnot.

which is pretty straight forward (and also "opt in").
Optionally, core can hook onto 'wp_restore_post_revision' and loop thru a filterable array of "versioned" meta so that the following could work:

add_filter( 'versioned_meta', 'my_versioned_meta' );
function my_versioned_meta( $meta_keys ) {
    $meta_keys[] = 'my_key';
    return $meta_keys;
}

patch introduces add_versioned_meta(), update_versioned_meta(), and delete_versioned_meta() adds the filter 'versioned_meta' and handles restoring versioned meta data in wp_restore_post_revision() Lightly tested.

I don't think we're ready to bless this for plugin use. As you noted, they can just use the lower-level functions if they know what they're doing. So for this go-around, I'd skip wrapping it up all pretty. Taking a look at your patch now.

new patch adds get_versioned_meta_keys

Replying to markjaquith:

I don't think we're ready to bless this for plugin use. As you noted, they can just use the lower-level functions if they know what they're doing. So for this go-around, I'd skip wrapping it up all pretty. Taking a look at your patch now.

Yeah, that's cool. I was just working on this because it blocks #20299 which in turn blocks a couple other interesting tickets like #23314 and #23539

Figured if I got a couple working patches up it might move those next round.

Taking a stab at this in 20564.4.diff​. The patch is quite self-explanatory.

20564.5.diff​ is same as .4, except the *_post_meta functions hi-jacking. Uses the underlying add_metadata where applicable.

In 23859:

Revision our post format postmeta.

props kovshenin, WraithKenny. see #20564.

See [23859]

Been seeing this lately in the postmeta table, post IDs 9212, 9213 and 9214 are revisions and all meta is empty (no values):

4040 	9214 	_wp_format_video 	
4039 	9214 	_wp_format_audio 	
4038 	9214 	_wp_format_image 	
4037 	9214 	_wp_format_quote_source 	
4036 	9214 	_wp_format_quote 	
4035 	9214 	_wp_format_url 	
4032 	9213 	_wp_format_video 	
4031 	9213 	_wp_format_audio 	
4030 	9213 	_wp_format_image 	
4029 	9213 	_wp_format_quote_source 	
4028 	9213 	_wp_format_quote 	
4027 	9213 	_wp_format_url 	
4022 	9212 	_wp_format_video 	
4021 	9212 	_wp_format_audio 	
4020 	9212 	_wp_format_image 	
4019 	9212 	_wp_format_quote_source 	
4018 	9212 	_wp_format_quote 	
4017 	9212 	_wp_format_url 	

Seems we are doing isset() and should be doing ! empty() somewhere.

its the if ( false === $meta_values ) line I think

In 20564-6.patch:

• When saving revisions (including autosaves): save the current post format as revision meta.
• When restoring from a revision, restore the format too.
• When saving revisions: add post format meta to the revision post only for the current post format, skip when empty.

Replying to azaozz: The original post contains all those meta keys with empty values as well, and the revision post is a complete copy, so I think that's the expected behavior. I also think that there should be a more general approach for all possible meta, rather than post formats only. Thoughts?

The original post contains all those meta keys with empty values as well...

Not sure that's needed either.

...the revision post is a complete copy, so I think that's the expected behavior.

No, not a complete copy. It's a copy of the post data, most fields are not revisioned. By default only title, content and excerpt are. That's why we have _wp_post_revision_fields().

Imagine a small site with 100 posts, each having 10 revisions. That's (1,100 * 7) nearly 8,000 rows in the post meta table just for post formats. And most will be empty. Now imagine a bigger site with 20,000 posts and 50 revisions each (that's not uncommon). Nearly 8,000,000 rows. And now imagine a really big site... :)

I also think that there should be a more general approach for all possible meta, rather than post formats only.

Yeah, when we open the API to revision post meta (agree with @markjaquith it's out of scope at the moment), thinking it should be opt-in. A lot of the post meta doesn't need to be saved in revisions.

I'm pretty sure it's been mentioned somewhere before, but we can probably store all the revision meta in a single serialized _revision_meta key. I agree that adding keys to revisioned meta should be opt-in and that it's out of scope for 3.6, I just dislike the post-format-ish approach at _wp_post_revision_meta_keys(), and the $single assumption in get_post_meta :)

We could, but having separate meta has some advantages. We could do a query to select posts by these meta fields which might be useful in some cases. Also that makes post formats more "futureproof".

Yes, currently the revisions meta is only for post formats, and the fields can only be "single" there. It's not very hard to turn parts of it into an API, perhaps 3.7?

Replying to azaozz: I think what Aaron meant in his comment is useful for posts only, but not really useful for revisions. I took a stab at the _revision_meta approach in 20564.6.diff​ which should keep the meta table clean and small for revisions. If this works out we can do the same for revisioned terms (#23893). Not really looking to fix this ticket, but rather lay a good foundation for whatever is next in 3.7 and 3.8.

P.S. Sorry for the .6 vs -6 naming conflict, that's trac's auto-naming kicking in :)

The only drawback I can think of for serializing all the meta into one field is size concerns. For formats, it shouldn't be a problem, but if it's opened up later as an API, what if some meta are very large? Potentially, the values would be saved into individual keys, but when combined they maybe too big to store... no? It's an edge case, but who knows.

For the unique key issue, if the key is actually unique, then get_post_meta( $revision['ID'], $meta_key ) should only have one value in the array making it unneeded to pass a unique flag to add_post_meta (since the loop only goes once). I had a unique param in my patch originally, but after I saw the commit I thought it unnecessary. Is there a different unique assumption that I'm missing?

Replying to kovshenin:

I took a stab at the _revision_meta approach in 20564.6.diff​ which should keep the meta table clean and small for revisions. If this works out we can do the same for revisioned terms (#23893).

Yeah, this works for post formats, not so sure for all meta (only need to add restoring of the post format from 20564-6.patch, perhaps to the same array). Still, having empty keys in the serialized array seems wasteful :)

As long as we know the post supports formats, we can recreate the empty keys programmatically, no need to store them in the DB. Example:

$all_meta = array_merge(
  array(
	'_wp_format_url' => '',
	'_wp_format_quote' => '',
	'_wp_format_quote_source' => '',
	...
  ),
  get_post_meta( $revision->ID, '_revision_meta', true )
);

This brings another interesting question: when restoring from a revision, do we overwrite all data stored in the main post's formats meta? If yes, do empty meta keys from the revision overwrite (i.e. remove) the corresponding values on the main post?

User case:

• User creates a post, no format set.
• User saves a draft creating a revision.
• User edits the post, sets format to quote, enters data in 'url' and 'quote' meta fields.
• User restores to the first revision. At this point do we delete the data entered in 'url' and 'quote' on the main post or do we keep it? Logically we should delete it, but if we are keeping all post formats data ever entered on the main post, the user may expect to find it if he sets the format to quote again.

Replying to WraithKenny:

...if the key is actually unique, then get_post_meta( $revision['ID'], $meta_key ) should only have one value in the array making it unneeded to pass a unique flag to add_post_meta.

True, but we will need to do $meta[0] everywhere. In the current code post formats meta doesn't support multiple keys, better to use the meta API properly and get/set a single key.

Replying to azaozz:

Only need to add restoring of the post format from 20564-6.patch, perhaps to the same array.

That's taxonomy and terms, not post meta, hence the similar patch in #23893 for taxonomy. If the the serialized _revision_meta approach works well for meta, we can take a similar approach with terms and not "pollute" the term_relationships table with relations to revisions :)

Still, having empty keys in the serialized array seems wasteful :)

Maybe. We can indeed check the emptiness of meta before storing their revisions to save some space, but I have mixed feelings about it. I don't think it will affect post formats all that much, but I keep thinking that a non-existent meta value is not the same as a stored empty string, especially for multiple values under one key. If there was a $default argument to get_post_meta like there is for get_option, not storing the empty values would be a deal-breaker. So I lean towards doing delete_post_meta for empty strings when saving post format metadata. Just thinking out loud here.

This brings another interesting question: when restoring from a revision, do we overwrite all data stored in the main post's formats meta? If yes, do empty meta keys from the revision overwrite (i.e. remove) the corresponding values on the main post?

I can try and answer this with a similar question :) When restoring from a revision, do empty fields (title or content) overwrite the corresponding values in the main post? Create a post with no title, save a few revisions, give it a title, save and restore a previous revision: do we set the title back to an empty string? The answer is yes, and I think post formats (and revisioned post meta in general) should not be any different.

In 20564.7.diff​:

• Merged some of the things from Andrew's patch
• Switched to single value lookups
• Save and retrieve post format via a _revision_post_format meta
• Filters get_the_terms to show the revisioned post format during previews
• Doesn't store empty meta values for revisions

This also covers #23893

We can indeed check the emptiness of meta before storing their revisions to save some space, but I have mixed feelings about it.

What I mean is at the lowest level: we know the expected keys, so array( 'a' => '', 'b' => '', 'c' => 'something' ); doesn't have to be serialized, saved to the db and unserialized again when getting the data. We can remove the empty keys, save array( 'c' => 'something' ); and then add the empty keys on getting the data. This will work in exactly the same way as saving the original array.

The same is true when getting/setting the post format meta on revision posts. We know there are 7 meta keys, no need to add the empty.

I can try and answer this with a similar question :) When restoring from a revision, do empty fields (title or content) overwrite the corresponding values in the main post?

That's not exactly the same. From the discussion on #19570, we are keeping the unused data for post formats. Basically this is irrelevant post meta unless the user decides to switch back to a previously used post format. The question here is: do we "roll back" that irrelevant data when restoring a revision, and yes, logically we should.

20564-8.patch builds on 20564.7.diff​ and adds updating of post format and post format meta for autosaves.

20564-8.patch​ looks good and works like a charm!

In 23928:

Revisions:

• Store the post format as meta on revisions (including autosaves).
• Add post formats data (post meta) when autosaving.
• Only add non-empty post formats data to revisions.
• Correct the post format when previewing a published post.

Props kovshenin, see #19570, see #20564.

@azaozz thanks for the commit! One little thing I noticed is the @since version in the docblock of the new _wp_preview_terms_filter function ([23928]). It says 2.6 but should be 3.6. I mistyped that in my patch, sorry!

[23936] for @since update.