Add metadata support for taxonomy terms

[sirzooro]

At this moment there is no dedicated way to store additional data for taxonomy. Plugin developers have to develop their own method for storing this data, e.g. by storing them encoded in description field or using set_option(). It will be good to add new functions for this, e.g. add_taxonomy_data()/get_taxonomy_data().

[filosofo]

Since taxonomies can be hierarchical, if a certain taxonomy needs metadata why not just have a descendant taxonomy that handles the metadata? I've done this for several plugins.

[sirzooro]

For non-hierarchical taxonomies this may be an option. For hierarchical ones you have to implement way to distinguish parent metadata from child. I know it can be done, but it will be tricky :)

[filosofo]

True. Another option is just to make another taxonomy, outside of the hierarchy, that handles the metadata.

Or maybe we should go to a general metadata table after all: #5183

[sirzooro]

Looks that with changes made in WP 2.9 for comments metadata this should be quite easy to implement - see attached path.

TODO: delete metadata when term is removed (and probably other things).

[sirzooro]

1st attempt (diff vs WP 2.9 RC1)

[hakre]

Can't we start to put the taxonomy meta functions into a class? I mean this is PHP 4 code, functions could be called staticly by plugin authors with only little overhead.

[hakre]

as suggested in #5183 I think it's worth to consider an "overall" meta api first. this can be factored upon the current comments metadata API which has been used as a pattern by sirzooro as well. Then Plugin authors can create on their own needs right away (and we would reduce code duplication as with the current patch).

[hakre]

Taken from #5183:

get_general_meta, update_general_meta, and delete_general_meta functions successfully.

By the way, update_meta and delete_meta would have been cleaner names, but they're already taken for admin post meta stuff.

[nacin]

#11847 closed as a duplicate of this.

See also #9674.

[mikeschinkel]

Replying to hakre:

as suggested in #5183 I think it's worth to consider an "overall" meta api first. this can be factored upon the current comments metadata API which has been used as a pattern by sirzooro as well. Then Plugin authors can create on their own needs right away (and we would reduce code duplication as with the current patch).

I really agree with this and wish that commentmeta had been general purpose instead of specific. I have use for linkmeta, termtaxonomymeta, termmeta, and termrelationshipmeta. I'd propose get_object_meta(), update_object_meta() delete_object_meta().

[nacin]

Not part of 3.0 scope.

[williamsba1]

I'm +1000 for this. Would love to utilize the update_metadata (and similar) functions, but with taxonomies it requires a custom table be created first. If wp_termdata existed it wouldn't be an issue.

[jeremyclarke]

+1, not every user needs this but many of us do and we need something that we can count on working forever.

[atoon]

+1, I believe this must be considered for the 3.1 release.

[ramiy]

+1

[mikeschinkel]

Questions on what people are wanting: Meta for term_id, meta for taxonomy_term_id, meta for object_id+term_taxonomy_id` (i.e. for term_relationships), or a meta for all of them?

[iamfriendly]

Personally, I think you should allow developers as much freedom as possible and therefore would ideally prefer meta on all of them, mikeschinkel.

A firm +1 from me on the 'idea' of this ticket.

[scribu]

Unfortunately, "developer freedom" comes at a high cost, much like "end-user freedom" when it comes to giving them a lot of options.

[sirzooro]

I think meta for term_id and for taxonomy_term_id would be enough. Meta for object_id+term_taxonomy_id can be stored as post meta.

[sirzooro]

BTW, CC list is getting longer and longer. Is it possible to include this ticket in 3.2 (preferable as Task)?

[scribu]

Currently, terms can be shared between taxonomies, although there is a trend against that: #5809

With such a shared term, it's much more likely that you will want to associate different metadata for different taxonomies.

Therefore, as it is now, if we add a term meta table, it should be connected via taxonomy_term_id only.

Flipping the coin, if we connect it via term_id only, it's easy to differentiate based on taxonomy, by adding the taxonomy name to the meta key, for example.

My point is that we should have one or the other, not both.

[scribu]

Replying to sirzooro:

BTW, CC list is getting longer and longer. Is it possible to include this ticket in 3.2 (preferable as Task)?

This won't become a task until consensus is reached, firstly on the usefulness of the feature, and secondly on the approach.

[mikeschinkel]

Replying to scribu:

Flipping the coin, if we connect it via term_id only, it's easy to differentiate based on taxonomy, by adding the taxonomy name to the meta key, for example.

My point is that we should have one or the other, not both.

FWIW, I have had need for both metadata for taxonomy_term_id and also for object_id+taxonomy_term_id (wp_term_relationships); I have not really had need for metadata for term_id.

[dd32]

FWIW, I have had need for both metadata for taxonomy_term_id and also for taxonomy_relationship_id;

The latter can be used with the former with the use of a prefix or suffix however, which tends to boil it back to only needing a tt_id metastore.

[mikeschinkel]

Replying to dd32:

The latter can be used with the former with the use of a prefix or suffix however, which tends to boil it back to only needing a tt_id metastore.

Sorry, I don't follow that at all. Can you give a use-case example to clarify?

[dd32]

Sorry, I don't follow that at all. Can you give a use-case example to clarify?

Example meta key with taxonomy_term_id, to apply to a specific tt for all relationships would be:

• 'colour' => red

If you needed relationship meta, ie. object_id+taxonomy_term_id for a specific tt to a specific object,

• $post->ID . '_colour' => red

My point, was that negates the need for a relationship meta table, and pushes it back to only needing the term_taxonomy meta

[mikeschinkel]

Replying to dd32:

Sorry, I don't follow that at all. Can you give a use-case example to clarify?

Example meta key with taxonomy_term_id, to apply to a specific tt for all relationships would be:

• 'colour' => red

If you needed relationship meta, ie. object_id+taxonomy_term_id for a specific tt to a specific object,

• $post->ID . '_colour' => red

My point, was that negates the need for a relationship meta table, and pushes it back to only needing the term_taxonomy meta

Thanks for the example.

So by that logic we don't actually need taxonomy meta at all. Instead we can just use wp_options with a prefix like this:

• 'taxterm_' . $taxonomy_term->taxonomy_term_id, and
• 'taxtermrel_' . $post->ID . '_' . $taxonomy_term->taxonomy_term_id

Right?

[dd32]

So by that logic we don't actually need taxonomy meta at all. Instead we can just use wp_options with a prefix like this:

Technically, Yes, It can be done that way, same as Postmeta can be done that way.

But I'm basing that off the current methodologies used in WordPress already for meta.

We have User Meta, which is per-user, however, in that same item, we can apply meta for user+site or user+network Meta, this is roughly the same as TT Meta, TT+Object Meta.

[dd32]

The reasoning for saying it like that, Is that Meta for objects always has a primary target, which is where the majority of the meta goes in 99% of scenario's, which is followed by a few other related spurs, which 10% of sites might use 1% of the time on average.

To make a specific table for (for example) taxonomy relationships to objects would probably fall into the 95/5% use case bin, where for the 5% that need it, 75% of those will find a prefix a working solution. The others will feel the need for a new table. Those that need the new table in the specific implementations are free to do so if they desire the extra performance gains that it may give them.

[mikeschinkel]

Replying to dd32:

The reasoning for saying it like that, Is that Meta for objects always has a primary target, which is where the majority of the meta goes in 99% of scenario's, which is followed by a few other related spurs, which 10% of sites might use 1% of the time on average.

To make a specific table for (for example) taxonomy relationships to objects would probably fall into the 95/5% use case bin, where for the 5% that need it, 75% of those will find a prefix a working solution. The others will feel the need for a new table. Those that need the new table in the specific implementations are free to do so if they desire the extra performance gains that it may give them.

Fair point. But let's look at statistics from another perspective.

For almost any site that uses or needs taxonomy meta the likelihood of having more than 1000 term_taxonomy records in a given taxonomy is very low; wouldn't you agree? As such, optimizing it doesn't really make much sense; we can easily simulate with wp_options and 'taxterm_' . $taxonomy_term->taxonomy_term_id without any scalability issues.

On the other hand the likelihood of having 1000 or even 10,000 or more wp_term_relationships records is reasonably high and the chances of 100,000+ or even 1 million such records is not complete remote. So leaving the only option in core for term_relationship meta to use a 255 byte key effective ensures it cannot be use for any site that might grow beyond a moderate number of post records in order to optimize for the case that won't ever need to scale.

Given that analysis if I was given the option to ONLY have meta for term_taxonomy_id OR for object_id+term_taxonomy_id I would pick the latter because it would be easy to simulate the former in a performant way with 0+term_taxonomy_id.

[mfields]

I'm all for this feature to be added and would like to share my experiences with taxonomy meta as well as my thoughts on why a term meta table is important in core.

I created a ​taxonomy images plugin a while ago and it has been really well received. It stores a serialized array of image->ID => term_taxonomy_id in a single row of the options table. While this is not ideal it seems to work and will accommodate thousands of image/term associations. I thought all was well and then one day I received a ​support request alerting me that my plugin did not retain associations when a user moved a blog using the ​WordPress Importer plugin.

Having a core term meta table that is able to be imported/exported from one installation to another is very important.

I would also like to become more involved in core. If this enhancement is accepted I am more than willing to help see it through for 3.2.

Best,
-Mike

[scribu]

@Mike and @dd32: If you're replying to the message exactly above, please don't quote the entire thing again. It uselessly clutters the screen. Thanks.

@mfields: Isn't there a hook in the exporter and importer for adding your custom tables?

[mfields]

@scribu: You've definitely inspired me to take a look at the code in the importer/exporter to see what I can do! Thanks for that. If it's relatively easy and sound to send custom data to and fro, then maybe a core termmeta table is not as necessary as I thought.

[mikeschinkel]

Related #17477

[scribu]

As Otto has said countless times, terms are just a convenient name for a group of posts in a taxonomy, which is a named set of groups. They're not meant to hold any data of their own.

If you find yourself needing to add additional fields to terms, you should actually make them posts and link them to other posts. See #14513

In other words, post-to-post relationships is the more appropriate solution.

[nacin]

We haven't ruled this out. Closing instead as maybelater.

[greenshady]

@nacin

You mentioned somewhere about writing out a guide on how you'd do this for core if implemented. Did you ever get around to that? I'd just like to future-proof some stuff I'm working on as much as possible.

Replying to scribu:

As Otto has said countless times, terms are just a convenient name for a group of posts in a taxonomy, which is a named set of groups. They're not meant to hold any data of their own.

If you find yourself needing to add additional fields to terms, you should actually make them posts and link them to other posts. See #14513

In other words, post-to-post relationships is the more appropriate solution.

What if someone wanted to create a "category image" plugin? Or, add SEO features (meta, keywords, title) to category archive pages?

These are just simple examples of extra fields someone might want to add to terms. Obviously, there are some things better handled by post-to-post relationships, but not everything fits into this scenario.

[nacin]

Replying to greenshady:

@nacin

You mentioned somewhere about writing out a guide on how you'd do this for core if implemented. Did you ever get around to that? I'd just like to future-proof some stuff I'm working on as much as possible.

I didn't, mainly because after a few conversations we couldn't come up with a good way to do this for core.

As described in #17477, we don't pass around term_taxonomy_id in our API, but rather term_id, which requires one to also pass around the taxonomy argument. For a meta table, ideally we'd want just one key (thus term_taxonomy_id) rather than two (term_id, taxonomy) but our current taxonomy API doesn't really allow for that (and our metadata API doesn't allow for the other). So I don't think this can happen in core until our API grows further.

There are three plugins in the directory [1]. IIRC, one uses term_taxonomy_id, one uses term_id (alone), one uses term_id + taxonomy. They seem to use different terminology for either function names, field names, or table names.

[1]

• ​http://wordpress.org/extend/plugins/simple-term-meta/ (removed, unsupported by author)
• ​http://wordpress.org/extend/plugins/taxonomy-metadata/
• ​http://wordpress.org/extend/plugins/meta-for-taxonomies/

[ramiy]

Found on WordPress codex - How to Add Metadata to a Term: ​http://codex.wordpress.org/Function_Reference/add_metadata#Add_Metadata_to_a_Term

(Using Metadata API located in wp-includes/meta.php)

[nacin]

Examples for things like that shouldn't be on the Codex.

[nacin]

I'd rather a low-level function like get_metadata() to not even be on the Codex.

[mikeschinkel]

Replying to nacin:

There are three plugins in the directory [1]. IIRC, one uses term_taxonomy_id, one uses term_id (alone), one uses term_id + taxonomy. They seem to use different terminology for either function names, field names, or table names.

​http://wordpress.org/extend/plugins/simple-term-meta/ (removed, unsupported by author)
​http://wordpress.org/extend/plugins/taxonomy-metadata/
​http://wordpress.org/extend/plugins/meta-for-taxonomies/

Nice list. Which is closest to an implementation that the core team would consider?

[dwenaus]

any movement on this for 3.4? Just as an example: right now I need to implement a taxonomy header image, and there is no clean way to do it. There are so many other use-cases for taxonomy meta.

[williamsba1]

I wrote a blog post a while back on storing taxonomy meta data in the options table:

​http://www.strangework.com/2010/07/01/how-to-save-taxonomy-meta-data-as-an-options-array-in-wordpress/

Not ideal for large amounts of data, but if you have a small number of taxonomy terms it works great!

[redrings]

would like to be able to add link and image fields to custom taxonomies

[ryno267]

really would love to see this in 3.5 or soon...

[ramiy]

I'm just thinking loud, but why not adding an extra theme feature:

add_theme_support( 'taxonomy-thumbnails' );

just like:

add_theme_support( 'post-thumbnails' );

using the new taxonomy metadata, of course, when it will be available.

[AliMH]

i don't think it's really a need for wordpress right now, i developed my own movie review site and though to have extra field per term but after a little thinking i found term description enough so just create a CPT and add all you need at a post then store postid in term description, when ever you wanna need anything (like images, rating,custom fields and..) will use that postid to show them, here is my example:
​http://www.animcentral.com/directors/kichi-mashimo

[082net]

As term and taxonomy is related to objects(almost to posts), I think we better go with 'nav menu' like system. And if we need 'REAL' metadata, then we can use postmeta system.

Advantage of following 'nav menu' system is much more than just using 'metadata', considering about what we need is... not a metadata but a relationship for objects or just an object.

Agree with ​scribu

[F J Kaiser]

As I need this regularly, I've writte my own solution: Missuses the "description" field as storage container for a serialized array. The only thing that was a little tricky was to display the plain description in the admin UI. Anyway this would be the solution I was hoping for: Simply convert the description field instead of adding a bunch of new fields. If there's a need for it, then we will see the use cases after we got it and move for a searchable solution with a later version. Just my two cents...

[husobj]

It's also worth noting here that in a lot of plugins I've seen that add meta for taxonomies, the metadata is stored against the term_id.

This means that if a term is used by multiple taxonomies you can only store one set of data for it rather than one for each taxonomy - better to attach to term_taxonomy_id perhaps?

[stephenh1988]

Replying to husobj:

It's also worth noting here that in a lot of plugins I've seen that add meta for taxonomies, the metadata is stored against the term_id.

I think normally plug-ins doing this are adding in a meta table for a specific taxonomy. In general, yes it would require the term_taxonomy_id to be used - unfortunately, the taxonomy API passes around term-taxonomy pairs rather than the term_taxonomy_id.

Nacin mentioned earlier that for that reason the API needs to 'grow'. I heard via twitter that a (long) roadmap was draw up. Sounded like it involved several table reshuffles... My point is that as eager as I am to see this in core - there is no immediate solution :(.

[sil.linguist]

Someone above mentioned that it is highly unlikely that a single taxonomy will have more than 1000 terms. I have a use case where I have a taxonomy with over 7000 terms. And these terms do have meta as well. In a way the post-to-post relationships makes sense for taxonomies this large and detailed. [1] Has anyone on the list here attempted a typology of use caes? Would such a discussion possibly bring us to a point where consensus could be achieved?

Not mentioned yet in the discussion, is the ability to turn any CPT into a taxonomy. If we go the route that taxonomies need to be just as flexible as CPTs (with respect to meta) then why does core support taxonomies as a separate object type? Just add to core the ability to make a CPT function as a taxonomies would have functioned. There is a plugin which sorta does this: ​http://wordpress.org/extend/plugins/cpt-onomies/

BTW: I deal with language and linguistic data.

[1] ​http://wordpress.org/extend/plugins/posts-to-posts/

[F J Kaiser]

Replying to sil.linguist:

Someone above mentioned that it is highly unlikely that a single taxonomy will have more than 1000 terms. I have a use case where I have a taxonomy with over 7000 terms.

Has anyone on the list here attempted a typology of use caes? Would such a discussion possibly bring us to a point where consensus could be achieved?

Just add one pretty common use case: Locations. In some installations I built, I got locations as hierarchical taxonomy - Country/State/District. There're currently (depending on the definition of state) ~196 countries in the world. You can guess the number of states and districts.

[ocean90]

#25902 was marked as a duplicate.

[TweetyThierry]

Hi all,

Personally I think there should the a wp_termmeta table (term_id, term_meta, term_value) which works the same way as the postmeta. This would handle scalability, keep it conventional and well organized. The saving part would simply use update_term_meta() and the deleting part would then be automated just like the post works with custom metas.

I am happy to contribute if necessary.

Thanks,

[helen]

Replying to TweetyThierry:

I suppose this was a "maybelater" because there are some other steps to take first. You'll probably be interested in this: ​http://make.wordpress.org/core/2013/07/28/potential-roadmap-for-taxonomy-meta-and-post-relationships/

[juliobox]

Since today ​http://make.wordpress.org/core/2014/05/24/were-ready-for-more-contributors-to-join-us-on-the-wp-metadata-ui-api-project/

[ericlewis]

Replying to TweetyThierry:

Hi all,

Personally I think there should the a wp_termmeta table (term_id, term_meta, term_value) which works the same way as the postmeta. This would handle scalability, keep it conventional and well organized. The saving part would simply use update_term_meta() and the deleting part would then be automated just like the post works with custom metas.

I am happy to contribute if necessary.

Thanks,

Hey TweetyThierry,

as this would be a game-changing ticket, it would require a blessed task team working on it during a cycle. This ticket was closed three years ago; as has been stated elsewhere, trac is a good place for working out implementation, not waiting around for someone to name a brilliant paradigm-shifting API.

If you'd like to be involved (or anyone else) I'd suggest you to join the ​metadata UI API team's next dev chat, as @juliobox suggests.

[boonebgorges]

In 4.1 we've begun to phase out shared taxonomy terms. See #5809, #21950. In a future release of WordPress, we'll have one-to-one correspondence between items in wp_terms and wp_term_taxonomy (and indeed, the two may be merged). See #30261, which blocks this ticket. So I think it's time to reopen and consider what a termmeta API would look like.

A few initial thoughts:

• Since term_id and term_taxonomy_id will be one-to-one, I think it makes sense to use the public term_id as the "owner" of termmeta. term_taxonomy_id thus becomes deprecated. (It could just as well be the other way around, though then we need a translation layer.)
• We'll need CRUD wrappers for add_metadata(), update_metadata(), etc.
• meta_query support for get_terms() and maybe other term-fetching functions.

[MikeSchinkel]

Replying to boonebgorges:

• Since term_id and term_taxonomy_id will be one-to-one, I think it makes sense to use the public term_id as the "owner" of termmeta. term_taxonomy_id thus becomes deprecated. (It could just as well be the other way around, though then we need a translation layer.)

It's a little thing, but I do so hope we'd go with term_id instead of term_taxonomy_id; my fingers object in a passive aggressive manner every time I try to type the latter; they inject typos into that long identifier name. :-)

• We'll need CRUD wrappers for add_metadata(), update_metadata(), etc.

What's the chance of revisiting the idea of a more generic meta_data table that would allow for other types of metadata in addition to metadata for term/taxonomy? I'm not suggesting we change post, user or comment meta but that we consider term meta being just meta with an object_type field?

[cjaimet]

Replying to boonebgorges:

I've been waiting for term meta data pretty much since I started WP development. I work with a large news agency and we use home/cat/tag pages extensively and serve several post lists to each with a modular layout defined by editors in the back end.

Right now, I'm using WP Large Options to store the JSON encoded configuration data in custom posts. It works well enough when serving data to an index page, but feels like a hack to me and isn't very queryable.

[marsjaninzmarsa]

I'll be glad if i can see this in 4.4 core, probably after providing feature-plugin for 4.3 (if we can close #30261 before 4.3).

It's very important thing for lot of plugin and theme developers...

[lgedeon]

I know that we are only a few months from having this in core now, but just want to share a method for doing term-meta without adding a table that works now and can be transitioned into using core in the future. ​https://github.com/lgedeon/term-meta

[marsjaninzmarsa]

Replying to lgedeon:

I know that we are only a few months from having this in core now, but just want to share a method for doing term-meta without adding a table that works now and can be transitioned into using core in the future. ​https://github.com/lgedeon/term-meta

There are also a few other plugins (even in repo) trying to achieve the same thing, and if they authors wanna to (and I think that they should want to), they can all provide in release corresponding to 4.4 conversion script and end up as just a compatibility layer, translating requests from theme or other plugins (before they have also been upgraded for use native implementation). @boonebgorges ​said on Slack that this could cause issues, but I think that transition of plugin features should be a plugin thing, not a core thing.

But now we have 4.3, and it's the best time to start working on real implementation. I'll be glad if I can cooperate with some of you on feature plugin for 4.3, which can be safely merged into core in 4.4. :)

[marsjaninzmarsa]

Ok, @boonebgorges ​explained concerns about implementation and possible conflicts, he got the point.

[flixos90]

I wasn't able to join today's Slack meeting on the taxonomy enhancements, but I just read through the chat - and although I'm also one of the people who has been looking forward to have term meta included for quite a while, the more I think about it, the more hesitant I am:

To me (and also others on the chat), the usage of metadata on posts is a clear distinctive feature of posts vs terms which I think should stay like that. Terms are simple objects, used to group posts together. If term meta was in Core, many people would probably add lots and lots of metadata to terms, which would basically lead to a similar behavior like when we have a post connected to another post (either with a plugin like P2P or a custom solution involving storing the other post's ID as meta).

I don't like the idea that people possibly use terms for something that should actually be posts. I might be drastic in my opinion, but I personally see it like that: If my taxonomy needs to store other data than its name, title and description, it's not a taxonomy. It's a post that should allow to be connected to another post.

The statement above has a few (very few) exceptions though. @krogsgard mentioned on Slack that a common use-case for term meta is the need for term images or term ordering. When I read this, it strengthened my opinion like so: if terms were to have images and an order field, I would never ever need to think about term meta again. And I think a lot of people who have run into this issue probably just thought the most straightforward way to solve this would be to add term meta. We have never thought of extending the terms table instead. But wouldn't it be possible to add a those two things into the table, in form of a term_image field (for an attachment ID) and a menu_order field (integer for the order)? This change could be incorporated into the improvements in #30262.

Sorry for the long post, I had to put it into words here - but here's the gist of it :)
It's not a big deal to add term meta by a plugin like ​https://wordpress.org/plugins/wp-term-meta/ if someone absolutely needs it. But since the addition of this feature to Core would probably interfere with a lot of plugins, significantly reduce performance on term queries and (my opinion) confuse the original purpose of terms, I prefer extending the terms table just for those common use-cases which would enhance terms, but not bloat them up.

[marsjaninzmarsa]

Replying to flixos90:

I wasn't able to join today's Slack meeting on the taxonomy enhancements

That was yesterday?? :O Damn...

I don't like the idea that people possibly use terms for something that should actually be posts. I might be drastic in my opinion, but I personally see it like that: If my taxonomy needs to store other data than its name, title and description, it's not a taxonomy. It's a post that should allow to be connected to another post.

The statement above has a few (very few) exceptions though. @krogsgard mentioned on Slack that a common use-case for term meta is the need for term images or term ordering. When I read this, it strengthened my opinion like so: if terms were to have images and an order field, I would never ever need to think about term meta again. And I think a lot of people who have run into this issue probably just thought the most straightforward way to solve this would be to add term meta. We have never thought of extending the terms table instead. But wouldn't it be possible to add a those two things into the table, in form of a term_image field (for an attachment ID) and a menu_order field (integer for the order)? This change could be incorporated into the improvements in #30262.

Examples from my recent projects:

• Color taxonomy attached to paintings CPT
• Product taxonomy attached to outpost CPT

In first case I needed name of color and slug based on name, that was obvious, but I also needed place for storing HEX value of color. And what with "Black and white" and "multicolor"? Second case was much more complex, with dedicated importer from corpo CRMs (50+ products, 50000+ outposts), each outpost and product with UID (different between each database and another database with mappings, independent imports from each database, nice mess). CPT was obvious choice for outposts and taxes for products, but I needed place for storing UIDs. Few months later had to be made import with product before debut on the market, so I needed also an option to show only products already on stock.

In both cases I needed filtering based on metafields and taxes, and in both cases I ended up with custom schemed data in comment, and I'm sure that comment field wasn't intended for data.

Name, title and description is enough if you're seeing WordPress only as blogging CMS, but I've done on top of it blogs, large companies sites, shops, two social apps, and two dedicated CRMs. I love flexibility and scalability of WP with CPT, but taxonomies in row with user management are last two things wasn't that flexible like the rest, it hurts sometimes...

Sorry for the long post, I had to put it into words here - but here's the gist of it :)
It's not a big deal to add term meta by a plugin like ​https://wordpress.org/plugins/wp-term-meta/ if someone absolutely needs it.

That isn't long-term solution because of interoperability and few more things.

But since the addition of this feature to Core would probably interfere with a lot of plugins

And prevent interfering with lot more of plugins in future.

significantly reduce performance on term queries

Only if you opt-in this taxonomy to support metadata?

and (my opinion) confuse the original purpose of terms, I prefer extending the terms table just for those common use-cases which would enhance terms, but not bloat them up.

Common for who? Common for 5% of users who wanna have term icons, or for 10% of devs who wanna build something custom on top of terms? (I've just fabricated this percents, but do you have better stats?)
I don't think that another (mandatory for every term) column in terms table can be faster than (opt-in) meta table when you're not using it (not using probably will be more common case than using), but it's only my intuition.

​https://wordpress.slack.com/archives/core/p1441310435002734 ← meeting archive for archive.

[MikeSchinkel]

Replying to marsjaninzmarsa:

Examples from my recent projects:

• Color taxonomy attached to paintings CPT
• Product taxonomy attached to outpost CPT

In both cases I needed filtering based on metafields and taxes, and in both cases I ended up with custom schemed data in comment, and I'm sure that comment field wasn't intended for data.

Name, title and description is enough if you're seeing WordPress only as blogging CMS, but I've done on top of it blogs, large companies sites, shops, two social apps, and two dedicated CRMs. I love flexibility and scalability of WP with CPT, but taxonomies in row with user management are last two things wasn't that flexible like the rest, it hurts sometimes...

It's not a big deal to add term meta by a plugin like ​https://wordpress.org/plugins/wp-term-meta/ if someone absolutely needs it.

That isn't long-term solution because of interoperability and few more things.

+1

[flixos90]

Replying to marsjaninzmarsa:

Examples from my recent projects:

• Color taxonomy attached to paintings CPT
• Product taxonomy attached to outpost CPT

In first case I needed name of color and slug based on name, that was obvious, but I also needed place for storing HEX value of color. And what with "Black and white" and "multicolor"? Second case was much more complex, with dedicated importer from corpo CRMs (50+ products, 50000+ outposts), each outpost and product with UID (different between each database and another database with mappings, independent imports from each database, nice mess). CPT was obvious choice for outposts and taxes for products, but I needed place for storing UIDs. Few months later had to be made import with product before debut on the market, so I needed also an option to show only products already on stock.

I guess it depends on the way you see posts vs terms. As I mentioned above, I would have created Color and Product as post types, adding a dropdown of all colors as a meta field for the paintings post type and a dropdown of all products to the outpost post type to create the relationships.

Name, title and description is enough if you're seeing WordPress only as blogging CMS, but I've done on top of it blogs, large companies sites, shops, two social apps, and two dedicated CRMs. I love flexibility and scalability of WP with CPT, but taxonomies in row with user management are last two things wasn't that flexible like the rest, it hurts sometimes...

I have been working with large sites too and did not run into problems there. For example, I've been working on an events site that has dedicated pages for events, artists, venues and event promoters, all post types, connected using meta. We then needed to integrate cities and states, connected to the venues. Since cities and states were just required to list their venues (and their venues' events), those were realized as taxonomies. If they had needed additional data, I would have realized them as additional post types instead.

But since the addition of this feature to Core would probably interfere with a lot of plugins

And prevent interfering with lot more of plugins in future.

Sure, if we integrated term meta, we should do it as soon as possible. But if we decide not to, we won't interfere with plugins that do it themselves.

significantly reduce performance on term queries

Only if you opt-in this taxonomy to support metadata?

I don't think opt-in for term meta is a good idea ("Decisions, not Options"). Moreover, there is no opt-in for post and user meta, so why would we do it for terms? I'd rather have a decision being made over the next weeks/months.

[marsjaninzmarsa]

Replying to flixos90:

Replying to marsjaninzmarsa:

Examples from my recent projects:

• Color taxonomy attached to paintings CPT
• Product taxonomy attached to outpost CPT

In first case I needed name of color and slug based on name, that was obvious, but I also needed place for storing HEX value of color. And what with "Black and white" and "multicolor"? Second case was much more complex, with dedicated importer from corpo CRMs (50+ products, 50000+ outposts), each outpost and product with UID (different between each database and another database with mappings, independent imports from each database, nice mess). CPT was obvious choice for outposts and taxes for products, but I needed place for storing UIDs. Few months later had to be made import with product before debut on the market, so I needed also an option to show only products already on stock.

I guess it depends on the way you see posts vs terms. As I mentioned above, I would have created Color and Product as post types, adding a dropdown of all colors as a meta field for the paintings post type and a dropdown of all products to the outpost post type to create the relationships.

And you would end up with nice fulltext search every time you wanna query items by color or product, nice. And with ~ 1 million extra rows in postmeta table (~20 products in each outpost), all with longtext, just for storing relations between posts...

Name, title and description is enough if you're seeing WordPress only as blogging CMS, but I've done on top of it blogs, large companies sites, shops, two social apps, and two dedicated CRMs. I love flexibility and scalability of WP with CPT, but taxonomies in row with user management are last two things wasn't that flexible like the rest, it hurts sometimes...

I have been working with large sites too and did not run into problems there. For example, I've been working on an events site that has dedicated pages for events, artists, venues and event promoters, all post types, connected using meta. We then needed to integrate cities and states, connected to the venues. Since cities and states were just required to list their venues (and their venues' events), those were realized as taxonomies. If they had needed additional data, I would have realized them as additional post types instead.

Let's go back to my second example, outposts and products. Hierarchical products. With filtering by parent product. And without storing relations to children product and all of ancestors in outposts, because I don't wanna rewrite 20000 posts each time when I change hierarchy.

This. And performance. This is tax territory, not posts territory.

significantly reduce performance on term queries

Only if you opt-in this taxonomy to support metadata?

I don't think opt-in for term meta is a good idea ("Decisions, not Options"). Moreover, there is no opt-in for post and user meta, so why would we do it for terms? I'd rather have a decision being made over the next weeks/months.

Yes, there is opt-in for post meta: ​https://codex.wordpress.org/Function_Reference/register_post_type#supports

There is no opt-in for user meta, because WordPress uses user meta for such things as nickname or capabilities...

[flixos90]

Replying to marsjaninzmarsa:

And you would end up with nice fulltext search every time you wanna query items by color or product, nice. And with ~ 1 million extra rows in postmeta table (~20 products in each outpost), all with longtext, just for storing relations between posts...

Okay granted, that's not the best solution either. In that case I would rather have a post_relationships table being introduced, but that's probably a little too much off-topic here.

significantly reduce performance on term queries

Only if you opt-in this taxonomy to support metadata?

I don't think opt-in for term meta is a good idea ("Decisions, not Options"). Moreover, there is no opt-in for post and user meta, so why would we do it for terms? I'd rather have a decision being made over the next weeks/months.

Yes, there is opt-in for post meta: ​https://codex.wordpress.org/Function_Reference/register_post_type#supports

Isn't that just for the ability to manually add custom fields? It basically is post meta, but you can always use the underlying CRUD functions for post meta even if that support is not enabled.

[marsjaninzmarsa]

Replying to flixos90:

Okay granted, that's not the best solution either. In that case I would rather have a post_relationships table being introduced, but that's probably a little too much off-topic here.

Probably. :)

significantly reduce performance on term queries

Only if you opt-in this taxonomy to support metadata?

I don't think opt-in for term meta is a good idea ("Decisions, not Options"). Moreover, there is no opt-in for post and user meta, so why would we do it for terms? I'd rather have a decision being made over the next weeks/months.

Yes, there is opt-in for post meta: ​https://codex.wordpress.org/Function_Reference/register_post_type#supports

Isn't that just for the ability to manually add custom fields? It basically is post meta, but you can always use the underlying CRUD functions for post meta even if that support is not enabled.

Hmm, dunno, I must check it out...
And you don't have add_taxonomy_support() function, so it cannot be moved 1:1 anyway. Performance when querying Taxes is indeed important thing, so we must ensure that taxmeta doesn't cause drop of speed for most of users not using it. I think that they just shouldn't been querying by default, and another if statement checking for optional meta_query param or so shouldn't impact performance at all.

[johnjamesjacoby]

I agree that a general object-to-object table would be extremely helpful, but those relationships will still need metadata and taxonomy-terms are the best way to accomplish this today without completely reinventing something new, which is not any less likely to have it's own issues ten years from today.

The Posts-to-Posts plugin is pure genius and I find it incredibly useful, but it's ambitious for WordPress core adoption when nothing in the Posts/Pages/Categories/Tags/Menus/Users objects needs it. A taxonomy term metadata table would be an absolutely huge improvement for all WordPress installations, and could be used in core later as taxonomy-terms become increasingly descriptive. (Then, best case scenario, terms are so hugely descriptive we migrate them into post-type taxonomy with a subtype column added to the posts table. /tangent)

A few of my anecdotal experiences and opinions:

• A performance compromise was made in bbPress 2.x's because of a lack of term metadata to keep counts of both topics & replies in a "Forum" post type vs. a taxonomy & terms.
• Without term-meta, it's impossible to describe a tag or category beyond it's literal description. Terms are crippled without metadata, and are infinitely powerful with it.
• Taxonomies & terms are everywhere on the web, and they have metadata attached to them. Github issues, Trac milestones & severities, etc... They have colors, timestamps, orders, priorities, and other little nuances that set their functionalities apart, but they're only real purpose is connecting a central object together with other similar objects.
• Taxonomies & terms are everywhere in real life, and we connect arbitrary metadata to them. Shirts, pants, socks. Hangers, folded, drawers. Large, 34/34, athletic. Black, blue, black. Center, top, bottom. Right now, you can only describe a "Shirt" as "This is a shirt" without any other designation.

Incoming screenshots will show off a few taxonomy term applications in the wild, to hopefully help steer this conversation back towards metadata and away from inventing something new, which IMO is out of scope for this feature & idea.

[johnjamesjacoby]

Github labels, using colors to visually separate them and indicate importance

[johnjamesjacoby]

Trac milestones, using dates to indicate order & completion

[johnjamesjacoby]

Trac resolutions, using order to indicate priority

[johnjamesjacoby]

Term colors plugin, to mimic Github labels

[johnjamesjacoby]

Term icons plugin, to attach a dashicon to a term

[johnjamesjacoby]

Term authors, similar to what's powering wordpress.org/plugins/

[cjaimet]

+1 @johnjamesjacoby

As a news publisher, we associate an enormous amount of meta data with tag and category records, including page layout definitions, advertising tags, sidebars, SEO titles, video URLs and data feeds. We mostly use WP Large Options for this, but true term meta data would be infinitely preferrable.

[boonebgorges]

Thanks for the thoughts, all.

If my taxonomy needs to store other data than its name, title and description, it's not a taxonomy. It's a post that should allow to be connected to another post.

This is a bit arbitrary. Why should "description" be included? Answer: because it has always been included :) Core should provide enough tools to devs to let them build what they need to build in an elegant way. For posts, this means metadata, along with tons of UI (think about CPTs in the Dashboard) and lots of template-related logic (think about the template hierarchy and the way URL parsing works). For terms, we've traditionally had an extremely limited feature set, and the general dev consensus seems to be that they're a bit hamstrung by the lack of term metadata. Adding this feature won't turn terms into posts. It'll just make the dev toolbox more poweful.

I've put a high-level outline of how I envision the implementation looking in core: ​https://make.wordpress.org/core/2015/09/04/taxonomy-term-metadata-proposal/ Feedback is welcome there. Once we have agreement on the high-level details, we can start the parallel projects of (a) deciding whether this is something we are sure we want, (b) working on patches, and (c) preparing outreach and documentation for plugin developers, especially those whose plugins will create critical conflicts with core termmeta. So get over there and leave your thoughts, if only a +1 :)

[flixos90]

Regardless of the discussion above, I'm looking forward to help. :)

I wonder how far we're gonna take this new feature though, in terms of UI especially. What about something like the custom-fields on posts so that the user can manually generate whatever fields he likes on terms? Or will that rather be plugin developer business?

[boonebgorges]

Replying to flixos90:

I wonder how far we're gonna take this new feature though, in terms of UI especially. What about something like the custom-fields on posts so that the user can manually generate whatever fields he likes on terms? Or will that rather be plugin developer business?

I left that out of the initial proposal. It's definitely something we can think about adding, though it probably doesn't have to be part of the initial rollout.

[marsjaninzmarsa]

I think we can think about this later, that can be handle by plugins (at least until we think about it in one of next releases), interface is secondary thing.

[dancameron]

Replying to boonebgorges:

Replying to flixos90:

I wonder how far we're gonna take this new feature though, in terms of UI especially. ...

... It's definitely something we can think about adding, though it probably doesn't have to be part of the initial rollout.

+1 to making this more about the API rather than dealing with a UI solution (that IMO is already resolved with numerous other APIs).

(edited because an emoji blocked the response)

[thomask]

I would like to propose for this discussion slightly different approach – move terms to posts. It might seem strange at the beginning, but if you think about it a little more, i believe you will see all the advantages. So please invest few seconds time to think about it.

All terms and posts need the same "fields" and settings. You need name, description, slug, date created, and also other post features would be welcomed – author/owner, shorttext… It would make everything much easier – you will not need to come with extra features, you would not add new tables – instead you will get rid of one table, future multilanguage features would be again easier and what is the most important – it would be possible to do things, that are not possible now – all post to post / post to taxonomy / taxonomy to post / taxonomy to taxonomy relations, not only taxonomy to post as now. Typical example – authors for books, or brands to cars.

I believe this would be also faster, as it would allow easier caching etc. And also the administration all all the relevant functions and pages would be much easier, you will not have to take care of two duplicating table lists, edit pages, meta boxes etc.

[johnjamesjacoby]

Replying to thomask:

I would like to propose for this discussion slightly different approach – move terms to posts. It might seem strange at the beginning, but if you think about it a little more, i believe you will see all the advantages. So please invest few seconds time to think about it.

This has been proposed before, even as part of the original thought-process. The posts-to-posts plugin accomplishes this quite well, and the value of connected multiple posts to multiple other posts is clear once you discover how difficult it is to do currently.

This said, we can map the loose schema of any object to any other object easily enough. Comments as posts? Users as posts? Sites as posts? Sure... there are plenty of similarities, but once we achieve universal-object bliss, we're left with a flexible schema that we can't easily change anymore because it affects every single other object in the system. We also end up with gaps where these objects don't overlap (count & term_group come to mind for terms..., user_email for users, and what about that old comment_karma column?)

Consider also that terms were intended to be more flexible than they were ever able to be. term_group was never implemented; shared terms were largely broken; term_order for relationship management was never implemented. These ideas couldn't easily be mapped to the existing posts schema, so they started in taxonomies and were never finished (not unlike comment-types, post-statuses, etc...)

What Boone & others are doing with Term metadata needs to happen in an evolutionary way, even if it eventually results in yours (and other's) observations being 100% correct about relating posts-to-posts, and the overlap of terms & posts.

[thomask]

Thank you for such detailed reply and it is great it was considered (even if you've realised, that it is not the right solution - i believe you are much higher experts then me). I can live with all the solutions, if they will allow also post2post relations etc.

So please at least consider unify all the relevant functions for administration, e.g. displaying term/post detail, term/post table for list pages, same function for widgets to both posts/terms etc. So that both the current term related functions and and post related functions would be just aliases to that global functions just with defined type.

I believe it would save a lot of energy to future development of WP and for web and plugins authors. If you look at it, praticaly the only difference between post list tables and term list tables is that the terms got "quick edit" on the side and posts got it inline, but this could (and should) be easily merged to one behaviour, so that users will not have to use different approaches for similar tasks. BTW here IMO the best would be folding (right) side quick edit, that would have all the advantages from both posts and terms - space for another columns, possibility to use it for new quick post ... and also advatage that it could contain the same type of widgets as detail (just they will be in one narrow column).

And the same for the term/post detail, where again the ability to define all the possible extra metaboxes with the same functions would be huge improvement.

And it could be also adopted for attachment details popup, where is also the right column with postdata, metas and termdata but now again with completely different approach then for any other type of content. And left part for attachments is just another view of the list table (similar as in eshops - show as table / show as thumbnails) and conversely this optional thumbnail view might be interesting feature also for posts and terms tables.

And this would be also future-proof and plugin-proof, if there would be any need to create another 3 tables with "post"-data, metadata and relations, than it would mean just another type definition, so that the tables would know what fields are available in the "post" table.

[boonebgorges]

All things being equal, it's incorrect to say that unifying terms and posts will improve performance and usability. Regarding performance (to echo johnjamesjacoby), it'd mean converting lots of the taxonomy component from what are currently straightforward relational MySQL queries to terrible meta queries. The caching bit is a red herring: the taxonomy component already has good caching built in (arguably more powerful than what's in posts); all that would be accomplished by combining the two is that we'd be able to reduce the amount of code required to implement caching. However, performance *without* caching would probably greatly decrease, given the huge amount of data that would be moved to a single set of database tables. Regarding dev usability: for the foreseeable future, the developer-facing APIs for terms vs posts will remain so distinct that it all but eliminates all potential API simplification that might come from merging the database schemas.

In any case, all of this is beside the point of the current ticket. Adding metadata to taxonomy terms is something that we can and probably should do today, without worrying about long-term plans for term storage. If we decided to merge terms and posts at some point down the road, we'd have to write a migration routine anyway. If anyone is interested in pursuing a data-type-merging project - be it terms, or comments, or users, or whatever - I urge you to create a proof-of-concept, and then to run some thorough benchmarks and post them somewhere (like a blog post) for discussion.

[boonebgorges]

10142.diff​ is a first pass at implementation. I haven't yet done 'meta_query' for get_terms() or wp_get_object_terms() - I first wanted to get the cache implementation straight, and in people's hands for feedback.

Some notes and areas for feedback:

• Termmeta caches are primed by default in get_terms() and wp_get_object_terms(). In each function, the priming can be disabled with update_term_meta_cache=false.
• Inside of the main $wp_query loop, termmeta caches are lazy-loaded. It works like this. (1) The cache is not primed during the main query ('update_term_meta_cache' is set to false in update_object_term_cache()). (2) When get_term_meta() is called within the loop, wp_lazyload_term_meta() checks to make sure that the terms themselves are being primed for the items in the loop. (3) If so, the function grabs the IDs of *all* terms associated with *all* posts queried in the loop ($wp_query->posts), and it primes the termmeta for those terms. (4) All subsequent calls to get_term_meta() within the loop will hit the cache. The purpose of this lazy-loading is to eliminate the termmeta query when no termmeta is in fact being used in the loop.
• wp_get_object_terms() only supports termmeta cache priming when fields is 'term_id', 'all', or 'all_with_object_id'. We need the term_id to prime the caches, and with other values of fields, it's not available. It'd be possible to refactor wp_get_object_terms() to make the cache priming more flexible, but I don't think it's worth the trouble (and the potential performance hits when using, eg, fields=name)

Thoughts and eyeballs are very welcome on this patch. I'll continue to plug away at the tax_query implementation.

==

On a somewhat related note, I've almost finished going through all the plugins in the repo that reference termmeta in one way or another. The results are not as bad as I'd thought - a decent number of plugins that have a very small number of users will be affected, but a very small number of popular plugins. I'll post results and recommendations soon. Based on the results, I think we can shoot for 4.4, if all goes well.

[thomask]

Following the previous discussion I have created Ticket #33931, where i added PDF with wireframes, that describe the complexity of current list screens and quick edit screen, including the taxonomy screens, and i have prepared wireframes of possible improved way how to list and (quick) edit all posts, taxonomies and media.

I think it would also solve a lot of practical problems, that would otherwise be with adding metadata to taxonomies and it would make it much easier to implement it, as with this apprach most functions could be reused for every item type.

I would love your opinions, especialy from @boonebgorges and @johnjamesjacoby

[johnjamesjacoby]

10142.dfff is exactly what I would have done. Thumbs up from me.

[boonebgorges]

10142.2.diff​ adds meta_query support to get_terms() and wp_get_object_terms(). It also cleans up some documentation.

In 10142.2.diff, I also added a routine to pre_schema_upgrade() that ensures that the index length on the meta_key field of wp_termmeta is 191. In my journey through the plugin repo, I found a number of plugins that created wp_termmeta, and almost all of them did it without specifying this limit on the index length. This seems like a case where we can fairly seamlessly inherit data from those plugins (whose table schemas otherwise match our own). @pento, I was hoping you could honor me with your esteemed opinion on this idea.

[boonebgorges]

In 34529:

Introduce metadata for taxonomy terms.

Adds a new table to the database schema (wp_termmeta), and a set of
*_term_meta() API functions. get_terms() and wp_get_object_terms()
now also support 'meta_query' parameters, with syntax identical to other
uses of WP_Meta_Query.

When fetching terms via get_terms() or wp_get_object_terms(), metadata for
matched terms is preloaded into the cache by default. Disable this behavior
by setting the new $update_term_meta_cache paramater to false.

To maximize performance, within WP_Query loops, the termmeta cache is *not*
primed by default. Instead, we use a lazy-loading technique: metadata for all
terms belonging to posts in the loop is loaded into the cache the first time
that get_term_meta() is called within the loop.

Props boonebgorges, sirzooro.
See #10142.

[boonebgorges]

In 34530:

After [34529], normalize documentation for get_terms() and wp_get_object_terms().

See #10142.

[boonebgorges]

In 34532:

After [34529], fix multisite site creation test.

The test Tests_Multisite_Site::test_created_site_details() checks for tables
that should be empty on a newly created MS site. termmeta is now one of those
tables.

See #10142.

[boonebgorges]

In 34538:

Bust term query cache when modifying term meta.

The 'last_changed' incrementor is used to invalidate the get_terms() query
cache. Since get_terms() queries may reference 'meta_query', changing term
metadata could change the results of the queries. So we invalidate the cache
on add, delete, and update.

See #10142.

[boonebgorges]

Marking this one resolved. Let's put new issues into a separate ticket. *GO TEAM WORDPRESS*

[boonebgorges]

In 34811:

Don't prime term meta cache in is_object_in_term().

Term meta is not necessary in this case, so priming the cache wastes a query.

See #10142, #32044.