Add AND query mode for taxonomy terms

[sebbb]

I'm trying to query the Rest API to get only posts that contain a given set of tags. If, for example, I request the following from the Rest API:

/wp-json/wp/v2/posts?tags=41+50

I get posts that don't satisfy all the tags. For example, the first post returned in the JSON response has the tags 45, 38, 46, 42, 35, 50, 43, 36, 37, 75, 76. The post doesn't have the tag with ID 41, while the expected output would be only posts that satisfy all the provided tags.

It looks like behind the scenes the API uses an OR logic instead of the correct AND logic. It returns posts that contain at least one of the provided tags. I'm experiencing the same exact problem with categories as well.

I tried different versions of the above URL, but I'm facing the same issue:

/wp-json/wp/v2/posts?tags=41&tags=50

/wp-json/wp/v2/posts?tags=41,50

Interestingly, tags_exclude seems to work perfectly and the following url correctly returns posts that have neither the tag ID 41 nor the tag ID 50:

/wp-json/wp/v2/posts?tags_exclude=41+50

[kadamwhite]

It looks like behind the scenes the API uses an OR logic instead of the correct AND logic.

Unfortunately this is exactly what is happening; the API does not currently support AND logic for querying records by taxonomy ID. We tracked this problem in our old ​GitHub repository as issue # 2768, but it doesn't look like that issue has made it here into trac -- thanks for opening the ticket.

We should have a way to query posts by taxonomy terms using an AND relation. I agree that the current behavior is highly unintuitive.

@rmccue may be able to clarify but I'd also comment that, as far as I understand it, id1+id2 or id1,id2 are both just proxies for the ?tags=id1&tags=id2 syntax, so there isn't a clear way to distinguish between AND or OR without introducing a new API query parameter.

[LewisCowles]

Is there a way for the existing OR functionality / bug to be provided via API endpoint or parameter to the existing endpoint to toggle between AND / OR?

I'm thinking it could be helpful for certain situations (find all posts in EITHER taxonomy X or Y, rather than find all posts in ONLY taxonomy X and Y).

Also does it only affect tags?

[sebbb]

Replying to LewisCowles:

Also does it only affect tags?

It seems to be affecting categories as well.

[LewisCowles]

Thanks. The reason I ask is that I'm going to borrow that bugged code. For a while now I've been thinking about provisioning a way for users to de-select taxonomy data to hide things they don't want. (Imagine if you could choose to hide unwanted political commentary).

It would require some work on author and editor part, but it's the first time I've seen something pre-built and will save me a few rainy days.

[jnylen0]

This is intended behavior, not a bug. I would expect "filter the posts list to posts that include tags 41,50" to return all posts that match any of these tags. "exclude tags 41,50" should not match any of those posts. If tags used AND, but exclude_tags used OR, this would be less consistent.

Having said that, adding an AND query mode would be a nice enhancement. The solution will likely look similar to the approach discussed in #39494.

For now, it should be possible to use the rest_{$this->post_type}_query ​filter -- possibly combined with register_rest_field (​docs) -- to make this functionality work in a plugin.

[sebbb]

It's too bad that this is not considered a bug 😢.

I would think that the alternate syntax using + (i.e.: tags=41+50) implies that only the posts that have both tag 41 and tag 50 should be returned.

I'm working on an app that needs that feature, so I'll certainly look into a DIY solution in the meantime.

Unfortunately I don't have any PHP, WordPress or API design background myself, so I wouldn't be able to create the patch.

[dd32]

#41908 was marked as a duplicate.

[earnjam]

In 41287.diff​, I've added support for a new {$tax}_and parameter which queries using an AND relationship between terms. This follows the model of the existing parameter {$tax}_exclude for a NOT IN type query.

I also considered something like {$tax}_strict for the parameter name, but went with _and because it more closely aligns with existing WP_Query arguments for tag__and and category__and

Also, it may be beyond the scope of this ticket, in which case I can split it off into a new ticket, but I also added a new parameter, relation, which allows you to specify an OR relationship between multiple taxonomies being queried. This more closely mimics the functionality for a tax_query argument on WP_Query.

So basically this is the current behavior:

URL	Tax Query Matches
?tags=1,2	tag 1 OR tag 2
?tags=1&categories=3	tag 1 AND category 3
?tags=1,2&categories=3	(tag 1 OR tag 2) AND category 3
?tags=1,2&categories=3,4	(tag 1 OR tag 2) AND (category 3 OR category 4)

With this patch, you can use a {$tax}_and type parameter to force AND matching on the terms within a specific taxonomy.

URL	Tax Query Matches
?tags_and=1,2	tag 1 AND tag 2
?tags_and=1,2&categories=3	tag 1 AND tag 2 AND category 3
?tags_and=1,2&categories_and=3,4	tag 1 AND tag 2 AND category 3 AND category 4

You can also use the relation parameter to allow you to make OR queries across multiple taxonomies.

URL	Tax Query Matches
?tags=1&categories=3&relation=OR	tag 1 OR category 3
?tags=1,2&categories=3&relation=OR	tag 1 OR tag 2 OR category 3
?tags=1,2&categories=3,4&relation=OR	tag 1 OR tag 2 OR category 3 OR category 4

Or you can combine the two together to make very complex tax queries.

URL	Tax Query Matches
?tags_and=1,2&tags=3&relation=OR	(tag 1 AND tag 2) OR tag 3
?tags_and=1,2&categories=3&relation=OR	(tag 1 AND tag 2) OR category 3
?tags_and=1,2&categories_and=3,4&relation=OR	(tag 1 AND tag 2) OR (category 3 AND category 4)
?tags_and=1,2&categories_exclude=3,4&relation=OR	(tag 1 AND tag 2) OR ( NOT category 3 OR category4 )

You can even use the relation parameter for queries within a single taxonomy that use multiple types of operators:

URL	Tax Query Matches
?tags_and=1,2&tags=3,4&relation=OR	(tag 1 AND tag 2) OR tag 3 OR tag 4
?tags_and=1,2&tags_exclude=2,3&relation=OR	(tag 1 AND tag 2) OR ( NOT tag 2 OR tag 3 )

I added several additional tests, but with so many new query combination possibilities (it goes from 4 to 18 possible combinations when querying 2 taxonomies), it might help to have a bit more coverage.

[earnjam]

I split out the relation parameter code and tests into #44326 because in hindsight it was clear it needed to be it's own ticket.

41287.2.diff​ just has the code and tests for {$tax}_and parameter support mentioned above.

[Clorith]

Just had someone ask about this, which is how I stumbled in here.

To me it makes sense to use 1+2 to get an AND effect, where 1,2 is the OR equivalent. This is how we do it everywhere else, so it makes sense to stick with the established format for, changing it seems counter intuitive to me, unless I'm missing something else here?

[nando99]

If I'm reading this right,&categories_and should work... but its not? Will it only work with tags_and?

[earnjam]

@nando99 It should work with any taxonomy in the patch I uploaded, but that was 9 months ago, so it may need a refresh. Core does not currently support AND type queries within any single taxonomy.

@Clorith 1+2 format currently works the same way as 1,2, so changing that would break backwards compatibility.

[TimothyBlynJacobs]

I think this approach makes sense, though I'm not a huge fan of the name. Would somethings like all_tags make sense? Or am I misunderstanding what this actually does?

[TimothyBlynJacobs]

5.7 introduces support for an object syntax for term queries to allow for specifying include_children. See #39494. We could extend syntax to have an operator field.

@earnjam would you be interested in working on this for 5.8?

[TimothyBlynJacobs]

#52468 was marked as a duplicate.

[stefanolsenn]

Any news on this, or workarounds? Any help is appreciated!

[TimothyBlynJacobs]

@stefanolsenn we're going to try and take a crack at this for 5.8. Assigning to @dlh.

[helen]

@TimothyBlynJacobs @dlh Have you had a chance to take a look at this? With the bugs-only beta period starting May 25 this needs to land very soon if it's going to be for 5.8.

[dlh]

I'm aiming to submit a PR for this tomorrow.

[dlh]

PR submitted: ​https://github.com/WordPress/wordpress-develop/pull/1249

The patch uses an enum of AND/OR, rather than AND/IN as used by WP_Tax_Query, for general readability and consistency with tax_relation, but that's easy to change if desired.

The patch doesn't include EXISTS in the enum only as it wasn't requested here, but I suppose that it could.

I'm open to feedback about those choices or any others in the PR.

[TimothyBlynJacobs]

This looks great to me, thanks @dlh! The only thing I think we should change is to lowercase the AND and OR. That way it matches what we do for things like order which accepts asc instead of ASC.

[desrosj]

Spoke with @TimothyBlynJacobs, and he is going to get this committed in the next few hours for 5.8.

[TimothyBlynJacobs]

I missed that we're actually already using the uppercase version for the tax_relation, so I'm just going to ship this as is.

[TimothyBlynJacobs]

In 51026:

REST API: Add support for modifying the term relation when querying posts.

By default, a post most contain any of the requested terms to be included in the response. This commit adds a new operator property that can be set to AND to require a post to contain all of the requested terms.

For example, /wp/v2/posts?tags[terms]=1,2,3&tags[operator]=AND will return posts that have tags with the ids of 1, 2, and 3.

Props dlh, earnjam, Clorith, jnylen0, sebbb.
Fixes #41287.