WordPress.org

Make WordPress Core

Opened 16 months ago

Last modified 7 weeks ago

#39965 assigned enhancement

REST API: not possible to request collection of posts across post types

Reported by: iseulde Owned by: flixos90
Milestone: 5.0 Priority: high
Severity: normal Version: 4.7
Component: REST API Keywords: rest-api
Focuses: Cc:

Description

See #38920 for a simple core use case.

There seems to be no way to query posts across multiple (or all) post types as you can do with WP_Query. You have to start making multiple requests, which is an inconvenient and non-optimal approach? Ideally this should be directly passed to the server and WP_Query.

Related: #39851.

Change History (30)

#1 @swissspidy
16 months ago

There are separate controllers for every post type, which definitely has its benefits. For queries across multiple post types, I would probably use batch requests. See https://developer.wordpress.org/rest-api/requests/#internal-requests.

For #38920 we could create a new endpoint that only returns the needed data without additional overhead.

#2 @iseulde
16 months ago

For that ticket it would be enough to create a separate endpoint with the needed data. Not sure if it's the best way to go in general though.

#3 follow-up: @jnylen0
11 months ago

  • Milestone changed from Awaiting Review to 4.9
  • Owner set to jnylen0
  • Status changed from new to assigned

Let's look into this for 4.9.

I'm thinking that /wp/v2/posts should support a special argument like ?any_type=true or perhaps ?type=post,page, ?type=*. There may be some difficulties due to differences in response structure (in core, posts and pages are somewhat different, and attachments are very different).

This ticket was mentioned in Slack in #core-restapi by jnylen. View the logs.


11 months ago

#5 in reply to: ↑ 3 ; follow-up: @danielbachhuber
11 months ago

Replying to jnylen0:

Let's look into this for 4.9.

I'm thinking that /wp/v2/posts should support a special argument like ?any_type=true or perhaps ?type=post,page, ?type=*. There may be some difficulties due to differences in response structure (in core, posts and pages are somewhat different, and attachments are very different).

For the record, this violates the principles of REST and I generally think it's a really bad idea.

Creating some batch GET endpoint would be another approach: https://github.com/WP-API/WP-API/issues/1506 GraphQL would be the inevitable conclusion.

#6 in reply to: ↑ 5 ; follow-up: @jnylen0
11 months ago

Replying to danielbachhuber:

For the record, this violates the principles of REST and I generally think it's a really bad idea.

While it's true that /wp/v2/posts currently means "posts with post_type of post", at least a post of another type is still a post.

Creating some batch GET endpoint would be another approach: https://github.com/WP-API/WP-API/issues/1506 GraphQL would be the inevitable conclusion.

This is a far larger task, and our WP.com batch endpoint has caused us a lot of very strange issues over the years. Some of them are specific to the setup there; some would eventually reoccur in multisite context with site and theme switching. I think it would ultimately be a pretty bad idea to ship this in core.

How about a separate endpoint like /wp/v2/posts/all_types?

#7 in reply to: ↑ 6 @danielbachhuber
11 months ago

Replying to jnylen0:

While it's true that /wp/v2/posts currently means "posts with post_type of post", at least a post of another type is still a post.

This is an incorrect assumption that will likely lead to negative downstream implications. In the context of the database schema, yes different content types are stored in the same table (and history shows some of the ramifications of that).

In the context of the REST API data model, not all posts are alike — in fact, they can be more different than similar.

Is a nav_item a type of post? Is a WooCommerce product a type of post? Both are shaped drastically different than a standard WordPress post. Any client code trying to handle arbitrarily shaped data is going to be a mess.

Furthermore, I believe trying to fit a type argument into WP_Posts_Controller would be a pretty substantial monkey-patch development job. Currently, a request to /wp/v2/posts follows a different code path than a request to /wp/v2/pages. You'd need to hack around the PHP architecture to get the controllers to behave consistently.

And so on.

How about a separate endpoint like /wp/v2/posts/all_types?

This seems like a hack to me. If there's one specific feature of Gutenberg for this use-case, then I'd suggest Gutenberg handle this entirely and leave the API as it is.

#8 follow-up: @jnylen0
11 months ago

This is for the feature of the link dialog that allows linking to a post or page. See #38920 for the core ticket to update this to use the REST API.

For reference, here is the code that currently drives this Ajax action: https://core.trac.wordpress.org/browser/tags/4.8/src/wp-includes/class-wp-editor.php#L1558

Thanks for explaining your concerns; I agree this isn't the best fit for the existing wp/v2 endpoints. However, shifting the responsibility to the client to batch requests isn't a great solution either, and this feature request has come up several times in other contexts, though I don't have a link at the moment.

We could write a minimal endpoint to provide this functionality and ship it in Gutenberg first.

Last edited 11 months ago by jnylen0 (previous) (diff)

#9 in reply to: ↑ 8 @danielbachhuber
11 months ago

For the purposes of the link dialog, I think you could make two separate requests (Posts and Pages), combine the results, and call that good enough for now. Given how much backwards compatibility Gutenberg is creating, I don't see it as entirely necessary to spend much time solving the problem for this particular use case.

#10 @iseulde
11 months ago

Starting to think it might be worth to create a separate search endpoint for things like this, as suggested earlier.

For #38920 we could create a new endpoint that only returns the needed data without additional overhead.

It would only return the common properties such as link, title, date and maybe a summary.

#11 @iseulde
11 months ago

(Because I think in the case of linking, you actually do want to search all non internal post types, such as WooCommerce products etc.)

#12 @afercia
11 months ago

  • Keywords rest-api added

This ticket was mentioned in Slack in #core-editor by aduth. View the logs.


10 months ago

This ticket was mentioned in Slack in #core by aduth. View the logs.


10 months ago

This ticket was mentioned in Slack in #core by jnylen. View the logs.


10 months ago

#16 @joehoyle
10 months ago

I think per @iseulde's point, a search endpoint would be best here. As @danielbachhuber said, having /wp/posts or similar return different types seems like a really bad idea, it totally breaks out controller - per - type model, schema for endpoint etc.

I'm still not sure why a /batch wouldn't suffice here though?

#17 @rmccue
10 months ago

I am much more in favour of a search endpoint which could search across different types of entities, and I think that's probably a good endpoint to have for both backend (link dialogue, etc) and frontend (site search, autocomplete, etc) usage.

Potentially, we could add an additional context (say, context=internal) to a search endpoint to allow searching types which don't have show_in_rest, but we would want to very carefully consider how this works and the use cases for it. I think a better idea is simply to require show_in_rest.

A generic /batch could also work, but I think a targeted search endpoint is a better fit, and it also avoids the problem of boiling the ocean (by trying to solve one use case with a very generic fix). This also fits nicer with things like pagination, sorting, etc.

This ticket was mentioned in Slack in #core-editor by youknowriad. View the logs.


10 months ago

This ticket was mentioned in Slack in #core-restapi by kadamwhite. View the logs.


9 months ago

This ticket was mentioned in Slack in #core-restapi by kadamwhite. View the logs.


9 months ago

#21 @kadamwhite
9 months ago

  • Milestone changed from 4.9 to Future Release

I think we're out of time for 4.9. I wouldn't want to rush a change like this in at the last minute; we'll try to get some movement on this early in the next cycle. Provisionally punting. @rmccue and @jnylen0 feel free to override if you disagree.

This ticket was mentioned in Slack in #core-editor by afercia. View the logs.


7 months ago

This ticket was mentioned in Slack in #core-restapi by kadamwhite. View the logs.


5 months ago

#24 @afercia
3 months ago

  • Milestone changed from Future Release to 5.0
  • Priority changed from normal to high

Discussed this issue during today's Gutenberg bug-scrub and heard there are dissenting opinions in the REST API team. I'd propose to do a quick use case example:

Say I’m writing a post and I want to promote one of my products. I want to link to my product. Are you really saying in Gutenberg I have to do that manually? Do you realize this won't be very... welcomed by users?

As I see it, and I'm pretty sure I'm not the only one, having the ability to link to any post type is paramount in a modern editor. Without this ability, Gutenberg will be pretty limited. At that point, better to switch back to AJAX :D

#25 @joehoyle
3 months ago

Say I’m writing a post and I want to promote one of my products. I want to link to my product. Are you really saying in Gutenberg I have to do that manually? Do you realize this won't be very... welcomed by users?

This is still possible in the REST API, just with multiple requests. It's up to the client on how it wants to present multiple types to the user. There's the api client (in this case gutenberg) that can do things for the user, like send multiple requests. Different post types can be very different, so the client should handle these as such. For example, don't assume you can just get "all posts across all types" and then display them. What if a post type doesn't support titles, etc. Different post types can be as different as a Tag and a Post, as far as the API is concerned. That's why the suggestion is a search endpoint (or similar) that would return multiple groupings of different objects.

#26 @danielbachhuber
8 weeks ago

Looping back on this issue, I think we need a read-only wp/v2/search endpoint that returns matching objects with title, link, and possibly excerpt.

By default, the endpoint would only return searchable posts where show_in_rest=true. However, a plugin could easily register more fields to the returned objects, use ElasticSearch to pull in a mixture of data (comments, users, etc.), and so on.

#27 @iseulde
8 weeks ago

@danielbachhuber Yes, I think that's exactly what we need. :) Maybe also with a date. I like that this could be extended beyond posts to include e.g. category or author archives.

This ticket was mentioned in Slack in #core-restapi by kadamwhite. View the logs.


8 weeks ago

#29 @kadamwhite
8 weeks ago

  • Owner changed from jnylen0 to flixos90

#30 @flixos90
7 weeks ago

An initial implementation is available at https://github.com/WordPress/gutenberg/pull/6489. I'll write tests for it later this week.

Note: See TracTickets for help on using tickets.