WordPress.org

Make WordPress Core

Opened 9 years ago

Closed 8 years ago

#17238 closed feature request (wontfix)

Parallel HTTP Requests

Reported by: sterlo Owned by:
Milestone: Priority: normal
Severity: normal Version:
Component: HTTP API Keywords:
Focuses: Cc:
PR Number:

Description

It would be cool if we could make multiple requests via HTTP in parallel rather than having each request stack up and block.

Some references to how some others are doing it:
http://pecl.php.net/package/pecl_http Of course that is in C. :)

also: http://www.php.net/manual/en/class.httprequestpool.php

The basic premise is, use a request pool. How we would implement such a thing in WordPress, I am not sure at the moment.

If we could do this then things that need to make external requests wouldn't need to block page/asset requests. Sites would speed up and unicorns would appear.

Change History (11)

#1 follow-up: @nacin
9 years ago

You can already do non-blocking requests in WordPress.

#2 in reply to: ↑ 1 @sterlo
9 years ago

Replying to nacin:

You can already do non-blocking requests in WordPress.

Is there any sort of documentation on that? I can't seem to find anything and I spoke with sivel and it sounded like that functionality wasn't there at the moment.

#3 @sterlo
9 years ago

Seemingly related: #16978

#4 follow-up: @nacin
9 years ago

See http://codex.wordpress.org/HTTP_API#Other_Arguments. We use non-blocking for just spawning the cron I think.

#5 in reply to: ↑ 4 @sterlo
9 years ago

  • Resolution set to invalid
  • Status changed from new to closed

Replying to nacin:

See http://codex.wordpress.org/HTTP_API#Other_Arguments. We use non-blocking for just spawning the cron I think.

Thanks! I will play with it.

#6 follow-up: @dd32
9 years ago

  • Milestone Awaiting Review deleted

non-blocking requests allow the request to be made, however no output from the server is captured.

The HTTP Extension can be useful (pecl http) however was just removed from core due to it's lack of functionality offered over that of cURL. cURL itself can handle multiple requests (which is where pecl http gets it's functionality from) but that isnt exposed in WP_HTTP

The main thing which prevents this being implemented on Streams and Fsockopen is that PHP is single-threaded (Sure, there are extensions for that, but lets not go there) resulting in functions for the http class having to be called regularly to check for output.

I'm not sure of your use-case here, but due to the above reasons, I'd not suggest it be added to core, and would've closed this ticket as wontfix myself. I would probably suggest using WordPress's cron system and firing off the http requests in their own WordPress instance/thread

#7 in reply to: ↑ 6 ; follow-up: @sterlo
9 years ago

  • Resolution invalid deleted
  • Status changed from closed to reopened

Replying to dd32:

non-blocking requests allow the request to be made, however no output from the server is captured.

Yes. But with ajax this can be done with ease.

The HTTP Extension can be useful (pecl http) however was just removed from core due to it's lack of functionality offered over that of cURL. cURL itself can handle multiple requests (which is where pecl http gets it's functionality from) but that isnt exposed in WP_HTTP

Yeah I noticed that from #16978 :)

I'm not sure of your use-case here, but due to the above reasons, I'd not suggest it be added to core, and would've closed this ticket as wontfix myself. I would probably suggest using WordPress's cron system and firing off the http requests in their own WordPress instance/thread

After playing with the idea a bit:

Non-blocking requests are cool. But still not the same as parallel requests.

Parallel requests that are intended to block. So if a page has two separate sets of data to display, the requests for that data can be made at the same time, instead of in parallel. So if both requests take 500ms - then your page load speed would be 500ms instead of 1000ms.

If it's deemed to be something not worth adding to the core - that's fine. I just figured I'd give the feature some thought.

#8 @jane
9 years ago

  • Milestone set to Future Release

Past freeze, punting.

#9 @cyberhobo
8 years ago

  • Cc cyberhobo@… added

#10 in reply to: ↑ 7 @kurtpayne
8 years ago

  • Cc kpayne@… added

Replying to sterlo:

Replying to dd32:

non-blocking requests allow the request to be made, however no output from the server is captured.

Yes. But with ajax this can be done with ease.

If you mean "use ajax for parallel requests" then that introduces new problems. Ajax has a security model that restricts requests to the same domain only.

If you mean "use ajax and proxy connections through wordpress' WP_HTTP classes" then that would only work when then action was initiated from a browser. Actions initiated from crons would not have access to this. These requests would also compete with other ajax requests in the browser. There is a maximum number of simultaneous ajax requests, too, that differs per browser (I believe the average right now is 8).

As you've pointed out, curl+php does support this with curl_multi_* functions http://us.php.net/manual/en/function.curl-multi-init.php

I agree that this would be cool to have, but it's not something that is supported by all of the WP_HTTP implementations (as dd32 said). I don't think this can be done properly.

#11 @dd32
8 years ago

  • Milestone Future Release deleted
  • Resolution set to wontfix
  • Status changed from reopened to closed

For all the above, I'm closing as wontfix for now. It's technically impossible for us to implement it in a pleasing way across all installations.

We can revisit in a few years when things may have changed in PHP, For now, if a plugin needs to implement it, they can use cURL directly (With the short comings that brings on some servers) or use multiple ajax requests from the client as mentioned above, or just queue up cron jobs if the result simply needs to be processed and stored.

Note: See TracTickets for help on using tickets.