Keep the connection open when doing upgrades or long-running operations

[markjaquith]

When we do long-running operations like upgrades-over-FTP, we may go a while without sending any data, which may cause the connection to be closed. We should investigate whether we can send some "hey, still here" dummy data down the pipe to keep the connection from being unceremoniously closed on us.

For example, Rackspace Cloud Sites runs behind load balancers that cut the connection after 30 seconds of no data. I was able to defeat it by manually flushing some dummy data like so:

<?php

for ( $i = 1; $i < 46; $i++ ) {
	echo "$i...<br />\r\n";
	flush();
	sleep( 1 );
}
echo "DONE";

[Denis-de-Bernardy]

see #10522, #10407, #11588 and #14049

[hubertnguyen]

Small note: the "Smush.it bulk" process does it really well. I tried on RackSpace Cloudsites, and it never times out, I left the process run for more than 45mn. Hope that Wordpress Core and plug-in update will never time out in the near future :)

/wp-admin/upload.php?page=wp-smushit-bulk

[hubertnguyen]

Replying to Denis-de-Bernardy:

see #10522, #10407, #11588 and #14049

Denis, I think that while #17301 may help with #14049, both are unrelated.

[dd32]

My initial thoughts on this were that Long-running functions (such as copy_dir() and unzip_file()) could execute a action every iteration, such as upgrader_tick.
A function hooked to that could then echo <!-- Some data --> every x seconds, If we had a method which left the FTP connection idle for a decent amount of time, we could also send a FTP NOOP command every x seconds.

Further to the last point, a FTP NOOP command would be good during long-running HTTP downloads when doing bulk upgrades, to keep the connection open.

[pento]

attachment:17301.diff​​ adds a keepalive tick to some long functions, as suggested by dd32.

[nacin]

Leaving this open for dd32.

[nacin]

Per conversation with dd32.

[hubertnguyen]

Since I originally had this issue with Cloud Sites, I wanted to update this thread. The reason why it was slow is because the updates were done via FTP, with the FTP user/pass defined in wp-config.php

What happened is that every single file IO was transiting through the public web because Cloud Site's FTP isn't located on a local machine (even if it was, I'm not sure it'd be faster).

Using define('FS_METHOD','direct') in wp-config.php, the problem goes away, but the update process is much more likely to fail because of permission issues. Plug-ins or WP core files uploaded via FTP may not have the proper group write permission. That's not an issue (for me) via FTP, but it is an issue when PHP wants to write the files directly.

Anyway, the bottom-line is: people will slow FTP uploads can use the direct method, at the risk of running into permission issues.

Regardless, it would be nice if things would never time out until completion of the update.

[pbiron]

refreshing patch

[pbiron]

17301.1.diff​ refreshes the patch so that it cleanly applies now. It also adds a DocBlock for connection_keepalive() (although the function description could certainly be improved :-)

[francina]

Hello, this ticket came up during a scrub. In order to move forward, please provide more information to help testers manually test the patch (including at Test Scrubs or Component Scrubs):

• What are the steps to reproduce?
• What are the steps to test?
• Are there any testing dependencies, such as a plugin or script?
• What is the expected behavior after applying the patch?

Thank you