FTP Extension-based filesystem fails on WP core upgrade/re-install

[Denis-de-Bernardy]

I've a bit out of ideas as to why it was failing (we ended up using the FTP Socket filesystem), but...

It failed in the middle of the upgrade, always at the same file (wp-blog-header.php). Some investigation revealed there were no permission problems of any kind. This leads me to suspect the FTP connection was basically lost in the middle of the upgrade.

The reason I think so is I had to work around a similar problem in some old core upgrade scripts that were based off of the Automatic WordPress Upgrades plugin (or whatever it was called). If my memory serves me well I ended up periodically disconnecting and reconnecting (like... every 30s).

Are we keeping our FTP connection alive in any way when using that transport? If not, we probably should. And a similar reasoning might be in order for the PHP HTTP Extension.

[dd32]

I'm not sure its timing out from inactivity, from what I've seen, happens half way through the process while theres heavy IO operations..

could be that the tunnel is disconnecting when it hits certain filenames though... (i'm thinking of the multiple dashes for some reason..)

[Denis-de-Bernardy]

I'm trying this fix atm: ftp_set_option($wp_filesystem->link, FTP_TIMEOUT_SEC, 600)

will report back when users who were affected try it.

[Denis-de-Bernardy]

I'm sure it's *not* timing out from inactivity, too. In my case, it's trying to upload 7M worth of zipped files... :-)

[dd32]

I'm sure it's *not* timing out from inactivity, too.

Which is why i was questioning, Surely it cant be a timeout.. FTP doesnt just timeout while you're transfering files..

There are a few sets of code which could do with a "check if its still alive, and if not, reconnect" however. ie. before the unziping. or before the copy_dir().

[Denis-de-Bernardy]

Replying to dd32:

I'm sure it's *not* timing out from inactivity, too.

Which is why i was questioning, Surely it cant be a timeout.. FTP doesnt just timeout while you're transfering files..

Actually (based on what I've been seeing anyway), it does. :-(

It transfers files, and then poof, gone. Reconnect and it works again. Still awaiting feedback from the user. I added this in a plugin, on the update_feedback hook, in case it makes any difference:

	function update_feedback($in = null) {
		static $done = false;
		global $wp_filesystem;
		
		if ( $done || !$_POST || !is_object($wp_filesystem) )
			return $in;
		
		if ( is_a($wp_filesystem, 'WP_Filesystem_FTPext') && $wp_filesystem->link ) {
			if ( @ftp_get_option($wp_filesystem->link, FTP_TIMEOUT_SEC) < 300 )
				@ftp_set_option($wp_filesystem->link, FTP_TIMEOUT_SEC, 600);
			$done = true;
		}
		
		return $in;
	} # update_feedback()

[dd32]

instead of using statics, a remove_filter is simpler usually :)

Thinking a bit more, Could it be Active vs. Passive? ..But then i think the FTPExt is set to passive mode only..

Or maybe, when the PHP Docs refer to "Network timeout" it could also refer to the timeout for each operation, ie. Its taking > timeout settings to transfer the file..

[Denis-de-Bernardy]

Replying to dd32:

instead of using statics, a remove_filter is simpler usually :)

It's actually not. It creates bugs in apply_filters() and do_action(), which were closed (as wontfix, for good reasons) when I reported them.

Thinking a bit more, Could it be Active vs. Passive? ..But then i think the FTPExt is set to passive mode only..

Best I'm understanding the code, both are passive always.

Or maybe, when the PHP Docs refer to "Network timeout" it could also refer to the timeout for each operation, ie. Its taking > timeout settings to transfer the file..

That's the thing. The docs make it sound as if it's for the connection (i.e. idle for X seconds = timeout), but based on the workaround code I had to use when I was maintaining my own FTP filesystem class (i.e. disconnect/reconnect after so many files), it actually is the entire sessions' timeout on a lot of platforms.

[dd32]

It's actually not. It creates bugs in apply_filters() and do_action(), which were closed (as wontfix, for good reasons) when I reported them.

Completely different scenario's. remove_filter will work in that case, Its only an issue when removing functions hooked to the current filter that is not the current function. (ie. As long as you're not attempting to remove a function that is going to be fired on THIS iteration of the stack)

but based on the workaround code I had to use when I was maintaining my own FTP filesystem class (i.e. disconnect/reconnect after so many files), it actually is the entire sessions' timeout on a lot of platforms.

Which sounds entirely possible, Just need to find a value which is going to work for all of core.. I know FTP takes sub 5 seconds in total on my new host.. I know it takes >120seconds on others.

[Denis-de-Bernardy]

Replying to dd32:

It's actually not. It creates bugs in apply_filters() and do_action(), which were closed (as wontfix, for good reasons) when I reported them.

Completely different scenario's. remove_filter will work in that case, Its only an issue when removing functions hooked to the current filter that is not the current function. (ie. As long as you're not attempting to remove a function that is going to be fired on THIS iteration of the stack)

No no, it also affects later functions on the same hook in some weird edge cases. But that's not the point of this ticket, is it? :-)

but based on the workaround code I had to use when I was maintaining my own FTP filesystem class (i.e. disconnect/reconnect after so many files), it actually is the entire sessions' timeout on a lot of platforms.

Which sounds entirely possible, Just need to find a value which is going to work for all of core.. I know FTP takes sub 5 seconds in total on my new host.. I know it takes >120seconds on others.

It took well over 90s on Jeff's host. I've bumped it to 600s (as I did for the timeout) to see how things went. Will report back when he replies.

[Denis-de-Bernardy]

see also #10522

[dd32]

Closing as duplicate of #10522