Suddenly getting error on all of my servers: Unable to locate WordPress content directory (wp-content).

[jobst]

Hi

I have now been getting these update errors, I have never had problems before. It always worked using ssh2.

This happened while I was overseas (5 weeks), so I am not sure when this started - but before I went ALL of my servers (CentOS 6.8 with php 5.6, all latest patches) where still able to update.

Suddenly I started getting "could not update wordpress" as I have automatic updates setup, now I can update some machines, some I cannot.

I thought I try one of the servers (the development server) with latest php 7.1, maybe that gets rid of that problem - it did not solve the issue, so I went back to the original php install.

What is really strange I have a development server/install of one website on a different server as the main server. Both installs are identical (as in core/plugins/etc) The main server I can happily update the wordpress site using ssh2, the development server I cannot update the wordpress site anymore - the ONLY difference is the domain name. Both servers run the nearly the same software (the development machine has a few more yum packages) both OS and web-environment are the same, they even have the same php.ini file.

When I change the development server from FS_METHOD=ssh2 to FS_METHOD=direct it works (after updating the file permissions), but I do not like this as the files need to be write-able by the user running the server, with ssh2 I can have different owner and group which is way more secure.

I too have another wordpress install that I cannot update. I know that all settings/keys/permissions etc are correct as it USED to work.

Help please!

[dd32]

@jobst are you absolutely sure nothing changed on the server? I ask, as iirc there haven't been any changes in the last six months that would cause it.

The ssh extension with php7 is known to be faulty in early versions of it though, later releases fixed it, but would cause such a error.

Feel free to dm me on slack @dd32 if you'd like help to debug it, there's not enough context available in your report to identify any specific cause.

[jobst]

@dd32 "nothing" is a probably wrong as I update all my servers frequently to get the latest security patches for all software installed - "Nothing" should mean "I have not installed anything new/else".

The thing is the server originally had php 5.6.29 on it - so I thought maybe this is the problem so I updated it to php 7.1.0, that made no difference.

This is a CentOS 6.8, so the php install is not from Redhat but rather webtatic. However, I have been using webtatic for years and has always been reliable.

Here are some other important items:

libssh2 1.4.2-2
pecl ssh2 0.13 stable

I re-installed the pecl ssh2, made no difference.

I, too, am back to php 5.6.29 (as all my machines are on that, makes development easier).

I can try to debug, but I would need a start where to begin - I haven't got much time in the moment.

As for slack - I am new to that and have not been using it. Not sure how to contact you at slack.

[jobst]

My error log has the following

[01-Feb-2017 03:53:41 UTC] PHP Warning:  file_put_contents(ssh2.sftp://Resource id #834/.maintenance): failed to open stream: operation failed in {WP_INSTALL}/wp-admin/includes/class-wp-filesystem-ssh2.php on line 259

which refers to this line

$ret = file_put_contents( $this->sftp_path( $file ), $contents );

in function "put_contents()".

This means, it actually gets a fair way into the code, it also gets pass the following line in "fs_connect()" in file "class_wp_upgrader.php"

if ( ! WP_Filesystem( $credentials, $directories[0], $allow_relaxed_file_ownership ) ) {

but fails here

case WP_CONTENT_DIR:
  if ( ! $wp_filesystem->wp_content_dir() )
    return new WP_Error('fs_no_content_dir', $this->strings['fs_no_content_dir']);

Now dumping the structure of "$wp_filesystem" does not show any path relating to "wp-content" ... here stops my skill a bit as I don't have a clue (yet) about the inheritance of the WP_Filesystem_Base class and where it could possibly find this.

[jobst]

Ok, I know more.

My confusion and missing info is related to 6 weeks of holiday. I now remember that I upgraded the development server to 5.6.29 (to test php) while all the other servers are still on 5.6.22. I remember that I had to help one of my other sysadmins to get setup for me to disappear and forgot all about that I upgraded the testing/development server (which holds another small/unimportant wordpress install).

The problem is everything later than 5.6.29 has this problem, it could also be a little earlier.

I cannot say where the problem originates as I do not have enough understanding of php, libssh2 and pecl ssh to choose a chair to sit on.

I only know it stopped working and I have proof is works for 5.6.22.
Also as soon as I made the development server 5.6.22 it worked again.

I am happy to help debugging, but I need some pointers.

[dd32]

Thanks @jobst, this could be something similar to #35517.

I don't think that this is anything to do with WordPress and is more likely that the libssh build you're using isn't compatible with either php or your ssh server.
This probably needs someone to duplicate it outside of WordPress and reported upstream to their bug tracker.

[jobst]

So is it the libssh2 library or the php pecl module?

[jobst]

I changed my class-wp-filesystem-ssh2.php as provided in #35517 and #37942.
I got further as some of the obstacles were the file testing (as in exist, size etc).
I got most of it fixed with using @ssh_sftp_* based functions, for example

public function exists($file) {
 // return file_exists( $this->sftp_path( $file ) );
 // jobst changed
 $stat = @ssh2_sftp_stat( $this->sftp_link, $file );
 if ( false === $stat ) {
  return false;
 }
 return (true);
}

But now it fails at (which is a harder one to fix)

public function put_contents($file, $contents, $mode = false ) {
 $ret = file_put_contents( $this->sftp_path( $file ), $contents );

It is getting this path passed (note that this has 0770 set as it is the upgrade dir but still fails)

(ssh2.sftp://Resource id #807/{WP_INSTALL}/wp-content/upgrade/download-now-for-woocommerce-U6lFyT/download-now-for-woocommerce/includes/somdn-meta.php

So I tried to re-write it:

    $fp = fopen( $this->sftp_path( $file ), "w");
    for ($written = 0; $written < strlen($contents); $written += $fwrite) {
      $fwrite = fwrite($fp, substr($contents, $written));
      if ($fwrite === false) {
        return $written;
      }
    }
    fclose($fp);

It fails at fopen, the way it is written it is really hard to do this with using ssh.sftp.* functions directly.

So here is my dilemma:

• I sit in the middle of a debate who has broken what
• who should fix it
• not many people are using it
• IMHO the whole thing should be written in a way NOT using file_put_content and file_get_content but ssh.sftp.* functions - then there would not be any problems.

Sorry, I am grumpy.

[jobst]

Replying to dd32:

Thanks @jobst, this could be something similar to #35517.

I don't think that this is anything to do with WordPress and is more likely that the libssh build you're using isn't compatible with either php or your ssh server.
This probably needs someone to duplicate it outside of WordPress and reported upstream to their bug tracker.

IMHO it is both Wordpress and pecl networking ssh2, and it is fixable.

I basically followed the instructions as laid out in these two links and it works for me:

• https://core.trac.wordpress.org/attachment/ticket/35517/class-wp-filesystem-ssh2.php.patch
• ​https://github.com/php/pecl-networking-ssh2/commit/17680cf039f0cfac53b5a2531fdb715b95e9cc42

My take on Wordpress and it's IMHO:
If you look at the pecl resource above it shows that after after php 7.0.28 and php 5.6.28 there is a need for "clean id" for function calls when using ssh2.sftp://, not an object as in "Resource #447" but just "447".

I have tried to implement this in "class-wp-filesystem-ssh2" like so:

 public function sftp_path( $path ) {
    error_log(__FUNCTION__.": LINK >".$this->sftp_link."< PATH >$path<");
	if ( '/' === $path ) {
		$path = '/./';
    }
    // not working
    $RetVal=print_r($this->sftp_link,true);
    error_log("RetVal >$RetVal<");
    $pos=strpos($RetVal,"Resource id",0);
    error_log("POS >".$pos."<");
    if($pos!==false)
    {
      error_log("YEP, object found");
      $RetVal = str_replace("Resource id #","",$RetVal);
      $RetVal = "ssh2.sftp://" . $RetVal . "/" . ltrim( $path, "/");
      error_log("RetVal >$RetVal<");
      return $RetVal;
    }
    else
    {
      // else
      error_log("NO, not object");
      return 'ssh2.sftp://' . $this->sftp_link . '/' . ltrim( $path, '/' );
    }
 }

but I could not get this to work :-(, it made no change.

So when I look at the code in "class-wp-filesystem-ssh2.php" there are already many functions used from the ssh2_sftp library (i.e. ssh2_sftp_realpath), so why not using one extra one called "ssh2_sftp_stat()" to return values for "exists(), is_file(), is_dir(), is-readable(), atime(), mtime()"? This way one can be sure that the values returned are correct! So I followed the instructions (35517) and changed my file and it works!

My take on Pecl Networking ssh2:
I added some extra lines to the the file ssh2_fopen_wrappers.c from the pecl tar.gz just to see whether I was interpreting the code correctly, and yes ALL IT DOES chopping the "Resouce id#" from the string before using it:

    bcgFile=fopen("/var/log/openwrapper.log", "a");
    if(bcgFile==NULL) {
		perror("Error opening file.");
    }

    /* Starting with 5.6.28, 7.0.13 need to be clean, else php_url_parse will fail */
    char *tmp = estrdup(path);

    fprintf(bcgFile, "BEFORE %s BEFORE\n", tmp);
    strncpy(tmp + (h-path), h + sizeof("Resource id #")-1, strlen(tmp)-sizeof("Resource id #"));
    fprintf(bcgFile, " AFTER %s AFTER", tmp);

    resource = php_url_parse(tmp);
    efree(tmp);

    fclose(bcgFile);

I can only say it works, I tried it on a number of installs, no problem.

[dd32]

Replying to jobst:

IMHO it is both Wordpress and pecl networking ssh2, and it is fixable.

I basically followed the instructions as laid out in these two links and it works for me:

• https://core.trac.wordpress.org/attachment/ticket/35517/class-wp-filesystem-ssh2.php.patch
• ​https://github.com/php/pecl-networking-ssh2/commit/17680cf039f0cfac53b5a2531fdb715b95e9cc42

Thanks for confirming it's a conflict between the SSH2 extension and more recent versions of PHP - It looks like an extension update is required for compatibility with PHP - I still don't think we should be working around this though.

[jobst]

Replying to dd32:

Thanks for confirming it's a conflict between the SSH2 extension and more recent versions of PHP - It looks like an extension update is required for compatibility with PHP - I still don't think we should be working around this though.

Why not?
Please explain.

If you look at the code of "class-wp-filesystem-ssh2.php" a mix of functions are called:

• PHP inbuilt functions, i.e. function exists() calls PHP's file_exists()
• SSH2 library functions, i.e. function cwd() calls PHP's ssh2_sftp_realpath()

That is inconsistent.
If there are any problems with any of the non ssh2 PHP functions that are not part of the ssh2 library there will be trouble ...

So why not using ALL functions from PHP ssh2 to circumvent the problems we currently have?
All I did in my "class-wp-filesystem-ssh2.php" file is using PHP ssh2 functions, for example I replaced the call to file_exists() with ssh2_sftp_stat() and it works, 100%.

ssh2_sftp_stat() does the same as file_exists() but has no problems, so what is wrong using it and why do you hesitate to change it?

All that I am suggesting stick to the ssh2 library doing the jobs required instead of using other functions.

It works, I and many other people can proof that, happily.