Add some initial PHPUnit tests for `WP_Filesystem_Direct`

[costdev]

This is a tests-only ticket.

WP_Filesystem_Direct is, by far, the most used filesystem abstraction class.

While some PHPUnit tests exist for a WP_Filesystem_MockFS class, this ticket aims to add PHPUnit tests for the WP_Filesystem_Direct class itself.

The other filesystem abstraction classes, WP_Filesystem_FTPext, WP_Filesystem_ftpsockets and WP_Filesystem_SSH2, do not have PHPUnit test runner configurations. While in future, we may reuse many of the same tests across filesystem abstraction classes, this ticket specifically targets WP_Filesystem_Direct.

This adds some initial tests for the WP_Filesystem_Direct filesystem abstraction class.

These tests do not aim to cover all paths, but to add line/branch coverage where source changes are not needed to facilitate testing or resolve bugs. The intention is to make it easier to know how to hit a line/branch when trying to cover paths in future.

This also introduces the WP_Filesystem_Direct_UnitTestCase class, which contains shared properties, set_up()/tear_down(), helpers and common data providers for the tests.

To allow constants to be set without affecting other tests in the test suite, some of these tests use the @runInSeparateProcess annotation ​[Ref]. To prevent an exception for Serialization of 'Closure' is not allowed, the @preserveGlobalState disabled annotation is also used ​[Ref].

Tests exist for the following methods:

• __construct()
• get_contents()
• get_contents_array()
• put_contents()
• cwd()
• chdir()
• chgrp()
• chmod()
• chown()
• getchmod()
• copy()
• move()
• delete()
• exists()
• is_file()
• is_dir()
• is_readable()
• is_writable()
• atime()
• mtime()
• size()
• touch()
• mkdir()
• rmdir()
• dirlist()

Trac ticket: https://core.trac.wordpress.org/ticket/57774

[costdev]

​PR 4106 is ready for review.

Line coverage: 74.56%
Branch coverage: 76.83%

To cover more lines/branches, source changes are needed to facilitate testing/fix bugs. This will be done in separate tickets.

[hellofromTonya]

Self-assigning for review and commit consideration.

[costdev]

This ticket was discussed in the bug scrub.

While this is a tests-only ticket and can be committed at any time, we're approaching RC1 and the PR is awaiting review, so I'm going to move this ticket to the 6.3 milestone.

Should it be reviewed and ready for committed prior to 6.2 RC1, feel free to bring it back into the 6.2 milestone for commit.

Additional props: @mukesh27

[oglekler]

@hellofromTonya, please take a look at this one 🙏

[hellofromTonya]

Moving into the 6.4 milestone.

@costdev sorry for the big delay. I'll get your PR reviewed and then committed.

[oglekler]

The patch needs to be refreshed, most likely the part in phpcs.xml.dist is not needed anymore, see:
​https://github.com/WordPress/wordpress-develop/commit/60d2d8ac75c4350a45e4c6970c7d0295b757f1f0

Great job, btw. Let's try to make it into the current milestone :)

[costdev]

Thanks @oglekler!

​PR 4106 has now been rebased on trunk and the conflict with phpcs.xml.dist resolved.

[oglekler]

This ticket was discussed during bug scrub.

@hellofromTonya can you please review this PR?

Add props to @mukesh27

[hellofromTonya]

As this ticket is for adding automated tests, it is not bound to the 6.4 release milestones and can be committed at any time before the 6.4 Dry Cycle starts.

[hellofromTonya]

To avoid confusion for 6.4 triagers during the beta cycle, I changed the component to Build/Test as this ticket tracks only tests.

[hellofromTonya]

While this ticket is not bound to a major release's schedule, tomorrow RC1 ships and trunk will be branched to open it for 6.5-alpha. Thus, given the short time remaining in 6.4, moving this to 6.5.

What's left? Code review and then commit. If that can get done during 6.4 RC cycle, could consider backporting the tests to the 6.4-branch.

[hellofromTonya]

Hey @costdev, we previously chatted about the patch's readiness. Fast wind to today, is it ready for code and commit review?

[costdev]

Hey @hellofromTonya! Yes I believe the PR should be ready for code and commit review.

[swissspidy]

In 57753:

Build/Test Tools: Add initial tests for the WP_Filesystem_Direct class.

Since WP_Filesystem_Direct is by far the most used filesystem abstraction class, this facilitates future changes with sufficient test coverage.

Props swissspidy, costdev, mukesh27.
Fixes #57774.

Merged in https://core.trac.wordpress.org/changeset/57753

[peterwilsoncc]

I've noticed these tests are causing a bunch of failures on most of the ​hosting providers running the test suite. Common issues appear to be disallowed operations and attempting to remove directories that are not empty.

Unfortunately there's no way of testing PRs against the hosting providers so it wasn't possible to preempt the issue but it's probably worth reverting the change for now as it's late in the release cycle and coming back to this in the 6.6 release cycle.

[swissspidy]

Thanks @peterwilsoncc, I was actually about to reopen or open a new one myself as we‘re actually working on the hosting tests this weekend and noticed the failures.

Not sure about reverting as they‘re just tests & not functionality, and also already in the 6.5 branch I think.

Would be good to first analyze the common failures a bit more to find the root cause. I‘d imagine we could also just skip the whole tests if lacking permissions for example.

[peterwilsoncc]

@swissspidy I've asked in the ​#hosting slack channel whether the hosting tests always run on trunk or on the branch for backport commits.

If they always run on trunk then I agree that there's not much point in reverting. If they run on the backport branch then it might be worth reverting from the 6.5 branch so the reports for backports are at least clear.

I‘d imagine we could also just skip the whole tests if lacking permissions for example.

I think that works.

[costdev]

I agree that we should establish the exact cause of the errors first.

For example, maybe the test runner isn't running with FS_METHOD set to direct.

So far, I see "Operation not permitted" for chown/chgrp(), and "Directory not empty" for rmdir().

The first seems clear as a permissions issue, but the error occurs in the actual class file, not the test file. That points to a legitimate error when trying to use the direct class in that environment.

For the "Directory not empty" failure, while I could theorise about possible causes, I'd be very curious to see further details specifically from those environments.

Trac ticket: https://core.trac.wordpress.org/ticket/57774

[swissspidy]

I was able to reproduce the issues locally as well.

Operation not permitted

The issue seems to be the line:

$expected_group = $this->is_windows() ? $original_owner : $original_owner + 1;

$original_owner is the result of fileowner(), which is the user ID of the current owner).

On my system there is no such user with ID+1, hence the operation is not permitted.

I think that's an easy thing to solve, just don't use another user ID.

Directory not empty

The tests seem to pass when run in isolation, so it's probably some missing cleanup elsewhere.

In my quick testing, fixing the Operation not permitted issue actually seemed to solve these ones here. So two birds with one stone.

Here's a PR: ​https://github.com/WordPress/wordpress-develop/pull/6286

If it passes we can commit this and then see how the hosting tests are afterwards.

[swissspidy]

There's a third issue I just noticed:

Tests_Filesystem_WpFilesystemDirect_Mkdir::test_should_set_chmod
The permissions are incorrect.
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'0644'
+'0755'

/home/wptestrunner/site/public_html/wp-test-runner/tests/phpunit/tests/filesystem/wpFilesystemDirect/mkdir.php:133

See ​https://make.wordpress.org/hosting/test-results/r57848/wpmudevbot-r57848/

Maybe that host sets FS_CHMOD_DIR to something custom?

After ​my PR, this test is the last one with an is_windows() check. Would be good if we could get rid of this env-specific check and maybe rely on comparing with FS_CHMOD_DIR instead.

[swissspidy]

In 57849:

Build/Test Tools: Make WP_Filesystem_Direct tests more robust.

Follow-up to [57753] to make tests more robust, as there were multiple permission-related errors in the hosting test results.
With this change, the tests now don’t try setting an owner that doesn’t exist.

Props peterwilsoncc, costdev, javiercasares.
See #57774.

Committed in https://core.trac.wordpress.org/changeset/57849

[swissspidy]

Since we're wrapping up 6.5 and most of the issues here have been resolved, I've opened #60817 to further investigate this one last test that fails on 1 host only.