'import' group tests fail using git repository

[enrico.sorcinelli]

Recently I switched to git for contributing to WordPress core. And of course, for convenience, I run tests under git tree.

In this situation, all tests pass with the exception of import group tests, since they require WordPress Import plugin that doesn't live under git working tree (these changesets [40532] and [40531] forced import group tests to fail instead of skip and they seems to revert the [27349]).

In the svn tree this doesn't occur because of svn:externals set to ​wordpress-importer.

As workaround we could alternatively:

• launch phpunit tests with --exclude-group import option or we

svn co https://plugins.svn.wordpress.org/wordpress-importer/trunk/ wordpress-importer

• under tests/phpunit/data/plugins/ directory and svn up it at regular basis, even if I think that the proposed patch would be a cleanier approach.

Anyway, in ​https://make.wordpress.org/core/handbook/contribute/git/, a notes about specific test actions above should be added.

PS: Probably there is, but I did not find a git repository for WordPress Importer plugin in order to add as submodule.

[enrico.sorcinelli]

Improved skipped test message

[jnylen0]

I don't think we should skip these tests - the changes in [40531] and [40532] were intentional. Without those changes, if someone has an incomplete test environment and they write patches against plugins or another feature that is left out of their local testing, then they're increasing the chance of writing a bad patch that fails a build later on. Quoting from #40533:

If the test data isn't valid or the environment is not capable of supporting the test then the test should fail, otherwise the tests are not reliable.

I added a new section to the handbook page on git that explains what is going on: ​https://make.wordpress.org/core/handbook/contribute/git/#unit-tests

Let's leave the test failures in place, but link to this new documentation from the test failure messages. Something like this:

This test requires the WordPress Importer plugin to be installed. See: ​https://make.wordpress.org/core/handbook/contribute/git/#unit-tests

[enrico.sorcinelli]

Now you've updated the handbook by explaining how to fix the failure when testing with git, I updated the patch by modifying the group import tests failure messages as you suggested.

[jnylen0]

Thanks for the patch. This will undoubtedly help people who are getting started with running the unit tests from a git repo.

[jnylen0]

In 41090:

Tests: Be more helpful when the wordpress-importer plugin is missing.

Link to some documentation that explains the problem and how to resolve it.

Props enrico.sorcinelli.
Fixes #41345.

[johnbillion]

In 41169:

Build/Test Tools: Clarify the error message when running the test suite without the WordPress Importer plugin present in the test suite.

See #41345