Avoid test skipping for situations that are actually errors or failures

[johnbillion]

Some tests contain calls to markTestSkipped() as a result of situations such as incorrect test data or unsuitable test environments. If the test data isn't valid or the environment is not capable of supporting the test then the test should fail, otherwise the tests are not reliable.

This is a catch-all ticket for attempting to remove a bunch of test skipping that falls outside of the is_multisite() skipping that's being handled in #40531.

[johnbillion]

In 40517:

Canonical: Don't skip tests if the test data is invalid.

See #40533

[johnbillion]

In 40519:

Build/Test Tools: Don't skip tests when php.net or dev.mysql.com are unreachable.

This sort of behaviour subtly hides the fact that these tests will never fail if the URLs where version number information is fetched from go away for one reason or another.

See #40533

[johnbillion]

In 40521:

Build/Test Tools: Remove a useless test.

See #33898, #40533

[johnbillion]

In 40523:

Build/Test Tools: Remove ancient UT ticket handling.

See #40533

[johnbillion]

In 40524:

Build/Test Tools: Remove more skipped tests that should actually be failures if their conditions aren't satisfied.

See #40533

[johnbillion]

In 40525:

Build/Test Tools: Replace test skipping with actual assertions when dealing with the DISALLOW_UNFILTERED_HTML, DISALLOW_FILE_MODS, and DISALLOW_FILE_EDIT constants.

If any of these constants are set during testing, they should cause the tests to fail, not to be skipped, otherwise we have a situation where failure conditions are never seen.

See #40533

[johnbillion]

In 40526:

Build/Test Tools: Don't trigger a skipped test when the built version of wp-embed.min.js isn't present.

The grunt command assertion prior to the check for the existence of the file is enough to safeguard this test.

See #34698, #40533

[johnbillion]

In 40527:

Build/Test Tools: Enable verbose mode in PHPUnit so we can see which tests are being skipped, and now that the number of skipped tests has been lowered.

See #40533, #40531

[johnbillion]

In 40528:

Build/Test Tools: Add some locale debugging to the Travis config so we can determine which locales are available to test with.

See #40533, #19861

[johnbillion]

In 40529:

Build/Test Tools: Partially revert [40524].

See #40533, #40535

[johnbillion]

In 40531:

Build/Test Tools: Convert more test skipping into hard failures. These dependencies should all be present when testing.

See #40533

[johnbillion]

In 40532:

Build/Test Tools: Convert more test skipping into hard failures. These dependencies should all be present when testing.

See #40533

[johnbillion]

In 40533:

Build/Test Tools: Remove unnecessary checks and skips that should instead cause failures if they ever fail.

See #40533

[johnbillion]

In 40544:

Build/Test Tools: Remove more unnecessary test skipping when erroneous situations occur.

See #40533

[obenland]

@johnbillion Can this be closed for 4.8?

[jnylen0]

Each of these failures should have some information about how to resolve it. For example, the importer plugin is missing by default if you use git to develop against WordPress (#41345). This isn't documented anywhere that I could find.

Ideally the error message should describe what is going on and point to documentation that explains how to resolve the issue. For the importer test failures, we can link to ​https://make.wordpress.org/core/handbook/contribute/git/#unit-tests.

[johnbillion]

In 41305:

Build/Test Tools: Remove ancient UT ticket handling for the 4.7 branch.

See #40533

Merges [40523] to the 4.7 branch.

[SergeyBiryukov]

In 46996:

Tests: In test_readme(), if the HTTP request to secure.php.net or dev.mysql.com failed, include response code in the error message.

See #40533, #44613.