Setting WP_DEFAULT_THEME to a custom theme causes test to fail

[mboynes]

Today I ran core's unit tests with Underscores active and the following test failed:

There was 1 failure:

1) Tests_Theme::test_default_themes_have_textdomain
Failed asserting that an array contains '_s'.

/var/www/wordpress-develop/tests/phpunit/tests/theme.php:166

This test is directly asserting that WP_DEFAULT_THEME is one of the included themes. I see no reason for this, especially in this test which offers no reference to WP_DEFAULT_THEME and is checking that the default themes have a textdomain.

[mboynes]

Remove assertion that WP_DEFAULT_THEME is one of the provided themes

[nacin]

So, it is there for a reason, believe it or not. I think I wrote this one.

When we add a new theme, we update WP_DEFAULT_THEME. This isn't something we'll forget. But it's very possible that we'll forget to update the $default_themes property of the Tests_Theme class. That's why this assertion is there.

I'm generally not against adjusting poor assumptions in our unit tests, but we also can't make a guarantee that they'll always pass when the default environment is changed.

[mboynes]

Thanks for the insight @nacin!

What if we added a filter to $this->default_themes?

I think there's a good use case for allowing someone to test their theme against the full suite of core unit tests, to ensure that customizations in the theme don't break core features. I'm open to other solutions to help accomplish that end goal.

[nacin]

I think we could break this into a separate test, and only do the assertion if WP_DEFAULT_THEME starts with "twenty*", and otherwise skip the test.

[jorbin]

Nacin's suggestion seems like the right fix for this situation.

[jorbin]

In 29946:

Check if WP_DEFAULT_THEME starts with twenty before asserting it is in default theme array

This fixes an issue that if you change WP_DEFAULT_THEME and run core unit tests, the tests fail since your theme isn't one of the hard coded lists of default themes. We need to keep this test to make sure that we update the array of default themes for use in other tests.

If we ever change the naming convention for default themes, this will need to be updated.

props nacin for initial idea
fixes #29925

[TobiasBg]

Let's maintain coding standards and white space for new commits, see patch.

[SergeyBiryukov]

In 29952:

Fix formatting in [29946].

props TobiasBg.
fixes #29925.

[dd32]

In 39065:

Themes: Update the unit tests to handle [39064] and #31550.

test_default_theme_in_default_theme_list() was always being skipped after #31550, this causes it to once again check that the unit tests are up to date and include the latest default theme.
test_default_themes_have_textdomain() didn't play happy when a default theme wasn't installed on a site.

See #31550, #29925, #38551.