Make WordPress Core

Opened 8 weeks ago

Last modified 8 weeks ago

#53781 new defect (bug)

Tests_Embed_Template tests fail when run in isolation

Reported by: jrf Owned by:
Milestone: Awaiting Review Priority: normal
Severity: minor Version:
Component: Build/Test Tools Keywords: needs-patch
Focuses: coding-standards Cc:


When running the Tests_Embed_Template test in isolation, two of the tests fail.

phpunit --filter Tests_Embed_Template
#) Tests_Embed_Template::test_oembed_output_post_with_thumbnail
DOMDocument::loadHTML(): AttValue: " expected in Entity, line: 10


#) Tests_Embed_Template::test_oembed_output_attachment
DOMDocument::loadHTML(): AttValue: " expected in Entity, line: 10


Accidentally discovered when I added the import group to the <groups> <excludes> and ran the complete test suite and those two tests - which are unrelated to the import tests - started erroring out.

A tentative conclusion from that can be that:

  1. The Tests_Embed_Template tests have an undeclared dependency on the tests in the import group.
  2. The Tests_Embed_Template tests only pass by accident due to the order in which the tests are run, which is not guaranteed and can be influenced in more recent PHPUnit versions, so cannot be relied upon.

Also see: https://phpunit.readthedocs.io/en/stable/configuration.html#the-executionorder-attribute

This needs further investigation.

Solution direction: The test should be made independent of the import tests.

Originally reported in https://core.trac.wordpress.org/ticket/53363#comment:39

Change History (1)

#1 @SergeyBiryukov
8 weeks ago

Thanks for the ticket!

Just noting that I cannot seem to reproduce yet, these tests run as expected for me:

S:\home\wordpress.test\develop-trunk>phpunit --filter Tests_Embed_Template
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

.................                                                 17 / 17 (100%)

You should really fix these slow tests (>150ms)...
 1. 303ms to run Tests_Embed_Template:test_oembed_output_post

Time: 53.52 seconds, Memory: 144.00 MB

OK (17 tests, 55 assertions)

That said, there might indeed be something unstable about these tests that needs further investigation.

Note: See TracTickets for help on using tickets.