Make WordPress Core

Opened 5 months ago

Last modified 2 days ago

#64225 new task (blessed)

Test tool and unit test improvements for 7.0

Reported by: desrosj's profile desrosj Owned by:
Milestone: 7.0 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch has-unit-tests
Focuses: Cc:

Description

This ticket is for various fixes and improvements in PHPUnit tests that don't have a more specific ticket, as well as general improvements to the GitHub Actions workflows that run automated testing.

Change History (71)

This ticket was mentioned in PR #10014 on WordPress/wordpress-develop by @rollybueno.


5 months ago
#1

  • Keywords has-patch has-unit-tests added

This test verifies that post meta cache persists (is not invalidated) when a post is updated, which documents the actual WordPress cache behavior.

Trac ticket:
https://core.trac.wordpress.org/ticket/64225
https://core.trac.wordpress.org/ticket/63167

This ticket was mentioned in PR #10057 on WordPress/wordpress-develop by @jignesh.nakrani.


5 months ago
#2

Update unsupported image format tests to use TGA instead of AVIF

Trac ticket:
https://core.trac.wordpress.org/ticket/64225
https://core.trac.wordpress.org/ticket/63167

Updated REST API test methods using the unsupported image format to test unsupported logic checks

This ticket was mentioned in PR #10060 on WordPress/wordpress-develop by @paulbonneau.


5 months ago
#3

Adding PHPUnit tests for get_post_states function

Trac ticket:
https://core.trac.wordpress.org/ticket/64225
https://core.trac.wordpress.org/ticket/63167

This ticket was mentioned in PR #10519 on WordPress/wordpress-develop by @desrosj.


5 months ago
#4

MySQL 9.5 is the most recent innovation release. This should be tested against appropriately.

Trac ticket: Core-64225

#5 @desrosj
5 months ago

In 61245:

Build/Test Tools: Add MySQL 9.5 to test strategies.

MySQL 9.5 is the most recent innovation release. This updates the GitHub Actions workflows testing against innovation release versions to use this new version.

See #64225.

@desrosj commented on PR #10519:


5 months ago
#6

Merged in r61245.

@westonruter commented on PR #10060:


5 months ago
#7

@paulbonneau There's a merge conflict. Could you resolve?

@paulbonneau commented on PR #10060:


5 months ago
#8

@paulbonneau There's a merge conflict. Could you resolve?

Hi @westonruter, the merge conflict has been resolved

This ticket was mentioned in PR #10642 on WordPress/wordpress-develop by @jonsurrell.


4 months ago
#9

These tests failed with semantically equivalent HTML while working on https://github.com/WordPress/wordpress-develop/pull/10639.

assertEqualHTML makes the tests more robust.

Trac ticket: https://core.trac.wordpress.org/ticket/64225

This ticket was mentioned in PR #10643 on WordPress/wordpress-develop by @jonsurrell.


4 months ago
#10

Fix class names or filename to match PHPUnit expectations.

PHPUnit can be run with a filename argument:

phpunit [options] UnitTest.php

However, certain test files and test suite class names do not align with PHPUnit expectations and it may fail to run any tests, for example:

./vendor/bin/phpunit tests/phpunit/tests/rest-api/rest-controller.php

Class rest-controller could not be found in /Users/jonsurrell/a8c/wordpress-develop/trunk/tests/phpunit/tests/rest-api/rest-controller.php

Trac ticket: https://core.trac.wordpress.org/ticket/64225

#11 @SergeyBiryukov
4 months ago

In 61391:

Tests: Use assertEqualHTML() in Script Modules HTML tests.

This aims to make the tests more robust.

Follow-up to [58579].

Props jonsurrell.
See #64225.

@SergeyBiryukov commented on PR #10642:


4 months ago
#12

Thanks for the PR! Merged in r61391.

This ticket was mentioned in PR #10644 on WordPress/wordpress-develop by @jonsurrell.


4 months ago
#14

These tests failed with semantically equivalent HTML while working on https://github.com/WordPress/wordpress-develop/pull/10641.

assertEqualHTML makes the tests more resilient to semantically irrelevant changes.

Trac ticket: https://core.trac.wordpress.org/ticket/64225

#15 @SergeyBiryukov
4 months ago

In 61392:

Tests: Use assertEqualHTML() in styles HTML tests.

This aims to make the tests more robust.

Follow-up to [287/tests], [749/tests], [25786], [31031], [36550], [46164], [48937], [52036], [61084], [61391].

Props jonsurrell.
See #64225.

@SergeyBiryukov commented on PR #10644:


4 months ago
#16

Thanks for the PR! Merged in r61392.

#18 @SergeyBiryukov
4 months ago

In 61394:

Tests: Update scripts tests to use semantic HTML comparison.

This aims to make the tests more robust.

Follow-up to [50167], [60295], [61391], [61392].

Props jonsurrell.
See #64225.

@SergeyBiryukov commented on PR #10649:


4 months ago
#19

Thanks for the PR! Merged in r61394.

#20 @SergeyBiryukov
4 months ago

In 61395:

Tests: Correst some test class names per the naming conventions.

Follow-up to [50167], [50409], [56748].

See #64225.

This ticket was mentioned in PR #10657 on WordPress/wordpress-develop by @jonsurrell.


4 months ago
#21

These tests failed with semantically equivalent HTML while working on https://github.com/WordPress/wordpress-develop/pull/10656.

assertEqualHTML makes the tests more robust.

Trac ticket: https://core.trac.wordpress.org/ticket/64225

#22 @jonsurrell
4 months ago

In 61409:

Tests: Update fonts tests to use semantic HTML comparison.

This aims to make the tests more robust.

Follow-up to [60295], [61391], [61392].

Props jonsurrell.
See #64225.

@jonsurrell commented on PR #10657:


4 months ago
#23

Merged in r61409.

#24 @SergeyBiryukov
4 months ago

In 61423:

Tests: Reset environment before performing assertions in populate_network() tests.

This aims to avoid affecting other tests in case of failure.

Follow-up to [60954], [61420].

See #64225.

This ticket was mentioned in PR #10727 on WordPress/wordpress-develop by @jonsurrell.


3 months ago
#25

assertEqualHTML makes the tests resilient to irrelevant syntactic changes.

Extracted from https://github.com/WordPress/wordpress-develop/pull/10639.

Trac ticket: https://core.trac.wordpress.org/ticket/64225

Follow up to r61394, r61391.

#26 @jonsurrell
3 months ago

In 61478:

Build/Test Tools: Use semantic HTML comparison in script tests.

The assertEqualHTML() method makes tests resilient to irrelevant syntactic changes in HTML output, focusing on semantic equivalence rather than exact string matching.

Developed in https://github.com/WordPress/wordpress-develop/pull/10727.

Follow up to [61394], [61391].

Props jonsurrell, cbravobernal.
See #64225.

@jonsurrell commented on PR #10727:


3 months ago
#27

Merged in r61478.

This ticket was mentioned in PR #10810 on WordPress/wordpress-develop by @huzaifaalmesbah.


3 months ago
#28

Ticket: https://core.trac.wordpress.org/ticket/64225

Adds unit tests to cover named entities, numeric entities, and edge cases in WP_HTML_Decoder, achieving 100% code coverage.

Changes:

  • Added test_decode_attribute_decodes_named_entities for standard named entities.
  • Added test_decode_attribute_decodes_hex_numeric_entities and test_decode_attribute_decodes_decimal_numeric_entities.
  • Added test_decodes_windows_1252_mapped_characters to verify C1 control mappings.
  • Added edge case tests for invalid numeric entities, null bytes, and 4-byte UTF-8 characters (emojis).
  • Added test_decodes_ambiguous_ampersands for context-specific behavior.

Testing:
Run npm run test:php -- --group html-api to verify. All tests pass.

This ticket was mentioned in PR #10847 on WordPress/wordpress-develop by @jonsurrell.


3 months ago
#29

This file was added in https://github.com/WordPress/wordpress-develop/pull/9014
but missing from r60516.

While reviewing another PR, I noticed Failed to load resource: net::ERR_FILE_NOT_FOUND errors in the qunit tests and investigated. It led me to this line which references a file that doesn't exist in the repo:

https://github.com/WordPress/wordpress-develop/blob/f0aa4ec597e284769f629e01fce3be7020a1a669/tests/qunit/index.html#L150

See https://core.trac.wordpress.org/ticket/63126.

Trac ticket: https://core.trac.wordpress.org/ticket/64225

@jonsurrell commented on PR #10847:


3 months ago
#30

Testing can confirm that there are no Failed to load resource: net::ERR_FILE_NOT_FOUND messages on CI:

https://github.com/WordPress/wordpress-develop/actions/runs/21587508566/job/62199177243?pr=10847

[Trunk does have them:
](https://github.com/WordPress/wordpress-develop/actions/runs/21586989732/job/62197454498)

Running "qunit:files" (qunit) task
Testing tests/qunit/compiled.html JQMIGRATE: Migrate is installed, version 3.4.1
Failed to load resource: net::ERR_FILE_NOT_FOUND
.....................................................................................................................................................................................................................OK
Testing tests/qunit/index.html JQMIGRATE: Migrate is installed with logging active, version 3.4.1
Failed to load resource: net::ERR_FILE_NOT_FOUND
.....................................................................................................................................................................................................................OK

@jonsurrell commented on PR #10847:


3 months ago
#31

I'm surprised this was not caught by quint and reported as an error:

https://qunitjs.com/api/extension/QUnit.onUncaughtException/

@mukesh27 commented on PR #10847:


3 months ago
#32

I'm surprised this was not caught by quint and reported as an error:

https://qunitjs.com/api/extension/QUnit.onUncaughtException/

+1 Let's explore why it was not shown the error and commit this one

#33 @jonsurrell
2 months ago

In 61625:

Build/Test Tools: Add theme keyboard navigation tests.

Add the theme keyboard navigation test file that omitted from [60516].

Follow-up to [60516].

Props abcd95, jonsurrell, mukesh27.
See #64225, #63126.

@jonsurrell commented on PR #10847:


2 months ago
#34

Merged in r61625.

This ticket was mentioned in PR #10950 on WordPress/wordpress-develop by @Rahmohn.


2 months ago
#35

Trac ticket: https://core.trac.wordpress.org/ticket/64225

## Use of AI Tools

Nill for coding

#36 @SergeyBiryukov
2 months ago

In 61714:

Tests: Move test_illegal_user_logins_single() to the correct place.

Follow-up to [35189], [35629], [35772], [61656].

See #64225.

#37 @SergeyBiryukov
2 months ago

In 61715:

Tests: Use named data provider in a balanceTags() test.

Follow-up to [661/tests], [45929].

Props rahmohn.
See #64225.

#38 @westonruter
7 weeks ago

In 61789:

Code Quality: Normalize object as type being supplied as comment, post, site, term, and user.

When constructing these objects, the specific instance type is redundantly used in a union with a generic object. A generic object can be passed directly from database row result. This also hardens get_post() to account for passing an object that lacks an ID property. Similarly, sanitize_term() is hardened to account for an object lacking a term_id property. Comprehensive unit tests are added for get_post() and sanitize_term().

Developed in https://github.com/WordPress/wordpress-develop/pull/11096

Props westonruter, apermo.
See #64238, #64225.

#39 @desrosj
7 weeks ago

In 61836:

Build/Test Tools: Increase timeout-minutes for PHPUnit workflow.

The GitHub Actions workflow responsible for running the PHPUnit test suite is frequently encountering the default timeout-minutes value of 20 since the changes in [61438]. The result is that the workflow is consistently unable to finish running and ends up being cancelled.

This bumps the default value to 40 until the overall speed of the build script can be improved.

See #64225, #64227, #64393.

#40 @desrosj
6 weeks ago

In 61875:

Build/Test Tools: Test against MySQL 9.6 & MariaDB 12.1.

These are now the latest Innovation and Rolling releases, respectively.

See #64225, #64227.

#41 @peterwilsoncc
6 weeks ago

In 61999:

Build/Test tools: Unify REST API group as restapi.

Replace @group rest-api with @group restapi to unify the tests under the one group name, chosen on the basis that majority rules.

Follow up to r61045, r61032.

See #64225.

#42 @desrosj
6 weeks ago

In 62022:

Build/Test Tools: Fix installation tests for PHP <= 7.3.

In MySQL 8.0, the default authentication plugin changed from mysql_native_password to caching_sha2_password, which is not available in PHP 7.3 and earlier. The installation testing workflow currently accounts for this, but only for PHP 7.2 & 7.3.

It also does not account for a change in MySQL 8.4 which changed the related flag for controlling this from --default-authentication-plugin to --authentication-policy.

This adjusts the flags and surrounding conditions to ensure the correct ones are passed for the right combinations.

Props johnbillion.
See #64225, #64227.

#43 @desrosj
6 weeks ago

In 62023:

Build/Test Tools: Add additional innovation release excludes.

This excludes MySQL 9.5 jobs for the installation testing workflow. Only the most recent innovation release of MySQL should be tested.

Follow up to [61875].

See #64225, #64227.

#44 @desrosj
5 weeks ago

Leaving this open for a bit longer since there seems to be a few open PRs remaining.

Created #64894 for the 7.1 release cycle.

This ticket was mentioned in PR #11381 on WordPress/wordpress-develop by @desrosj.


4 weeks ago
#45

This fixes a failing tst that seems to be caused by the fact that MySQL changed the content on release pages.

Unfortunately, it does not seem like there is an official API available from MySQL itself. However, endoflife.date is an open source, MIT licensed, reasonably maintained project that does have an endpoint available.

Trac ticket: Core-64225.

## Use of AI Tools

AI Assistance: Yes
Tool(s): Claude Code
Model(s): Sonnet 4.6
Used for: Searching for possible APIs to use to replace parsing an HTML document.

@desrosj commented on PR #11381:


4 weeks ago
#46

Tested with the $this->skipOnAutomatedBranches() commented out so that the test would run within the PR's workflow runs. Tests pass.

@desrosj commented on PR #11381:


4 weeks ago
#47

Merged in r62170.

@SergeyBiryukov commented on PR #10950:


3 weeks ago
#48

Thanks for the PR! Merged in r61715.

This ticket was mentioned in PR #11403 on WordPress/wordpress-develop by @westonruter.


3 weeks ago
#49

See Slack discussion raised by @sirreal.

Erroneously introduced in https://github.com/WordPress/wordpress-develop/commit/a0b67495342265ee4de947d783569c23fa7adca2 (r41773) for Core-40104. The PHP tags were surely copied incorrectly from the PHP source file as opposed to being copied from the source code view in the browser.

Trac ticket: https://core.trac.wordpress.org/ticket/64225

## Use of AI Tools

None

#50 @westonruter
3 weeks ago

In 62184:

Build/Test Tools: Remove erroneous PHP tags for translations from QUnit HTML file.

The markup had surely been copied from the PHP source file, as opposed to being copied from the rendered HTML, as it should have been.

Developed in https://github.com/WordPress/wordpress-develop/pull/11403

Follow-up to r41773.

Props westonruter, jonsurrell, desrosj, SergeyBiryukov.
See #64225, #40104.

@westonruter commented on PR #11403:


3 weeks ago
#51

Committed in r62184 (15e60ff)

#52 @SergeyBiryukov
3 weeks ago

In 62205:

Tests: Move data providers and helpers in Tests_REST_Server for consistency.

This ensures that data providers or helper functions used by a single test are located next to the test, for consistency with the rest of the test suite.

Follow-up to [37905], [37943], [45809], [47239], [47260], [47351], [48947], [49252], [49257], [51960], [53110], [56096], [59032].

See #64225.

#53 @SergeyBiryukov
2 weeks ago

In 62207:

Tests: Adjust Unicode tests for consistency.

Includes:

  • Adding missing @covers tags.
  • Correcting test class names as per the naming conventions.
  • Moving wp_check_invalid_utf8() tests to their own file, separate from wp_scrub_utf8().

Follow-up to [60630], [60793], [61000].

See #64225.

#54 @SergeyBiryukov
2 weeks ago

In 62208:

Tests: Move wp_dropdown_languages() tests to their own file.

This aims to make the tests more discoverable and easier to expand.

Follow-up to [36631], [39169], [43359], [44514].

See #64225.

#55 @SergeyBiryukov
2 weeks ago

In 62213:

Tests: Add missing @covers tags for some multisite tests.

See #64225.

#56 @SergeyBiryukov
2 weeks ago

In 62218:

Tests: Add missing @covers tags for some multisite tests.

Follow-up to [62213].

See #64225.

#57 @SergeyBiryukov
2 weeks ago

In 62222:

Tests: Add missing @covers tags for some multisite tests.

Follow-up to [62213], [62218].

See #64225.

This ticket was mentioned in PR #11524 on WordPress/wordpress-develop by @sagardeshmukh.


13 days ago
#58

## What

This PR makes two types of test improvements across three test files.


---


### 1. Add missing @covers annotations

Adds @covers tags to test classes that were missing them, as part of the
ongoing effort in #64225:


| File | Annotation added |

|

| tests/phpunit/tests/rewrite/addRewriteRule.php | @covers ::add_rewrite_rule |
| tests/phpunit/tests/rewrite/addRewriteEndpoint.php | @covers ::add_rewrite_endpoint |
| tests/phpunit/tests/blocks/wpBlockTypeRegistry.php | @covers WP_Block_Type_Registry |


Both add_rewrite_rule() and add_rewrite_endpoint() are defined in
src/wp-includes/rewrite.php.


Without @covers, PHPUnit cannot correctly attribute code coverage to the
functions under test, which leads to inaccurate coverage reports on Codecov.


---


### 2. Refactor repeated invalid block name tests to use a data provider

In Tests_Blocks_wpBlockTypeRegistry, four separate test methods were
structurally identical — each registered one invalid block name and asserted
false:


  • test_invalid_non_string_names()
  • test_invalid_names_without_namespace()
  • test_invalid_characters()
  • test_uppercase_characters()


These have been consolidated into a single test_invalid_block_names() method
backed by a data_invalid_block_names() data provider with named cases,
following the same pattern used in recently updated tests across the suite.


Before: 4 methods × ~9 lines each = ~36 lines
After: 1 method + 1 data provider = ~21 lines


Named data sets also make PHPUnit failure output more readable — instead of
test_invalid_characters you see `test_invalid_block_names with data set
"invalid characters"`, which is clearer at a glance.


---


## Testing

Ran all three affected test classes locally:


npm run test:php -- --filter "Tests_Rewrite_AddRewriteRule|Tests_Rewrite_AddRewriteEndpoint|Tests_Blocks_wpBlockTypeRegistry"


OK (20 tests, 42 assertions)


All passing. The data provider runs each case as its own PHPUnit test, so the
assertion count is unchanged.


---


## Trac

See https://core.trac.wordpress.org/ticket/64225

---

_This is my first contribution to WordPress core. I've enabled "Allow edits
and access to secrets by maintainers". Happy to adjust anything based on
review feedback._


---

This ticket was mentioned in PR #11532 on WordPress/wordpress-develop by @sagardeshmukh.


13 days ago
#59

Tests: Add missing @covers tags for blocks, oembed, and rewrite tests.

Follow-up to [62213], [62218], [62222].

Fixes https://core.trac.wordpress.org/ticket/64225

#60 @SergeyBiryukov
12 days ago

In 62223:

Tests: Add missing @covers tags for some oEmbed tests.

Props sagardeshmukh.
See #64225.

#61 @SergeyBiryukov
11 days ago

In 62224:

Tests: Rename some oEmbed test classes as per the naming conventions.

Follow-up to [34903], [62223].

See #64225.

#62 @SergeyBiryukov
10 days ago

In 62227:

Tests: Rename some oEmbed test classes as per the naming conventions.

Includes moving the data provider after the corresponding test for consistency with the rest of the test suite.

Follow-up to [34903], [62223], [62224].

See #64225.

#63 @SergeyBiryukov
10 days ago

In 62228:

Tests: Expand @covers tags for WP_oEmbed tests.

Follow-up to [62223].

See #64225.

#64 @SergeyBiryukov
9 days ago

In 62229:

Tests: Expand @covers tags for WP_Embed tests.

Includes removing the external-http group for a WP_Embed::run_shortcode() test which does not perform any HTTP requests.

Follow-up to [50448], [62223], [62228].

See #64225.

#65 @SergeyBiryukov
7 days ago

In 62234:

Tests: Rename some oEmbed test classes as per the naming conventions.

Follow-up to [37708], [37892], [62224], [62227].

See #64225.

#66 @SergeyBiryukov
6 days ago

In 62237:

Tests: Add missing @covers tags for some multisite tests.

Follow-up to [62213], [62218], [62222].

See #64225.

#67 @SergeyBiryukov
5 days ago

In 62243:

Tests: Remove external-http group from a get_theme_feature_list() test.

This particular test checks the list of theme features hardcoded into Core and does not perform an external API request.

Follow-up to [39906].

See #64225.

#68 @SergeyBiryukov
4 days ago

In 62244:

Tests: Add missing @covers tags for some rewrite tests.

Props sagardeshmukh.
See #64225.

#69 @SergeyBiryukov
4 days ago

In 62245:

Tests: Use a data provider in a WP_Block_Type_Registry test for invalid block names.

Includes adding missing @covers tags.

Follow-up to [43742], [51491].

Props sagardeshmukh.
See #64225.

@SergeyBiryukov commented on PR #11524:


4 days ago
#70

Thanks for the PR! Merged in r62244 and r62245.

#71 @SergeyBiryukov
2 days ago

In 62246:

Tests: Add missing @covers tags for some rewrite tests.

Follow-up to [36181], [62244].

Props sagardeshmukh.
See #64225.

Note: See TracTickets for help on using tickets.