Run E2E tests with and without `SCRIPT_DEBUG` enabled

[Clorith]

The current setup for the E2E tests is to use the default .env generation, which sets SCRIPT_DEBUG to true. This is to capture any console warnings that are generated during the test runs, and make sure they are also surfaced.

Adding a matrix which allows us to run it with SCRIPTS_DEBUG set to both true and false would allow us to catch scenarios where the minified and non-minified files may differ, for example, the minification process has resulted in erroneous files.

The E2E tests initially always perform tests with SCRIPT_DEBUG set to true, to capture any console warnings that may be present.

By adding a matrix where we test both scenarios, we can capture failures where minified files are causing problems as well.

The test intentionally applies fail-fast: false as well, to ensure that both tests can finish, as there may be different failures in them, and one should not prevent the other to ensure we get as clear a picture as possible.

Trac ticket: [](https://core.trac.wordpress.org/ticket/58661)

[joemcgill]

Given that this is related to Build/Test tools, I'm suggesting we move this to the current release cycle and make it task. It would be valuable to the rest of the release process to have this change in place to avoid missing errors that are only visible when not using minified files, i.e. SCRIPT_DEBUG set to true.

I agree this is a good addition.

The e2e test environment is installed with npm run env:install which already supports defining script debug via the LOCAL_SCRIPT_DEBUG environment variable, if I'm not mistaken. Could that be used instead of manually calling the docker-compose command?

The e2e test environment is installed with npm run env:install which already supports defining script debug via the LOCAL_SCRIPT_DEBUG environment variable, if I'm not mistaken. Could that be used instead of manually calling the docker-compose command?

I was considering it, but then I became unsure, because I believe the .env file that core bundles will take priority when parsing for values, and it has that value defined and set to true. As the modification of the file would cause the final check, the one that looks for modifications to the git history, it was safer to just run the CLI command and flip the value within the already gitignored wp-config.php file that way.

A local environment variable should take precedence over the .env file as that's the default behaviour of Dotenv.

I'm not too worried, just seems we might as well use that method of configuration as it's already in place.

It looks like the env variables are being set correctly during the workflows after the latest update. 👍🏻

[johnbillion]

In 56113:

Build/Test Tools: Run E2E tests with and without SCRIPT_DEBUG enabled.

This adds a matrix to the e2e test runs on GitHub Actions so the tests run both with and without SCRIPT_DEBUG enabled. This allows us to catch scenarios where the minified and non-minified files may differ.

Props Clorith, joemcgill

Fixes #58661

https://core.trac.wordpress.org/changeset/56113

[johnbillion]

Reopening as I forgot I wanted to improve the job names.

[johnbillion]

In 56114:

Build/Test Tools: Improve the name of the e2e test jobs on GitHub Actions.

Fixes #58661

[desrosj]

In 58330:

Build/Test Tools: Make use of new reusable workflows for 6.1.

This updates the 6.1 branch to utilize the new reusable workflows in trunk introduced in [58165].

This also includes backports for a some additional improvements and bug fixes that are necessary for the local development environment to continue working long term:

• The image and platform properties for the mysql container have been updated to always prefer amd64 containers (#60822).
• macos-13 is now pinned for MacOS jobs instead of macos-latest (#61340).
• Removes the performance testing workflow. This workflow was overhauled in 6.4 to use Playwright. Continuing to support Puppeteer-based performance testing in 6.1-6.3 (which was historically very flaky) in a reusable workflow outweighs the benefit.
• Run E2E tests with and without SCRIPT_DEBUG (#58661).
• Migrating to Docker Compose V2 (#60901).
• Removing the version property from docker-compose.yml (#59416).
• Improvements to how artifacts and comments for Playground testing are generated.
• Removing SVN related commands causing failures (#61216).
• Updating the actions/github-scripts action to the latest version.

Merges [56113], [56114], [57918], [58157], [57124], [57125], [57249] to the 6.1 branch.

Props johnbillion, joemcgill, swissspidy, thelovekesh, narenin, mukesh27, JeffPaul, peterwilsoncc, zieladam, ockham, SergeyBiryukov, jorbin.
See #61340, #60822, #61216, #60901, #61101, #59416, #59805, #61213, #58661.

[desrosj]

In 58357:

Build/Test Tools: Make use of new reusable workflows for 6.0.

This updates the 6.0 branch to utilize the new reusable workflows in trunk introduced in [58165].

This also includes backports for a some additional improvements and bug fixes that are necessary for the local development environment to continue working long term:

• The image and platform properties for the mysql container have been updated to always prefer amd64 containers (#60822).
• macos-13 is now pinned for MacOS jobs instead of macos-latest (#61340).
• Run E2E tests with and without SCRIPT_DEBUG (#58661).
• Migrating to Docker Compose V2 (#60901).
• Removing the version property from docker-compose.yml (#59416).
• Improvements to how artifacts and comments for Playground testing are generated.
• Removing SVN related commands causing failures (#61216).
• Updating the actions/github-scripts action to the latest version.
• Move the Memcached container into the Docker Compose config (#55700).
• Configure Xdebug modes in the local Docker environment (#56022).

Merges [53895], [53552], [56113], [56114], [57918], [58157], [57124], [57125], [57249] to the 6.0 branch.

Props johnbillion, joemcgill, swissspidy, thelovekesh, narenin, mukesh27, JeffPaul, peterwilsoncc, zieladam, ockham, SergeyBiryukov, jorbin, Clorith, afragen.
See #55700, #61340, #60822, #61216, #60901, #61101, #59416, #59805, #61213, #58661.

[desrosj]

In 58358:

Build/Test Tools: Make use of new reusable workflows for 5.9.

This updates the 5.9 branch to utilize the new reusable workflows in trunk introduced in [58165].

This also includes backports for a some additional improvements and bug fixes that are necessary for the local development environment to continue working long term:

• The image and platform properties for the mysql container have been updated to always prefer amd64 containers (#60822).
• macos-13 is now pinned for MacOS jobs instead of macos-latest (#61340).
• Run E2E tests with and without SCRIPT_DEBUG (#58661).
• Migrating to Docker Compose V2 (#60901).
• Removing the version property from docker-compose.yml (#59416).
• Improvements to how artifacts and comments for Playground testing are generated.
• Removing SVN related commands causing failures (#61216).
• Updating the actions/github-scripts action to the latest version.
• Move the Memcached container into the Docker Compose config (#55700).
• Configure Xdebug modes in the local Docker environment (#56022).

Merges [53895], [53552], [56113], [56114], [57918], [58157], [57124], [57125], [57249] to the 5.9 branch.

Props johnbillion, joemcgill, swissspidy, thelovekesh, narenin, mukesh27, JeffPaul, peterwilsoncc, zieladam, ockham, SergeyBiryukov, jorbin, Clorith, afragen.
See #55700, #61340, #60822, #61216, #60901, #61101, #59416, #59805, #61213, #58661.

[desrosj]

In 58597:

Build/Test Tools: Make use of new reusable workflows for 5.8.

This updates the 5.8 branch to utilize the new reusable workflows in trunk introduced in [58165].

This also includes backports for a some additional improvements and bug fixes that are necessary for the local development environment to continue working long term:

• The image and platform properties for the mysql container have been updated to always prefer amd64 containers (#60822).
• macos-13 is now pinned for MacOS jobs instead of macos-latest (#61340).
• Run E2E tests with and without SCRIPT_DEBUG (#58661).
• Migrating to Docker Compose V2 (#60901).
• Removing the version property from docker-compose.yml (#59416).
• Improvements to how artifacts and comments for Playground testing are generated.
• Removing SVN related commands causing failures (#61216).
• Updating the actions/github-scripts action to the latest version.
• Move the Memcached container into the Docker Compose config (#55700).
• Configure Xdebug modes in the local Docker environment (#56022).
• Cache the results of PHP_CodeSniffer runs (#49783).

Merges [52179], [53895], [53552], [56113], [56114], [57918], [58157], [57124], [57125], [57249] to the 5.8 branch.

Props johnbillion, joemcgill, swissspidy, thelovekesh, narenin, mukesh27, JeffPaul, peterwilsoncc, zieladam, ockham, SergeyBiryukov, jorbin, Clorith, afragen, jrf.
See #48783, #55700, #61340, #60822, #61216, #60901, #61101, #59416, #59805, #61213, #58661.

[desrosj]

In 58598:

Build/Test Tools: Make use of new reusable workflows for 5.7.

This updates the 5.7 branch to utilize the new reusable workflows in trunk introduced in [58165].

This also includes backports for a some additional improvements and bug fixes that are necessary for the local development environment to continue working long term:

• The image and platform properties for the mysql container have been updated to always prefer amd64 containers (#60822).
• macos-13 is now pinned for MacOS jobs instead of macos-latest (#61340).
• Run E2E tests with and without SCRIPT_DEBUG (#58661).
• Migrating to Docker Compose V2 (#60901).
• Removing the version property from docker-compose.yml (#59416).
• Improvements to how artifacts and comments for Playground testing are generated.
• Removing SVN related commands causing failures (#61216).
• Updating the actions/github-scripts action to the latest version.
• Cache the results of PHP_CodeSniffer runs (#49783).
• A fix to grunt clean to prevent script-loader-packages.php from being deleted (#53606).

Merges [51355], [52179], [56113], [56114], [57918], [58157], [57124], [57125], [57249] to the 5.7 branch.

Props johnbillion, joemcgill, swissspidy, thelovekesh, narenin, mukesh27, JeffPaul, peterwilsoncc, zieladam, ockham, SergeyBiryukov, jorbin, Clorith, afragen, jrf.
See #48783, #61340, #60822, #61216, #60901, #61101, #59416, #59805, #61213, #58661, #53606.

[desrosj]

In 58605:

Build/Test Tools: Make use of new reusable workflows for 5.6.

This updates the 5.6 branch to utilize the new reusable workflows in trunk introduced in [58165].

This also includes backports for a some additional improvements and bug fixes that are necessary for the local development environment to continue working long term:

• The image and platform properties for the mysql container have been updated to always prefer amd64 containers (#60822).
• macos-13 is now pinned for MacOS jobs instead of macos-latest (#61340).
• Run E2E tests with and without SCRIPT_DEBUG (#58661).
• Migrating to Docker Compose V2 (#60901).
• Removing the version property from docker-compose.yml (#59416).
• Improvements to how artifacts and comments for Playground testing are generated.
• Removing SVN related commands causing failures (#61216).
• Updating the actions/github-scripts action to the latest version.
• Cache the results of PHP_CodeSniffer runs (#49783).
• Move the Memcached container into the Docker Compose config (#55700).
• Configure Xdebug modes in the local Docker environment (#56022).
• Improvements to the healthcheck command for the mysql container (#58867).
• A fix to grunt clean to prevent script-loader-packages.php from being deleted (#53606).

Merges [51355], [51673], [52179], [53552], [53895], [56113], [56114], [56464], [57918], [58157], [57124], [57125], [57249] to the 5.6 branch.

Props johnbillion, joemcgill, swissspidy, thelovekesh, narenin, mukesh27, JeffPaul, peterwilsoncc, zieladam, ockham, SergeyBiryukov, jorbin, Clorith, afragen, jrf.
See #49783, #58867, #61340, #60822, #61216, #60901, #61101, #56022, #59416, #59805, #61213, #58661, #53606.