Make WordPress Core

Opened 8 months ago

Closed 8 months ago

#58661 closed task (blessed) (fixed)

Run E2E tests with and without `SCRIPT_DEBUG` enabled

Reported by: clorith's profile Clorith Owned by: johnbillion's profile johnbillion
Milestone: 6.3 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch
Focuses: Cc:

Description

The current setup for the E2E tests is to use the default .env generation, which sets SCRIPT_DEBUG to true. This is to capture any console warnings that are generated during the test runs, and make sure they are also surfaced.

Adding a matrix which allows us to run it with SCRIPTS_DEBUG set to both true and false would allow us to catch scenarios where the minified and non-minified files may differ, for example, the minification process has resulted in erroneous files.

Change History (13)

This ticket was mentioned in PR #4747 on WordPress/wordpress-develop by @Clorith.


8 months ago
#1

  • Keywords has-patch added

The E2E tests initially always perform tests with SCRIPT_DEBUG set to true, to capture any console warnings that may be present.

By adding a matrix where we test both scenarios, we can capture failures where minified files are causing problems as well.

The test intentionally applies fail-fast: false as well, to ensure that both tests can finish, as there may be different failures in them, and one should not prevent the other to ensure we get as clear a picture as possible.

Trac ticket: [](https://core.trac.wordpress.org/ticket/58661)

This ticket was mentioned in Slack in #core by clorith. View the logs.


8 months ago

#3 @johnbillion
8 months ago

  • Milestone changed from Awaiting Review to 6.4

#4 @joemcgill
8 months ago

  • Keywords 2nd-opinion added
  • Milestone changed from 6.4 to 6.3
  • Type changed from enhancement to task (blessed)

Given that this is related to Build/Test tools, I'm suggesting we move this to the current release cycle and make it task. It would be valuable to the rest of the release process to have this change in place to avoid missing errors that are only visible when not using minified files, i.e. SCRIPT_DEBUG set to true.

@johnbillion commented on PR #4747:


8 months ago
#5

I agree this is a good addition.

The e2e test environment is installed with npm run env:install which already supports defining script debug via the LOCAL_SCRIPT_DEBUG environment variable, if I'm not mistaken. Could that be used instead of manually calling the docker-compose command?

@Clorith commented on PR #4747:


8 months ago
#6

The e2e test environment is installed with npm run env:install which already supports defining script debug via the LOCAL_SCRIPT_DEBUG environment variable, if I'm not mistaken. Could that be used instead of manually calling the docker-compose command?

I was considering it, but then I became unsure, because I believe the .env file that core bundles will take priority when parsing for values, and it has that value defined and set to true. As the modification of the file would cause the final check, the one that looks for modifications to the git history, it was safer to just run the CLI command and flip the value within the already gitignored wp-config.php file that way.

@johnbillion commented on PR #4747:


8 months ago
#7

A local environment variable should take precedence over the .env file as that's the default behaviour of Dotenv.

I'm not too worried, just seems we might as well use that method of configuration as it's already in place.

joemcgill commented on PR #4747:


8 months ago
#8

It looks like the env variables are being set correctly during the workflows after the latest update. 👍🏻

#9 @johnbillion
8 months ago

  • Keywords 2nd-opinion removed
  • Owner set to johnbillion
  • Status changed from new to reviewing

#10 @johnbillion
8 months ago

  • Resolution set to fixed
  • Status changed from reviewing to closed

In 56113:

Build/Test Tools: Run E2E tests with and without SCRIPT_DEBUG enabled.

This adds a matrix to the e2e test runs on GitHub Actions so the tests run both with and without SCRIPT_DEBUG enabled. This allows us to catch scenarios where the minified and non-minified files may differ.

Props Clorith, joemcgill

Fixes #58661

#12 @johnbillion
8 months ago

  • Resolution fixed deleted
  • Status changed from closed to reopened

Reopening as I forgot I wanted to improve the job names.

#13 @johnbillion
8 months ago

  • Resolution set to fixed
  • Status changed from reopened to closed

In 56114:

Build/Test Tools: Improve the name of the e2e test jobs on GitHub Actions.

Fixes #58661

Note: See TracTickets for help on using tickets.