Make WordPress Core

Opened 4 weeks ago

Closed 3 days ago

#63564 closed defect (bug) (fixed)

Dev environment should incorporate enhancements from wp-env for speed and non-interactive usage

Reported by: westonruter's profile westonruter Owned by: westonruter's profile westonruter
Milestone: 6.8.2 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch fixed-major dev-reviewed
Focuses: Cc:

Description (last modified by westonruter)

In gutenberg#49168 (fixed in gutenberg#50007), the cli container is kept running so that repeated calls to WP-CLI run very quick and avoid noisy messages like:

[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running   

This should be done for the wordpress-develop environment as well, as currently repeated calls to env:cli are very slow. For example, running npm run env:install includes multiple calls to the cli container, and it takes 15 seconds on my machine. However, if the cli container is already running, then it takes just 4 seconds.

Additionally, when attempting to use env:cli in a non-interactive context (e.g. piping content from stdin or using in a shell script), Docker emits an error:

$ echo "WP Develop" | npm --silent run env:cli -- option set blogname 
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
the input device is not a TTY

Finally, the arguments from npm run env:cli are not not being passed in a way that allows for spaces to be used in an argument name. For example:

$ npm --silent run env:cli option set blogname "WordPress Develop"
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
Error: Too many positional arguments: Develop

This is a follow-up to #63543.

Change History (14)

#1 @westonruter
4 weeks ago

  • Description modified (diff)
  • Status changed from assigned to accepted

This ticket was mentioned in PR #8969 on WordPress/wordpress-develop by @westonruter.


4 weeks ago
#2

  • Keywords has-patch added

Trac ticket: https://core.trac.wordpress.org/ticket/63564

  • Start cli container when running env:start. This greatly speeds up calls to WP-CLI since the container is already running rather than having to start up for each call. See https://github.com/WordPress/gutenberg/pull/50007.
  • Facilitate calls to env:cli in non-interactive context (non-TTY) to allow piping content into commands or use in shell scripts. Also see https://github.com/WordPress/gutenberg/pull/50007.
  • Fix passing arguments to WP-CLI from env:cli so that arguments with spaces are passed as expected.
  • Fix JSHint issues.

# Running env:install

## Before

$ time npm run env:install

> WordPress@6.9.0 env:install
> node ./tools/local-env/scripts/install.js

[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
Success: Generated 'wp-config.php' file.
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
Success: Updated the constant 'WP_DEBUG' in the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
Success: Added the constant 'WP_DEBUG_LOG' to the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
Success: Added the constant 'WP_DEBUG_DISPLAY' to the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
Success: Added the constant 'SCRIPT_DEBUG' to the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
Success: Added the constant 'WP_ENVIRONMENT_TYPE' to the 'wp-config.php' file with the value 'local'.
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
Success: Added the constant 'WP_DEVELOPMENT_MODE' to the 'wp-config.php' file with the value 'core'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
Success: Database reset.
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
Success: WordPress installed successfully.

real    0m14.993s
user    0m0.681s
sys     0m0.399s

## After

$ time npm run env:install

> WordPress@6.9.0 env:install
> node ./tools/local-env/scripts/install.js

Success: Generated 'wp-config.php' file.
Success: Updated the constant 'WP_DEBUG' in the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'WP_DEBUG_LOG' to the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'WP_DEBUG_DISPLAY' to the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'SCRIPT_DEBUG' to the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'WP_ENVIRONMENT_TYPE' to the 'wp-config.php' file with the value 'local'.
Success: Added the constant 'WP_DEVELOPMENT_MODE' to the 'wp-config.php' file with the value 'core'.
Success: Database reset.
Success: WordPress installed successfully.

real    0m4.018s
user    0m0.598s
sys     0m0.334s

# Running a single WP-CLI command

## Before

$ time npm run env:cli post list

> WordPress@6.9.0 env:cli
> node ./tools/local-env/scripts/docker.js run --rm cli post list

[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
+----+--------------+-------------+---------------------+-------------+
|| ID || post_title   || post_name   || post_date           || post_status ||
+----+--------------+-------------+---------------------+-------------+
|| 1  || Hello world! || hello-world || 2025-06-09 22:03:24 || publish     ||
+----+--------------+-------------+---------------------+-------------+

real    0m2.074s
user    0m0.125s
sys     0m0.108s

## After

$ time npm run env:cli post list

> WordPress@6.9.0 env:cli
> node ./tools/local-env/scripts/docker.js exec cli wp --allow-root post list

+----+--------------+-------------+---------------------+-------------+
|| ID || post_title   || post_name   || post_date           || post_status ||
+----+--------------+-------------+---------------------+-------------+
|| 1  || Hello world! || hello-world || 2025-06-09 22:03:24 || publish     ||
+----+--------------+-------------+---------------------+-------------+

real    0m0.682s
user    0m0.126s
sys     0m0.059s

# Passing an argument with spaces to WP-CLI

## Before

$ time npm --silent run env:cli option set blogname "WordPress Develop"
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
Error: Too many positional arguments: Develop


real    0m1.818s
user    0m0.137s
sys     0m0.071s

## After

$ time npm --silent run env:cli option set blogname "WordPress Develop"
Success: Value passed for 'blogname' option is unchanged.

real    0m0.773s
user    0m0.117s
sys     0m0.095s

# Passing data to WP-CLI via STDIN

## Before

$ echo "WP Develop" | time npm --silent run env:cli -- option set blogname 
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                0.0s 
the input device is not a TTY
        0.22 real         0.12 user         0.05 sys

## After

$ echo "WP Develop" | time npm --silent run env:cli -- option set blogname
Success: Updated 'blogname' option.
        0.81 real         0.11 user         0.09 sys

# Calling WP-CLI from shell script

Given a watch.sh script that runs PHP unit tests for the comment group whenever a couple of the comments PHP files are modified:

fswatch tests/phpunit/tests/comment/commentForm.php src/wp-includes/comment-template.php | while read -r modified_file; do
	clear
	echo "Modified file: $modified_file"
	npm --silent run test:php -- --group=comment
done

## Before

$ ./watch.sh
Modified file: /Users/westonruter/repos/wordpress-develop/tests/phpunit/tests/comment/commentForm.php
the input device is not a TTY

## After

$ ./watch.sh
Modified file: /Users/westonruter/repos/wordpress-develop/tests/phpunit/tests/comment/commentForm.php
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 9.6.23 by Sebastian Bergmann and contributors.

Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

...............................................................  63 / 510 ( 12%)
............................................................... 126 / 510 ( 24%)
............................................................... 189 / 510 ( 37%)
............................................................... 252 / 510 ( 49%)
............................................................... 315 / 510 ( 61%)
............................................................... 378 / 510 ( 74%)
............................................................... 441 / 510 ( 86%)
............................................................... 504 / 510 ( 98%)
......                                                          510 / 510 (100%)

Time: 00:03.489, Memory: 195.00 MB

OK (510 tests, 1260 assertions)

#3 follow-up: @sandeepdahiya
4 weeks ago

  • Keywords needs-testing added

Hi @westonruter,

When I try to apply the patch, I encounter the following issue:

Running "patch:https://github.com/WordPress/wordpress-develop/pull/8969.diff" (patch) task
patching file .github/workflows/reusable-test-local-docker-environment-v1.yml
patching file docker-compose.yml
Hunk #1 succeeded at 110 (offset -1 lines).
patching file package.json
patching file tools/local-env/scripts/docker.js
patching file tools/local-env/scripts/install.js
Hunk #1 FAILED at 13.
1 out of 1 hunk FAILED -- saving rejects to file tools/local-env/scripts/install.js.rej
patching file tools/local-env/scripts/start.js
patching file tools/local-env/scripts/utils.js

Done.

The failed hunk appears to correspond to this line in install.js:

// Create wp-config.php.
-wp_cli( `config create --dbname=wordpress_develop --dbuser=root --dbpass=password --dbhost=mysql --force --config-file=${process.env.LOCAL_DIR}/../wp-config.php` );
+wp_cli( `config create --dbname=wordpress_develop --dbuser=root --dbpass=password --dbhost=mysql --force --config-file="wp-config.php"` );

I already have a wp-config.php file in my root directory, so I'm unsure why this part of the patch is failing or if it’s even necessary to create the config file in my case.

#4 in reply to: ↑ 3 ; follow-up: @SirLouen
4 weeks ago

Replying to sandeepdahiya:

I already have a wp-config.php file in my root directory, so I'm unsure why this part of the patch is failing or if it’s even necessary to create the config file in my case.

Get the latest changes from trunk and pull them again before applying the patch.

#5 in reply to: ↑ 4 @sandeepdahiya
4 weeks ago

Replying to SirLouen:

Get the latest changes from trunk and pull them again before applying the patch.

Thank you @SirLouen.

In my case, the test results are:

#Before-patch:

$ time npm run env:install

> WordPress@6.9.0 env:install
> node ./tools/local-env/scripts/install.js

[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Generated 'wp-config.php' file.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Updated the constant 'WP_DEBUG' in the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Added the constant 'WP_DEBUG_LOG' to the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
Success: Added the constant 'WP_DEBUG_DISPLAY' to the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Added the constant 'SCRIPT_DEBUG' to the 'wp-config.php' file with the raw value 'true'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Added the constant 'WP_ENVIRONMENT_TYPE' to the 'wp-config.php' file with the value 'local'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Added the constant 'WP_DEVELOPMENT_MODE' to the 'wp-config.php' file with the value 'core'.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: Database reset.
[+] Creating 2/2
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
Success: WordPress installed successfully.

real    0m29.395s
user    0m0.212s
sys     0m0.151s

#After-patch: (much cleaner and faster)

time npm run env:install

> WordPress@6.9.0 env:install
> node ./tools/local-env/scripts/install.js

Success: Generated 'wp-config.php' file.
Success: Updated the constant 'WP_DEBUG' in the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'WP_DEBUG_LOG' to the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'WP_DEBUG_DISPLAY' to the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'SCRIPT_DEBUG' to the 'wp-config.php' file with the raw value 'true'.
Success: Added the constant 'WP_ENVIRONMENT_TYPE' to the 'wp-config.php' file with the value 'local'.
Success: Added the constant 'WP_DEVELOPMENT_MODE' to the 'wp-config.php' file with the value 'core'.
Success: Database reset.
Success: WordPress installed successfully.

real    0m18.533s
user    0m0.185s
sys     0m0.167s

#Before-patch:

$ time npm run env:cli post list

> WordPress@6.9.0 env:cli
> node ./tools/local-env/scripts/docker.js run --rm cli post list

[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
+----+--------------+-------------+---------------------+-------------+
| ID | post_title   | post_name   | post_date           | post_status |
+----+--------------+-------------+---------------------+-------------+
| 1  | Hello world! | hello-world | 2025-06-13 15:38:57 | publish     |
+----+--------------+-------------+---------------------+-------------+

real    0m4.975s
user    0m0.138s
sys     0m0.152s

#After-patch:

$ time npm run env:cli post list

> WordPress@6.9.0 env:cli
> node ./tools/local-env/scripts/docker.js exec cli wp --allow-root post list

+----+--------------+-------------+---------------------+-------------+
| ID | post_title   | post_name   | post_date           | post_status |
+----+--------------+-------------+---------------------+-------------+
| 1  | Hello world! | hello-world | 2025-06-13 16:01:46 | publish     |
+----+--------------+-------------+---------------------+-------------+

real    0m3.520s
user    0m0.120s
sys     0m0.137s

#Before-patch:

$ time npm --silent run env:cli option set blogname "My World"
[+] Creating 2/2
 ✔ Container wordpress-develop-php-1    Running                                                                                                                                                                              0.0s 
 ✔ Container wordpress-develop-mysql-1  Running                                                                                                                                                                              0.0s 
Error: Too many positional arguments: World


real    0m5.503s
user    0m0.165s
sys     0m0.136s

#After-patch (argument passing was also successful):

time npm --silent run env:cli option set blogname "My World"
Success: Value passed for 'blogname' option is unchanged.

real    0m3.703s
user    0m0.075s
sys     0m0.167s

This is the result for

$ npm run test:e2e

Running 25 tests using 1 worker:
................................................................................................
  2 failed
    [chromium] › tests\e2e\specs\gutenberg-plugin.test.js:25:6 › Gutenberg plugin › should activate
    [chromium] › tests\e2e\specs\install.test.js:34:6 › WordPress installation process › should install WordPress with pre-existing database credentials
  23 passed (2.1m)

Its a lot nicer experience now. @westonruter appreciate the efforts.

Last edited 4 weeks ago by sandeepdahiya (previous) (diff)

#6 @westonruter
4 weeks ago

  • Resolution set to fixed
  • Status changed from accepted to closed

In 60308:

Build/Test Tools: Improve dev environment's CLI in speed, non-interactive usage, and argument handling.

  • Start cli container when running env:start. This greatly speeds up calls to WP-CLI since the container is already running rather than having to start up for each call.
  • Facilitate calls to env:cli in non-interactive context (non-TTY) to allow piping content into commands or use in shell scripts.
  • Fix passing arguments to WP-CLI from env:cli so that arguments with spaces are passed as expected.
  • Fix JSHint issues.

This aligns the wordpress-develop environment closer to wp-env. See https://github.com/WordPress/gutenberg/pull/50007.

Props westonruter, jorbin, SirLouen, sandeepdahiya.
Fixes #63564.

#7 @westonruter
4 weeks ago

  • Keywords fixed-major added; needs-testing removed
  • Resolution fixed deleted
  • Status changed from closed to reopened

Reopening for 6.8.2 consideration.

#8 @westonruter
4 weeks ago

With this change, if you previously invoked npm run env:cli with a wp then this must be omitted now.

However, I just realized that actually it was supposed to work without an explicit wp, for example npm run env:cli pwd previously would print out /var/www and doesn't get routed to WP-CLI, whereas now this results in a WP-CLI error:

Error: 'pwd' is not a registered wp command. See 'wp help' for available commands.

See comment from @jorbin:

Since this is technically a breaking change, I searched github for npm run env:cli wp and there is only one reference which seems to be from a fork of a demo rather than actual production code, but it still might be worth a PR there after this lands and a quick note in the core room so folks are aware. I don't think it needs a dev note though since this isn't a public API.

So it seems in practice that this command is always used for WP-CLI. See also another GitHub search.

If we want to restore the ability to run arbitrary commands, perhaps this should be introduced as a separate command instead of overloading env:cli. As it stands right now, the this overloading could cause ambiguity, when you expect to run a system command but actually there is a WP-CLI command registered with the same name.

This ticket was mentioned in Slack in #core by westonruter. View the logs.


4 weeks ago

#10 @westonruter
4 weeks ago

Another downside of the previous overloading logic is that it adds additional latency because it first checks whether the provided command is a valid WP-CLI command before proceeding to then actually call the command with WP-CLI.

This ticket was mentioned in Slack in #core by zunaid321. View the logs.


3 weeks ago

This ticket was mentioned in Slack in #core by audrasjb. View the logs.


3 days ago

#13 @westonruter
3 days ago

  • Keywords dev-reviewed added

Reviewed by @audrasjb.

#14 @westonruter
3 days ago

  • Resolution set to fixed
  • Status changed from reopened to closed

In 60432:

Build/Test Tools: Improve dev environment's CLI in speed, non-interactive usage, and argument handling.

  • Start cli container when running env:start. This greatly speeds up calls to WP-CLI since the container is already running rather than having to start up for each call.
  • Facilitate calls to env:cli in non-interactive context (non-TTY) to allow piping content into commands or use in shell scripts.
  • Fix passing arguments to WP-CLI from env:cli so that arguments with spaces are passed as expected.
  • Fix JSHint issues.

This aligns the wordpress-develop environment closer to wp-env. See https://github.com/WordPress/gutenberg/pull/50007.

Reviewed by audrasjb.
Merges [60308] to the 6.8 branch, minus the change to .github/workflows/reusable-test-local-docker-environment-v1.yml which was made irrelevant in [60092].

Props westonruter, jorbin, SirLouen, sandeepdahiya.
Fixes #63564.

Note: See TracTickets for help on using tickets.