WordPress.org

Make WordPress Core

Opened 14 months ago

Last modified 3 weeks ago

#48301 assigned task (blessed)

Backport the simplified local environment

Reported by: desrosj Owned by: desrosj
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch
Focuses: Cc:

Description

This is a continuation ticket of #47767.

In that ticket, a Docker-based local WordPress Development environment was added to trunk.

This ticket is being opened to accomplish the 3rd goal listed on the ticket, It should be simple to backport to old branches, so we can more easily set up test environments when backporting patches to old branches.

https://core.trac.wordpress.org/ticket/47767#comment:6 has an initial proof of concept for backporting the environment to 3.7, though it needs to be updated to account for the changes made further down in the ticket.

Change History (56)

This ticket was mentioned in PR #125 on WordPress/wordpress-develop by desrosj.


12 months ago

This ticket was mentioned in PR #133 on WordPress/wordpress-develop by desrosj.


11 months ago

This ticket was mentioned in PR #134 on WordPress/wordpress-develop by desrosj.


11 months ago

This ticket was mentioned in PR #135 on WordPress/wordpress-develop by desrosj.


11 months ago

This ticket was mentioned in PR #136 on WordPress/wordpress-develop by desrosj.


11 months ago

This ticket was mentioned in PR #137 on WordPress/wordpress-develop by desrosj.


11 months ago

#7 @desrosj
11 months ago

  • Keywords has-patch added; needs-patch removed

I finally got some working branches pushed to my GitHub fork. You can find the PRs linked above.

I started by only going back to 4.8. At this point, I stop encountering new problems. Solving these first will most likely make going back further go much more smoothly.

For each .env file, I have changed LOCAL_PHP-latest to load the highest version of PHP supported by that branch as the default to prevent any potential issues as newer latest builds are released. The LOCAL_MYSQL versions in the .env files need to be updated in the same fashion. I have been less involved in the MySQL version support changes, so I need to research when support for each new MySQL version was added.

I think making a MySQL version support page like the one for PHP versions would be very useful.

Some Notes

  • I kept build matrices exactly the same as they are now in each branch for this ticket. Trimming or expanding the build matrices can be explored in separate tickets after this transition is made.
  • To install the new required NPM packages (dotenv, dotenv-expand, and wait-on) and update the npm-shrinkwrap.json file correctly, I ran `npm install --save-dev --save-exact packagename@1.0.0.
  • Updating the npm-shrinkwarp-json file will cause fsevents to be re-added. This needs to be manually removed to avoid compatibility issues with non MacOS servers (including Travis, Docker, and the build server). See #38657.
  • npm ci does not appear to work on branches < 5.0. I switched the command to npm install.

Issues:

A few of the pull request Travis jobs are failing due to a time out when running npm ci. I'm not sure why as these jobs pass on my fork.

5.1 Branch

PHP 7.3, 7.2, 7.1: For some reason, the Tests_Image_Editor_Imagick::test_rotate and Tests_Image_Editor_Imagick::test_flip tests are reading the RGB values of the manipulated images incorrectly, getting an off by one value. This issue does not occur locally for me when running the test suite within the Docker container with the same versions configured.

PHP 5.4, 5.3, 5.2: The following error is being returned for these jobs:

/usr/local/bin/docker-php-ext-enable: 5: cd: can't cd to /usr/local/lib/php/extensions/no-debug-non-zts-20060613

5.0 Branch

PHP 7.3=: There is a Fatal error: Class PHPUnit_Util_Test may not inherit from final class (PHPUnit\Util\Test) in /var/www/tests/phpunit/includes/phpunit6-compat.php on line 18 happening. I believe that this is being caused because PHPUnit 7 is packaged in the PHP 7.1-7.3-fpm PHPUnit Docker images, but WordPress 5.0 only supported up to PHPUnit 6.

PHP 5.2: The same error is being encountered as PHP 5.4-5.2 above.

4.9 Branch

The wait-on >= 3.0.0 NPM package does not support NPM 6.9.1, which is the version used in this branch. Downgrading the package to 2.1.2 seems to work, but there is a potentially unrelated JavaScript error:

/home/travis/build/desrosj/wordpress-develop/tools/local-env/scripts/install.js:33 .then( () => { TypeError: Cannot read property 'then' of undefined at Object

This could also just be a compatibility issue with the install.js script and NPM 6.9.1.

For PHP 5.2, The same error is being caused above.

4.8 Branch

The same issues described for 4.9 also occur in this branch.

#8 @desrosj
11 months ago

Experimented with changing npm ci to npm install in the 5.2 branch. It fixed the timeout issue with the PHP 7.3 job, but introduced a new error for the 5.6 job: cb() never called!.

Also, here is a link to the Docker image for PHPUnit on PHP 7.1 loading PHPUnit 7: https://hub.docker.com/layers/wordpressdevelop/phpunit/7.1-fpm/images/sha256-7c8bccf4a538f8918c9223904cf1d59ff9811eba2836ef83391986659eb7afd5

Moving forward as support for newer versions of PHPUnit is added, there will likely be a need for more granularity here. @pento any thoughts on how to handle this?

#9 @prbot
10 months ago

noisysocks commented on PR #133:

Test test test

#10 @prbot
10 months ago

desrosj commented on PR #133:

Test

#11 @desrosj
7 months ago

  • Type changed from defect (bug) to enhancement

This ticket was mentioned in Slack in #core-committers by desrosj. View the logs.


6 months ago

#13 @desrosj
7 weeks ago

Related: #50401.

I've opened https://github.com/WordPress/wpdev-docker-images/pull/44, which seems to fix the issues with the PHP 5.2/5.3/5.4 containers.

Backporting this should be much easier now with the GitHub Actions.

#14 @desrosj
7 weeks ago

  • Owner set to desrosj
  • Status changed from new to assigned

#15 @desrosj
5 weeks ago

In 49335:

Build/Test Tools: Specify a type when using wp config set.

This prevents an error if constants are not predefined when running the Docker install script while using older combinations of WordPress/PHP/WP-CLI.

See #48301.

#16 @desrosj
5 weeks ago

In 49340:

Build/Test Tools: Specify a type when using wp config set.

This prevents an error if constants are not predefined when running the Docker install script while using older combinations of WordPress/PHP/WP-CLI.

Backports [49335] to the 5.5 branch.
See #48301.

#17 @desrosj
5 weeks ago

In 49341:

Build/Test Tools: Specify a type when using wp config set.

This prevents an error if constants are not predefined when running the Docker install script while using older combinations of WordPress/PHP/WP-CLI.

Backports [49335] to the 5.4 branch.
See #48301.

#18 @desrosj
5 weeks ago

In 49342:

Build/Test Tools: Specify a type when using wp config set.

This prevents an error if constants are not predefined when running the Docker install script while using older combinations of WordPress/PHP/WP-CLI.

Backports [49335] to the 5.3 branch.
See #48301.

#19 @prbot
4 weeks ago

desrosj commented on PR #133:

This has become difficult to sync. Going to open a fresh PR.

#20 @prbot
4 weeks ago

desrosj commented on PR #134:

This has become difficult to sync. Going to open a fresh PR.

#21 @prbot
4 weeks ago

desrosj commented on PR #136:

This has become difficult to sync. Going to open a fresh PR.

#22 @prbot
4 weeks ago

desrosj commented on PR #137:

This has become difficult to sync. Going to open a fresh PR.

This ticket was mentioned in PR #693 on WordPress/wordpress-develop by desrosj.


4 weeks ago

  • Keywords has-unit-tests added

This backports the local Docker environment to the 5.2 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

This ticket was mentioned in PR #695 on WordPress/wordpress-develop by desrosj.


4 weeks ago

This backports the local Docker environment to the 5.2 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

This ticket was mentioned in PR #696 on WordPress/wordpress-develop by desrosj.


4 weeks ago

This backports the local Docker environment to the 5.1 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

This ticket was mentioned in PR #697 on WordPress/wordpress-develop by desrosj.


4 weeks ago

This backports the local Docker environment to the 4.9 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

This ticket was mentioned in PR #698 on WordPress/wordpress-develop by desrosj.


4 weeks ago

This backports the local Docker environment to the 4.8 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

This ticket was mentioned in PR #699 on WordPress/wordpress-develop by desrosj.


4 weeks ago

This backports the local Docker environment to the 4.7 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

This ticket was mentioned in PR #700 on WordPress/wordpress-develop by desrosj.


4 weeks ago

This backports the local Docker environment to the 4.6 branch.

Trac ticket: https://core.trac.wordpress.org/ticket/48301.

#30 @desrosj
4 weeks ago

  • Keywords needs-testing has-unit-tests removed
  • Type changed from enhancement to task (blessed)

I've spent a huge amount of time on this the last few months. I'd like to wrap this up, and I think we're in a state where we can backport . It's a blocker for backporting the GitHub action workflows to older branch, which will is a huge step towards finishing a transition off Travis CI.

I've resolved almost all of the issues above. A large number of them were being caused by missing packages or incorrect configurations within the Docker containers (mainly for PHP 5.4 and earlier).

The one remaining issue is that the PHP 5.2 Docker container is not yet able to be used. For more information on that, see the ongoing GitHub PR (TLDR: WP-CLI does not support PHP 5.2, and the PHPUnit test suite requires version 3.6 on PHP 5.2, which is no longer available). For the time being, the 5.2 jobs will continue using Travis' precise image, and will not be switched to use the Docker container.

Here is a list of remaining blockers:

#46073/[45067]: This was resolved in the 5.2 branch. This will need to be backported all the way to 3.7. Because the Travis environment does not include the needed Imagick components, the related tests are always skipped. But when the tests shift to being run within the Docker container, Imagick is present and this error presents itself.

[48486]: This change backported some error detection within tests confirming PDF thumbnail generation functionality to the 5.0 branch. However, the variable name within the test_pdf_preview_doesnt_overwrite_existing_jpeg() test was inadvertently incorrect. Similar to the last item, this test function was never reached because of a lack of the presence of Imagick. Within the Docker container, it now causes a test error. This needs to be committed to 5.0 and backported to the 4.7 branch.

#51624: This fixes a compatibility issue with Composer 2.0. This has been backported to the 5.3 branch, but it looks like it will also need to be merged back through 5.1 because composer.json/composer.lock files are present. The travis:format command does call composer to check for coding standards issues, but it does so within the Travis image, which still runs Composer 1.x. When the Docker container is backported, this will now run within the container, which has Composer 2.0.

Because the PHP_CodeSniffer is updated as a result of this fix, there are a few minor, spacing and alignment coding standards issues that will be flagged during the code base scan. I have included these fixes in the PRs, and they should be OK to make/include in the next release for the 5.2 and 5.1 branches. The total number is 6 files with coding standards changes, which should not cause issues with auto-updates.

Problems with old Node versions: In the 4.5 and lower branches, the version of NodeJS required drops to NodeJS < 6. This becomes problematic because the required versions of the dotenv, dotenv-expand, wait-on NPM packages do not support these older versions of NodeJS. I'd like to take a step back and look holistically at NodeJS in older branches. This has been discussed in 51603#comment:15 (see also #51682, #48140).

Today's Plan
My plan for after this comment is to backport everything outlined today back to the 4.6 branch (all open PRs above). After that, we can solve the NodeJS problem, and finish backporting.

#31 @desrosj
4 weeks ago

In 49514:

Build Tools: Fix running installing Composer dependencies using Composer 2.0.

This updates the dealerdirect/phpcodesniffer-composer-installer package to allow installing version 0.7.0 which supports Composer 2.0.

It also includes several minor spacing/alignment coding standards fixes that are made as a result of the package update.

Props itowhid06, jrf.
Merges [49306] to the 5.2 branch.
See #51624, #48301.

#32 @desrosj
4 weeks ago

In 49515:

Build Tools: One additional coding standards fix now detected after [49514].

See #51624, #48301.

#33 @desrosj
4 weeks ago

In 49516:

Build Tools: Fix running installing Composer dependencies using Composer 2.0.

This updates the dealerdirect/phpcodesniffer-composer-installer package to allow installing version 0.7.0 which supports Composer 2.0.

It also includes several minor spacing/alignment coding standards fixes that are made as a result of the package update.

Props itowhid06, jrf.
Merges [49306] to the 5.1 branch.
See #51624, #48301.

#34 @desrosj
4 weeks ago

In 49517:

Media: Improve and fix rotate/flip image tests.

Fix off-by-one error in pixel color checks for rotate and flip image tests. Change to using PNG with single pixel to ensure that errors are caught in the future, rather than lost in JPEG noise.

Props Fuegas, mikeschroder.
Merges [45067] to the 5.1 branch.
See #46073, #48301.

#35 @desrosj
4 weeks ago

In 49518:

Media: Improve and fix rotate/flip image tests.

Fix off-by-one error in pixel color checks for rotate and flip image tests. Change to using PNG with single pixel to ensure that errors are caught in the future, rather than lost in JPEG noise.

Props Fuegas, mikeschroder.
Merges [45067] to the 5.0 branch.
See #46073, #48301.

#36 @desrosj
4 weeks ago

In 49519:

Media: Improve and fix rotate/flip image tests.

Fix off-by-one error in pixel color checks for rotate and flip image tests. Change to using PNG with single pixel to ensure that errors are caught in the future, rather than lost in JPEG noise.

Props Fuegas, mikeschroder.
Merges [45067] to the 4.9 branch.
See #46073, #48301.

#37 @desrosj
4 weeks ago

In 49520:

Media: Improve and fix rotate/flip image tests.

Fix off-by-one error in pixel color checks for rotate and flip image tests. Change to using PNG with single pixel to ensure that errors are caught in the future, rather than lost in JPEG noise.

Props Fuegas, mikeschroder.
Merges [45067] to the 4.8 branch.
See #46073, #48301.

#38 @desrosj
4 weeks ago

In 49521:

Media: Improve and fix rotate/flip image tests.

Fix off-by-one error in pixel color checks for rotate and flip image tests. Change to using PNG with single pixel to ensure that errors are caught in the future, rather than lost in JPEG noise.

Props Fuegas, mikeschroder.
Merges [45067] to the 4.7 branch.
See #46073, #48301.

#39 @desrosj
4 weeks ago

In 49522:

Media: Improve and fix rotate/flip image tests.

Fix off-by-one error in pixel color checks for rotate and flip image tests. Change to using PNG with single pixel to ensure that errors are caught in the future, rather than lost in JPEG noise.

Props Fuegas, mikeschroder.
Merges [45067] to the 4.6 branch.
See #46073, #48301.

#40 @desrosj
4 weeks ago

In 49523:

Tests: Fix an incorrect variable name in PDF tests.

This causes a PHP error when running the test suite with PDF rendering supported.

See #50573, #48301.

#41 @desrosj
4 weeks ago

In 49526:

Tests: Fix an incorrect variable name in PDF tests.

This causes a PHP error when running the test suite with PDF rendering supported.

Merges [49523] to the 4.7 branch.
See #50573, #48301.

#42 @desrosj
4 weeks ago

In 49527:

Build/Test Tools: Backport the local Docker environment to the 5.2 branch.

This commit introduces the Docker-based local WordPress development environment to the 5.2 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 5.2 branch.
See #48301, #47767.

#44 @desrosj
4 weeks ago

In 49528:

Build/Test Tools: Backport the local Docker environment to the 5.2 branch.

This commit introduces the Docker-based local WordPress development environment to the 5.2 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 5.1 branch.
See #48301, #47767.

#46 @desrosj
4 weeks ago

In 49529:

Build/Test Tools: Backport the local Docker environment to the 5.0 branch.

This commit introduces the Docker-based local WordPress development environment to the 5.0 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Until existing blockers with the PHP 5.2 Docker container can be solved, the PHP 5.2 test job will remain using the Travis precise image.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 5.0 branch.
See #48301, #47767.

#48 @desrosj
4 weeks ago

In 49530:

Build/Test Tools: Backport the local Docker environment to the 4.9 branch.

This commit introduces the Docker-based local WordPress development environment to the 4.9 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Until existing blockers with the PHP 5.2 Docker container can be solved, the PHP 5.2 test job will remain using the Travis precise image.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 4.9 branch.
See #48301, #47767.

#50 @desrosj
4 weeks ago

In 49531:

Build/Test Tools: Backport the local Docker environment to the 4.8 branch.

This commit introduces the Docker-based local WordPress development environment to the 4.8 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Until existing blockers with the PHP 5.2 Docker container can be solved, the PHP 5.2 test job will remain using the Travis precise image.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 4.8 branch.
See #48301, #47767.

#52 @desrosj
4 weeks ago

In 49532:

Build/Test Tools: Backport the local Docker environment to the 4.7 branch.

This commit introduces the Docker-based local WordPress development environment to the 4.7 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Until existing blockers with the PHP 5.2 Docker container can be solved, the PHP 5.2 test job will remain using the Travis precise image.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 4.7 branch.
See #48301, #47767.

#53 @desrosj
4 weeks ago

In 49533:

Build/Test Tools: Backport the local Docker environment to the 4.6 branch.

This commit introduces the Docker-based local WordPress development environment to the 4.6 branch and converts the Travis test jobs to utilize this environment for easier and more consistent testing.

Until existing blockers with the PHP 5.2 Docker container can be solved, the PHP 5.2 test job will remain using the Travis precise image.

Merges [45745,45762,45783-45784,45800,45819,45885,46320,46999,47225,47912,48121,49335,49358,49360,49362] to the 4.6 branch.
See #48301, #47767.

This ticket was mentioned in Slack in #core by sergey. View the logs.


3 weeks ago

Note: See TracTickets for help on using tickets.