Travis: Download Chromium for e2e tests only

[SergeyBiryukov]

Every other commit or so, tests randomly fail with a message like this:

> puppeteer@2.1.1 install /home/travis/build/WordPress/wordpress-develop/node_modules/puppeteer
> node install.js
ERROR: Failed to download Chromium r722234! Set "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" env variable to skip download.
{ Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27)
  -- ASYNC --
    at BrowserFetcher.<anonymous> (/home/travis/build/WordPress/wordpress-develop/node_modules/puppeteer/lib/helper.js:111:15)
    at Object.<anonymous> (/home/travis/build/WordPress/wordpress-develop/node_modules/puppeteer/install.js:66:16)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:741:12)
    at startup (internal/bootstrap/node.js:285:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:739:3) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! puppeteer@2.1.1 install: `node install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the puppeteer@2.1.1 install script.

Chromium is only needed for e2e tests and should not be downloaded for PHP or JS tests.

This will also speed up the tests a bit by not downloading ~120 MB over and over again on every run.

[SergeyBiryukov]

In 47449:

Tests: Download Chromium for e2e tests only.

This reduces the chance of random failures due to network timeouts and avoids the overhead of unnecessarily downloading ~120 MB for PHP or JS tests.

Fixes #49621.

[SergeyBiryukov]

Reopening for backporting to the 5.4 branch after a second committer's review.

[SergeyBiryukov]

For some more context, this happened during the RC2 release process and can happen again when working on the final release or subsequent minor releases. Random failures like these that don't indicate any issues with our own code are annoying and reduce our trust in the tests.

[kirasong]

+1

Looks good to me to backport to 5.4

[gziolo]

We have the same conditions in Gutenberg :+1:

There is a patch that should disable Chromium download by default but it's blocked by the issue in the latest version of Puppeteer, see ​https://github.com/WordPress/gutenberg/pull/20215 for more.

[SergeyBiryukov]

In 47459:

Tests: Use PUPPETEER_SKIP_CHROMIUM_DOWNLOAD environment variable instead of config setting.

This is more consistent with the similar approach in Gutenberg Travis setup.

Follow-up to [47449].

See #49621.

[SergeyBiryukov]

In 47460:

Tests: Download Chromium for e2e tests only.

This reduces the chance of random failures due to network timeouts and avoids the overhead of unnecessarily downloading ~120 MB for PHP or JS tests.

Reviewed by mikeschroder, gziolo.
Merges [47449] and [47459] to the 5.4 branch.
Fixes #49621.

[desrosj]

In 49227:

Build/Test Tools: Download Chromium for E2E tests only.

This prevents Chromium from being downloaded in workflows where it is not required.

See #49621, #50649, #50401.

[gziolo]

@desrosj Interesting, it shouldn't be an issue for the most recent version of @wordpress/scripts. There was a change introduced that install Chromium only when e2e test is executed for the first time. Well, it doesn't harm to have this variable set but it feels redundant unless something went wrong.