e2e timeout errors: waiting for selection `.components-snackbar`

[hellofromTonya]

The CI is experiencing too many e2e time out errors, which may indicate a flaky test or instability. This ticket is opened to investigate and monitor for stability improvements.

Error:

TimeoutError: waiting for selector `.components-snackbar` failed: timeout 30000ms exceeded
    at new WaitTask (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/puppeteer-core/src/common/DOMWorld.ts:780:28)
    at DOMWorld.waitForSelectorInPage (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/puppeteer-core/src/common/DOMWorld.ts:632:22)
    at Object.internalHandler.waitFor (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/puppeteer-core/src/common/QueryHandler.ts:78:19)
    at DOMWorld.waitForSelector (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/puppeteer-core/src/common/DOMWorld.ts:490:25)
    at Frame.waitForSelector (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/puppeteer-core/src/common/FrameManager.ts:1163:47)
    at Page.waitForSelector (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/puppeteer-core/src/common/Page.ts:2100:29)
    at publishPost (/home/runner/work/wordpress-develop/wordpress-develop/node_modules/@wordpress/e2e-test-utils/build/@wordpress/e2e-test-utils/src/publish-post.js:19:14)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at Object.<anonymous> (/home/runner/work/wordpress-develop/wordpress-develop/tests/e2e/specs/edit-posts.test.js:25:3)

Capturing CI job runs where it failed:

• ​CI job run and ​Slack
• ​CI job run and ​Slack
• ​CI job run and ​Slack

[youknowriad]

I noticed these as well, I think these failures are probably there since these tests have been added. My suspicions is CSS animations, in the Gutenberg tests relying on the same helpers, there's no flakiness, but there are two extra things done on the Gutenberg e2e setup:

• Reduce animations (which I attempted in Core here ​https://github.com/WordPress/wordpress-develop/commit/fdb2dacc199382cd0800838596b5717cbf8e211f#diff-cd20a88b9a0d8ff93f4ebc81383c60b53312bb8aed35909f2ae80ec6094dc924R132-R134 and committed let's see if it improves things
• The "disable-css-animation" plugin which is only loaded in the e2e test setup in the Gutenberg test environment. I didn't do this on Core yet because that plugin is not available on the core's environment yet but if things stay unstable after my commit above, we could try doing this as well.

[isabel_brison]

All of these failures are happening in the publishPost function imported from the e2e-test-utils package. What's interesting is that function is used in most of the tests from that suite and only one of them is consistently failing: shows a single post after one is published with the correct title (though allows an existing post to be deleted using the Trash button also failed in one of those runs).

[justinahinon]

This happens very randomly (at least on local env). I've run the test up to 20 times, for i in {1..20}; do npm run test:e2e -- tests/e2e/specs/edit-posts.test.js; done and had only one failure.

@isabel_brison in the ​publishPost() function, could we wait for the content in the body panel instead of the snack bar? See ​https://cln.sh/Jsm9GY.