Lacking of thorough e2e testing

[amandasilvasantos]

While WordPress currently uses Playwright for E2E testing, there are notable limitations in terms of test coverage depth, maintenance effort, and non-technical contributor accessibility. I propose introducing testRigor into the E2E test suite as a complementary tool.

Key Benefits

• Plain English Tests: Write tests in natural language, making them accessible to all contributors.
• Low Maintenance: Resilient to UI changes thanks to AI-driven selectors.
• Tests across devices: Run tests across browsers and devices with no extra setup.
• Reusable Test Steps: Abstract common flows into shareable, readable steps.
• Faster Execution: Cloud-based parallel test runs speed up feedback loops.
• Non-Developer Friendly: Empowers QA, designers, and content creators to contribute with tests.

https://core.trac.wordpress.org/ticket/63657#no0

I set up an environment for tests with testRigor and integrated into CI. It is ready to test the latest version and further improved with even more tests in plain english.

Addition of new E2E tests in plain english.

Add successful ci integration

Add testRigor tests

[SirLouen]

Replying to amandasilvasantos:

While WordPress currently uses Playwright for E2E testing, there are notable limitations in terms of test coverage depth, maintenance effort, and non-technical contributor accessibility. I propose introducing testRigor into the E2E test suite as a complementary tool.

Looks good, I think we should implement it.

[TobiasBg]

Replying to SirLouen:

Looks good, I think we should implement it.

Then we should leave the ticket open :-)

[amandasilvasantos]

Replying to TobiasBg:

Replying to SirLouen:

Looks good, I think we should implement it.

Then we should leave the ticket open :-)

Hi! Do you guys think it's possible for us to get a mainteiner to create the github env variables in order to proceed the testing of this enhancement? What can I do to have further progress with this PR? Thanks in advance!

[SirLouen]

Replying to TobiasBg:

Replying to SirLouen:

Looks good, I think we should implement it.

Then we should leave the ticket open :-)

@TobiasBg I was being totally sarcastic with this shameless spamming reporter.
Do your diligence, and you will know what I'm talking about.

[johnbillion]

Let's close this off properly. The WordPress core project is not going to invest time currently into an alternative e2e test platform, especially one that operates via a third party SaaS.

The existing e2e infrastructure uses Playwright, the same as the Gutenberg project, and switching to a different e2e platform doesn't magically increase coverage.

[TobiasBg]

@SirLouen: Sorry for not understanding your sarcasm or not "doing my diligence"... Please remember that it's not always easy to spot humor in written texts (especially for non-native speakers), so that it's often better to refrain from using it in work tools like Trac or to only use it in addition, and maybe marked appropriately, to a proper explanation of an action. Based on your first reply, I simply thought that closing the ticket as invalid happened by accident, which is why I had reopened it.

[SirLouen]

Replying to TobiasBg:

Please remember that it's not always easy to spot humor in written texts (especially for non-native speakers)

It was not humorous, I was being completely hostile against the spammer, in spammer's language.

so that it's often better to refrain from using it in work tools like Trac or to only use it in addition, and maybe marked appropriately, to a proper explanation of an action. Based on your first reply, I simply thought that closing the ticket as invalid happened by accident, which is why I had reopened it.

If you want to further discuss on this, I invite you to come by in today's meeting @ 3PM in #core-test Slack channel.