Overview of e2e tests to be written

[isabel_brison]

#49507 adds e2e tests for the edit posts page. The goal is to write tests for all the admin, but we should be clear on which parts need most testing and what exactly to test. To kick things off, I'm making a list of all the pages, some of the tests we should write, and some questions that occurred to me. Feedback on what to add, change or remove is very welcome!

Dashboard:

• Quick draft section: add title, save draft, navigate to posts page and check it's there.

Updates page: not sure how we should test this one.

Categories:

• Add new category: add name, click add new category button, check if it appears in list
• Check that bulk deleting all categories leaves “uncategorized” untouched
• Add child category
• Delete category
• Search for category
• Search for non-existing category

Tags: same as categories.

Media:

• Add image to library
• Edit image alt text, title, desc
• Search for image
• Search for non-existing image

Pages: same as posts (see #49507)

Comments:

• Create post, view it, add a comment, check that it displays in comments page.

Themes:

• Search for a theme
• Activate a theme
• Delete a theme, then search for it
• Add new theme and activate it

Customize: this is pretty theme-dependent, so we should pick a theme and check a few flows on it.

Widgets and Menus: might not be worth writing tests yet as these screens are being re-written.

Theme editor:

• Add a line of CSS in the theme stylesheet and check for it in the front end?

Plugins: how should we test this page?

Users:

• Add new user
• Search for user
• change user role
• Check admin user can't change itself to subscriber

Profile: there are lots of options here, should we test changes to each of them? Some may not be relevant, e.g. profile pic just takes us to Gravatar.

Import: We could test whether the plugins install correctly, but any further than that should be unnecessary (the plugins should have their own tests)

Export/Export personal data: would it be worth trying to check the contents of the downloaded files?

Site health: there's no real interaction on this page, so maybe we can skip it.

Erase personal data: not sure how to test this one.

Settings: like profile, lots of options here. Test each individually?

[francina]

Dropping some links I mentioned this morning on Dev Chat here, so everything is under one roof :)

• Core blog post with a discussion about setup ​https://make.wordpress.org/core/2020/08/07/e2e-end-to-end-testing-in-core-working-group-proposal/
• Document with test cases created during February Yoast Contributor Day (they can be turned into instructions for automated tests) ​https://docs.google.com/document/d/1g-D3Vht0ofR7JZ3xBJiQEtdW24XTiIfzvUf5fsOJQ88/edit#heading=h.20jvhpux4c2a

Thanks :🙌

[dariak]

Hello! The overview looks very good!
Here are some of my ideas on this:

Updates page:

• expected WordPress version is shown
• Plugins, Themes and Translations sections are present

Themes (additions to already suggested use cases):

• check that the current/default Theme is active

Theme editor (additions to already suggested use cases):

• just simply add "hello world" output on a homepage (it can be easier to assert)

Plugins:

• search for a plugin, install it, activate and deactivate

Profile:

• change default data and save changes

Export/Export personal data:
I think it is enough to check that a download is triggered.

Erase personal data:

• have a test user created in setUp of test data, search for this user, and check that this user appears in Requester. I do not think the real request needs to be executed and tested on this step

Settings:
I would check here all the defaults which are filled in for all Settings options.

Trac ticket: https://core.trac.wordpress.org/ticket/52895

[justinahinon]

I am drafting here ​https://github.com/WordPress/wordpress-develop/pull/1231 the Core categories tests

Related Trac ticket: https://core.trac.wordpress.org/ticket/52895.

Related Trac ticket: https://core.trac.wordpress.org/ticket/52895.

[justinahinon]

I am going to split this ticket in multiple smaller tickets that will focus on specific parts to test.

[francina]

I believe this can then become a task, but I will defer to more experienced contributors :)

[joyously]

Customize: this is pretty theme-dependent, so we should pick a theme and check a few flows on it.

There are several core options in the Customizer, like site identity, tagline, custom CSS, menus. But the loading and saving of all the registered options is Customizer code. Also the save as draft feature is core.

Does an end-to-end test mean that the main function of the page is tested, or that all the features on the page are tested?

[hellofromTonya]

Replying to joyously:

Does an end-to-end test mean that the main function of the page is tested, or that all the features on the page are tested?

e2e testing is simulating a real world scenario to test functionality and performance from a user/product point-of-view. Click or do this workflow and then validate the expected result.

Yes, it is testing a feature and the functionality that delivers that feature. How? By running it in a complete way (end-to-end) which exercises that specific feature's software and systems to ensure all of the pieces are working together to deliver the expected behavior.

Does this make sense?

[swissspidy]

#48977 was marked as a duplicate.

[swissspidy]

#55042 was marked as a duplicate.

Playwright e2e test for filtering media by type

Trac ticket: https://core.trac.wordpress.org/ticket/52895

playwright e2e test to filter media by date

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test to validate media list view

Added Playwright e2e test for bulk edit pages

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test to validate previewing a page

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test to validate taxonomy - Categories & Tags

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test for bulk delete of pages

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test for quick edit of pages

Trac ticket: https://core.trac.wordpress.org/ticket/52895

…antly from trash

Added Playwright e2e test for undo trashed post and delete posts permanently

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test for edit page

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Add Playwright e2e test to add a new page

Trac ticket: https://core.trac.wordpress.org/ticket/52895

Trac ticket: https://core.trac.wordpress.org/ticket/52895
Also moved dashboard.test.js to dashboard directory

Trac ticket: https://core.trac.wordpress.org/ticket/52895
It has test

1. To trash a post
2. To Restore post from trash

E2E tests were running with twentytwentyone, made changes to work with current theme twentytwentyfour

Also made the changes in dashboard test to make it work with new activated theme

https://core.trac.wordpress.org/ticket/52895

Added Playwright e2e test to validate theme editor

Added Playwright e2e test to validate theme editor

https://core.trac.wordpress.org/ticket/52895
It has tests to

• upload and install an older version of the plugin
• upload and update to a newer version of the plugin
• remove the installed plugin successfully'

https://core.trac.wordpress.org/ticket/52895
It has tests to

• Add a new theme
• Activate a theme
• Delete a theme

https://core.trac.wordpress.org/ticket/52895

It has tests to

• View the comment from backend
• Quick edit the comment
• Delete the comment

https://core.trac.wordpress.org/ticket/52895

https://core.trac.wordpress.org/ticket/52895

It has tests to

• hide/show widgets using Screen Options
• move widgets using arrows
• collapse and expand widgets

https://core.trac.wordpress.org/ticket/52895

https://core.trac.wordpress.org/ticket/52895

Thanks for the first pass on this. I think it needs to be expanded a bit before it's ready. I would like to see an additional type besides image that is uploaded and that test to ensure that the correct number of items is in the media directory before filtering and then at each step of the way (i.e. if an mp3 is uploaded along with the two images, check that there are 3 normally, 1 with audio, 0 with video, and 2 with image)

Also, it would be nice to not only test the grid view but also test the list view

This doesn't seem to actually test the filtering by date method. I think we need to upload the files and change the dates in order to do so

One change that I would like to see is that the restoring of the default value should be moved into an afterAll action so that if the assertion fails for some reason, the default value is still restored.

Hey @aaronjorbin
Thanks for the review. I agree that makes sense, somehow I missed this. I have added now in afterEach, please have a look.
afterAll can't be used as page fixtures are created per test basis

Hi @aaronjorbin I have made the requested changes. Can you please recheck?

Trac ticket: https://core.trac.wordpress.org/ticket/52895

This PR has test to

• create privacy page
• changes the Privacy Policy Page
• validates the Privacy Policy Guide Title
• copies WordPress text to clipboard

To make the "copies WordPress text to clipboard" test work, in Playwright config, this needs to be added

permissions: ['clipboard-read', 'clipboard-write'], // Allows clipboard read and write access