Make test coverage reports easier to digest

[jorbin]

Right now, code coverage reports are uploaded as artifacts to the github action. This means that in order to view the coverage reports, you need to download, unzip, and then run a local server which can be a bit cumbersome. It would be better if we could more easily view the codecoverage

[jorbin]

One idea of a place we can put the code coverage is to upload it to a new repo that uses github pages. ​https://github.com/marketplace/actions/push-a-file-to-another-repository can be used to push the html files.

[desrosj]

I think using a repo for the time being could work well until we find an external service we can utilize.

[swissspidy]

I've used Codecov and Coveralls successfully in the past. Rather easy to set up with PHPUnit & GitHub Actions, although I do recommend a good config to reduce noise.

[desrosj]

@swissspidy do you recommend one over the other?

We would definitely not generate coverage on every commit (daily would probably be more appropriate). Codecov documents support for this using ​Flags, but I couldn't find an equivalent on Coveralls.

Both seem to be 100% free for open source projects, and both have an officially maintained GitHub Action.

[swissspidy]

I'd lean towards Codecov as it does have more powerful features like ​flags (e.g. to separate unit tests and integration tests) and ​merging reports (e.g. to merge multisite and non-multisite test coverage reports, or to merge JS and PHP coverage reports), which are useful when you run multiple tests.

[desrosj]

Great! Let's try this out in the new year. Anything that we can add to help us prevent blunders is a good thing.

Better visualize our code coverage in order to avoid blunders

This will likely take a while to get working correctly

See: https://core.trac.wordpress.org/ticket/52141

[jorbin]

​https://codecov.io/github/aaronjorbin/wordpress-develop/commit/f73b27a7eb9e5f71de5ab7646a2616fba1da381b is an example of what we can get from codecov

[jorbin]

In 49903:

Build/Test: Send Code Coverage reports to Codecov.

Help visualize and explore code coverage reports to avoid blunders.

Props desrosj, swissspidy.
Fixes #52141.

[desrosj]

Just following up that the coverage reports can be found here: ​https://codecov.io/gh/WordPress/wordpress-develop.

[desrosj]

In 49931:

Build/Test Tools: Change the frequency of code coverage reporting.

Before the coverage reports were submitted to Codecov.io, HTML coverage reports were compressed into ZIP files and uploaded to the workflow run as an artifact. A weekly schedule was chosen to run this workflow because generating a coverage report is more time consuming, and the resulting reports are quite large (~150-200MB each).

This changes the schedule for the code coverage workflow from weekly to daily and eliminates the ZIP artifacts that were previously generated. This will ensure the code coverage data found at ​https://codecov.io/gh/WordPress/wordpress-develop is relatively accurate on any given day of the week without needlessly consuming artifact storage.

Props jorbin.
See #50401, #52141.

[desrosj]

#46373 was marked as a duplicate.

👍 https://core.trac.wordpress.org/changeset/49903