Make WordPress Core

Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#52141 closed feature request (fixed)

Make test coverage reports easier to digest

Reported by: jorbin's profile jorbin Owned by: jorbin's profile jorbin
Milestone: 5.7 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch
Focuses: Cc:


Right now, code coverage reports are uploaded as artifacts to the github action. This means that in order to view the coverage reports, you need to download, unzip, and then run a local server which can be a bit cumbersome. It would be better if we could more easily view the codecoverage

Change History (14)

#1 @jorbin
3 years ago

One idea of a place we can put the code coverage is to upload it to a new repo that uses github pages. can be used to push the html files.

#2 @desrosj
3 years ago

  • Milestone changed from Awaiting Review to 5.7

I think using a repo for the time being could work well until we find an external service we can utilize.

#3 @swissspidy
3 years ago

I've used Codecov and Coveralls successfully in the past. Rather easy to set up with PHPUnit & GitHub Actions, although I do recommend a good config to reduce noise.

#4 @desrosj
3 years ago

@swissspidy do you recommend one over the other?

We would definitely not generate coverage on every commit (daily would probably be more appropriate). Codecov documents support for this using Flags, but I couldn't find an equivalent on Coveralls.

Both seem to be 100% free for open source projects, and both have an officially maintained GitHub Action.

#5 @swissspidy
3 years ago

I'd lean towards Codecov as it does have more powerful features like flags (e.g. to separate unit tests and integration tests) and merging reports (e.g. to merge multisite and non-multisite test coverage reports, or to merge JS and PHP coverage reports), which are useful when you run multiple tests.

#6 @desrosj
3 years ago

Great! Let's try this out in the new year. Anything that we can add to help us prevent blunders is a good thing.

This ticket was mentioned in PR #833 on WordPress/wordpress-develop by aaronjorbin.

3 years ago

  • Keywords has-patch added

Better visualize our code coverage in order to avoid blunders

This will likely take a while to get working correctly


#9 @jorbin
3 years ago

  • Owner set to jorbin
  • Resolution set to fixed
  • Status changed from new to closed

In 49903:

Build/Test: Send Code Coverage reports to Codecov.

Help visualize and explore code coverage reports to avoid blunders.

Props desrosj, swissspidy.
Fixes #52141.

#10 @desrosj
3 years ago

Just following up that the coverage reports can be found here:

#11 @desrosj
3 years ago

In 49931:

Build/Test Tools: Change the frequency of code coverage reporting.

Before the coverage reports were submitted to, HTML coverage reports were compressed into ZIP files and uploaded to the workflow run as an artifact. A weekly schedule was chosen to run this workflow because generating a coverage report is more time consuming, and the resulting reports are quite large (~150-200MB each).

This changes the schedule for the code coverage workflow from weekly to daily and eliminates the ZIP artifacts that were previously generated. This will ensure the code coverage data found at is relatively accurate on any given day of the week without needlessly consuming artifact storage.

Props jorbin.
See #50401, #52141.

#12 @desrosj
3 years ago

#46373 was marked as a duplicate.

This ticket was mentioned in Slack in #core by sergey. View the logs.

3 years ago

Note: See TracTickets for help on using tickets.