WordPress.org

Make WordPress Core

Opened 4 weeks ago

Closed 3 weeks ago

Last modified 8 days ago

#52141 closed feature request (fixed)

Make test coverage reports easier to digest

Reported by: jorbin Owned by: jorbin
Milestone: 5.7 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch
Focuses: Cc:

Description

Right now, code coverage reports are uploaded as artifacts to the github action. This means that in order to view the coverage reports, you need to download, unzip, and then run a local server which can be a bit cumbersome. It would be better if we could more easily view the codecoverage

Change History (13)

#1 @jorbin
4 weeks ago

One idea of a place we can put the code coverage is to upload it to a new repo that uses github pages. https://github.com/marketplace/actions/push-a-file-to-another-repository can be used to push the html files.

#2 @desrosj
4 weeks ago

  • Milestone changed from Awaiting Review to 5.7

I think using a repo for the time being could work well until we find an external service we can utilize.

#3 @swissspidy
4 weeks ago

I've used Codecov and Coveralls successfully in the past. Rather easy to set up with PHPUnit & GitHub Actions, although I do recommend a good config to reduce noise.

#4 @desrosj
4 weeks ago

@swissspidy do you recommend one over the other?

We would definitely not generate coverage on every commit (daily would probably be more appropriate). Codecov documents support for this using Flags, but I couldn't find an equivalent on Coveralls.

Both seem to be 100% free for open source projects, and both have an officially maintained GitHub Action.

#5 @swissspidy
4 weeks ago

I'd lean towards Codecov as it does have more powerful features like flags (e.g. to separate unit tests and integration tests) and merging reports (e.g. to merge multisite and non-multisite test coverage reports, or to merge JS and PHP coverage reports), which are useful when you run multiple tests.

#6 @desrosj
4 weeks ago

Great! Let's try this out in the new year. Anything that we can add to help us prevent blunders is a good thing.

This ticket was mentioned in PR #833 on WordPress/wordpress-develop by aaronjorbin.


4 weeks ago

  • Keywords has-patch added

Better visualize our code coverage in order to avoid blunders

This will likely take a while to get working correctly

See: https://core.trac.wordpress.org/ticket/52141

#9 @jorbin
3 weeks ago

  • Owner set to jorbin
  • Resolution set to fixed
  • Status changed from new to closed

In 49903:

Build/Test: Send Code Coverage reports to Codecov.

Help visualize and explore code coverage reports to avoid blunders.

Props desrosj, swissspidy.
Fixes #52141.

#10 @desrosj
12 days ago

Just following up that the coverage reports can be found here: https://codecov.io/gh/WordPress/wordpress-develop.

#11 @desrosj
12 days ago

In 49931:

Build/Test Tools: Change the frequency of code coverage reporting.

Before the coverage reports were submitted to Codecov.io, HTML coverage reports were compressed into ZIP files and uploaded to the workflow run as an artifact. A weekly schedule was chosen to run this workflow because generating a coverage report is more time consuming, and the resulting reports are quite large (~150-200MB each).

This changes the schedule for the code coverage workflow from weekly to daily and eliminates the ZIP artifacts that were previously generated. This will ensure the code coverage data found at https://codecov.io/gh/WordPress/wordpress-develop is relatively accurate on any given day of the week without needlessly consuming artifact storage.

Props jorbin.
See #50401, #52141.

#12 @desrosj
10 days ago

#46373 was marked as a duplicate.

Note: See TracTickets for help on using tickets.