WordPress.org

Make WordPress Core

Opened 11 months ago

Closed 11 months ago

Last modified 10 months ago

#52141 closed feature request (fixed)

Make test coverage reports easier to digest

Reported by: jorbin Owned by: jorbin
Milestone: 5.7 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch
Focuses: Cc:

Description

Right now, code coverage reports are uploaded as artifacts to the github action. This means that in order to view the coverage reports, you need to download, unzip, and then run a local server which can be a bit cumbersome. It would be better if we could more easily view the codecoverage

Change History (14)

#1 @jorbin
11 months ago

One idea of a place we can put the code coverage is to upload it to a new repo that uses github pages. https://github.com/marketplace/actions/push-a-file-to-another-repository can be used to push the html files.

#2 @desrosj
11 months ago

  • Milestone changed from Awaiting Review to 5.7

I think using a repo for the time being could work well until we find an external service we can utilize.

#3 @swissspidy
11 months ago

I've used Codecov and Coveralls successfully in the past. Rather easy to set up with PHPUnit & GitHub Actions, although I do recommend a good config to reduce noise.

#4 @desrosj
11 months ago

@swissspidy do you recommend one over the other?

We would definitely not generate coverage on every commit (daily would probably be more appropriate). Codecov documents support for this using Flags, but I couldn't find an equivalent on Coveralls.

Both seem to be 100% free for open source projects, and both have an officially maintained GitHub Action.

#5 @swissspidy
11 months ago

I'd lean towards Codecov as it does have more powerful features like flags (e.g. to separate unit tests and integration tests) and merging reports (e.g. to merge multisite and non-multisite test coverage reports, or to merge JS and PHP coverage reports), which are useful when you run multiple tests.

#6 @desrosj
11 months ago

Great! Let's try this out in the new year. Anything that we can add to help us prevent blunders is a good thing.

This ticket was mentioned in PR #833 on WordPress/wordpress-develop by aaronjorbin.


11 months ago

  • Keywords has-patch added

Better visualize our code coverage in order to avoid blunders

This will likely take a while to get working correctly

See: https://core.trac.wordpress.org/ticket/52141

#9 @jorbin
11 months ago

  • Owner set to jorbin
  • Resolution set to fixed
  • Status changed from new to closed

In 49903:

Build/Test: Send Code Coverage reports to Codecov.

Help visualize and explore code coverage reports to avoid blunders.

Props desrosj, swissspidy.
Fixes #52141.

#10 @desrosj
11 months ago

Just following up that the coverage reports can be found here: https://codecov.io/gh/WordPress/wordpress-develop.

#11 @desrosj
11 months ago

In 49931:

Build/Test Tools: Change the frequency of code coverage reporting.

Before the coverage reports were submitted to Codecov.io, HTML coverage reports were compressed into ZIP files and uploaded to the workflow run as an artifact. A weekly schedule was chosen to run this workflow because generating a coverage report is more time consuming, and the resulting reports are quite large (~150-200MB each).

This changes the schedule for the code coverage workflow from weekly to daily and eliminates the ZIP artifacts that were previously generated. This will ensure the code coverage data found at https://codecov.io/gh/WordPress/wordpress-develop is relatively accurate on any given day of the week without needlessly consuming artifact storage.

Props jorbin.
See #50401, #52141.

#12 @desrosj
11 months ago

#46373 was marked as a duplicate.

This ticket was mentioned in Slack in #core by sergey. View the logs.


10 months ago

Note: See TracTickets for help on using tickets.