WordPress.org
Get WordPress

Make WordPress Core

Opened 2 years ago

Last modified 2 years ago

#56087 new enhancement

Configure Codecov status checks

Reported by: desrosj's profile desrosj Owned by:
Milestone: Awaiting Review Priority: low
Severity: normal Version:
Component: Build/Test Tools Keywords:
Focuses: Cc:

Description

There are two ​status checks associated with Codecov reporting being attached to commits: ​project and ​patch.

Patch status only measures the lines adjusted in the commit where project measures the overall project coverage and compares the before and after for the current commit.

Because a coverage report is not generated for every commit, the project check sometimes attaches misleading information to a commit when it does run. For example, see ​this commit. No files in the src directory were changed, but the test coverage is reported as declining.

It seems that any decline in coverage is also considered a failure.

Change History (1)

#1 @desrosj
2 years ago

I think there are a few things that could possibly be changed (listed in no particular order and each being an independent change).

  • Coverage reports could be submitted on every commit instead of once per day on a schedule, and when any related files are changed. This would make
  • Increase the threshold to a value other than 0% (maybe 1-3%). This would allow some level of coverage decline.
  • Disable the project status check and only run the patch check.
Note: See TracTickets for help on using tickets.