Make WordPress Core

Opened 4 years ago

Closed 3 years ago

Last modified 3 years ago

#52786 closed defect (bug) (fixed)

Fix the code coverage reporting

Reported by: jrf's profile jrf Owned by: desrosj's profile desrosj
Milestone: 5.8 Priority: normal
Severity: normal Version: 5.7
Component: Build/Test Tools Keywords: has-patch has-unit-tests
Focuses: Cc:

Description

As reported by Juliette on #51734:


So... this broke the code coverage reporting as it seems it was forgotten to update the code coverage configuration in the phpunit.xml.dist file.

Last commit on which code coverage was run is commit [50436]
See: https://app.codecov.io/gh/WordPress/wordpress-develop/

It can also been seen in the GH Actions code coverage runs which report:

==> Reading reports
    - file not found at wp-code-coverage-single-clover-cded2eb52aa71a398d1d0cd87f8674b383512c0b.xml
--> No coverage data found.

I'll upload the patch which should fix this in a moment and which basically reverts [48847].
Note: this patch will also need to be backported.

I'll also upload a second patch (optional) which would (should) allow for manually triggering code coverage runs. Committing that patch would allow us, in the future, to test that specific build changes do not break the code coverage build by manually triggering a build on a branch/PR.

Attachments (3)

52786-allow-manual-triggering-codecov-workflow.patch (759 bytes) - added by desrosj 4 years ago.
52786-fix-codecoverage-config.patch (7.3 KB) - added by desrosj 4 years ago.
52786.diff (574 bytes) - added by desrosj 4 years ago.

Download all attachments as: .zip

Change History (14)

@desrosj
4 years ago

#1 @jrf
4 years ago

  • Keywords has-patch added; needs-patch removed

#2 @desrosj
4 years ago

  • Milestone changed from Awaiting Review to 5.8

I copied the original patches over from #51734 to have them in one place.

52786.diff is an expanded version of 52786-allow-manual-triggering-codecov-workflow.patch that will trigger a build when the phpunit.xml.dist file is updated. Since this file configures how the test suite runs and how code coverage reports are compiled, it makes sense to run the workflow every time this is updated.

CC @johnbillion.

#3 @jrf
4 years ago

Own ticket ? Okay, fair enough.

In the 52786.diff patch, I think it would be useful to add the tests/phpunit/multisite.xml file and the code coverage workflow file .github/workflows/test-coverage.yml as files which should trigger the workflow when changed.

To confirm that my original patch 52786-fix-codecoverage-config.patch actually fixes the issue, see this test run I did:
https://codecov.io/gh/WordPress/wordpress-develop/tree/trac-51734%2Ffix-code-coverage-reporting/src

This ticket was mentioned in PR #1092 on WordPress/wordpress-develop by jrfnl.


4 years ago
#4

  • Keywords has-unit-tests added

#5 @johnbillion
4 years ago

Huh, I created this ticket but Trac timed out and I thought it didn't get saved so I went and did something else. Thanks for following up.

#6 @desrosj
3 years ago

  • Owner set to desrosj
  • Resolution set to fixed
  • Status changed from new to closed

In 50592:

Build/Test Tools: Fix code coverage reporting to generate report from src.

Since [50441-50442] switched the test workflows to run from src instead of build, code coverage reporting has stopped working. This was caused by the code coverage configuration continuing to reference build. This corrects the configuration so coverage reporting can resume.

This change also introduces the workflow_dispatch event to the workflow, which will allow committers to manually run the workflow when desired. For example, to confirm changes to the test suite do not break reporting.

This also adds the phpunit.xml.dist and tests/phpunit/multisite.xml files to the paths list. Since these files are responsible for configuring the test suite and code coverage reporting, any changes to them should verify that no problems were introduced.

Props jrf, johnbillion.
Fixes #52786. See #51734.

jrfnl commented on PR #1092:


3 years ago
#7

Closing as merged.

#8 @desrosj
3 years ago

In 50595:

Build/Test Tools: Run code coverage workflow when the file is updated.

When the workflow file is updated, it should always be run to verify changes.

Follow up to [50592].

See #52786.

#9 @desrosj
3 years ago

In 50601:

Build/Test Tools: Backport several build tool fixes & improvements.

This backports a handful of test fixes related to generating code coverage reporting, as well as some devDependency updates.

Backports [50591-50595,50597-50598] to the 5.7 branch.
See #52624, #52786, #51734.

#10 @desrosj
3 years ago

In 50602:

Build/Test Tools: Backport GitHub Action and build improvements to the 5.6 branch.

This backports several build and test tool improvements to the 5.6 branch. Most notably, this includes:

  • The changes required to allow each workflow to be triggered by the workflow_dispatch event so that tests can be run on a schedule [50590].
  • The ability to run PHPUnit tests from src instead of build [50441-50443].
  • Splitting single site and multisite tests into parallel jobs [50379].
  • Split slow tests into separate, parallel jobs for PHP 5.6 [50444].
  • Better branch and path scoping for GitHub Action workflows when running on pull_request [50432,50479].
  • Several devDependency updates.

Merges [50267,50299,50379,50387,50413,50416,50432,50435-50436,50441-50444,50446,50473-50474,50476,50479,50485-50487,50545,50579,50590,50592,50598] to the 5.6 branch.
See #50401, #51734, #51801, #51802, #52548, #52608, #52612, #52623, #52624, #52625, #52645, #52653, #52658, #52660, #52667, #52786.

#11 @desrosj
3 years ago

In 50603:

Build/Test Tools: Backport GitHub Action and build improvements to the 5.5 branch.

This backports several build and test tool improvements to the 5.5 branch. Most notably, this includes:

  • The changes required to allow each workflow to be triggered by the workflow_dispatch event so that tests can be run on a schedule [50590].
  • The ability to run PHPUnit tests from src instead of build [50441-50443].
  • Splitting single site and multisite tests into parallel jobs [50379].
  • Split slow tests into separate, parallel jobs for PHP 5.6 [50444].
  • Better branch and path scoping for GitHub Action workflows when running on pull_request [50432,50479].
  • Several devDependency updates.

Merges [50267,50299,50379,50387,50413,50416,50432,50435-50436,50441-50444,50446,50473-50474,50476,50479,50485-50487,50545,50579,50590,50592,50598] to the 5.5 branch.
See #50401, #51734, #51801, #51802, #52548, #52608, #52612, #52623, #52624, #52625, #52645, #52653, #52658, #52660, #52667, #52786.

Note: See TracTickets for help on using tickets.