WordPress.org

Make WordPress Core

Opened 10 months ago

Closed 9 months ago

Last modified 8 months ago

#52034 closed defect (bug) (fixed)

Generate a test coverage report with a GitHub Action workflow

Reported by: desrosj Owned by: desrosj
Milestone: 5.7 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch
Focuses: Cc:

Description

To help increase test coverage, let's use GitHub actions to generate a nightly code coverage report.

While this is not the only metric to consider, introducing some reporting is better than no reporting, and hopefully will shed some light into areas of core with little to no automated testing.

Change History (8)

This ticket was mentioned in Slack in #core by desrosj. View the logs.


10 months ago

This ticket was mentioned in PR #807 on WordPress/wordpress-develop by desrosj.


10 months ago

  • Keywords has-patch added

This generates a code coverage report on a nightly cron with a GitHub Action workflow (the push event will be removed when committed to trunk.

For now, reports are only generated for single site, multisite, and REST API results, as those are the only setups/groups that would generate any meaningful reports.

Trac ticket: https://core.trac.wordpress.org/ticket/52034.

#3 @desrosj
10 months ago

  • Resolution set to fixed
  • Status changed from assigned to closed

In 49797:

Build/Test Tools: Allow code coverage reports to be generated when running tests as a multisite.

This adds the information needed to generate a code coverage report to the multisite ruleset.

Fixes #52034.

#4 @desrosj
10 months ago

  • Resolution fixed deleted
  • Status changed from closed to reopened

[49797] was meant for #52045.

This ticket was mentioned in Slack in #core by sergey. View the logs.


9 months ago

#6 @prbot
9 months ago

desrosj commented on PR #807:

Since the default retention period is 90 days, are there any follow-up plans with those uploaded reports before they get deleted?

I accidentally nested my response here to a different comment for a specific line of code:

For now, the idea is just to surface the report so anyone interested in writing tests can find an area to focus on.

We could submit the reports to a third party service. Do you have any that you recommend?

#7 @desrosj
9 months ago

  • Resolution set to fixed
  • Status changed from reopened to closed

In 49834:

Build/Test Tools: Generate a code coverage report using GitHub Actions.

This introduces a new GitHub Action workflow to generate a code coverage report when running the PHPUnit test suite as both a single and multisite install.

The workflow will run once per week on Sunday at 00:00 UTC.

For now, the reports are not submitted anywhere, but they will be uploaded to the workflow run as ZIP file artifacts where they will persist for 90 days.

Making code coverage reports more readily available will hopefully better highlight areas of Core with poor coverage, and encourage more contributors to help increase test coverage of those areas.

Props ocean90, johnbillion.
Fixes #52034.

This ticket was mentioned in Slack in #core by sergey. View the logs.


8 months ago

Note: See TracTickets for help on using tickets.