WordPress.org

Make WordPress Core

Opened 20 months ago

Last modified 7 weeks ago

#46373 new task (blessed)

Add code coverage tracking

Reported by: pento Owned by:
Milestone: 5.6 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch needs-refresh
Focuses: Cc:

Description

To help track what we're testing, code coverage is useful.

Let's investigate the services that can provide this.

Attachments (1)

46373.diff (3.6 KB) - added by pento 20 months ago.

Download all attachments as: .zip

Change History (13)

@pento
20 months ago

#1 @pento
20 months ago

46373.diff gets codecov (kind of) working.

There are some unit test failures when it runs, though: https://travis-ci.org/WordPress/wordpress-develop/jobs/499749838

Results: https://codecov.io/gh/WordPress/wordpress-develop/branch/try%2Fcodecov

@whyisjake, @joyously: y'all were going to look at this, here's a starting point. You should be able to connect services to your personal GitHub accounts for testing, but let me know if you need something connected to the WordPress organisation.

#2 @whyisjake
20 months ago

Accidentally started on this yesterday with a PR to core...

https://github.com/WordPress/wordpress-develop/pull/42

Thanks for kicking this off @pento.

#3 @johnbillion
19 months ago

Coverage reports in PHPUnit require Xdebug, which we disable on Travis for performance reasons. Do these coverage reports actually work?

#4 @pento
19 months ago

46373.diff uses phpdbg instead of xdebug, because it's a whole lot faster.

#5 follow-up: @jorbin
19 months ago

  • Milestone changed from 5.2 to 5.3

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

This ticket was mentioned in Slack in #core by david.baumwald. View the logs.


13 months ago

#7 @desrosj
13 months ago

  • Keywords has-patch needs-refresh added

Somewhat related: #39265, #39237.

The patch needs a refresh to account for the tests now being executed within the local Docker environment.

#8 in reply to: ↑ 5 @netweb
13 months ago

  • Milestone changed from 5.3 to 5.4

Replying to jorbin:

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

Same, I’m removing from 5.3, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

#9 @SergeyBiryukov
9 months ago

  • Milestone changed from 5.4 to Future Release

#10 @jrf
3 months ago

  • Milestone changed from Future Release to 5.6

I'd highly recommend for this ticket to be revisited for WP 5.6.

As explained in more detail in #50902, there are significant changes in PHP 8.0 which we will only be able to detect breakage from via unit tests.

Knowing which code is not, or insufficiently, covered by tests will allow a more focused approach for expanding the test suite to ensure compatibility with PHP 8.0.

Regarding CI integration: only one build running all tests is needed with code coverage enabled.
The Xdebug/PHPUnit combination has received lots of work over the past year or so, so with PHPUnit 9.x, it is likely that using Xdebug (enabled only for that one build) will be the fastest option, though testing it is recommended.

Xdebug will also allow for enabling the - far more accurate - path and branch coverage reports as introduced in PHPUnit 9.3.x, though those do make the code coverage reporting extremely slow, so should probably only be used locally. For more information, see: https://doug.codes/php-code-coverage

This ticket was mentioned in Slack in #core-site-health by jrf. View the logs.


8 weeks ago

#12 @ayeshrajans
7 weeks ago

Having switched pretty much all my projects to use pcov, I'd like to propose to use pcov as well. It's an extension focused on code coverage alone (unlike xdebug 2.x, but similar to unrelesed xdebug 3.x with code coverage mode). PHPUnit 9.3 supports pcov too.

Last edited 7 weeks ago by ayeshrajans (previous) (diff)
Note: See TracTickets for help on using tickets.