Make WordPress Core

Opened 3 years ago

Closed 13 months ago

#46373 closed task (blessed) (duplicate)

Add code coverage tracking

Reported by: pento Owned by:
Milestone: Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch needs-refresh
Focuses: Cc:

Description

To help track what we're testing, code coverage is useful.

Let's investigate the services that can provide this.

Attachments (1)

46373.diff (3.6 KB) - added by pento 3 years ago.

Download all attachments as: .zip

Change History (17)

@pento
3 years ago

#1 @pento
3 years ago

46373.diff gets codecov (kind of) working.

There are some unit test failures when it runs, though: https://travis-ci.org/WordPress/wordpress-develop/jobs/499749838

Results: https://codecov.io/gh/WordPress/wordpress-develop/branch/try%2Fcodecov

@whyisjake, @joyously: y'all were going to look at this, here's a starting point. You should be able to connect services to your personal GitHub accounts for testing, but let me know if you need something connected to the WordPress organisation.

#2 @whyisjake
3 years ago

Accidentally started on this yesterday with a PR to core...

https://github.com/WordPress/wordpress-develop/pull/42

Thanks for kicking this off @pento.

#3 @johnbillion
3 years ago

Coverage reports in PHPUnit require Xdebug, which we disable on Travis for performance reasons. Do these coverage reports actually work?

#4 @pento
3 years ago

46373.diff uses phpdbg instead of xdebug, because it's a whole lot faster.

#5 follow-up: @jorbin
3 years ago

  • Milestone changed from 5.2 to 5.3

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

This ticket was mentioned in Slack in #core by david.baumwald. View the logs.


2 years ago

#7 @desrosj
2 years ago

  • Keywords has-patch needs-refresh added

Somewhat related: #39265, #39237.

The patch needs a refresh to account for the tests now being executed within the local Docker environment.

#8 in reply to: ↑ 5 @netweb
2 years ago

  • Milestone changed from 5.3 to 5.4

Replying to jorbin:

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

Same, I’m removing from 5.3, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

#9 @SergeyBiryukov
2 years ago

  • Milestone changed from 5.4 to Future Release

#10 @jrf
18 months ago

  • Milestone changed from Future Release to 5.6

I'd highly recommend for this ticket to be revisited for WP 5.6.

As explained in more detail in #50902, there are significant changes in PHP 8.0 which we will only be able to detect breakage from via unit tests.

Knowing which code is not, or insufficiently, covered by tests will allow a more focused approach for expanding the test suite to ensure compatibility with PHP 8.0.

Regarding CI integration: only one build running all tests is needed with code coverage enabled.
The Xdebug/PHPUnit combination has received lots of work over the past year or so, so with PHPUnit 9.x, it is likely that using Xdebug (enabled only for that one build) will be the fastest option, though testing it is recommended.

Xdebug will also allow for enabling the - far more accurate - path and branch coverage reports as introduced in PHPUnit 9.3.x, though those do make the code coverage reporting extremely slow, so should probably only be used locally. For more information, see: https://doug.codes/php-code-coverage

This ticket was mentioned in Slack in #core-site-health by jrf. View the logs.


17 months ago

#12 @ayeshrajans
17 months ago

Having switched pretty much all my projects to use pcov, I'd like to propose to use pcov as well. It's an extension focused on code coverage alone (unlike xdebug 2.x, but similar to unrelesed xdebug 3.x with code coverage mode). PHPUnit 9.3 supports pcov too.

Last edited 17 months ago by ayeshrajans (previous) (diff)

This ticket was mentioned in Slack in #core by helen. View the logs.


15 months ago

#14 @helen
15 months ago

  • Milestone changed from 5.6 to 5.7

Moving this out into 5.7, it's possible it'll end up addressed during the 5.6 cycle but I don't want to be tied to it and we're doing a lot of work migrating off of Travis right now so will have a whole new set of considerations to make.

This ticket was mentioned in Slack in #core by hellofromtonya. View the logs.


13 months ago

#16 @desrosj
13 months ago

  • Milestone 5.7 deleted
  • Resolution set to duplicate
  • Status changed from new to closed

I had missed this ticket before opening #52141. Since this ticket also handles adding code coverage reporting, let's close this out and follow up tickets can be created to make changes.

Note: See TracTickets for help on using tickets.