WordPress.org

Make WordPress Core

Opened 23 months ago

Closed 2 weeks ago

#46373 closed task (blessed) (duplicate)

Add code coverage tracking

Reported by: pento Owned by:
Milestone: Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch needs-refresh
Focuses: Cc:

Description

To help track what we're testing, code coverage is useful.

Let's investigate the services that can provide this.

Attachments (1)

46373.diff (3.6 KB) - added by pento 23 months ago.

Download all attachments as: .zip

Change History (17)

@pento
23 months ago

#1 @pento
23 months ago

46373.diff gets codecov (kind of) working.

There are some unit test failures when it runs, though: https://travis-ci.org/WordPress/wordpress-develop/jobs/499749838

Results: https://codecov.io/gh/WordPress/wordpress-develop/branch/try%2Fcodecov

@whyisjake, @joyously: y'all were going to look at this, here's a starting point. You should be able to connect services to your personal GitHub accounts for testing, but let me know if you need something connected to the WordPress organisation.

#2 @whyisjake
23 months ago

Accidentally started on this yesterday with a PR to core...

https://github.com/WordPress/wordpress-develop/pull/42

Thanks for kicking this off @pento.

#3 @johnbillion
22 months ago

Coverage reports in PHPUnit require Xdebug, which we disable on Travis for performance reasons. Do these coverage reports actually work?

#4 @pento
22 months ago

46373.diff uses phpdbg instead of xdebug, because it's a whole lot faster.

#5 follow-up: @jorbin
22 months ago

  • Milestone changed from 5.2 to 5.3

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

This ticket was mentioned in Slack in #core by david.baumwald. View the logs.


16 months ago

#7 @desrosj
16 months ago

  • Keywords has-patch needs-refresh added

Somewhat related: #39265, #39237.

The patch needs a refresh to account for the tests now being executed within the local Docker environment.

#8 in reply to: ↑ 5 @netweb
16 months ago

  • Milestone changed from 5.3 to 5.4

Replying to jorbin:

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

Same, I’m removing from 5.3, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

#9 @SergeyBiryukov
12 months ago

  • Milestone changed from 5.4 to Future Release

#10 @jrf
5 months ago

  • Milestone changed from Future Release to 5.6

I'd highly recommend for this ticket to be revisited for WP 5.6.

As explained in more detail in #50902, there are significant changes in PHP 8.0 which we will only be able to detect breakage from via unit tests.

Knowing which code is not, or insufficiently, covered by tests will allow a more focused approach for expanding the test suite to ensure compatibility with PHP 8.0.

Regarding CI integration: only one build running all tests is needed with code coverage enabled.
The Xdebug/PHPUnit combination has received lots of work over the past year or so, so with PHPUnit 9.x, it is likely that using Xdebug (enabled only for that one build) will be the fastest option, though testing it is recommended.

Xdebug will also allow for enabling the - far more accurate - path and branch coverage reports as introduced in PHPUnit 9.3.x, though those do make the code coverage reporting extremely slow, so should probably only be used locally. For more information, see: https://doug.codes/php-code-coverage

This ticket was mentioned in Slack in #core-site-health by jrf. View the logs.


5 months ago

#12 @ayeshrajans
4 months ago

Having switched pretty much all my projects to use pcov, I'd like to propose to use pcov as well. It's an extension focused on code coverage alone (unlike xdebug 2.x, but similar to unrelesed xdebug 3.x with code coverage mode). PHPUnit 9.3 supports pcov too.

Last edited 4 months ago by ayeshrajans (previous) (diff)

This ticket was mentioned in Slack in #core by helen. View the logs.


3 months ago

#14 @helen
3 months ago

  • Milestone changed from 5.6 to 5.7

Moving this out into 5.7, it's possible it'll end up addressed during the 5.6 cycle but I don't want to be tied to it and we're doing a lot of work migrating off of Travis right now so will have a whole new set of considerations to make.

This ticket was mentioned in Slack in #core by hellofromtonya. View the logs.


2 weeks ago

#16 @desrosj
2 weeks ago

  • Milestone 5.7 deleted
  • Resolution set to duplicate
  • Status changed from new to closed

I had missed this ticket before opening #52141. Since this ticket also handles adding code coverage reporting, let's close this out and follow up tickets can be created to make changes.

Note: See TracTickets for help on using tickets.