Add code coverage tracking

[pento]

To help track what we're testing, code coverage is useful.

Let's investigate the services that can provide this.

[pento]

46373.diff​ gets codecov (kind of) working.

There are some unit test failures when it runs, though: ​https://travis-ci.org/WordPress/wordpress-develop/jobs/499749838

Results: ​https://codecov.io/gh/WordPress/wordpress-develop/branch/try%2Fcodecov

@whyisjake, @joyously: y'all were going to look at this, here's a starting point. You should be able to connect services to your personal GitHub accounts for testing, but let me know if you need something connected to the WordPress organisation.

[whyisjake]

Accidentally started on this yesterday with a PR to core...

​https://github.com/WordPress/wordpress-develop/pull/42

Thanks for kicking this off @pento.

[johnbillion]

Coverage reports in PHPUnit require Xdebug, which we disable on Travis for performance reasons. Do these coverage reports actually work?

[pento]

46373.diff​ uses phpdbg instead of xdebug, because it's a whole lot faster.

[jorbin]

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

[desrosj]

Somewhat related: #39265, #39237.

The patch needs a refresh to account for the tests now being executed within the local Docker environment.

[netweb]

Replying to jorbin:

I’m removing from 5.2, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

Same, I’m removing from 5.3, but if there is traction this could come back in as a tool chain request like this can happen at just about any time.

[jrf]

I'd highly recommend for this ticket to be revisited for WP 5.6.

As explained in more detail in #50902, there are significant changes in PHP 8.0 which we will only be able to detect breakage from via unit tests.

Knowing which code is not, or insufficiently, covered by tests will allow a more focused approach for expanding the test suite to ensure compatibility with PHP 8.0.

Regarding CI integration: only one build running all tests is needed with code coverage enabled.
The Xdebug/PHPUnit combination has received lots of work over the past year or so, so with PHPUnit 9.x, it is likely that using Xdebug (enabled only for that one build) will be the fastest option, though testing it is recommended.

Xdebug will also allow for enabling the - far more accurate - path and branch coverage reports as introduced in PHPUnit 9.3.x, though those do make the code coverage reporting extremely slow, so should probably only be used locally. For more information, see: ​https://doug.codes/php-code-coverage

[ayeshrajans]

Having switched pretty much all my projects to use pcov, I'd like to propose to use pcov as well. It's an extension focused on code coverage alone (unlike xdebug 2.x, but similar to unrelesed xdebug 3.x with code coverage mode). PHPUnit 9.3 supports pcov too.

[helen]

Moving this out into 5.7, it's possible it'll end up addressed during the 5.6 cycle but I don't want to be tied to it and we're doing a lot of work migrating off of Travis right now so will have a whole new set of considerations to make.

[desrosj]

I had missed this ticket before opening #52141. Since this ticket also handles adding code coverage reporting, let's close this out and follow up tickets can be created to make changes.