Make WordPress Core

Opened 2 years ago

Last modified 2 years ago

#56545 assigned defect (bug)

Add a gutenberg plugin enabled run to PHPUnit Tests workflow

Reported by: jorbin's profile jorbin Owned by: desrosj's profile desrosj
Milestone: Awaiting Review Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch close
Focuses: Cc:

Description

[54118] caused a fatal if you are running trunk and the gutenberg plugin. This isn't the first time that something has happened when backporting code. I think we may want to add a run of the automated PHP tests with the current stable gutenberg plugin enabled to prevent this. I think we should only worry about trunk for this.

Change History (4)

#1 @desrosj
2 years ago

  • Owner set to desrosj
  • Status changed from new to assigned

This ticket was mentioned in PR #3306 on WordPress/wordpress-develop by desrosj.


2 years ago
#2

  • Keywords has-patch added

#3 @desrosj
2 years ago

  • Keywords reporter-feedback added

I've created a PR with a proof of concept that adds a job activating the plugin before running tests. I used trunk for the Gutenberg repo because ideally issues like the one described should be caught before a release of the plugin goes out.

I think in theory testing this sounds great, but I'm not sure it's as simple as installing and activating the Gutenberg plugin before running tests.

As Gutenberg releases new versions, the way various blocks are processed, displayed, etc. can change. If the plugin changes any of the outcomes for tests looking for specific outcomes in Core's PHPUnit test suite it will result in failures.

If the block editor packages were published and updated in trunk more often, this wouldn't be a problem. But it's not uncommon for 5-15 Gutenberg releases to be merged into trunk at once.

What about creating a new test group for the Gutenberg plugin (similar to how we test for xDebug) with a small handful of tests just to confirm there are no fatal errors when the plugin is active in trunk (and maybe perform a few other assertions)?

I'm also thinking that this may be better suited for the Gutenberg repository. It could run on merge, and the plugin should have unit tests that are more accurate than the ones in Core (related #56266, #56611).

#4 @desrosj
2 years ago

  • Keywords close added; reporter-feedback removed
  • Milestone changed from 6.1 to Awaiting Review

I'm punting this one to Awaiting Review with RC1 tomorrow. My findings so far are that the Gutenberg repo actually does test against trunk, but for some reason it only detects these types of issues in release/##.# branches.

We need to figure out why trunk of Gutenberg does not detect these fatal errors to make a recommendation. It wouldn't hurt to _also_ run the tests with Gutenberg in the core test suite, but in the Gutenberg repo may be the better location.

Note: See TracTickets for help on using tickets.