Block Processor: Extracting full block missing nested content.

[dmsnell]

The behavior of WP_Block_Processor::extract_full_block_and_advance() should produce an identical output to what parse_blocks() would return on the same substring of input.

It has difficulties in certain nested contexts. In the following case, the trailing </ul> is missing. In several test cases the trailing HTML span is missing, at least from innerContent.

<!-- wp:columns --><ul><!-- wp:column --><li>Hi</li><!-- /wp:column --></ul><!-- /wp:columns -->

There is likely a problem in the recursive part of the algorithm, or in recognizing HTML spans, or in performing proper stack accounting, or all three.

Reported by @jonsurrell

Trac ticket: Core-64537

The behavior of WP_Block_Processor::extract_full_block_and_advance() should produce an identical output to what parse_blocks() would return on the same substring of input.

It has difficulties in certain nested contexts. In the following case, the trailing </ul> is missing. In several test cases the trailing HTML span is missing, at least from innerContent.

<ul><li>Hi</li></ul><!-- /wp:columns>

There is likely a problem in the recursive part of the algorithm, or in recognizing HTML spans, or in performing proper stack accounting, or all three.

Only one of the tests seem to have any block attributes. Is this intentional, if not would it make sense to make sure that more of those don't affect the block processor vs. parse blocks?

I can add more, @aaronjorbin, but I think I only really felt like one was essential for this specific bug, mostly to ensure that it works _at all_ vs. having a big gap. The attribute testing is handled by other unit tests which parse block delimiters, but I suppose here it could be useful to ensure that attributes are properly assigned in nested contexts (vs. being assigned to a block’s containing outer block)

@aaronjorbin in the latest push I’ve added tests that I think should be sufficient to ensure that block attribute parsing doesn’t run amok. That is, four levels deep I’m testing _no attributes_, _empty attributes_, _attributes_, and _attributes with the same key but different value as its parent block_.

As a side note to this change, I intentionally don’t want to verify the error-handling behavior of malformed posts. That’s a more complicated subject, because the default parser is technically not spec-compliant, and if we assert those behaviors on the current parse_blocks() we will lock ourselves in to that ad-hoc error-resolution.

Coincidentally, I am proposing a change in error-handling in wordpress/Gutenberg#74427 in the default parser to resolve a longstanding memory-explosion time bomb.

[dmsnell]

In 61509:

Blocks: Ensure extract_full_block_and_advance() matches parse_blocks()

The behavior of WP_Block_Processor::extract_full_block_and_advance() should produce an identical output to what parse_blocks() would return on the same substring of input.

Unfortunately, when HTML spans followed inner blocks, they were being omitted in the output parse tree. This was due to an omission in the original code which would look for those blocks before advancing again after calling extract_full_block_and_advance() recursively.

This patch adds the missing check and resolves the discrepancy.

Developed in: ​https://github.com/WordPress/wordpress-develop/pull/10769
Discussed in: https://core.trac.wordpress.org/ticket/64538

Follow-up to [60939].

Props dmsnell, jonsurrell, jorbin.
Fixes #64537.

[dmsnell]

Reopening for review for 6.9.1 backport.

[jonsurrell]

[61509] looks good to backport to 6.9.

[jorbin]

Setup a PR to ensure all tests pass with this on the 6.9 branch and there are no complications with a backport.

# This is a backport test PR for [61509]

The behavior of WP_Block_Processor::extract_full_block_and_advance() should produce an identical output to what parse_blocks() would return on the same substring of input.

Unfortunately, when HTML spans followed inner blocks, they were being omitted in the output parse tree. This was due to an omission in the original code which would look for those blocks before advancing again after calling extract_full_block_and_advance() recursively.

This patch adds the missing check and resolves the discrepancy.

Developed in: ​https://github.com/WordPress/wordpress-develop/pull/10769.
Discussed in: https://core.trac.wordpress.org/ticket/64538.

Trac ticket: https://core.trac.wordpress.org/ticket/64537

This diff should match d3068ae now.

[jonsurrell]

In 61530:

Blocks: Ensure extract_full_block_and_advance() matches parse_blocks()

The behavior of WP_Block_Processor::extract_full_block_and_advance() should produce an identical output to what parse_blocks() would return on the same substring of input.

Unfortunately, when HTML spans followed inner blocks, they were being omitted in the output parse tree. This was due to an omission in the original code which would look for those blocks before advancing again after calling extract_full_block_and_advance() recursively.

This patch adds the missing check and resolves the discrepancy.

Developed in: ​https://github.com/WordPress/wordpress-develop/pull/10769
Discussed in: https://core.trac.wordpress.org/ticket/64538

Follow-up to [60939].

Reviewed by jonsurrell.
Merges [61509] to the 6.9 branch.

Props dmsnell, jonsurrell, jorbin.
Fixes #64537.