Ensure HTML Processor step adds to element_queue

[jonsurrell]

There are some cases where the step functions in the HTML processor return true, but do not add anything to the element queue. When step returns true, it should indicate the progress has been made progressing the document and there are tokens to process.

​There is a todo in code right now describing this situation:

In some cases, probably related to the adoption agency
algorithm, this call to step() doesn't create any new
events. Calling it again creates them. Figure out why
this is and if it's inherent or if it's a bug. Looping
until there are events or until there are no more
tokens works in the meantime and isn't obviously wrong.

The HTML specification does not close HTML or BODY tags (pop them off the stack of open elements) when their tag closers are encountered. The HTML processor correctly handles this behavior, however it "pauses" by returning true from the step functions. This behavior is incorrect behavior for step because there are no additional tokens to be processed (nothing has been added to the element queue).

When encountering these closers, internal state is updated correctly, but the parser should continue processing without pausing.

Trac ticket: https://core.trac.wordpress.org/ticket/62583

• Proceed past BODY and HTML close tags
• DROPME: Add exception in failure case
• Do not return true when closing HTML (but not popping off stack)
• Do not return from step if active formatting POP did not occur
• Do not return from foreign tag closer if stack not modified

Trac ticket:

[gziolo]

In 59500:

HTML API: Step past closing HTML, BODY tags

The HTML specification does not close HTML or BODY tags (pop them off the stack of open elements) when their tag closers are encountered. The HTML processor correctly handled this behavior, however it incorrectly "paused" by returning true from the step functions.

Props jonsurrell, dmsnell, gziolo.
Fixes #62583.

Committed with https://core.trac.wordpress.org/changeset/59500.

[jonsurrell]

​PR 7904 contains more cases for consideration (although it's not yet ready for review). Reopening.

[audrasjb]

As per today's bugscrub: This ticket was committed and reopened to handle further use case, so we'll keep it open in the milestone for now, but can we have a status update on the remaining work @gziolo @jonsurrell? It would be nice to get this closed before beta 1 next week. Thanks!

[jonsurrell]

There is some valid work in ​PR 7904, but I don't expect that work to be finished or merged in 6.8. The changes in that PR are not essential and are fine to bump to a future release.

I'm not sure what the preferred approach is. Would it be best to close this ticket with the changes that are already merged, then open another ticket with the ongoing work is ready to continue?

[audrasjb]

Yes, let's close this ticket as fixed then. Follow-up ticket can be opened for further changes.