convert_smilies() fails on large tags

[podpirate]

I have an image with huge data-url src (1.6M) in the post content.
The post content is not displayed, but (with WP_DEBUG on) there is a php error instead.

PHP Error output:

 Warning: count(): Parameter must be an array or an object that implements Countable in /Users/joern/www/vhosts/wordpress.local/httpdocs/stable/wp-includes/formatting.php on line 3357

Som digging revealed, that preg_split() in function convert_smilies() fails with a PREG_RECURSION_LIMIT_ERROR.

IMHO there should a check for preg_last_error() whether preg_split() was successfull, and if not the function should just return its input.

Any thoughts on this?
I'd be happy to craft a patch.

[SergeyBiryukov]

Hi there, welcome back to WordPress Trac! Thanks for the ticket.

IMHO there should a check for preg_last_error() whether preg_split() was successfull, and if not the function should just return its input.

Makes sense to me :)

…o avoid issues in ticket #51019

Use preg_last_error() inside convert_smilies() to check for any preg_split errors before running count().
If there are errors, sends a PHP warning with the error message, then returns the original text.

Running count() on the array after there are preg_split errors can cause post content to not display (see ticket).

Trac ticket: https://core.trac.wordpress.org/ticket/51019

Hi @nathkrill! 👋

Thank you for your contribution to WordPress! 💖

It looks like this is your first pull request to wordpress-develop. Here are a few things to be aware of that may help you out!

No one monitors this repository for new pull requests. Pull requests must be attached to a Trac ticket to be considered for inclusion in WordPress Core. To attach a pull request to a Trac ticket, please include the ticket's full URL in your pull request description.

Pull requests are never merged on GitHub. The WordPress codebase continues to be managed through the SVN repository that this GitHub repository mirrors. Please feel free to open pull requests to work on any contribution you are making.

More information about how GitHub pull requests can be used to contribute to WordPress can be found in ​this blog post.

Please include automated tests. Including tests in your pull request is one way to help your patch be considered faster. To learn about WordPress' test suites, visit the ​Automated Testing page in the handbook.

If you have not had a chance, please review the ​Contribute with Code page in the ​WordPress Core Handbook.

The ​Developer Hub also documents the various ​coding standards that are followed:

• ​PHP Coding Standards
• ​CSS Coding Standards
• ​HTML Coding Standards
• ​JavaScript Coding Standards
• ​Accessibility Coding Standards
• ​Inline Documentation Standards

Thank you,
The WordPress Project

Use preg_last_error() inside convert_smilies() to check for any preg_split errors before running count().
If there are errors, sends a PHP warning with the error message, then returns the original text.

Running count() on the array after there are preg_split errors can cause post content to not display (see ticket).

This is a reformat of a previous pull request to get the PHP tests to pass.

Trac ticket: https://core.trac.wordpress.org/ticket/51019

Same, +1 Up 🙌

Is there any ETA on this..?

[rupw]

FWIW I'd probably just check $textarr === false rather than preg_last_error(), but looks good to me.

Another report on WPSE: ​https://wordpress.stackexchange.com/q/409084

[jorbin]

#59927 was marked as a duplicate.

[jorbin]

I would like for there to be an automated test for this, but otherwise, it is something that should be fixed for 6.5.

[audrasjb]

As per today's bugscrub:
Some changes were requested in the current PR, and it still needs unit tests. Let's keep it in the milestone for a week.

@nathkrill are you able to refresh your PR with the requested changes? Thank you!

[rajinsharwar]

Let's keep this in the milestone for some more days, and if there is still not much activity, we might need to bump this again.

[nhrrob]

We have reviewed in today's bug scrub.
It has changes requested. And unit test is required.

Let's keep it in the milestone one more week.

Bailing out in convert_smiles() is preg_split() is false.

Trac ticket: https://core.trac.wordpress.org/ticket/51019

[rajinsharwar]

I added a new PR along with the unit tests. So, I am just bailing out and returning the input text directly, if the preg_replace fails. Added a unit test as well.

I removed the part to show the error message, because the preg_last_error_msg() was introduced in PHP 8, and users with PHP 7 or before will encounter a Fatal error.

Also, we can also use some switch statements to use the preg_last_error() error code, to display error messages based on those. But, I think that would introduce so many strings, and much more complexity than required, for this edge case.

Feel free to share your thoughts if you think we should output the error message as well. 🙂

Testing instructions:

1. Enable WP_DEBUG
2. Add the below code in functions.php

function test(){
	$text =  '<p><img alt="" src="data:image/png;base64,' . str_repeat( "iVBORw0KGgoAAAAN", 99999 ) . '="></p>' ;
	convert_smilies( $text );
}

add_action( 'init', 'test' );

You will notice some Fatal errors.

3. Apply the patch, and repeat the process. You shouldn't notice any Fatal error.

[audrasjb]

As per today's bug scrub, this ticket still needs testing. Moving to 6.7.

[dmsnell]

This same bug was recently discussed within Automattic and @bjorsch identified a problem with the regex being written in a way that invites unrestrained backtracking.

Converting '/(<.*>)/U' into /(<[^>]*>/U would likely resolve this particular problem, as it would get rid of all the backtracking when attempting to poorly segment the HTML.

A more comprehensive fix that I've proposed making is to replace this code altogether with the HTML API, which will eliminate a number of bugs, including this one.

Trac ticket: Core-51019.

Among other issues, convert_smilies() will fail when parsing documents with very long attributes, such as those containing images as data URIs. This occurs frequently when importing content from cloud document editors.

In this patch, the function is refactored to rely on the HTML API instead of its custom PCRE pattern to split and tokenize an HTML document. This avoids the backtracking problem that came with the PCRE function and it improves the reliability of how the function matches HTML text nodes.

Fixes #51019.

[dmsnell]

The patch I just posted should solve this through use of the HTML API instead of regexes to parse the HTML. It should have the added benefit of being able to reliable parse any given HTML. The only remaining quirk is that the way we track CODE and PRE elements is prone to error if they aren't properly balanced, but this is probably fine, and the consequences are merely that smilies won't get converted in such a failure case - it won't crash the site.

Might be an interesting test for the 6.7 cycle since we now have a long time to test it.

One change present in this updated version is that text nodes containing smilies _and other HTML character references_ will be modified such that other character references are decoded as well. This is because the HTML API only exposes decoded strings so that Core doesn't have to try and do all string replacement logic multiple times (as evidenced in this existing code where it checks for the UTF-8 encoded non-breaking space and for   - this approach is guaranteed to corrupt data)