Make WordPress Core

Opened 2 years ago

Last modified 8 days ago

#53465 assigned task (blessed)

PHP 8.1.: the default value of the flags parameter for htmlentities() et all needs to be explicitly set

Reported by: jrf's profile jrf Owned by: hellofromtonya's profile hellofromTonya
Milestone: 6.3 Priority: normal
Severity: normal Version:
Component: General Keywords: php81 has-patch has-unit-tests
Focuses: coding-standards Cc:

Description

From the PHP 8.1 changelog:

htmlspecialchars(), htmlentities(), htmlspecialchars_decode(),
html_entity_decode() and get_html_translation_table() now use
ENT_QUOTES | ENT_SUBSTITUTE rather than ENT_COMPAT by default. This means
that ' is escaped to ' while previously it was left alone.
Additionally, malformed UTF-8 will be replaced by a Unicode substitution
character, instead of resulting in an empty string.

Ref: https://github.com/php/php-src/blob/28a1a6be0873a109cb02ba32784bf046b87a02e4/UPGRADING#L149-L154

If effect this means that the output of the above mentioned functions may be different depending on the PHP version and the passed text string, unless the $flags parameter is explicitly passed.

I've run an initial scan over WordPress core with a new (not yet published) sniff for PHPCompatibility and this flags 33 issues.

  • 1 issue in GetID3 which should be fixed upstream and the copy of GetID3 used in WP should be updated once the issue is fixed.
  • 1 issue in PHPMailer which should be fixed upstream and the copy of PHPMailer used in WP should be updated once the issue is fixed.
  • 1 issue in SimplePie which should be fixed upstream and the copy of SimplePie used in WP should be updated once the issue is fixed.
  • And 30 issues in WP Core native code or code from external dependencies which are no longer maintained externally.

Detailed issue list: https://gist.github.com/jrfnl/9d56b4053faa62a0fe91dea1b14839bf

To fix this issue, the $flags parameter should be explicitly passed in each of these function calls.

Some investigation will be needed for each of these instances to determine what will be the optimal value for $flags.

Take note that the "old" parameter default in the function signature is documented as ENT_COMPAT, while in the parameter detail documentation, it states that the default, in actual fact, is ENT_COMPAT | ENT_HTML401.

However, by the looks of it, the full range of flag constants is available to us, which is at least one less problem.
There is no mention of any of the flags being added since PHP 5.6.
Ref: https://php-legacy-docs.zend.com/manual/php5/en/string.constants

It is strongly recommended to make sure that for each of these at least one unit test exists which exposes the difference in output between PHP < 8.1 and PHP 8.1 to safeguard the fixes which will be added for the future.

Also see:

Change History (17)

#2 @hellofromTonya
19 months ago

  • Milestone changed from 5.9 to 6.0

With 4 days left until 5.9 Beta 1, moving this to 6.0.

#3 follow-up: @GaryJ
15 months ago

According to https://www.php.net/manual/en/function.htmlentities.php, the default for flags for PHP 8.1 was "changed from ENT_COMPAT to ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML401".

I don't know what difference that ENT_HTML401 might make (there's also a ENT_HTML5 flag available as well), but might be something else to consider.

#4 in reply to: ↑ 3 @jrf
15 months ago

Replying to GaryJ:

According to https://www.php.net/manual/en/function.htmlentities.php, the default for flags for PHP 8.1 was "changed from ENT_COMPAT to ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML401".

I don't know what difference that ENT_HTML401 might make (there's also a ENT_HTML5 flag available as well), but might be something else to consider.

The ENT_HTML401 makes no difference as the value of that constant is 0, so the "bit" value remains the same with or without.
The ENT_HTML401 part is not an actual change, just better documentation.

See:

#5 @costdev
13 months ago

  • Milestone changed from 6.0 to 6.1

With 6.0 RC1 tomorrow, I'm moving this to the 6.1 milestone.

#6 @rafiahmedd
11 months ago

I think we can discuss this ticket on the next dev chat on slack. And it seems to be an important one.

#7 @jrf
11 months ago

FYI: If I remember correctly, @SergeyBiryukov is working on patches for this ticket together with other people from the Yoast WP Core team.

#8 @SergeyBiryukov
11 months ago

Yes, @aristath, @justinahinon, @poena, @afercia, and I worked on this recently in group sessions and should have a PR ready for review soon :)

This ticket was mentioned in Slack in #core by chaion07. View the logs.


8 months ago

This ticket was mentioned in Slack in #hosting-community by javier. View the logs.


8 months ago

This ticket was mentioned in Slack in #hosting-community by jadonn. View the logs.


8 months ago

#12 @audrasjb
8 months ago

  • Milestone changed from 6.1 to 6.2

With WP 6.1 RC 1 scheduled tomorrow (Oct 10, 2022), there is not much time left to address this ticket. Given it still needs a patch, let's move this ticket to 6.2.

Ps: if you were about to send a patch and if you feel it is realistic to commit it in the next few hours, please feel free to move this ticket back to milestone 6.1.

This ticket was mentioned in PR #3429 on WordPress/wordpress-develop by SergeyBiryukov.


8 months ago
#13

  • Keywords has-patch has-unit-tests added; needs-patch removed

This ticket was mentioned in Slack in #core by costdev. View the logs.


4 months ago

#15 @costdev
4 months ago

This ticket was discussed during the recent bug scrub.

@SergeyBiryukov Do you have time to give the PR a quick bump to see results on GitHub Actions?

#16 @hellofromTonya
3 months ago

  • Milestone changed from 6.2 to 6.3
  • Owner set to hellofromTonya
  • Status changed from new to assigned

The last scheduled 6.2 beta happened today. RC1 is next week. Moving this ticket to 6.3 and self-assigning ownership to help shepherd its completion.

This ticket was mentioned in Slack in #core by oglekler. View the logs.


8 days ago

Note: See TracTickets for help on using tickets.