wp_extract_urls not returning full URL with & query separator

[trex005]

Related : 29314

When formatting a link, and even when inserting a URL in the visual editor in WordPress the & separator used for the query variables will be properly encoded into &. However, when you then pass that string to wp_extract_urls, it incorrectly truncates the URL at the &.

Example :
​https://www.youtube.com/watch?v=exvUH2qKLTU&start=25
Becomes
​https://www.youtube.com/watch?v=exvUH2qKLTU&

[johnbillion]

Thanks for the report!

[voldemortensen]

Fixes the regex used in wp_extract_urls. Adds unit tests.

[johnbillion]

Those tests could probably benefit from some URLs with trailing punctuation, eg. http://example.com;.

[voldemortensen]

I was just writing a comment saying that the ; character would now be allowed in urls, which has the potential to allow broken urls. But I think its better to allow some broken urls (we can't help if someone enters a bad link), than to not allow valid urls. Adding patch with suggested unit tests.

This is a refresh of Trac #30580 and adds additional test cases.

Trac ticket: https://core.trac.wordpress.org/ticket/30580

[ironprogrammer]

Thanks to @voldemortensen for past work on this 😄

I've pushed up ​PR 2283 which refreshes this patch, and adds some additional test cases.

Tests Added

• URLs with either named or numbered entities, e.g. & or &.
• URL with non-escaped ampersand (common for manually written or edited URLs).
• URLs with typos, such as an extraneous semicolon, or a missing semicolon on an HTML entity.

This fix unblocks unit testing needed for #38197.

[ironprogrammer]

The proposed ​PR 2283 has been updated to include backward compatibility support for the removal of unused semicolons (;).

Props @costdev, @hellofromTonya

[peterwilsoncc]

I think the ​linked pull request is good to commit as an improvement.

The issue of semi-colons being stripped remains even though they're perfectly valid characters in a URL. There's still a little to do to make sure that's dealt with correctly so it can be fixed on a follow up ticket.

[peterwilsoncc]

In 53044:

Formatting: Account for HTML entities in wp_extract_urls().

Prevent wp_extract_urls() trimming HTML entities within URLs. Correctly escaped URLs such as ​https://youtube.com/watch?v=dQw4w9WgXcQ&t=1 will now be extracted as ​https://youtube.com/watch?v=dQw4w9WgXcQ&t=1 rather than truncated.

Props trex005, voldemortensen, johnbillion, ironprogrammer, costdev, hellofromtonya.
Fixes #30580

Closing as completed in r53044.