add_query_arg() should esc_url_raw() REQUEST_URI

[jorbin]

add_query_arg assumes that the query argument is an acceptable query argument. In order to help developers from accidently making a URL an unacceptable URL.

Some related tickets: #16859, #22951, and #22300.

The security team has reviewed this and ok'd it being worked on in public.

https://core.trac.wordpress.org/ticket/58902

This will also need unit tests.

[wpsharif]

I've checked this and I did error log the URL with this changes. It's working fine.

My configuration:
-Windows 11
-Apache Version: 2.4.54.2
-PHP Version: 8.1.13
-MySQL Version: 8.0.31

Updated unit tests

[SergeyBiryukov]

Since esc_url_raw() is a wrapper for sanitize_url(), could we use the latter directly here?

All of the other instances in core were replaced in [53455] / #55852, except for two that accidentally snuck in later.

[SergeyBiryukov]

Replying to SergeyBiryukov:

Since esc_url_raw() is a wrapper for sanitize_url(), could we use the latter directly here?

All of the other instances in core were replaced in [53455] / #55852, except for two that accidentally snuck in later.

Created a ticket for those: #59247.

[ivanzhuck]

@SergeyBiryukov, I've replaced esc_url_raw() with sanitize_url() in the PR

[oglekler]

I think that the problem in question is not covered by Unit tests and change in the last patch is changing test that according to comment is dedicated to test another issue: #4903

[oglekler]

@jorbin can you provide an example URL for Unit test? Thank you!

[ivanzhuck]

@oglekler

The string baz=1 is not a valid relative URL. If we send it as a parameter for the function
esc_url_raw() it returns http://baz=1 that is also not valid URL. We can't use unacceptable URL as a positive test case, because the ticket is about preventing that. So we should add ? before baz=1 to make the URL correct. Do you agree with me?

P.S. The change doesn't affect the issue from #4903

[oglekler]

@ivanzhuck you are changing the test line that was added for another issue 11 years ago:
https://core.trac.wordpress.org/changeset/1192/tests
I believe that it should be left like it is.

This is the tests dedicated to URLs:
​https://github.com/WordPress/wordpress-develop/blob/trunk/tests/phpunit/tests/formatting/escUrl.php

To make sure that add_query_arg() will always return a sanitized URL, I assume it needs a separate test with URLs where there is something to sanitize.

[ivanzhuck]

@oglekler

well... I think we can't leave the old test line as is, because the test fails with it after applying the patch from @audrasjb. We should change one of both: the test or the code)

root@83c80e784dc4:/var/www# ./vendor/bin/phpunit --group add_query_arg
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 9.6.13 by Sebastian Bergmann and contributors.

Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

F......                                                             7 / 7 (100%)

Time: 00:00.382, Memory: 171.50 MB

There was 1 failure:

1) Tests_Functions::test_add_query_arg
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'baz=1&foo=1'
+'http://baz=1?foo=1'

/var/www/tests/phpunit/tests/functions.php:539
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:106

FAILURES!
Tests: 7, Assertions: 91, Failures: 1.

[ivanzhuck]

@oglekler

1. I moved the checkup for the issue #4903 to the end of the test function. Now it runs only if URL passed as a parameter to add_query_arg(). And doesn't run for cases when URL was taken from $_SERVERREQUEST_URI? as sinitize_url()returns not valid value for the line 'baz=1'as it is unacceptable URL.
2. I added a separate test case to make sure add_query_arg() returns sanitized URLs

Please review

[oglekler]

@SergeyBiryukov please take a look, possibly we overcomplicate things :)

[oglekler]

It looks like right now we are running out of time before RC1, so, I am moving this ticket to the next milestone.

[oglekler]

It is possible to test the patch somehow? 🤔 Possibly the patch needs refresh.

I hope we can make it in this milestone and will not be drugging it again 😅

[oglekler]

We have RC1 today, and it looks like we will not make it in this milestone, so I am moving this ticket to the next one.

Most likely, the patch needs refresh and possibly testing instructions.

Hypothetically, Can unescape or escaped $_SERVERREQUEST_URI? in this place break something? 🤔

[stoyangeorgiev]

Moving this one to a Future Release as it was punted multiple times.

[SirLouen]

Reproduction and Patch Test Report

Description

🟠 This report validates that the issue can be reproduced and the patch is working as expected with some warnings included in the notes.

Environment

• WordPress: 6.9-alpha-60093-src
• PHP: 8.4.6
• Server: nginx/1.27.5
• Database: mysqli (Server: 8.4.5 / Client: mysqlnd 8.4.6)
• Browser: Chrome 136.0.0.0
• OS: Windows 10/11
• Theme: Twenty Twenty 2.9
• MU Plugins: None activated
• Plugins:
  • Test Reports 1.2.0

Reproduction Steps

1. Add the coded provided in Supplemental Artifacts to whatever method you prefer to run code in WP
2. Create a post and add the shortcode [url_query_demo_shortcode]
3. 🐞 Resulting URLs are wrong

Expected Results (according to the report)

• Using add_query_arg the resulting URL should be sanitized, hence right despite being poorly made by the developer.

Actual Results with the Patch

• ✅ The resulting URL are correctly formatted.

Additional Notes

• In this case, unit tests are pretty straightforward and testing could be done with them, but still, I have added a very simple tests to showcase this patch in case anyone wants to see visually where is the problem with this.
• 🟠 I'm finding that not all proposed test cases are useful (they are still passing with or without URL sanitization, so they are basically useless). I think some of them can be trimmed, although none will cause any harm being left there.
• ⚠️ I'm not 100% confident if it's the right solution that the URL is sanitized after adding a add_query_arg, or if it should return some sort of notice, for a poorly formatted URL.

Supplemental Artifacts

Here I provide the code to create a shortcode and test this issue:

function url_query_demo_shortcode() {
        $output  = '<div>';
        $output .= '<h2>URL Query String Demo</h2>';

        $urls_with_query_string = array(
                'http://example.com/two words?foo=1' => 'http://example.com/two%20words?foo=1&bar=2',
                'http;//example.com?foo=1'                       => 'http://example.com?foo=1&bar=2',
                'example.com?foo=1'                              => 'http://example.com?foo=1&bar=2',
        );

        $output .= '<table>';
        $output .= '<tr><th>Original URL</th><th>Expected</th><th>Result w/ QV</th></tr>';

        foreach ( $urls_with_query_string as $wrong_url => $expected ) {
                $_SERVER['REQUEST_URI'] = $wrong_url;
                $with_bar               = add_query_arg( array( 'bar' => '2' ) );
                $result = ( $with_bar !== $expected ) ? '❌' : '✅';
                $output .= '<tr>';
                $output .= '<td>' . $wrong_url . '</td>';
                $output .= '<td>' . $expected . '</td>';
                $output .= '<td>' . $with_bar . '</td>';
                $output .= '<td>' . $result . '</td>';
                $output .= '</tr>';
        }

        $output .= '</table>';

        $output .= '</div>';

        return $output;
}
add_shortcode( 'url_query_demo', 'url_query_demo_shortcode' );