WP rewrite rule bug with & in url path

[stephdau]

We've noticed a peculiar bug in the standard rewrite rules for wp if & is included in any url, as part of the path, not the query string.

EGs:

• ​http://tekartist.org/&nbsp/anything
• ​http://ma.tt/&blah/test

The rewrite rule serves the default page, without returning a 404.

My best guess at this time is that what is happening is that the inclusion of & in a rewritten path is actually seen as a query param (eg: treated as /index.php?&nbsp=...)

[stephdau]

So, a bit of context: we found the above issue when the msn bot got stuck in an infinite loop on an install because of a bad link from somwhere that ended up being sent to the server as "&nbsp/blah" instead of "/blah".

Having the & anywhere else in the url does not create an issue, only when the 1st character after the root of the wp install.

• EG: gave a 404, as expected: ​http://my.host/my/path/to/wp/nbsp&/test

• EG: just loaded the top page: ​http://my.host/my/path/to/wp/&nbsp/test

So in my local install, I fixed it with the following rule, which simply redirects offending url to the same one without the ampersand, therefore giving the proper wp 404 response:

RewriteRule ^&(.*)$ $1 [R=301,L]

As in:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /~epsi/wptrunk/
RewriteRule ^&(.*)$ $1 [R=301,L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /~epsi/wptrunk/index.php [L]
</IfModule>
# END WordPress

See attached patch for suggested wp-includes/rewrite.php fix.

[stephdau]

patch for /wp-includes/rewrite.php

[ryan]

Do setups that don't use mod_rewrite experience the same problem (pathinfo, IIS rewrite). If so we should solve it in WP's query handler.

[ryan]

Perhaps $_SERVERQUERY_STRING? is being set for these requests, thus confusing WP::handle_404().

[stephdau]

• instances without mod_rewrite: in my test, the problem did not affect these
• query_string: indeed, that is the source of the issue. I tried other ways, such a suggested fix for MediaWiki (​http://www.mediawiki.org/wiki/Manual:Short_URL/Ampersand_solution), but the above is what worked best, again, in my tests.

[stephdau]

As far as having the fix in the query handler: there are up and down sides to that, but it'd be just as valid a solution, and if nothing else, it'd be easier to control, and for upgrades, since a .htaccess solution (as provided in patch) would have the downside of people having to physically go to the permalinks settings page and re-saving their configs (unless handled in the upgrade, which unless I'm mistaken, isn;t a best practice).

[stephdau]

And although it's not a core thing, we might also want to try things out with donncha's wp-super-cache.

[nacin]

No patch, moving out of 3.0 for now.

[loushou]

This problem is already solved in 4.0. Looks like a simple patch to the WP_MatchesMapRegex() class solved the problem:

  public function callback($matches) {
    $index = intval(substr($matches[0], 9, -1));
    return ( isset( $this->_matches[$index] ) ? urlencode($this->_matches[$index]) : '' );
  }

The urlencode bit, on the return line, resolves the & issue.

This should be closed.