Multisite: 404 template isn't used in network installs when a file extension is used in the URL

[henry.wright]

The WordPress 404 template isn't used when certain URLs are visited. For example:

• ​https://commons.gc.cuny.edu/sites/file.php
• ​https://blogs.harvard.edu/file.php

Note this seems to happen in network installs only. I can't reproduce when using a single install.

[pbiron]

This only happens when the file extension is .php and is because of the .htaccess rewrite rules that are used with multisite.

multisite sub-directory rewrite rules

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]

# add a trailing slash to /wp-admin
RewriteRule ^([_0-9a-zA-Z-]+/)?wp-admin$ \$1wp-admin/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
RewriteRule ^([_0-9a-zA-Z-]+/)?(wp-(content|admin|includes).*) \$2 [L]
# the following rule is the "culprit" and is necessary for URLs like:
# https://example.com/sub-site/wp-login.php
RewriteRule ^([_0-9a-zA-Z-]+/)?(.*\.php)$ \$2 [L]
RewriteRule . index.php [L]

multisite sub-domain rewrite rules

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]

# add a trailing slash to /wp-admin
RewriteRule ^([_0-9a-zA-Z-]+/)?wp-admin$ \$1wp-admin/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
RewriteRule ^(wp-(content|admin|includes).*) \$1 [L]
# the following rule is the "culprit" and is necessary for URLs like:
# https://sub-site.example.com/wp-login.php
RewriteRule ^(.*\.php)$ \$1 [L]
RewriteRule . index.php [L]

[pbiron]

Because of those 2 .htaccess rewrite rules the web server, and not WP, handles those requests, which is why the 404 template never gets called.

[apedog]

ErrorDocument directive is your friend.

​https://httpd.apache.org/docs/2.4/custom-error.html

[henry.wright]

@apedog

The pattern matching described by @pbiron could be improved rather than fixing this at the server level

[pbiron]

@henrywright Unfortunately, as far as I know, the ​RewriteCond syntax is not expressive enough to "improve the pattern matching".

However, I'm not sure if this is what @apedog was suggesting in his comment (and I'd never thought of this before), but since the ​ErrorDocument directive is allowed in .htaccess, changing the multiste mod_rewrite rules to the following seems to works:

new multisite sub-directory rewrite rules

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
# the following is the new rule.
# it forces WP to handle the request when apache detects a 404 as a result of
# the "RewriteRule ^([_0-9a-zA-Z-]+/)?(.*\.php)$ $2 [L]" rule.
ErrorDocument 404 /index.php

# add a trailing slash to /wp-admin
RewriteRule ^([_0-9a-zA-Z-]+/)?fwp-admin$ $1wp-admin/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
RewriteRule ^([_0-9a-zA-Z-]+/)?(wp-(content|admin|includes).*) $2 [L]
RewriteRule ^([_0-9a-zA-Z-]+/)?(.*\.php)$ $2 [L]
RewriteRule . index.php [L]

new multisite sub-domain rewrite rules

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
# the following is the new rule.
# it forces WP to handle the request when apache detects a 404 as a result of
# the "RewriteRule ^(.*\.php)$ $1 [L]" rule.
ErrorDocument 404 /index.php

# add a trailing slash to /wp-admin
RewriteRule ^wp-admin$ wp-admin/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
RewriteRule ^(wp-(content|admin|includes).*) $1 [L]
RewriteRule ^(.*\.php)$ $1 [L]
RewriteRule . index.php [L]

I've tried the above on a few local multisites and it seems to work. I haven't tested it enough to know whether it will cause unintended consequences. Would love to get some more feedback from others.

As I've never used nginx, I have no idea whether there is an equivalent for it's rules (or IIS, for that matter).

[apedog]

@henrywright
I've no experience with multisite, so I've no idea why WordPress's default .htaccess rules are as they are.
I was treating this as a support question. Figuring this ticket would be flagged as invalid by someone else down the road, I thought I'd chime in with an easy solution.

Adding an ErrorDocument rule by default might be considered an opinionated solution, that could affect others in ways I don't know.
I've no idea if WordPress should redirect server 404's to a WordPress document. And even if so, if it perhaps should redirect to /404.php and not /index.php.

Adding a line of code to your .htaccess is an easy and reasonable fix.

[limestreet]

I think that's a bug.
I was able to reproduce the issue, it's not just causing that errors are handled by the server - it creates inifinite redirect loop for non existent directory and 500 Internal error.
When you try to open non existent directory with the trailing slash at the end you get redirect loop (to itself) - example url:

http://example.com/wp-content/non-existent/

when there is no trailing slash at the end you get 404 error and that error is handled by the server too - example:

http://example.com/wp-content/non-existent

I think the issue is caused by the following rule:

RewriteRule ^(wp-(content|admin|includes).*) $1 [L]

As far as I understand it's just rewriting URI to itself...
So it looks like it may be removed.
If it's okay I may provide a patch for that issue.

Best,
Patryk

[limestreet]

Here are the old related tickets:
https://core.trac.wordpress.org/ticket/20746
https://core.trac.wordpress.org/ticket/31664