WordPress.org

Make WordPress Core

Opened 6 years ago

Closed 6 years ago

#13115 closed defect (bug) (fixed)

Wordpress should not serve robots.txt when it is not installed at the root of a site

Reported by: solarissmoke Owned by: ryan
Milestone: 3.0 Priority: normal
Severity: normal Version: 3.0
Component: Rewrite Rules Keywords: has-patch dev-feedback
Focuses: Cc:

Description

Current behaviour:
Visiting http://some.domain.tld/wordpress/robots.txt causes wordpress to generate a robots.txt file, when actually such a file below the root directory has no meaning.

Desired behaviour: the robots.txt file is only meaningful when placed at the root of a domain. So wordpress should only do robots.txt handling if it is installed at the root of a domain, otherwise return 404 for all such requests.

Is this too late for 3.0? Feel free to punt. I'm working on a patch anyway though.

Attachments (2)

13115.patch (654 bytes) - added by solarissmoke 6 years ago.
set is_robots only if wp is at the root of the domain, otherwise set 404. This looks a little clumsy to me, so a better implementation is welcome
13115-v2.patch (890 bytes) - added by solarissmoke 6 years ago.
add robots.txt rewrite rule only if wp is installed in root directory

Download all attachments as: .zip

Change History (7)

@solarissmoke
6 years ago

set is_robots only if wp is at the root of the domain, otherwise set 404. This looks a little clumsy to me, so a better implementation is welcome

#1 @solarissmoke
6 years ago

  • Keywords dev-feedback added

The rewrite_rules are processed relative to Wordpress' installation directory, so nothing there can be changed directly there as far I can see.

One possible alternative:

Have a static robots.txt file and have Wordpress create/modify it when privacy options are changed AND wordpress is installed at the root of a domain.

#2 @nacin
6 years ago

We should probably do this check instead when generating the rewrite rules, and decline to add robots.txt to the rules if we're not root.

We do the similar parse_url() check in rewrite.php, query.php, classes.php, etc. -- that seems okay.

#3 @solarissmoke
6 years ago

Thanks nacin, that makes more sense. Here's a better patch.

@solarissmoke
6 years ago

add robots.txt rewrite rule only if wp is installed in root directory

#4 @scribu
6 years ago

  • Keywords has-patch added

#5 @nacin
6 years ago

  • Resolution set to fixed
  • Status changed from new to closed

(In [14273]) Don't serve dynamic robots.txt when WP is not installed at the root of a domain. props solarissmoke, fixes #13115

Note: See TracTickets for help on using tickets.