WordPress.org

Make WordPress Core

Opened 5 years ago

Closed 5 years ago

#13115 closed defect (bug) (fixed)

Wordpress should not serve robots.txt when it is not installed at the root of a site

Reported by: solarissmoke Owned by: ryan
Milestone: 3.0 Priority: normal
Severity: normal Version: 3.0
Component: Rewrite Rules Keywords: has-patch dev-feedback
Focuses: Cc:

Description

Current behaviour:
Visiting http://some.domain.tld/wordpress/robots.txt causes wordpress to generate a robots.txt file, when actually such a file below the root directory has no meaning.

Desired behaviour: the robots.txt file is only meaningful when placed at the root of a domain. So wordpress should only do robots.txt handling if it is installed at the root of a domain, otherwise return 404 for all such requests.

Is this too late for 3.0? Feel free to punt. I'm working on a patch anyway though.

Attachments (2)

13115.patch (654 bytes) - added by solarissmoke 5 years ago.
set is_robots only if wp is at the root of the domain, otherwise set 404. This looks a little clumsy to me, so a better implementation is welcome
13115-v2.patch (890 bytes) - added by solarissmoke 5 years ago.
add robots.txt rewrite rule only if wp is installed in root directory

Download all attachments as: .zip

Change History (7)

@solarissmoke5 years ago

set is_robots only if wp is at the root of the domain, otherwise set 404. This looks a little clumsy to me, so a better implementation is welcome

comment:1 @solarissmoke5 years ago

  • Keywords dev-feedback added

The rewrite_rules are processed relative to Wordpress' installation directory, so nothing there can be changed directly there as far I can see.

One possible alternative:

Have a static robots.txt file and have Wordpress create/modify it when privacy options are changed AND wordpress is installed at the root of a domain.

comment:2 @nacin5 years ago

We should probably do this check instead when generating the rewrite rules, and decline to add robots.txt to the rules if we're not root.

We do the similar parse_url() check in rewrite.php, query.php, classes.php, etc. -- that seems okay.

comment:3 @solarissmoke5 years ago

Thanks nacin, that makes more sense. Here's a better patch.

@solarissmoke5 years ago

add robots.txt rewrite rule only if wp is installed in root directory

comment:4 @scribu5 years ago

  • Keywords has-patch added

comment:5 @nacin5 years ago

  • Resolution set to fixed
  • Status changed from new to closed

(In [14273]) Don't serve dynamic robots.txt when WP is not installed at the root of a domain. props solarissmoke, fixes #13115

Note: See TracTickets for help on using tickets.