page name /robots-txt/ creates confused with robots.txt

[Jean-LucfromBrussels]

Hi,

I have created a page (not a post) called /robots-txt/. It is an information page about the way /robots.txt works. The real content of the page is not displayed; in fact, what is displayed is this :

User-agent: *
Disallow:

Addresses like /robots-txt/crawl-delay/ work correctly. What can I do to make /robots-txt/ work as well ?

Example here: ​http://www.annuaire-info.com/robots-txt/

Jean-Luc

[ionfish]

This is to do with the way that WordPress' rewrite rules work, and that it automatically generates a robots.txt file depending on the site's privacy options.

[Jean-LucfromBrussels]

Thanks ionfish,

I understand that. I reported this as a bug, because I do not think there is a good reason to confuse robots-txt with robots.txt.

I will reformulate my question: Is there a way to make WordPress rewrite robots.txt and not robots-txt ?

I changed line 808 in /wp-includes/rewrite.php in

$robots_rewrite = array('robots\.txt$' => $this->index . '?robots=1');

but this did not help.

[DD32]

but this did not help.

It would've if you cleared the Rewrite rules :)

The WordPress Rewrite rules are cached in the database, You need to clear them after making changes to the rules. Bumping the $wp_db_version up a notch will cause it to re-create them.

The attached patch is exactly what you posted, with the DB version bumped up

[ryan]

(In [9650]) Escape dot in robots rewrite rule. Props Jean-LucfromBrussels and DD32. fixes #8176