do_robots() should not serve robots.txt as text/html
|Reported by:||JeremyVisser||Owned by:||ryan|
WordPress SVN /branches/2.1 rev 5096
Also affects /trunk (rev 5116) and WordPress MU /tags/1.1.1.
In wp-includes/functions.php, the do_robots() function serves up the contents of a robots.txt file. If you try to view the file in a web browser, the line breaks will not show up properly. This is because the mime-type served is the default text/html which instructs the browser to parse the robots.txt file as HTML. This should not be the case.
text/plain is a good mime-type to use, and behaves correctly.
The attached patch will cause the function to serve robots.txt as text/plain.