id	summary	reporter	owner	description	type	status	priority	milestone	component	version	severity	resolution	keywords	cc
11918	do_robots() Enhancement	miqrogroove		"do_robots(), in wp-includes/functions.php, is currently responsible for handling robots.txt requests for sites that do not have a robots.txt file.

The default rules are quite lame.  All it does is allow or disallow the entire site based on the privacy setting.

There should be default rules such as

{{{
Disallow: /?s=
Disallow: /search/
Disallow: /wp-includes/
Disallow: *?replytocom
}}}"	enhancement	closed	normal	3.0	Optimization		normal	fixed		sirzooro secondv