WordPress.org
Get WordPress

Make WordPress Core

Opened 7 years ago

Closed 4 years ago

#43557 closed defect (bug) (maybelater)

Remove wp-admin/admin-ajax.php from robots.txt - sure give ppl the ability to add it, remove the default

Reported by: phuquocgreentours's profile phuquocgreentours Owned by:
Milestone: Priority: normal
Severity: normal Version: 4.9.4
Component: General Keywords:
Focuses: Cc:

Description

This is a follow-up to #33156.

This is dumb...

My site returns 0 and HTTP status code of 400 when hitting admin-ajax.php.
Of course, Google is going to have a hard time interpreting that.

All the links on the page Google complained about are Javascript executed requests. There are no bare ​https://site/wp-admin/admin-ajax.php?someparam=getsomedata that Googlebot can follow. Everything is executed from JS and obviously requires the correct params.

I don't want errors from Googlebot...

Looking for how to remove this default robots.txt rubbish.

Expose robots.txt to the WP API so whoever needs this can add it.
Making it default is ludicrous :/

Change History (2)

This ticket was mentioned in ​Slack in #core by noisysocks. ​View the logs.
4 years ago

#2 @noisysocks
4 years ago

  • Milestone Awaiting Review deleted
  • Resolution set to maybelater
  • Status changed from new to closed

Hi @phuquocgreentours. The default is to have Allow: /wp-admin/admin-ajax.php as this is best for the majority of sites. You can use the robots_txt filter to change this.

Note: See TracTickets for help on using tickets.