WordPress.org

Make WordPress Core

Opened 6 months ago

Closed 6 months ago

Last modified 6 months ago

#47711 closed defect (bug) (fixed)

noindex, wp_die and admin-ajax.php

Reported by: harryfear Owned by: SergeyBiryukov
Milestone: 5.3 Priority: normal
Severity: normal Version: 3.4
Component: Administration Keywords: good-first-bug has-patch needs-testing
Focuses: administration, rest-api Cc:
PR Number:

Description

Hi folks,

I've found what appears to be a bug that affects fresh WordPress installs, so I am sure it's about the core and not themes or plugins.

If one browses directly to "admin-ajax.php", the URL dies with code 400 as a result of the:

wp_die( '0', 400 );

Further down in the file, there is the noindex header:

@header( 'X-Robots-Tag: noindex' );

My issue arose due to Google reporting a "robots blocked but indexed" issue https://webmasters.stackexchange.com/questions/123439/how-to-allow-only-admin-ajax-php-to-be-crawled.

But when it dies it seems not to send the noindex header, at least on my hosting environment. I don't have any other issues with this host. My host is a basic shared Cpanel host with a bunch of different WordPress sites/domains running on it. I think this is something that could be looked at from the WordPress side as a possible bug?

On some others' publicly-browsable sites, I can see that the noindex header *is* sent along with the 400 error status.

I don't myself have access to multiple different hosting environments to test this with, so I'm throwing it out here for advice/expertise/support/alarm-raising.

The noindex header instruction is set explicitly lower down in the admin-ajax.php file. If I copy that line up in way of a hot fix tweak on the code, the issue is resolved for me with Google:

// Require an action parameter
if ( empty( $_REQUEST['action'] ) ) {
	@header( 'X-Robots-Tag: noindex' ); // I moved this up and it solves the issue
	wp_die( '0', 400 );
}

I also contacted my host and we found that a .htaccess directive hot-fixes my issue on all my domains for now:

<FilesMatch "admin-ajax.php">
Header set X-Robots-Tag "noindex"
</FilesMatch>

Attachments (3)

47711.diff (192.1 KB) - added by robi-bobi 6 months ago.
47711.2.diff (192.1 KB) - added by robi-bobi 6 months ago.
47711.3.diff (809 bytes) - added by robi-bobi 6 months ago.
47711.3.diff

Download all attachments as: .zip

Change History (14)

#1 @harryfear
6 months ago

  • Summary changed from noindex, wpkill and admin-ajax.php to noindex, wp_die and admin-ajax.php

#2 @SergeyBiryukov
6 months ago

  • Component changed from General to Administration
  • Keywords needs-patch good-first-bug added
  • Milestone changed from Awaiting Review to 5.3
  • Version changed from 5.2.2 to 3.4

Hi @harryfear, welcome to WordPress Trac! Thanks for the report.

Yes, apparently the header should be sent earlier, after send_origin_headers(), so that it applies before wp_die() as well.

For reference, the current line was introduced in [20288].

Last edited 6 months ago by SergeyBiryukov (previous) (diff)

#3 @harryfear
6 months ago

Great, glad I'm not going mad. I'll watch this ticket.

@robi-bobi
6 months ago

#4 @robi-bobi
6 months ago

  • Keywords has-patch added; needs-patch removed

I've submitted a patch for this.

#5 @SergeyBiryukov
6 months ago

Thanks for the patch, @robi-bobi!

Looks like it only updates the package-lock.json file, but not wp-admin/admin-ajax.php. Could you update it with the correct changes?

#6 @robi-bobi
6 months ago

Thanks @SergeyBiryukov
Yeah, this is my first patch for this project, so a help from someone more familiar with WordPress processes would be great:

I used grunt upload_patch:47711, which added the patch, but in it I see:

  • package-lock.json file changes, isn't this supposed to be ignored?
  • my changes from wp-admin/admin-ajax.php are indeed not in there. I do see them if I use svn diff command (where I also see the package_json stuff, again, should I ignore this locally)
Last edited 6 months ago by robi-bobi (previous) (diff)

#7 @garrett-eclipse
6 months ago

  • Keywords needs-refresh added

Hi @robi-bobi

There's some great assistance here - https://make.wordpress.org/core/handbook/tutorials/working-with-patches/#creating-a-patch-with-the-command-line

I followed that when I started. And one of the things I learned when doing a patch from SVN is to either specify the file or the folder, in my case I just always run svn diff on the src folder;
svn diff src > #####.diff

You'll want to create the patch file before running the upload_patch command. And I assume it picked up that file as you weren't specific as to what folder (src) to diff.

In your case I suggest the following commands in this order;
svn diff src > 47711.2.diff
grunt upload_patch:47711
*The only time I personally don't focus my diff on src is when there's unit tests and then I include the tests folder like;
svn diff src tests > #####.diff

Hope that helps,
P.S. Another tip that took me a while to realize is if your patch introduces new files you first have to do an svn add before creating the patch file;
svn add src/path/to/file.txt

@robi-bobi
6 months ago

#8 @robi-bobi
6 months ago

Thanks @garrett-eclipse !
I did follow the tutorial you linked and did exactly as it instructs (did not target the src because of that).

I followed your suggestion, but it still uploaded the same file as before.

Finally I did svn diff while targeting src, and uploaded the diff manually. Hope that this works.

Please use only the 47711.3.diff when merging/reviewing.

@robi-bobi
6 months ago

47711.3.diff

#9 @garrett-eclipse
6 months ago

  • Keywords needs-testing added; needs-refresh removed

Glad the manual approached worked for you @robi-bobi the patch 47711.3.diff looks good to me.

Weird that the grunt task isn't working for you though. Feel free to find me on Slack (@garrett-eclipse) and I can help you troubleshoot. Or come join the new contributor meeting on Slack in the #core channel on the 2nd and 4th Wednesdays at 19:00 UTC. Everyone is quite helpful there.

#10 @SergeyBiryukov
6 months ago

  • Owner set to SergeyBiryukov
  • Resolution set to fixed
  • Status changed from new to closed

In 45650:

Administration: In admin-ajax.php, send X-Robots-Tag header earlier, so that it applies before wp_die() when no action parameter was provided.

Props robi-bobi, harryfear, garrett-eclipse.
Fixes #47711.

#11 @robi-bobi
6 months ago

Thank you @SergeyBiryukov and @garrett-eclipse for helping me complete my first patch.
Thanks @harryfear for describing the issue so well.

Note: See TracTickets for help on using tickets.