Make WordPress Core


Ignore:
Timestamp:
09/02/2019 02:26:55 AM (5 years ago)
Author:
peterwilsoncc
Message:

#43590: Use robots meta tag to better discourage search engines.

This changes the "discourage search engines" option to output a noindex, nofollow robots meta tag. Disallow: / is removed from the robots.txt to allow search engines to discover they are requested not to index the site.

Disallowing search engines from accessing a site in the robots.txt file can result in search engines listing a site with a fragment (a listing without content).

Props donmhico, jonoaldersonwp.
Fixes #43590.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/src/wp-includes/general-template.php

    r45926 r45928  
    29872987 *
    29882988 * Outputs a noindex meta tag that tells web robots not to index the page content.
    2989  * Typical usage is as a wp_head callback. add_action( 'wp_head', 'wp_no_robots' );
     2989 * Typical usage is as a {@see 'wp_head'} callback. add_action( 'wp_head', 'wp_no_robots' );
    29902990 *
    29912991 * @since 3.3.0
     2992 * @since 5.3.0 Echo "noindex,nofollow" if search engine visibility is discouraged.
    29922993 */
    29932994function wp_no_robots() {
    2994     echo "<meta name='robots' content='noindex,follow' />\n";
     2995    if ( get_option( 'blog_public' ) ) {
     2996        echo "<meta name='robots' content='noindex,follow' />\n";
     2997        return;
     2998    }
     2999
     3000    echo "<meta name='robots' content='noindex,nofollow' />\n";
    29953001}
    29963002
Note: See TracChangeset for help on using the changeset viewer.