Make WordPress Core


Ignore:
Timestamp:
09/02/2019 02:26:55 AM (4 years ago)
Author:
peterwilsoncc
Message:

#43590: Use robots meta tag to better discourage search engines.

This changes the "discourage search engines" option to output a noindex, nofollow robots meta tag. Disallow: / is removed from the robots.txt to allow search engines to discover they are requested not to index the site.

Disallowing search engines from accessing a site in the robots.txt file can result in search engines listing a site with a fragment (a listing without content).

Props donmhico, jonoaldersonwp.
Fixes #43590.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/tests/phpunit/tests/general/template.php

    r45028 r45928  
    613613        $this->assertSame( $expected, $result );
    614614    }
     615
     616    /**
     617     * @ticket 43590
     618     */
     619    function test_wp_no_robots() {
     620        // Simulate private site (search engines discouraged).
     621        update_option( 'blog_public', '0' );
     622        $actual_private = get_echo( 'wp_no_robots' );
     623        $this->assertSame( "<meta name='robots' content='noindex,nofollow' />\n", $actual_private );
     624
     625        // Simulate public site.
     626        update_option( 'blog_public', '1' );
     627        $actual_public = get_echo( 'wp_no_robots' );
     628        $this->assertSame( "<meta name='robots' content='noindex,follow' />\n", $actual_public );
     629    }
    615630}
Note: See TracChangeset for help on using the changeset viewer.