Introduce 'noindex' filter for robots meta tag

[MartyThornley]

As things are right now, themes and plugins add robots meta tag lines by just adding an action to wp_head, with no way of knowing who has added what. Because it is not done as a filter, there is no way to prioritize or alter things.

As far as I can see there are two functions that are hooked to wp_head to add the meta tag for robots, the noindex() and wp_no_robots() function in '/wp-includes/general-template.php'. All noindex() does is look to see if a blog is marked private, then calls wp_no_robots(). In other places, like the signup and login pages, wp_no_robots is manually added.

To help clear up the confusion, we could just call wp_no_robots whenever we need to add a robots meta tag, use wp_no_robots to do some default logic, then allow plugins to filter it depending on where we are.

Working on a quick patch now and will add.

[MartyThornley]

add some logic to wp_no_robots, unhook noindex from wp_head

[MartyThornley]

To explain a few things in the first go at this...

Added some default ideas for what values should be added when. Maybe overkill or maybe not the exact right things? Not exactly a robots expert here, just wish it was more editable. :)

Needed to grab the $post so we can see where we are when filtering. As a bonus, the empty $post as it is done here, works to detect things like wp-signup.php, where wp_no_robots() has been manually added.

[MartyThornley]

cleaner default meta tag

[MartyThornley]

After rethinking a little, came up with this simpler way of doing the default meta tag. Something like this would let core stay simple and still let everyone filter it.

Maybe even get rid of all the wp_query and idea of passing the $post, just let anything tying it to do that if they need to?

[obenland]

wp_no_robots() is expected to output the meta tag. Changing that would likely break backwards compatibility.
Having said that, these changes could probably be made in noindex().

[Mte90]

new refreshed patch

[Mte90]

Looking at the code add that in no_index require that we pass a parameter to wp_no_robots but that method is used in many parts of the code.
So for me the best solution is the filter inside the wp_no_robots method, after all is a method to alter the value of this tag values and in case there are no value not print at all.
So I think that not created problems for back compatibility because the behaviour is right also on old systems if that filter will be used.

[flixos90]

If I understand the conversation here correctly, we should be able to consider this covered by #51511. Closing for now; please reopen if there is something else to do here.