Update Search Engine Visibility text following XML sitemap launch

[jonoaldersonwp]

#50117 adds XML sitemaps to core.

Whilst it's a non-goal of that project to add user controls for XML sitemaps, checking the discourage search engines from indexing the site option (In Settings > Reading) does disable XML sitemaps entirely.

Given this, we should update the wording around this option to make this clear - and whilst we're in there, we could also do a little housekeeping.

I propose that we change the message to:

Search Engine Visibility
Discourage search engines from indexing this site (and disable WordPress' XML sitemaps)
Note: It is up to search engines to honor this request, and this setting won't prevent them from crawling your website.

[arpitgshah]

Hey,

Do you want to update the text whenever we have update the sitemap?

Can you please explain me more with screenshots?

Thanks,
Arpit G Shah

[afercia]

A similar text in the "At a glance" dashboard widget would need to be updated as well. See screenshot below.

[arpitgshah]

Do we need to change in both the files?

[Presskopp]

disregard

[swissspidy]

A similar text in the "At a glance" dashboard widget would need to be updated as well.

Not sure actually. There's not much space there to change the text.

Anyway, my suggestion on Slack was to not add to much technical details (what are XML sitemaps? What is noindex?) to the copy there, and instead link to helphub.

Maybe something like this:

Discourage search engines from indexing this site
This will also disable XML sitmaps.
It is up to search engines to honor this request, and this setting won't prevent them from crawling your website. Learn more about search engine visibility.

[Presskopp]

Because of the reason you mentioned to not add too much technical stuff, I ask myself if we even need the information "This will also disable XML sitmaps." here. Who wants to have XML-Sitemaps while hiding his site from search engines? And at the moment the checkbox gets unchecked, they will be there.

[afercia]

Replying to swissspidy:

A similar text in the "At a glance" dashboard widget would need to be updated as well.

Not sure actually. There's not much space there to change the text.

A good UI should be designed around content. Not the other way around :)

[swissspidy]

Reminds me...wasn't there a ticket somewhere dedicated to changing that dashboard widget text? I remember it being quite long.

[afercia]

Replying to swissspidy:

Reminds me...wasn't there a ticket somewhere dedicated to changing that dashboard widget text? I remember it being quite long.

#35288 which produced the fundamental change of adding an icon :)

[knutsp]

Replying to Presskopp:

Who wants to have XML-Sitemaps while hiding his site from search engines?

I would. Feed is there, HTML output is there, API's are not disabled, for viewing and testing. One may want to check the sitemap before going public. The link to then sitemap (index) should just not be linked to by robots.txt.

This probably not the right ticket, then, but the idea to disable sitemaps when site is not "public" in the eyes of search engines is wrong. This should rely on noindex, and the sitemap should just not be submitted to search engines to avoid feedback saying the submitted pages are "noindexed". The existence of sitemap(s) is not the same as it/they being submitted.

Google threats RSS/Atom feeds as partial sitemaps, if submitted. The existence of those should be treated equally by WordPress.

[swissspidy]

Ha, good to know! Well then, open for suggestions for the dashboard widget. Perhaps we can add the same "learn more" link to helphub there?

IMHO if we have a support link we don't need to mention XML sitemaps in the UI copy specifically.

---

@knutsp Wrong ticket to discuss this, but an interesting point. Can you perhaps add this to the ​make/core post or the ​GitHub repo for visibility?

---

We might want to hold off on discussing this ticket here in detail until that point is addressed

[pbiron]

Replying to swissspidy:

Anyway, my suggestion on Slack was to not add to much technical details (what are XML sitemaps? What is noindex?) to the copy there, and instead link to helphub.

Maybe something like this:

Discourage search engines from indexing this site
This will also disable XML sitmaps.
It is up to search engines to honor this request, and this setting won't prevent them from crawling your website. Learn more about search engine visibility.

I think adding a "Learn more" link (to a page on HelpHub) is a great idea, so that the text of this setting can be as simple as possible.

As mention during the ​#core-sitemaps meeting this morning, the URL of that link should be filterable. A sitemaps or SEO plugin might alter the core default behavior of what happens with sitemaps for a site that is not public, e.g., enabling sitemaps for testing purposes while still not adding it to robots.txt...and that plugin should be able to provide a link to their documentation of how the "Discourage search engines from indexing this site" setting affects sitemaps.