#16416 closed defect (bug) (fixed)
Settings -> Privacy "block search engines" option is not accurate
Reported by: | designsimply | Owned by: | westi |
---|---|---|---|
Milestone: | 3.5 | Priority: | normal |
Severity: | normal | Version: | |
Component: | Administration | Keywords: | has-patch ux-feedback dev-feedback |
Focuses: | Cc: |
Description
There is an option on the Settings -> Privacy page says it will "block search engines", but that does not always happen because checking the option does not block posts already established in the search engine. Proposing updating the language so it's more accurate.
Current:
"I would like to block search engines, but allow normal visitors"
Proposed:
"I would like to ask search engines not to index this site, but allow normal visitors"
Attachments (15)
Change History (78)
#2
@
14 years ago
checking the option does not block posts already established in the search engine.
I'm not sure what you mean. If a robot is blocked from re-visiting a URL, the search engine will eventually remove that URL from it's index.
Therefore, I think the text is accurate.
#5
follow-up:
↓ 6
@
14 years ago
- Component changed from General to Administration
- Keywords 3.2-early added
- Milestone changed from Awaiting Review to Future Release
- Owner set to westi
- Status changed from new to accepted
Ask and normal sound fine to me.
robots.txt is only a request and only obeyed by good citizen search engines - in some cases bad citizens of the web will use them to find hidden content.
Normal was already in the string for a long time and sounds fine to me.
#6
in reply to:
↑ 5
@
14 years ago
Replying to westi:
Ask and normal sound fine to me.
robots.txt is only a request and only obeyed by good citizen search engines - in some cases bad citizens of the web will use them to find hidden content.
Normal was already in the string for a long time and sounds fine to me.
FWIW, I've recently had several test domains be indexed by Google after implementing analytics on them, even though I had robots.txt
in full disallow mode. I had to request removal to get the pages out of the index, and I had to write a plugin that returned 403 for any page Googlebot visited on those sites to keep it out of the index. YMMV.
#7
@
14 years ago
Issues with robots.txt are not limited to "misbehaving" search engines - pages from sites blocked with robots.txt can show up in Google or any other search engine.
I like "ask" since it's technically accurate while still being human, but I don't think "normal" is necessary since search engines should not really be thought of as visitors anyway. Instead, how about:
I would like to ask search engines not to index this site, but allow direct visitors
Also, hope this would not be considered hijacking the thread, but I think the "I would like" part is unnecessary and can be removed - those three words are wasting too much valuable user concentration that can be used on figuring out this not very intuitive concept.
#9
@
14 years ago
There are a lot of text issues on this screen. "Site Visibility" isn't correct. It's visible either way. It's about what you're asking robots (Search Engines is probably a fine colloquial term for robots) to do.
"I would like..." "I would like..." lose those.
Maybe frame the whole thing in terms of search engines.
Search Engine Visibility:
- Allow search engines to index this site.
- Ask search engines to not index this site.
Note: neither of these options physically blocks access to your site — it is up to search engines to honor your request.
#11
follow-up:
↓ 14
@
14 years ago
Nevermind "100%" access. It doesn't block "1%" access. Maybe just "Neither of these options forcibly blocks access"
#13
@
13 years ago
- Cc heymrpro@… added
I don't think [18554] creates a need to change anything in the help tab for this screen, but someone might want to double-check that. The Right Now box in the Dashboard still says "Search Engines Blocked"; about to upload a patch for that.
#14
in reply to:
↑ 11
@
13 years ago
I like where this has gone!
Replying to markjaquith:
Nevermind "100%" access. It doesn't block "1%" access. Maybe just "Neither of these options forcibly blocks access"
What is the thought behind using the term "neither"? It makes me think that *both* are requests not to index.
What is lost if the options were more vertically separated and the note reads something like:
Note: This option doesn't block access to your site — it is up to search engines to honor your request.
#15
@
13 years ago
+1 to Mark Jaquith's suggestion to rebuild the whole thing around search engines, and remove "normal visitors" out of the equation.
However, once you go down that road, I think it makes the most sense to just remove this whole page and replace it with a single toggle box someplace: "Prevent search engines from indexing my site".
#16
@
13 years ago
+1 on tranforming it into a "Privacy" section with a single checkbox on the General Settings page.
#18
@
13 years ago
+1 on tranforming it into a "Privacy" section with a single checkbox on the General Settings page.
Might it not be more aptly included in the Reading settings? I can give a patch for this a go, but I think it might be more suited for inclusion in the Reading settings.
#20
@
13 years ago
See also: #18841 - Privacy settings are useless when Default permalinks are enabled
#21
@
13 years ago
- Cc mike@… added
I started working on #18841 and couldn't help but take a shot at renaming 'Privacy' to 'Search Indexing.'
Language changes are always hard:
- I choose this to try to reflect what is happening while leaving the door open for search-related items like sitemaps or SEO options (knowing those are usually handled by plugins)
Note, this patch also updates the Help menu to the new format from #18690. With those upgraded Help menus, moving this single setting to the Reading settings seems reasonable. The Reading settings could have the current Privacy help-info added as an option-specific tab.
#22
@
13 years ago
That latest .5 diff moves the setting from the Privacy Options page to the Reading Options page.
I think I prefer keeping it on its own page. That's so SEO & Sitemap plugins might use that existing page with add_settings_section()/add_settings_field() instead of always building their own settings pages.
#23
@
13 years ago
Hope it's OK to put in a reminder that the related UI text on the Dashboard screen, which links to this, needs re-wording too.
#24
@
13 years ago
Good reminder. One of my own: My patches don't affect the main navigation in WP-Admin, just starting points right now.
#25
follow-up:
↓ 26
@
13 years ago
Is this ticket still "milestone: future release" given that [18554] is in the beta? I believe there's still an incompatibility bug with the related phrasing on the Dashboard screen.
#26
in reply to:
↑ 25
@
13 years ago
Replying to dougwrites:
Is this ticket still "milestone: future release" given that [18554] is in the beta?
Moving Privacy settings to General/Reading settings is definitely a Future Release material, but RightNowBlocked.patch would probably make sense for 3.3 in terms of consistency. 16416.RightNowBlocked.2.patch is a slight variation of the text.
#29
@
12 years ago
- Milestone changed from Future Release to 3.5
I'd like to eliminate options-privacy.php from 3.5. I was originally moving it to options-general.php. I understand why options-reading.php is enticing. Given core's usage, reading makes sense. If however WP.com were to dissolve options-privacy, they'd probably move it to general. That is because WP.com (like some plugins) add an actual "private site" option here.
Technically, a site being private only applies to the frontend, though, so reading is still not incorrect. I'm also not too concerned about redirecting plugins' privacy sections and fields to reading.
16416.general.diff moves it to general and adds a "Date and Time" group to split things up over there. Moving it to options-reading will be a bit cleaner, but does require more than 16416.5.diff.
#30
follow-up:
↓ 31
@
12 years ago
- Keywords has-patch ux-feedback added; needs-patch removed
16416.reading.diff shifts blog_public to options-reading instead. It is also a bit more inclusive, catching options-privacy.php links on the dashboard, options-writing, and the welcome panel. (It was removed from the welcome panel because they just got through installing WP, where they were asked, and I don't think "search engine visibility" is that important otherwise, such as for multisite welcome's.)
It also toys with an idea for how to present blog_public. We can call it 'Search Engine Visibility' unless someone is using the blog_privacy_selector action, in which case it can revert to 'Site Visibility' automatically.
Additionally, when the action isn't used, we can get away with a checkbox only. While checkboxes are generally cleaner, the radio buttons do give you more allowances with the text; specifically, "Ask search engines not to...". It's easier to describe the negative "ask not" rather than the positive "allow", and it's more accurate, which is not something you can do with a checkbox.
I like how the label changes based on the action usage; not sold on the checkbox.
#31
in reply to:
↑ 30
@
12 years ago
Replying to nacin:
16416.reading.diff shifts blog_public to options-reading
I'd prefer this over settings->general. Reading is the section I'd look to first (if I didn't know where to look). Plus, I'm not a huge fan of the "general" section, as it's not super transparent what I'll find there without clicking the link.
#32
follow-up:
↓ 33
@
12 years ago
Note that there's some text and link under Update Services on the Settings -> Writing screen which relates to the same setting and links to the Privacy Settings screen.
#33
in reply to:
↑ 32
@
12 years ago
Replying to johnbillion:
Note that there's some text and link under Update Services on the Settings -> Writing screen which relates to the same setting and links to the Privacy Settings screen.
Yep, that's in 16416.reading.diff.
#34
@
12 years ago
+1 for 16416.general.diff and the label changes. Also undecided on the checkbox
In .5 I put these into Reading because it seemed easily justified. But I consider General a better place:
Search indexing is a core consideration when you're running any site, so these should be important enough to go front & center in Options.
(Have to admit that using WP.com as an argument for putting these into General almost made me vote for Reading. Petty jealousy though, I wish .org had more in common with .com at times)
What do you think about adding a hook to stop output of these 'Site Visibility' settings?
If a plugin like 'Registered Users Only' or WP.com creates a level of site privacy that overrides these settings, then hiding them from the UI would simplify Options & reduces the chance of users' confusion.
#37
@
12 years ago
I wanted to get this in. What still needs review after [21838] —
- The text/language in general.
- Whether we can invert the checkbox, as the options for language is better when inverted. I think we can get away with this if we set value="0" on the box and then in sanitize_option() convert === null to 1. (As options.php uses 'null' when nothing is posted... The fix in [21292] makes this actually distinguishable now.)
- Whether we should register this whole option using add_settings_field(), to hypothetically allow it to be unhooked. Note that there's no actual API for removing fields and sections, but I'd consider that to be good enough for now. Note also #21507.
#38
@
12 years ago
16416.diff inverts the checkbox, with the label "Ask search engines to <strong>not index</strong> this site".
options.php sets $value to be null by default. It then pulls the value from $_POST (which can only be a string or array), trims that value if it is not an array, and calls stripslashes_deep() on it (which only affects strings, and strings inside arrays).
The problem is that the trim occurs even if the value was not pulled from POST. That means null is converted to a string, and my trick doesn't work. A slight tweak means null can be passed to update_option(), which converts it to an empty string anyway, so there's no change in functionality — but, it provides some extra context in sanitize_option().
How it looks: http://cl.ly/image/2C0p3M1y2f41
#39
@
12 years ago
+1 registering this with add_settings_field()
(but I still prefer this in Options > General)
Only slightly off topic:
Seems like an API for hiding sections and/or fields is a good idea. One of the top requests from site owners is to hide individual settings or to re-mix the settings pages. An API for that would enable this instead of hiding whole admin pages based on user levels.
#40
@
12 years ago
#43
@
12 years ago
- Resolution fixed deleted
- Status changed from closed to reopened
Re-opening to discuss hook-ability.
#44
follow-ups:
↓ 45
↓ 61
@
12 years ago
Do we want to apply [21851] to wp-admin/install.php
and wp-signup.php
as well?
http://core.trac.wordpress.org/browser/tags/3.4.2/wp-admin/install.php#L132
http://core.trac.wordpress.org/browser/tags/3.4.2/wp-signup.php#L95
#45
in reply to:
↑ 44
@
12 years ago
Replying to SergeyBiryukov:
Do we want to apply [21851] to
wp-admin/install.php
andwp-signup.php
as well?
http://core.trac.wordpress.org/browser/tags/3.4.2/wp-admin/install.php#L132
http://core.trac.wordpress.org/browser/tags/3.4.2/wp-signup.php#L95
I think we should remove the "privacy" phrases from these pages to speed up the install/sign-up process. A new user shouldn't faced with such a special option.
#47
@
12 years ago
- Cc xoodrew@… added
16416.4-help.diff Updates the help text from 'radio button' to 'checkbox' and the new option label from r21851
#48
@
12 years ago
This ticket was reopened to discuss hook-ability, I hope it's OK to still comment on copy.
"Discourage search engines from indexing this site"
- The term "discourage" doesn't seem right for a setting. It's ambiguous about what the setting actually does, whereas something like "ask" that was suggested before is a more accurate description. Also, the note "it is up to search engines to honor your request" doesn't make as much sense in the context of "discourage".
- "indexing this site" requires you to understand how search engines work. Something like "ask search engines to exclude your site from search results" might be less accurate but more helpful to users.
It might even be worth considering something like "Ask Google, Bing and Yahoo to exclude your site from search results".
#49
@
12 years ago
This was discussed nearly ad nauseam during an IRC meeting: https://irclogs.wordpress.org/chanlog.php?channel=wordpress-dev&day=2012-08-29&sort=asc#m447025.
That said, I could re-consider based on "Ask search engines to exclude your site from search results".
#50
@
12 years ago
- Keywords dev-feedback added
From what I see, this is what's left:
- 16416.4-help.diff Updates the help text to match [21851], but could be adjusted if the language changes again.
- Decide whether to remove the checkbox from wp-signup and/or install
- Decide on hook-ability
#52
@
12 years ago
We might as well propagate the updated text to signup and install, although I'd be fine with punting that.
#53
@
12 years ago
16416.5.diff changes the install/signup language. It's flipped to the positive because the box is checked by default.
#54
follow-up:
↓ 55
@
12 years ago
As mentioned in comment:45, 16416.6.patch removes the privacy search engines option from the signup/install page.
#55
in reply to:
↑ 54
@
12 years ago
Replying to ocean90:
As mentioned in comment:45, 16416.6.patch removes the
privacysearch engines option from the signup/install page.
Good call. If we remove it, the option still defaults to 'on', correct?
#56
@
12 years ago
I'm okay with removing this on install. I'm not sure I'd like to mess with wp-signup.php at all - we know basically nothing about how this page is used in the wild.
I'm also quite okay with closing this as fixed.
#58
@
12 years ago
- Resolution set to fixed
- Status changed from reopened to closed
Let's close this for 3.5. We can revisit install.php and wp-signup.php in 3.6, as well as any pluggable or language changes. Nice work, everyone.
#61
in reply to:
↑ 44
@
11 years ago
Replying to SergeyBiryukov:
Do we want to apply [21851] to
wp-admin/install.php
andwp-signup.php
as well?
Follow-up: #27628
I'm not keen on the use of the term "ask", nor the term "normal". What about some thing like:
"I don't want search engines to index this site. This does not block visitors, and unfortunately, this may not affect what content search engines have already indexed."