"Silence is golden" scenarios should return a 404 HTTP header

[jonoaldersonwp]

Direct requests to system file structures/folders (e.g., ​https://yoast.com/app/wp-content/) currently return a blank page with a 200 HTTP status. This informs search engines and agents that the page/state is 'okay', which may result in them being crawled, indexed, or otherwise exposed.

This causes a number of SEO and analytics headaches, and, is generally poor practice.

Whilst these directories/files technically do exist, and therefore arguably should return a 200 status, the important thing to consider is that no user-facing page/content exists at the URL. We're not trying to represent the 'physical' file structure of the site (which we're already extremely abstracted from), we're trying to signpost what is, or isn't, a valid page/response. That an actual file/folder resides at the requested URL is irellevant.

In the short-term, we should simply set a 404 header in all 'Silence is golden' templates.

In the long-term, we should aim to find a way to allow all request to 'Silence is golden' files to return the 404 template of the active theme. This may come with complexities, where we can't rely on system configurations/capabilities or folder structures to be consistent between sites.

As an aspiration, we should aim that all requests to a WP site return an appropriate HTTP status and experience based on the user's context, regardless of the actual file/directory structure.

[vanyukov]

If it's just about adding headers - attached a patch. Also seems logical to increase the file size in get_mu_plugins(), to adjust for the new file size.

[santilinwp]

Hello, I would suggest changing all templates with the "silence is golden" sentence and adding a call to a function like:

wp_silence_is_golden();

that would handle the logic of redirecting to the 404 page of the current theme.

[desrosj]

Thanks for this detailed description, @jonoaldersonwp!

I am going to close this one out as a duplicate of #17619 because that ticket predates this one, and there is some relevant discussion there.