WordPress.org

Make WordPress Core

Opened 6 months ago

Last modified 4 weeks ago

#47577 new enhancement

Streamline detecting and enabling HTTPS

Reported by: flixos90 Owned by:
Milestone: Awaiting Review Priority: normal
Severity: normal Version:
Component: Administration Keywords: 2nd-opinion needs-unit-tests has-patch
Focuses: Cc:
PR Number:

Description

Of all the WordPress sites today, 63.4% are using HTTPS. While this is already better than the average for the entire web, it is far from optimal. More and more modern web APIs require usage of HTTPS, let alone the security implications of not using it.
In order to close that gap, it must be easier for administrators to switch their WordPress site to HTTPS, especially if it is already supported by their environment.

In order to provide accurate recommendations to site owners about switching their site to HTTPS, we need to know whether HTTPS is even supported by their server and domain. We have been reliably detecting HTTPS support in the PWA plugin for a while, and the same logic could be used in core.

Based on the result of the HTTPS support detection, we would recommend one of the following:

  • If supported, recommend to change the WordPress site URL, as that's all that's needed.
  • If not supported, recommend talking to the web host about enabling HTTPS.

This provide more accurate recommendations for the respective situation a site is in.

In order to properly enable HTTPS it is also crucial to not have mixed content links. Performing extensive database replacements is unfeasible for WordPress core itself, so we should instead replace URLs in content pointing to http:// versions of the page with their https:// counterparts on the fly. While this would be unnecessary for sites that properly have switched all their content to HTTPS, the overhead is minimal and acceptable. Last but not least, if somebody still doesn't want it, those checks should be removable easily because of the filter usage.

Attachments (5)

47577.diff (7.6 KB) - added by flixos90 6 months ago.
Screenshot 2019-06-20 at 13.47.12.png (65.5 KB) - added by flixos90 6 months ago.
Screenshot of when HTTPS is not enabled, but supported by the environment
Screenshot 2019-06-20 at 13.55.15.png (58.1 KB) - added by flixos90 6 months ago.
Screenshot of when HTTPS is not enabled and also not supported by the environment
47577.2.diff (10.6 KB) - added by miinasikk 3 months ago.
Includes adding Content-Security-Policy header with filter to opt out for Content-Security-Policy-Report-Only
Screenshot 2019-08-27 at 11.36.43.png (92.5 KB) - added by miinasikk 3 months ago.
HTTPS section with additional information about Content-Security-Policy

Download all attachments as: .zip

Change History (18)

@flixos90
6 months ago

#1 follow-up: @flixos90
6 months ago

  • Keywords has-patch added; needs-patch removed

47577.diff makes the following changes:

  • Introduce wp_is_using_https(), which detects whether the site uses HTTPS based on the WordPress site URL.
  • Introduce wp_is_https_supported() which detects whether HTTPS is supported by the environment (using a cached request to the HTTPS version of the site).
  • Enhance the Site Health test for HTTPS status to use the new function and provide accurate recommendation based on the results.
  • If HTTPS is active, filter and replace non-HTTP links to the site in content with their HTTPS counterparts.

@flixos90
6 months ago

Screenshot of when HTTPS is not enabled, but supported by the environment

@flixos90
6 months ago

Screenshot of when HTTPS is not enabled and also not supported by the environment

#2 @earnjam
6 months ago

It would be nice to allow filtering the Site Health action link/button for talking to your host about HTTPS in the same way we do for the PHP upgrade notices.

Hosts will have differing documentation about how to accomplish it on their environments and they may want to link directly to that.

#3 @johnbillion
6 months ago

Related: #28521 (for considerations related to filtering URL schemes).

#4 in reply to: ↑ 1 @westonruter
6 months ago

Replying to flixos90:

  • If HTTPS is active, filter and replace non-HTTP links to the site in content with their HTTPS counterparts.

I suggest also adding a upgrade-insecure-requests CSP directive to automatically handle this outside fo the content: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/upgrade-insecure-requests

#5 follow-up: @flixos90
5 months ago

@earnjam

It would be nice to allow filtering the Site Health action link/button for talking to your host about HTTPS in the same way we do for the PHP upgrade notices.

I like that suggestion, however I'd prefer if we approached this iteratively and opened a follow-up issue to make that URL filterable. The complexity we've learned about during the Servehappy project is that we probably wouldn't want them to replace the wordpress.org support URL, so we'd need to add more content and tweak the UI to allow for that, which would require more discussion.

@westonruter

I suggest also adding a upgrade-insecure-requests CSP directive to automatically handle this outside fo the content

That's worth exploring. I'm wondering whether that would cause problems with URLs pointing to external websites, that may still not be on HTTPS though - how does the directive deal with images or links from such websites? The other concern is that in order to add CSP headers into core, it may be better to work on a simple centralized solution as a developer API that would allow managing those directives.

#6 in reply to: ↑ 5 ; follow-up: @westonruter
5 months ago

Replying to flixos90:

I suggest also adding a upgrade-insecure-requests CSP directive to automatically handle this outside fo the content

That's worth exploring. I'm wondering whether that would cause problems with URLs pointing to external websites, that may still not be on HTTPS though - how does the directive deal with images or links from such websites? The other concern is that in order to add CSP headers into core, it may be better to work on a simple centralized solution as a developer API that would allow managing those directives.

Hyperlinks to other websites would be ignored since merely linking to another site does not cause a request. Images, videos, iframes, and other resources would need to be upgraded even on external domains as otherwise there would be insecure mixed content warnings.

Per MDN:

The upgrade-insecure-requests directive will not ensure that users visiting your site via links on third-party sites will be upgraded to HTTPS for the top-level navigation

#7 in reply to: ↑ 6 ; follow-up: @miinasikk
4 months ago

Replying to westonruter:

Hyperlinks to other websites would be ignored since merely linking to another site does not cause a request. Images, videos, iframes, and other resources would need to be upgraded even on external domains as otherwise there would be insecure mixed content warnings.

How would you handle the case where external images are not available via HTTPS? Or are you thinking that perhaps upgrade-insecure-requests could be optional? Or maybe require handling first and/or logging the issues that need fixing?

Per MDN:

Note that, if the requested resource is not actually available via HTTPS, the request will fail without any fallback to HTTP

#8 in reply to: ↑ 7 ; follow-up: @westonruter
4 months ago

Replying to miinasikk:

How would you handle the case where external images are not available via HTTPS? Or are you thinking that perhaps upgrade-insecure-requests could be optional? Or maybe require handling first and/or logging the issues that need fixing?

Yeah, I'm not sure. I think ultimately the insecure external assets would just need to fail. It's something the site owner (or theme/plugin developer) would need to fix (or find a replacement).

Without upgrade-insecure-requests browsers will still load insecure images. Other insecure subresources, scripts in particular, are blocked entirely. So it seems better to have this in place so that the upgrade is performed for the majority of subresources that should be available over HTTPS.

It would be nice if upgrade-insecure-requests could be configured to only upgrade non-image subresources, leaving images to flag insecure content warnings, but that doesn't seem to be the case.

So while upgrade-insecure-requests should be the default, it makes sense that a developer should be able to turn off that default behavior and instead opt to find insecure requests via:

Content-Security-Policy-Report-Only: default-src https:; report-uri /endpoint

This would require the support of a plugin like Reporting API. It's also not something that should be the default for all sites.

The 80% solution here—for the majority of users who don't know how to debug things like this—seems to be just to upgrade-insecure-requests. This should be come less and less of a problem as more sites support HTTPS.

#9 in reply to: ↑ 8 @miinasikk
4 months ago

Replying to westonruter:

The 80% solution here—for the majority of users who don't know how to debug things like this—seems to be just to upgrade-insecure-requests. This should be come less and less of a problem as more sites support HTTPS.

If we would add the upgrade-insecure-requests e.g. by adding 'wp_headers' filter to https-detection.php then this would mean that all the sites that already are using HTTPS would benefit from that. I'm wondering about the cases though where there might be hundreds of posts of which some might use third-party resources which might not support HTTPS, and this would happen without any notice about it after WP update. Not sure if there is a good way to inform about this behavior change, other than stating it in the added Security section. Or do you think it should probably be acceptable as-is since it would work for 80% of the cases? Not sure what's the general policy for these cases, is 80% rule the policy? :)

So while upgrade-insecure-requests should be the default, it makes sense that a developer should be able to turn off that default behavior and instead opt to find insecure requests via:

Content-Security-Policy-Report-Only: default-src https:; report-uri /endpoint

This would require the support of a plugin like Reporting API. It's also not something that should be the default for all sites.

We could add a filter which by default would add the upgrade-insecure-requests but when set to the opposite value (false vs true depending on the filter) would add Content-Security-Policy-Report-Only instead.

#10 @westonruter
4 months ago

If a site switches to HTTPS but they have HTTP resources, then (as I understand):

  1. Without upgrade-insecure-requests, insecure scripts will be blocked and insecure media will cause warnings.
  2. With upgrade-insecure-requests, all insecure resources will automatically be updated to HTTPS. If the hosts for the resources support HTTPS, then everything will work seamlessly. If a hosted resource is not available on HTTPS (scripts and media), then it will fail.

In both cases, the user would need to check their site to check for problems (unless it's for a new install). But my expectation is that upgrade-insecure-requests will generally result in fewer errors, though the ones that do occur (e.g. for images) will be harder (failures instead of warnings). So 2 seems like a better 80% solution.

@miinasikk
3 months ago

Includes adding Content-Security-Policy header with filter to opt out for Content-Security-Policy-Report-Only

@miinasikk
3 months ago

HTTPS section with additional information about Content-Security-Policy

#11 @miinasikk
3 months ago

Added the header + also information about Content-Security-Policy. That will only be visible if the HTTPS is not configured correctly though, the sites already using HTTPS won't see this. Not sure if this makes sense. Thoughts?

#12 follow-up: @flixos90
3 months ago

@westonruter

If a hosted resource is not available on HTTPS (scripts and media), then it will fail.

I'm okay with including upgrade-insecure-requests, but your point here leads me to believe that we should do a bit more to make sure it doesn't break users' sites. How about we also issue a request to a random media file, and potentially even a core asset, to make the same check (only if the root URL is different, e.g. when using a CDN)? Only if all of them support HTTPS, we'll tell the user that their site supports HTTPS. It is admittedly a special case, but probably enough people are using a CDN (and hopefully they're all good enough to support HTTPS) to make this worth having.

@miinasikk

Added the header + also information about Content-Security-Policy.

I'd prefer to not expose this to the user, at least not in here - it's very technical information and most folks will have no idea what it is. If we want to expose it, then I'd prefer just doing it in the Info part of Site Health, with all the other technical data.

After reading the previous discussion and reviewing the patch, I'd prefer if we just set the upgrade-insecure-requests value in the filter and not bothered about the very specific report-only cases. You'd need to use a filter to modify the behavior anyway, and you could just as well remove our filter if you wanted to do so. Let's keep the code simple here, covering the vast majority of cases. What we should probably do though is check, if the Content-Security-Policy header is already set, and if so append our value (maybe we should even check whether it's not already set). We don't wanna override other CSP headers people may be using already.

#13 in reply to: ↑ 12 @miinasikk
4 weeks ago

Replying to flixos90:

How about we also issue a request to a random media file, and potentially even a core asset, to make the same check (only if the root URL is different, e.g. when using a CDN)? Only if all of them support HTTPS, we'll tell the user that their site supports HTTPS. It is admittedly a special case, but probably enough people are using a CDN (and hopefully they're all good enough to support HTTPS) to make this worth having.

Are you suggesting that we should check if a random media file that's already on the site supports HTTPS and if it doesn't then consider the site not supporting HTTPS?

Some concerns with that:

  • We would check just one random media file. This would mean that it could happen that the chosen file is from CDN but it could also be from some other random source / be a leftover or irrelevant and not accurately reflect the site being ready for HTTPS.
  • How would you choose the random media file, could it differ at different times and have different results?
  • This wouldn't really reflect accurately if the site supports HTTPS, having external resources not supporting HTTPS means that the external site isn't supporting HTTPS, not the local.

If the CDN doesn't support HTTPS then maybe it would be good to change the service, too.

Also, the sites that aren't already supporting HTTPS aren't going to have the CSP header anyway -- these sites would actually need to take steps to convert the site to HTTPS where some issues can be expected. It would probably be unexpected for the sites already supporting HTTPS when the resources suddenly display broken, for those, however, the information about being or not being HTTPS ready wouldn't be relevant anymore. WDYT?

I'd prefer to not expose this to the user, at least not in here - it's very technical information and most folks will have no idea what it is. If we want to expose it, then I'd prefer just doing it in the Info part of Site Health, with all the other technical data.

Fair point that it's very technical information and out of the place. The technical information does make sense under Info more, however, we should first actually check if the header hasn't been overwritten by a filter before displaying the information about it. Also, if we already display the CSP header information then should others be displayed as well? Perhaps you're right and it's not that important to display it in such detail.

It would be good to have information under the Status with the HTTPS information though, however, not sure what would be a good way to tell that "External resources that don't support HTTPS might break after switching to HTTPS -- make sure that you don't have those!". Do you think it'd be unnecessary information for the user and we could just skip all the information part? Perhaps I'm overthinking.

After reading the previous discussion and reviewing the patch, I'd prefer if we just set the upgrade-insecure-requests value in the filter and not bothered about the very specific report-only cases. You'd need to use a filter to modify the behavior anyway, and you could just as well remove our filter if you wanted to do so.

That's true, however, I think that the report-only case would be helpful for transitioning, we could document the filter better to make it more clear what's it for, it's very likely that the report-only header is not something that everyone is aware of and the header would be just removed instead of trying to transition. So having that filter could help.

What we should probably do though is check, if the Content-Security-Policy header is already set, and if so append our value (maybe we should even check whether it's not already set). We don't wanna override other CSP headers people may be using already.

It wouldn't make sense to just append our value without knowing what's there before, that would cause duplicate directives and not really work (e.g. having default-src twice). Checking first if it already exists and adding the header only then is a good point, no need to overwrite the exising efforts.

Note: See TracTickets for help on using tickets.