WordPress.org

Make WordPress Core

Opened 16 months ago

Last modified 3 days ago

#47577 new enhancement

Detect HTTPS support and provide guidance

Reported by: flixos90 Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: Security Keywords: has-patch has-unit-tests
Focuses: administration Cc:

Description (last modified by flixos90)

Of all the WordPress sites today, 63.4% are using HTTPS. While this is already better than the average for the entire web, it is far from optimal. More and more modern web APIs require usage of HTTPS, let alone the security implications of not using it.
In order to close that gap, WordPress should do better to actively recommend administrators to switch their non-HTTPS site to use HTTPS, especially if their current environment already technically supports it.

In order to provide accurate recommendations to site owners about switching their site to HTTPS, we need to know whether HTTPS is even supported by their server and domain. This has been reliably detected in the PWA plugin for a while, and similar logic could be used in core.

Based on the result of the HTTPS support detection, we would recommend one of the following:

  • If supported, recommend to change the WordPress site URL, as that's all that's needed.
  • If not supported, recommend talking to the web host about enabling HTTPS.

This provide more accurate recommendations for the respective situation a site is in. Then, in separate follow-up tickets, we should look at simplifying the migration from HTTP to HTTPS itself which today is far too complex for the majority of WordPress users.

Attachments (5)

47577.diff (7.6 KB) - added by flixos90 16 months ago.
Screenshot 2019-06-20 at 13.47.12.png (65.5 KB) - added by flixos90 16 months ago.
Screenshot of when HTTPS is not enabled, but supported by the environment
Screenshot 2019-06-20 at 13.55.15.png (58.1 KB) - added by flixos90 16 months ago.
Screenshot of when HTTPS is not enabled and also not supported by the environment
47577.2.diff (10.6 KB) - added by miinasikk 14 months ago.
Includes adding Content-Security-Policy header with filter to opt out for Content-Security-Policy-Report-Only
Screenshot 2019-08-27 at 11.36.43.png (92.5 KB) - added by miinasikk 14 months ago.
HTTPS section with additional information about Content-Security-Policy

Download all attachments as: .zip

Change History (25)

@flixos90
16 months ago

#1 follow-up: @flixos90
16 months ago

  • Keywords has-patch added; needs-patch removed

47577.diff makes the following changes:

  • Introduce wp_is_using_https(), which detects whether the site uses HTTPS based on the WordPress site URL.
  • Introduce wp_is_https_supported() which detects whether HTTPS is supported by the environment (using a cached request to the HTTPS version of the site).
  • Enhance the Site Health test for HTTPS status to use the new function and provide accurate recommendation based on the results.
  • If HTTPS is active, filter and replace non-HTTP links to the site in content with their HTTPS counterparts.

@flixos90
16 months ago

Screenshot of when HTTPS is not enabled, but supported by the environment

@flixos90
16 months ago

Screenshot of when HTTPS is not enabled and also not supported by the environment

#2 @earnjam
16 months ago

It would be nice to allow filtering the Site Health action link/button for talking to your host about HTTPS in the same way we do for the PHP upgrade notices.

Hosts will have differing documentation about how to accomplish it on their environments and they may want to link directly to that.

#3 @johnbillion
16 months ago

Related: #28521 (for considerations related to filtering URL schemes).

#4 in reply to: ↑ 1 @westonruter
16 months ago

Replying to flixos90:

  • If HTTPS is active, filter and replace non-HTTP links to the site in content with their HTTPS counterparts.

I suggest also adding a upgrade-insecure-requests CSP directive to automatically handle this outside fo the content: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/upgrade-insecure-requests

#5 follow-up: @flixos90
16 months ago

@earnjam

It would be nice to allow filtering the Site Health action link/button for talking to your host about HTTPS in the same way we do for the PHP upgrade notices.

I like that suggestion, however I'd prefer if we approached this iteratively and opened a follow-up issue to make that URL filterable. The complexity we've learned about during the Servehappy project is that we probably wouldn't want them to replace the wordpress.org support URL, so we'd need to add more content and tweak the UI to allow for that, which would require more discussion.

@westonruter

I suggest also adding a upgrade-insecure-requests CSP directive to automatically handle this outside fo the content

That's worth exploring. I'm wondering whether that would cause problems with URLs pointing to external websites, that may still not be on HTTPS though - how does the directive deal with images or links from such websites? The other concern is that in order to add CSP headers into core, it may be better to work on a simple centralized solution as a developer API that would allow managing those directives.

#6 in reply to: ↑ 5 ; follow-up: @westonruter
16 months ago

Replying to flixos90:

I suggest also adding a upgrade-insecure-requests CSP directive to automatically handle this outside fo the content

That's worth exploring. I'm wondering whether that would cause problems with URLs pointing to external websites, that may still not be on HTTPS though - how does the directive deal with images or links from such websites? The other concern is that in order to add CSP headers into core, it may be better to work on a simple centralized solution as a developer API that would allow managing those directives.

Hyperlinks to other websites would be ignored since merely linking to another site does not cause a request. Images, videos, iframes, and other resources would need to be upgraded even on external domains as otherwise there would be insecure mixed content warnings.

Per MDN:

The upgrade-insecure-requests directive will not ensure that users visiting your site via links on third-party sites will be upgraded to HTTPS for the top-level navigation

#7 in reply to: ↑ 6 ; follow-up: @miinasikk
15 months ago

Replying to westonruter:

Hyperlinks to other websites would be ignored since merely linking to another site does not cause a request. Images, videos, iframes, and other resources would need to be upgraded even on external domains as otherwise there would be insecure mixed content warnings.

How would you handle the case where external images are not available via HTTPS? Or are you thinking that perhaps upgrade-insecure-requests could be optional? Or maybe require handling first and/or logging the issues that need fixing?

Per MDN:

Note that, if the requested resource is not actually available via HTTPS, the request will fail without any fallback to HTTP

#8 in reply to: ↑ 7 ; follow-up: @westonruter
15 months ago

Replying to miinasikk:

How would you handle the case where external images are not available via HTTPS? Or are you thinking that perhaps upgrade-insecure-requests could be optional? Or maybe require handling first and/or logging the issues that need fixing?

Yeah, I'm not sure. I think ultimately the insecure external assets would just need to fail. It's something the site owner (or theme/plugin developer) would need to fix (or find a replacement).

Without upgrade-insecure-requests browsers will still load insecure images. Other insecure subresources, scripts in particular, are blocked entirely. So it seems better to have this in place so that the upgrade is performed for the majority of subresources that should be available over HTTPS.

It would be nice if upgrade-insecure-requests could be configured to only upgrade non-image subresources, leaving images to flag insecure content warnings, but that doesn't seem to be the case.

So while upgrade-insecure-requests should be the default, it makes sense that a developer should be able to turn off that default behavior and instead opt to find insecure requests via:

Content-Security-Policy-Report-Only: default-src https:; report-uri /endpoint

This would require the support of a plugin like Reporting API. It's also not something that should be the default for all sites.

The 80% solution here—for the majority of users who don't know how to debug things like this—seems to be just to upgrade-insecure-requests. This should be come less and less of a problem as more sites support HTTPS.

#9 in reply to: ↑ 8 @miinasikk
15 months ago

Replying to westonruter:

The 80% solution here—for the majority of users who don't know how to debug things like this—seems to be just to upgrade-insecure-requests. This should be come less and less of a problem as more sites support HTTPS.

If we would add the upgrade-insecure-requests e.g. by adding 'wp_headers' filter to https-detection.php then this would mean that all the sites that already are using HTTPS would benefit from that. I'm wondering about the cases though where there might be hundreds of posts of which some might use third-party resources which might not support HTTPS, and this would happen without any notice about it after WP update. Not sure if there is a good way to inform about this behavior change, other than stating it in the added Security section. Or do you think it should probably be acceptable as-is since it would work for 80% of the cases? Not sure what's the general policy for these cases, is 80% rule the policy? :)

So while upgrade-insecure-requests should be the default, it makes sense that a developer should be able to turn off that default behavior and instead opt to find insecure requests via:

Content-Security-Policy-Report-Only: default-src https:; report-uri /endpoint

This would require the support of a plugin like Reporting API. It's also not something that should be the default for all sites.

We could add a filter which by default would add the upgrade-insecure-requests but when set to the opposite value (false vs true depending on the filter) would add Content-Security-Policy-Report-Only instead.

#10 @westonruter
14 months ago

If a site switches to HTTPS but they have HTTP resources, then (as I understand):

  1. Without upgrade-insecure-requests, insecure scripts will be blocked and insecure media will cause warnings.
  2. With upgrade-insecure-requests, all insecure resources will automatically be updated to HTTPS. If the hosts for the resources support HTTPS, then everything will work seamlessly. If a hosted resource is not available on HTTPS (scripts and media), then it will fail.

In both cases, the user would need to check their site to check for problems (unless it's for a new install). But my expectation is that upgrade-insecure-requests will generally result in fewer errors, though the ones that do occur (e.g. for images) will be harder (failures instead of warnings). So 2 seems like a better 80% solution.

@miinasikk
14 months ago

Includes adding Content-Security-Policy header with filter to opt out for Content-Security-Policy-Report-Only

@miinasikk
14 months ago

HTTPS section with additional information about Content-Security-Policy

#11 @miinasikk
14 months ago

Added the header + also information about Content-Security-Policy. That will only be visible if the HTTPS is not configured correctly though, the sites already using HTTPS won't see this. Not sure if this makes sense. Thoughts?

#12 follow-up: @flixos90
14 months ago

@westonruter

If a hosted resource is not available on HTTPS (scripts and media), then it will fail.

I'm okay with including upgrade-insecure-requests, but your point here leads me to believe that we should do a bit more to make sure it doesn't break users' sites. How about we also issue a request to a random media file, and potentially even a core asset, to make the same check (only if the root URL is different, e.g. when using a CDN)? Only if all of them support HTTPS, we'll tell the user that their site supports HTTPS. It is admittedly a special case, but probably enough people are using a CDN (and hopefully they're all good enough to support HTTPS) to make this worth having.

@miinasikk

Added the header + also information about Content-Security-Policy.

I'd prefer to not expose this to the user, at least not in here - it's very technical information and most folks will have no idea what it is. If we want to expose it, then I'd prefer just doing it in the Info part of Site Health, with all the other technical data.

After reading the previous discussion and reviewing the patch, I'd prefer if we just set the upgrade-insecure-requests value in the filter and not bothered about the very specific report-only cases. You'd need to use a filter to modify the behavior anyway, and you could just as well remove our filter if you wanted to do so. Let's keep the code simple here, covering the vast majority of cases. What we should probably do though is check, if the Content-Security-Policy header is already set, and if so append our value (maybe we should even check whether it's not already set). We don't wanna override other CSP headers people may be using already.

#13 in reply to: ↑ 12 @miinasikk
11 months ago

Replying to flixos90:

How about we also issue a request to a random media file, and potentially even a core asset, to make the same check (only if the root URL is different, e.g. when using a CDN)? Only if all of them support HTTPS, we'll tell the user that their site supports HTTPS. It is admittedly a special case, but probably enough people are using a CDN (and hopefully they're all good enough to support HTTPS) to make this worth having.

Are you suggesting that we should check if a random media file that's already on the site supports HTTPS and if it doesn't then consider the site not supporting HTTPS?

Some concerns with that:

  • We would check just one random media file. This would mean that it could happen that the chosen file is from CDN but it could also be from some other random source / be a leftover or irrelevant and not accurately reflect the site being ready for HTTPS.
  • How would you choose the random media file, could it differ at different times and have different results?
  • This wouldn't really reflect accurately if the site supports HTTPS, having external resources not supporting HTTPS means that the external site isn't supporting HTTPS, not the local.

If the CDN doesn't support HTTPS then maybe it would be good to change the service, too.

Also, the sites that aren't already supporting HTTPS aren't going to have the CSP header anyway -- these sites would actually need to take steps to convert the site to HTTPS where some issues can be expected. It would probably be unexpected for the sites already supporting HTTPS when the resources suddenly display broken, for those, however, the information about being or not being HTTPS ready wouldn't be relevant anymore. WDYT?

I'd prefer to not expose this to the user, at least not in here - it's very technical information and most folks will have no idea what it is. If we want to expose it, then I'd prefer just doing it in the Info part of Site Health, with all the other technical data.

Fair point that it's very technical information and out of the place. The technical information does make sense under Info more, however, we should first actually check if the header hasn't been overwritten by a filter before displaying the information about it. Also, if we already display the CSP header information then should others be displayed as well? Perhaps you're right and it's not that important to display it in such detail.

It would be good to have information under the Status with the HTTPS information though, however, not sure what would be a good way to tell that "External resources that don't support HTTPS might break after switching to HTTPS -- make sure that you don't have those!". Do you think it'd be unnecessary information for the user and we could just skip all the information part? Perhaps I'm overthinking.

After reading the previous discussion and reviewing the patch, I'd prefer if we just set the upgrade-insecure-requests value in the filter and not bothered about the very specific report-only cases. You'd need to use a filter to modify the behavior anyway, and you could just as well remove our filter if you wanted to do so.

That's true, however, I think that the report-only case would be helpful for transitioning, we could document the filter better to make it more clear what's it for, it's very likely that the report-only header is not something that everyone is aware of and the header would be just removed instead of trying to transition. So having that filter could help.

What we should probably do though is check, if the Content-Security-Policy header is already set, and if so append our value (maybe we should even check whether it's not already set). We don't wanna override other CSP headers people may be using already.

It wouldn't make sense to just append our value without knowing what's there before, that would cause duplicate directives and not really work (e.g. having default-src twice). Checking first if it already exists and adding the header only then is a good point, no need to overwrite the exising efforts.

#14 @flixos90
10 months ago

@miinasikk @westonruter

I wonder whether we should take a step back here and not upgrade-insecure-requests for now, except those that go against the actual site URL. While this can just as well cause issues because of e.g. media or assets on a separate host, I think it keeps the work here more scoped. Exploring upgrade-insecure-requests could be the second step - I'm afraid this gets lost in a can of worms otherwise.

Arguably, most WordPress sites serve all their files from the same origin, so for those the simple HTTP to HTTPS replacement should work. The complexity of checking for files that are served from different origins leads me to think that we should defer that work for now. Potentially it even is plugin territory: For example, a plugin hooking media up a CDN could (if that CDN doesn't already use HTTPS anyway) make use of wp_is_using_https() to act accordingly.

Last but not least, we need to keep in mind that users can at least change their URLs back to HTTP, should any resource unexpectedly cause their site to break.

#15 @westonruter
10 months ago

As noted in No More Mixed Messages About HTTPS, Chromium is going to start blocking all non-HTTP subresources on pages loaded over HTTPS:

In a series of steps starting in Chrome 79, Chrome will gradually move to blocking all mixed content by default. To minimize breakage, we will autoupgrade mixed resources to https://, so sites will continue to work if their subresources are already available over https://. Users will be able to enable a setting to opt out of mixed content blocking on particular websites, and below we’ll describe the resources available to developers to help them find and fix mixed content.

It states that HTTP-resources will be autoupgraded to HTTPS, at least audio, video, and images. However, it doesn't mention scripts or iframes.

So if WordPress served upgrade-insecure-requests it would seem that this would preempt what Chromium is already doing. It would prevent broken media as well eventually prevent broken scripts and iframes which would not get automatically upgraded, as far as I understand.

I'm not sure what Firefox or Safari are planning in this regard.

#16 @flixos90
3 weeks ago

Revisiting this ticket, I believe we're suffering of some scope creep here, so it would make sense to split this ticket into multiple (closely related, but different) efforts:

  1. Detect HTTPS support and provide guidance
  2. Streamline migrating from HTTP to HTTPS
  3. Upgrade insecure requests when site is using HTTPS

Technically the above order would also be a dependency chain for these three, but discussion on the different efforts can happen separately and focus on one topic.

I'm going to move forward and simplify this ticket to cover 1., then open new tickets for 2. and 3.

#17 @flixos90
3 weeks ago

  • Component changed from Administration to Security
  • Description modified (diff)
  • Focuses administration added
  • Keywords needs-patch added; 2nd-opinion has-patch removed
  • Milestone changed from Awaiting Review to Future Release
  • Summary changed from Streamline detecting and enabling HTTPS to Detect HTTPS support and provide guidance

I've reduced scope of this ticket as mentioned above and opened #51437 and #51438 for the above tasks 2. and 3.

This ticket was mentioned in PR #568 on WordPress/wordpress-develop by felixarntz.


3 weeks ago

  • Keywords has-patch added; needs-patch removed

Trac ticket: https://core.trac.wordpress.org/ticket/47577

## Screenshots

HTTPS already supported:
<img width="1064" alt="Screenshot 2020-10-02 at 14 01 56" src="https://user-images.githubusercontent.com/3531426/94969916-67638100-04b8-11eb-9ca7-892b4f23ac61.png">

HTTPS not supported:
<img width="1065" alt="Screenshot 2020-10-02 at 14 02 57" src="https://user-images.githubusercontent.com/3531426/94969972-7d714180-04b8-11eb-8a52-b176972d3c14.png">

#19 @flixos90
4 days ago

  • Keywords has-unit-tests added; needs-unit-tests removed

In the latest commit in the PR https://github.com/WordPress/wordpress-develop/pull/568, I've added unit test coverage. I've also added a fix where now the siteurl option is accessed directly in wp_is_https_supported(). This is necessary because otherwise (via site_url()) the URL's scheme will be adjusted based on whether the current request uses HTTPS or not, which shouldn't affect the URL set here. @westonruter What do you think about this? I would think this may be an issue for the PWA plugin similarly?

#20 @westonruter
3 days ago

@flixos90 Yeah, calling site_url() to determine the underlying URL scheme seems problematic since set_url_scheme() will override that underlying scheme.

So it seems that get_option('siteurl') will work, and it will also account for the case where someone has defined the siteurl via the WP_SITEURL constant, since this is enforced via a lower-level option_siteurl filter rather than the site_url filter.

The only concern I have is if a site is using the site_url filter to manipulate the siteurl to be something else rather than what is returned by get_option('siteurl'). I suppose that would be a rare scenario. But one possibility to ensure to account for that would be to apply the filters yourself:

<?php
$site_url = apply_filters( 'site_url', get_option( 'siteurl' ), '/', null, get_current_blog_id() );
Note: See TracTickets for help on using tickets.