Make WordPress Core

Opened 16 months ago

Last modified 11 days ago

#28507 accepted task (blessed)

Secure oEmbeds

Reported by: johnbillion Owned by: johnbillion
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: Embeds Keywords: ongoing
Focuses: Cc:

Description (last modified by johnbillion)

We need to audit our oEmbed providers and determine:

  • Which ones don't support embedding an https URL
  • Which ones don't support embedding content over SSL

If we have providers in core which do not support embedding content over SSL then we (or the WP.com team) should make contact and see if they're open to implementing it. This is pretty much a prerequisite for #28249 as it stands.

Problem providers:

ProviderCore supports HTTPS URLEndpoint recognises HTTPS URLEmbed supports HTTPSNotes
dai.lyNoNo-Invalid SSL certificate (points to dailymotion.com)
hulu.comYesYesNoInvalid SSL certificate (points to Akamai)
photobucket.comNoYesNoSite doesn't resolve over HTTPS
poll.fmYesYesYesInvalid SSL certificate (points to polldaddy.com)
instagr.amYesYes-Invalid SSL certificate (points to instagram.com)
ted.comYesYesYesAlmost there, just some mixed content in embeds
kck.stYesYes-Domain isn't available over HTTPS

Recently fixed providers:

  • flic.kr - HTTPS everywhere. Regex corrected in r28834.
  • slideshare.net - HTTPS embeds since r28834.
  • wordpress.tv - HTTPS embeds for HTTPS URLs.
  • meetup.com and meetu.ps- HTTPS embeds for HTTPS URLs.
  • instagram.com - HTTPS everywhere since r31710.
  • dailymotion.com - Uses the HTTPS oEmbed endpoint since r34587.
  • smugmug.com - Embeds now use HTTPS by default.
  • funnyordie.com - Embeds are now protocol-relative and cert is now valid.
  • imgur.com - Embeds are now protocol-relative.
  • collegehumor.com - HTTPS embeds for HTTPS URLs.
  • animoto.com and video214.com - Embeds now use HTTPS by default. See also r34588.

Ok providers:

  • youtube.com and youtu.be - HTTPS embeds via the scheme=https parameter.
  • vimeo.com - Embeds are protocol-relative.
  • flickr.com - HTTPS everywhere (same for flic.kr).
  • polldaddy.com - Embeds are served over HTTPS if the parent container uses HTTPS. Effectively protocol-relative via JavaScript.
  • twitter.com - HTTPS everywhere.
  • soundcloud.com - HTTPS everywhere. (Minor note: their oEmbed response includes an http URL for the thumbnail on their CDN, but it resolves over https if you change it.)
  • rdio.com and rd.io - HTTPS embeds by default.
  • spotify.com - HTTPS everywhere.
  • issuu.com - Embeds are served over HTTPS if the parent container uses HTTPS. Effectively protocol-relative via JavaScript.
  • mixcloud.com - Embeds are protocol-relative.
  • tumblr.com - Embeds are partly HTTPS and partly protocol-relative.
  • vine.co - HTTPS everywhere.
  • scribd.com - HTTPS embeds by default.

Attachments (1)

28507.diff (2.5 KB) - added by johnbillion 16 months ago.

Download all attachments as: .zip

Change History (63)

comment:1 @johnbillion16 months ago

Immediate task list:

  • Add support for https URLs for flic.kr.
  • Switch oEmbed endpoint to https for flickr.com, flic.kr, and slideshare.net.
Last edited 14 months ago by johnbillion (previous) (diff)

comment:2 @ircbot16 months ago

This ticket was mentioned in IRC in #wordpress-dev by johnbillion. View the logs.

@johnbillion16 months ago

comment:3 @johnbillion16 months ago

  • Keywords has-patch added

28507.diff is a patch for the above immediate task list.

comment:4 @ircbot16 months ago

This ticket was mentioned in IRC in #wordpress-dev by johnbillion. View the logs.

comment:5 @jkudish16 months ago

As part of some partnership work that I'm doing for WordPress.com, I have been in contact with engineers at Instagram. I contacted them about this issue today. I'll reply back on this ticket when I hear back.

My team at Automattic (we work on partnerships) will see if we can contact most of the other providers as well.

comment:6 @johnbillion16 months ago

That would be fantastic jkudish, thank you. Do also keep us posted on what wp.com are planning with relation to their SSL switchover.

comment:7 @johnbillion16 months ago

In 28834:

Switch to SSL for the Flickr and Slideshare oEmbed endpoints. Add support for SSL embeds on flic.kr. See #28507.

comment:8 @ircbot16 months ago

This ticket was mentioned in IRC in #wordpress-dev by wonderboymusic. View the logs.

comment:9 @azaozz16 months ago

Related: #28195 and the first-run patch for handling secure oEmbed in the admin (wpView) https://core.trac.wordpress.org/attachment/ticket/28195/28195.16.patch.

We may need to change some of the regexp/strings for the providers in order to easily detect which of them support https.

Last edited 16 months ago by azaozz (previous) (diff)

comment:10 @azaozz15 months ago

In 28919:

Secure embeds in the editor (first run):

  • When the user pastes an embeddable http URL, try to get the https embed.
  • If an embed provider doesn't support ssl embeds, show a placeholder/error message.
  • Revise the way we return error messages.

See #28195, #28507.

comment:11 @DrewAPicture15 months ago

In 28949:

Introduce an annotated list of oEmbed providers, their flavors, whether they support SSL, and when they were added to the oembed_providers filter docs.

See #28507.
Fixes #28372.

comment:12 @DrewAPicture15 months ago

In 28950:

Remove duplicate of the 'oembed_providers' filter accidentally introduced in [28949].

Move annoted table of oEmbed providers into the existing filter docs.

See #28507.
Fixes #28372.

comment:13 @ircbot15 months ago

This ticket was mentioned in IRC in #wordpress-dev by azaozz. View the logs.

comment:14 @stephdau15 months ago

This is to let you know that WordPress.tv now has a valid SSL certificate, and could be whitelisted as an SSL-Ok providers.

comment:15 @stephdau15 months ago

WordPress.tv: Actually, not fully true [yet], sorry... It does have a valid SSL cert, but work needs to be done for embeds to be protocol-relative ready, etc.

comment:16 @johnbillion15 months ago

Thanks Stephane. It also looks like the oEmbed endpoint fails if the scheme of the video URL doesn't match the scheme of the endpoint URL. For example this fails:


comment:17 @stephdau15 months ago

Going to look into either making it work, or find the right people to get it to, this morning.
I'll get back to you. :)

comment:18 @stephdau15 months ago

WordPress.tv update: I have the protocol-relative and mismatching schemes figured out, getting a patch reviewed before applying it (wp.tv runs on wp.com).

On the other hand, I'm hitting a temporary wall with wp.tv itself playing its videos (though VideoPress) under SSL. Digging further into that.

comment:19 @stephdau15 months ago

Happy to report that https://wordpress.tv/ is now SSL-ready, for playback on the site, or embedded.

oEmbed responses now use protocol-relative URLs for the video location, which work under both HTTP and HTTPS.


{"type":"video","version":"1.0","title":null,"width":400,"height":225,"html":"<embed src=\"\/\/v.wordpress.com\/3JiLCPst\" type=\"application\/x-shockwave-flash\" width=\"400\" height=\"225\" allowscriptaccess=\"always\" allowfullscreen=\"true\" wmode=\"transparent\"><\/embed>"}


<?xml version="1.0" encoding="utf-8" standalone="yes"?>
	<html>&lt;embed src=&quot;//v.wordpress.com/3JiLCPst&quot; type=&quot;application/x-shockwave-flash&quot; width=&quot;400&quot; height=&quot;225&quot; allowscriptaccess=&quot;always&quot; allowfullscreen=&quot;true&quot; wmode=&quot;transparent&quot;&gt;&lt;/embed&gt;</html>

The protocol mismatch issue was also handled.

The most network-inquisitives in the crowd might notice some 404s in the load process, upon playback, but those will soon be handled as well, and should not affect playback on the site, or embedded. :)

We're also hoping to look into iframe support in the future, but no timeframe on that.

comment:20 @johnbillion15 months ago

In 29110:

Add support for secure wordpress.tv embeds (thanks stephdau). See #28507.

comment:21 @ircbot15 months ago

This ticket was mentioned in IRC in #wordpress-dev by DrewAPicture. View the logs.

comment:22 follow-up: @johnbillion15 months ago

  • Keywords has-patch removed
  • Milestone changed from 4.0 to Future Release

The issues with Flickr and Slideshare were fixed in r28834. Moving to future milestone for ongoing SSL issues with the providers.

jkudish: Were you able to get anywhere with Instagram or any of the other providers?

comment:23 in reply to: ↑ 22 @jkudish14 months ago

Replying to johnbillion:

jkudish: Were you able to get anywhere with Instagram or any of the other providers?

Instagram is aware of the situation and working on it.

I believe we've contacted some of the others but will make sure we get them all this week.

comment:24 @jkudish14 months ago

My team and I have now contacted the remaining providers. We'll keep this ticket updated if/when folks reply.

comment:25 @stephdau14 months ago

Talked to devs at scribd.com, and pointed out the details of the redirect issue with their oembed endpoint, and an issue when passing https urls to the same endpoint (invalid url). They're on it. Will post updates here.

comment:26 @stephdau14 months ago

Talked to the devs at TED, and they're hoping to have SSL support within the next month, which might make it possible to whitelist them for 4.0. I'll post updates on that here too.

comment:27 @stephdau14 months ago

I've reported the SSL and HTTPS related issues to Meetup.com on GitHub: https://github.com/meetup/api/issues/38

comment:28 @jkudish14 months ago

Imgur is aware and working on it as well

comment:29 @jkudish14 months ago

revision3.com sent me a predef reply, not sure I actually reached a real dev, if someone else has a contact there, please pursue it

comment:30 @stephdau14 months ago

funnyordie.com update: in talk with them, and will be in a "tech call" with them next Monday or Tuesday (Aug 11/12) to help them with supporting SSL across the board.

comment:31 @stephdau14 months ago

Meetup.com update, from their dev on https://github.com/meetup/api/issues/38#issuecomment-50941609

so I should have queued up next week a fix that should return secure photo urls if
 * the X-Meta-Photo-Host header is set to secure
 * the request to the oembed endpoint was made over https
 * the url starts with https
evaluated in that order

Rules 2 and 3 should be the one solving our issues.
Not seeing it deployed yet, we'll see later in the week.

comment:32 @stephdau14 months ago

FunnyOrDie.com update: just had a phone meeting with them today: they are proceeding forward with SSL, and are in discussion with Akamai to do so. They currently do not have a precise timeline for SSL support, but it's in the works.

I have passed along this ticket number, and they'll update it when they are ready (or let me know to).

Last edited 14 months ago by stephdau (previous) (diff)

comment:33 @stephdau14 months ago

Meetup.com update: their fix is now live, meaning that:

  • assets (images) in oembed response will now be returned under SSL if we query the oembed endpoint under SSL
  • same is the case if we query with a URL starting with HTTPS, as per cited example in comment:1 (although not techincally a valid one, since Meetup.com does not support public content under SSL, nor has plans to).

I do note that any query returns image URLs under SSL though (which works).
Made a note of that to them on GitHub.

Last edited 14 months ago by stephdau (previous) (diff)

comment:34 @johnbillion14 months ago

Thanks for the updates Stephane. Great stuff.

Looks like DailyMotion has recently added support for HTTPS embeds. I'm going to update this ticket description with a table layout so we can more easily see where we're at.

comment:35 @johnbillion14 months ago

  • Description modified (diff)
  • Owner set to johnbillion
  • Status changed from new to accepted

I've added a tabular view to the ticket description to aid our sanity.

My task list:

  • Audit the providers that we've added in 4.0
  • Find a meetu.ps URL to audit
  • Switch polldaddy.com oEmbed endpoint to HTTPS as it now redirects there

comment:36 @johnbillion14 months ago

  • Description modified (diff)

comment:37 @johnbillion14 months ago

In 29476:

Switch the Polldaddy oEmbed endpoint to HTTPS as it now redirects there. See #28507.

comment:38 @johnbillion14 months ago

  • Description modified (diff)

Animoto was the only provider missing from the list. Animoto embeds are HTTPS by default, but there's some mixed content in there when playing a video. Found and tested a meetu.ps URL (works fine). Updated the list.

comment:39 @stephdau14 months ago

Awesome. We'll get them all, eventually. :)

comment:40 @ircbot13 months ago

This ticket was mentioned in IRC in #wordpress-dev by stephdau. View the logs.

comment:41 @pento12 months ago

instagram.com now loads over SSL, but it does give a content warning (the profile image is loaded over HTTP).

instagr.am still tries to provide the certificate for instagram.com.

comment:42 @GunGeekATX7 months ago

I believe Instagram is forcing everything to HTTPS now. I tested it out by adding a provider:

add_filter( 'oembed_providers', 'pn_test_instagram_oembed' );
function pn_test_instagram_oembed( $providers ) {
	$providers ['#https://instagr(\.am|am\.com)/p/.*#i'] = array( 'https://api.instagram.com/oembed', true );
	return $providers;

Appears to be working: https://petenelson.com/instagram-https-embed-test/


comment:43 @johnbillion7 months ago

Confirmed. Everything's redirecting to HTTPS.

instagr.am still isn't available over HTTPS, but embeds for them continue to work as expected.

comment:44 @johnbillion7 months ago

In 31710:

Allow https URLs for Instagram embeds, and switch to https for its oEmbed API endpoint.

See #28507.

comment:45 @johnbillion7 months ago

  • Description modified (diff)

comment:46 @johnbillion7 months ago

  • Description modified (diff)

comment:47 @johnbillion7 months ago

  • Description modified (diff)

comment:48 @johnbillion7 months ago

  • Description modified (diff)

comment:49 @johnbillion7 months ago

  • Description modified (diff)

comment:50 @johnbillion7 months ago

In 31711:

Some updates to the oEmbed provider table.

See #28507

comment:51 @johnbillion7 months ago

  • Description modified (diff)

comment:52 @johnbillion6 weeks ago

  • Description modified (diff)

comment:53 @johnbillion11 days ago

In 34587:

Switch to the https oEmbed endpoint for Dailymotion so secure embeds are supported.

See #28507

comment:54 @johnbillion11 days ago

  • Description modified (diff)

comment:55 @johnbillion11 days ago

  • Description modified (diff)

comment:56 @johnbillion11 days ago

  • Description modified (diff)

comment:57 @johnbillion11 days ago

  • Description modified (diff)

comment:58 @johnbillion11 days ago

  • Description modified (diff)

comment:59 @johnbillion11 days ago

In 34588:

Switch to the https oEmbed endpoint for Animoto because it redirects there anyway.

See #28507

comment:60 @johnbillion11 days ago

  • Description modified (diff)

comment:61 @DrewAPicture11 days ago

In 34589:

Docs: Adjust the table of providers in the hook doc for oembed_providers to use "No" instead of "!" to signify lack of SSL support.

When displayed in the Code Reference, the "!" doesn't convey enough information.

See #32246. See #28507.

comment:62 @johnbillion11 days ago

  • Keywords ongoing added
Note: See TracTickets for help on using tickets.