Secure oEmbeds

[johnbillion]

We need to audit our oEmbed providers and determine:

• Which ones don't support embedding an https URL
• Which ones don't support embedding content over SSL

If we have providers in core which do not support embedding content over SSL then we (or the WP.com team) should make contact and see if they're open to implementing it. This is pretty much a prerequisite for #28249 as it stands.

---

Problem providers:

None!

Recently fixed providers:

• flic.kr - HTTPS everywhere. Regex corrected in r28834.
• slideshare.net - HTTPS embeds since r28834.
• wordpress.tv - HTTPS embeds for HTTPS URLs.
• meetup.com and meetu.ps- HTTPS embeds for HTTPS URLs.
• instagram.com - HTTPS everywhere since r31710.
• instagr.am - HTTPS URLs are now supported.
• dailymotion.com - Uses the HTTPS oEmbed endpoint since r34587.
• dai.ly - Cert is now valid for the dai.ly domain.
• smugmug.com - Embeds now use HTTPS by default.
• funnyordie.com - Embeds are now protocol-relative and cert is now valid.
• imgur.com - Embeds are now protocol-relative.
• collegehumor.com - HTTPS embeds for HTTPS URLs.
• animoto.com and video214.com - Embeds now use HTTPS by default. See also r34588.
• kck.st - Cert is now valid for the kck.st domain.
• poll.fm - Cert is now valid for the poll.fm domain, redirects to crowdsignal.com
• photobucket.com - Removed in #45399
• hulu.com - HTTPS everywhere since r45385.

Ok providers:

• youtube.com and youtu.be - HTTPS everywhere.
• vimeo.com - Embeds are protocol-relative.
• flickr.com - HTTPS everywhere (same for flic.kr).
• polldaddy.com - Embeds are served over HTTPS if the parent container uses HTTPS. Effectively protocol-relative via JavaScript.
• twitter.com - HTTPS everywhere.
• soundcloud.com - HTTPS everywhere. (Minor note: their oEmbed response includes an http URL for the thumbnail on their CDN, but it resolves over https if you change it.)
• rdio.com and rd.io - HTTPS embeds by default.
• spotify.com - HTTPS everywhere.
• issuu.com - Embeds are served over HTTPS if the parent container uses HTTPS. Effectively protocol-relative via JavaScript.
• mixcloud.com - Embeds are protocol-relative.
• tumblr.com - Embeds are partly HTTPS and partly protocol-relative.
• vine.co - HTTPS everywhere.
• scribd.com - HTTPS embeds by default.
• ted.com - HTTPS embeds for HTTPS URLs.
• videopress.com - HTTPS embeds for HTTPS URLs.
• reverbnation.com - HTTPS embeds by default.
• speakerdeck.com - Embeds are protocol-relative.
• facebook.com - HTTPS everywhere.

[johnbillion]

Immediate task list:

• Add support for https URLs for flic.kr.
• Switch oEmbed endpoint to https for flickr.com, flic.kr, and slideshare.net.

[johnbillion]

28507.diff​ is a patch for the above immediate task list.

[jkudish]

As part of some partnership work that I'm doing for WordPress.com, I have been in contact with engineers at Instagram. I contacted them about this issue today. I'll reply back on this ticket when I hear back.

My team at Automattic (we work on partnerships) will see if we can contact most of the other providers as well.

[johnbillion]

That would be fantastic jkudish, thank you. Do also keep us posted on what wp.com are planning with relation to their SSL switchover.

[johnbillion]

In 28834:

Switch to SSL for the Flickr and Slideshare oEmbed endpoints. Add support for SSL embeds on flic.kr. See #28507.

[azaozz]

Related: #28195 and the first-run patch for handling secure oEmbed in the admin (wpView) https://core.trac.wordpress.org/attachment/ticket/28195/28195.16.patch.

We may need to change some of the regexp/strings for the providers in order to easily detect which of them support https.

[azaozz]

In 28919:

Secure embeds in the editor (first run):

• When the user pastes an embeddable http URL, try to get the https embed.
• If an embed provider doesn't support ssl embeds, show a placeholder/error message.
• Revise the way we return error messages.

See #28195, #28507.

[DrewAPicture]

In 28949:

Introduce an annotated list of oEmbed providers, their flavors, whether they support SSL, and when they were added to the oembed_providers filter docs.

See #28507.
Fixes #28372.

[DrewAPicture]

In 28950:

Remove duplicate of the 'oembed_providers' filter accidentally introduced in [28949].

Move annoted table of oEmbed providers into the existing filter docs.

See #28507.
Fixes #28372.

[stephdau]

This is to let you know that WordPress.tv now has a valid SSL certificate, and could be whitelisted as an SSL-Ok providers.

[stephdau]

WordPress.tv: Actually, not fully true [yet], sorry... It does have a valid SSL cert, but work needs to be done for embeds to be protocol-relative ready, etc.

[johnbillion]

Thanks Stephane. It also looks like the oEmbed endpoint fails if the scheme of the video URL doesn't match the scheme of the endpoint URL. For example this fails:

​http://wordpress.tv/oembed?url=https://wordpress.tv/2014/06/07/andrew-nacin-wordcamp-connecticut-keynote/

[stephdau]

Going to look into either making it work, or find the right people to get it to, this morning.
I'll get back to you. :)

[stephdau]

WordPress.tv update: I have the protocol-relative and mismatching schemes figured out, getting a patch reviewed before applying it (wp.tv runs on wp.com).

On the other hand, I'm hitting a temporary wall with wp.tv itself playing its videos (though VideoPress) under SSL. Digging further into that.

[stephdau]

Happy to report that ​https://wordpress.tv/ is now SSL-ready, for playback on the site, or embedded.

oEmbed responses now use protocol-relative URLs for the video location, which work under both HTTP and HTTPS.

JSON:

{"type":"video","version":"1.0","title":null,"width":400,"height":225,"html":"<embed src=\"\/\/v.wordpress.com\/3JiLCPst\" type=\"application\/x-shockwave-flash\" width=\"400\" height=\"225\" allowscriptaccess=\"always\" allowfullscreen=\"true\" wmode=\"transparent\"><\/embed>"}

XML

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<oembed>
	<type>video</type>
	<version>1.0</version>
	<title></title>
	<width>400</width>
	<height>225</height>
	<html>&lt;embed src=&quot;//v.wordpress.com/3JiLCPst&quot; type=&quot;application/x-shockwave-flash&quot; width=&quot;400&quot; height=&quot;225&quot; allowscriptaccess=&quot;always&quot; allowfullscreen=&quot;true&quot; wmode=&quot;transparent&quot;&gt;&lt;/embed&gt;</html>
</oembed>

The protocol mismatch issue was also handled.

• ​http://wordpress.tv/oembed/?url=http%3A%2F%2Fwordpress.tv%2F2014%2F06%2F13%2Fmel-choyce-creating-a-great-portfolio-website%2F&format=json
• ​http://wordpress.tv/oembed/?url=https%3A%2F%2Fwordpress.tv%2F2014%2F06%2F13%2Fmel-choyce-creating-a-great-portfolio-website%2F&format=json
• ​https://wordpress.tv/oembed/?url=http%3A%2F%2Fwordpress.tv%2F2014%2F06%2F13%2Fmel-choyce-creating-a-great-portfolio-website%2F&format=json
• ​https://wordpress.tv/oembed/?url=https%3A%2F%2Fwordpress.tv%2F2014%2F06%2F13%2Fmel-choyce-creating-a-great-portfolio-website%2F&format=json

The most network-inquisitives in the crowd might notice some 404s in the load process, upon playback, but those will soon be handled as well, and should not affect playback on the site, or embedded. :)

We're also hoping to look into iframe support in the future, but no timeframe on that.

[johnbillion]

In 29110:

Add support for secure wordpress.tv embeds (thanks stephdau). See #28507.

[johnbillion]

The issues with Flickr and Slideshare were fixed in r28834. Moving to future milestone for ongoing SSL issues with the providers.

jkudish: Were you able to get anywhere with Instagram or any of the other providers?

[jkudish]

Replying to johnbillion:

jkudish: Were you able to get anywhere with Instagram or any of the other providers?

Instagram is aware of the situation and working on it.

I believe we've contacted some of the others but will make sure we get them all this week.

[jkudish]

My team and I have now contacted the remaining providers. We'll keep this ticket updated if/when folks reply.

[stephdau]

Talked to devs at scribd.com, and pointed out the details of the redirect issue with their oembed endpoint, and an issue when passing https urls to the same endpoint (invalid url). They're on it. Will post updates here.

[stephdau]

Talked to the devs at TED, and they're hoping to have SSL support within the next month, which might make it possible to whitelist them for 4.0. I'll post updates on that here too.

[stephdau]

I've reported the SSL and HTTPS related issues to Meetup.com on GitHub: ​https://github.com/meetup/api/issues/38

[jkudish]

Imgur is aware and working on it as well

[jkudish]

revision3.com sent me a predef reply, not sure I actually reached a real dev, if someone else has a contact there, please pursue it

[stephdau]

funnyordie.com update: in talk with them, and will be in a "tech call" with them next Monday or Tuesday (Aug 11/12) to help them with supporting SSL across the board.

[stephdau]

Meetup.com update, from their dev on ​https://github.com/meetup/api/issues/38#issuecomment-50941609

so I should have queued up next week a fix that should return secure photo urls if
 * the X-Meta-Photo-Host header is set to secure
 * the request to the oembed endpoint was made over https
 * the url starts with https
evaluated in that order

Rules 2 and 3 should be the one solving our issues.
Not seeing it deployed yet, we'll see later in the week.

[stephdau]

FunnyOrDie.com update: just had a phone meeting with them today: they are proceeding forward with SSL, and are in discussion with Akamai to do so. They currently do not have a precise timeline for SSL support, but it's in the works.

I have passed along this ticket number, and they'll update it when they are ready (or let me know to).

[stephdau]

Meetup.com update: their fix is now live, meaning that:

• assets (images) in oembed response will now be returned under SSL if we query the oembed endpoint under SSL
• same is the case if we query with a URL starting with HTTPS, as per cited example in comment:1 (although not techincally a valid one, since Meetup.com does not support public content under SSL, nor has plans to).

I do note that any query returns image URLs under SSL though (which works).
Made a note of that to them on GitHub.

[johnbillion]

Thanks for the updates Stephane. Great stuff.

Looks like DailyMotion has recently added support for HTTPS embeds. I'm going to update this ticket description with a table layout so we can more easily see where we're at.

[johnbillion]

I've added a tabular view to the ticket description to aid our sanity.

My task list:

• Audit the providers that we've added in 4.0
• Find a meetu.ps URL to audit
• Switch polldaddy.com oEmbed endpoint to HTTPS as it now redirects there

[johnbillion]

In 29476:

Switch the Polldaddy oEmbed endpoint to HTTPS as it now redirects there. See #28507.

[johnbillion]

Animoto was the only provider missing from the list. Animoto embeds are HTTPS by default, but there's some mixed content in there when playing a video. Found and tested a meetu.ps URL (works fine). Updated the list.

[stephdau]

Awesome. We'll get them all, eventually. :)

[pento]

instagram.com now loads over SSL, but it does give a content warning (the profile image is loaded over HTTP).

instagr.am still tries to provide the certificate for instagram.com.

[GunGeekATX]

I believe Instagram is forcing everything to HTTPS now. I tested it out by adding a provider:

add_filter( 'oembed_providers', 'pn_test_instagram_oembed' );
function pn_test_instagram_oembed( $providers ) {
	$providers ['#https://instagr(\.am|am\.com)/p/.*#i'] = array( 'https://api.instagram.com/oembed', true );
	return $providers;
}

Appears to be working: ​https://petenelson.com/instagram-https-embed-test/

​https://api.instagram.com/oembed?url=https://instagram.com/p/0DU6jJIvyw/

[johnbillion]

Confirmed. Everything's redirecting to HTTPS.

instagr.am still isn't available over HTTPS, but embeds for them continue to work as expected.

[johnbillion]

In 31710:

Allow https URLs for Instagram embeds, and switch to https for its oEmbed API endpoint.

See #28507.

[johnbillion]

In 31711:

Some updates to the oEmbed provider table.

See #28507

[johnbillion]

In 34587:

Switch to the https oEmbed endpoint for Dailymotion so secure embeds are supported.

See #28507

[johnbillion]

In 34588:

Switch to the https oEmbed endpoint for Animoto because it redirects there anyway.

See #28507

[DrewAPicture]

In 34589:

Docs: Adjust the table of providers in the hook doc for oembed_providers to use "No" instead of "!" to signify lack of SSL support.

When displayed in the Code Reference, the "!" doesn't convey enough information.

See #32246. See #28507.

[swissspidy]

There's a patch on #36274 addressing YouTube and Vimeo, whose API endpoints are currently configured with http://, but redirect to https:// versions anyway. We should add this for 4.6.

[swissspidy]

#36274 was marked as a duplicate.

[swissspidy]

28507.2.diff​ is the patch from #36274 (props zsusag) to change the YouTube endpoint URLs to https as they redirect there.

[paulschreiber]

I just found #36274, which was marked as a duplicate of this ticket. It looks like the patch didn't make it for 4.6. Can we get that in to 4.7?

[swissspidy]

Yeah, I think we can add 28507.3.diff​ in 4.7

[johnbillion]

In 38365:

Embeds: Always use the HTTPS endpoint for YouTube embeds. The scheme parameter is no longer required as all YouTube assets now use HTTPS.

See #36274, #28507
Props zsusag, tollmanz

[johnbillion]

In 38366:

Embeds: Many of our oEmbed providers now default to HTTPS embeds, redirect to the HTTPS oEmbed endpoint, or have complete support for HTTPS even if they don't default to HTTPS.

This change defaults to using HTTPS endpoints for oEmbeds for those providers that have full HTTPS support and don't redirect back to HTTP when clicking through from the embed. It covers:

• Vimeo
• SmugMug
• Scribd
• WordPress.tv
• SoundCloud
• Meetup
• issuu
• Mixcloud
• TED

See #28507

[johnbillion]

In 41680:

Embeds: Switch the collegehumor.com oEmbed endpoint to HTTPS, as it redirects there by default now.

See #28507

[johnbillion]

In 45385:

Embeds: Use HTTPS for the Hulu oEmbed endpoint, as it redirects to HTTPS now.

Fixes #28507

[johnbillion]

In 45386:

Embeds: Remove the "Supports HTTPS" column from the whitelisted oEmbed provider list now that all supported providers use HTTPS.

See #28507