Remove Facebook and Instagram as an oEmbed Source

[whyisjake]

Facebook and Instagram are dropping support of unauthenticated oEmbeds on ​October 24th.

They have a new endpoint, but it depends on generating an app ID with a developer account. We should remove the endpoint and allow them to support via a plugin in the future.

[whyisjake]

@mkaz pointed out this would need to be updated to include the block editor too.

[joyously]

How will that work with legacy embeds? Is the data that is stored in post meta still usable?

[mkaz]

Related Block Editor issue: ​https://github.com/WordPress/gutenberg/issues/24389

[mkaz]

@joyously It will require some testing to confirm, but previously for broken embeds the link just shows.

So if there was an Instagram link embedding a photo, instead of a photo showing it would be a link to the Instagram post.

[johnbillion]

Yeah there's nothing that core can do about existing embeds. If the oEmbed endpoint for a service ceases to work and the embed HTML isn't in the post meta cache, then it will stop working.

[ayeshrajans]

I started a plugin ​https://github.com/Ayesh/WordPress-oEmbed-Facebook-Instagram (WIP) to bring these sites back with the new API. The plugin will need to expose a new option to enter Instagram/Facebook App access keys, after which the plugin can complement the embeds with authenticated oEmbed requests.

[ayeshrajans]

FWIW, I completed the work on the plugin mentioned above. It currently has a v1.0 git tag that will replace/add the new authenticated API call support. I submitted the plugin for review, fingers crossed it gets approved.

[ayeshrajans]

​https://wordpress.org/plugins/oembed-plus/

This plugin (from the GitHub repo in my previous comments) brings back the Facebook/Instagram oEmbed support, but it now requires to enter API credentials.

[mkaz]

@whyisjake @johnbillion If the plan is to remove for 5.5.1, we should probably merge and backport this PR ​https://github.com/WordPress/gutenberg/pull/24472

[davisshaver]

Does WordPress.org have any institutional relationship with Facebook? Given the scope of this change I wonder if a channel could be created to discuss issues of mutual interest.

Reconsideration or postponement would both be nice outcomes given the potential issues here, but even if those aren't options, Facebook could provide guidance or support which may assist the community in implementing a transition strategy.

From a journalistic perspective these oEmbed links are often material to reporting/storytelling. In the event that oEmbed links revert to raw links, readers may not understand that they need to click the link to view the supporting material, and story flow/comprehension may be impacted as a result. It would be unfortunate for this regression to occur at all, but especially at the end of October when American media will be in the final countdown to a presidential election.

Even better than a postponement would be some means of allowing open access for editorial purposes. My understanding is that the "o" in "oEmbed" stands for open. While the oEmbed spec doesn't appear to directly comment on this, at the least requiring authorization for oEmbed endpoints seems to be unusual. Embedly is one major provider that does this, but they also serve a different use case in the oEmbed landscape than most services. Facebook may ultimately still need to restrict the endpoint but there could be relevant considerations of oEmbed's design intent that may cause them to reconsider.

Lastly, if authorization does become necessary, I wonder if major hosting providers like WordPress.com, WordPress VIP, and other managed platforms could provide this feature for clients using specially designated app access tokens. Given the ubiquity of WordPress and Facebook we have an opportunity to benefit a large percentage of overall sites on the internet if we convene a small number of companies to address this at a systems level.

[mkaz]

@davisshaver Thank you for your considerate comment. I'll try to address each below, but the short of it is that the change is going to happen on Oct 24.

A couple of us did have a discussion with Facebook as they reached out. It is a new company policy that all API requested are authenticated by Oct 24. This is also in their announcement post ​https://developers.facebook.com/blog/post/2020/08/04/Introducing-graph-v8-marketing-api-v8. The date and change in policy do not seem negotiable.

I agree that oEmbed is a great feature and should be open, not requiring authentication, regardless, it is not our API to decide. I think the only real issue would be Facebook using the "oEmbed" name to describe the endpoints when they are not indeed open, but that is more a quibble and doesn't change much of anything.

Lastly, I do work for Automattic and thus have some insight to WordPress.com and VIP. I believe they will be creating and offering a solution to their customers. However, I'm not working directly on that, and this isn't really the place to promote.

We did suggest to Facebook to reach out and talk to all major hosts prior to help reduce the impact.

[davisshaver]

Thanks @mkaz – I appreciate the additional context. The only shot I see is possibly making the journalistic argument. I wonder if we could at least try to brief the Facebook Journalism Project on the issue here, even if no change in plans is possible I think FJP would be interested to understand the probable impact on news organizations in a hectic time leading up to the election. Even postponing 2-4 weeks would be of significant value.

[khag7]

Is there precedent for WordPress displaying some type of notice on the admin area for an upcoming breaking change? "Hey you, your Facebook and Instagram oEmbeds are going to stop working Oct 24. Check out this plugin that can keep them working"

If we were to do something like that in 5.5.1, and then disable Facebook and Instagram oEmbeds after Oct 24, it might avoid instances of missing oEmbed content for users. On the other hand, it might set the precedent that we want to display that type of notice and it becomes expected, or is perceived poorly.

[joyously]

Facebook has made other breaking changes, but WP didn't have any integration, so it didn't matter. It was up to plugins to adapt. With embed being in core, WP has to adapt. I think it's best to simply remove support for that embed type. Plugins can take up the slack. But it does bring up the concept of making all embeds generic, meaning that WP only supplies the basic infrastructure for the embed, and plugins do the rest. That way core doesn't have to keep up with all the different providers.
This might be a good time to switch to that.

[ayeshrajans]

A dev note can help a lot. In the WP core backend, all provided are treated the same, but Gutenberg has custom snippets for IG/FB, which might be the source for an end-user notice.

Fingers crossed we can ship an update with the heads-up before FB takes the API down.

[whyisjake]

A dev note could def. help. This will throw notices on all non-trunk/5.5.x releases too.

Regarding the timing, if 5.5.1 releases ahead of October 24th, I don't think we should merge this. Since the API will self-deprecate, I think we should leave it in until after. (Like we did with the Hulu changes.) See #50676, [48512].

[audrasjb]

Moving to milestone 5.5.2 as we are close to release 5.5.1 RC.

[paaljoachim]

@mkaz added a comment to make core.

"...a heads up on a change for Facebook and Instagram embeds in Gutenberg 9.0 going out Sept 16.

Gutenberg PR-24472 removes the Facebook and Instagram embeds—this was merged probably a release earlier than it should, but so be it, has to be done at some point.
​https://github.com/WordPress/gutenberg/pull/24472

With the embeds removed from Gutenberg the impact isn’t too dramatic as it sounds, since any pasted link in Gutenberg calls out to the service and looks for oEmbed support. Facebook and Instagram will still support oEmbed until Oct 24, 2020.

However, the Facebook and Instagram blocks will no longer show in the Block Editor inspector if searching for the blocks, but all the embeds and even new embeds will still work until Facebook makes the change to the service.

Link to the comment:
​https://make.wordpress.org/core/2020/09/15/dev-chat-agenda-september-16th-2020/#comment-39780

[Clorith]

Since this may impact users unknowingly, it is possible to push a dashboard notice to users who have facebook/instagram embeds in their content, showing for site admins, as a one-off that can be dismissed.

We've previously done post-update-processing to clean up comments, so the idea of looking over content for an embed isn't completely outlandish, and would help with those who don't follow WordPress' usual channels to learn of this?

[dimadin]

Why should we make exception here? It's not the first time oEmbed support was discontinued for a provider, and I don't remember anything specific was done then.

[johnbillion]

A dashboard notice definitely isn't desirable, but a message which replaces existing embeds when editing a post might be.

[bridgetwillard]

As a writer and social media manager, who loves and depends upon oEmbeds (rather than screenshots), I completely understand the logic behind removal.

I completely agree this should become plugin-dependent.

Facebook is famous for changing the rules on their playground. And, honestly, APIs be like that sometimes. Right?

So, as far as communicating this to the general public, we should fill in the holes of people's education.

Something like:

"WordPress 5.5.2 has an important update that affects oEmbeds of Facebook properties. This includes Instagram and Facebook links.

How Does This Affect You?

Previously, any WordPress blogger could simply insert a URL of a Facebook or Instagram post and the image and post would render. Like magic, but definitely technical.

Unfortunately, Facebook has made access to their API a bit more complicated which, cheers to them, helps protect the privacy of its user base.

If you're worried about this update, check your posts for links that don't unfurl/render. They will appear as regular links. It isn't broken, it just doesn't render. Instead, use the link like you would any link -- in a sentence -- with accessibility in mind.

You can always include a screenshot with alt tags and a caption as well."

[jb510]

Replying to johnbillion:

A dashboard notice definitely isn't desirable, but a message which replaces existing embeds when editing a post might be.

I disagree here. I think a dashboard notice _is_ desirable.

Otherwise we're not preemptively warning people in a way they can prepare and transition to another solution, we're letting them know the same instant it's going to break (when editing a specific post). I don't think we can safely assume cached data is going to persist forever either, plenty of routines out there purge transient data before its stated expiration date.

I see this as potentially being similar to the problems seen in dropping JQM. It'll cause _avoidable and silent breakage_ client side without even any error logging for a site developer to pick up on. In hindsight what ideally would have happened with JQM would have been incorporating the detection code from Enable jQuery Migrate Helper into core temporarily, or simply (haha) installing that plugin automatically on behalf of users.

Core can "see" the calls to these specific cached embeds, core ought therefore be able to easily warn folks with a notice that these calls are being made before they fail.

That notice can then direct folks to consider enabling a plugin that would maintain pulling these embeds.

[sippis]

Whilst I agree that showing a dashboard notice can be helpful for some user groups like bloggers and take away some of the burden from support forums, it might cause panicky messages sent elsewhere.

Imagine a corporate website which has many content updaters, tens of pages and hundreds of posts - in which only a handful of Facebook or Instagram oEmbeds. Then some busy marketing manager logs into the dashboard to see the message and sends right away message to their hosting or agency partner to ensure that the whole site and/or social media wall does not break down. The support burden is just moved to other parties.

Which brings to my actual points:

• The message in the notice should be as clear as possible, preferably linking to support article where change is told in more detail.
• There needs to be a hook to remove the content check, in case the hosting or agency partner decides to inform their customer in another way. Yup - you can remove the notice with already existing hooks, but if doing so the check is still made in the background and it doesn't make much sense if results aren't shown anywhere.

And then to one idea: Why clutter up the dashboard with notice, since we have the Site Health Check? Isn't this that kind of a thing that would fit into the purpose of that quite nicely?

[Clorith]

Being the one that first brought up messaging, I'm going to re-consider that right now.

Existing embeds are "cached" (I'm using quotes, because it's not really a cache in the traditional sense), some cleanup plugins may clear them out for example, so we don't know how reliable they are for every site, but by and large, from a core perspective they should remain available for now.

If something else cleans them up, it should be their responsibility to notify users of the potential drawbacks of doing so. I do think that it is reasonable to get the removal of the embeds from core done before they stop working though. It will reduce user frustration if they try to use it and it fails I believe.

From a Site Health perspective, it doesn't really fit in, it could alert users that they have cached oEmbeds form Facebook/Instagram, but what if they have a plugin that handles the embeds properly for them?

[TimothyBlynJacobs]

I think a full on admin notice might be too much. But what about something contextual. For instance, when you paste a Facebook link that we previously would've been able to oEmbed, we show an inline notice and open the block directory with a search for "Facebook". That way we can guide users to a more seamless upgrade path.

[desrosj]

I've been doing some testing on this today. @mkaz ​shared the ​Facebook documentation link yesterday that has a way to simulate how the API will act after the deprecation date.

You can simulate the experience dropping this in an MU plugin:

add_filter( 'oembed_fetch_url', function( $provider, $url ) {
	if ( ! in_array( $provider, array( 'facebook', 'instagram' ), true ) ) {
		return $url;
	}

	return add_query_arg( 'breaking_change', 'oembed', $url );
}, 10, 2 );

This snippet will show how new attempts to embed Facebook/Instagram posts will be act once the deprecation date is reached (a 400 code will be returned). However, it doesn't tell us what changes will happen to existing, cached embeds.

When a link is embedded in a post, the HTML response from the provider is cached in post meta (unless the embed is placed within a widget's content, then it is cached in the hidden oembed_cache post type in a similar way). In the example above, this is the cached markup for the Instagram post:

<blockquote class="instagram-media" data-instgrm-captioned data-instgrm-permalink="https://www.instagram.com/p/CEU6QukJkQy/?utm_source=ig_embed&amp;utm_campaign=loading" data-instgrm-version="12" style=" background:#FFF; border:0; border-radius:3px; box-shadow:0 0 1px 0 rgba(0,0,0,0.5),0 1px 10px 0 rgba(0,0,0,0.15); margin: 1px; max-width:580px; min-width:326px; padding:0; width:99.375%; width:-webkit-calc(100% - 2px); width:calc(100% - 2px);"><div style="padding:16px;"> <a href="https://www.instagram.com/p/CEU6QukJkQy/?utm_source=ig_embed&amp;utm_campaign=loading" style=" background:#FFFFFF; line-height:0; padding:0 0; text-align:center; text-decoration:none; width:100%;" target="_blank"> <div style=" display: flex; flex-direction: row; align-items: center;"> <div style="background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 40px; margin-right: 14px; width: 40px;"></div> <div style="display: flex; flex-direction: column; flex-grow: 1; justify-content: center;"> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 100px;"></div> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 60px;"></div></div></div><div style="padding: 19% 0;"></div> <div style="display:block; height:50px; margin:0 auto 12px; width:50px;"><svg width="50px" height="50px" viewBox="0 0 60 60" version="1.1" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink"><g stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"><g transform="translate(-511.000000, -20.000000)" fill="#000000"><g><path d="M556.869,30.41 C554.814,30.41 553.148,32.076 553.148,34.131 C553.148,36.186 554.814,37.852 556.869,37.852 C558.924,37.852 560.59,36.186 560.59,34.131 C560.59,32.076 558.924,30.41 556.869,30.41 M541,60.657 C535.114,60.657 530.342,55.887 530.342,50 C530.342,44.114 535.114,39.342 541,39.342 C546.887,39.342 551.658,44.114 551.658,50 C551.658,55.887 546.887,60.657 541,60.657 M541,33.886 C532.1,33.886 524.886,41.1 524.886,50 C524.886,58.899 532.1,66.113 541,66.113 C549.9,66.113 557.115,58.899 557.115,50 C557.115,41.1 549.9,33.886 541,33.886 M565.378,62.101 C565.244,65.022 564.756,66.606 564.346,67.663 C563.803,69.06 563.154,70.057 562.106,71.106 C561.058,72.155 560.06,72.803 558.662,73.347 C557.607,73.757 556.021,74.244 553.102,74.378 C549.944,74.521 548.997,74.552 541,74.552 C533.003,74.552 532.056,74.521 528.898,74.378 C525.979,74.244 524.393,73.757 523.338,73.347 C521.94,72.803 520.942,72.155 519.894,71.106 C518.846,70.057 518.197,69.06 517.654,67.663 C517.244,66.606 516.755,65.022 516.623,62.101 C516.479,58.943 516.448,57.996 516.448,50 C516.448,42.003 516.479,41.056 516.623,37.899 C516.755,34.978 517.244,33.391 517.654,32.338 C518.197,30.938 518.846,29.942 519.894,28.894 C520.942,27.846 521.94,27.196 523.338,26.654 C524.393,26.244 525.979,25.756 528.898,25.623 C532.057,25.479 533.004,25.448 541,25.448 C548.997,25.448 549.943,25.479 553.102,25.623 C556.021,25.756 557.607,26.244 558.662,26.654 C560.06,27.196 561.058,27.846 562.106,28.894 C563.154,29.942 563.803,30.938 564.346,32.338 C564.756,33.391 565.244,34.978 565.378,37.899 C565.522,41.056 565.552,42.003 565.552,50 C565.552,57.996 565.522,58.943 565.378,62.101 M570.82,37.631 C570.674,34.438 570.167,32.258 569.425,30.349 C568.659,28.377 567.633,26.702 565.965,25.035 C564.297,23.368 562.623,22.342 560.652,21.575 C558.743,20.834 556.562,20.326 553.369,20.18 C550.169,20.033 549.148,20 541,20 C532.853,20 531.831,20.033 528.631,20.18 C525.438,20.326 523.257,20.834 521.349,21.575 C519.376,22.342 517.703,23.368 516.035,25.035 C514.368,26.702 513.342,28.377 512.574,30.349 C511.834,32.258 511.326,34.438 511.181,37.631 C511.035,40.831 511,41.851 511,50 C511,58.147 511.035,59.17 511.181,62.369 C511.326,65.562 511.834,67.743 512.574,69.651 C513.342,71.625 514.368,73.296 516.035,74.965 C517.703,76.634 519.376,77.658 521.349,78.425 C523.257,79.167 525.438,79.673 528.631,79.82 C531.831,79.965 532.853,80.001 541,80.001 C549.148,80.001 550.169,79.965 553.369,79.82 C556.562,79.673 558.743,79.167 560.652,78.425 C562.623,77.658 564.297,76.634 565.965,74.965 C567.633,73.296 568.659,71.625 569.425,69.651 C570.167,67.743 570.674,65.562 570.82,62.369 C570.966,59.17 571,58.147 571,50 C571,41.851 570.966,40.831 570.82,37.631"></path></g></g></g></svg></div><div style="padding-top: 8px;"> <div style=" color:#3897f0; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:550; line-height:18px;"> View this post on Instagram</div></div><div style="padding: 12.5% 0;"></div> <div style="display: flex; flex-direction: row; margin-bottom: 14px; align-items: center;"><div> <div style="background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(0px) translateY(7px);"></div> <div style="background-color: #F4F4F4; height: 12.5px; transform: rotate(-45deg) translateX(3px) translateY(1px); width: 12.5px; flex-grow: 0; margin-right: 14px; margin-left: 2px;"></div> <div style="background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(9px) translateY(-18px);"></div></div><div style="margin-left: 8px;"> <div style=" background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 20px; width: 20px;"></div> <div style=" width: 0; height: 0; border-top: 2px solid transparent; border-left: 6px solid #f4f4f4; border-bottom: 2px solid transparent; transform: translateX(16px) translateY(-4px) rotate(30deg)"></div></div><div style="margin-left: auto;"> <div style=" width: 0px; border-top: 8px solid #F4F4F4; border-right: 8px solid transparent; transform: translateY(16px);"></div> <div style=" background-color: #F4F4F4; flex-grow: 0; height: 12px; width: 16px; transform: translateY(-4px);"></div> <div style=" width: 0; height: 0; border-top: 8px solid #F4F4F4; border-left: 8px solid transparent; transform: translateY(-4px) translateX(8px);"></div></div></div></a> <p style=" margin:8px 0 0 0; padding:0 4px;"> <a href="https://www.instagram.com/p/CEU6QukJkQy/?utm_source=ig_embed&amp;utm_campaign=loading" style=" color:#000; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px; text-decoration:none; word-wrap:break-word;" target="_blank">📸 Take a look at Hurricane Laura pictured by @Astro_SEAL from 254 miles above Earth. ⁣ ⁣ For more than five decades, we have used the vantage point of space to understand and explore our home planet, improve lives, and safeguard our future. In addition to observations from the International Space Station (@ISS), our @NASAEarth satellites monitor hurricanes to bring new capabilities and tools for studying them.⁣ ⁣ Credit: NASA/Chris Cassidy⁣ ⁣ #Hurricane #ExtremeWeather #NASA #EarthFromSpace #StormClouds #Atlantic</a></p> <p style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; line-height:17px; margin-bottom:0; margin-top:8px; overflow:hidden; padding:8px 0 7px; text-align:center; text-overflow:ellipsis; white-space:nowrap;">A post shared by <a href="https://www.instagram.com/nasa/?utm_source=ig_embed&amp;utm_campaign=loading" style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px;" target="_blank"> NASA</a> (@nasa) on <time style=" font-family:Arial,sans-serif; font-size:14px; line-height:17px;" datetime="2020-08-25T21:04:57+00:00">Aug 25, 2020 at 2:04pm PDT</time></p></div></blockquote><script async src="//www.instagram.com/embed.js"></script>

We know this markup will persist, but we don't know how the behavior will change. The best guess we can make is to remove the JS file to simulate that file also returning a 400 type error. After removing the JS file in the Facebook and Instagram embed, the before and after images I uploaded are the results.

I'd note I don't particularly like how the Facebook embed becomes just a plain blockquote. It's not clear to users that clicking the links in that embed will navigate you to Facebook. The Instagram "fallback" is actually quite nice.

This is also the best guess we can really make at this time without more direction from Facebook. It is possible that they change the contents of the JS file to manipulate cached embeds, perhaps to display a warning that the site is using an old method to embed content or that the request is not properly authenticated.

[Clorith]

Great work @desrosj!

I agree the blockquote on the facebook embed isn't pretty, and lacks any reference to what it might have been (the instagram one isn't as bad, since it links you off to the source in an obvious way).

There's not too much we can do here, but at least it gives an educated guess for marketing to work with when working on relaying information, which I think is fantastic.

[smub]

Just wanted to comment here, since our Smash Balloon Instagram and FB feed plugins are already used by over 1.2 million users, we have added the oEmbed functionality to it.

This was easy for us to do because to create custom feeds, users would have to use the API key anyways. Over the years, our plugin has simplified that process to make it very user friendly.

I think it’d be helpful to recommend these because we DO NOT require users to go through a complicated FB app setup process or anything.

Instagram Feeds – ​https://wordpress.org/plugins/instagram-feed/
Facebook Feeds – ​https://wordpress.org/plugins/custom-facebook-feed/

[audrasjb]

Given Facebook and Instagram Embeds were deprecated upstream, I think it is now safe to commit the exiting patch for 5.5.2.

[whyisjake]

Dev note added here: ​https://make.wordpress.org/core/2020/09/22/facebook-and-instagram-embeds-to-be-deprecated-october-24th/

[whyisjake]

In 49359:

Embeds: Remove Facebook and Instagram as an oEmbed Source

Facebook has depracated all non-authenticated endpoints for Facebook and Instagram.

See: ​https://developers.facebook.com/docs/plugins/oembed-legacy
See also: ​https://make.wordpress.org/core/2020/09/22/facebook-and-instagram-embeds-to-be-deprecated-october-24th/

With this change, endpoints are being removed. If a site is dependent on this feature, they need to pass either an app or client token. There are a few plugins that add this functionality.

Fixes #50861.
Props johnbillion, joyously, mkaz, dimadin, ayeshrajans, davisshaver, paaljoachim, Clorith, bridgetwillard, jb510, sippis, Clorith, TimothyBlynJacobs, desrosj, smub, audrasjb, whyisjake.

[whyisjake]

Reopening for inclusion in 5.5.2.

[SergeyBiryukov]

Looks good to me.

[whyisjake]

In 49361:

Embeds: Remove Facebook and Instagram as an oEmbed Source
Facebook has depracated all non-authenticated endpoints for Facebook and Instagram.

This commit brings the changes from [49359] to the 5.5 branch.

See: ​https://developers.facebook.com/docs/plugins/oembed-legacy
See also: ​https://make.wordpress.org/core/2020/09/22/facebook-and-instagram-embeds-to-be-deprecated-october-24th/

With this change, endpoints are being removed. If a site is dependent on this feature, they need to pass either an app or client token. There are a few plugins that add this functionality.

Fixes #50861.
Props johnbillion, joyously, mkaz, dimadin, ayeshrajans, davisshaver, paaljoachim, Clorith, bridgetwillard, jb510, sippis, Clorith, TimothyBlynJacobs, desrosj, smub, audrasjb, whyisjake.