Twitter oembeds not working because of Twitter API change

[yurivictor]

Twitter restricting api.twitter.com to SSL/TLS traffic, per ​https://dev.twitter.com/discussions/24239.

/ht Connor Jennings.

[ocean90]

Confirmed.

Related: #23419

[nacin]

Things to investigate:

• If we make this change, we abandon sites that don't support OpenSSL. That's OK, since obviously Twitter forced our hand on that. I've reached out to the developer advocate who announced this (​https://twitter.com/nacin/status/423548186167226368).
• Do we (and if not, why not) follow the redirect here? Is the issue OpenSSL, WordPress, or Twitter? (See possibility below...)

So — while ​http://api.twitter.com/1/statuses/oembed.json?id=423546680999301121 redirects in a browser, yurivictor pointed out that curl --head gets a 403. Why? Would be good to investigate and report that back to Twitter. If they can fix that, then all existing WordPress installs (that support OpenSSL) should follow this redirect without an issue.

[gcorne]

Replying to nacin:

So — while ​http://api.twitter.com/1/statuses/oembed.json?id=423546680999301121 redirects in a browser, yurivictor pointed out that curl --head gets a 403. Why? Would be good to investigate and report that back to Twitter. If they can fix that, then all existing WordPress installs (that support OpenSSL) should follow this redirect without an issue.

So thanks to ​strict-transport-security, browsers are not actually redirecting. Instead the browser is using https protocol even if the user enters ​http:// in the location bar.

Here are the headers from an https request that show the strict-transport-security: max-age=631138519 header being sent:

$ curl -v 'https://api.twitter.com/1/statuses/oembed.json?id=423546680999301121' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36' -b 'guest_id=v1%3A138981846426213327'
* Adding handle: conn: 0x7f9763004000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f9763004000) send_pipe: 1, recv_pipe: 0
* About to connect() to api.twitter.com port 443 (#0)
*   Trying 199.16.156.40...
* Connected to api.twitter.com (199.16.156.40) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_RC4_128_SHA
* Server certificate: api.twitter.com
* Server certificate: VeriSign Class 3 Secure Server CA - G3
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /1/statuses/oembed.json?id=423546680999301121 HTTP/1.1
> Host: api.twitter.com
> Cookie: guest_id=v1%3A138981846426213327
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36
> 
< HTTP/1.1 200 OK
< cache-control: must-revalidate, max-age=3153600000
< content-length: 794
< content-type: application/json;charset=utf-8
< date: Wed, 15 Jan 2014 21:04:43 GMT
< expires: Fri, 22 Dec 2113 21:04:43 GMT
< last-modified: Wed, 15 Jan 2014 21:04:43 GMT
* Server tfe is not blacklisted
< server: tfe
< strict-transport-security: max-age=631138519
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-transaction: 9e09e54fb5666417
< x-xss-protection: 1; mode=block

[MnLGmz]

Hello,

I would like to know when Wordpress will release this bug fix?

Thanks

[mac2net]

OK, I was in the process of migrating a site from one domain to the other that tracks a few hash tags and converts tweets to individual posts of embedded tweets.
I made the above change at 2:36 EST to the new site but not the old.

It appears that starting from 3:26 EST, embedding is now working on the site where I made this patch and it's still not working on the old site.

So hopefully twitter has corrected the problem and this fix now works.

PS: I just added the patch to the old site and sent a test tweet with an has tag that I am tracking and it works!

[ocean90]

mac2net, curl --head still returns 403. So nothing is fixed.

(If you mean that the patch works for you, you shouldn't close the ticket as worksforme.)

[yurivictor]

The patch I posted works for us too, but I try not to patch core on live servers and there are bigger issues here because of different server architectures, and Twitter and ombeds in general, at least larger than a patch, in my opinion, so I'd recommend keeping this open.

In the mean time, this dirty plugin (short-term solution) works for us:

​https://gist.github.com/yurivictor/8444326

[nacin]

In 26967:

Switch Twitter oEmbed to SSL due to a Twitter API change. Ah, the open web.

props yurivictor, cojennin.
see #26844.

[nacin]

In 26968:

Realign the oEmbed providers. see #26844.

[nacin]

In 26969:

Switch Twitter oEmbed to SSL due to a Twitter API change that broke embedding across all WordPress sites.

Merges [26967] to the 3.8 branch.

props yurivictor, cojennin.
fixes #26844.

[nacin]

The Twitter conversation I linked to above continued. It would certainly be nice if Twitter at least instituted a redirect for this traffic, but I'm not holding my breath. I've subscribed to the discussion thread (as requested by the Twitter employee) and we'll see what happens. Either way, we can ship this in 3.8.1.

[martinburnsuk]

3.8.1 still fails with

• "Problem Connecting to Twitter" error output to page (if URL is on a line on its own)
• the URL with a null link output to page (if [embed] short code used)

[Ipstenu]

I can't reproduce this. I don't get an error at all on 3.8.1 or Trunk. Have you made sure to test this with all plugins off? I suspect you're having a one-off issue.

[nacin]

New tickets for any additional issues, please.