Make WordPress Core

Opened 6 years ago

Last modified 14 months ago

#46986 accepted defect (bug)

DNT Parameter automatically for Vimeo oEmbed requests

Reported by: djc71889's profile djc71889 Owned by: garrett-eclipse's profile garrett-eclipse
Milestone: Future Release Priority: normal
Severity: major Version: 4.9
Component: Embeds Keywords: has-patch needs-dev-note has-unit-tests 2nd-opinion
Focuses: privacy Cc:

Description

This is a follow-up to #41784.

Summary: The “do not track” (?dnt=1) embed parameter is currently being automatically applied to all oEmbed requests for Vimeo videos, seemingly those embedded in self-hosted versions of WordPress only.

We’ve traced this back to the change made to address this ticket #41784 (https://core.trac.wordpress.org/changeset/41345) The reason this is only coming up now is that Vimeo only just recently added oEmbed support for DNT- https://github.com/vimeo/player.js/pull/403

The issue with this WordPress change is that adding a DNT parameter to Vimeo embeds is intended to automatically block all tracking in the Vimeo player, including player statistics. Stats are a big feature of Vimeo which many users rely on, so adding this parameter is currently leading to user confusion when stats are not collected. Many users aren’t aware that this parameter is being added for them by WordPress, nor do they understand the implications. Vimeo support currently receives 1-2 support tickets from users a day who are affected by this change and unaware of the parameter being added by WordPress, and it's ability to disable stats.

Documentation for the DNT parameter on Vimeo can be found here- https://vimeo.zendesk.com/hc/en-us/articles/360001494447-Using-Player-Parameters

Example webpages with DNT:

https://foodtv.dk/video/verdens-hurtigste-pastaret/
https://foodsthathealyoucbd.com/cbd-buyer-beware-know-what-youre-buying/

Check out this link for evidence of the DNT parameter in the iframe

Proposal: WordPress should not automatically append the DNT parameter. Users should have the option to add DNT if they determine that it fits their needs.

Note that unlike Twitter (which was mentioned in the initial report, the Vimeo player does not contain cookies which are unessential to player functionality (like saving language preferences or viewer statistics) It does not track 'non-essential' cookies like google analytics and other third party cookies (used for ad serving, etc).

Attachments (4)

46986.diff (1.5 KB) - added by garrett-eclipse 4 years ago.
Exclude dnt on vimeo.com providers and update existing oembed_fetch_url filter to provide function example for re-enabling dnt on vimeo.com
46986.2.diff (2.1 KB) - added by Mista-Flo 4 years ago.
Fix if state and add unit test
46986.3.diff (3.0 KB) - added by garrett-eclipse 4 years ago.
Add additional inverse unit test to confirm the example function provided is successful in allowing users to add dnt back conditionally.
46986.4.diff (3.0 KB) - added by garrett-eclipse 4 years ago.
Update vimeo links in unit tests to use embed sample video from Vimeo as suggested by Meta.

Download all attachments as: .zip

Change History (34)

#1 @garrett-eclipse
5 years ago

  • Focuses privacy added

#2 @mikesaville
5 years ago

I concur, this is rather frustrating. We host our videos on Vimeo, but they're only view able through the member area of our site, where we have them embedded. As a result, we can't use the standard embed block (which we found out only when all our Vimeo stats were zero). Vimeo has some massive analysis tools that we just can't access in the oEmbed's current iteration.

If we just cut and paste the Vimeo embed script into a html block, does WordPress add the DNT function too do you know? I think I may have to check later.

EDIT: Just checked, if you don't use the Vimeo embed block and just use the Vimeo embed code in a HTML block, the stats ARE recorded at Vimeo.

I hope that a future update adds it as a checkable option rather than a default that you can't override easily.

Last edited 5 years ago by mikesaville (previous) (diff)

#3 @johnbillion
4 years ago

#50869 was marked as a duplicate.

#4 follow-up: @freshyjon
4 years ago

This is also causing an issue that may have been overlooked.

If there are numerous Vimeo videos embedded on the page, previously (by default), Vimeo would only allow one of those video to play at a time. If a user played one video — then started playing another video — it would then pause the previous video that was playing. Again, this is default Vimeo behavior. However, when the Vimeo URL has the parameter of dnt=1 on the URL, it causes that default behavior to be overridden. Therefore, it allows the user to play multiple videos at once (which is NOT default behavior, and is arguably not very “Accessible friendly”).

Ideally, this would be looked in to further, since it now overrides default/expected Vimeo behavior. I argue that it should be removed from Vimeo URLs, or at least have an option to disable it from oEmbeds.

#5 follow-up: @johnbillion
4 years ago

  • Keywords 2nd-opinion added

I'll be completely honest, I don't know how best to fix this. According to MDN:

The DNT (Do Not Track) request header indicates the user's tracking preference. It lets users indicate whether they would prefer privacy rather than personalized content.

This makes no mention of collecting view statistics which can be gathered independently of tracking a user for personalisation purposes. It seems to me that Vimeo is being overly cautious with respect to the dnt parameter.

That said, I appreciate that this is affecting the stats collection for video producers, which is not ideal. It's unfortunate that there's nothing in the oEmbed spec that covers this, and off the top of my head there's no other header or parameter that could be used instead of dnt.

Vimeo embeds _could_ be excluded from using the dnt parameter, which isn't ideal, but realistically it might be about the only option.

This ticket was mentioned in Slack in #accessibility by ryokuhi. View the logs.


4 years ago

#7 in reply to: ↑ 5 @SergeyBiryukov
4 years ago

  • Milestone changed from Awaiting Review to 5.6

Replying to johnbillion:

This makes no mention of collecting view statistics which can be gathered independently of tracking a user for personalisation purposes. It seems to me that Vimeo is being overly cautious with respect to the dnt parameter.

Yes, that's my understanding here as well. They seem to be using it to block all kinds of statistics, rather than just user tracking.

Vimeo embeds _could_ be excluded from using the dnt parameter, which isn't ideal, but realistically it might be about the only option.

Since this is causing accessibility issues as is, per comment:4, let's go with that for now.

This ticket was mentioned in Slack in #core by metalandcoffee. View the logs.


4 years ago

#9 @Mista-Flo
4 years ago

  • Keywords needs-patch added

#10 @helen
4 years ago

I think a patch approach here would be to add a filter around the line added in [41345] that filters an array of providers *not* to apply the dnt query arg to, with Vimeo in there as a default item in the array. We may need to move this before the $provider variable is changed, or use a separate variable for the changed $provider URL.

#11 @hellofromTonya
4 years ago

5.6 Beta 1 is less than 2 weeks away. Adding a note here to invite a contributor to add a patch in the coming days, i.e. to keep it on track for consideration to land in the 5.6 release.

@garrett-eclipse
4 years ago

Exclude dnt on vimeo.com providers and update existing oembed_fetch_url filter to provide function example for re-enabling dnt on vimeo.com

#12 @garrett-eclipse
4 years ago

  • Keywords has-patch needs-testing needs-unit-tests needs-dev-note added; 2nd-opinion needs-patch removed
  • Owner set to garrett-eclipse
  • Status changed from new to accepted
  • Type changed from enhancement to defect (bug)
  • Version set to 4.9

In 46986.diff I've made an initial patch on a simplistic approach focused on vimeo.com dnt.

The dnt setup is followed by the oembed_fetch_url which allows you to update the oembed $provider already. So instead of a new filter I just provided a @since update and function example for how users can re-enable dnt for vimeo.com now that we've excluded that one provider specifically.

Please test not only the change but also the function example.

I added needs-unit-tests but am unsure if that's necessary but may be helpful in confirming dnt is not present on vimeo.com urls. This will though need a dev note.

This ticket was mentioned in Slack in #core by garrett-eclipse. View the logs.


4 years ago

#14 @freshyjon
4 years ago

Here is an example that was mentioned in a WordPress.org support topic: https://wordpress.org/support/topic/vimeo-oembed-now-has-dnt1-parameter/#post-13481656

@Mista-Flo
4 years ago

Fix if state and add unit test

#15 @Mista-Flo
4 years ago

  • Keywords has-unit-tests added; needs-unit-tests removed

Hi there,

thanks for your patch @garrett-eclipse, I have uploaded another patch to fix the if state, the code was doing the opposite of what we wanted. It was adding the dnt attribute only to vimeo oembeds.

I have also added one unit test, but I am not sure it's a good one, I have copy pasted the URL of a public video from vimeo.

#16 @garrett-eclipse
4 years ago

Nice hahaha thanks for catching that, I thought I was just looking at the log backwards but guess it was my logic ;)

And appreciate the unit test, it's working nicely. I tried to find a wp.org vimeo account or video we could use to be official but I couldn't find one. Maybe we can get #meta to setup an account with a single video for this purpose.

Also I applied one additional unit test, now that I see how you've done it there, to apply the function example I provided in the docblock which confirms the inverse of default can easily be achieved with the existing filter.

@garrett-eclipse
4 years ago

Add additional inverse unit test to confirm the example function provided is successful in allowing users to add dnt back conditionally.

#17 @Mista-Flo
4 years ago

Nice one, thanks for the enhancement.

Yes for the vimeo URL that was my concern, I did not find any official WordPress account to rely on.

This ticket was mentioned in Slack in #meta by garrett-eclipse. View the logs.


4 years ago

This ticket was mentioned in Slack in #core-media by garrett-eclipse. View the logs.


4 years ago

#20 @garrett-eclipse
4 years ago

I've made meta#5467 to create a Vimeo account and upload a video for unit tests.

I don't feel we should block inclusion of this into core to wait on that, especially as this is unit tests and not front-end core code. We can always update unit tests once meta is processed.

@garrett-eclipse
4 years ago

Update vimeo links in unit tests to use embed sample video from Vimeo as suggested by Meta.

#21 @garrett-eclipse
4 years ago

Meta Feedback from dd32 (Thank you 🙏):

Instead of using WordPress test accounts, could we instead fall back onto using a video uploaded by the embed company instead?

For example, for Vimeo, anything from Vimeo Staff https://vimeo.com/staff or the test video from https://player.vimeo.com/embed should be safe to use going forward. edit: The player embed page actually has a bunch of test videos with various states, such as not-embeddable, hosted by official vimeo test accounts, seems perfect for this use-case.

Having official WordPress accounts for every embed platform is not going to be viable from a management perspective IMHO, and having accounts everywhere with only a single W logo video doesn't lead to a super great experience when it ends up in search results.

Where WordPress does have a use for the service (Such as Youtube) using one of our uploads such as a release video does make sense though.

I've updated the patch 46986.4.diff to use the Vimeo Embed Sample found here - https://player.vimeo.com/embed

Want to give this last one a re-test @Mista-Flo and if happy mark for committers review.

#22 @Mista-Flo
4 years ago

  • Keywords needs-testing removed

Thanks for the update Garrett, it was worth it. I have tested the patch again, unit tests are passing so all good :D

#23 @garrett-eclipse
4 years ago

  • Keywords commit added

Thanks @Mista-Flo that's great. Let's move this forward to get a committers review.

This ticket was mentioned in Slack in #core by sncoker. View the logs.


4 years ago

#25 in reply to: ↑ 4 ; follow-up: @adakaleh
4 years ago

Before deciding on this, I think several points ought to be clarified:

Replying to djc71889:

Note that unlike Twitter (which was mentioned in the initial report, the Vimeo player does not contain cookies which are unessential to player functionality (like saving language preferences or viewer statistics) It does not track 'non-essential' cookies like google analytics and other third party cookies (used for ad serving, etc).

The above statement is false. Vimeo's player stores an unessential tracking cookie called vuid. It's unessential because the video player works without it when DNT is used. This cookie can be used by Vimeo to track people across the web through its embedded videos.

Vimeo also sets the sync_active key in local storage. This is what Vimeo uses to stop videos from playing simultaneously in different tabs and iframes. It can also be used to track people across websites, which is why Vimeo blocks it when DNT is used.

The stats mentioned in https://vimeo.com/analytics can be obtained without unique identifiers. The same viewer might be counted more than once, but Vimeo could still show its clients "where they’re coming from, how they’re finding your videos, and what devices they’re watching them on" (and more) without tagging each visitor with a unique ID. Even so, much of this data is perhaps too invasive and it's arguably a good thing that DNT prevents its collection.

As for the issue raised by freshyjon:

it allows the user to play multiple videos at once (which is NOT default behavior, and is arguably not very “Accessible friendly”).

How is this an accessibility issue? To me it looks like a tiny inconvenience, far outweighed by the privacy costs of going through with this change.

To conclude, please be mindful of the privacy implications herein. I suggest not allowing Vimeo to push its tracking-by-default exception into WordPress.

#26 follow-up: @johnbillion
4 years ago

  • Keywords 2nd-opinion added; commit removed
  • Milestone changed from 5.6 to Future Release

Punting based on the above. Can someone else confirm this?

#27 in reply to: ↑ 25 @garconis
4 years ago

How is this an accessibility issue? To me it looks like a tiny inconvenience, far outweighed by the privacy costs of going through with this change.

Because if someone is watching a video, then plays a different one, it's difficult for them to understand why two sources of audio is now playing. If they're on a device with controls other than a mouse (e.g., keyboard only, or no visual screen), it'd likely be a challenge to find the original playing video and then figure out how to pause it... so that both aren't playing.

Beyond issues like that, it's just a bad user experience and unexpected.

#28 in reply to: ↑ 26 @adakaleh
4 years ago

Replying to johnbillion:

Can someone else confirm this?

How to confirm: open a video embed without DNT (example), open the browser's dev tools and go to Storage (Firefox) or Application -> Storage (Chrome). The vuid cookie is set before you press play and sync_active (in local storage) is set after.

This is the script that generates the vuid cookie: https://f.vimeocdn.com/js_opt/modules/utils/vuid.min.js
It is minified, but quite readable if pretty-printed. Note that it explicitly sets the SameSite attribute to "None", so even though browsers are changing the default to "Lax", Vimeo will still be able to use this cookie from a third party context.

Replying to garconis:

it's just a bad user experience

The problem is that people don't experience tracking. It's generally done to them without their knowledge or (informed) consent. To make it more concrete: if a Vimeo employee walked in and took notes whenever you opened a page with embedded video, you would probably consider that a worse experience than having to pause a video before playing a different one.

Replying to garconis:

and unexpected

This is based on user preference. I actually think most people (with or without accessibility problems) expect to be able to play multiple videos simultaneously - for instance, to play music in the background while focusing on another video. Without DNT, Vimeo only allows one video to play at a time, even in separate tabs. This type of behavior can be frustrating for users. YouTube doesn't have this, and since YouTube embeds are far more widespread, they drive most users' expectations.

Plus there's the issue of inconsistent behavior when embeds from multiple providers are mixed within a page.

This would, however, be a nice optional feature if it could be done consistently and without tracking, which is what browsers and browser extensions are in a position to do.

#29 @georgestephanis
3 years ago

Just noting here that if someone does need to disable dnt for vimeo videos, it's not hard to do on your own site via a filter like so:

add_filter( 'oembed_fetch_url', function( $provider ) {
	if ( strpos( $provider, 'vimeo.com' ) !== false ) {
		$provider = remove_query_arg( 'dnt', $provider );
	}
	return $provider;
} );

The discussion in this thread is more about whether the default that WordPress ships with should change, and if show how to effectuate that change.

Version 0, edited 3 years ago by georgestephanis (next)

#30 @mikecl
14 months ago

Hi, I just discovered this, as I'm using Vimeo on my site. Are there any updates on this?

Note: See TracTickets for help on using tickets.