MCE Views for Audio/Video

[wonderboymusic]

Turns out, you can render players in TinyMCE. This was not easy.

[wonderboymusic]

This bent my brain in half: 27389.2.diff​

• Pauses all players when toggling between editor modes
• Pauses all players when popping open the media modal
• When "Update" is pressed: the MCE view's data-wpview-text attr is updated, the views are refreshed, and the specific type of media's render method is called to remove the player and re-add it
• wp.media.audio|video's shortcode method is now called update, which is more semantically representative.
• Triggers a ready event when a wp.mce.View is constructed
• Bind ready handlers for wp.mce.audio|video.View for when the page initially loads so we have a reference to the view's root element
• Removed the event attributes that were present in a previous patch's Underscore templates

[wonderboymusic]

27389.3.diff​ has many improvements:

• Previously, if you had the same shortcode in your post twice, it would only get rendered once. Fixed that.
• When rendering a wp.mce.View, the current node context is passed to a ready event, so when you do have multiple shortcodes that are equal, you know which one you are dealing with.
• For videos in the editor, force preload="none" - will let the video play immediately. Any of the other preload values cause the spinner to appear for many seconds. This defies logic.

[wonderboymusic]

In 27528:

Add MCE Views for audio and video. Please clear your browser cache so that you get the latest TinyMCE stylesheet.

• Move TinyMCE shortcode handling from wpgallery plugin to mce-view.js
• Force preload="none" when rendering media in the editor to ensure fast loading (I realize this sounds illogical)
• Move audio and video tag builder logic in media-template.php into PHP funcs that can be reused by any code passing data.model to an Underscore template
• Pause all players when moving between editor tabs and when moving from the editor to editing in the media modal.
• Rename wp.media.audio|video.shortcode() to the more appropriate wp.media.audio|video.update() that now returns a wp.shortcode object instead of a string.
• Add necessary MediaElement css files to $mce_css
• In wp.mce.View.render(), support multiple instances of the same shortcode
• When rendering wp.mce.Views, fire a ready event that passes the current MCE View root element as context

See #27389.

[wonderboymusic]

In 27529:

Version bump for TinyMCE stylesheet.

See #27389, [27528].

[wonderboymusic]

In 27530:

Revert [27528] until Flash in Firefox behaves :(

See #27389.

[wonderboymusic]

I tested this on every browser with every supported audio/video type: 27389.5.diff​

tl;dr MediaElement's Plugin Bridge (Flash/Silverlight) and TinyMCE do not intermingle well.

Subsequently, I added an isCompatible method to whitelist types that MEjs should parse, otherwise replace with a placeholder and bail.

[wonderboymusic]

posted a screenshot that shows intermingling of supported/unsupported videos in TinyMCE. What sucks: almost every type plays just fine in the media modal. I think there is something in the global space that doesn't translate from the MEjs bridge to TinyMCE - the errors propagate from the JS VM though, not from the app #yay

[wonderboymusic]

In 27533:

When rendering Underscore templates for Audio/Video details, use 2 new PHP functions to render the markup for audio and video tags: wp_underscore_audio_template() and wp_underscore_video_template().

Future JS templates can follow the convention of expecting data to be passed to them containing a model property.

See #27389.

[wonderboymusic]

In 27534:

Load MediaElement's CSS when TinyMCE is loaded via $mce_css in editor_settings(). Add some baseline styles in wp-content.css for audio, video, and embed tags to ensure their max-width is 100%, regardless of whether MEjs is implemented in TinyMCE.

See #27389.

[wonderboymusic]

In 27535:

Make media.view.VideoDetails.prepareSrc a static class method instead of an instance method. Properly pairs it with its incrementing instances property.

See #27389.

[wonderboymusic]

In 27536:

In media.model.PostMedia, for these 2 scenarios:

• src is set, and 'Add a Source' results in the same file (or a file with the same extension) being added
• src is set, and 'Replace Audio|Video' is used, which will add a model property named by the attachment's extension

... call model.unset( 'src' ).

See #27389.

[wonderboymusic]

In 27537:

Rather than extending wp.media.mixin, only borrow coerce when necessary in wp.media.audio|video|collection. wp.media.mixin will make sense to be mixed in for classes that expect to interact with the player. More universal methods can be added and inherited by all those who extend their prototype with it.

Assuming someone had implemented players in the editor, pause all players when switching between editor tabs. A method, pauseAllPlayers, has been added to wp.media.mixin.

See #27389.

[wonderboymusic]

In 27538:

Move removePlayer() and unsetPlayer() to wp.media.mixin so that other classes can take advantage of them.

• removePlayer() is an alternative to MediaElement's player removal method that does not re-add the audio|video element back to the DOM.
• unsetPlayer() will check for an existing player property on the passed instance and pause all players before calling unsetPlayer()

See #27389.

[wonderboymusic]

In 27539:

The canPlayType property for audio and video in JS is so bad that the official valid responses are "probably" and "maybe". There are many cases where we might want to know if an audio|video tag is gonna blow up in our face before even attempting to make a MediaElementPlayer instance out of it.

The best (and most cautious) way to tackle this is to whitelist types by browser. Imagine that one implemented MEjs in TinyMCE's rich editor mode, this would be very helpful.

Add isCompatible( $media ) to wp.media.mixin. Future features will use this.

See #27389.

[wonderboymusic]

In 27542:

In wp.media.mixin, add ua and a compat map to separate and clarify browser detection and support. Leverage Underscore list iterators in isCompatible().

See #27389.

[wonderboymusic]

27389.7.diff​ is a refresh against all of the code churn. Does not take [27544] into account yet.

[wonderboymusic]

In 27615:

Add MCE views for audio and video shortcodes. When the shortcode does not contain a source that supports native playback, just show the filename.

• Remove the audio/video shortcode parsing from the wpgallery plugin.
• Make mce-view a dependency of media-audiovideo
• Introduce wp.mce.video, wp.mce.audio, wp.mce.media, and wp.mce.media.View
• Rename wp.media.audio|video.shortcode() to wp.media.audio|video.update() since it is called on Update and returns a wp.shortcode object now.
• In wp.mce.View.render(), fire a ready event when the placeholder is being parsed and pass the current node to the event handler.

See #27389, #27437.

[wonderboymusic]

New tickets for specific bugs.

[wonderboymusic]

In 27658:

When an HTML5 fallback button is pressed in the Audio or Video Details state, filter the library by that specific mime-type when the Add Audio|Video Source is activated.

See #27389.

[wonderboymusic]

There are at least 2 patches that will need to be applied - one is the timeout on initial page load, the other is related to cut-copy-paste-undo and memory leaks

[wonderboymusic]

27389.combined.diff​ combines the patches. As far as memory management goes, TinyMCE itself leaks memory. My initial profiling (empty-editor.png​) was done against an empty Visual editor, each additional profile was done by going from Visual-Text-Visual then taking another snapshot. The second set of snapshots (one-audio.png​) was done with one audio player in the editor. I did some other tests with 21 players in the editor and the ratios were the same.

My code in the patch ensures that MediaElement's cleanup routine is run on every player when TinyMCE rebuilds itself, even when multiple players are bound to the same view - this happens when duplicate shortcodes exists. I tested this with 20 players, 7 being the exact same.

On page load, the players are initialized after a 500ms timeout to ensure that the stylesheets are loaded. Subsequent builds happen after a 10ms refresh. The broken looking players in FF were due to a race condition with the editor stylesheet.

The patch also contains @gcorne's view refresh fixes in wp.mce.View

[gcorne]

A change to the UndoManager was ​committed upstream that adds the additional context to the BeforeAddUndo hook that will allow us to prevent undo steps from being added when the body of a wpview changes and some sort of action triggers TinyMCE to attempt to add an undo level.

[wonderboymusic]

27389.combined.3.diff​ cleans up all player instances in a view when the view refreshes and the first new player is being parsed.

Since we are blowing away the inner DOM for MCE views when transition away from Visual or Undo-ing, this is necessary to clean up MEjs cruft from previously created players. They all have to be done at once to prevent extras from leaking - say you go from Visual tab to Text, delete a shortcode, and then tab back to Visual, we don't want orphaned and unbound player instances.

This is a joy to debug. Patch also removes the TinyMCE changes which are forthcoming from upstream.

[azaozz]

Replying to wonderboymusic:

There is no need to use jQuery inside the editor (TinyMCE includes Sizzle too). This bit:

_.each( tinymce.editors, function( editor ) { 
  var doc; 
  if ( editor.plugins.wpview ) { 
    doc = editor.getDoc(); 
    size += $( doc ).find( '[data-wpview-text="' + self.encodedText + '"]' ).length; 
  } 
} );

would be better as:

_.each( tinymce.editors, function( editor ) { 
  if ( editor.plugins.wpview ) { 
    size += editor.dom.select( '[data-wpview-text="' + self.encodedText + '"]' ).length; 
  } 
} );

Also self.encodedText can be quite long, might slow down or even break the selector? If you need to count only specific views, perhaps select all views first with views = editor.dom.select( '[data-wpview-text]' ); then loop over them and count only the specific views by matching the attribute value with editor.dom.getAttrib( viewNode, 'data-wpview-text' ) === self.encodedText.

[azaozz]

Thinking about this more: would it be better to have a "global" unbind event triggered in wpview? Seems the three events that force DOM rebuilding in the editor are undo, redo, and loadContent. We can easily fire an view.unbind() on these.

[wonderboymusic]

An unbind event would negate a lot of the code in my patch, which is a good thing. All of the memory cleanup could happen in the same place, before ready fires.

Once that is exposed, I can refactor my patch.

[azaozz]

27389-unbind-02.patch looks good, @wonderboymusic could you confirm it works as expected.

[wonderboymusic]

27389.9.diff​ adds a few of the things from my patch that are still necessary.

1. Need to set a longer timeout on initial page load to avoid the race condition with the editor stylesheet setting content width
2. Need to push players onto a stack, a single view can render multiple instances if the shortcode is duplicated. Thus, the unbind routine needs to iterate and remove and then reset.

Everything looks good, and this approach is much cleaner. John Resig envisioned code-heroes/cowboys like you two when he wrote Secrets of a JavaScript Ninja.

[azaozz]

27389.9.diff looks good. 27389.10.diff just adds couple more inline comments.

[nacin]

If azaozz, gcorne, and wonderboymusic are fine with this, me too.

[azaozz]

In 28084:

TinyMCE wpViews:

• Prevent undo steps from being added when the body of a wpview changes.
• Add unbind() to handle cleanup on DOM rebuilding in TinyMCE.
• Ensure that MediaElement's cleanup routine is run on every player in all instances of the editor.
• Initialize the players after some delay to ensure CSS is loaded.

Props gcorne and wonderboymusic, fixes #27389