WordPress.org

Make WordPress Core

Opened 15 months ago

Last modified 2 months ago

#23285 new feature request

Implement an AMD JavaScript loader in WordPress

Reported by: auniquename Owned by:
Milestone: Awaiting Review Priority: normal
Severity: normal Version:
Component: Script Loader Keywords:
Focuses: Cc:

Description

AMD = Asynchronous Module Definition

See comment: http://core.trac.wordpress.org/ticket/22896#comment:29

I think this is a must for the future viability of WordPress and plugins written for it.

Multiple versions of JavaScript libraries are a fact of life and we have to deal with it somehow.

If WordPress provides the framework for the use of multiple versions of JavaScript libraries then a lot of the problems we see with WordPress now will be a thing of the past.

This represents a major re-think of the current WordPress JavaScript loading paradigm, but I think it is necessary and would be well worth it.

I'm not a terribly advanced developer, and there might be better solutions out there or others to come, but for now, I think this is the way to go.

Example:

http://requirejs.org/docs/why.html

http://requirejs.org/docs/whyamd.html

https://github.com/amdjs/amdjs-api/wiki/AMD

It does not have to be RequireJS, but it seems the most mature and popular of the implementations.

From the unminified jQuery source:

// Expose jQuery as an AMD module, but only for AMD loaders that
// understand the issues with loading multiple versions of jQuery
// in a page that all might call define(). The loader will indicate
// they have special allowances for multiple jQuery versions by
// specifying define.amd.jQuery = true. Register as a named module,
// since jQuery can be concatenated with other files that may use define,
// but not use a proper concatenation script that understands anonymous
// AMD modules. A named AMD is safest and most robust way to register.
// Lowercase jquery is used because AMD module names are derived from
// file names, and jQuery is normally delivered in a lowercase file name.
// Do this after creating the global so that if an AMD module wants to call
// noConflict to hide this version of jQuery, it will work.
if ( typeof define === "function" && define.amd && define.amd.jQuery ) {
	define( "jquery", [], function () { return jQuery; } );
}

Note: multiple versions of jQuery

Change History (16)

comment:1 auniquename15 months ago

Though thinking about it, perhaps the existing *_enqueue_scripts model could be modified to be AMD literate while preserving backward compatibility by doing the AMD implementation transparently behind the WordPress API. (?)

Version 0, edited 15 months ago by auniquename (next)

comment:2 scribu15 months ago

  • Keywords reporter-feedback added

If WordPress provides the framework for the use of multiple versions of JavaScript libraries then a lot of the problems we see with WordPress now will be a thing of the past.

Problems such as?

comment:3 auniquename15 months ago

See ticket linked above (and again below):

http://core.trac.wordpress.org/ticket/22896

Plugin collisions with other plugins and the core functionality of WordPress (usually, but not always, in admin).

I have had to resort to an ugly and inefficient hack to ensure that the jQuery I want to use in my plugin does not conflict with the one used in WordPress or with the jQuery used by other plugins. And to attempt to have a plugin with a working lifespan longer than the WordPress and jQuery release cycles.

However I have hit other significant issues with jQuery itself and I am attempting to report issues there, but their bug tracker registration is broken and I have to resort to their forums.

Basically, neither WordPress or jQuery is ready for a plugin as complicated as I want to produce. I hope by submitting this feedback that someday they will be.

Ultimately the goal is to use multiple jQuery UI themes in a plugin in such a way that it does not break anything else and will work for an extended period of time with little to no maintenance (see long comment in ticket linked above).

As I see it, this needs to happen. Though perhaps I expect too much?

Last edited 15 months ago by auniquename (previous) (diff)

comment:4 wonderboymusic15 months ago

"Basically, neither WordPress or jQuery is ready for a plugin as complicated as I want to produce"

What does that even mean? Please describe so someone can help you get there - I doubt it's impossible

comment:5 scribu15 months ago

Since jQuery hardcodes the module name as 'jquery', I don't see how AMD would help with having two versions loaded separately.

From the ticket you mention:

With jQuery there is a better way to do this: let the WordPress jQuery load first, then load your desired version in the footer or right before it's needed [...]

http://core.trac.wordpress.org/ticket/22896#comment:30

Last edited 15 months ago by scribu (previous) (diff)

comment:6 follow-up: auniquename15 months ago

@scribu - Roger. I've got that now. Though I will use the solution outlined later in that ticket. WordPress does not necessarily need to use AMD to implement multiple js lib versions dependencies, but why re-invent the wheel? Alternatively, if plugin authors are encouraged to bundle and isolate jQuery themselves as discussed in that other ticket, then why make the jQuery used internally by WordPress available to plugin authors as it is now at all? And why is the solution I have arrived at with the assistance here not documented? (I know, it is in the jQuery docs, but the WordPress codex leads one to register their scripts with dependencies on the jQuery included with WordPress)

@wonderboymusic - Basically, it comes down to using multiple fully css scoped jQuery UI themes on a single page that will continue to work across at least a few releases of WordPress and jQuery. (If called for, depending on how the user sets the plugin options, and without breaking any other plugins or native WordPress functionality.) I am not so much concerned with what happens in admin as on the front end. I now have a way to make WordPress do it by isolating my jQuery and its plugins as discussed (but this is not the standard or documented way to use jQuery in WordPress). Currently the challenge that I face now that I can reliably isolate jQuery and its plugins in WordPress is the failings of jQuery UI itself. When you fully css scope a theme (every single css selector, not just the ones they scope in the themeroller - to preserve a theme behaviour across mutliple jQuery UI versions without collision with other jQuery UI themes from earlier or later versions) - you soon discover that many of the jQuery UI functions do not really take into account a css scoped theme at all (even a partially scoped one as the themeroller does).

What happens is that things like overlays, and element clones used for animations and effects fall out of the css scope by being appended to elements (usually body) outside your defined css scope - and one cannot apply their jQuery UI scoping class to the body of a document without potentially affecting jQuery UI themes elements added before or after the ones you are working with.

Hopefully that fills in some of the blanks for you. Right now my issue is with overlays (I have my own hack for that and it has been reported in the jQuery bug tracker), and with the explode effect falling out of theme scope, which has not been reported, and I seem to be unable to register on their bug tracker all names no matter how unique (even tried GUIDs) are reported as taken. Grrr. So I have registered on their forums to report some of this stuff.

In jQuery UI 1.9.2 scoped overlays did not appear at all and had to be kicked into scope, now in 1.10.0 they appear in scope but out of z-order and over the element they are supposed to be behind. (I have hacks for both cases.) What is clear is that the jQuery UI developers are not testing under multiple scoped themes at all. That is half the point of jQuery UI - themed UIs!!! Arg!

The reason I wanted to use jQuery UI 1.10.0 is that it corrects the dialog falling out of scope by allowing it to be appended to elements other than the body without a hack that adversely affects the body dimensions. i.e. this:
http://stackoverflow.com/questions/2199888/custom-css-scope-jquery-ui-dialog-themes
is no longer required.

However there is still the overlay and explode issues (both existed prior to 1.10.0) and now icon button focus and sprite issues with the themes as implemented in 1.10.0 that did not exist in 1.9.2.

So neither product is really ready for what I want to make them do, but it is so close... I can taste it. But the bulk of this reply is actually for the broken jQuery bug tracker...

The issue relating directly to this ticket is that I still see the need to use some way to handle multiple versions of jQuery in WordPress in a standard way. I suggested AMD as it seems to be the direction this type of thing is going, though it need not be done that way, there is more than one way to skin this cat. The alternative, and perhaps better solution (?) is not to encourage plugin authors to use the jQuery included in WordPress at all and lock it down for internal use only, and document or link the proper way to isolate your own copy of jQuery to use in your plugin as I have arrived at for my current solution. It is all about plugin stability and longevity.

When a new plugin breaks an old one on a site, all the client knows is that their site is broken. Or when a WordPress update breaks a plugin that a site depends on... the user is inclined not to update, but what if that update contains a critical security update? I see a lot of job postings to fix old WordPress sites that have been compromised. The current situation does not really do much to prevent that from happening. I found a way to do it, and then a better ("the right way") to do with with the help here, but how many plugin authors will go to the trouble that I have to ensure that?

And lo and behold... as I have been writing this:

  • Work around some misconfigurations that may have caused some JavaScript in the WordPress admin area to fail.

...

  • Server-side request forgery (SSRF) and remote port scanning via pingbacks. Fixed by the WordPress security team.
  • Cross-site scripting (XSS) via shortcodes and post content. Discovered by Jon Cave of the WordPress security team.
  • Cross-site scripting (XSS) in the external library Plupload. Plupload 1.5.5 was released to address this issue.

http://codex.wordpress.org/Version_3.5.1

Though I see this after updating:

External Libraries
WordPress now includes the Underscore and Backbone JavaScript libraries. TinyMCE, jQuery, jQuery UI, and SimplePie have all been updated to the latest versions.

Not quite so. WordPress' jQuery and jQuery UI remain at 1.8.3 and 1.9.2 respectively, while the current versions are 1.9.0 and 1.10.0. WordPress' Backbone is at 0.9.2 not the current 0.9.10 ...

This adds weight to my argument.

Last edited 15 months ago by auniquename (previous) (diff)

comment:7 in reply to: ↑ 6 SergeyBiryukov15 months ago

Replying to auniquename:

Alternatively, if plugin authors are encouraged to bundle and isolate jQuery themselves as discussed in that other ticket, then why make the jQuery used internally by WordPress available to plugin authors as it is now at all?

Quite the contrary, plugin authors are encouraged to use the jQuery version bundled with WordPress, and update the plugins accordingly. Your situation, as described in ticket:22896:29, sounds like an edge case to me.

comment:8 follow-up: auniquename15 months ago

Perhaps so in my case, as I am more particular than most.

That said, my comments on people not wanting to update in order to preserve the functionality of unmaintained plugins stand, and that is damaging overall to the reputation of WordPress, particularly with regard to security.

People have custom plugins written for their site alone, and then may never hear from the developer again and are unable to maintain code themselves. Ya, their bed, lie in it, yada, yada... But if we can prevent that, then why not?

I also have this crazy half baked idea of snapshotting the current version of jQuery in WordPress when a plugin that references it is installed, then storing it in the db and feed the plugin that copy henceforth... until the plugin itself is updated. I haven't really thought that all the way through though. Probably lots of pitfalls.

I just don't see any reason why not include some way to deal with js lib version dependencies. Not doing so causes more problems than doing so as I see it.

Also, the release cycle is frequent enough to make all this a lot of work, too much? Even just thinking about how to write a plugin that supports multiple versions of WordPress and multiple versions of js libs is giving me braincramps. I cannot count on users updating their sites to the latest versions, so if I want my plugin to work on the most sites possible I need to write it to account for a whole quarterly history of jQuery revisions back a few years at least. Why put people through that? Overall man hours expended to implement an AMD style loader or some kind of mechanism to deal with multiple versions of js libs in WordPress would be considerably less. Set it and forget it I say. Computers and software are meant for automating such arduous and burdensome tasks.

Last edited 15 months ago by auniquename (previous) (diff)

comment:9 auniquename15 months ago

  • Keywords reporter-feedback removed

comment:10 in reply to: ↑ 8 bpetty15 months ago

Putting aside @auniquename's personal issues working with jQuery and other JS libs themselves, the idea of using RequireJS with multiversion support actually does have some merit, and is at least worth some investigation, discussion, and possible patch down the road.

As was mentioned, WP does update bundled JS libraries very frequently, and they are often updated to API incompatible versions. They might as well be since there's no stable API policy around 3rd party JS libraries still. Most of the libraries WP bundles are bundled because they are quality, popular libraries, and so they often do have good policies around backwards compatibility even when upgrading to API incompatible versions (often only removing really old API like WP occasionally does, or infrequently used API). So it doesn't happen very frequently, but plugins do occasionally break when WP is updated and those JS libraries the plugin was originally built on are updated with it to API incompatible versions. That is the definition of "API incompatible", code originally written to work with older versions is not guaranteed to work with the new API incompatible branch.

I'd be surprised if the plugin review team hasn't seen this with old plugins. Do they even review old plugins, particularly ones with high incompatible votes in the compatibility matrix, or only new ones?

Anyway, I believe these are the same issues @auniquename is referring to. They are also the same issues I've referred to in the past when discussing plugin API manifest versioning. I actually do believe this is one approach to the problem that could potentially work.

Replying to auniquename:

People have custom plugins written for their site alone, and then may never hear from the developer again and are unable to maintain code themselves. Ya, their bed, lie in it, yada, yada... But if we can prevent that, then why not?

Sorry, but you'll generally find that most of the WP community doesn't care about these people, and the response you'll almost always find from WP developers is "upgrade your plugins or fix them" - and if their developer has abandoned them or disappeared, the answer is hire another. You won't get far with that argument.

Also keep in mind though that if you feel like you're being pushed back, it's because you're making a suggestion to change something without even providing a patch to review and test out. Anyone that agrees with you in a situation like this seems to believe they are volunteering to do the work (and I wish they didn't because they aren't). When that happens, they're always inclined to defend however it is that WP currently handles (or doesn't handle) that issue.

comment:11 bpetty15 months ago

  • Cc bpetty added

comment:12 auniquename15 months ago

Right. So why build something over and over and over when you can build it once? I'm glad you see the benefit of the idea.

If it has to wait for me to do it for WordPress myself, then so be it. Fine, I can deal with that. But it will take me much longer than a more experienced or skilled developer. I'm a chemist by education and training, not a computer scientist. (Though I understand all they teach in school now is Java with a big easy button.) Also, I would test it to death so that it is guaranteed to work and be near perfect in every way.

Anyone that agrees with you in a situation like this seems to believe they are volunteering to do the work (and I wish they didn't because they aren't).

(This is an excellent point, thank you for reminding me.)

The problem is that it will not pay the rent and I have about 8 weeks left until I am on the street. So it just is not going to happen right now. Sorry I don't have a patch for you, but the idea is a good one. My dilemma continues unresolved in the other ticket, there are just too many possible use cases.

I am not alone in my issues with jQuery UI, it is far from personal, I just seem to have found more bugs than others by pushing the product to its (intended) limits.

And whether you like it or not, compromised WP sites reflect poorly on WP as much as on their owners.

Sorry if this post comes across as snarky, I do not mean it to be, I'm just tired and frustrated and should really go to bed. Everyone participating in this discussion has been most civil and helpful and I appreciate that a great deal, it is a rare thing on the internet.

Last edited 15 months ago by auniquename (previous) (diff)

comment:13 lkraav6 months ago

  • Cc leho@… added

comment:14 ericlewis4 months ago

+1 on this.

There's basic performance boosts that would be gained from utilizing AMD.

comment:15 nacin3 months ago

  • Component changed from General to Script Loader

comment:16 ircbot2 months ago

This ticket was mentioned in IRC in #wordpress-dev by tierra. View the logs.

Note: See TracTickets for help on using tickets.