WordPress.org

Make WordPress Core

Opened 18 months ago

Last modified 9 months ago

#22316 new enhancement

Plugin Dependencies (Yet Another Plugin Dependencies Ticket)

Reported by: Viper007Bond Owned by:
Milestone: Awaiting Review Priority: normal
Severity: normal Version: 3.4.2
Component: Plugins Keywords: dev-feedback
Focuses: Cc:

Description

Previously: #10190 #11308 #13296 and I'm sure many more

It's been a few years since we looked at plugin dependencies and this still seems to be a feature people really, really want, especially for shared functionality that isn't a plugin in itself. For example a PHP library that isn't popular enough to be in core but is popular enough to be bundled in multiple plugins.

A bunch of us sat down and talked about this at this year's WordPress Community Summit and there was a lot of enthusiasm for this type of functionality.

We didn't know about the existing tickets at the time but the general summary of what we came up with was this:

  • Plugins list WP.org slugs of their dependencies in their readme.txt, or perhaps better their plugin's header.
  • When you go to install a plugin via the plugin directory UI in the admin area, the WP.org API returns a list of dependencies along with the data about the plugin being installed. WP would say like "these following dependencies will also be installed". This means it's seamless to the user -- they install a plugin and the other plugin(s) that are needed get installed too.
  • No versioning support. It's too complicated and what if one plugin wants an older version of a dependency than another plugin does? If your plugin is listing another as a dependency, then it's your job to make sure it stays compatible with the latest version of the dependency. On the flip side, hopefully plugins that get listed as dependencies are made to be forwards and backwards compatible.
  • Probably not allowing the disabling of plugins that are dependencies while their dependents are active. This seems better than disabling the dependents when the dependency is disabled ("why did Foo get disabled? I only disabled Bar!").
  • On plugin re-activation or on activation of a plugin uploaded via FTP, make sure it's dependencies are already installed. If not, offer to install them. If installed but disabled, just enable them for the user.

So while the previous tickets were closed as wontfix in the past, I think this is worth taking another look at. A lot of planning and thought will be required though to get this right.

Change History (68)

comment:1 batmoo18 months ago

  • Cc batmoo@… added

comment:2 danielbachhuber18 months ago

  • Cc danielbachhuber added

comment:3 mordauk18 months ago

  • Cc pippin@… added

comment:4 apeatling18 months ago

  • Cc apeatling added

comment:5 husobj18 months ago

  • Cc ben@… added

comment:6 DJPaul18 months ago

  • Cc djpaul@… added

comment:7 tollmanz18 months ago

  • Cc tollmanz@… added

comment:8 sc0ttkclark18 months ago

  • Cc lol@… added

comment:9 kraftbj18 months ago

  • Cc bk@… added

comment:10 blobaugh18 months ago

  • Cc ben@… added

comment:11 beaulebens18 months ago

  • Cc beau@… added

comment:12 boonebgorges18 months ago

  • Cc boonebgorges@… added

I'm working on a project where we've built an implementation of plugin dependencies on top of scribu's solution. It includes versioning support, so that you can require a minimum or maximum version of a given plugin. We've also got support for auto-installing dependent plugins.

See https://github.com/cuny-academic-commons/commons-in-a-box/blob/master/admin/plugins-loader.php for the basic dependency stuff + versioning, and https://github.com/cuny-academic-commons/commons-in-a-box/blob/master/admin/plugin-install.php for the plugin installation/upgrade dependency logic.

comment:13 williamsba118 months ago

  • Cc brad@… added

comment:14 griffinjt18 months ago

  • Cc thomasgriffinmedia@… added

comment:15 goldenapples18 months ago

  • Cc goldenapplesdesign@… added

comment:16 nvwd18 months ago

  • Cc nowell@… added

comment:17 scribu18 months ago

  • Cc scribu added

comment:18 sabreuse18 months ago

  • Cc sabreuse added

comment:19 griffinjt18 months ago

On the topic of versioning dependencies, I discussed this at length with a friend of mine who is a PHP/Java developer and we came up with this: https://github.com/bubba-h57/PHP-Library-Version-Framework. Using namespaces plus a factory was the only viable way we could find to fix the issue of class version dependencies, but that can't even be considered until WordPress support PHP 5.3 and most plugin devs won't be able to implement something like it.

While I think header comments are nice, I think a more robust API should be created for handling this, especially when you start getting into the issue of forcing activation/deactivation with dependencies. Maybe header comments could be used to initialize the API, and from there plugin dev could utilize a class with hooks/filters.

External plugin dependencies (e.g. commercial plugins) should also at least be considered should something like this make it into core. There doesn't need to be a specific way to handle external items but at least some hooks or filters to allow commercial plugins to make use of it.

I'd be more than happy to contribute to the planning and development process of this. It's why I created TGM Plugin Activation in the first place. :-)

comment:20 wpsmith18 months ago

  • Cc travis@… added

comment:21 kurtpayne18 months ago

  • Cc kurtpayne added

comment:22 follow-up: scribu18 months ago

Replying to griffinjt:

While I think header comments are nice, I think a more robust API should be created for handling this, especially when you start getting into the issue of forcing activation/deactivation with dependencies.

Header comments are preffered because they can be parsed without activating the plugin first.

On the issue of forcing activation, when you specify a dependency, you say "I _need_ this to be active".

If you don't need it to be active, it's not a dependency; it's a nice-to-have, which should be specified using a separate header or some other method.

The issue is: is it safe for plugin X be auto installed and activated? For example, you wouldn't want BuddyPress to be auto-installed and activated, since it changes your site in a 100 ways.

But, I think this should be specified by the library, with a header like "Provides: metabox". Besides providing a stable name for the library, it could also signal that it's safe to auto-activate.

comment:23 rmccue18 months ago

We want to make sure that we think about circular dependencies as well, just to make sure we don't get stuck in infinite loops.

In terms of working out a dependency tree, Composer ported openSUSE's Libzypp dependency solver, which seems complex but might be a good start to actually doing that if we strip it down (and de-PHP5.3 it).

comment:24 jbobich18 months ago

  • Cc jbobich added

comment:25 follow-up: rmccue18 months ago

Replying to griffinjt:

On the topic of versioning dependencies, I discussed this at length with a friend of mine who is a PHP/Java developer and we came up with this: https://github.com/bubba-h57/PHP-Library-Version-Framework. Using namespaces plus a factory was the only viable way we could find to fix the issue of class version dependencies, but that can't even be considered until WordPress support PHP 5.3 and most plugin devs won't be able to implement something like it.

We discussed this and decided that we'd only implement it at a basic level, since resolving multiple versions is going to be a whole new layer of complexity. It then becomes the plugin's problem to ensure that they maintain forwards- and backwards-compatibility.

External plugin dependencies (e.g. commercial plugins) should also at least be considered should something like this make it into core. There doesn't need to be a specific way to handle external items but at least some hooks or filters to allow commercial plugins to make use of it.

I think as long as there are filters for this, it should be fine. i.e. Depends: abc def othersite:ghi would work, I think.

comment:26 griffinjt18 months ago

Replying to scribu:

Header comments are preffered because they can be parsed without activating the plugin first.

On the issue of forcing activation, when you specify a dependency, you say "I _need_ this to be active".

If you don't need it to be active, it's not a dependency; it's a nice-to-have, which should be specified using a separate header or some other method.

You are correct, and I see the benefit of header comments now.

The issue is: is it safe for plugin X be auto installed and activated? For example, you wouldn't want BuddyPress to be auto-installed and activated, since it changes your site in a 100 ways.

But, I think this should be specified by the library, with a header like "Provides: metabox". Besides providing a stable name for the library, it could also signal that it's safe to auto-activate.

+1 to this

comment:27 in reply to: ↑ 25 griffinjt18 months ago

Replying to rmccue:

I think as long as there are filters for this, it should be fine. i.e. Depends: abc def othersite:ghi would work, I think.

That would be perfect. Nothing drastic, but at least a way to commercial plugin devs to take advantage of it, kind of like how you can tap into plugins_api for updates outside of the repo.

comment:28 follow-up: MikeSchinkel18 months ago

  • Cc mike@… added

Ironically I brought up a discussion on wp-hackers yesterday to address this issue, but proposed "libraries" for this use case because of the past pushback to adding plugin dependency support into core.

One aspect of the library proposal is that libraries would be hidden from users and I think it's critical that if address this. Plugins add a lot of conceptual overhead for the user and we all know that many blog posts and peer-to-peer discussions at WordPress meetup groups tell users "limit the number of plugins you use."

Imagine a plugin that uses four (4) other plugins as dependencies; if a user installs that plugin now they have five (5) active visible plugins and their anxiety shoots through the roof. This will cause plugin and theme developers to avoid using many or any dependencies even if modularity is the best solution otherwise because they know uninformed users will tell their peers "Avoid that theme/plugin; it adds too many other plugins" even when the code of adding those plugins are essential to accomplish the task.

Drupal has had module dependencies for when I started using it in 2008 and the conceptual overload for users was a problem with it too.

An example of this type of dependent plugin would be an API client SDK for a company like MailChimp or FreshBooks. Why does the user need to see that a plugin that wraps one of those APIs is active? Would it not be much better to simply to have them see the plugin that uses it?

Proposal: Consider doing one of the following:

  1. Allow plugins to market themselves as a library and if marked can't be activated by themselves and would be invisible to the user.
  2. Display dependent plugins only when the admin user clicks a "Show dependent plugins" link in the plugin list.
  3. Implement the library proposal from wp-hackers instead.

Another question to ponder is: "How much overhead does each plugin add vs. using require() or a class auto-loader?" Will 100 plugins being activated slow down a site vs. require()ing 100 PHP files? If so, we may seriously consider a "library" concept instead of or in addition to plugin dependencies.

Last edited 18 months ago by MikeSchinkel (previous) (diff)

comment:29 in reply to: ↑ 22 MikeSchinkel18 months ago

Replying to scribu:

But, I think this should be specified by the library, with a header like "Provides: metabox". Besides providing a stable name for the library, it could also signal that it's safe to auto-activate.

This would be nice because it would require officially delineating a list of "features" that a plugin could add. That could then be used as metadata on WordPress.org plugin search as well as other locations.

Last edited 18 months ago by MikeSchinkel (previous) (diff)

comment:30 taylorde18 months ago

  • Cc td@… added

comment:31 stephenh198818 months ago

  • Cc contact@… added

comment:32 in reply to: ↑ 28 ; follow-up: Viper007Bond18 months ago

Replying to MikeSchinkel:

Another question to ponder is: "How much overhead does each plugin add vs. using require() or a class auto-loader?" Will 100 plugins being activated slow down a site vs. require()ing 100 PHP files? If so, we may seriously consider a "library" concept instead of or in addition to plugin dependencies.

Why don't you test and see? ;)

Active plugins being loaded are just done via an <code>include()</code> or <code>require()</code>. There's basically no difference. For that matter the include itself has a trivial amount of overhead.

As the saying goes, it's quality not quantity. You could run 500 plugins if you wanted to.

But this is off-topic from this ticket. :)

comment:33 in reply to: ↑ 32 MikeSchinkel18 months ago

Replying to Viper007Bond:

Why don't you test and see? ;)

Isn't one of the benefits a community provides being to multiply the time and effort of its participants? It seems like a foolish waste of time to set up a benchmark when one question can potentially leverage the experience of someone else who has already evaluated.

Active plugins being loaded are just done via an <code>include()</code> or <code>require()</code>. There's basically no difference. For that matter the include itself has a trivial amount of overhead.

As the saying goes, it's quality not quantity. You could run 500 plugins if you wanted to.

My memory was that more happened during plugin load than it does (I just reviewed the code.) OTOH running 500 plugins would do 500 include()s for every page load whereas having the ability to load on demand would be useful for PHP files that provide supporting functionality to only selected URLs.

Maybe a "Load" header for "on-demand" or "always" would be useful and then a "require_plugin()" function that would load if not already loaded?

But this is off-topic from this ticket. :)

Really? Seems a discussion of potential ramifications of dependencies would be on-topic.

comment:34 follow-up: scribu18 months ago

The performance concern is valid. I've heard tales of ruby gems having 20+ dependencies, most of which weren't warranted.

We should be able to do on demand loading, especially if we use the WP_Dependencies class, but it's something we can easily add in a v2.0 iteration.

comment:35 in reply to: ↑ 34 MikeSchinkel18 months ago

Replying to scribu:

We should be able to do on demand loading, especially if we use the WP_Dependencies class, but it's something we can easily add in a v2.0 iteration.

That's cool, it'll be less of a problem early on. Just as long as we don't for some reason paint ourselves into a corner that won't let us address later.

comment:36 pogidude18 months ago

  • Cc pogidude added

comment:37 follow-ups: dougal18 months ago

  • Cc dougal@… added

From what I've seen in the Drupal space, dependencies help encourage devs to collaborate on common APIs that they can all use for their own modules. For example, a CDN caching API plugin -- One author can build an S3 plugin on top of it, another dev could build a MaxCDN plugin, or Rackspace Cloud, or whatever. Or a base Twitter API module could be available to build upon. Can you image how much repeated code/effort has gone into the bajillion Twitter plugins currently in the directory?

comment:38 in reply to: ↑ 37 sc0ttkclark18 months ago

Major +1 on this discussion, heavily needed and +1 on @dougal's specific notes:

via dougal:

dependencies help encourage devs to collaborate on common APIs that they can all use for their own modules

comment:39 pbaylies18 months ago

  • Cc pbaylies added

comment:40 in reply to: ↑ 37 MikeSchinkel18 months ago

Replying to dougal:

Dependencies help encourage devs to collaborate on common APIs that they can all use for their own modules. ... Can you image how much repeated code/effort has gone into the bajillion Twitter plugins currently in the directory?

+1000.

comment:41 in reply to: ↑ 37 sabreuse18 months ago

Replying to dougal:

dependencies help encourage devs to collaborate on common APIs that they can all use for their own modules.... Can you image how much repeated code/effort has gone into the bajillion Twitter plugins currently in the directory?

I want this for the twitter factor alone. But on top of that, I want to see how much of a shift there can be toward looking at the whole plugin sphere as potentially interrelated (hookable, contributable...) and not just each as its own separate little world.

comment:42 alexkingorg18 months ago

  • Cc public@… added

comment:43 MikeSchinkel18 months ago

Something else to consider about dependent plugins which I'd like to pose as somewhat rhetorical questions:

  • Are plugins that themes and other plugins depend on mostly things an end-user would install independently, or are they mostly things that an end-user would likely never install on their own?
  • If the former, what are some examples? But if the latter would it make sense to treat these "depended on" plugins differently than existing plugins?

Specifically, if these "depended on" plugins are mostly API/libraries would it not make sense to consider (also) pulling them from GitHub instead of the WordPress plugin repository where end-users go to search for plugins? GitHub is emerging as the place developers post code to share and collaborate (note: I personally prefer Hg for version control, but GitHub has no peer in enabling collaboration.) The primary benefit to GitHub IMO is the fork/pull model that allows developers to more easily collaborate.

The current plugin repository makes it very difficult to collaborate (I believe that issue was discussed at the summit, directly or indirectly regarding abandoned plugins?) If API/libraries are pulled from GitHub then these emergent APIs will likely be fewer in number but much better in depth and robustness. Rather than have every developer create yet another API/library for the same purpose it will be easier to make an enhancement to an existing API/library because of forking, and easier to get the original developer to accept the enhancement because of pull requests.

Which brings up a corollary, should API/library plugins even be included in the WordPress plugin repository where end-users go looking for plugins to install?

comment:44 toscho18 months ago

  • Cc info@… added

comment:45 johnciacia18 months ago

  • Cc johnciacia added

comment:46 mzaweb18 months ago

  • Cc mzaweb added

comment:47 whiteshadow18 months ago

  • Cc whiteshadow added

comment:48 jbrinley18 months ago

  • Cc jonathanbrinley@… added

comment:49 ashfame18 months ago

  • Cc ashishsainiashfame@… added

comment:50 jleedy18 months ago

  • Cc joseph@… added

comment:51 georgestephanis18 months ago

In the scope of the discussion, are we also talking about letting themes be dependent on named plugins via a method in core? I've seen http://tgmpluginactivation.com/ fill something of a need for that already -- but I see that as a much larger use case. We could potentially abstract out a lot of functionality code from themes, and into plugins, and make them more interchangeable. And the code to do it for both plugins and themes having the dependencies should cross-pollinate nicely.

comment:52 mbijon18 months ago

  • Cc mike@… added

The workflow for TMG Plugin Activation is one I think could work for a broad pool of WordPress users. It has pre-install notices and makes all installed plugins visible to users, no hidden libs or invisible activations.

As for the plugin-proliferation that any dependency manager might expose, we already have that and worse. Having many named and shared plugins/libs seems better than having just a few plugins where each has it's own copy of jQueryUI + different lightboxes + overlapping OAuth libs. The overlapping code means even more total code is loaded and there's a greater risk of errors. Even a basic dependency manager would help resolve that.

I don't like the idea of "Provides: metabox" though. Not because of circular dependencies, but because until the WP.org repository resolves issues regarding social/pull coding and ownership transitions we'll be better served to have as little overlap as possible between plugins. If there is overlap we would need to provide a way to filter/set which of many provided libs is used by each of many different plugins.

What might be making this more complex for WordPress is that plugins are so often standalone data, logic and view bundles. That's a bit by necessity of just getting things working, but as a community we also don't recognize good, base plugins enough. An example is the Twitter plugin comment: if people want to compete with 20 other Twitter plugins on features or UX that's *great*. The more of that we have the better for finding what works. ...but if each competing plugin writes their own and/or bundles OAuth code, that's the problem I think we need to solve here. There should be a whole set of OAuth plugins, and a whole other set of Twitter/Facebook/sharing plugins that don't have their own OAuth code.

comment:53 mbijon18 months ago

It may be worth looking at the Composer JSON schema, https://github.com/composer/composer/blob/master/res/composer-schema.json

The info provided there might not all fit in our Header comments, but if the WP.org plugin repo made the rest of it available then dependency/library management in core or as a plugin would be a lot easier. It might also help non-WP.org repos to be more functional. They already exist, but tend to be hacked together and tied to the owner's own plugin "frameworks." Why not support them, but raise the bar for interop a bit?

comment:54 mrwweb17 months ago

  • Cc info@… added

In response to @MikeSchinkel, I would certainly hope that theme dependencies on plugins are included in the scope of this ticket, and I also think that's where you would see a lot of plugins that end users would install on their own. Tons of themes bundle widgets, shortcodes, custom post types, etc. that are also already offered by plugins. Getting that stuff out of themes and into plugins would be a big win for theme developers (and users).

However, it raises the issue of whether this ticket would address themes/plugins that support integration with a plugin but don't require it. I love that modularized approach that would lead to lighter-weight sites and cleaner admins, but integrating with a plugin is definitely different than depending on it.

comment:55 norcross17 months ago

  • Cc andrew@… added

re: visual overhead

is there any reason there couldn't be a 'dp-plugins' (for dependencies) similar to the mu-plugins folder for these things to be stored? Adding another tab on the top row (next to 'must use' or whatever). This will allow them to stay contained, be useful, and not overload / scare a user when seeing a bunch of new plugins.

comment:56 jltallon16 months ago

  • Cc jltallon@… added

Plugin *and* Theme dependencies:


We have found ourselves needing certain functionalities from (activated) plugins for themes to work properly (modular code), specially w.r.t. Custom Post Types: the CPT in a plugin, with some code to help use it, while the display logic is in the theme itself.

I hereby propose extending the functionality to theme headers too. Versioned dependencies are a must if we want to enable code sharing.
For a good example on what *definitively* works, look at Debian. We can use the same format, too:

-- cpt-shared-3/cpt-shared-plugin.php --

/*
 Plugin: cpt-shared
 Version: 3
*/

-- cpt-foo-plugin-1.0/foo-plugin.php --

/*
 Plugin: CPT FOO Functionality
 Version: 1.0
 Provides: cpt-foo
 Depends: cpt-shared (>= 2)
*/

-- my-super-duper-theme/functions.php --

/*
 Theme: My Super Duper Theme 1
 Depends: cpt-foo (>= 1.0)
*/

comment:57 jltallon16 months ago

Shared library / on-demand loading functionality:


I would certainly split this issue into a separate issue.

What we have done (I'm ready to share the code if wanted) is to implement our shared functionality into a series of plugins:

  • A shared "framework" plugin, which registers an autoloader and registers a base namespace where most *base* functionality resides (i.e. base and utility/shared classes)
  • Functionality-providing plugins, which depend on the framework, each providing a certain set of functionalities (usually within classes derived from the interfaces contained into the framework, so that one can mix and match components)
  • The plugin-dependencies plugin (v1.2 by Scribu + patches) to try and avoid an inconsistent mess

Hence, the "libraries" versioning here is not based on particular class versions (which has the potential of becoming messy) but of *library* versions (just like it is done with sonames)

E.g.: "library" A v1.2 provides versions x, y and z of classes Foo, Bar and Baz respectively. So another plugin depending on this shared functionality needs only "Depends: library (>= 1.2)"

comment:58 jltallon16 months ago

SUMMARY PROPOSAL:

  1. plugin-dependencies (w/ versioned dependencies and theme dependencies) into core
  1. Class Loader (we can certainly even use the UniversalClassLoader from Symfony here) into core
  1. "Library" plugins have one index.php file which just registers their (sub-)namespace with the ClassLoader. All later loading is demand-based. The declare "Provides:" and "Version:" à la Debian
  1. Functionality plugins work as usual (perfect backwards compatibility), though they can (and should, IMHO) use "libraries" when possible. They declare dependencies if at all posible, so that we can fail gracefully if pre-conditions are not met (as opposed to just WSOD'ing)
  1. Themes can optionally depend on certain functionalities to be present (via de same "Depends" mechanism). The theme chooser UI would need to be refreshed to include some feedback for this, though.

Ideally, and with the auto-loader within core, quite some of WordPress' itself can be demand-loaded: most of the admin (e.g. WP_Table/WP_List), RSS functionality, image processing, etc.
The performance boost can indeed be huge (we have benchmarked the performance gain of doing this with plugins, I expect the gain for WP itself to be even more noticeable)

Last edited 16 months ago by SergeyBiryukov (previous) (diff)

comment:59 ethitter15 months ago

  • Cc erick@… added

comment:60 coolmann14 months ago

  • Cc coolmann added

comment:61 atimmer14 months ago

  • Cc atimmermans@… added

comment:62 retlehs13 months ago

  • Cc retlehs added

comment:63 a.hoereth12 months ago

  • Cc a.hoereth@… added

comment:64 pauldewouters12 months ago

  • Cc pauldewouters@… added

comment:65 bobbingwide12 months ago

  • Cc herb@… added

I've been delivering plugins with plugin dependency logic since April 2012. I developed the "full" solution in the oik base plugin and provided lazy code in the dependent plugins.
Since there's no core API, each plugin that has a dependency needs to deliver the same basic code. I tried to develop a solution that worked during activation; but that was a nightmare. I plumped for responding to "admin_notices".

add_action( "admin_notices", "oiku_activation" );
/**
 * Implement the "admin_notices" action for oik-user
 */ 
function oiku_activation() {
  static $plugin_basename = null;
  if ( !$plugin_basename ) {
    $plugin_basename = plugin_basename(__FILE__);
    add_action( "after_plugin_row_" . $plugin_basename, __FUNCTION__ );   
    require_once( "admin/oik-activation.php" );
  }  
  $depends = "oik:2.0-alpha.0329,oik-fields:1.18.0325";
  oik_plugin_lazy_activation( __FILE__, $depends, "oik_plugin_plugin_inactive" );
}

See http://plugins.svn.wordpress.org/oik-nivo-slider/trunk/admin/oik-activation.php

This solution works for dependent plugins which are: missing, inactive, not at the required minimum version. With the right API built into core the above code could reduce to a one line filter.

Note: plugins which are dependent on others wait for the appropriate "plugin loaded" signal before attempting to use those plugins APIs.

The challenge I now face is where the missing or out-of-date dependent plugin's repository isn't wordpress.org. e.g. oik-fields. I have a similar challenge when it comes to determining which jQuery libraries are available!

comment:66 whiteshadow12 months ago

  • Cc whiteshadow removed

comment:67 whiteshadow12 months ago

  • Cc whiteshadow@… added

comment:68 jarednova9 months ago

  • Cc jarednova added
Note: See TracTickets for help on using tickets.