Make WordPress Core

Opened 9 years ago

Closed 3 years ago

Last modified 19 months ago

#32101 closed task (blessed) (fixed)

Ability to mark plugin as unmanaged

Reported by: damonganto's profile damonganto Owned by: sergeybiryukov's profile SergeyBiryukov
Milestone: 5.8 Priority: normal
Severity: major Version: 4.1.2
Component: Plugins Keywords: has-patch needs-testing needs-docs has-dev-note
Focuses: Cc:

Description

We've had a rather unpleasent incident where we used a certain plugin name for a private plugin, but then another plugin appeared under the same name in the theme directory, marking our plugin for an update. Rest assured someone clicked on the update button, replacing our plugin with some other.

Such conflicts may seem rare, but they do happen. Adding an optional, defaults to false, private field to the theme header (like npm/Bower) to disable update checking for such a plugin would be an easy fix.

Attachments (3)

32101.diff (3.5 KB) - added by markjaquith 9 years ago.
32101.2.diff (3.7 KB) - added by DrewAPicture 8 years ago.
filter_to_header_data.patch (721 bytes) - added by mweichert 8 years ago.
Allow filters to modify plugin/theme header data

Download all attachments as: .zip

Change History (105)

#1 @SergeyBiryukov
9 years ago

  • Milestone changed from Awaiting Review to WordPress.org

Related: #13928, #14179, #23318.

#2 @markjaquith
9 years ago

  • Milestone changed from WordPress.org to Awaiting Review

I'm in favor of this.

"Private" is a good name for the header.

Moving this out of the WordPress.org milestone as it will be all WordPress core changes.

#3 follow-up: @dd32
9 years ago

I'm also in favour of this, but would greatly prefer to add some kind of unique ID header instead to make matching on the WordPress.org side far more black & white.

This is probably a duplicate of #23318 for that reason

#4 in reply to: ↑ 3 @markjaquith
9 years ago

Replying to dd32:

I'm also in favour of this, but would greatly prefer to add some kind of unique ID header instead to make matching on the WordPress.org side far more black & white.

Why are these mutually exclusive? We need backwards compatible matching still, so a unique ID header would be an optional addition authors could add. The lack of a unique ID header can't mean "private". I think plugin/theme privacy should be something that people specifically opt in to.

@markjaquith
9 years ago

#5 @markjaquith
9 years ago

32101.diff is my pass at 'Private' header support for plugins and themes.

#6 @joostdevalk
9 years ago

I love the idea. I don't like the term "Private" though, because people would also use this for plugins that have other update endpoints...

#7 @dd32
9 years ago

Why are these mutually exclusive?

I don't think they're mutually exclusive, but I think one can surpass the other very quickly in usefulness.

For example, I expect that any plugin using a third party update script will set this flag too - Why? Because it's the only way for them to opt-out of updates when their plugin is disabled (or their updater code breaks.. which happens).

I agree that there are a lot of plugins on sites that are never going to have updater support and are 100% site specific; I understand that's where your thought process is here - but it just seems redundant IMHO to eventually have two fields which do somewhat of the same thing, but not really.

That all being said, I'm really still +1 for this, because we need some kind of solution for this sort of mess :)

#8 @joostdevalk
9 years ago

I fully agree with @dd32 above. Especially also on getting this in regardless because we need movement here :)

#9 follow-up: @jorbin
9 years ago

For a name, what about something verbose such as "WordPress.orgUpdates". This would simply allow plugins/themes to flag that they don't want to be checked vs WordPress.org

#10 @georgestephanis
9 years ago

Plugin repo would need an update before core, so repo plugins don't add in 'private' and get permastuck with no more updates ever.

Perhaps have repo parse 'private' as letting users remove their plugin from public view and new downloads/updates, so it never even builds zips of tags flagged as private in the tag readme?

#11 follow-up: @khromov
9 years ago

Great idea, but I echo what Joost said about plugins with custom update endpoints needing to still be able to piggyback on the update process somehow without complicating the current way it's done. (Thus breaking existing custom updaters.)

#12 in reply to: ↑ 9 @GeekStreetWP
9 years ago

Replying to jorbin:

For a name, what about something verbose such as "WordPress.orgUpdates". This would simply allow plugins/themes to flag that they don't want to be checked vs WordPress.org

make .org "_internal" and private plugins "_external"?

#13 in reply to: ↑ 11 ; follow-up: @jdgrimes
9 years ago

Replying to khromov:

Great idea, but I echo what Joost said about plugins with custom update endpoints needing to still be able to piggyback on the update process somehow without complicating the current way it's done. (Thus breaking existing custom updaters.)

A while ago a created a fork of the plugin update code for a plugin. Instead of hard-coding the handling of requests based on a single remote URL and API, I abstracted the code in such a way that it is remote API agnostic. All that is needed to receive updates from a given URL is to set the Channel header like this: Channel: wordpress.org. The channel can have whatever kind of API it wants, but for non-standard APIs you have to have the code installed that will handle them. This is similar to how custom update APIs are handled now, except that instead your custom code only needs to worry about interacting with the remote channel, not hooking into WordPress all over the place. Parts of it could probably be merged back into WordPress without breaking back-compat.

This ticket was mentioned in Slack in #core by ocean90. View the logs.


9 years ago

#15 @jb510
9 years ago

I'm a huge fan of this idea.

I'm also wondering though if this is the right avenue to lay the groundwork for core support of 3rd party update endpoints. Rather than a boolean for private, why not be explicit with registered endpoints like 'wporg', 'github', 'external'?

#16 @Rarst
9 years ago

Mark jumped on the patch so fast I hadn't had the time to collect my thoughts on at contributor day. :)

The whole updates system in WP is a very layered issue.

On a broad level WP extension ecosystem is engineered with single set of official repositories in mind. So far it resists possibility or alternate repositories technically (overreaching "push" model, inconvenient to override) and organizationally (alternate infrastructure implementations are banned from being distributed through official channels).

As an implementation detail the data about all installed extension is getting pushed to official repositories, with implications to privacy and often causing wrong updates being matched and served. In current code this behavior is extremely inconvenient to override (requires manipulating intermediary data in HTTP requests being made) and requires extension to be active to be able to.

In my opinion in scope of this ticket the Private flag makes sense to opt out extension from being subject to any data pushes. It is important that implementation as header flag will still function when extension is disabled.

Trying to "solve" the general updates situation, however, is much more involved technically and organizationally.

TL;DR let's just get this in because it's badly needed... and leave dreaming of more flexible update system for another day

#17 @juliobox
9 years ago

Hello everyone

I already got this idea earlier so i love that.

In my mind it was like that:
If no new header is provided, let the actual behavior do its job.
If the new header is provided, let's say, 'Repo' just for the demo, then hit this URL.
Yes "Repo" is a URL.
So instead of hit the w.org page, it will hit my own api endpoint.
WP will add the correct final url part like /update-check/ or /info/ when needed.
My API endpoint has to respect the guidelines for returning correct values. I'm free to send more variables like a licence string.

Thank you for reading me.

#18 @jb510
9 years ago

Also related: #25314 #28660

Still in favor of getting a quick simple patch in for this ASAP.

#19 @Ipstenu
9 years ago

I like this idea. About once every 3-5 months it comes up that someone's got a self-hosted plugin that was stomped on by a new one added to .org. The basic idea of not updating anything flagged private or 'notorg' or whatever is a good idea.

However George brings up my biggest concern (which admittedly we already have today)

Perhaps have repo parse 'private' as letting users remove their plugin from public view and new downloads/updates, so it never even builds zips of tags flagged as private in the tag readme?

The tl;dr of my thoughts is this. I don't trust the majority plugin devs to handle this properly because it's a convoluted set of priorities.

This is based solely on the volume of emails we get about 'how do tags and stable releases work?'

I don't think our repo API handles the situation today in a good way. Right now we have five states:

  • Pending approval
  • Approved by pending upload
  • Open
  • Closed
  • Closed but serving updates (no new installs)

We rarely use that last one. However I would be happier if the private flag could trigger #5, which would allow someone to close the plugin and allow for security updates.

I also have a fear that people will, stupidly, edit their plugins to add that private flag when they want to 'fork' a plugin and leave themselves in a dangerous place for security updates. While, yes, they did it to themselves, we're enabling them to shoot their own feet.

#20 @mikejolley
9 years ago

Big +1 from me. We see this all the time with our premium plugins - even with our "woocommerce-" prefix, authors on .org copy the same naming structure causing headaches in support.

#21 @Otto42
9 years ago

I like 32101.diff from @markjaquith adding the "Private" header. Not sure about the use of "true" here, but it's fine.

Regarding the suggestions after it, about allowing updates from elsewhere, I think that while those topics are related, they're separate ideas.

Allowing for updates from other APIs: #25314

  • Note that this would be something we would have to police for in the plugin and themes directory. WordPress.org directories are hosting sites, not listing sites. We don't allow code there to update from elsewhere. However, what your non-free plugin hosted from your own site does is not our concern so making that simpler is fine by me. Probably should be more than a header though, maybe a better filter or action. But that's discussion for over there.

Using unique ID headers to identify plugins and prevent mismatching: #23318

  • Big +1 to this. Almost all the work for this is in the API though, not much to do in core.

Having the private header do special things in the repos: (part of this ticket, mentioned by @ipstenu and @georgestephanis)

  • Not necessary, because we would actively block plugins and themes in the WordPress.org repository from having this header at all. It doesn't work that way, because obviously if we send a Private plugin to a system, it will never check for updates for that plugin ever again. So we'd prevent it entirely. Such a patch would be made to theme-check to prevent the theme from being uploaded, and to the plugin pre-commit scripts to prevent such a change from being committed, even by accident. Somebody accidentally makes it private, we send that ZIP to their users, they're now locked out of updates for it forever without manually editing the field.

Usefulness of this by non-free plugins:

  • High at first, less so after #23318 goes in and gets widespread.

#22 @Ipstenu
9 years ago

Re having the private header do special things in the repos:

.. we would actively block plugins and themes in the WordPress.org repository from having this header at all .. Such a patch would be made [...] to the plugin pre-commit scripts to prevent such a change from being committed, even by accident.

Awesome. That solves up my logistical worries. Go forth and API :)

#23 @gMagicScott
9 years ago

Considering that one has to choose to host their plugin in the dot org repo, upload, and have it reviewed, perhaps the header should be opt-in instead of opt-out like currently recommend.

I'm not familiar with the infrastructure and code that powers the repo, but I'd imagine that the extra header could be injected during the process of zipping the plugin for delivery. There could also be a measure in place to prevent a commit that removes the header.

With the opt-out style, in a perfect world, we are asking countless people to update all their private plugins with a new header. How much benefit is there with there still being tons of private plugins without the new header. There is some degree of control over what and how many plugins are in the official repository.

I think I saw someone mention that such a header could interfere with private plugins with a custom upgrade routine. It shouldn't be too much of an issue since that routine will never fire when the plugin is deactivated. I think improving custom update APIs would be a new ticket.

Last edited 9 years ago by gMagicScott (previous) (diff)

#24 @TJNowell
9 years ago

This patch looks good to me, +1

#25 @nhuja
9 years ago

Anything that makes life easier on support because some bad ass author just wants to use a popular theme name/plugin name to give the original author some headaches in the support. Even requests didn't help. If we can implement this, that would be one great achievement in life. ;-)

My support on this one. Please add this to core..

Last edited 9 years ago by nhuja (previous) (diff)

#26 @wonderboymusic
9 years ago

  • Keywords has-patch added
  • Milestone changed from Awaiting Review to 4.4
  • Owner set to markjaquith
  • Status changed from new to assigned

Let us try to get this in.

#27 @DrewAPicture
8 years ago

  • Keywords needs-refresh added

32101.diff needs a refresh.

@DrewAPicture
8 years ago

#28 @DrewAPicture
8 years ago

  • Keywords needs-testing added; needs-refresh removed

32101.2.diff refreshes the patch for current trunk. Will try to get some eyes on this, would be nice to get this nugget in for 4.4.

#29 @DrewAPicture
8 years ago

  • Owner changed from markjaquith to DrewAPicture
  • Status changed from assigned to accepted

#30 follow-up: @jorbin
8 years ago

Thinking about this a little more, I think some UI to ensure users know that a plugin is unmanaged is going to be needed. Users expect plugins to receive updates. And they expect the plugin screens to be where they find out about upgrades. They should know that a plugin they are using isn't upgradable in the normal tradditional sense.

When this goes in, we should create a meta ticket to have the plugin repo block any attempts at adding this header. Ideally, via a pre-commit SVN hook so it can never ever be added.

#31 in reply to: ↑ 30 @DrewAPicture
8 years ago

Replying to jorbin:

Thinking about this a little more, I think some UI to ensure users know that a plugin is unmanaged is going to be needed.

I disagree with needing a UI element. I think users are more jarred by having their third-party plugins overwritten with one from the repo when they click update.

Users expect plugins to receive updates. And they expect the plugin screens to be where they find out about upgrades. They should know that a plugin they are using isn't upgradable in the normal tradditional sense.

The onus is on third-party developers to provide an update mechanism for their plugins whether users expect them or not, and that isn't really relevant here, in my opinion. This solves a very specific and longstanding problem. We aren't changing the expectation of using the plugins screens for updates. We're proposing to not overwrite one plugin with another one just because there's a matching slug for a plugin the repo.

When this goes in, we should create a meta ticket to have the plugin repo block any attempts at adding this header. Ideally, via a pre-commit SVN hook so it can never ever be added.

Noted.

This ticket was mentioned in Slack in #meta by drew. View the logs.


8 years ago

#33 in reply to: ↑ 32 @DrewAPicture
8 years ago

Replying to slackbot:

This ticket was mentioned in Slack in #meta by drew. View the logs.

@jorbin: Conversation started, FYI

#34 @Ipstenu
8 years ago

Thinking about this a little more, I think some UI to ensure users know that a plugin is unmanaged is going to be needed.

I think Jorbin has a point, but I also think that's the scope of another ticket. "More meaningful status information on the plugins and themes pages."

Flagging a plugin as 'not managed via wp.org' would be as nice as having a way to say "This plugin hasn't been updated in over 2 years..." as a method for educating users.

However for this ticket, just getting the ability in, to stop users from the undesirable experience of "Why did my plugin get overwritten?!?!" when I have to close a plugin for shenanigans cannot be measured.

#35 @wonderboymusic
8 years ago

  • Status changed from accepted to assigned

#36 @GaryJ
8 years ago

I realise the patch has all but been committed, but instead of a Private header, how about a more descriptive Update header?

Value can be false, to opt-out of all updates, but could also be a URL that points to a custom endpoint? Lack of header effectively means a value of the WordPress.org plugin update endpoint.

#37 @knutsp
8 years ago

Update seems like a better idea as a header name. This is more extensible and future proof.

In the first iteration (4.4) this value should only accept N/No/None/F/False meaning no updates from wp.org repo.

Later this may have other values like Major/Minor/Security/Manual or whatever when new features regarding (auto) updates may be desirable and implemented. Both opt-in and opt-out. In any case this should hold information that instructs the update mechanism in core how to handle updates.

I see a future where some widely used plugins may opt-in to enable pushing security updates regardless of auto update settings (done manually today, through the security team), and some plugins to opt-out of any automatic updates (if allowed as a future decision).

Private as header has far less such future opportunities.

#38 @Otto42
8 years ago

@GaryJ: WordPress is not going to include any code to have updates come from another URL anytime soon, so having a header to specify a URL which isn't going to ever be used by core does not make any sense. If a plugin is going to update from a different endpoint, then it's going to need to include code for it to do so, and the URL can be contained in there. There's no benefit to having a URL in the header for an update check that will never be implemented.

The purpose of a "Private" header is not only to prevent updates, but also to prevent any information such as the plugin header from being sent out as part of the update check process. Privacy in these headers is a concern that has been expressed by others, and while w.org does not store this plugin header information (because we ultimately don't really care about your plugin headers), not everybody believes me when I tell them that (because people are paranoid). Having a way for them to simply not send this info for plugins they don't want to send is a simple and easy fix to add right now.

#39 @DrewAPicture
8 years ago

  • Type changed from enhancement to task (blessed)

#40 @mordauk
8 years ago

This is working great for me.

I've tested both with plugins distributed on WordPress.org and on plugins with external repositories.

#41 follow-up: @mweichert
8 years ago

I like this addition, but would it be possible to add a filter to by-pass this for specific plugins that are marked private? We use the WP update mechanism to update our plugins, and use existing WP filters to modify which url is used to both check for updates and retrieve updates.

Perhaps this could be extended by adding a filter to modify the parsed plugin/theme header data?

Adding patch.

Thanks,
Mike

@mweichert
8 years ago

Allow filters to modify plugin/theme header data

#42 in reply to: ↑ 41 ; follow-up: @Otto42
8 years ago

Replying to mweichert:

I like this addition, but would it be possible to add a filter to by-pass this for specific plugins that are marked private?

That would negate the point of the header. Additionally, it wouldn't change the behavior when the plugin is deactivated.

We use the WP update mechanism to update our plugins, and use existing WP filters to modify which url is used to both check for updates and retrieve updates.

Right, and that means that you don't need this header at all. You're already changing where the update checks go. The purpose of the Private header is to prevent updates on a plugin. Not to redirect them elsewhere.

#43 in reply to: ↑ 42 @mweichert
8 years ago

Replying to Otto42:

Right, and that means that you don't need this header at all.

That's right, I don't really need this additional header - but I can see how it would be useful for others. My point was that if this header gets added, the filters I mentioned should also be added to customize how specific "Private" plugins are managed. For instance, I have a "Private" plugin, so it probably makes sense to add the "Private" header to indicate that it's not actually available on wordpress.org - otherwise we'll be asked why a private plugin doesn't have the "Private" header. But I don't want to actually by-pass WP's mechanism of checking for updates.

Additionally, the filters I suggested are useful outside the context of this ticket.

#44 follow-up: @nvartolomei
8 years ago

The purpose of the Private header is to prevent updates on a plugin. Not to redirect them elsewhere.

To prevent updates from WordPress.org repo. We still need a way to provide updates to those 3rd party commercial plugins and themes.

#45 in reply to: ↑ 44 @damonganto
8 years ago

Replying to nvartolomei:

The purpose of the Private header is to prevent updates on a plugin. Not to redirect them elsewhere.

To prevent updates from WordPress.org repo. We still need a way to provide updates to those 3rd party commercial plugins and themes.

See comment 38. That's a different use case.

This ticket was mentioned in Slack in #core by wonderboymusic. View the logs.


8 years ago

#47 @DrewAPicture
8 years ago

  • Milestone changed from 4.4 to WordPress.org

#48 follow-up: @aspexi
8 years ago

I am trying to understand this ticket status - has this been already implemented? I'm looking at 4.4-beta4 but nothing there related to it.

If this has not been imlpemented yet, do we know when can it happen?

Apart from above, can you suggest any workaround for blocking "private" plugin from updating / replacing by other plugin from wordpress.org? I still want my plugin to check 3rd party for update though.

#49 in reply to: ↑ 48 @chriscct7
8 years ago

Replying to aspexi:

I am trying to understand this ticket status - has this been already implemented? I'm looking at 4.4-beta4 but nothing there related to it.

If this has not been imlpemented yet, do we know when can it happen?

Apart from above, can you suggest any workaround for blocking "private" plugin from updating / replacing by other plugin from wordpress.org? I still want my plugin to check 3rd party for update though.

This ticket is still open, awaiting implementation by the .org meta team. It is not tied to a particular release.

#50 @benoitchantre
8 years ago

#14179 : same issue / improvement needed for themes

This ticket was mentioned in Slack in #core by benoitchantre. View the logs.


8 years ago

#52 @Ipstenu
8 years ago

Semi-related to https://meta.trac.wordpress.org/ticket/666

Should we also have this cover not showing the "view details" link if the plugin is unmanaged? Or should that be it's own ticket?

#53 @dd32
8 years ago

Should we also have this cover not showing the "view details" link if the plugin is unmanaged? Or should that be it's own ticket?

If this is implemented so that update checks are not done for these plugins, then the WordPress.org slug will be unknown which will cause the View Details link to vanish.

#54 @swissspidy
8 years ago

#37713 was marked as a duplicate.

This ticket was mentioned in Slack in #core by sergey. View the logs.


7 years ago

This ticket was mentioned in Slack in #core by clorith. View the logs.


7 years ago

#57 @ryno267
7 years ago

this just tells me the private plugin should have had more unique prefixing. don't see the point of this. but that's just me.

#58 follow-up: @Ipstenu
7 years ago

@ryno267 The issue is that with 65,000 plugins hosted here, plus everything else, it's impossible to make every plugin have a unique slug that will never ever be taken by someone else hosted here. It's part of being good curators of the open source community :)

#59 in reply to: ↑ 58 @ryno267
7 years ago

Replying to Ipstenu:

it's impossible

@Ipstenu define "impossible"... I understand what you're saying but I have plenty of personal plugins in many client sites that have never had an issue. By using a hash of the following or just by simply having your companyname_clientname_pluginname... or randomnumberhere_pluginname. I just find it hard to believe "it's impossible to make every plugin have a unique slug that will never ever be taken by someone else hosted here".
But with all the other things to focus on in WP if this is really that important to focus on then so be it.

#60 @Clorith
7 years ago

#41214 was marked as a duplicate.

#61 @lev0
6 years ago

Has there been any progress on this? It's always been an issue. I've seen plenty of discussion and examples of preventing updates on themes & plugins, whether it's because they're private or frozen at a specific version, or whatever. All rely on the plugin (or another) being active. Unique names can be discovered, especially themes', thus be commandeered. I don't see why this is not deemed important.

From the comments presented here, and examples in other package managers, it seems pretty sensible to implement a plugin's "private" status as an update namespace. Its absence would mean be an implicit WordPress.org value for backward compatibility of auto-updates; anything else means not WP.org. The namespace could serve as an install-unique (not universal-) key for attaching hooks to do 3rd party update checks, and archive fetching, so the rest of WordPress's update system is unchanged. If nothing attaches to those unique hooks, WordPress can warn the admin that they aren't receiving the updates they should. A reserved value like "none" could be disallowed for hooks so only manual updates are allowed for those, again warning the admin. The outcome would be passive protection of plugins, and sanctioned update integration.

Surely you can find a header name that does not already exist in the WP.org repo.

#62 @SergeyBiryukov
5 years ago

#45780 was marked as a duplicate.

#63 @ocean90
4 years ago

#50034 was marked as a duplicate.

#64 @apedog
4 years ago

  • Severity changed from normal to major

I've now had a plugin unceremoniously deleted from a client's website when scheduled plugin automatic-update have run (#50034 which was marked a duplicate of this).
The plugin in question uses a custom library to update from a github repo. Which it does successfully. It is configured properly. It knows where to look when updating. On the Plugins page it does not show update information taken from wp.org repo but rather from the correct github repo.

However when this plugin is added to the automatic updates routine (using Easy Updates Manager), it results in its deletion. With a single notice appearing once:

The plugin has been deactivated due to an error: Plugin file does not exist.

Yup. The update routine failed (not the update of my plugin, which had no updates, just the twice-daily version-check routine) and the plugin just vanished. *poof*

This is actually the second time I've encountered this naming-conflict problem for a plugin of mine. In both cases I had chosen a plugin name with no apriori naming conflict. However at some later point someone else had also written a plugin with the same generic name and submitted that to the wp.org repository. From that point on my plugin's proper functioning is broken.

This is very much a backward-compatibility breaking defect. And as such (bc-breaking) this ticket's severity should be set to major IMO (higher-ups may naturally disagree and set this back to normal). Plugins magically disappearing is a bad experience. Not a "normal" experience.

A solution would be to add a new plugin/theme header field:

Plugin Repository: WordPress | Github | github.com | Private

Such a solution would also be extensible/scalable. If at any point in the future WP core will elect to support updating plugins from different sources, this will be possible.

With the upcoming auto-updates functionality being added to WordPress core, this issue needs to be addressed.
The solution suggested by some of simply disabling update checks for private plugins is at best a stop-gap measure. And its a handicap of the plugin's functionalities.
The suggestions of changing a plugin's name/slug or artificially setting a higher version are (to put it mildly) insulting.

#65 @apedog
4 years ago

Also,
My original ticket #50034 was closed as a duplicate of this - and obviously this is the same issue.

However this ticket sugges (by its title) the "Ability to mark plugin as unmanaged" which is quite the opposite of what I had intended. I very much want to manage my plugin. I simply want core to not let wp.org plugins supersede my plugin.

A decision should be made about the way forward. And if a new direction is chosen - the title of this ticket should change to reflect that.

#66 follow-up: @Ipstenu
4 years ago

As of April, plugins no longer accepts submissions of a slug that is known to exist in the wild, similar to themes. We also have a double check when we review, so for the future this is somewhat mitigated.

Existing plugins? Technically they've had this problem for all this time, and anyone impacted should have 'resolved' it one way or the other, either with a rename or hooking into the updater and slapping it down.

IF we were to add that, I would want an SVN commit blocker on anyone trying to upload something to .org with that header as anything but WordPress, in order to prevent them from accidentally breaking their plugins the other way. Or just block that header entirely.

#67 @apedog
4 years ago

  • Severity changed from major to normal

First off I'd like to admit an embarrassing fact..
The plugin slug conflict did NOT cause the deletion of my plugin on auto-update.
My plugin was configured to update from github - and wp core respected that.
What happened was that I had accidentally moved the plugin inside another plugin's directory. (I believe wp also looks one directory down the hierarchy - so it was still available on the Plugins page as it always had been). My plugin was deleted when its "parent" plugin had auto-updated.
So that's that. I'm sorry if I sent anyone chasing down this wrong rabbit hole.

As it stands, the plugin retains its association with the github repository/version so long as its activated. On deactivation it reverts to being erroneously associated with the plugin of the same name on WordPress.org and displays the notice suggesting an upgrade. ie. to be replaced by the wp.org plugin (which is at a higher version).

IF we were to add that, I would want an SVN commit blocker on anyone trying to upload something to .org with that header as anything but WordPress, in order to prevent them from accidentally breaking their plugins the other way. Or just block that header entirely.

That sounds very reasonable.

Last edited 4 years ago by apedog (previous) (diff)

#68 in reply to: ↑ 66 @jb510
4 years ago

  • Severity changed from normal to major

Replying to Ipstenu:

As of April, plugins no longer accepts submissions of a slug that is known to exist in the wild, similar to themes.

While useful against someone poisoning the ecosystem with a plugin named 'advanced-custom-fields-pro', that isn't the point of this ticket. This ticket is not about well know commercial plugins, it's about private plugins not "known to exist in the wild".

This is still needed in core as the code I used to use to disable plugin updates in private plugins has broken at least twice over the last decade. Abdicating responsibility for "all things not .org" continues to be a great frustration for many and this remains a "major" issues IMHO.

#69 @Ipstenu
4 years ago

While useful against someone poisoning the ecosystem with a plugin named 'advanced-custom-fields-pro', that isn't the point of this ticket. This ticket is not about well know commercial plugins, it's about private plugins not "known to exist in the wild".

Yeah, I'm not talking about well-known-commercial plugins either.

The bar is super low (100 users at the moment) for 'is this a plugin known to .org?' and keep in mind, by that we mean "Is it a plugin that someone has installed on a WP site that's phoned home to momma for updates?"

Which means all plugins on a site that contacts .org for any updates, which absolutely covers exactly who you're talking about.

I do actually understand the ticket, and it's always going to be 'both' here. That's why I said 'somewhat' mitigated. It's not solved by any means, but we're doing our best outside this to make it less possible to damage sites.

  1. We don't want WP.org plugins to conflict with plugins not hosted here, regardless of how many users.
  1. We don't want plugins hosted here to accidentally hurt themselves and break updates.

Basically? I don't want to hurt ANY users :) Thanks.

#70 @apedog
4 years ago

I think actually the onus shouldn't be on WP.org plugins repo. Regardless of the naming policy WP.org repo adopts.

If someone wants to upload a generic google-tag-manager utility plugin to WP.org, she shouldn't be blocked from using that generic name just because another google-tag-manager plugin already exists "in the wild".

This is a WP core issue. Not a WP.org plugins-repo issue.
The idea is to allow any plugin with any name to tell WP core: "I am not a WP.org plugin. I update from X" or "I am not a WP.org plugin. And I don't update at all. Leave me alone."

The point of adding this as a header (and not a hook) is so the plugin can communicate this to core even when deactivated.

This reduces the onus on the plugins repository to manage plugin naming conventions. And it affords plugin authors better control of their plugins. Be they publicly-available plugins (used by 100 installations) or a tailor-made plugin (used by a single website). I can write a small google-tag-manager plugin with hard-coded ID for a client and never worry about a potential future conflict when someone adds a similar plugin to the WP.org repo.

#71 @Ipstenu
4 years ago

Besides the fact that 'google-tag-manager' is disallowed for trademark reasons (unless you're Google), I disagree. Partly. I don't think the sole responsibility is (nor should be) on the plugin review team. But I do think we need to make sure we don't make things worse.

The issue isn't just if YOU, who owns apedog-tag-manager and has it installed somewhere wants to now host on .org, damn the consequences. It's what if someone else does it? A real-world example for you: End users upload other people's plugins to .org semi-regularly. I assume it's because they think it's uploading to their site, but none of those people have replied so I don't know for sure.

There's more layers to "We need to make sure plugin X in the wild doesn't get broadsided by a new plugin on .org" and having multiple checks and balances will help us prevent the damage.

The idea is to allow any plugin with any name to tell WP core: "I am not a WP.org plugin. I update from X" or "I am not a WP.org plugin. And I don't update at all. Leave me alone."

Sure. And by blocking people from using those names, we prevent someone from installing a plugin that already exists and breaking their site. (i.e. you have apedog-tag-manager as a non-org plugin AND as a .org plugin, and someone tries to install both... What happens?)

We want to prevent that too, at least as much as we can :)

We're never going to prevent every possible permutation of errors, but that's not a reason not to stop what we know we can.

#72 @apedog
4 years ago

I'm not in disagreement with WP.org plugin naming policy.
I see what you deal with on a regular basis. I'm confident your decisions in the matter are well-thought-out and balanced.

I'm only saying these are two entirely separate issues: The WP.org naming policy (which is what you're talking about), and how plugins should communicate their status to WP core (which is what I and @jb510 above trying to address).

If five years from now WP.org changes its naming policy (for whatever good reasons may arise), plugin authors today should still have a way to safeguard their plugins from accidentally being associated with a plugin hosted on WP.org. Which is the current default in core.


And on an unrelated (and slightly facetious) note:

Besides the fact that 'google-tag-manager' is disallowed for trademark reasons (unless you're Google)

Really?
I actually use this plugin. Please don't remove it. It does what it says it does.
https://wordpress.org/plugins/google-tag-manager/

#73 follow-up: @Ipstenu
4 years ago

I'm disagreeing that they're entirely separate. I think they're running in parallel right here. Agree to disagree :)

(I'm not going to defend or argue Google's trademark enforcement.)

#74 in reply to: ↑ 73 @jb510
4 years ago

Replying to Ipstenu:

I'm disagreeing that they're entirely separate. I think they're running in parallel right here. Agree to disagree :)

I _know_ your intentions are good, but I really do disagree. These issues are entirely separate.

The core issue in this ticket isn't a plugin with 100 users gotten from some other source conflicting with .org. It's that a plugin with one user, who chose to name their private custom plugin "seed-core" and is in use on one site has to jump through hoops "just in case" some day someone submits a plugin to .org with the slug "seed-core".

The cleanest simplest solution is above, add a line to the header block to declare its a private. I really don't get how that one patch takes over 5 years to get into core. It's exactly these sorts of tickets that sapped all interest I ever had in trac and core myself.

Now, again, per above, it _could_ be a simple start to something so much better that just a private flag. That header line could declare an alternative source for updates. Imagine the utility in that just for testing beta versions before release to .org? Imagine the utility in that to developers with private plugins like many of mine that are used across dozens of sites.

WP's power comes from its flexibility and extensibility, and giving alternative sources core support would be FANTASTIC help to those needing that sort of flexibility. Yes, they can write their own, but why should they when they could still make use of core tooling for secure, fail-safe, auto-updates of our private plugins which we are use across a few, or even hundreds of sites... instead of home-brewed updaters far more likely to have security and reliability issues than core's.

#75 @gregorlove
4 years ago

+1 for the "private" header. I've experienced this happen with a custom plugin we wrote named "member-directory" and also with a theme.

There needs to be an easy way built in to core for developers to tell WP "do not check wp.org for updates to this plugin/theme."

#76 @apedog
4 years ago

Can the title of this ticket/task be updated to "Ability to mark plugin as non-WP.org repo plugin"?

With the introduction of auto-updates in 5.5 the need for a plugin header marking a plugin as non-wp.org will become relevant for custom plugins that ARE managed - just not managed on wp.org.
The original wording of this ticket - to mark a plugin as "private" or "unmanaged" - no longer expresses the use of this feature correctly.

I would also suggest opening a separate ticket for how the wp.org repo handles slugs for new plugin submissions. While these issues are related, they are "perpendicular" to each other. The resolution of one does not affect the resolution of the other.

  • The need of a plugin (especially a self-managed/self-updating plugin) to self-identify as non-wp.org. Allowing custom/private/in-the-wild plugin authors to safely turn-on auto-updates for their plugins without having to worry about a future naming conflict being introduced. (this ticket)
  • Updating the naming policy of wp.org plugin repo to alleviate such naming conflicts arising for new plugins added to the repo. (separate ticket)

#77 @dougwollison
4 years ago

+1 for a "Private" header on plugins and themes, at least as a basic solution until a more versatile one like others have suggested can be properly implemented.

I recently had a scare where a custom theme on a client's site was flagged for an update because at some point in the past 8 years a theme with the same name was published to the public repo. Thankfully the client almost never bothers performing plugin/theme/core updates.

As a precaution, I ended up going through every site I ever built in the past 8 years (and still had access to) and change the version number on each custom theme/plugin to the project's date, in order to minimize the chance of that happening in the future. The operative word here is minimize, as at least one plugin (GA Google Analytics) uses the full date without periods as the version number (e.g. 20200815). At this point, I wouldn't be surpised if someone gets the idea to use unix timestamps as their version numbers.

This ticket was mentioned in Slack in #core-auto-updates by david.baumwald. View the logs.


4 years ago

#79 @lev0
3 years ago

It's really not hard to imagine a scenario where someone creates a puppet WP account, then a non-useless plugin that maliciously targets a site's custom one. It isn't only the little sites, e.g. one could use the name Ari Bloomberg to reduce suspicion when attempting to publish a plugin with the same name as one visible in the markup of https://www.bloomberg.com/professional/. After a plugin gets approved, the actual published code is not directly from that process but from the developer's SVN commits, which could include un-vetted malicious operations. That site may have measures and procedures to mitigate such an attack but not everyone does. The plugin might be removed promptly but it'd only take one update to infect a site. It's quite disappointing that it's still unreasonably difficult to prevent, considering a patch has been submitted.

#80 follow-up: @Ipstenu
3 years ago

We actually put checks in now, you cannot submit a plugin with a permalink that's used in or outside of WP.org

Also we do a little more than just say "Oh Ari Bloomberg submitted a plugin?" :) You have to actually use an email from the domain if you're claiming to be an official plugin. Not to mention we punt trademarks pretty hard these days.

Sure something could slip in, but it's a lot harder now.

Now that said, I would still prefer we have a way to say "Don't update this via .org, ever" for non-org plugins, just as a matter of course.

#81 in reply to: ↑ 80 @lev0
3 years ago

Replying to Ipstenu:

We actually put checks in now, you cannot submit a plugin with a permalink that's used in or outside of WP.org

Does this mean WP.org is logging every single install's private plugins, even if they have no front-end traces? What about intranet sites? This seems both hacky and privacy-violating to me.

Also we do a little more than just say "Oh Ari Bloomberg submitted a plugin?" :) You have to actually use an email from the domain if you're claiming to be an official plugin. Not to mention we punt trademarks pretty hard these days.

I think you've missed the point there. The account in that scenario would be trying to appear unrelated to Bloomberg the company, in which case they wouldn't be expected to have an @bloomberg address.

Sure something could slip in

And therein lies the problem.

Now that said, I would still prefer we have a way to say "Don't update this via .org, ever" for non-org plugins, just as a matter of course.

This is what we're all waiting for.

#82 @apedog
3 years ago

It seems there's a consensus for this feature.
What should this plugin header be named?
"Unmanaged" and "Private" are misnomers.
This header would apply both to plugins that are unmanaged and to plugins that are managed outside of wp.org (github updaters and the like).
This header would also be useful for public plugins published "in the wild". Not just private plugins.

#83 @knutsp
3 years ago

WordPress Repo: $value

Default $value = true (and some friends), all other values means no updates from wporg, as ignore/local/private/unmaintained/external/whatever you mean.

Use $value = filter_var( $value, FILTER_VALIDATE_BOOLEAN );
Returns bool true for '1', 'true', 'on' and 'yes', but bool false anything else.

#84 in reply to: ↑ 13 @lev0
3 years ago

The suggestion by jdgrimes make a lot of sense:

A while ago I created a fork of the plugin update code for a plugin. Instead of hard-coding the handling of requests based on a single remote URL and API, I abstracted the code in such a way that it is remote API agnostic. All that is needed to receive updates from a given URL is to set the Channel header like this: Channel: wordpress.org.

It's concise, doesn't add extra descriptive terms that are already evident from context (e.g. WordPress Repository:), and makes it extensible for future update methods. It would immediately provide a patch path for this issue with a reserved word, e.g. Channel: none. If that's not descriptive enough Update Channel: could do. Omission of the header would obviously default to whatever WP.org wishes to reserve for itself, if not the quoted suggestion above.

The downside of using terms like "Repo" or "Repository" in the header name is the implication of consistent, remote archives of code that can be utilised with read-only operations. Those terms are usually associated with package managers that have strict format and delivery standards, but it's not necessary to define how updates should be delivered.

This ticket was mentioned in Slack in #core-auto-updates by apedog. View the logs.


3 years ago

#86 follow-ups: @afragen
3 years ago

I’ve always thought something like this was plugin territory, so I wrote one.

https://github.com/afragen/skip-updates

It will skip any updates only for plugins or themes hosted on dot org.

#87 in reply to: ↑ 86 ; follow-up: @apedog
3 years ago

Replying to afragen:

I’ve always thought something like this was plugin territory, so I wrote one.

https://github.com/afragen/skip-updates

It will skip any updates only for plugins or themes hosted on dot org.


This ticket is about allowing a custom plugin to self-dis-associated from wp.org repo. Not neccessarily disabling its updates altogether. The plugin might still have its own update routine from another repo (eg. using https://github.com/afragen/github-updater). It's more about "namespacing". Allowing plugin authors to confidently create custom plugins without having to worry about a same-name plugin later being added to the wp.org repo and messing with the current plugin's functionality.

This is not plugin territory. It's very much a wp-core issue.

#88 in reply to: ↑ 86 ; follow-ups: @lev0
3 years ago

Replying to afragen:

I’ve always thought something like this was plugin territory

Thank you for illustrating this problem perfectly. Your plugin isn't taken. If your plugin doesn't protect itself, someone named "Skip Jones" could create it and have it do something else update-y. Maybe no name-based shenanigans are required and it could simply skip updates outside of business hours so that someone would be on the clock in case anything needs fixing. Now every plugin and theme that was protected by your plugin is trashed. Maybe your plugin does protect itself but someone disabled it, or it failed for some other reason, and both it and the other plugins get wiped out.

Last edited 3 years ago by lev0 (previous) (diff)

#89 in reply to: ↑ 87 ; follow-up: @afragen
3 years ago

Replying to apedog:

Replying to afragen:

I’ve always thought something like this was plugin territory, so I wrote one.

https://github.com/afragen/skip-updates

It will skip any updates only for plugins or themes hosted on dot org.


This ticket is about allowing a custom plugin to self-dis-associated from wp.org repo. Not neccessarily disabling its updates altogether. The plugin might still have its own update routine from another repo (eg. using https://github.com/afragen/github-updater). It's more about "namespacing". Allowing plugin authors to confidently create custom plugins without having to worry about a same-name plugin later being added to the wp.org repo and messing with the current plugin's functionality.

This is not plugin territory. It's very much a wp-core issue.

All this plugin does is dis-associate other plugins or themes from dot org.

It really does work flawlessly with GitHub Updater or likely any other custom updater.

#90 in reply to: ↑ 88 @afragen
3 years ago

Replying to lev0:

Replying to afragen:

I’ve always thought something like this was plugin territory

Thank you for illustrating this problem perfectly. Your plugin isn't taken. If your plugin doesn't protect itself, someone named "Skip Jones" could create it and have it do something else update-y. Maybe no name-based shenanigans are required and it could simply skip updates outside of business hours so that someone would be on the clock in case anything needs fixing. Now every plugin and theme that was protected by your plugin is trashed. Maybe your plugin does protect itself but someone disabled it, or it failed for some other reason, and both it and the other plugins get wiped out.

I guess all I would need to protect myself and anyone else who wants to use this as a solution is to submit it to the plugin repository. 😉

I’ll do that in the morning.

#91 in reply to: ↑ 89 @apedog
3 years ago

Replying to afragen:

All this plugin does is dis-associate other plugins or themes from dot org.

It really does work flawlessly with GitHub Updater or likely any other custom updater.

That's good. I couldn't tell from the title (or the README). "Skip Updates" might be a bit of a misnomer.

Nontheless, as per @lev0's comment above - this should be in core. Not dependant on plugins. Especially as plugins can be disabled during normal operation for whatever reasons.

Last edited 3 years ago by apedog (previous) (diff)

#92 in reply to: ↑ 88 @afragen
3 years ago

Problem solved. https://wordpress.org/plugins/skip-updates/

Replying to lev0:

Replying to afragen:

I’ve always thought something like this was plugin territory

Thank you for illustrating this problem perfectly. Your plugin isn't taken. If your plugin doesn't protect itself, someone named "Skip Jones" could create it and have it do something else update-y. Maybe no name-based shenanigans are required and it could simply skip updates outside of business hours so that someone would be on the clock in case anything needs fixing. Now every plugin and theme that was protected by your plugin is trashed. Maybe your plugin does protect itself but someone disabled it, or it failed for some other reason, and both it and the other plugins get wiped out.

#93 @SergeyBiryukov
3 years ago

In 50921:

Plugins: Add support for Update URI header.

This allows third-party plugins to avoid accidentally being overwritten with an update of a plugin of a similar name from the WordPress.org Plugin Directory.

Additionally, introduce the update_plugins_{$hostname} filter, which third-party plugins can use to offer updates for a given hostname.

If set, the Update URI header field should be a URI and have a unique hostname.

Some examples include:

  • https://wordpress.org/plugins/example-plugin/
  • https://example.com/my-plugin/
  • my-custom-plugin-name

Update URI: false also works, and unless there is code handling the false hostname, the plugin will never get an update notification.

If the header is present, the WordPress.org API will currently only return updates for the plugin if it matches the following format:

  • https://wordpress.org/plugins/{$slug}/
  • w.org/plugin/{$slug}

If the header has any other value, the API will not return a result and will ignore the plugin for update purposes.

Props dd32, DavidAnderson, meloniq, markjaquith, DrewAPicture, mweichert, design_dolphin, filosofo, sean212, nhuja, JeroenReumkens, infolu, dingdang, joyously, earnjam, williampatton, grapplerulrich, markparnell, apedog, afragen, miqrogroove, rmccue, crazycoders, jdgrimes, damonganto, joostdevalk, jorbin, georgestephanis, khromov, GeekStreetWP, jb510, Rarst, juliobox, Ipstenu, mikejolley, Otto42, gMagicScott, TJNowell, GaryJ, knutsp, mordauk, nvartolomei, aspexi, chriscct7, benoitchantre, ryno267, lev0, gregorlove, dougwollison, SergeyBiryukov.
See #14179, #23318, #32101.

#94 @SergeyBiryukov
3 years ago

  • Milestone changed from WordPress.org to 5.8
  • Owner changed from DrewAPicture to SergeyBiryukov
  • Status changed from assigned to accepted

#95 @SergeyBiryukov
3 years ago

  • Keywords needs-dev-note added

This ticket was mentioned in Slack in #core by sergey. View the logs.


3 years ago

#97 @audrasjb
3 years ago

  • Keywords needs-docs added

This ticket was mentioned in Slack in #core by sergey. View the logs.


3 years ago

#99 @SergeyBiryukov
3 years ago

  • Resolution set to fixed
  • Status changed from accepted to closed

Let's call this one fixed for 5.8 in [50921]. Please create new tickets for any follow-up issues.

See #14179 for themes, and #23318 for using unique ID headers to identify plugins and prevent mismatching.

Thanks everyone!

This ticket was mentioned in Slack in #core by sergey. View the logs.


3 years ago

#102 @SergeyBiryukov
19 months ago

In 53933:

Themes: Add support for Update URI header.

This allows third-party themes to avoid accidentally being overwritten with an update of a theme of a similar name from the WordPress.org Theme Directory.

Additionally, introduce the update_themes_{$hostname} filter, which third-party themes can use to offer updates for a given hostname.

If set, the Update URI header field should be a URI and have a unique hostname.

Some examples include:

  • https://wordpress.org/themes/example-theme/
  • https://example.com/my-theme/
  • my-custom-theme-name

Update URI: false also works, and unless there is code handling the false hostname, the theme will never get an update notification.

If the header is present, the WordPress.org API will currently only return updates for the theme if it matches the following format:

  • https://wordpress.org/themes/{$slug}/
  • w.org/theme/{$slug}

If the header has any other value, the API will not return a result and will ignore the theme for update purposes.

Follow-up to [50921].

Props dd32, meloniq, costdev, audrasjb, DavidAnderson, markjaquith, DrewAPicture, mweichert, design_dolphin, filosofo, sean212, nhuja, JeroenReumkens, infolu, dingdang, joyously, earnjam, williampatton, grapplerulrich, markparnell, apedog, afragen, miqrogroove, rmccue, crazycoders, jdgrimes, damonganto, joostdevalk, jorbin, georgestephanis, khromov, GeekStreetWP, jb510, Rarst, juliobox, Ipstenu, mikejolley, Otto42, gMagicScott, TJNowell, GaryJ, knutsp, mordauk, nvartolomei, aspexi, chriscct7, benoitchantre, ryno267, lev0, gregorlove, dougwollison, leemon, SergeyBiryukov.
See #14179, #23318, #32101.

Note: See TracTickets for help on using tickets.