WordPress.org

Make WordPress Core

Opened 3 years ago

Last modified 11 months ago

#32101 assigned task (blessed)

Ability to mark plugin as unmanaged

Reported by: damonganto Owned by: DrewAPicture
Milestone: WordPress.org Priority: normal
Severity: normal Version: 4.1.2
Component: Plugins Keywords: has-patch needs-testing
Focuses: Cc:

Description

We've had a rather unpleasent incident where we used a certain plugin name for a private plugin, but then another plugin appeared under the same name in the theme directory, marking our plugin for an update. Rest assured someone clicked on the update button, replacing our plugin with some other.

Such conflicts may seem rare, but they do happen. Adding an optional, defaults to false, private field to the theme header (like npm/Bower) to disable update checking for such a plugin would be an easy fix.

Attachments (3)

32101.diff (3.5 KB) - added by markjaquith 3 years ago.
32101.2.diff (3.7 KB) - added by DrewAPicture 3 years ago.
filter_to_header_data.patch (721 bytes) - added by mweichert 3 years ago.
Allow filters to modify plugin/theme header data

Download all attachments as: .zip

Change History (63)

#1 @SergeyBiryukov
3 years ago

  • Milestone changed from Awaiting Review to WordPress.org

Related: #13928, #14179, #23318.

#2 @markjaquith
3 years ago

  • Milestone changed from WordPress.org to Awaiting Review

I'm in favor of this.

"Private" is a good name for the header.

Moving this out of the WordPress.org milestone as it will be all WordPress core changes.

#3 follow-up: @dd32
3 years ago

I'm also in favour of this, but would greatly prefer to add some kind of unique ID header instead to make matching on the WordPress.org side far more black & white.

This is probably a duplicate of #23318 for that reason

#4 in reply to: ↑ 3 @markjaquith
3 years ago

Replying to dd32:

I'm also in favour of this, but would greatly prefer to add some kind of unique ID header instead to make matching on the WordPress.org side far more black & white.

Why are these mutually exclusive? We need backwards compatible matching still, so a unique ID header would be an optional addition authors could add. The lack of a unique ID header can't mean "private". I think plugin/theme privacy should be something that people specifically opt in to.

@markjaquith
3 years ago

#5 @markjaquith
3 years ago

32101.diff is my pass at 'Private' header support for plugins and themes.

#6 @joostdevalk
3 years ago

I love the idea. I don't like the term "Private" though, because people would also use this for plugins that have other update endpoints...

#7 @dd32
3 years ago

Why are these mutually exclusive?

I don't think they're mutually exclusive, but I think one can surpass the other very quickly in usefulness.

For example, I expect that any plugin using a third party update script will set this flag too - Why? Because it's the only way for them to opt-out of updates when their plugin is disabled (or their updater code breaks.. which happens).

I agree that there are a lot of plugins on sites that are never going to have updater support and are 100% site specific; I understand that's where your thought process is here - but it just seems redundant IMHO to eventually have two fields which do somewhat of the same thing, but not really.

That all being said, I'm really still +1 for this, because we need some kind of solution for this sort of mess :)

#8 @joostdevalk
3 years ago

I fully agree with @dd32 above. Especially also on getting this in regardless because we need movement here :)

#9 follow-up: @jorbin
3 years ago

For a name, what about something verbose such as "WordPress.orgUpdates". This would simply allow plugins/themes to flag that they don't want to be checked vs WordPress.org

#10 @georgestephanis
3 years ago

Plugin repo would need an update before core, so repo plugins don't add in 'private' and get permastuck with no more updates ever.

Perhaps have repo parse 'private' as letting users remove their plugin from public view and new downloads/updates, so it never even builds zips of tags flagged as private in the tag readme?

#11 follow-up: @khromov
3 years ago

Great idea, but I echo what Joost said about plugins with custom update endpoints needing to still be able to piggyback on the update process somehow without complicating the current way it's done. (Thus breaking existing custom updaters.)

#12 in reply to: ↑ 9 @GeekStreetWP
3 years ago

Replying to jorbin:

For a name, what about something verbose such as "WordPress.orgUpdates". This would simply allow plugins/themes to flag that they don't want to be checked vs WordPress.org

make .org "_internal" and private plugins "_external"?

#13 in reply to: ↑ 11 @jdgrimes
3 years ago

Replying to khromov:

Great idea, but I echo what Joost said about plugins with custom update endpoints needing to still be able to piggyback on the update process somehow without complicating the current way it's done. (Thus breaking existing custom updaters.)

A while ago a created a fork of the plugin update code for a plugin. Instead of hard-coding the handling of requests based on a single remote URL and API, I abstracted the code in such a way that it is remote API agnostic. All that is needed to receive updates from a given URL is to set the Channel header like this: Channel: wordpress.org. The channel can have whatever kind of API it wants, but for non-standard APIs you have to have the code installed that will handle them. This is similar to how custom update APIs are handled now, except that instead your custom code only needs to worry about interacting with the remote channel, not hooking into WordPress all over the place. Parts of it could probably be merged back into WordPress without breaking back-compat.

This ticket was mentioned in Slack in #core by ocean90. View the logs.


3 years ago

#15 @jb510
3 years ago

I'm a huge fan of this idea.

I'm also wondering though if this is the right avenue to lay the groundwork for core support of 3rd party update endpoints. Rather than a boolean for private, why not be explicit with registered endpoints like 'wporg', 'github', 'external'?

#16 @Rarst
3 years ago

Mark jumped on the patch so fast I hadn't had the time to collect my thoughts on at contributor day. :)

The whole updates system in WP is a very layered issue.

On a broad level WP extension ecosystem is engineered with single set of official repositories in mind. So far it resists possibility or alternate repositories technically (overreaching "push" model, inconvenient to override) and organizationally (alternate infrastructure implementations are banned from being distributed through official channels).

As an implementation detail the data about all installed extension is getting pushed to official repositories, with implications to privacy and often causing wrong updates being matched and served. In current code this behavior is extremely inconvenient to override (requires manipulating intermediary data in HTTP requests being made) and requires extension to be active to be able to.

In my opinion in scope of this ticket the Private flag makes sense to opt out extension from being subject to any data pushes. It is important that implementation as header flag will still function when extension is disabled.

Trying to "solve" the general updates situation, however, is much more involved technically and organizationally.

TL;DR let's just get this in because it's badly needed... and leave dreaming of more flexible update system for another day

#17 @juliobox
3 years ago

Hello everyone

I already got this idea earlier so i love that.

In my mind it was like that: If no new header is provided, let the actual behavior do its job. If the new header is provided, let's say, 'Repo' just for the demo, then hit this URL. Yes "Repo" is a URL. So instead of hit the w.org page, it will hit my own api endpoint. WP will add the correct final url part like /update-check/ or /info/ when needed. My API endpoint has to respect the guidelines for returning correct values. I'm free to send more variables like a licence string.

Thank you for reading me.

#18 @jb510
3 years ago

Also related: #25314 #28660

Still in favor of getting a quick simple patch in for this ASAP.

#19 @Ipstenu
3 years ago

I like this idea. About once every 3-5 months it comes up that someone's got a self-hosted plugin that was stomped on by a new one added to .org. The basic idea of not updating anything flagged private or 'notorg' or whatever is a good idea.

However George brings up my biggest concern (which admittedly we already have today)

Perhaps have repo parse 'private' as letting users remove their plugin from public view and new downloads/updates, so it never even builds zips of tags flagged as private in the tag readme?

The tl;dr of my thoughts is this. I don't trust the majority plugin devs to handle this properly because it's a convoluted set of priorities.

This is based solely on the volume of emails we get about 'how do tags and stable releases work?'

I don't think our repo API handles the situation today in a good way. Right now we have five states:

  • Pending approval
  • Approved by pending upload
  • Open
  • Closed
  • Closed but serving updates (no new installs)

We rarely use that last one. However I would be happier if the private flag could trigger #5, which would allow someone to close the plugin and allow for security updates.

I also have a fear that people will, stupidly, edit their plugins to add that private flag when they want to 'fork' a plugin and leave themselves in a dangerous place for security updates. While, yes, they did it to themselves, we're enabling them to shoot their own feet.

#20 @mikejolley
3 years ago

Big +1 from me. We see this all the time with our premium plugins - even with our "woocommerce-" prefix, authors on .org copy the same naming structure causing headaches in support.

#21 @Otto42
3 years ago

I like 32101.diff from @markjaquith adding the "Private" header. Not sure about the use of "true" here, but it's fine.

Regarding the suggestions after it, about allowing updates from elsewhere, I think that while those topics are related, they're separate ideas.

Allowing for updates from other APIs: #25314

  • Note that this would be something we would have to police for in the plugin and themes directory. WordPress.org directories are hosting sites, not listing sites. We don't allow code there to update from elsewhere. However, what your non-free plugin hosted from your own site does is not our concern so making that simpler is fine by me. Probably should be more than a header though, maybe a better filter or action. But that's discussion for over there.

Using unique ID headers to identify plugins and prevent mismatching: #23318

  • Big +1 to this. Almost all the work for this is in the API though, not much to do in core.

Having the private header do special things in the repos: (part of this ticket, mentioned by @ipstenu and @georgestephanis)

  • Not necessary, because we would actively block plugins and themes in the WordPress.org repository from having this header at all. It doesn't work that way, because obviously if we send a Private plugin to a system, it will never check for updates for that plugin ever again. So we'd prevent it entirely. Such a patch would be made to theme-check to prevent the theme from being uploaded, and to the plugin pre-commit scripts to prevent such a change from being committed, even by accident. Somebody accidentally makes it private, we send that ZIP to their users, they're now locked out of updates for it forever without manually editing the field.

Usefulness of this by non-free plugins:

  • High at first, less so after #23318 goes in and gets widespread.

#22 @Ipstenu
3 years ago

Re having the private header do special things in the repos:

.. we would actively block plugins and themes in the WordPress.org repository from having this header at all .. Such a patch would be made [...] to the plugin pre-commit scripts to prevent such a change from being committed, even by accident.

Awesome. That solves up my logistical worries. Go forth and API :)

#23 @gMagicScott
3 years ago

Considering that one has to choose to host their plugin in the dot org repo, upload, and have it reviewed, perhaps the header should be opt-in instead of opt-out like currently recommend.

I'm not familiar with the infrastructure and code that powers the repo, but I'd imagine that the extra header could be injected during the process of zipping the plugin for delivery. There could also be a measure in place to prevent a commit that removes the header.

With the opt-out style, in a perfect world, we are asking countless people to update all their private plugins with a new header. How much benefit is there with there still being tons of private plugins without the new header. There is some degree of control over what and how many plugins are in the official repository. I think I saw someone mention that such a header could interfere with private plugins with a custom upgrade routine. It shouldn't be too much of an issue since that routine will never fire when the plugin is deactivated. I think improving custom update APIs would be a new ticket.

Last edited 3 years ago by gMagicScott (previous) (diff)

#24 @TJNowell
3 years ago

This patch looks good to me, +1

#25 @nhuja
3 years ago

Anything that makes life easier on support because some bad ass author just wants to use a popular theme name/plugin name to give the original author some headaches in the support. Even requests didn't help. If we can implement this, that would be one great achievement in life. ;-)

My support on this one. Please add this to core..

Last edited 3 years ago by nhuja (previous) (diff)

#26 @wonderboymusic
3 years ago

  • Keywords has-patch added
  • Milestone changed from Awaiting Review to 4.4
  • Owner set to markjaquith
  • Status changed from new to assigned

Let us try to get this in.

#27 @DrewAPicture
3 years ago

  • Keywords needs-refresh added

32101.diff needs a refresh.

@DrewAPicture
3 years ago

#28 @DrewAPicture
3 years ago

  • Keywords needs-testing added; needs-refresh removed

32101.2.diff refreshes the patch for current trunk. Will try to get some eyes on this, would be nice to get this nugget in for 4.4.

#29 @DrewAPicture
3 years ago

  • Owner changed from markjaquith to DrewAPicture
  • Status changed from assigned to accepted

#30 follow-up: @jorbin
3 years ago

Thinking about this a little more, I think some UI to ensure users know that a plugin is unmanaged is going to be needed. Users expect plugins to receive updates. And they expect the plugin screens to be where they find out about upgrades. They should know that a plugin they are using isn't upgradable in the normal tradditional sense.

When this goes in, we should create a meta ticket to have the plugin repo block any attempts at adding this header. Ideally, via a pre-commit SVN hook so it can never ever be added.

#31 in reply to: ↑ 30 @DrewAPicture
3 years ago

Replying to jorbin:

Thinking about this a little more, I think some UI to ensure users know that a plugin is unmanaged is going to be needed.

I disagree with needing a UI element. I think users are more jarred by having their third-party plugins overwritten with one from the repo when they click update.

Users expect plugins to receive updates. And they expect the plugin screens to be where they find out about upgrades. They should know that a plugin they are using isn't upgradable in the normal tradditional sense.

The onus is on third-party developers to provide an update mechanism for their plugins whether users expect them or not, and that isn't really relevant here, in my opinion. This solves a very specific and longstanding problem. We aren't changing the expectation of using the plugins screens for updates. We're proposing to not overwrite one plugin with another one just because there's a matching slug for a plugin the repo.

When this goes in, we should create a meta ticket to have the plugin repo block any attempts at adding this header. Ideally, via a pre-commit SVN hook so it can never ever be added.

Noted.

This ticket was mentioned in Slack in #meta by drew. View the logs.


3 years ago

#33 in reply to: ↑ 32 @DrewAPicture
3 years ago

Replying to slackbot:

This ticket was mentioned in Slack in #meta by drew. View the logs.

@jorbin: Conversation started, FYI

#34 @Ipstenu
3 years ago

Thinking about this a little more, I think some UI to ensure users know that a plugin is unmanaged is going to be needed.

I think Jorbin has a point, but I also think that's the scope of another ticket. "More meaningful status information on the plugins and themes pages."

Flagging a plugin as 'not managed via wp.org' would be as nice as having a way to say "This plugin hasn't been updated in over 2 years..." as a method for educating users.

However for this ticket, just getting the ability in, to stop users from the undesirable experience of "Why did my plugin get overwritten?!?!" when I have to close a plugin for shenanigans cannot be measured.

#35 @wonderboymusic
3 years ago

  • Status changed from accepted to assigned

#36 @GaryJ
3 years ago

I realise the patch has all but been committed, but instead of a Private header, how about a more descriptive Update header?

Value can be false, to opt-out of all updates, but could also be a URL that points to a custom endpoint? Lack of header effectively means a value of the WordPress.org plugin update endpoint.

#37 @knutsp
3 years ago

Update seems like a better idea as a header name. This is more extensible and future proof.

In the first iteration (4.4) this value should only accept N/No/None/F/False meaning no updates from wp.org repo.

Later this may have other values like Major/Minor/Security/Manual or whatever when new features regarding (auto) updates may be desirable and implemented. Both opt-in and opt-out. In any case this should hold information that instructs the update mechanism in core how to handle updates.

I see a future where some widely used plugins may opt-in to enable pushing security updates regardless of auto update settings (done manually today, through the security team), and some plugins to opt-out of any automatic updates (if allowed as a future decision).

Private as header has far less such future opportunities.

#38 @Otto42
3 years ago

@GaryJ: WordPress is not going to include any code to have updates come from another URL anytime soon, so having a header to specify a URL which isn't going to ever be used by core does not make any sense. If a plugin is going to update from a different endpoint, then it's going to need to include code for it to do so, and the URL can be contained in there. There's no benefit to having a URL in the header for an update check that will never be implemented.

The purpose of a "Private" header is not only to prevent updates, but also to prevent any information such as the plugin header from being sent out as part of the update check process. Privacy in these headers is a concern that has been expressed by others, and while w.org does not store this plugin header information (because we ultimately don't really care about your plugin headers), not everybody believes me when I tell them that (because people are paranoid). Having a way for them to simply not send this info for plugins they don't want to send is a simple and easy fix to add right now.

#39 @DrewAPicture
3 years ago

  • Type changed from enhancement to task (blessed)

#40 @mordauk
3 years ago

This is working great for me.

I've tested both with plugins distributed on WordPress.org and on plugins with external repositories.

#41 follow-up: @mweichert
3 years ago

I like this addition, but would it be possible to add a filter to by-pass this for specific plugins that are marked private? We use the WP update mechanism to update our plugins, and use existing WP filters to modify which url is used to both check for updates and retrieve updates.

Perhaps this could be extended by adding a filter to modify the parsed plugin/theme header data?

Adding patch.

Thanks, Mike

@mweichert
3 years ago

Allow filters to modify plugin/theme header data

#42 in reply to: ↑ 41 ; follow-up: @Otto42
3 years ago

Replying to mweichert:

I like this addition, but would it be possible to add a filter to by-pass this for specific plugins that are marked private?

That would negate the point of the header. Additionally, it wouldn't change the behavior when the plugin is deactivated.

We use the WP update mechanism to update our plugins, and use existing WP filters to modify which url is used to both check for updates and retrieve updates.

Right, and that means that you don't need this header at all. You're already changing where the update checks go. The purpose of the Private header is to prevent updates on a plugin. Not to redirect them elsewhere.

#43 in reply to: ↑ 42 @mweichert
3 years ago

Replying to Otto42:

Right, and that means that you don't need this header at all.

That's right, I don't really need this additional header - but I can see how it would be useful for others. My point was that if this header gets added, the filters I mentioned should also be added to customize how specific "Private" plugins are managed. For instance, I have a "Private" plugin, so it probably makes sense to add the "Private" header to indicate that it's not actually available on wordpress.org - otherwise we'll be asked why a private plugin doesn't have the "Private" header. But I don't want to actually by-pass WP's mechanism of checking for updates.

Additionally, the filters I suggested are useful outside the context of this ticket.

#44 follow-up: @nvartolomei
3 years ago

The purpose of the Private header is to prevent updates on a plugin. Not to redirect them elsewhere.

To prevent updates from WordPress.org repo. We still need a way to provide updates to those 3rd party commercial plugins and themes.

#45 in reply to: ↑ 44 @damonganto
3 years ago

Replying to nvartolomei:

The purpose of the Private header is to prevent updates on a plugin. Not to redirect them elsewhere.

To prevent updates from WordPress.org repo. We still need a way to provide updates to those 3rd party commercial plugins and themes.

See comment 38. That's a different use case.

This ticket was mentioned in Slack in #core by wonderboymusic. View the logs.


3 years ago

#47 @DrewAPicture
3 years ago

  • Milestone changed from 4.4 to WordPress.org

#48 follow-up: @aspexi
3 years ago

I am trying to understand this ticket status - has this been already implemented? I'm looking at 4.4-beta4 but nothing there related to it.

If this has not been imlpemented yet, do we know when can it happen?

Apart from above, can you suggest any workaround for blocking "private" plugin from updating / replacing by other plugin from wordpress.org? I still want my plugin to check 3rd party for update though.

#49 in reply to: ↑ 48 @chriscct7
3 years ago

Replying to aspexi:

I am trying to understand this ticket status - has this been already implemented? I'm looking at 4.4-beta4 but nothing there related to it.

If this has not been imlpemented yet, do we know when can it happen?

Apart from above, can you suggest any workaround for blocking "private" plugin from updating / replacing by other plugin from wordpress.org? I still want my plugin to check 3rd party for update though.

This ticket is still open, awaiting implementation by the .org meta team. It is not tied to a particular release.

#50 @benoitchantre
2 years ago

#14179 : same issue / improvement needed for themes

This ticket was mentioned in Slack in #core by benoitchantre. View the logs.


2 years ago

#52 @Ipstenu
2 years ago

Semi-related to https://meta.trac.wordpress.org/ticket/666

Should we also have this cover not showing the "view details" link if the plugin is unmanaged? Or should that be it's own ticket?

#53 @dd32
2 years ago

Should we also have this cover not showing the "view details" link if the plugin is unmanaged? Or should that be it's own ticket?

If this is implemented so that update checks are not done for these plugins, then the WordPress.org slug will be unknown which will cause the View Details link to vanish.

#54 @swissspidy
21 months ago

#37713 was marked as a duplicate.

This ticket was mentioned in Slack in #core by sergey. View the logs.


20 months ago

This ticket was mentioned in Slack in #core by clorith. View the logs.


12 months ago

#57 @ryno267
11 months ago

this just tells me the private plugin should have had more unique prefixing. don't see the point of this. but that's just me.

#58 follow-up: @Ipstenu
11 months ago

@ryno267 The issue is that with 65,000 plugins hosted here, plus everything else, it's impossible to make every plugin have a unique slug that will never ever be taken by someone else hosted here. It's part of being good curators of the open source community :)

#59 in reply to: ↑ 58 @ryno267
11 months ago

Replying to Ipstenu:

it's impossible

@Ipstenu define "impossible"... I understand what you're saying but I have plenty of personal plugins in many client sites that have never had an issue. By using a hash of the following or just by simply having your companyname_clientname_pluginname... or randomnumberhere_pluginname. I just find it hard to believe "it's impossible to make every plugin have a unique slug that will never ever be taken by someone else hosted here". But with all the other things to focus on in WP if this is really that important to focus on then so be it.

#60 @Clorith
11 months ago

#41214 was marked as a duplicate.

Note: See TracTickets for help on using tickets.