WordPress Update Notification

[mdawaffe]

I swear there's a ticket for this already, but I can't find it.

Without grabbing any extra information from wordpress.org, WordPress can be made to determine if a user's version of WordPress is out of date given the following assumption: That the WordPress Development Blog is running the current stable version of WordPress (this may be a bad assumption - as of now it is running an alpha version).

Adding a nicer (markup/CSS) version of

$latest_version = parse_url($rss->channel['generator']);
parse_str($latest_version['query'], $latest_version);
$latest_version = (float) $latest_version['v'];
if ( -1 == version_compare(get_bloginfo('version'), $latest_version) ) 
        echo 'A newer version of WordPress is available.  Please upgrade.';

below the dev blog rss fetch would do the trick (type casting as float may or may not be deemed necessary).

wordpress.org could even be set up to make clever links in the above upgrade notice easy. ​http://wordpress.org/development/version/$version_number (or something similar) could point to that release's permalink.

[westi]

The ​Wordpress Version Check plugin can help achieve this - the source for the simple xml-rpc webservice can be provided for wp.org hosting if required.

[skippy]

I think this is a great re-use of functionality; provided we can get the devblog to stick to the latest stable release. Let the dev's personal sites run whatever version they want, but keep the official site running the official release.

[westi]

1st pass at a patch for svn head

[westi]

Improved patch with new message text

[westi]

This possibly needs updating for 2.1.

I will have a look once the webservices return.

They seem to have been lost in the new wordpress.org redesign.

[Nazgul]

Westi, is this still feasable for 2.1 or should we bump it to 2.2?

[westi]

This one needs a good refresh removing has-patch

[rob1n]

I'm kicking some ideas around. And I'll talk to Matt about some of them.

[westi]

I do have some ideas for this.

My basic idea was to refresh the patch as follows:

1. Use ajax to call the checking code from the dashboad
2. Cache the result and check every n minutes, where n probably = 15
3. Use REST to get the info from a specific url ecoding the version number into the url

My feeling for the returned info would be to allow a message and a severity to be returned. Preferably with one call to the REST service.

The severity would be used to modify how the info was displayed so as to hilight a more severe issue such as an easily exploitable vulnerability.

I would have been working on this but have been snowed under at work :-(

[rob1n]

That's what I was thinking (and 15 minutes is waaay too frequent... maybe 6 or 12 hours). A REST interface would be great. Apparently rpc.wordpress.org existed at one point in time, but it is no more. If you want to take this, it's all yours :).

[westi]

Replying to rob1n:

That's what I was thinking (and 15 minutes is waaay too frequent... maybe 6 or 12
hours). A REST interface would be great. Apparently rpc.wordpress.org existed at one
point in time, but it is no more. If you want to take this, it's all yours :).

Yes the rpc.wordpress.org did exist before the redesign although I'm not sure we'll go with the exact data we had before.

As for the refresh interval it should be noted that the 15minutes is only applicable to dashboard page loads if we put the message there or at worse admin page loads.

15 minutes has worked well with my Version Check plugin without any issues my end with load.

I would expect to use the same interval used for the dashboard feeds.

[rob1n]

Sounds good.

[rob1n]

A "script."

• WP: Load Dashboard, makes AJAX call to rpc.wordpress.org (or whatever). Could be run through another local PHP file to cache it in the DB, but I don't think it's needed. The request has GET or POST variables (probably GET) such as current, which has the current version.
• RPC: Compare $_GETcurrent? with the current version, most likely using PHP's version_compare() (if we do the RPC in PHP). If it's fine, then return an XML response with <status>0</status> or something. If it's not up to date, then <status>1</status> <latest>2.3.1</latest> <type>critical/major/minor</type>.
• WP: Use JS magic to parse the XML response (I know this is doable somehow). Based on what's returned, show an i18ned message in the Dashboard alerting the user to such, and provide a download link to ​http://wp.org/latest.tar.gz and ​http://wp.org/latest.zip

I'll work on this in the next few days.

[majelbstoat]

If you're going to parse a response in JS, you might want to consider JSON as your transfer format. It is possible to parse XML, but far easier to parse JSON. (One line in fact, if you just eval it). If security is a concern, I'm sure jQuery has JSON parsing functionality - Prototype certainly does. And if the WordPress server is running PHP 5.2, there is a built-in JSON extension for creating it as well...

[rob1n]

Most likely a bounce through a local PHP file. JSON it is. jQuery has great JSON stuff.

[westi]

Replying to rob1n:

A "script."

• WP: Load Dashboard, makes AJAX call to rpc.wordpress.org (or whatever). Could be run through another local PHP file to cache it in the DB, but I don't think it's needed. The request has GET or POST variables (probably GET) such as current, which has the current version.

I think AJAX to local php file is better to allow check to be cached as we don't want every WordPress install hitting rpc.wordpress.org for every dashboard load.

• RPC: Compare $_GETcurrent? with the current version, most likely using PHP's version_compare() (if we do the RPC in PHP). If it's fine, then return an XML response with <status>0</status> or something. If it's not up to date, then <status>1</status> <latest>2.3.1</latest> <type>critical/major/minor</type>.

RPC should just be flat text files if possible spread the computational load of all the many remote requests onto the server making the request so as to not overload rpc.wordpress.org with too much work.

Using a REST api such as : ​http://rpc.wordpress.org/update/1.0/wordpress-version to make the request which can then return some data:

1. latest relevant version number for this branch (2.0 or 2.2)
2. update type.
3. link to blog posting.

• WP: Use JS magic to parse the XML response (I know this is doable somehow). Based on what's returned, show an i18ned message in the Dashboard alerting the user to such, and provide a download link to ​http://wp.org/latest.tar.gz and ​http://wp.org/latest.zip

Parse the response in PHP and cache it so that you only check every 15 mins at most.

Some other comments:

1. Consider making this update info available on more than just the dashboard - some people will not necessarily visit the dashboard very often - if they did they would already see the blog posts!
2. This feature must be auto disabled for private blogs to enforce their privacy.
3. We need to consider which users should trigger the check and get it displayed to them.
4. Provide a hook that is fired when a new message is detected to allow plugins to do things at this time.

[rob1n]

Great points, westi. I'm planning to take a crack at this in the next few weeks.

[pishmishy]

XML RPC server (as a modification to WordPress XML RPC, example for testing only)

[pishmishy]

I hope I've not stepped on anyone's toes by submitting these patches. I've written some code that uses a simple XMLRPC interface to periodically check if an upgrade is available. I've not gone down an AJAX/Javascript route.

I've included a sample XMLRPC server (actually a modification to WordPress's XMLRPC Server) for testing purposes; there's no reason why a standalone couldn't be used. You'll want to modify updates.php to point to your own XMLRPC server and testing can be done by modifying wp-includes/version.php (remembering to clear out the cached values from the options table).

[pishmishy]

Replying to pishmishy:

(remembering to clear out the cached values from the options table)

I'm not familiar with the wp_cache_ functions (and can't find their documentation!) so I'm not suer if I should be using them rather than the options table for these values. Am I right in thinking that the cache isn't always enabled (i.e. if the cache directory isn't present or writable) and so the options table is the safer choice?

[westi]

Replying to pishmishy:

I hope I've not stepped on anyone's toes by submitting these patches.

Not at all.

However, I am -1 to the current patches.

I think the following things are very important for the implementation of this functionality:

1. Only one request should be made each time to the update notification server.
2. Downtime on the update server must not slow the load time of the blog - i.e. at least use ajax like the dashboard does and probably run the check calls using the builting cron functionality.
3. XML-RPC is a bit heavy for this as it requires processing on the serverside at the update beacon - a REST based service would be much better - e.g. ​http://update-service.org/wordpress/<version>/ would return the update info for <version> of WordPress.

[pishmishy]

Replying to westi:

1. Only one request should be made each time to the update notification server.

Agreed.

1. Downtime on the update server must not slow the load time of the blog - i.e. at least use ajax like the dashboard does and probably run the check calls using the builting cron functionality.

The dashboard feeds don't work if Javascript is disabled. I don't think this is acceptable for something more critical which is why I've avoided javascript and ajax.

I don't know much about the cron functionality - I'll look into that - will have to be careful to make sure that WordPress installs don't all hit the server at the same time.

1. XML-RPC is a bit heavy for this as it requires processing on the serverside at the update beacon - a REST based service would be much better - e.g. ​http://update-service.org/wordpress/<version>/ would return the update info for <version> of WordPress.

Agreed - far less complicated. I'll work on (at least ;-)) the two points we've agreed on this evening.

James

[santosj]

What about mirrors? You can hit the original server once a day or every 5 hours to get the trusted mirrors (that are up and responding). Then randomily hit one of them.

You will most likely find that trying to get from another server is going to take a while. Something like this would probably need a real cron job, because the function might slow down the site terribly.

Replying to pishmishy:

Replying to westi:

1. Only one request should be made each time to the update notification server.

Agreed.

1. Downtime on the update server must not slow the load time of the blog - i.e. at least use ajax like the dashboard does and probably run the check calls using the builting cron functionality.

The dashboard feeds don't work if Javascript is disabled. I don't think this is acceptable for something more critical which is why I've avoided javascript and ajax.

I don't know much about the cron functionality - I'll look into that - will have to be careful to make sure that WordPress installs don't all hit the server at the same time.

1. XML-RPC is a bit heavy for this as it requires processing on the serverside at the update beacon - a REST based service would be much better - e.g. ​http://update-service.org/wordpress/<version>/ would return the update info for <version> of WordPress.

Agreed - far less complicated. I'll work on (at least ;-)) the two points we've agreed on this evening.

James

[westi]

A few other comments that have come to mind:

1. Update check should happen often - I have found that every 15 minutes works well for the update service I already provide (​http://blog.ftwr.co.uk/wordpress/wp-version-check/) - the bandwidth used is minimal. (Last month the ~1000 unique users of my service used only 10MB of bandwidth)
2. Update check should timeout quickly on it's connection to the remote server - to alleviate page load problems on downtime of the update service.
3. Hooks to perform actions when a new version is detected would be a bonus - this could allow plugins to trigger admin emails / auto-upgrades etc.

[pishmishy]

1. Downtime on the update server must not slow the load time of the blog - i.e. at least use ajax like the dashboard does and probably run the check calls using the builting cron functionality.

I don't think the built in cron functionality is a perfect answer; the page load that triggers the scheduled function will still be slowed with it. I think it's better to slow an admin's page load than a random reader.

I've changed the code to use a single call to a REST interface with curl timing out after five seconds. I'll post the updated code after I've added a hook.

[pishmishy]

slight tweaks to previous version - ammends existing WordPress files

[pishmishy]

Attached the latest version of my patches, uses a REST interface this time with only a single HTTP call. Added a hook for plugins to perform actions on an upgrade notification; added code for scheduling functions with wp-cron if decided necessary.

[westi]

Replying to pishmishy:

Attached the latest version of my patches, uses a REST interface this time with only a single HTTP call. Added a hook for plugins to perform actions on an upgrade notification; added code for scheduling functions with wp-cron if decided necessary.

+1

This looks really nice.

I think the only thing left for me before we get this live it to decide exactly how much of the text displayed should be hard coded as it is now or should be returned by the REST service.

For me I would like to see all the displayed text be returned from the REST service to enable very specific messages - like when a security release is made this could be highlighted and when a major version release is made a short list of new features could be displayed.

I suppose I would still prefer if the test was populated via ajax just to ensure snappy page loads an all.. ;-).

Thank you for your work on this.... this is something I haven't found the time to redo in tooo long!

[pishmishy]

Replying to westi:

I think the only thing left for me before we get this live it to decide exactly how much of the text displayed should be hard coded as it is now or should be returned by the REST service.

For me I would like to see all the displayed text be returned from the REST service to enable very specific messages - like when a security release is made this could be highlighted and when a major version release is made a short list of new features could be displayed.

I think displaying all the text in the REST service might be difficult from an internationalization point of view as we'll be stuck with whatever phrases the server sends us. Perhaps having the rest output as "2.2.2, This version was released to fix a security problem" instead of just "2.2.2" might be useful. Then combine the version number, set phrases and messages together into something that can at least be partly translated by WordPress.

[westi]

Replying to pishmishy:

Replying to westi:

I think the only thing left for me before we get this live it to decide exactly how much of the text displayed should be hard coded as it is now or should be returned by the REST service.

For me I would like to see all the displayed text be returned from the REST service to enable very specific messages - like when a security release is made this could be highlighted and when a major version release is made a short list of new features could be displayed.

I think displaying all the text in the REST service might be difficult from an internationalization point of view as we'll be stuck with whatever phrases the server sends us. Perhaps having the rest output as "2.2.2, This version was released to fix a security problem" instead of just "2.2.2" might be useful. Then combine the version number, set phrases and messages together into something that can at least be partly translated by WordPress.

True it does make internationalisation difficult.

Maybe we need a set of phrases which can then be translated and the REST service lets you know which phrase to use and provides the configurable part of the phrase (e.g. version number:

Some example phrases (excuse my poor english):

1. WordPress version %s, a major functionality update has been released.
2. WordPress version %s, is available with fixes for %n known security issues.
3. WordPress version %s, is available with fixes for %n bugs.

IMHO all these strings should also link to the blog post announcing the release (another piece of info which the rest service should provide)

We should then be able to extend the list in the future if we have other update messages we want to display.

[pishmishy]

I've added the ability to use a set of phrases. Would a link to the blog post be in place of the link to the download page or in addition to?

[_erik_]

1 statement: nice work guys.
3 ideas:

1) pls make sure you do NOT hardcode an update-url. you might piss off all those people putting l10n'd wordpress archives together. (eg: take wordpress-deutschland.org for example. the german zip/tar is established and comes within hours. btw: wordpress-germany already has all this and it is working fine. (except for the branches check) see: ​http://tinyurl.com/yo4t7l (plugin-page) ​http://tinyurl.com/ypflz6 (plugin download) and ​http://static.wordpress-deutschland.org/version.xml (the response)
I'm not saying this is better or ideal, but worth considering as it would minimize your l10n worries and takes care of the users habits.

2) where's the easter-egg for alpha-testers? ;)

[_erik_]

3) on displaying: please make sure to provide enough semantic-elements and CSS-classes/IDs so that it could be properly fitted into ANY admin-theme. (i think of absolute positioning = preferred) as i hate my navigation consumin too much space.

4) the hook is a really good idea to give plugin-authors the chance to tell their admins, that either something might break (-> "go see my plugin-page for updates, etc") or at least

PS: Olaf can surely also give detailed information on the traffic and download-numbers of his own attempt. contact info: ​http://blogshop.de/kontakt/

[pishmishy]

wp-admin/includes/updates.php - improved translation

[pishmishy]

Replying to _erik_:

1) pls make sure you do NOT hardcode an update-url.

Done, the urls are now translatable. If you do change one URL make sure you change the other. No point telling someone that an update is available when it's not been made ready for their translation yet.

2) where's the easter-egg for alpha-testers? ;)

A suitably witty comment is now displayed

3) on displaying: please make sure to provide enough semantic-elements and CSS-classes

I'm not sure what you mean. I've placed the warning into a div and suggested a change to the CSS file but there's nothing stopping alternative CSS being used. If I've missed the point - please let me know.

I've also added the blog URL to the REST resource.

James

[matt]

(In [5892]) First pass at upgrade notification for core, see #1476. Also includes some minor footer styling changes.

[matt]

I've got an implementation of this both the client side and server side at api.wordpress.org. I've deliberately left out some things from this discussion, trying to keep it to the bare minimum required for the initial functionality, as simple as possible. The call is versioned so we can always add more later.

It is also fairly self-contained so could be turned into a plugin for older versions, and the web service API returns correct responses for all versions of WP.

The thing I'm least happy with is how the notification looks. I'm very open to some visual suggestions there. Play with your version.php to see the different states.

[westi]

Replying to matt:

I've got an implementation of this both the client side and server side at api.wordpress.org. I've deliberately left out some things from this discussion, trying to keep it to the bare minimum required for the initial functionality, as simple as possible. The call is versioned so we can always add more later.

It is also fairly self-contained so could be turned into a plugin for older versions, and the web service API returns correct responses for all versions of WP.

The thing I'm least happy with is how the notification looks. I'm very open to some visual suggestions there. Play with your version.php to see the different states.

Cool, this looks good.

Some comments:

1. Is it worth documenting the API somewhere (either at api.wordpress.org on on the codex)
2. I think the call to wp_version_check should run on a hook rather than at include time - hook it onto "init" is probably best.
3. I don't think we should do the check at all if the blog is marked as private / hidden from Google - this check is effectively announcing it.
4. I think we should warn the user if the update check fails - i.e if the last check time is too old - so that they are not left with the false assumption they are up-to-date.

[Viper007Bond]

That is one ugly ass (well, plain) footer. Hopefully it's just temporary...

[matt]

1. Yes I would like when you visited api.wordpress.org that it had a list of its methods and a dev reference for them. What to pass and what to expect back.
2. Agreed.
3. I'm on the fence about this. The purpose of a hidden-from-Google blog isn't to be invisible it's just to not be in search engines, and this doesn't impact that at all. By the same logic we should remove RSS feeds on the dashboard. I think a tin-foil hat mode should be left to a plugin.
4. Judging from Akismet support, it's a fairly common problem for hosts to firewall or close off outgoing HTTP access from PHP. I think this would be handy on a diagnostics page somewhere in WP.

Viper, the header notice will definitely change.

[matt]

(In [5894]) Switch to init action. Hat tip: westi. See #1476

[santosj]

4. Well, on this, I have some experience bypassing this. Very difficult, depending on how crappy (paranoid) the host is, but possible. My suggestion would be to use the three or four methods and if they all fail, then tell the user to switch hosts or to wordpress.com.

Curl
Fopen (if ini_settings allows for this. however, PHP 4 and PHP 5 has an extra ini setting for allowing fopen and closing include(_once)/require(_once)
Sockets (fsockopen) (second prefer for PHP 4)
Streams (prefer for PHP 5 and PHP 4)
HTTP (PHP 5.2) (if exists, and if anyone knows how to use it, then use this for PHP 5.2)

Yeah, it can be nasty, but for full support, pretty much required work. As far as I know, at least one of these should allow for bypassing the allow_fopen_urls ini config. If not, then just spit out some error.

[matt]

(In [5904]) Better styling for update notice, fixes #1476

[matt]

I got the better styling so I'm closing out this ticket.

For the HTTP opening issues, right now it's the same way we do it in other places in WP, but I'm very very open to making this smarter, though to get in for 2.3 it needs to happen before Tuesday.

Basically there are 4-5 places in WP and also in Akismet that make a remote HTTP call, not to mention we bundle the Snoopy class. Pretty much everywhere we do the fopen thing I did here, but I'd be fine with normalizing all those one-offs into a utility function, but it'd only be really useful if it did the open failover stuff mentioned above. I'm sure it will help some, but we still need a documentation aspect because most of the times I've run into it it's been a firewall issue not php config.

If you're interested in that, go ahead and open a new ticket for it. I'll be happy to test it in the environment I have access to.

[Viper007Bond]

Something's up. It's currently not displaying my version number on my localhost install using the latest SVN. It used to work fine.

[Viper007Bond]

I suspect it may have to do with a timed out request for the latest version (my install was a taking a while to load, so I restarted Apache).

Regardless of the cause though, we need to always display the version even if we can't say if it's the latest version or not. Simply put, there's some logic error going on somewhere.

[Viper007Bond]

Will open a new ticket.