Update SimplePie to Latest Version and possibly include PHP-MF2

[dshanske]

SimplePie Version 1.4 is now available.

​https://github.com/simplepie/simplepie/releases

It is the first release in 3 years and has a large number of bug fixes. It also is the first version that includes support for new types of feeds as described by the IndieWebCamp community.

These feeds are made up of h-feed and h-entry microformats, which allows the feed to be parsed from the html rather than a separate xml file.

SimplePie will now detect when it finds such a feed and parse them when it finds the php-mf2 parser available.

[jorbin]

@rmccue - thoughts?

[slushman]

Patch updating SimplePie to version 1.4.

[slushman]

I've run into an issue with a plugin trying to read an RSS feed from a site with a Lets Encrypt (self-signed) cert. It appears this was fixed in [May 2015](​https://github.com/simplepie/simplepie/pull/407) and that fix is part of SimplePie 1.4.
Submitting patch to update SimplePie to version 1.4.

[jhabdas]

SimplePie 1.5 has been released. It still does not support PHP 7.0, which is what I believe WP suggests. It seems apt to deprecate or make private fetch_feed() entirely and let the plugins manage the use cases and forward-facing support role.

[Presskopp]

1.5.1 is latest now

[aquakitty]

#42515 was marked as a duplicate.

[etruel]

Hi all, someone recommended me to tell the following here. I'll try to say some different behaviours that we have in our plugin WPeMatico by reading different feeds.

I can't give too much technical details, but we're attending so much tickets of lot of users with problems using the Simplepie 1.3.1 included in WordPress that we solved just by using the new version 1.5 or newer.

Almost all the new things we found in the 1.5 version (already to download in simplepie.org) was related with fixed issues and improvements. (​https://github.com/simplepie/simplepie/releases)

Using 1.3.1 included in WP, some of the feeds with problem are solved with force_feed and others by changing to a known User-agent. (But in 1.5 works by default)

There are feeds giving different errors in 1.3.1 than 1.5. And some of them work well in 1.5 and not in 1.3 (1.3 tried with and without force_feed)

For example ​http://thenationonlineng.net/feed/
Works well with 1.5.

1.3.1 = ERROR: The feed ​http://thenationonlineng.net/feed/ cannot be parsed. Simplepie said: This XML document is invalid, likely due to invalid characters. XML error: Reserved XML Name at line 2, column 38

Many errors that produce notices like examples below are fixed easily by using the new 1.5:

ERROR: Feed ​https://www.cryptocoinsnews.com/feed could not be parsed. (SimplePie said: A feed could not be found at ​https://www.cryptocoinsnews.com/feed. A feed with an invalid mime type may fall victim to this error, or SimplePie was unable to auto-discover it.. Use force_feed() if you are certain this URL is a real feed.)

ERROR: Feed ​https://www.youtube.com/feeds/videos.xml?channel_id=UCCWYNcRdRteOF3wIRVeaJZQ could not be parsed. (SimplePie said: cURL error 60: SSL certificate problem: unable to get local issuer certificate)

Hope this helps.

cheers

etruel

[swissspidy]

#44309 was marked as a duplicate.

[wpshades]

The latest release of SimplePie found at - ​https://github.com/simplepie/simplepie/releases

[wpshades]

Hi everyone!

I have just submitted a patch that includes the latest release of SimplePie(1.5.2).
If there's any way in which I can help test this out, please let me know.

Thanks!

[dmenard]

Hi!

I would like to second this request.

With WP 4.9.8 and PHP 7.2, the WordPress RSS widget generates many warnings like this:

Warning: A non-numeric value encountered in wordpress\wp-includes\SimplePie\Parse\Date.php on line 694

10	SimplePie->get_items( $start = ???, $end = ??? )			...\class-simplepie.php:2812
11	SimplePie_Item->get_date( $date_format = 'U' )				...\class-simplepie.php:2925
12	SimplePie_Parse_Date->parse( $date = '2018-10-02T08:38:56Z' )		...\Item.php:773
13	SimplePie_Parse_Date->date_w3cdtf( $date = '2018-10-02T08:38:56Z' )	...\Date.php:603

These are caused by the way SimplePie 1.3.1 parses dates and by the ​new E_WARNINGs introduced in PHP 7.1.

​A simple fix for this has been made in july 2016 and is included ​since SimplePie 1.4.2.

So upgrading SimplePie to the latest version would solve these warnings.

Thanks!

[miunosoft]

Just in case for someone who are facing this issue and need to get away from the warnings without modifying the core, here is a plugin, Fix - SimplePie Errors: ​https://github.com/michaeluno/_fix-simplepie-errors).

[pento]

#45436 was marked as a duplicate.

[Znuff]

This bug has been present for years and it affects all production websites on current PHP versions.

There have been patches submitted, people even made plugins.

How come this is taking so long to be adopted in the core?

It's spamming our servers logs so hard that we ended up at 3GB in 3 days. It's incredible.

[pento]

#42684 was marked as a duplicate.

[pento]

PHP 5.2 support was dropped in SimplePie 1.4, 1.5 requires PHP 5.3+.

I'm moving this to the 5.2 milestone, when we'll be bumping WordPress' minimum PHP version to 5.6.

[knutsp]

#42515 was marked as a duplicate.

[desrosj]

5.2 beta is in less than 2 days. The latest patch needs to be refreshed to apply cleanly to trunk, and this needs to be tested. Punting to 5.3.

@rmccue do you have any thoughts about this?

[dshanske]

@rmccue was assigned this task 3 years ago and has not commented on this or noted his acceptance of the assignment. Can I suggest that, if he is occupied with other things, this revert to unassigned in the hopes someone else will take this on?

[Hareesh Pillai]

Adding refreshed patch

[elexem]

not sure I understand the implications of the latest entries in this thread.
Can we expect the error-causing-behaviour to go away sometime soon? Looking into /wp-includes/SimplePie seems to still be on ver 1.3.1 while ​https://github.com/simplepie/simplepie/releases/ lists 1.5.2 as released early Aug 2018 - so perhaps I'm too incompetent to grasp the monumental hindrances impeding adapting that.
Either way, it would be nice if that could be mended; the nuisant oversight pelts us with error lines we could clearly do without; 91953 in the last cycle.

Cheers

[dshanske]

No one with privileges to commit wants to take ownership of it.

[Znuff]

How about mark this as a BUG instead of an "enhancement" and mark it as a blocker?

The amount of pussyfooting around this issue is astounding.

At this rate, I'm gonna have to patch that file manually until Wordpress 8.0. And I'm sick of patching it every release.

[elexem]

I'll have to agree with @Znuff here.

+1 :/

[dshanske]

Okay. I will classify it as a bug...

[kirasong]

@stevenkword as Feeds maintainer, what do you think about upgrading this as part of 5.3?

[stevenkword]

Replying to mikeschroder:

@stevenkword as Feeds maintainer, what do you think about upgrading this as part of 5.3?

Thanks for putting this on my radar. The current version in WP is 1.3.1, last updated 6 years ago.

I think a major prerequisite for this would be to have better test coverage across the feeds component. I began introducing tests a few years back, but they are still not complete.

I think this is a great idea to inherit the bugfixes, but I'd like to ensure we have 1:1 tests against every function touched in the SimplePie changelogs between 1.3.1 and 1.5. I'd always want this big of a change to land in the release early to allow for a long bake time.

[dshanske]

SimplePie has its own test coverage. Do we need to redo them or simply incorporate them?

[davidbaumwald]

This ticket still needs a decision, and with 5.3 Beta 1 only a few days away, this is being moved to Future Release.

[jrf]

Related #47746 - the current version used of SimplePie contains a PHP 7.4 compatibility issue.

This issue has been patched upstream, but is not contained in a tagged release yet. Tagging a new release has been requested: ​https://github.com/simplepie/simplepie/issues/620

[jrf]

SimplePie has released a new version including the changes needed for PHP 7.4 support: ​https://github.com/simplepie/simplepie/releases/tag/1.5.3

[stevenkword]

Replying to jrf:

SimplePie has released a new version including the changes needed for PHP 7.4 support: ​https://github.com/simplepie/simplepie/releases/tag/1.5.3

Due to the late introduction into this cycle, I'm very hesitant for a push. The PHP 7.4 support doesn't concern me necessarily since there is a patch available in #47746, but I would like this sort of request to hit a release early. Let's aim for 5.4 with a tag of 'early' so that we can properly kick the tires. I'm not nearly as worried about breaking Core functionality as I am dependent plugins.

We should do some discovery against the largest plugin consumers of the library and confirm compatibility first.

[desrosj]

#44087 was marked as a duplicate.

[Hareesh Pillai]

#29238 was marked as a duplicate.

[nasiralamreeki]

WordPress really needs to take more seriously updating external libraries. Years between updating libraries from upstream is unacceptable. There’s tremendous benefits to getting these libraries updated faster.

We can’t wait for theme and plugin developers forever to test. Sometimes we have to push the web forward by pushing ahead with changes even if they break themes and plugins as this in turn will push developers to update their code.

We pushed Gutenberg ahead even though most major themes and plugins weren’t ready for it.

[Ipstenu]

@stevenkword Here's a list of everyone using SimplePie: ​https://wpdirectory.net/search/01DT3A1NPYN8FRVGACQ4FTPDHE

If you need a more precise search, let me know and I'll get it for you. But this is a constant bugbear with random plugins and we could use this updated.

[stevenkword]

Replying to Ipstenu:

@stevenkword Here's a list of everyone using SimplePie: ​https://wpdirectory.net/search/01DT3A1NPYN8FRVGACQ4FTPDHE

If you need a more precise search, let me know and I'll get it for you. But this is a constant bugbear with random plugins and we could use this updated.

Thank you for the ping, @Ipstenu! The list is more that sufficient for now. I'm all in favor of implementing the latest stable release of Simple Pie based on trac feedback and having looked over the changelogs. I'd like to reach out to a sample set of plugins from the list to make sure we don't have any blind spots. It seems like a no-brainer from here, but I just want to double check with a few different types of use case plugins (aggregators, podcasts, xml sync, etc) to ensure we've been thorough and allow the opportunity for testing. Now is a great time since we're early into the release cycle.

[stevenkword]

Replying to stevenkword:

Replying to Ipstenu:

@stevenkword Here's a list of everyone using SimplePie: ​https://wpdirectory.net/search/01DT3A1NPYN8FRVGACQ4FTPDHE

If you need a more precise search, let me know and I'll get it for you. But this is a constant bugbear with random plugins and we could use this updated.

Thank you for the ping, @Ipstenu! The list is more that sufficient for now. I'm all in favor of implementing the latest stable release of Simple Pie based on trac feedback and having looked over the changelogs. I'd like to reach out to a sample set of plugins from the list to make sure we don't have any blind spots. It seems like a no-brainer from here, but I just want to double check with a few different types of use case plugins (aggregators, podcasts, xml sync, etc) to ensure we've been thorough and allow the opportunity for testing. Now is a great time since we're early into the release cycle.

Is anyone able to assist with doing some compatibility checks against plugins from this list with the latest patch applied to trunk?

[etruel]

Replying to stevenkword:

Is anyone able to assist with doing some compatibility checks against plugins from this list with the latest patch applied to trunk?

I could make some tests. How can I help ?
I have WordPress beta tester plugin 2.1.0 with WordPress 5.3.1-alpha-46728

Update:: I have WordPress beta tester plugin 2.1.0 with WordPress 5.4-alpha-46750 from trunk
Simplepie 1.3.1

[diddledani]

Updating simplepie to latest will also close #48615.

[dd32]

#49451 was marked as a duplicate.

[audrasjb]

Hi,

With 5.4 Beta 3 approaching and the Beta period reserved for bugs introduced during the cycle, this is being moved to Future Release. If any maintainer or committer feels this should be included or wishes to assume ownership during a specific cycle, feel free to update the milestone accordingly.

[Znuff]

Is this a joke? Are you for real? AGAIN?!

[nasiralamreeki]

Replying to Znuff:

Is this a joke? Are you for real? AGAIN?!

The kicking these updates including jquery which is updated down the road for years is the disappointing reality of WordPress as a project.

[leemon]

Replying to Znuff:

Is this a joke? Are you for real? AGAIN?!

If the bug is not Gutenberg-related, the probability of being fixed is near zero now

[antoine2711]

Replying to Znuff:

How about mark this as a BUG instead of an "enhancement" and mark it as a blocker?

The amount of pussyfooting around this issue is astounding.

At this rate, I'm gonna have to patch that file manually until WordPress 8.0. And I'm sick of patching it every release.

+1

This PR updates SimplePie to version 1.5.4 for testing.

Trac ticket: https://core.trac.wordpress.org/ticket/36669.

This PR updates SimplePie to version 1.5.4 for testing.

Trac ticket: https://core.trac.wordpress.org/ticket/36669.

[desrosj]

I'd like to kindly remind everyone that time is a limited resource. The WordPress project is maintained almost entirely by volunteers (with the exception of a very, very small handful of people). While it is frustrating to see an issue repeatedly miss each new release, it is not because the ticket is being avoided. Leaving negative comments does nothing to progress the ticket closer to a resolution, and probably has the opposite affect. Newer contributors will be hesitant to jump in to help.

I've created a PR with the latest version of SimplePie (and included the refreshed patch here), and it looks like all of the tests that are currently included pass.

@stevenkword I dug in and looked at some of the top plugins using SimplePie. I ​re-ran the scan linked above by @Ipstenu and there were a few additional plugins. I looked at the top 5 plugins ordered by number of active installs. This is what I found:

• UpdraftPlus: There are 2 occurrences of SimplePie in their code base. One never runs because it is intentionally disabled, and the other just pulls their news blog's RSS feed in and includes it in emails when backups run. I tested this out and looks like it works without issue.
• Duplicator: Looks like this plugin has a list of all core WordPress files and classes and does not actually utilize the class.
• WP Statistics: SimplePie is only included in their code base to identify what is crawling/accessing a site.
• Google Tag Manager: SimplePie is only included in their code base to identify bots accessing the site.
• Caldera Forms: simplepie is mentioned once, but looks like it is code from Wonolog, a package that is included with Composer.

I also looked at the top 5 plugins ordered by the most occurrences of "simplepie" with at least 1,000 active installs:

• RSSImport (10,000 active): This plugin worked when I used the shortcode and the default URL [RSSImport], but broke when I attempted to pass a different feed URL to the shortcode with the feedurl argument. I am not sure if this is a block editor, plugin issue, or an issue from the upgrade.
• WP Social Blogroll (1,000 active): This plugin doesn't even activate on 5.4. Based on the support forums, it stopped working at WordPress version 4.7. It also has not been updated in 7 years.
• FeedWordPress (30,000 active): The plugin still works and successfully pulls in posts from a feed, but there are a few warnings thrown when SimplePie is upgraded.

Declaration of FeedWordPie_Item::get_id($hash = false) should be compatible with SimplePie_Item::get_id($hash = false, $fn = 'md5') in wp-content/plugins/feedwordpress/feedwordpie_item.class.php on line 4
Declaration of FeedWordPress_Parser::parse(&$data, $encoding) should be compatible with SimplePie_Parser::parse(&$data, $encoding, $url = '') in wp-content/plugins/feedwordpress/feedwordpress_parser.class.php on line 28

• WPide (50,000 active) and AceIDE (20,000 active): These had a lot of shared code. They only had "simplepie" in strings within JS files.
• Libsyn Publisher Hub (4,000 active): I was unable to test this because it appears you need an active Libsyn account, which is paid. But, I looked at the occurrences of "simplepie" in the code, and it seems to be very standard use of SimplePie.

This research considered and the 5.5 release cycle being very, very early, I think we could make this change and publish a blog post on Make to test thoroughly to help identify problems. We should still work to increase the overall test coverage.

Another thing I wanted to mention was that I was unable to find a full changelog for SimplePie on the GitHub repository. I could be missing it, though. When we make this change, we should have a full changelog somewhere that we can link to or include in a dev note or documentation.

[dshanske]

Current maintainer of SimplePie is fairly responsive and would be interested in any help he could give, as far as he implied to me.

[desrosj]

Opened an ​issue upstream requesting a changelog be added to the official repository.

[desrosj]

Closed out #33003 as reported-upstream as it appears to have been fixed and will be solved once this update is committed.

I also ​reported a separate PR to help progress PHPCompatibility testing in #49922. I plan on requesting a new release from the maintainer so those fixes can be included. But if that doesn't happen in time, the changes in that PR should be included in this update.

Spent some time today building a proper changelog upstream: ​https://github.com/simplepie/simplepie/pull/646

There are several other SimplePie tickets (#37763, #21872, #42254, #29204, #43357 at first glance - I am sure there are more). The last box that should be checked before updating the library is to figure out if:

• Will be solved by this update.
• Will be made worse by this update.
• Will become irrelevant after this update.

This will hopefully identify any additional compatibility issues or blockers not being identified by the current less than adequate test coverage in Core.

[antoine2711]

Hi, I just want to take the time to thank you for you work on this issue. It's good to see it going forward. It's very much appreciated. Best Regards, Antoine

Replying to desrosj:

Closed out #33003 as reported-upstream as it appears to have been fixed and will be solved once this update is committed.

I also ​reported a separate PR to help progress PHPCompatibility testing in #49922. I plan on requesting a new release from the maintainer so those fixes can be included. But if that doesn't happen in time, the changes in that PR should be included in this update.

Spent some time today building a proper changelog upstream: ​https://github.com/simplepie/simplepie/pull/646

There are several other SimplePie tickets (#37763, #21872, #42254, #29204, #43357 at first glance - I am sure there are more). The last box that should be checked before updating the library is to figure out if:

• Will be solved by this update.
• Will be made worse by this update.
• Will become irrelevant after this update.

This will hopefully identify any additional compatibility issues or blockers not being identified by the current less than adequate test coverage in Core.

[johnbillion]

I spent a couple of hours looking through SimplePie changes since 1.3.3 to see if I could identify anything that gives me cause for concern, and the tl;dr is no I didn't. The majority of the changes are bug fixes and several iterations on PHP compatibility.

I think at this point our best approach is to get this into 5.5 early (ie. ASAP), followed up by communication with the authors of the plugins mentioned above who are using SimplePie directly and ask them to do some testing and check for deprecated notices or warnings. We could extend that to plugins that use fetch_feed() but that will probably have diminishing returns.

1.5.5 was just tagged, I believe @desrosj is going to submit a patch to include the latest changes.

[desrosj]

36669.3.diff​ updates to 1.5.5, which only includes a PHP compatibility fix (see #49922), and some doc fixes.

Also, marking as 3.5 since that was the last release this was updated.

Build is passing on my attached PR. Let's get this in so it can be more widely tested.

[desrosj]

Also, forgot to mention that @johnbillion and I discussed including the new Redis and Memcached cache classes and we decided it was best to include them.

• SimplePie won't know they are not present.
• We don't currently exclude anything else in the library from inclusion.
• Makes it easier to continue maintaining the library.

[desrosj]

In 47733:

External Libraries: Update the SimplePie library to the latest version (1.5.5).

This brings SimplePie in sync with the most up to date version, 1.5.5.

This update brings many bug fixes, small enhancements, and PHP compatibility fixes for newer versions of PHP.

For a full list of changes, see ​https://github.com/simplepie/simplepie/blob/master/CHANGELOG.md#155-may-1-2020.

Props dshanske, slushman, etruel, wpshades, dmenard, desrosj, hareesh-pillai, stevenkword, jrf, Ipstenu, johnbillion.
Fixes #36669.

This was merged into core in https://core.trac.wordpress.org/changeset/47733.

[ocean90]

#42515 was marked as a duplicate.

[shojibur]

#50315 was marked as a duplicate.

[desrosj]

This was noted in the following dev note: ​https://make.wordpress.org/core/2020/07/01/external-library-updates-in-wordpress-5-5-call-for-testing/.

There was also a post published on the Make Plugins blog in early May: ​https://make.wordpress.org/plugins/2020/05/01/simplepie-updated-for-wordpress-5-5/

The plugin team also should have sent an email to all plugins with active installs that appear to be using the library based on some rough static searches on wpdirectory.net.

[Hareesh Pillai]

#51347 was marked as a duplicate.