WordPress.org

Make WordPress Core

Opened 4 years ago

Last modified 3 months ago

#11585 new enhancement

WordPress should cache failed feed fetches so as to avoid overloading feed sources

Reported by: Denis-de-Bernardy Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version: 2.9
Component: Feeds Keywords: has-patch
Focuses: performance Cc:

Description

Following up on #11219, which fixed the cause of that particular error, but not the one that related to caching of feed errors.

When SimplePie fails to fetch or parse a feed, it should not hammer whichever server the feed came from on every page load. This is bad for two reasons: it disturbs the originating server, and it tremendously slows down page loads when the feed is in an RSS widget on the front end.

Instead, we should cache the error for a reasonably long amount of time (15 minutes? An hour? More?) and bypass SimplePie until that duration expires.

Attachments (2)

11585.diff (3.6 KB) - added by peterchester 9 months ago.
11585.2.diff (6.9 KB) - added by peterchester 9 months ago.

Download all attachments as: .zip

Change History (9)

comment:1 hakre4 years ago

Idea makes sense. I know from one site I use simplepie on and where I do logging that feels fail from time to time like out of nothing. Tend to a smaller timespan like 15 minutes.

comment:2 mtdewvirus4 years ago

What about caching the feed items twice? Once with a long cache expiration and once with the current expiration. If the current cache expires and the feed isn't available, repopulate from the long expiration cache and set a shorter expiration to check if the feed is back up. If the feed is available, update both caches. This way, a visitor doesn't need to see an error and will still see content.

comment:3 nacin4 years ago

  • Milestone changed from 3.0 to Future Release

Needs a patch.

comment:4 peterchester9 months ago

Taking a look at this...

peterchester9 months ago

comment:5 peterchester9 months ago

  • Keywords has-patch added; needs-patch removed

Ok, I think I have it. But honestly, this part of the code is pretty hard to follow. I've tested locally with working and broken feed urls but it would be great if this could be tested more thoroughly with feeds that are throttled and / or temporarily disabled.

Some questions that came up in the process:

  • Should we be displaying errors in the RSS widget on the frontend when a feed is not working or should we just hide the widget?
  • The error message for a bad feed appears in the RSS admin when you first enter it. But after saving it again, the error disappears because the feed is cached (empty). Should we be effectively caching the error in the widget admin?

comment:6 peterchester9 months ago

Just confirmed that the error message for a failed feed only showed up on the first save before this fix so that's not a new issue if it's an issue at all.

I'm also adding a new patch with a solution for separating the cache timing from the error cache timing. Initial testing on that came out positively. To test cache timings use the filters 'wp_feed_cache_transient_lifetime' (the existing one) vs. 'wp_feed_error_cache_transient_lifetime' (the new one)

To test, add the following to your functions.php file to easily see the timeouts if you load a functional feed vs. a broken one.

add_filter( 'wp_feed_cache_transient_lifetime', 'my_feed_cache_transient_lifetime' );
function my_feed_cache_transient_lifetime() {
	return 30; // 30 second cache life time for successful content
}

add_filter( 'wp_feed_error_cache_transient_lifetime', 'my_feed_error_cache_transient_lifetime' );
function my_feed_error_cache_transient_lifetime() {
	return 5; // 5 second cache life time for errors.
}

I tested broken feeds by entering a url in the RSS widget that i knew was totally bunk and one that was loading xml that made Simplepie sad vs. my control of a feed on a live site that was working fine.

peterchester9 months ago

comment:7 nacin3 months ago

  • Component changed from Performance to Feeds
  • Focuses performance added
Note: See TracTickets for help on using tickets.