WordPress.org

Make WordPress Core

Opened 21 months ago

Closed 3 months ago

#45303 closed defect (bug) (reported-upstream)

XML parsing should be done in chunks, causing parser out of memory

Reported by: amanmanglik Owned by:
Milestone: Priority: normal
Severity: normal Version: 4.9.8
Component: Import Keywords:
Focuses: Cc:

Description

SimplePie in wordpress reads xml as a single chunk instead of multiple chunks as recommended by xml_parse documentation. This is resulting is parser running out of memory internally and the error is gets suppressed. Its the cause for blogger importer plugin failing with >10MB xml files and possibly other issues.

Full context and a unpolished fix in the support thread here -
https://wordpress.org/support/topic/xml-parsing-not-in-chunks-parser-out-of-memory/

I have verified the bug and my fix on wordpress version 4.9.8 but i am guessing this has existed for a while.

Change History (4)

#1 @swissspidy
21 months ago

  • Component changed from General to Import

#2 @dd32
21 months ago

SimplePie is an external library WordPress uses, this should be reported upstream to them. Although WordPress uses an old version of the library, it looks like the code in question still exists: https://github.com/simplepie/simplepie/blob/master/library/SimplePie/Parser.php#L166-L172

#3 @amanmanglik
21 months ago

Reported it to SimplePie as well but i would request doing a local fix if SimplePie doesnt pick it up fast as Wordpress is the one being affected. Its essentially blocking all potential users who want to migrate from Blogger to WP but cant because their export wont load. My friend, whose blog i am helping with, gave up on the migration months back before involving me recently.

https://github.com/simplepie/simplepie/issues/598

Last edited 21 months ago by amanmanglik (previous) (diff)

#4 @desrosj
3 months ago

  • Milestone Awaiting Review deleted
  • Resolution set to reported-upstream
  • Status changed from new to closed

Hi @amanmanglik,

Thanks again for this ticket. My apologies it did not receive attention for a long period of time.

I am going to close this issue out as reported-upstream. Based on the feedback from the library maintainer on your PR, it seems that the change suggested is just "too fundamental of a change for how SimplePie was designed".

Because SimplePie is an external library, it's not feasible to make modifications to the library in WordPress as it creates the potential for future compatibility issues and would further complicate the process of updating in the future.

If it becomes apparent that this is an issue that many more users are experiencing, the issue upstream should be reopened to be reconsidered.

Note: See TracTickets for help on using tickets.