Ticket #5357 (closed enhancement: fixed)

Opened 4 years ago

Last modified 4 years ago

wxr import: don't load the whole file at once

Reported by: tellyworth Owned by: anonymous
Priority: normal Milestone: 2.5
Component: General Version:
Severity: normal Keywords:
Cc:

Description

The wxr importer currently loads all posts into a large array before processing them. That makes large imports impossible on servers with a small memory limit.

The enclosed patch refactors import/wordpress.php to keep only one post at a time in memory. I've keep most of the original code intact to minimize breakage. It's no faster or slower than before. There are other optimizations that could be made but this is a necessary first step.

Unit tests are in  http://svn.automattic.com/wordpress-tests/wp-testcase/test_import_wp.php

Attachments

importer-2pass-3.patch Download (7.5 KB) - added by tellyworth 4 years ago.

Change History

The sample export files used for unit testing are here if anyone wants to confirm those tests by hand:  http://svn.automattic.com/wordpress-tests/wp-testdata/export/

big-export.xml has 500 posts with comments.

comment:2   ryan4 years ago

  • Status changed from new to closed
  • Resolution set to fixed

(In [6336]) Don't load entire wxr import file at once to conserve memory. Props tellyworth. fixes #5357

  • Milestone changed from 2.5 to 2.4
Note: See TracTickets for help on using tickets.