WordPress.org

Make WordPress Core

Opened 6 years ago

Last modified 6 months ago

#28146 assigned enhancement

Export Filter for Post IDs

Reported by: zourbuth Owned by:
Milestone: Priority: normal
Severity: normal Version: 2.1
Component: Export Keywords: has-patch needs-refresh
Focuses: administration Cc:
PR Number:

Description

I know there is an action hook export_wp we can use to create our own export function for custom post queries.

The idea is to apply filters for the $post_ids at wp-admin\includes\export.php line 87

$post_ids = $wpdb->get_col( "SELECT ID FROM {$wpdb->posts} $join WHERE $where" );

to

$post_ids = apply_filters( 'export_post_ids', $wpdb->get_col( "SELECT ID FROM {$wpdb->posts} $join WHERE $where" ) );

so we can do custom post queries to generate the post id(s).

Attachments (9)

export.php (16.8 KB) - added by zourbuth 6 years ago.
export.2.php (16.8 KB) - added by zourbuth 6 years ago.
28146.diff (706 bytes) - added by GaryJ 5 years ago.
28146.2.diff (791 bytes) - added by MikeHansenMe 5 years ago.
added additional param docs
28146.3.diff (6.8 KB) - added by sc0ttkclark 5 years ago.
Refactor export_wp to use a WP_Query loop, add filter for $query_args to override from export_filters field input
28146.4.diff (7.8 KB) - added by sc0ttkclark 5 years ago.
Use the more performant postin for paginating posts by $post_ids, add a filter to $post_ids for people to include things that would be impossible with the query args filter on it's own -- for instance -- Including attachments whose post_parentin $post_ids
28146.5.diff (7.7 KB) - added by sc0ttkclark 5 years ago.
Fix post type handling to not overwrite $args
28146.6.diff (7.8 KB) - added by ivanblagdan 4 years ago.
A patch working against the current trunk.
28146.7 (8.0 KB) - added by ivanblagdan 4 years ago.

Download all attachments as: .zip

Change History (31)

@zourbuth
6 years ago

#1 @zourbuth
6 years ago

Sorry, I forgot to pass the $args to the filter.

$post_ids = apply_filters( 'export_post_ids', $wpdb->get_col( "SELECT ID FROM {$wpdb->posts} $join WHERE $where" ), $args );

@zourbuth
6 years ago

#2 @netweb
6 years ago

Related: #22435

@GaryJ
5 years ago

#3 @GaryJ
5 years ago

  • Keywords has-patch added

#4 @SergeyBiryukov
5 years ago

  • Milestone changed from Awaiting Review to 4.1

@MikeHansenMe
5 years ago

added additional param docs

#5 follow-up: @johnbillion
5 years ago

  • Keywords 2nd-opinion added
  • Milestone changed from 4.1 to Future Release

I think we'd be better off adding a filter to the initial query SQL, rather than applying a filter to the post IDs after the query runs. Unless there is a use case for filtering the post IDs array rather than the SQL?

#6 in reply to: ↑ 5 ; follow-up: @sc0ttkclark
5 years ago

Replying to johnbillion:

I think we'd be better off adding a filter to the initial query SQL, rather than applying a filter to the post IDs after the query runs. Unless there is a use case for filtering the post IDs array rather than the SQL?

I'd really like to see a filter on $join / $where pieces so I can add custom taxonomy support or whatever else I might want to.

Although, I'm just going to throw this out there.. I have no idea why this couldn't easily use a get_posts with 'fields' => 'ids' and change all of the manual SQL to utilize WP_Query args, with a filter on the args before they go into WP_Query.

I know #22435 proposes some more sweeping changes for abstraction and cleanup, but WP_Query would ultimately be the most flexible option for this as a solution to the overall problem (not being able to control the Post IDs that get sent to the export output)

#7 in reply to: ↑ 6 ; follow-up: @SergeyBiryukov
5 years ago

Replying to sc0ttkclark:

I have no idea why this couldn't easily use a get_posts with 'fields' => 'ids' and change all of the manual SQL to utilize WP_Query args, with a filter on the args before they go into WP_Query.

Related: comment:7:ticket:27048

#8 in reply to: ↑ 7 @sc0ttkclark
5 years ago

  • Keywords close added; 2nd-opinion removed

I propose we close this ticket in favor of the resolution of #27048, we'll just be sure to add a filter to address the needs of overriding with specific post IDs via $query_args = array( 'post__in' => .... );

#9 @sc0ttkclark
5 years ago

  • Keywords close removed

I now think it's best to tackle this ticket separate from #27048 so it can be more likely to merge in as it's simplified from the ability to actually select multiple post types to merely allowing you to filter everything from the WP_Query args.

@sc0ttkclark
5 years ago

Refactor export_wp to use a WP_Query loop, add filter for $query_args to override from export_filters field input

@sc0ttkclark
5 years ago

Use the more performant postin for paginating posts by $post_ids, add a filter to $post_ids for people to include things that would be impossible with the query args filter on it's own -- for instance -- Including attachments whose post_parentin $post_ids

#10 @sc0ttkclark
5 years ago

  • Keywords needs-unit-tests added

Reposting because formatting got messed up -- Latest patch uses the more performant post__in for paginating posts by $post_ids, add a filter to $post_ids for people to include things that would be impossible with the query args filter on it's own -- for instance -- Including attachments whose post_parent__in $post_ids

@sc0ttkclark
5 years ago

Fix post type handling to not overwrite $args

#11 @westonruter
4 years ago

  • Milestone changed from Future Release to 4.4
  • Version changed from 3.9 to 2.1

Note that being able to filter which posts are being exported is very important for plugins being able to ensure that any post dependencies are also included among the exported data. For instance, if a post being exported references another post of a different post type among its postmeta, then this referenced post should be able to be included among the exported posts.

I'm going to suggest this for 4.4.

#12 @westonruter
4 years ago

  • Owner set to westonruter
  • Status changed from new to accepted

Are there any concerns with this patch, other than unit tests being needed?

#13 @ivanblagdan
4 years ago

I've tried running this patch and it failed. Here's the output:

Running "patch:https://core.trac.wordpress.org/ticket/28146" (patch) task
[?] Please select a patch to apply: 28146.5.diff​ (7.7 KB) - added by sc0ttkclark 7 months ago.
patching file src/wp-admin/includes/export.php
Hunk #1 succeeded at 42 (offset 3 lines).
Hunk #2 succeeded at 147 (offset 3 lines).
Hunk #3 FAILED at 433.
Hunk #4 succeeded at 579 (offset 15 lines).
1 out of 4 hunks FAILED -- saving rejects to file src/wp-admin/includes/export.php.rej

#14 @sc0ttkclark
4 years ago

Where is "433"? Does that mean line 433 in the patch is broken?

#15 @ivanblagdan
4 years ago

As far as I can tell, its related to a docbloc:

23ce2efd src/wp-admin/includes/export.php (Scott Taylor               2015-05-28 21:40:27 +0000 445)  /**
23ce2efd src/wp-admin/includes/export.php (Scott Taylor               2015-05-28 21:40:27 +0000 446)   * @global WP_Query $wp_query
23ce2efd src/wp-admin/includes/export.php (Scott Taylor               2015-05-28 21:40:27 +0000 447)   */

Heres the export.php.rej as well:

***************
*** 377,397 ****
  	do_action( 'rss2_head' );
  	?>
  
- <?php if ( $post_ids ) {
- 	global $wp_query;
  
- 	// Fake being in the loop.
- 	$wp_query->in_the_loop = true;
  
- 	// Fetch 20 posts at a time rather than loading the entire table into memory.
- 	while ( $next_posts = array_splice( $post_ids, 0, 20 ) ) {
- 	$where = 'WHERE ID IN (' . join( ',', $next_posts ) . ')';
- 	$posts = $wpdb->get_results( "SELECT * FROM {$wpdb->posts} $where" );
  
- 	// Begin Loop.
- 	foreach ( $posts as $post ) {
- 		setup_postdata( $post );
- 		$is_sticky = is_sticky( $post->ID ) ? 1 : 0;
  ?>
  	<item>
  		<title><?php
--- 433,460 ----
  	do_action( 'rss2_head' );
  	?>
  
+ <?php
+ 	if ( $post_ids ) {
+ 		$posts_per_page = 20;
+ 
+ 		$wp_query_args = array(
+ 			'post_type'      => 'any',
+ 			'post_status'    => 'any',
+ 			'posts_per_page' => $posts_per_page,
+ 			'post__in'       => array_slice( $post_ids, 0, $posts_per_page )
+ 		);
+ 
+ 		$query = new WP_Query( $wp_query_args );
  
+ 		$page = 0;
  
+ 		// Paginate posts 20 at a time
+ 		while ( $query->have_posts() ) {
+ 			// Begin Loop.
+ 			while ( $query->have_posts() ) {
+ 				$query->the_post();
  
+ 				$is_sticky = is_sticky( $post->ID ) ? 1 : 0;
  ?>
  	<item>
  		<title><?php

@ivanblagdan
4 years ago

A patch working against the current trunk.

#17 @ivanblagdan
4 years ago

I've tried writing a couple of tests for this as well, but getting export_wp() to run gets me nowhere with the test stalling at the command line. There are 25 posts created via setUpBeforeClass() as a sample.

Here's a stab at running it in a test class, found this approach among the other tests somewhere.

	function do_export(){
		ob_start();
		try {
			@require_once(ABSPATH . 'wp-admin/includes/export.php');
			export_wp();
			$out = ob_get_clean();
		} catch (Exception $e) {
			$out = ob_get_clean();
			throw($e);
		}
		return $out;
	}

Not sure if I'm handling this wrong, any suggestions for a test are appreciated.

#18 @westonruter
4 years ago

  • Owner westonruter deleted
  • Status changed from accepted to assigned

#19 @DrewAPicture
4 years ago

  • Keywords needs-refresh needs-screenshots added

@ivanblagdan: Hi, thanks for the work so far. Looks like 28146.6.diff needs a refresh against current trunk. Edit: Ignore the screenshots comment, wrong ticket.

@wonderboymusic: Any suggestions on how we could go about testing export_wp()? See comment:17.

Last edited 4 years ago by DrewAPicture (previous) (diff)

@ivanblagdan
4 years ago

#20 @wonderboymusic
4 years ago

  • Keywords needs-unit-tests needs-screenshots removed

Downloaded the patch, sight unseen - I have 21 posts in the database. Chose to exports Posts that are Drafts. The XML file stopped filling up at 45MB after running for 30 seconds.

#needs-refresh

#21 @wonderboymusic
4 years ago

  • Milestone changed from 4.4 to Future Release

#22 @hlashbrooke
4 years ago

Just a note to say that the patch proposed in #27048 also fixes the issue discussed in this ticket as it adds a filter to the array of Post IDs as a part of the general improvements to the exporter.

Note: See TracTickets for help on using tickets.