Make admin backend unit-tests friendly

[sirzooro]

Recently I started writing unit tests for my plugins. Beside tests focused on functionality (where I directly call my plugin functions) I write some basic integrations tests, which should test that WordPress will call my function, pass data in expected format and recognize data returned from it. For frontend it is quite easy - theme API is well-defined, so I can write something like this:

public function test_something() {
	// add new post
	$post_id = wp_insert_post( array( ... ) );
	$this->assertGreaterThan( 0, $post_id );
	
	// go to post page
	$this->go_to( get_permalink( $post_id ) );
	
	// main loop
	$checked_post = false;
	while ( have_posts() ) {
		the_post();
		if ( $post_id == get_the_ID() ) {
			$checked_post = true;
			
			// test that content is modified
			ob_start();
			the_content();
			$result = ob_get_clean();
			$this->assertEquals( '...', $result );
		}
	}
	
	// make sure test above was executed
	$this->assertTrue( $checked_post );
}

Unfortunately this is not true for admin backend - there most of code is written directly at file level (not in functions and classes), so I would need to either duplicate this code in my tests (bad approach, because would have to monitor original code for changes), or test using whole file (either load it directly or use Selenium) - in this case test would be more complicated.

Therefore I logged this ticket, to start discussion how we can perform refactoring of admin backend to make it more tests-friendly, and how to test it more thoroughly. Most probably we would also need to modify the testing framework (e.g. introduce new admin_go_to() method).

[sirzooro]

Some non-admin page would also benefit from such refactoring, e.g. wp-login.php.

[jorbin]

While I think this is a worthy goal, I think that having such a large ticket won't serve us nearly as well as tickets for each individual page much like with the hook documentation and jshint. I think scheduling an IRC discussion and then tracking the overall progress through a make/core post makes the most sense.

[bpetty]

Personally, I think this generally just involves modularizing each individual functional component/responsibility into more general (and reusable) admin APIs like WP_Screen. Each one of those components will have their own back-compat issues to deal with, and will each need to be considered individually anyway as their respective API designs are discussed, tested, approved, and finally applied. I think this is something everyone has already been doing as we identify the best potential areas that can/should be addressed at that given moment. This has always just been an ongoing task, and we don't need a ticket to track a continuous task involving a general programming practice.

You did suggest a specific global approach though, so let me address that.

• Actually including the admin PHP endpoint files themselves just makes them significantly more complex and not only changes the environment in which they run (something unit tests should avoid when possible), but might not even be possible for certain admin pages, and likely incompatible with numerous plugins which often won't be expecting to be buffered (although this obviously only matters for plugins that choose to make use of these types of tests themselves).
• This also encourages (as you did yourself) extremely volatile assertions based on content being returned from the output buffer, and this is a specifically discouraged practice when writing unit tests. We need to be writing tests against the functionality of specific individual methods, which is both more reliable, but also more helpful in isolating and identifying broken behaviors.
• PHPUnit contains it's own ​output buffering functionality that we don't need to duplicate or interfere with, but even using that isn't encouraged: "A test that emits output will fail in strict mode."

The output buffer will frequently change significantly in unpredictable ways, but in ways that remain perfectly functional without actually breaking anything. Unit tests can't account for that in any efficient manner, this would require rewriting those unit tests every time some content is moved around a bit. This is why it's referred to as "units of code", functionality that is being tested needs to be as isolated as possible from external environmental factors.

[wonderboymusic]

#33413 and #33813 are making the code more modular and unit test-able