Plugin tests don't work anymore

[ziodave]

I cannot run anymore plugin tests using:

• bash bin/install-wp-tests.sh wordpress_test root localhost latest
• phpunit

I get the following issue:

Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
PHP Warning:  Cannot modify header information - headers already sent by (output started at /private/tmp/wordpress-tests-lib/includes/bootstrap.php:61) in /private/tmp/wordpress/wp-includes/pluggable.php on line 1121
PHP Stack trace:
PHP   1. {main}() /usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar:0
PHP   2. PHPUnit_TextUI_Command::main() /usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar:614
PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:129
PHP   4. PHPUnit_TextUI_Command->handleArguments() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:138
PHP   5. PHPUnit_TextUI_Command->handleBootstrap() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:606
PHP   6. PHPUnit_Util_Fileloader::checkAndLoad() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:778
PHP   7. PHPUnit_Util_Fileloader::load() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/Util/Fileloader.php:76
PHP   8. include_once() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/Util/Fileloader.php:92
PHP   9. require() /Users/david/Developer/wordpress/wp-cli/wp-content/plugins/my-plugin/tests/bootstrap.php:13
PHP  10. require_once() /private/tmp/wordpress-tests-lib/includes/bootstrap.php:85
PHP  11. wp_not_installed() /private/tmp/wordpress/wp-settings.php:109
PHP  12. wp_redirect() /private/tmp/wordpress/wp-includes/load.php:447
PHP  13. header() /private/tmp/wordpress/wp-includes/pluggable.php:1121

Warning: Cannot modify header information - headers already sent by (output started at /private/tmp/wordpress-tests-lib/includes/bootstrap.php:61) in /private/tmp/wordpress/wp-includes/pluggable.php on line 1121

Call Stack:
    0.0175     525024   1. {main}() /usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar:0
    0.0188     736872   2. PHPUnit_TextUI_Command::main() /usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar:614
    0.0188     737104   3. PHPUnit_TextUI_Command->run() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:129
    0.0188     737536   4. PHPUnit_TextUI_Command->handleArguments() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:138
    0.0209    1090048   5. PHPUnit_TextUI_Command->handleBootstrap() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:606
    0.0210    1098592   6. PHPUnit_Util_Fileloader::checkAndLoad() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/TextUI/Command.php:778
    0.0210    1098816   7. PHPUnit_Util_Fileloader::load() phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/Util/Fileloader.php:76
    0.0211    1103040   8. include_once('/Users/david/Developer/wordpress/wp-cli/wp-content/plugins/my-plugin/tests/bootstrap.php') phar:///usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar/phpunit/Util/Fileloader.php:92
    0.0215    1167200   9. require('/private/tmp/wordpress-tests-lib/includes/bootstrap.php') /Users/david/Developer/wordpress/wp-cli/wp-content/plugins/my-plugin/tests/bootstrap.php:13
    0.1498    1881776  10. require_once('/private/tmp/wordpress/wp-settings.php') /private/tmp/wordpress-tests-lib/includes/bootstrap.php:85
    0.1637    4138264  11. wp_not_installed() /private/tmp/wordpress/wp-settings.php:109
    0.1941    5599752  12. wp_redirect() /private/tmp/wordpress/wp-includes/load.php:447
    0.1943    5608920  13. header() /private/tmp/wordpress/wp-includes/pluggable.php:1121

I've been able to track down the issue in this revision:

r28965 | wonderboymusic | 2014-07-03 04:16:33 +0300 (Thu, 03 Jul 2014) | 4 lines

Transient timeouts are untestable when `WP_INSTALLING` is defined and set to ANY value. Transient timeouts cannot be tested during multisite runs because `wpmu_create_blog()` sets `WP_INSTALLING`. I have zero idea how these tests ever passed.

See #28706.

[ziodave]

I traced down the issue to this line being removed:

define( 'WP_INSTALLING', true );

More details:

===================================================================
--- includes/install.php    (revision 28964)
+++ includes/install.php    (revision 28965)
@@ -9,7 +9,6 @@
 $config_file_path = $argv[1];
 $multisite = ! empty( $argv[2] );

-define( 'WP_INSTALLING', true );
 require_once $config_file_path;
 require_once dirname( __FILE__ ) . '/functions.php';

[SergeyBiryukov]

See [28965].

[ziodave]

In order to complete the information for the ticket, the procedure to enable plugins unit tests is described here:
​https://github.com/wp-cli/wp-cli/wiki/Plugin-Unit-Tests

[ziodave]

I published the patch file here:
​https://gist.githubusercontent.com/ziodave/10fdc7ebfb2288be2876/raw/2649d21d4255e0487ae9c19afaaa9263ebb14999/r28965-patch

I currently apply the patch in the install-wp-tests.sh script after svn co --quiet http://develop.svn.wordpress.org/trunk/tests/phpunit/includes/.

(​https://github.com/wp-cli/wp-cli/blob/master/templates/install-wp-tests.sh, line 45)

[jdgrimes]

It seems that for some reason when WP checks is_blog_installed(), it is returning false. I'm not able to reproduce this. It seems to me that it might be an issue related to caching. Is some kind of persistent caching enabled during the tests?

[danielbachhuber]

Could be related to ​https://github.com/wp-cli/wp-cli/pull/1076

[jdgrimes]

@ziodave — What is the output of, e.g., var_dump( wp_cache_get( 'is_blog_installed' ) );?

Could you try running that somewhere before WP runs wp_not_installed()? (Or, just add it to right before wp_not_installed() in wp-settings.php.)

[ziodave]

bool(false)

[ziodave]

Replying to danielbachhuber:

Could be related to ​https://github.com/wp-cli/wp-cli/pull/1076

I don't think the issue is related with wp-cli. The failing code is in WordPress svn:
​http://develop.svn.wordpress.org/trunk/tests/phpunit/includes/

You should be able to reproduce it running the script at ln 53 of bootstrap.php.

[wonderboymusic]

In 28988:

Bring back define( 'WP_INSTALLING', true ); in phpunit/includes/install.php, which was removed in [28965]. tl;dr = wp-cli. Unit tests still pass in multisite because of the skips I added.

Fixes #28738, #28706.