Opened 10 years ago
Last modified 4 years ago
#31092 new defect (bug)
travis-ci builds skip lots of tests due to environment configuration
Reported by: | boonebgorges | Owned by: | |
---|---|---|---|
Milestone: | Future Release | Priority: | normal |
Severity: | normal | Version: | |
Component: | Build/Test Tools | Keywords: | needs-patch |
Focuses: | Cc: |
Description
See #30284.
travis-ci builds (eg https://travis-ci.org/aaronjorbin/develop.wordpress/jobs/47713725) are skipping a bunch of tests. I haven't yet looked into it in detail, but I'm guessing it's a combination of the following:
- Missing image-related PHP extensions (gd, exif)
- ru_RU system domain must be installed (
Tests_DB::test_locale_floats()
) - MySQL versions don't support utf8mb4
- WordPress Importer plugin not available
Ideally, the only tests skipped in CI are those that depend on *WordPress* configurations, and we run separate builds for each relevant config. In the case of our current tests, the only WP config that matters is multisite, and we're already running separate multisite tests, so we should expect to see that some tests are skipped on MS, and others skipped on non-MS. All OS, PHP, MySQL, and webserver-related skips should be eliminated.
Anyone who's interested in chasing this down, here's how I'd go about it:
- Fork https://github.com/aaronjorbin/develop.wordpress and set it up to run on Travis.
- Modify the build so that it runs
phpunit -v
. I think you should be able to do just by changing thescript
command in .travis.yml - Check in and push up. The first Travis run should give you a full report of what's being skipped, and why
- Start hacking at the
before_script
section as necessary to get the necessary prerequisites installed. http://docs.travis-ci.com/user/languages/php/#PHP-installation and the following sections should be a good starting point.
Change History (4)
#2
@
10 years ago
Related: #30462 Test multiple MySQL versions on Travis (And utf8mb4 is in there as a bonus)
#4
@
4 years ago
Was scrubbing Build/Test Tool tickets and found this one. I think that the local Docker environment (see #47767) has solved a number of the configuration issue resulting in skipped tests.
For example, the Docker containers now contain GD, EXIF, and the ru_RU domain.
- #49720 was created for the WordPress Importer plugin issue.
- The
utf8mb4
issue has actually flipped. The tests currently skipped requireutf8mb4
support to be missing.
I also checked the codebase and opened #51737 for the only instance of version_compare()
checking for a specific PHP version.
Not going to close this out just yet, but a lot has changed to the test environment used and we'll just need to compare/update the things that need to be addressed.
https://travis-ci.org/aaronjorbin/develop-wordpress-verbose/builds/47830269 is up and running tests verbose right now.