ImageMagick PDF test failures

[SergeyBiryukov]

Background: #48145, #50466.

After [48329], there are ​test failures on Travis that appear to be unrelated to the commit:

There was 1 error:

1) Tests_Image_Functions::test_pdf_preview_doesnt_overwrite_existing_jpeg
Undefined index: sizes

/var/www/tests/phpunit/tests/image/functions.php:620

--

There were 3 failures:
1) Tests_Image_Functions::test_wp_generate_attachment_metadata_pdf
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
-Array &0 (
-    'sizes' => Array &1 (
-        'full' => Array &2 (
-            'file' => 'wordpress-gsoc-flyer-pdf.jpg'
-            'width' => 1088
-            'height' => 1408
-            'mime-type' => 'image/jpeg'
-        )
-        'medium' => Array &3 (
-            'file' => 'wordpress-gsoc-flyer-pdf-232x300.jpg'
-            'width' => 232
-            'height' => 300
-            'mime-type' => 'image/jpeg'
-        )
-        'large' => Array &4 (
-            'file' => 'wordpress-gsoc-flyer-pdf-791x1024.jpg'
-            'width' => 791
-            'height' => 1024
-            'mime-type' => 'image/jpeg'
-        )
-        'thumbnail' => Array &5 (
-            'file' => 'wordpress-gsoc-flyer-pdf-116x150.jpg'
-            'width' => 116
-            'height' => 150
-            'mime-type' => 'image/jpeg'
-        )
-    )
-)
+Array &0 ()

/var/www/tests/phpunit/tests/image/functions.php:466

2) Tests_Image_Functions::test_crop_setting_for_pdf
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
-Array &0 (
-    'sizes' => Array &1 (
-        'full' => Array &2 (
-            'file' => 'wordpress-gsoc-flyer-pdf.jpg'
-            'width' => 1088
-            'height' => 1408
-            'mime-type' => 'image/jpeg'
-        )
-        'medium' => Array &3 (
-            'file' => 'wordpress-gsoc-flyer-pdf-300x300.jpg'
-            'width' => 300
-            'height' => 300
-            'mime-type' => 'image/jpeg'
-        )
-        'large' => Array &4 (
-            'file' => 'wordpress-gsoc-flyer-pdf-791x1024.jpg'
-            'width' => 791
-            'height' => 1024
-            'mime-type' => 'image/jpeg'
-        )
-        'thumbnail' => Array &5 (
-            'file' => 'wordpress-gsoc-flyer-pdf-116x150.jpg'
-            'width' => 116
-            'height' => 150
-            'mime-type' => 'image/jpeg'
-        )
-    )
-)
+Array &0 ()

/var/www/tests/phpunit/tests/image/functions.php:531

3) Tests_Image_Functions::test_fallback_intermediate_image_sizes
The `test-size` was not added to the metadata.
Failed asserting that false is true.

/var/www/tests/phpunit/tests/image/functions.php:572

This appears to be caused by wp_generate_attachment_metadata() failing to generate the metadata for PDF files, due to wp_get_image_editor() returning a WP_Error object:

 ["errors"]=>
  array(1) {
    ["invalid_image"]=>
    array(1) {
      [0]=>
      string(115) "attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/408"
    }
  }
  ["error_data"]=>
  array(1) {
    ["invalid_image"]=>
    string(29) "/tmp/wordpress-gsoc-flyer.pdf"
  }
}

It's not quite clear why this suddenly started failing now, I guess something has changed in the environment recently.

Props @enrico.sorcinelli for helping with debugging here.

[SergeyBiryukov]

In 48338:

Tests: Add some assertions to PDF tests for better debugging.

Props enrico.sorcinelli.
See #50573.

[SergeyBiryukov]

It's interesting that the tests ​fail on PHP 7.1+ and pass on 5.6 and 7.0.

[azaozz]

Seems caused by upgraded ImageMagick or Postscript on the server. Given the very last moment these errors appeared, thinking the tests can be bypassed for now. Then get them fixed (if at all possible) and enabled for beta-2.

[SergeyBiryukov]

In 48341:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

See #50573.

[SergeyBiryukov]

In 48366:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 5.4 branch.

See #50573.

[kirasong]

Replying to azaozz:

Seems caused by upgraded ImageMagick or Postscript on the server.

<snip>

I agree -- probably ImageMagick.
I suspect that it is due to #45813, and we should be able to work around it by writing our own policy.xml.

As mentioned in the other ticket, it's probably also worthwhile to find out whether we can get the entry removed from the Ubuntu package now that the security issue has, as far as I'm aware, been resolved.

[SergeyBiryukov]

In 48483:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 5.3 branch.

See #50573.

[SergeyBiryukov]

In 48484:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 5.2 branch.

See #50573.

[SergeyBiryukov]

In 48485:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 5.1 branch.

See #50573.

[SergeyBiryukov]

In 48486:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 5.0 branch.

See #50573.

[SergeyBiryukov]

In 48487:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 4.9 branch.

See #50573.

[SergeyBiryukov]

In 48488:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 4.8 branch.

See #50573.

[SergeyBiryukov]

In 48489:

Tests: Temporarily skip PDF tests if they fail due to ImageMagick permission errors.

Merges [48338], [48341] to the 4.7 branch.

See #50573.

[SergeyBiryukov]

Closing in favor of #45813, let's continue there.

[kirasong]

I'm questioning whether this should be part of #45813, since it'd be good for core to be running Imagick tests whether or not the default settings are fixed upstream.

[desrosj]

It looks like the last chunk of [48489] has the incorrect variable. I'll be correcting this and backporting to the 4.7 branch as it blocks the progress of #48301.

[desrosj]

In 49523:

Tests: Fix an incorrect variable name in PDF tests.

This causes a PHP error when running the test suite with PDF rendering supported.

See #50573, #48301.

[desrosj]

In 49524:

Tests: Fix an incorrect variable name in PDF tests.

This causes a PHP error when running the test suite with PDF rendering supported.

Merges [49523] to the 4.9 branch.
See #50573, #48301.

[desrosj]

In 49526:

Tests: Fix an incorrect variable name in PDF tests.

This causes a PHP error when running the test suite with PDF rendering supported.

Merges [49523] to the 4.7 branch.
See #50573, #48301.