CURLOPT_BUFFERSIZE can corrupt downloaded file

[nextendweb]

One of our user has a website who tried to update our plugin through WordPress update system. Our update url points to our private server, so it is not wordpress.org related. We had tens of thousands updates for this very same file and this is the only site which had this issue, so I think it is not related how our server serves the file

The error is md5_mismatch: The checksum of the file (4a3b968a44c585a2883e687d61c251fb) does not match the expected checksum value (231732259d67fe83ed6fc02d7ad9be57).

I started to debug this issue, I had the original zip file and the corrupted one which WordPress downloaded. I diffed the files and at some point it seems like that different parts are mixed up. Like the wrong stream part got inserted at some point. And it happens all the time.

<?php

require_once __DIR__ . '/wp-load.php';
require_once ABSPATH . 'wp-admin/includes/file.php';

$url = "https://***********";


$download_file = download_url( $url, 300, false );

var_dump($download_file);

For this download Curl is used, so I head over to /wp-includes/Requests/Transport/cURL.php

And when I removed CURLOPT_BUFFERSIZE or changed anything else (Requests::BUFFER_SIZE+1, Requests::BUFFER_SIZE-1), download_url function started to behave as it should and there was no md5 error anymore.

<?php
curl_setopt($this->handle, CURLOPT_BUFFERSIZE, Requests::BUFFER_SIZE);

Site info:
WordPress 5.8.1
PHP 7.4.24
Curl 7.71.0
LiteSpeed V7.9

[nextendweb]

Exactly one 1160 bytes block is missing in the middle of the file, which is the value of Requests::BUFFER_SIZE.

[nextendweb]

We had another user with the very same error, so I created a PHP bug report as it seems like its an issue under the hood: ​https://bugs.php.net/bug.php?id=81516

[nextendweb]

It's clear now that it is not a bug in WordPress. It looks like it's related to a security module called i360 developed by ​https://www.imunify360.com/ which identifies the chunk as a threat and does not write that chunk onto the disk.

So I think we can close this issue.