id	summary	reporter	owner	description	type	status	priority	milestone	component	version	severity	resolution	keywords	cc
20057	Media upload for multi-webserver setups introduces a nasty race condition that could corrupt uploaded files	archon810		"I am in the process of scaling a Wordpress blog with several million monthly pageviews, and I have designed a plan to scale it in the highest availability manner possible.

'''The setup'''
Each web server that hosts HTML/PHP, static files, and uploads will be replicated via rsync from time to time and sitting behind a load balancer. Rsync will run every 5min to 1 hour, and to mitigate the 404s in the uploads, I put together an nginx setup that automatically tries a different upstream server in its configuration when it encounters 404s. This allows any web server to go down at any time, and the system to run as if nothing happened. This also gives me freedom for rsyncing periodically rather than immediately and avoids 404s completely.

'''The problem'''
Now, the bug (note, I'm using Windows Live Writer which automatically names uploads image.png, but I could see this potentially happen without WLW too since WP seems to automatically name files on disk in case of collisions).

Let's say we have server A and B. The site name is foo.com. Let's also say B is out of date by an hour and a bunch of files got uploaded to A that aren't on B. Say, A has image.png and image2.png and B has only image.png.

Now, the issue is that if the load balancer directs the new post uploader (the new post contains a single image) to server B, the file that it will create on disk will be named image2.png rather than image3.png. So now B will have a file that's different from A's but is named the same way.

The main problem is that the file name is given based on what's available on the disk rather than according to the database. It's easy for this race condition (between rsyncs) to destroy the integrity of the files. Furthermore, and I've experienced this first hand), if you delete the attachment from the Wordpress UI, it could actually delete the wrong file from the wrong server as a result.

Seeing this, I can't continue with my scaling plan until file names are assigned by the database rather than the file system or I figure out how to mitigate that. I don't want to force all writers to use only server A for uploads using its direct IP, as I want HA (high availability).

Thank you."	defect (bug)	new	normal	Awaiting Review	Media	3.3.1	major			admin@…