[Make Money?]

Thanks for that SoftDux

I got these additional suggestions too

Is anyone out there using any of these methods with good results?

1) "sync is probably not viable - for large numbers of files, there's a big CPU and IO hit when you do the sync. Too much to run frequently enough to avoid round-robin problems. There are other tools that are better at synchronizing multiple servers, but I'm not aware of a good real-time option.

If you do find or build a fast sync tool, you could session affinity at the load balancer to keep a user on their web head for a particular period of time (5 minutes is standard for HTTPS connections - same approach is fine for HTTP too). This would avoid the short-term "where'd my file go" problem, but there's still a risk of someone viewing a page before the files hit their webserver.

2) Another approach is to stick all the files on your database box and have it export them to the web heads via NFS. NFS has its issues, but this is pretty simple to set up, especially over your servers private network. Still a single point of failure, but hopefully your database box has RAID to be a bit more on the safe side.

3) You could go completely external and upload your files to Amazon S3. It would be pretty simple to set up a local caching proxy that keeps frequently-requested files on local disk. I'd be surprised if someone isn't working on this for Vbulletin. Big benefit is the reliability of S3.

There's also low-level disk replication, like DRBD. You'd need a spare disk in each web head that would be synced in real-time between boxes. A bit harder to set up, but would definitely handle this and probably work pretty well over a gigabit private link.

The last general class of fixes are distributed filesystems of various ilks. From GFS and Lustre all the way down to MogileFS (not a real filesystem, but a fantastic file synchronization handler). If someone wrote MogileFS hooks into Vbulletin, that would be a great option. The others I think are just too much of a headache to set up on a small set of servers."