External Attachment Storage

[SaintDog]

I know this has been asked before and generally points to a specific release here at vBulletin.org, however I am looking for a simple modification that will allow me to do the same without having to integrate a different download/attachment system.

Has anyone been able to successfully have their attachments ported to a seperate server that would mind sharing the code or Ext/Plugin to do so?

[Zachery]

I know that erwin had done it in the past, there are possibly several ways of doing this. Either ftping files to a remote server during the upload, or rysyncing the attachments folder to the remote server. Then uploading nessary files to the remote server to process things, like attachment.php global.php config.php etc.

[SaintDog]

I knew that Erwin had accomplished this, however I wasn't sure if anyone had made such coding public as of yet. It'd be a great feature to have as a side option or even in future versions of vBulletin.

I may have to take a look at those files and see what I can come up with, though if anyone else has already completed this task, I'd be very interested to speak with them and see how they accomplished this.

[Zachery]

Any reason you want to store files externally?

[SaintDog]

Originally Posted by Zachery

Any reason you want to store files externally?

Specifically, to lessen the load on a single server and off-load it to another. While on new communities, it wouldn't mean much, though on more active communities it could drop the load. The less activity specifically placed to a single server, the more speed available for other comsuming resources and queries.

Overall, to save CPU cycles, save memory and drop the load altogether. Why strain the server any more than it needs to be when you could simply offset the load to another if you have it available?

[Zachery]

You would probally notice a better preformance increasae by offlodaing all of your static files to an external server, or even another services. lighttpd is a great server, I've moved 95% of the static content on one of the sites i help run to an lighttpd server and the server was able to survive the E3 barrage, both the apache machine and the image/content servers.

[SaintDog]

Seperating the images & attachments from the static PHP files to an external server would be doing just that, however, remaining on an Apache based system.

The static files themselves do not force a load on the server, however utilizing and storing the attachments to the database or by using the file system does when there is such high usage to these areas of the community.

Offsetting the load of the attachments and images specifically to another server will reduce the load since a great deal of the activity is there.

This is not so much for your average community with posts, threads and users online, it's for an approach using vBulletin more as a CMS and download-specific tool.

[Zachery]

You'd be suprised how big of a difference offloading the other static content (not attachments) makes. We went from loads of 5,10,20, duruing preE3 week to 1~ the entire week of E3 while our servers were hammered. (we had more traifc after E3 as well). Apache is not the greatest static file server when you've got 4000 people trying to download large files.

[SaintDog]

With the current setup, everything resides on the same server and while the loads are not high by any means, there's room for improvement and seperating that which gains the most traffic is the best starting point. In this case, it's the images and attachments.

The other static files simply aren't hit enough, or even used as much to warrant moving them to their own server. It's kind of the opposite end of the spectrum.

Most people use vBulletin as a community software, in this case, more of a download and image manager.

[Zachery]

But you are ignoreing the fact of how often images are called, even if cached. Theres quite a few images here that apache would not have to deal with serving. Since i left your hosting I've moved from my own hosting company to consulting for lots of bigboards now and have been doing fieldtests of differnt methods to improve website preformance under high usage loads. I'm currently looking into not only clustering but also in general spacing systems to do specific tasks and working on finding programs that better accomplish tasks. Apache is a decent program for processing php, but lighttpd is doing a much much better job serving static content (images etc) and I'll be testing how it does with php in the naer future. However there are already a few very large boards who've made this transition and its been wonderful.