Attachments

[Kevlar]

How do large boards handle attachments? Right now I have about 7GB of attachments in my file system. I was just curious how others deal with this issue. Especially if you have more than one web server... :confused:

[Xenon]

We have about 1.5 GB in the DB, but i do get the feeling, that we should change that.....

during the normal process that's not a problem but during upgrades that table is most of the time responsible for the timeouts

[Erwin]

I have attachments on a separate server with some minor hacks to the attachment php files and templates.

[Paul M]

Anyone with more than a few hundred MB of attachements should move them out of the DB and into the file system. Personally I would always use the file system anyway, I don't think attachments belong in the database (the only time it's useful to have them in the DB is when moving server)

[Kevlar]

I have attachments on a separate server with some minor hacks to the attachment php files and templates.

I guess... the next question would be, how did you come across those minor hacks to allow you to run the attachments from a different server? :surprised:

[AdminNation]

I guess... the next question would be, how did you come across those minor hacks to allow you to run the attachments from a different server? :surprised:

Given some of the hacks he's released, its pretty safe to assume that he meant that he figured out how to mod the files accordingly. There is no publically released hack that tells you how to do it.

[Trigunflame]

<a href="http://mysqldump.azundris.com/archives/36-Serving-Images-From-A-Database.html" target="_blank">http://mysqldump.azundris.com/archiv...-Database.html</a>

This is a very good read for anyone that would like more "exact" information on why Files stored in Database for a busy site is not a good idea; as well as a compromise.

Keep the master storage in the Database, while providing a local cache in the file system.

[Erwin]

I guess... the next question would be, how did you come across those minor hacks to allow you to run the attachments from a different server? :surprised:
It's actually easier than it looks - make sure subdomains are included in the cookie, then have an attachment server that runs a strip down vbulletin with just attachment.php, global.php and some include files, modify attachment.php, modify the attachment templates to point to the subdomain for the attachment server, and you're done.

[Paul M]

It's actually easier than it looks - make sure subdomains are included in the cookie, then have an attachment server that runs a strip down vbulletin with just attachment.php, global.php and some include files, modify attachment.php, modify the attachment templates to point to the subdomain for the attachment server, and you're done.I would be interested to know how that is covered by the licence since you are running two copies of vb ?

[who is chris c]

well if the forum powered it all (on the backend) it wouldnt be a violation it still would be 1 copy (just not in the directory structure they supplied)

[Kevlar]

It's actually easier than it looks - make sure subdomains are included in the cookie, then have an attachment server that runs a strip down vbulletin with just attachment.php, global.php and some include files, modify attachment.php, modify the attachment templates to point to the subdomain for the attachment server, and you're done.

Thanks for the insight... I'm going to have to try this today and see if it helps lessen the load on my primary server (as it's currently buckling).

[Erwin]

I would be interested to know how that is covered by the licence since you are running two copies of vb ?
I run a few webservers with multiple copies of vB. :) But they are within the same network and for the same site - Jelsoft allows that.

[GrCh_Jeff]

Please explain how I would do this legally? Thanx.

[kerplunknet]

Please explain how I would do this legally? Thanx.
Only one copy can be available to the public, for public use.

[GrCh_Jeff]

would you explain this please?
But they are within the same network and for the same site - Jelsoft allows that
I want to somehow run a second server same members same site name linked to the present server for storage and upload/download uses.storage for the site.

[Stargesicht]

It's actually easier than it looks - make sure subdomains are included in the cookie, then have an attachment server that runs a strip down vbulletin with just attachment.php, global.php and some include files, modify attachment.php, modify the attachment templates to point to the subdomain for the attachment server, and you're done.
Can you post a step by step way to do this????
thanks

[scotty]

Can you post a step by step way to do this????


i've just implemented it and here is my way:

AdminCP => Options => Cookies => Cookie-Domain
set it to .yourdomain.com - with the two dots!!!

setup a subdomain called attachment.yourdomain.com
(in my case this is located on the same server as my master-webserver, so I don't have to deal with the attachment path)

copy the following files in the root:
attachment.php
global.php
newattachment.php

copy the following files in root\includes:
class_core.php
class_dm.php
class_dm_attachment.php
class_hook.php
class_image.php
class_upload.php
config.php
functions.php
functions_editor.php
functions_file.php
functions_forumlist.php
functions_log_error.php
functions_misc.php
functions_newpost.php
init.php

go to the templates section of your board:
replace all occurrences of
attachment.php
with
http://attachment.yourdomain.com/attachment.php

and

replace all occurrences of
newattachment.php
with
http://attachment.yourdomain.com/newattachment.php

last step:
add a replacement variable to all your styles:
http://www.yourdomain.com/
with
http://attachment.yourdomain.com/

this last step is for the "attach" tag

delete all your cookies (maybe via filesystem) and test the installation.

have fun :)

[Stargesicht]

Thank you .
I test it tomorrow

But:

(in my case this is located on the same server as my master-webserver, so I don't have to deal with the attachment path)
Whats the way when there two different servers (with different IP)???

[scotty]

Whats the way when there two different servers (with different IP)???
you must have a look at your attachment (absolute) path: this must match the path on your attachment server and NOT on your webservers.

example:

your webservers directory structure is:
/home/users/forum/htdocs/www for the php files
/home/users/forum/htdocs/attachments for the attachments

but your attachment server has the following structure:
/home/sites/attachments/htdocs/www for the php files
/home/sites/attachments/htdocs/attachments for the attachments

than is the last path the correct one for the CP.

[zooki]

hi currently i am hosting on a friend's server. He has given me some space and i also have a server with the shared host Servage.
Re: Scotty's Server Attachments

At the moment my website is on the server. at domainName.COM

I wish to have my attachments on the Servage Server. as it has a lot of diskspace and bandwidth.... Perhaps under my domainName.NET address. Understand?

If there are vbulletin license issues, i am willing to purchase a second copy. Although i have nothing at all at the .net address.. it will be simply to host the files. for my website.

What changes would i need to make to the cookies and the other settings to implement this? If you can help me out Scotty or anyone else, it would be really great and special thanks.

[G2place]

I have a similar request....

I have our website on a server.... Great.
Presently it is storing attachments in the database.... Not so great.

So, I want to use the vBulletin Attachment Manager to store them elsewhere.
To merely put them in a different folder on the forum server is easy....
I modified the Attachments to say
"Attachments are currently being stored in the filesystem at ../files"
(note... this is the sub-dir I created with appropriate permissions).
This works great.

So, let's move them to a Different server.
I have a different server and can access it via IPaddress.
It has a folder under the root called /files, with all the appropriate permissions.
So I change the Attachment manager to say
Attachments are currently being stored in the filesystem at
http://xxx.xxx.xxx.xxx/files
No sir... it just doesn't like this... the error message I get is

"We attempted to write a test file to http://xxx.xxx.xxx.xxx/files, but we were unsuccessful. Please verify that the web server has write/delete permissions for this path."

For the life of me I can't figure out what I am doing wrong.

Any help ?????

[orban]

http:// isn't a protocol to write files. I think you have to code your own attachment solution to store them on a seperate server.

[deegan]

Make sure you copy those attachment files to your PC just in case...

I accidentlly lost my smilies moving to a new server...Thankfully, I backed up the /images & /gallery directories to my pc beforehand...took awhile, but worth it...

[Snatch]

But, they way to move attachments to a another Server with link the attachment.php to the new Domain ist not the best way. For example, what is if the extern server also full?
The best way is, to modify the Forum so, that it stores the Real-URL to The images like: www.forums.de/imagesdir/image.jpg than you can use many HDDs and many Servers.

But I don’t now how can I do this

Whit the way from “scotty” I can copy al my attachments to the Upload Server, that’s great. But what is, if the HDD full an I but in a new one? Than I have the same Problem as now.

GreeTz
Snatch

[netwind]

I use light frontend http server which serve attachments/images and backend server which serve php.
attachment.php with simple hack points frontend where get attachement body and this work. client not give direct link anytime.

[DJ XtAzY]

Currently i have 14gigs of attachments..... on a single server.

[scotty]

The best way is, to modify the Forum so, that it stores the Real-URL to The images like: www.forums.de/imagesdir/image.jpg be carefull: use this method only, if you have no permissions on your attachments! the main advandtage to use attachment.php is, that you have full control over the rights to download depending on used forum & user groups!

Whit the way from ?scotty? I can copy al my attachments to the Upload Server, that?s great. But what is, if the HDD full an I but in a new one? Than I have the same Problem as now.than you'll need an attachment server with a bigger HDD. Pack your attachment into a gzip/tar, copy per FTP directly to the new server and put it in the same directory structure, as the old one. I've done this a few times when moving to bigger hardware.

[Snatch]

be carefull: use this method only, if you have no permissions on your attachments! the main advandtage to use attachment.php is, that you have full control over the rights to download depending on used forum & user groups!

than you'll need an attachment server with a bigger HDD. Pack your attachment into a gzip/tar, copy per FTP directly to the new server and put it in the same directory structure, as the old one. I've done this a few times when moving to bigger hardware.

Yew, but I have 240GB of Attachements. The Max HDD Size is about 300GB. So this way is not the right for me.

[woodysfj40]

Max SCSI HDD size is about 145GB.......in Raid10, you are stuck at 278.47 in useable space....if you limit to 4 drives....

four 7200rpm 750GB SATA drives in a Raid10 configuration results in a useable space of 1430.51GB

should be sufficient....

[Snatch]

I mean the Max HDD size on my Server.

[Klopek]

you must have a look at your attachment (absolute) path: this must match the path on your attachment server and NOT on your webservers.

example:

your webservers directory structure is:
/home/users/forum/htdocs/www for the php files
/home/users/forum/htdocs/attachments for the attachments

but your attachment server has the following structure:
/home/sites/attachments/htdocs/www for the php files
/home/sites/attachments/htdocs/attachments for the attachments

than is the last path the correct one for the CP.


That doesn't work. The CP doesn't allow to set a path that doesn't exist on the server (but it exists on the attachment server). How can I set it otherwise ?

[dave6901_2000]

Has anyone ever got this to work with 2 different IPS ??
I have one server in HK and the other in the USA I wish to have my USA Server handle all my attachments and images

[Jafo232]

Curious, but why not mount a network drive as the Vb attachment directory?

I have a server on my network that I just use for backups. Basically all the other servers have it mounted in a directory named /backup. All I have to do is perform tapes on the backup server instead of all of them.

I don't see why the same couldn't be done for attachments/avatars..

[amcd]

Jafo232, there can be many reasons. Your suggestion about network storage is good if the issue is disk space shortage, but there can be many other issues/objectives.

It may be desirable to run the forums on a multi-carrier, BGP network which costs much more (touted by hosts as premium bandwidth) and the attachments on a lower cost network (value or volume bandwidth).

A big board may want to reduce load on its main webserver by offloading some functions.

A board using shared hosting does not have the option of network mounts, but it can use multiple hosting accounts quite easily if vbulletin is capable of distributed functioning.

Just a few. Hope you get the idea.

[dave6901_2000]

My main goal is to reduce server load and also save on B/W my board holds 1000's of Attachments
My US server I have Tons of room and B/W to spare

[Zi55]

i've just implemented it and here is my way:

AdminCP => Options => Cookies => Cookie-Domain
set it to .yourdomain.com - with the two dots!!!

setup a subdomain called attachment.yourdomain.com
(in my case this is located on the same server as my master-webserver, so I don't have to deal with the attachment path)

copy the following files in the root:
attachment.php
global.php
newattachment.php

copy the following files in root\includes:
class_core.php
class_dm.php
class_dm_attachment.php
class_hook.php
class_image.php
class_upload.php
config.php
functions.php
functions_editor.php
functions_file.php
functions_forumlist.php
functions_log_error.php
functions_misc.php
functions_newpost.php
init.php

go to the templates section of your board:
replace all occurrences of
attachment.php
with
http://attachment.yourdomain.com/attachment.php

and

replace all occurrences of
newattachment.php
with
http://attachment.yourdomain.com/newattachment.php

last step:
add a replacement variable to all your styles:
http://www.yourdomain.com/
with
http://attachment.yourdomain.com/

this last step is for the "attach" tag

delete all your cookies (maybe via filesystem) and test the installation.

have fun :)
First thanks too much scotty for the way
Did anyone try this way in seperate attachment server ?
and are it have any bugs etc .. ?
Thanks ,
Zi55

[Leo Renard]

I always have attachments disabled. If anyone really needs to upload anything, they can at my uploading site (Not on the same server)

[Nitro212]

but that method only works for storing attachments on the db? what if i want to store them on file system in another server.. will that still work?

[ferreo]

What you can do is to move your attachments to a separate server in a file system and point the vB install to a NFS mount directory that is local to the web server(s). I have it done that way and it works like a charm.

[Mickie D]

but that method only works for storing attachments on the db? what if i want to store them on file system in another server.. will that still work?

yes it does work with file system in fact its only for file system attachment storage

i have this working and its doing really well but as i have posted on vbulletin.com there is a few cookie issues i am experiencing but i will find the reason sooner or later.

but it works 100%

[heeroyun]

How can I store the attachment on the file system instead of the db ?

[ferreo]

How can I store the attachment on the file system instead of the db ?

vB Admin Control Panel - Attachments - Attachment Storage Type - Move Items from Database into File System

--------------- Added 1205777228 at 1205777228 ---------------

First thanks too much scotty for the way
Did anyone try this way in seperate attachment server ?
and are it have any bugs etc .. ?
Thanks ,
Zi55


I am going to try this out today and will let you know if I succeed.

[Zi55]

Anyone fix cookies problems ??

[Golzarion]

well, If I want to move attachments to the folder instead of database ..... I have to make a folder change mode 777 and also I have to disable safe mode ...

Aren't these dangerous ? or risky ?

how can we secure folder ch mode 777 ?

can we have this folder on root ? or secure it by using htaccess file?

[amcd]

You can have the attachments folder outside your webroot. It doesn't really have to be chmod 0777. Just make sure the webserver/php user has full access to it.

[valdet]

This is a very helpful thread. Thanks a lot guys.