![]() |
Big Boarders & Backups..
My last few projects have been with boards that fit the big board description. Hundreds of thousands of members, millions of posts, and tons of activity.
Usually I am hired on to help with fixing issues with plugins, writing custom plugins, optimizing, etc.. My last three clients, in addition to other things, all wanted consultation on backing up data. For the most part, they all had in common one feeling and that was that they had little to no faith in their dedicated hosting provider when it came to the backups. Either they did not understand it, or were somehow discouraged when they tried to use it last. My last/current client actually bought me a drive and begged me to backup his data for him (for a fee of course).. Well, I have always had my own system for backing up my servers which allows me to have incremental backups so you could have hourly/weekly/monthly/yearly backups. Using rsync and hard links, I have multiple backups without using a ton of storage. Basically if something happened to my data, I could get back the last hours version of it. So I set him up with the same thing and he was ecstatic.. Gave him FTP access to the server where I stored his backups (using the drive he bought :) ) and he could see, and download all of his files at any interval.. Got me thinking about the demand for such a thing. Is this common with big boarders here? What would happen to you tomorrow if your servers hard drive(s) suffered some ill fate (power surge, water damage, dead drive, etc..)? Are you confident in your backup strategy? The reason why I ask is, if there is a demand out there for it, I might consider running a small service to do it, but before I even bother, I thought of asking here first.. |
I'd be interested in such a service, but depending on how much it would cost would be a factor, because my dedicated server is costing me enough already.
|
I doubt many want a backup service that is not located at the same physical place or at least have a direct highspeed network connection to such a location.
I would not have to push my backups over the regular internet conneections. |
Funny, I always thought the idea of having a remote backup a selling point. If say that same physical place caught on fire and burned down what good would having a backup system be there?
My backups transmit over the regular Internet yes, via SSH.. I haven't even considered a price.. |
Obviously, the big issue with big-boards is regular backups to the mysql db. For me, mysqlhotcopy took too long to use frequently. I ended up having to go with binary logs and weekly mysqlhotcopies.
arn |
A physical remote location is good in case of fire etc., but you should ensure that there is a fast direct connection between the 2 locations.
|
Ive noticed its become common for a hosting company to have a backup on remote these days.
|
Quote:
|
I can understand that.. Many of the dedicated solutions I have seen include some sort of backup service.. Mine would be compliment that backup and offer some level of redundancy..
Thinking of prices, I was contemplating about $100/mo to backup 25gb of data... Opinions? |
I do a full mysqldump of the db once a day at the least busy hour and the .tgz is rsynced to another server 3000 kilometers away.
|
Quote:
|
Quote:
|
Quote:
|
It doesn't look bad, although I only see daily backups. Doesn't appear to cover hourly, weekly or monthly.. Could just be me.. In fact, it only appears to have one backup.. The daily one..
The samba share feature is a good idea (provided it is read only).. However, I could see people abusing the "No data transfer limit" feature in combination with the share. You could basically use the share to serve large files (i.e. videos). |
Why pay money when you can do it yourself, like the pro's?
Just use rsync, it is the only method that will not break your busy servers while doing a backup. What do you think they use when they do your backup? Heh... Edit: Check what bqinternet.com uses as procedure. Features: Daily backup using rsync technology SSH encryption On-the-fly compression Get back online instantly with a Samba mount FTP access No data transfer limit No setup fee |
If you peruse the mods forum, you can see how technically apt your average Vbulletin owner is. Asking them to handle rsync would be an interesting follow..
|
Quote:
|
If the average vBulletin owner is not capable to perform a rsync process, then the above listed service will come handy to them.
However, you have to look at rsync process like you would copy files from a folder to another... really easy. Setting up your own local rsync mirror is just a matter of configuring the rsyncd daemon to make your portage directory available for syncing. That's all. :) |
Backing up however needs to be more than just mirroring the site. Lets say, for whatever reason, config.php is wiped out, or corrupted. The mirror server will just copy it, or delete it I suppose. Or a file gets hacked, it will get copied to the mirror.
By using incremental backups with hard linking, you can keep snapshots on the backup server to allow for rollbacks to certain times (hourly, daily, weekly, monthly).. I guess what I am saying, is rsync alone is not all that helpful. |
Quote:
|
Quote:
|
Quote:
Only a few resources (out of many available): Mike Rubel's rsync snapshots - the original rsync script rsnapshot - based off Mike Rubel's article, my preferate Dirvish - the most feature-filled of these programs Output on rsnapshot: Code:
[root@localhost /]# rsnapshot -v daily Code:
[root@localhost /]# rsnapshot du example.com/home/ Have your pick, or if you are not happy with this solution, simply use rdiff-backup. It is based on rsync and stores incremental rdiff deltas with the backup, with which it is possible to recreate any backup point. You problem is solved in 2 seconds. :) Compared to rdiff-backup, rsync is faster, so it is often the better choice when pure mirroring is required. Also, rdiff-backup does not have a separate server like rsyncd (instead it relies on ssh-based networking and authentication). However, rdiff-backup uses much less memory than rsync, on large directories. Quote:
|
Actually, I have done custom work for quite a few big boarders who barely had knowledge of FTP, much less the terminal, negating both points I think.
|
That corresponds with my above statement: "you have money to hire an admin..."
In other words, you were hired to do a backup by the big board owner. If I have money, I would never bother to learn all those "rsync thingies", I will hire you to do it for me. I will spend my time to get more clients who will advertise on my board, because I'm a business guy, not a Linux lover. :) |
I wrote a script that shuts down Apache and MySQL each night, does an rsync of the DB directory to a backup directory on the server, restarts Apache & MySQL and then rsync's the backup directory to dedicated backup server. I maintain an encrypted tunnel between a server at home and the firewall of my remote network and rsync the backup directory of the backup server to a system at my home nightly. I also run a mysqldump of the DB once a week and rsync it to the backup server and home. There is no site without the data and I've been doing this long enough to fear and respect Murphy's Law. Eric
|
There is not need to shut down your server. You are basically deny access to users during that backup time. :)
Use rsnapshot, it is transparent... and it does incremental backups hourly. |
I can understand shutting down httpd/mysqld I guess, when you are backing up the database binaries, myself though, I just do an automated mysql dump and let rsync back up the dump file.
|
Quote:
TecK, I haven't heard of rsnapshot. I'll give a look. Thanks for the tip. Eric |
My pleasure, Eric.
link: http://www.rsnapshot.org/ screenshots: http://www.rsnapshot.org/screenshots.html |
All times are GMT. The time now is 03:53 AM. |
Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
![]() |
|
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|