Ultimate Vbulletin Backup Guide || With or Without ROOT ACCESS with over 5GB Backups
 

Hey Guys,

I own a VBULLETIN , its kind of big one at this time.
Here are the stats as i write the guide :
Threads :: 979,845
Posts :: 1,159,042
Members :: 181,012

All of us know, BACKING up a big board or even a medium one is literally a big pain.
All of us need a system to backup forums to a place , of course AUTOMATICALLY.

I searched a lot of thing, tried everything possible to backup things.
But yes, i faced many issues like these :

1. Incomplete or Faulty backups due to database being continuously used by traffic.
2. The website is already loading the servers & with the MYSQL Being dumped makes the servers heavy leaving the sites inaccessible for longer time before it becomes normal.
3. If you do this manually means when the website is getting hits, you end up with a backup that many times wont restore properly.

Yes, it has happened with me so many times that i was fed up with it and finally decided to make a short working script for myself. Now, its confirmed working i am sharing it here with you guys.

Considering the comments , the script is now working :

1. With SSH LOGIN to Backup Servers.
2. Without SSH Login to Backup Servers
3. Can be RUN VIA CRON Setup, scroll down for instructions.

Mission :

We will backup the database locally and then finally back it up to the REMOTE BACKUP server.

With SSH LOGIN :

Requirements :

1. Dedicated Servers.
2. Root access of course.
3. A BackUp Server with root access.


Process :

As everything is to be done AUTOMATED, we need to set up the SOURCE SERVER & DEST SERVER to have PASSWORD LESS logins.

How to do that :

I prefer to get a TWO WAY SETUP Done cause sometimes You may need it.

Now, we move to Backups :

All of us know about the MYSQL DUMP Command, so here is the command that you can use directly :

backup_YourSiteName__`date '+%m-%d-%Y'`.sql
Backup SQL file with website name & date of backups.

So, this command will back up your database to the BACKUP directory & include your Website name & DATE of course you can change the file name as you want to.

Now, we need to take the backups in a way that :

1. It Does not OVERLOAD the servers.
2. It does not allow the DB access at all else the backups are ruined.

To solve this, we can close the forums , put the SITE UNDER MAINTENANCE Pages, BUT EVERYTHING failed for me.. I did all this, it gives me backup but of course loads the servers as the SERPS traffic keep on visiting the forums and you can not stop them at all.

So, i finally decided to let the FORUMS BECOME UNREACHABLE to take the backups & its a matter of minutes to back up things.
Now, to automate this thing, i just wrote a small script :

Now,what are these steps doing :

1. cd /path/to/your/backup/directory/
Goes to your backup directory.

2. cp -r /home/backup/* /home/backup_backup/
Copies the last done backup to ANOTHER BACKUP DIRECTORY, IN Case of my forums, i prefer this, you can leave it out.

3. service httpd stop
STOPS THE HTTPD Services leaving the sites inaccessible & REDUCING THE SERVER LOADS TO BARE MINIMUM.

4. rm ./*
mysqldump --user Your_DB_User_Here --password=Your_DB_PASS_here Your_DB_NAME_Here > /path/to/your/backup/directory/backup_YourSiteName__`date '+%m-%d-%Y'`.sql

Removes the OLD BACK in that directory, takes a new backup, stores the new backup as a file with DATE.
I DID NOT TAKE A GZIP backup as i have sometimes got CORRUPT BACKUPs with it, SQL files always worked for me, its your choice.

5. service httpd start
Starts the server httpd again, making the sites accessible.

6. scp *.sql root@xxx.xxx.xxx.xxx:/backup/

Now, we did the PASSWORD LOGIN SETUP ABOVE, this command copies your backup securely to the backup servers via SCP , so no worries.

Advantages :

1. The backups are always good & worked fine for me. I never had a faulty backup or corrupt sql file with this method.

2. The server loads NEVER SHOOT SKY HIGH as only MYSQLDUMP works at a time before the servers are open again.

3. Yes, the site becomes inaccessible for a couple of minutes in my case 1 Minute 20 seconds at present. But, its better then to show TIMED OUT ERRORS for 10 minutes in which SERVER LOAD is so high that the site lags badly.

4. Backups are safe in local storage as well as the REMOTE STORAGE.

WITHOUT SSH Login:

How Do I Setup a Cron Job To Backup Data Automatically?

Just add cron job as per your requirements:

How can i use the FTP CLIENT on this ..??

If you guys want to use FTP Clients for the back up & do not have ROOT access to set up SCP access, you can use this one :

1. Install the NCFTP Client.

Use yum or apt-get package manager to install ftp client called ncftp.

2. Replace the command for SCP with NCFTP :

Use ncftp command instead of SCP :

This will login via NCFTP Client , then move to your backup directory & use the MPUT command to dump the backup to the FTP & on completion will quit the session.


And about GRACEFULLY CLOSING THE Vbulletin :

If your VB gets around medium traffic , it can be tried but I will always prefer you to SHUT DOWN the HTTPD for just few minutes.

I know it will make the website inaccessible but will ENSURE THAT Backups are complete and can be SAFELY restored without any errors , as you know when the dump stats the DATABASE is still in use for one or other queries & this also lags the backups & may lead to corrupt backups.

The script worked for me , so i just shared with you.
Of course, you can take my help in anything above i will gladly help you with things. But I WONT BE RESPONSIBLE FOR ANY DATA LOSS occurring by application of the above things.:p

I hope you will use the script and if you do, I WOULD REQUEST A WORD "THANKS" in this post.

I also provide professional VBULLETIN HOSTING & Solutions, now backups too.;)
You can reach me at : support@buyrdp.com

Enjoy with VB.
Regards

Great article, many thanks for posting.

3 comments though...

1) Many admins can't afford a secondary backup server, however many dedicated hosts provide a free FTP backup server - it might be worth adding some details of performing an FTP copy of the backup.

2) Rather than shutting down the webserver it is possible to gracefully close vBulletin, have a look at this:

https://vborg.vbsupport.ru/showthread.php?t=254529

3) It is also probably worth detailing how to setup the backup script as a cron job

Thanks for appreciating this article.
I have done editing as per needed.

1. Added the FTP scripts on the post.

2. Shutting the WEB SERVER is a VERY GOOD option for those who get medium to high traffic. Loosing 2 minutes of audience is always better then getting a corrupt backup.
And it depends on the guy taking backups, you can always remove it.

3. Cron have been added and explained.

Nice Article, 5 Stars ;).

And my 2 cents? Always close the board on a backup, always no matter what - we have too many fearless "coders" out here who think they know what's best - NO! Common sense says don't have anyone interacting with the database while your making a backup for various reasons.

Agreed to you , Thanks a lot for this appreciating 5 Stars :)

And yes, Stopping total Interaction with the database is always USEFUL :

1. It gets you a complete working backup.
2. It never lets you OVERLOAD the server which means your website is back online within few seconds of SWITCHING HTTPD service back on as the server load never goes high up while backups are on.

I can not find any mod which can do this & i dont think it can be really done except what i did above for my method.
Thanks a lot for all appreciation, hope the guys using this post a word of Thanks here.

Sounds cool. I currently run a script called automysqlbackup.sh (http://sourceforge.net/projects/automysqlbackup/), my db is 570meg gzip. I then rsync my site to a local ubuntu box which in turn brings down the dumped db. I extract the db's from time to time and they extract without error. I also have used the dbs to build dev servers. I have not run into any problems so far. Do you think this is ok? I am also wondering about stopping http, even if you do this are there still not sessions attached to the mysql db? Does stopping http nuke all connections to the db? I also run lightspeed webserver so does my command to stop that before my backup differ? Thanks!

Or - more easy - install MySQL on your local comp and setup replica. Then power it on, wait until it's in sync, lock the tables (e.g. stop the slave), do the backup and start the slave again.

Result is you won't need a root access for SSH (rather dangerous to allow this), your SSH keys still use a passphrase (required to have) and your backups will be clean as well.

And nope - your local MySQL don't need to run all the time

To me ,
The method is cool, i have used that one lots of times for backups.
But there is a difference with your DB of 570MB & 5GB.

Mine are over 6.5 GB now & when you try to dump that big server loads shoots up high , i have seen loads of over 50 at times.
This is why we need solution to backup without shooting that load which of course messes up DB , lags the server and hangs your website for a longer time period , while my method safely takes the backup, then does an scp that backup to remote server and turns your server back on.
There are several methods to dump small DB's but for big ones you need something like that one.

Think you can not do it on automation , you need to manually do it everytime.
For my method, its all automated.

Hmm - dunno what's so hard typing "net start mysql" or "/etc/rc.d/mysql start" on a command line. Automated SSH clients like Tunnelier or some SSH Agent can even automate the SSH login process while giving some more kind of security than using keys without a passphrase.

And - advantage with Replica is you don't need to lock tables on the server so there's no need to set the board offline. :) With some very easy client side script it all can be done without much user interaction :)

Great article