PDA

View Full Version : Vbulletin Database Backup Pro (Cron Support)


Pages : [1] 2

Trigunflame
03-20-2005, 10:00 PM
VBulletin Database Backup Pro

https://vborg.vbsupport.ru/ (https://vborg.vbsupport.ru/vborg_miscactions.php?do=installhack&threadid=78486)

NOTICE: 02/03/2006
This system is being "ReBorn" into a new product "Vbulletin Backup Pro" working from a New Codebase and hopefully implementing all the features and fixing all the bugs that were present in the previous releases.
This will be developed specifically for the vbulletin 3.5 system but will probably maintain compatabilty with the pre-existing 3.0 versions.



Summary:

Vbulletin Database Backup Pro is just what it says; a professional tool for your Vbulletin installation. This script will handle the automated backup of your forums in a secure, data-conscious and optimized way.


Introduction:

I want to interest all of you in the ultimate solution for automated database backup from within Vbulletin crontab system.
And potentially more so than a standard crontab.

It's been said, backing up without direct shell access is futile. Wrong. This script is optimized to work just fine with even the largest of databases. (Note: If you have an extremely large database, make sure to use the mysqldump binary option that is selected by default.)

The reason for the creation of this script was due to needing a better system of automated backup. I think me and a lot of people have used either a standard cron system, or brad.loo's small database cron; while they can be somewhat effective, they do not provide the functionality or a total solution without having to extensively mod it.

This system has automatic failovers for MySQLdump, as well as the compression programs used where applicable. All in all, this script is a one stop shop for handling your backup needs on a automatic basis or from the control panel without having to use shell and do a dump manually.


Features:

Absolutely NO Vbulletin File Modifications
Fully automized backup using the Vbulletin Cron system.
Remotely store backups on another srever via builtin FTP libraries.
Keep Remote FTP Server directory SYNC'd with Local Filesystem.
Dump using shell based, MySQLdump
Dump using an optimized PHP based Dump
System will Failover from MySQLDump binary straight Into a PHP Dump
Various other Failovers for file/dir creations & deletion.
Automatic pruning & control of a Specified max backups at any one time.
Do a Full database dump OR just the tables you want. Such as post, forums, threads & users.
Combine individual tables from a dump into 1 SQL File.
Some optimizations for InnoDb tables AND MySQL 4
System LOCKS tables while dumping, to prevent database corruption.
Can Shutdown forum while backup is in progress. And then REOPEN it automatically.
Customized Errors are Logged back into the Vbulletin Cron Log
Log path for MySQLDump Errors are customizable as well as Dump Path
Customize the Full Paths to binarys that will be used if needed.
Customize which execution command to use from PHP if using MySQLdump & certain compressors.
Specifiy the Dump Switches MySQLDump will use if selected.
Choose the option to Compress your Backups or not.
Choose from gzip, bzip2, tar, tar gz, tar bz2, zip for compressions.
And Many Many More...


Change log:

3/30/2005 - Patch v1.9
Added configuration directives to control the PHP Dump optimizations for MySQL4, InnoDB, and Locking Tables. Some people were having some reported problems with restoring dumps; I believe this may have been the cause, I was using optimizations hardcoded directly for MySQL 4+ support, they are now MySQL 3/4+.


3/25/2005 - Patch v1.8
Lots of bug fixes regarding directory paths. If dump path was null, the script would translate the path as starting at / root.
Incorporated Full Remote FTP Support.
This includes remote offloading of dumps, keeping them rotated in sync with local backups, as well as removing Local backups after transfer to Remote Server.


3/23/2005 - Patch v1.7
Added ability to Repair & Optimize tables before backup.


3/23/2005 - Patch v1.6
A few configuration changes, I didn't save in the mysqlbackupconfig.php


3/22/2005 - Patch v1.5
Made the LOCK Tables mechanism optional.
Note: Use the shutdown Forums option, if you disable the LOCK Tables.
Made some adjustments to the cronbackup.php might help with the array_merge errors, and it may not..
Took the eval out for the parsing the configuration, they are referenced in hardcode now.
Modified the date feature for backups. Remember just use 'mdy' or 'm_d_y'.


3/22/2005 - Patch v1.4
Added Backup Rotating (Pruning) support to limit max backups at any one time.
Make sure to replace all files, even the mysqlbackupconfig.php as it contains 3 new configuration options.


3/21/2005 - Patch v1.3
Added some optimization code for InnoDB. Will help when you reload the database.
Reason: Thought about Erwin, I believe he mentioned he used InnoDB.


3/21/2005 - Patch v1.2
Modified previous sql/directory cleanup. Better code.


3/21/2005 - Patch v1.1
Fix for table prefixes, updated config with info about it.
Bug located by: Nexialys


3/21/2005 - Initial Patch v1.0
Forgot to add unbuffered streams for fwrite(), this will speed up the compression of the database backup.


3/21/2005 - Initial release. v1.0
Script was just released today, waiting on bug reports or any features that may need added to the script that I have missed.

To-Do:

AdminCP Based Dump Control Panel to Test & Run Customized Configurations
Create a Database Restore Tool, that works through Shell or PHP.
Redesign the error system for better debugging, with severity levels.
Finish implementing shell based support of Secure SSH SCP File Transfer Support
Full Backup of Directory Tree for all of your Files
Fix the bug regarding serializing/unserializing on some PHP Setups


Known Issues:

Need to work on some of the Raw Windows Commands.
Some of the stuff isn't reporting the errors as correctly as they should.


Instructions:

Download the zip file.
Place mysqlbackupconfig.php, mysqlbackup.php, ftp.lib.php, pclzip.lib.php in the forums 'includes' folder.
Place cronbackup.php in your 'includes/cron' folder.
Add a New Cron Task in your Scheduled Task Section. Refer to the screenshot if you need help.
Read and Modify the configuration data in the mysqlbackupconfig.php
Go back to the Scheduled Task section, and run the task. See what messages you get.
Note: pclzip.lib.php is an Modified Version of this open source GPL Php Zip Library.http://www.phpconcept.net
CHMOD 0777 the Directory you will be saving Dumps/Logs to !!!
IF POSSIBLE. YOU OR YOUR HOST SHOULD TURN OFF SAFEMODE/OPENBASEDIR



Help Debugging:

This may come as a surprise to some people.. but I do all my code in notepad, thus it may not look formatted right for people that use editors. If you want to view the code in "my" style, how its supposed to look.
Open my files in Notepad, with Font size 10-Regular-Lucida Console.

As for An errors you may get, when you post;
Post your PHP Version, MySQL Version, and the Error as is Shown Please.


Further Info:

I would like a lot of people to try this script and give me feedback on what you would like changed, what Needs changed and so on and so forth. With the size of this script, it more than likely not gonna work perfect on every system.. so just tell me whats wrong and I will try to get it fixed.


https://vborg.vbsupport.ru/ (https://vborg.vbsupport.ru/vborg_miscactions.php?do=installhack&threadid=78486)

elastomaniac
03-21-2005, 05:48 AM
looks great! will install shortly

Q-v-n-s-Q
03-21-2005, 05:56 AM
GJ, but do you have restore hack??

Trigunflame
03-21-2005, 06:01 AM
No, but I can work on it :) Until then, use phpmyadmin, or shell.

Marco van Herwaarden
03-21-2005, 06:16 AM
This really sounds great, will be doing some tests today if i have time.

neocorteqz
03-21-2005, 06:25 AM
Very Nice.

I coded a custom script myself, but it seems not wanna work with vB's cron.

This should make a great replacement. :)

Works good, except when I manually ran the script in the Cron section i get this outputted.

DB Backup

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in /global.php on line 329

Warning: in_array() [function.in-array]: Wrong datatype for second argument in /includes/functions.php on line 2359

Warning: Invalid argument supplied for foreach() in /includes/functions.php on line 2364
MySQL Backup was completed successfully with no errors.

Like I said, the backup was sucessful, Just I get that error. :)

Again very Nice work.

Trigunflame
03-21-2005, 06:56 AM
Very Nice.

I coded a custom script myself, but it seems not wanna work with vB's cron.

This should make a great replacement. :)

Works good, except when I manually ran the script in the Cron section i get this outputted.

DB Backup

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in /global.php on line 329

Warning: in_array() [function.in-array]: Wrong datatype for second argument in /includes/functions.php on line 2359

Warning: Invalid argument supplied for foreach() in /includes/functions.php on line 2364
MySQL Backup was completed successfully with no errors.

Like I said, the backup was sucessful, Just I get that error. :)

Again very Nice work.

Edit:

I ran this on my test linux system AND windows system, and cannot reproduce this error.. it may be a problem with your current setup. The merge arrays sounds like something have to do with the cache templates.. and that error I believe is unique to PHP5.

Acido
03-21-2005, 09:13 AM
It was tested with large databases?
I have a 1GB Database, i'm not sure to test it in my installation...

nexialys
03-21-2005, 09:21 AM
...add the installation process in a text files please, it will be easier to follow for reinstall etc...

Trigunflame
03-21-2005, 09:22 AM
It was tested with large databases?
I have a 1GB Database, i'm not sure to test it in my installation...

The script by default will use the MySqlDump command utility to handle backing up your database.

For a database of your size, I would sugget to

1. Use the MySqlDump utility as is default in the script.
2. For compression, select the tgz option (use shell tar with gzip), as this will also use the command line to compress your backup.

Trigunflame
03-21-2005, 09:23 AM
...add the installation process in a text files please, it will be easier to follow for reinstall etc...

There is no installation process.

You upload the files, modify configuration, and add a task to cron.
If that can't be figured out, maybe they shouldn't be using the script :ermm:

Edit:

I added the "installation" to a text file. Although I still don't see the point to it being that.. it's common sense.

nexialys
03-21-2005, 09:34 AM
error:
$this->OPTIONS = $this->MYSQL->query_first("SELECT `data` FROM `datastore` WHERE `title` = 'options';");is missing the table_prefix: $this->OPTIONS = $this->MYSQL->query_first("SELECT `data` FROM `".TABLE_PREFIX."datastore` WHERE `title` = 'options';");
in mysqlbackup.php ...

Trigunflame
03-21-2005, 09:39 AM
error:
$this->OPTIONS = $this->MYSQL->query_first("SELECT `data` FROM `datastore` WHERE `title` = 'options';");is missing the table_prefix: $this->OPTIONS = $this->MYSQL->query_first("SELECT `data` FROM `".TABLE_PREFIX."datastore` WHERE `title` = 'options';");
in mysqlbackup.php ...

I always forget that, I never use prefixes..

Edit. Updated zip file/configuration with patch.

Edit2. Optimized directory/file cleanup routine.

Freezerator
03-21-2005, 10:07 AM
Looks great. I will try it soon.

Trigunflame
03-21-2005, 10:31 AM
Looks great. I will try it soon.

I hope you do :) The more people that test it, the better.

SnowBot
03-21-2005, 11:02 AM
Can this also deleted backups so they dont stockpile on the server like the old hack? If i go away on holiday i have to turn the old hack OFF as not to be flooded with backups on the server.

nexialys
03-21-2005, 11:04 AM
maybe keep the last 2 or 3, depending on a config we can add to the hack... admins would be able to set the number of old backups to be kept...

SnowBot
03-21-2005, 11:06 AM
maybe keep the last 2 or 3, depending on a config we can add to the hack... admins would be able to set the number of old backups to be kept...

Yes great idea.
* SnowBot always likes options :)

Trigunflame
03-21-2005, 11:14 AM
Can this also deleted backups so they dont stockpile on the server like the old hack? If i go away on holiday i have to turn the old hack OFF as not to be flooded with backups on the server.

Ya I can code that in there, I actually created a log rotate script for brad.loos old hack a while back, I'll go ahead and remake one for this.

MrNase
03-21-2005, 11:52 AM
Can it upload backups to another server? If not that would be a great thing to add :)

I will install it when it has this feature.. Webspace is limited so I can't just upload them to my forum's space :(

Trigunflame
03-21-2005, 11:55 AM
Can it upload backups to another server? If not that would be a great thing to add :)

I will install it when it has this feature.. Webspace is limited so I can't just upload them to my forum's space :(

Yes, im workin on FTP support.

nexialys
03-21-2005, 11:55 AM
Can it upload backups to another server? If not that would be a great thing to add :)
... OR email the backup to the owner... but i suppose that 45gigs backups will be hard to email.. ;)

Trigunflame
03-21-2005, 11:57 AM
... OR email the backup to the owner... but i suppose that 45gigs backups will be hard to email.. ;)

Don't know if that would be prudent or not.. It is a possibility tho, I will add it to the To-Do List.

Reeve of shinra
03-21-2005, 01:51 PM
Great add-on and am looking forward to some of the suggestions that seem to be planned.

Trigunflame
03-21-2005, 02:25 PM
Click install then, lol.

Edit:
Just uploaded version 1.3 with a small optimization fix for InnoDb users.

error_22
03-21-2005, 02:45 PM
I get this when I run the task:

Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing.)

-------------------
what am I doing wrong???

Thanks
Niklas

Trigunflame
03-21-2005, 02:56 PM
Thats telling you it could not open the destination file. That could mean 2 things.

1. The destination directory for backups you specified does not exist.
2. It Does exist, but you havnt chmod'd it 777

kevinnguyen
03-21-2005, 03:40 PM
does it affect the site performance and consume lots of bw?

Trigunflame
03-21-2005, 03:52 PM
does it affect the site performance and consume lots of bw?

Thats why it runs on vbulletins cron system. It is ran whenever you have it scheduled to run.

zetetic
03-21-2005, 04:35 PM
Thats why it runs on vbulletins cron system. It is ran whenever you have it scheduled to run.
Thanks, this looks great! I'm going to remove brad.loo's (a year later and still waiting for version 2.0 :p) and install this now.

One nitpick: Any reason you don't pull the db config info from includes/config.php?

Trigunflame
03-21-2005, 05:01 PM
Thanks, this looks great! I'm going to remove brad.loo's (a year later and still waiting for version 2.0 :p) and install this now.

One nitpick: Any reason you don't pull the db config info from includes/config.php?

Because I have a lot of commentary in the mysqldump configuration file, and see no point in cluttering up vbulletins primary config file.

zetetic
03-21-2005, 05:10 PM
Because I have a lot of commentary in the mysqldump configuration file, and see no point in cluttering up vbulletins primary config file.
Sorry - I'm not a coder but do you have to modify the primary config file to get the db info out of it? Can't you just require ./includes/config.php in mysqlbackupconfig.php instead of putting the db login info in there?

I'm just thinking it's best to limit the number of places where your db login info is.

Trigunflame
03-21-2005, 05:22 PM
Sorry - I'm not a coder but do you have to modify the primary config file to get the db info out of it? Can't you just require ./includes/config.php in mysqlbackupconfig.php instead of putting the db login info in there?

I'm just thinking it's best to limit the number of places where your db login info is.

Im using my own settings array to store the info, I could include the primary config file, but I would still have to reference the specified variables.

As they are PHP files, and are simply for configurion having no functions for direct output, it's not gonna hurt anything.

zetetic
03-21-2005, 05:26 PM
Okay cool, thanks for explaining. I just ran the backup and it worked great. Thanks. :)

Trigunflame
03-21-2005, 06:09 PM
Update:

Version 1.4, included backupRotate support to prune old backups, and keep a limited amount of backups at any one time.

paratek
03-21-2005, 11:45 PM
I keep getting mysql error: Access denied. I am using the same username and password that i use in my VB config.

paratek

neocorteqz
03-22-2005, 12:46 AM
Edit:

I ran this on my test linux system AND windows system, and cannot reproduce this error.. it may be a problem with your current setup. The merge arrays sounds like something have to do with the cache templates.. and that error I believe is unique to PHP5.
Well if it is, then you figured it out then, I'm running php 5.0.3.

kall
03-22-2005, 03:46 AM
How about this error?


mysql error: Table 'datastore' was not locked with LOCK TABLES

mysql error number: 1100

Carl Anderson
03-22-2005, 04:48 AM
Parse error: parse error, expecting `T_STRING' or `T_VARIABLE' or `'{'' or `'$'' in /home/goldis/public_html/forums/includes/mysqlbackup.php(1414) : eval()'d code on line 1

Thanks Carl

Great job I hope I can get this going...

Trigunflame
03-22-2005, 06:31 AM
I keep getting mysql error: Access denied. I am using the same username and password that i use in my VB config.

paratek

Can you paste the error directly as it shows?

How about this error?

What MySQL Version?

Well if it is, then you figured it out then, I'm running php 5.0.3.

That would be an error with VB's templates, the MySQL Backup script does not use them. I may know a small fix for it tho.

Parse error: parse error, expecting `T_STRING' or `T_VARIABLE' or `'{'' or `'$'' in /home/goldis/public_html/forums/includes/mysqlbackup.php(1414) : eval()'d code on line 1

Thanks Carl

Great job I hope I can get this going...

Eval's will only give that error, if you've messed up something its trying to evaluate. In other words, you probably messed some part of the configuration file up.

Trigunflame
03-22-2005, 07:12 AM
Update, 1.5;

Read the changelog, I made quite a few changes.

neocorteqz
03-22-2005, 07:29 AM
That would be an error with VB's templates, the MySQL Backup script does not use them. I may know a small fix for it tho.

I'll stay tuned. :)

Trigunflame
03-22-2005, 07:34 AM
I'll stay tuned. :)

Upload version 1.5, including the configuration and remodify it.

Or if you want, just upload cronbackup.php I added 4 null arrays for templates, that might fix your merge error.. if not, doubt theres anything I can do about it.

kall
03-22-2005, 08:37 AM
What MySQL Version?


4.0.23-standard

Should I just try the latest update?

Trigunflame
03-22-2005, 08:40 AM
4.0.23-standard

Should I just try the latest update?

Try it with, and without the LOCK tables option.

kall
03-22-2005, 08:58 AM
Try it with, and without the LOCK tables option.
With - same error.

Without (assuming that means, to change LOCK to 0) - Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing.)

Trigunflame
03-22-2005, 09:41 AM
With - same error.

Without (assuming that means, to change LOCK to 0) - Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing.)

If your getting the destination SQL error, means the directory your trying to write to is not CHMOD 0777 most likely.

Paste your current configuration, in a PM to me.

Edit: I also remodified the config file, I accidentally had TYPE set to 0, make sure you have

$backup['TYPE'] = 1;

Also: Try and get SafeMode turned off for your domain if possible, you always want to use mysqldump if possible.

In the config file also.

Trigunflame
03-22-2005, 10:28 AM
Update:

patch 1.7;
Option to repair & optimize tables before the backup process.

MrNase
03-22-2005, 10:39 AM
Thank you very much for considering my feature suggestion :banana:

I will install it as soon as it can transfer my backups via FTP :)

The thing is: I have a small webserver standing in the basement. Itworks as a FTP-Server and Iam trying to upload the backups to thisserver. Hope that works :)

Trigunflame
03-22-2005, 10:45 AM
Ya, im still workin on stuff, I'll try to have it done soon.

paratek
03-22-2005, 03:36 PM
Here is the error i get:

Database error in vBulletin 3.0.7:

Invalid SQL: LOCK TABLES `access` READ, `adminhelp` READ, mysql error:

THE REST DELETED

Access denied for user: 'xxxxxxxxx@localhost' to database 'xxx-xxx_forums'



mysql error number: 1044

Date: Tuesday 22nd of March 2005 09:30:44 AM
Script: http://xxxxxxxx.com/forums/admincp/cronadmin.php?do=runcron&cronid=17
Referer: http://xxxxxxxx.com/forums/admincp/index.php?
Username: paratek
IP Address: xx.xxx.xxx.xx

Trigunflame
03-22-2005, 04:15 PM
Here is the error i get:

1. Try turning off LOCK tables, in the configuration first.

You might not have LOCK priviledges or something.

paratek
03-22-2005, 04:53 PM
Ok Progress...i turn off the lock feature, and using mysql type backup i get the Failover was initiated for MySQLDump Binary but backup was succesfull error....however, i can not find the backup anywhere. If i change it to php dump built into the program it does work and the backup file is in my directory. I am just concerned that i am turning locks off, as you said it was not recommended.

paratek
03-22-2005, 05:01 PM
One last thing, and i am sure it was not the intended use of the program to manually do backups with it. Any backups i do on the same day, are just written over, as the file name format is saved using the date. If i do 1 backup per day, fine, it just saves the file as the next date. If i do multiple backups in one day, they will just get written over.

Trigunflame
03-22-2005, 05:17 PM
One last thing, and i am sure it was not the intended use of the program to manually do backups with it. Any backups i do on the same day, are just written over, as the file name format is saved using the date. If i do 1 backup per day, fine, it just saves the file as the next date. If i do multiple backups in one day, they will just get written over.

Yes you are correct, the main function is cron. Until I create an actual "webpage" based center for the script, its running the cron function everytime you click "Run Task". So it's doing its intended job, by running the cron function.

When I finish making the AdminCP based page for it, you will be able to test different settings, customize backups, do individual dumps etc...

Note:
Yes, not using locks isnt as secure.. but if you have the shutdown forum option enabled, that is almost as good. Its basically turning the forum off to your members while the backup is in progress; that will prevent anyone from writing to the database. So, it Is your 2nd best option, just make sure its enabled.

paratek
03-22-2005, 05:37 PM
Well thanks for the reply. Just smacked the install. Last question. What happens to the users already in the forums when the cron starts and shuts the board down? I haven't been able to test this myself yet.

Trigunflame
03-22-2005, 05:57 PM
Well thanks for the reply. Just smacked the install. Last question. What happens to the users already in the forums when the cron starts and shuts the board down? I haven't been able to test this myself yet.

They will just get a page notifying them the board is temporarily closed, its built-in to vbulletin.

kall
03-22-2005, 07:00 PM
Marvellous!

All working fine now...I can only assume my path was a bit bung..I have /backups directories dotted around the server, but it didn't find them. :)

Trigunflame
03-22-2005, 07:12 PM
Marvellous!

All working fine now...I can only assume my path was a bit bung..I have /backups directories dotted around the server, but it didn't find them. :)

Thats good to know :), was hoping it wasn't my code lol..

Brad
03-22-2005, 07:23 PM
Good job :)

BaconDelight
03-22-2005, 07:36 PM
So if I understand correctly, if the lock option is on, but the option to close the board isn't, the board would remain visible, but it would be read-only, right? What would a user see if they attempted to, say, make a post while in this state?

Trigunflame
03-22-2005, 07:38 PM
Good job :)

Thanks, glad to have your blessings, lol.. :banana:

Trigunflame
03-22-2005, 07:42 PM
So if I understand correctly, if the lock option is on, but the option to close the board isn't, the board would remain visible, but it would be read-only, right? What would a user see if they attempted to, say, make a post while in this state?

Yes you are correct. If they tried to make a post, Vbulletin would spit out a MySQL error notifying you of an error with that query.

When using a lock, it prevents any threads from doing a write to that table, that is to keep consistency while you are doing the backup.

Note: I strongly suggest to use the lock options & the shutdown option. Just make a pretty message to be displayed, notifying your members its just routine backup procedure.

-=Sniper=-
03-22-2005, 10:31 PM
excellent hack dude! votes for hack of the month award!

neocorteqz
03-22-2005, 11:20 PM
Upload version 1.5, including the configuration and remodify it.

Or if you want, just upload cronbackup.php I added 4 null arrays for templates, that might fix your merge error.. if not, doubt theres anything I can do about it.
I'm not worried, it still functions.
I'll still upgrade to the latest (1.7).

Thanks. :)

Trigunflame
03-23-2005, 10:58 AM
Almost done with the Remote FTP Addition.
---------------

Reason its taking long, adding support for PHP Module "FTP" as well as a straight "fsockopen" implementation, this will allow everyone to be able to use this part of the script.

MrNase
03-24-2005, 09:07 PM
Can't wait till you are finished :)

My forums are growing and Iam just too lazy to backup the data myself :D

Trigunflame
03-25-2005, 01:44 PM
Major update folks.

Fixed quite a few directory bugs, AND !

Fully integrated/automated FTP Support.
This will allow seamless transfer of your data from one server to the next VIA ftp protocol.

paratek
03-25-2005, 01:54 PM
Sweet!

Hey Trigun, the directions didn't include an update with what to do withthe ftp.php file in the package. I assume in the same directory as the first 3?

Trigunflame
03-25-2005, 01:57 PM
Sweet!

Hey Trigun, the directions didn't include an update with what to do withthe ftp.php file in the package. I assume in the same directory as the first 3?

Thanks i forgot to add; ya your right stick it in the includes directory.

Trigunflame
03-25-2005, 03:26 PM
Update:

Removing "send backup as email" from To-Do list, ive been looking at lots of methods on sending large attachments via email, and its just not practical with php, especially because of memory limit issues, and the fact that as far as I know, the entire data of the attachment would have to be base64_encoded and then sent in 1 function call.

The problem with this again lies in the problem of memory_limit errors. Dumps hundreds of megabytes in size, having to put all of that in one string and compress, just not practical.

MrNase
03-25-2005, 11:52 PM
This is just great :)

It's 3 in the morning now so that I will install and test it after I got some sleep :)

Trigunflame
03-26-2005, 11:05 AM
This is just great :)

It's 3 in the morning now so that I will install and test it after I got some sleep :)

Ok thanks, im still working on the rest now.

Trigunflame
03-26-2005, 11:34 AM
Update:

Had to reupload, accidentally had the FTP option turned on by default -_-.

neocorteqz
03-26-2005, 04:58 PM
too many updates. :p

Thanks for this. Certainly saves me time.

Carl Anderson
03-26-2005, 08:04 PM
Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing. (/backups/03_26_05.sql))

I do have a folder called backups and it is granted 777 permissions in my root directory?

Any ideas?

Trigunflame
03-26-2005, 10:03 PM
(/backups/03_26_05.sql)

You are using a forward slash, that would be telling the program to start from the root filesystem directory / which you do not have access to.

Change

/backups

to

backups

or

./backups

AxEsLoCkEd
03-27-2005, 09:10 AM
installed nice script

telc
03-28-2005, 02:36 AM
How will this work with large forums where the databases are over 4 gigs. When the cron executes, will the browser still need to be open? Will it continue to run if the browser is closed after the cron is triggered? Also when you set it to Shutdown the forums, does it only shut them down during the Dump or are they still shutdown durring the gzip?

Carl Anderson
03-28-2005, 02:44 AM
(/backups/03_26_05.sql)

You are using a forward slash, that would be telling the program to start from the root filesystem directory / which you do not have access to.

Change

/backups

to

backups

or

./backups

Thanks I did as you suggested and I get the same.. I do have a directory with 777 permissions set at ./backups

here is what I get..

Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing. (./backups/03_27_05.sql))

Trigunflame
03-28-2005, 11:32 AM
Thanks I did as you suggested and I get the same.. I do have a directory with 777 permissions set at ./backups

here is what I get..

Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing. (./backups/03_27_05.sql))

Try using the full path to the backup file instead. Inside the AdminCP take a look at your phpinfo() it should give you the full path to your document root.

Have you tried

$_SERVER['DOCUMENT_ROOT'].'/backups';

that would be placing the backup in the backups folder directly inside whatever your html directory is.

Other than that, I don't know what to tell you, other than get safemode turned off if it is currently on.

Trigunflame
03-28-2005, 11:33 AM
How will this work with large forums where the databases are over 4 gigs. When the cron executes, will the browser still need to be open? Will it continue to run if the browser is closed after the cron is triggered? Also when you set it to Shutdown the forums, does it only shut them down during the Dump or are they still shutdown durring the gzip?

Using a PHP INI Directive to prevent termination of the script, and to have unlimited life until the script is finished.

Basically what this means is, that say you loaded the page; the cron executes and even if you close the browser the script keeps on running in the background.

With a database as large as that, make sure you do have SafeMode Off, is that will guarantee the directives are set to ensure your backup without prematurely stopping.

telc
03-28-2005, 11:36 AM
Using a PHP INI Directive to prevent termination of the script, and to have unlimited life until the script is finished.

Basically what this means is, that say you loaded the page; the cron executes and even if you close the browser the script keeps on running in the background.

With a database as large as that, make sure you do have SafeMode Off, is that will guarantee the directives are set to ensure your backup without prematurely stopping.


Also when you set it to Shutdown the forums, does it only shut them down during the Dump or are they still shutdown durring the gzip?

Trigunflame
03-28-2005, 01:31 PM
Download Version 1.8.3

--------

I did have the Restore Forum function after the compression handler.
Now I put it directly after the actual mysql dump.

So the process is like this:

Close Forum
Dump Database
Open Forum
Compress

Dolamite
03-28-2005, 07:37 PM
would there be any way to modify this hack (works fine btw) to use the admin panel for doing its backups?

i ask because i have just been informed (from people on vbulletin.com no less) that because of my mysql version 4.1.10a, this is the cause of all my backups done via the terminal to be "corrupted"

i was hoping for an automated system that used the vbulletin built in admin functions for doing the backups... since somehow those are not getting corrupted (also they are 2MB smaller which is odd).

Any help appreciated...

Trigunflame
03-28-2005, 08:15 PM
would there be any way to modify this hack (works fine btw) to use the admin panel for doing its backups?

i ask because i have just been informed (from people on vbulletin.com no less) that because of my mysql version 4.1.10a, this is the cause of all my backups done via the terminal to be "corrupted"

i was hoping for an automated system that used the vbulletin built in admin functions for doing the backups... since somehow those are not getting corrupted (also they are 2MB smaller which is odd).

Any help appreciated...

1. Read the my first post, it list the To-Do, and yes backups directly from the admincp.

2. You already can "sorta", by running the Task manually once you have added it to the cron task scheduler.

3. My "PHP" Dump method is actually better than Vbulletins, so if you want to use that; take a look at the Configuration and change the Backup type from

0 which is MySQLDump to 1 which is the PHP Dump.

Dolamite
03-28-2005, 08:42 PM
ah.. duh i guess i should have read that whole file.... anyhow... i really appreciate you helping me with this....and wonderful hack...i'll be sure to click the install button for you

Trigunflame
03-28-2005, 09:35 PM
ah.. duh i guess i should have read that whole file.... anyhow... i really appreciate you helping me with this....and wonderful hack...i'll be sure to click the install button for you

Thanks :), im still working on some more features, so check back whenever you can.

Zoints
03-29-2005, 09:23 AM
I get the following error:

Could not fetch database options. Shutting backup down.

I double checked my uploaded forums and DB username/password/name and it's all there and right.

Thanks.

Trigunflame
03-29-2005, 01:45 PM
I get the following error:



I double checked my uploaded forums and DB username/password/name and it's all there and right.

Thanks.

Please read the configuration file, I have it all explained in there. I can't explain why would you get that error, but anyway until its resolved, turn off the shutdown option in the config file.

Dolamite
03-29-2005, 03:57 PM
i'm really impressed...now that i've examined it better.... your "mysqlbackupconfig.php" file is about the cleanest code i've ever seen in a vb hack.... nice job.

Dennis Olson
03-29-2005, 04:39 PM
A couple of things -

Can you provide SCP support in addition to FTP? We exclusively use SCP for security reasons.

My database is about 16 Gb. Tell me what I should do in order to assure a good backup.

Thanks!

red_baron2000
03-29-2005, 05:10 PM
one word: excellent :)

thank you.

Trigunflame
03-29-2005, 06:15 PM
A couple of things -

Can you provide SCP support in addition to FTP? We exclusively use SCP for security reasons.

My database is about 16 Gb. Tell me what I should do in order to assure a good backup.

Thanks!

-------------------------------------------------------------
Yes, I will start working on an SCP interface ASAP.
-------------------------------------------------------------

As for your backup, if I were you, Make Absolute Sure you have SafeMode off so that you can use the shell commands (mysqldump).

As for compression, I would turn Compression OFF, and take care of that manually whenever you need to. Compressing 16gb will take a lot of CPU anyway you look at it, so I dont recommend doing that automatically; do it when you need it done.

Just do a straightup backup to the .sql file or MAYBE using the tar option to just archive it. If I were you however, I would just backup straight to the sql file with no compression/archiving whatsoever.

Also, make sure you are storing your attachments in the FileSystem; that is an option inside the AdminCP, Look for it. It's hard to think how you could have 16gb of just text.

Zachery
03-29-2005, 06:28 PM
A couple of things -

Can you provide SCP support in addition to FTP? We exclusively use SCP for security reasons.

My database is about 16 Gb. Tell me what I should do in order to assure a good backup.

Thanks!
Dennis you would be best off having a shell script written to do this and have it done by a real cron instead.

Trigunflame
03-29-2005, 06:29 PM
i'm really impressed...now that i've examined it better.... your "mysqlbackupconfig.php" file is about the cleanest code i've ever seen in a vb hack.... nice job.

Thanks, im still working on teh script though, so lots to improve upon.

Trigunflame
03-29-2005, 06:30 PM
one word: excellent :)

thank you.

Thankyou 2 :)

Dennis Olson
03-30-2005, 12:18 AM
Zach, I *do* have that right now (if you PM me, I can send you a copy, and you can sanitize it for use by vB users...) I just want to do the whole thing via the vB cron if I can. Nice & neat that way...

Dennis Olson
03-30-2005, 12:19 AM
It's hard to think how you could have 16gb of just text.

I have 1.3 MILLION posts.... ;)

Trigunflame
03-30-2005, 12:24 AM
It's hard to think how you could have 16gb of just text.

I have 1.3 MILLION posts.... ;)

Ouch -_-.

Rambo
03-30-2005, 01:23 AM
Nice script,

But i do seem to get error's when importing the database using either Shell, or bigdump.php, but i don't seem to get errors when i just manually go into vbulletin admincp -> import and maintanence -> database backup.

Good mod, but sadly i cannot use it, will have to keep doing it manually ><

Trigunflame
03-30-2005, 01:29 AM
Nice script,

But i do seem to get error's when importing the database using either Shell, or bigdump.php, but i don't seem to get errors when i just manually go into vbulletin admincp -> import and maintanence -> database backup.

Good mod, but sadly i cannot use it, will have to keep doing it manually ><

Well you have to give me more info.

Is the dump being done by PHP or Mysqldump? If it's via PHP dump, then I actually know why it might not be working, Im using some directives designed specifically for mysql 4 / innodb, that might not be working for you.

If thats the case, I will patch the script with options to enable/disable mysql 4/innodb optimizations.

Rambo
03-30-2005, 09:08 AM
Well you have to give me more info.

Is the dump being done by PHP or Mysqldump? If it's via PHP dump, then I actually know why it might not be working, Im using some directives designed specifically for mysql 4 / innodb, that might not be working for you.

If thats the case, I will patch the script with options to enable/disable mysql 4/innodb optimizations.

I believe the dump is being done by Mysqldump, don't think i touched that part of the settings.

bonnmac
03-30-2005, 11:27 AM
great hack!!! should be the best of the YEAR !!! not the month!

Trigunflame
03-30-2005, 02:09 PM
I believe the dump is being done by Mysqldump, don't think i touched that part of the settings.

Run the cron task manually, or look at the cron logs and tell me.

Mysqldump shouldnt cause any errors remaking the db, as its automated through the binary, and I don't touch its output in the script.

Trigunflame
03-30-2005, 02:48 PM
Update Version 1.9

Added configuration support for Enabling MySQL4/InnoDB/Lock Tables for the Directives placed inside the MySQL Dumps.

Greebo
03-30-2005, 06:11 PM
First: Thank you for an excellent addition - much needed! Automatic backups == GOOOOOOD!

Second: Bug found?

Running with LOCK off, I got these on every table:
LOCK TABLES `access` WRITE;
UNLOCK TABLES;


If Lock is turned off, shouldn't the export NOT have locks in it? It's not good if I have to manually edit out the LOCKS because I can't perform locks in my DB and I need to restore quickly from the latest backup. :) Its ok - but not having to edit would be much better. :)

Note - this is using today's update mysqlbackup109.zip - downloaded at 12:56pm EST - about an hour and a half after your update.

Trigunflame
03-30-2005, 07:59 PM
First: Thank you for an excellent addition - much needed! Automatic backups == GOOOOOOD!

Second: Bug found?

Running with LOCK off, I got these on every table:
LOCK TABLES `access` WRITE;
UNLOCK TABLES;


If Lock is turned off, shouldn't the export NOT have locks in it? It's not good if I have to manually edit out the LOCKS because I can't perform locks in my DB and I need to restore quickly from the latest backup. :) Its ok - but not having to edit would be much better. :)

Note - this is using today's update mysqlbackup109.zip - downloaded at 12:56pm EST - about an hour and a half after your update.

If using the PHP Dump, with LockTables set off, it works as needed.
However if it is using MySQLDump; by default I have the switches set at '--opt' which provides the best performance dump but they add locks and other directives to the dump.

In the updated zip file, I have included in the configuration the most compatible switches while giving a good dump, although I still have '--opt' as default.

The switches are, '--add-drop-table --extended-insert --quick'. Look in the config file and change $backup['SWITCHES'] to the alternate setup that I have listed in the config.

Greebo
03-31-2005, 01:02 PM
You know what, the problem was mine - I had $backup['LOCKTABLES'] set to 1. Missed changing that when I set it up yesterday.

Thanks!

Trigunflame
03-31-2005, 06:12 PM
Your welcome, im still working on SCP support btw. Should finish it, if I can plow through some of my client projects today.

neocorteqz
04-04-2005, 06:14 AM
Any particular reason it's showing up in the logs 4 times?


10296 DB Backup 03:10, 4th Apr 2005 MySQL Backup was completed successfully with no errors.
10295 DB Backup 03:07, 4th Apr 2005 MySQL Backup was completed successfully with no errors.
10294 DB Backup 03:06, 4th Apr 2005 MySQL Backup was completed successfully with no errors.
10293 DB Backup 03:05, 4th Apr 2005 MySQL Backup was completed successfully with no errors.


Also I have it set to run at 2am sunday, and it's running at 3am Monday morning. :confused:

Actually I think this was my fault, It was running more than once, I had set only the 2AM, and not the Minutes.

So it was * 2 * * Sun for the cron instead of 00 2 * * Sun

EDIT:

It's showing the file ownership as not being me. and I can't do a 'chown' on it, because of course I'm not root, but I do own the folder it's in. How do I make the script use me as the file owner?

I'm having a small problem extracting it. I'm using tgz, so I'm extracting like this

tar xvz MM_DD_YY.tar.gz


and it just sits there doing nothing.

That is the right command yes?

almqdad
04-04-2005, 08:37 AM
does this thing work with 3.06

Greebo
04-04-2005, 01:10 PM
does this thing work with 3.06
Should, I've got it on 3.0.7 and it works fine. :)

cybermomcafe
04-04-2005, 05:30 PM
//Fixed Problem// the vb3_threadviews table was showing as in use and had crashed... just ran the repair/optomize twice and it took care of it. :)

Working great on two of my forums, not sure what the problem is on the third though...

I'm getting this error when I try to run the script from the Scheduled Task Manager:

at the very bottom: 'vb3_threadviews.MYI' where's it getting that from?

Database error in vBulletin 3.0.3:

Invalid SQL: LOCK TABLES `vb3_access` READ, `vb3_adminhelp` READ, `vb3_administrator` READ, `vb3_adminlog` READ, `vb3_adminutil` READ, `vb3_announcement` READ, `vb3_attachment` READ, `vb3_attachmenttype` READ, `vb3_attachmentviews` READ, `vb3_avatar` READ, `vb3_bans` READ, `vb3_bbcode` READ, `vb3_calendar` READ, `vb3_calendarcustomfield` READ, `vb3_calendarmoderator` READ, `vb3_calendarpermission` READ, `vb3_connections` READ, `vb3_cpsession` READ, `vb3_cron` READ, `vb3_cronlog` READ, `vb3_customavatar` READ, `vb3_customprofilepic` READ, `vb3_datastore` READ, `vb3_deletionlog` READ, `vb3_editlog` READ, `vb3_event` READ, `vb3_faq` READ, `vb3_forum` READ, `vb3_forumpermission` READ, `vb3_gamechallenges` READ, `vb3_gamenews` READ, `vb3_games` READ, `vb3_gamesessions` READ, `vb3_holiday` READ, `vb3_icon` READ, `vb3_ignors` READ, `vb3_imagecategory` READ, `vb3_imagecategorypermission` READ, `vb3_language` READ, `vb3_mailqueue` READ, `vb3_messages` READ, `vb3_moderation` READ, `vb3_moderator` READ, `vb3_moderatorlog` READ, `vb3_passwordhistory` READ, `vb3_phrase` READ, `vb3_phrasetype` READ, `vb3_pm` READ, `vb3_pmreceipt` READ, `vb3_pmtext` READ, `vb3_poll` READ, `vb3_pollvote` READ, `vb3_post` READ, `vb3_post_parsed` READ, `vb3_posthash` READ, `vb3_postindex` READ, `vb3_profilefield` READ, `vb3_ranks` READ, `vb3_regimage` READ, `vb3_reminder` READ, `vb3_reputation` READ, `vb3_reputationlevel` READ, `vb3_rooms` READ, `vb3_rss_settings` READ, `vb3_rss_user` READ, `vb3_search` READ, `vb3_session` READ, `vb3_setting` READ, `vb3_settinggroup` READ, `vb3_smilie` READ, `vb3_stats` READ, `vb3_strikes` READ, `vb3_style` READ, `vb3_subscribeevent` READ, `vb3_subscribeforum` READ, `vb3_subscribethread` READ, `vb3_subscription` READ, `vb3_subscriptionlog` READ, `vb3_template` READ, `vb3_thread` READ, `vb3_threadrate` READ, `vb3_threadviews` READ, `vb3_upgradelog` READ, `vb3_user` READ, `vb3_useractivation` READ, `vb3_userban` READ, `vb3_userfield` READ, `vb3_usergroup` READ, `vb3_usergroupleader` READ, `vb3_usergrouprequest` READ, `vb3_usernote` READ, `vb3_userpromotion` READ, `vb3_usertextfield` READ, `vb3_usertitle` READ, `vb3_utt_store_action` READ, `vb3_utt_store_asvs` READ, `vb3_utt_store_bank` READ, `vb3_utt_store_history` READ, `vb3_vbookie_bets_placed` READ, `vb3_vbookie_groups` READ, `vb3_vbookie_item_options` READ, `vb3_vbookie_items` READ, `vb3_vbookie_news` READ, `vb3_word` READ;
mysql error: Can't open file: 'vb3_threadviews.MYI'. (errno: 145)

mysql error number: 1016

Date: Monday 04th of April 2005 02:26:54 PM
Script: http://www.***********.com/vb/admin/cronadmin.php?do=runcron&cronid=14
Referer: http://www.**********.com/vb/admin/
Username: *******
IP Address: ************

RichieBoy67
04-04-2005, 08:29 PM
If this has been covered, sorry... I just don't have time right now to read through all the posts... It is working for me but I am getting the following error...

Failover was initiated for MySQLDump Binary, but dump completed successfully.

Any way I can fix this??

Trigunflame
04-04-2005, 10:40 PM
Any particular reason it's showing up in the logs 4 times?


10296 DB Backup 03:10, 4th Apr 2005 MySQL Backup was completed successfully with no errors.
10295 DB Backup 03:07, 4th Apr 2005 MySQL Backup was completed successfully with no errors.
10294 DB Backup 03:06, 4th Apr 2005 MySQL Backup was completed successfully with no errors.
10293 DB Backup 03:05, 4th Apr 2005 MySQL Backup was completed successfully with no errors.


Also I have it set to run at 2am sunday, and it's running at 3am Monday morning. :confused:

Actually I think this was my fault, It was running more than once, I had set only the 2AM, and not the Minutes.

So it was * 2 * * Sun for the cron instead of 00 2 * * Sun

EDIT:

It's showing the file ownership as not being me. and I can't do a 'chown' on it, because of course I'm not root, but I do own the folder it's in. How do I make the script use me as the file owner?

I'm having a small problem extracting it. I'm using tgz, so I'm extracting like this

tar xvz MM_DD_YY.tar.gz


and it just sits there doing nothing.

That is the right command yes?

Try using

tar xzvf MM_DD_YY.tar.gz it worked fine me in test, if not just change it to use straight .gz encoding.

then just use

gunzip MM_DD_YY.gz



-------------

As for it showing up in your log at multiple times, I can not yet explain that... it has not occured for me on any of the forums I have tried this on, make sure you dont have multiple cron entries for the backup either.

As for the CHMOD problem, I will add into the script to CHMOD the file after it is saved, that should fix the problem hopefully...

Trigunflame
04-04-2005, 10:45 PM
//Fixed Problem// the vb3_threadviews table was showing as in use and had crashed... just ran the repair/optomize twice and it took care of it. :)

Working great on two of my forums, not sure what the problem is on the third though...

I'm getting this error when I try to run the script from the Scheduled Task Manager:

at the very bottom: 'vb3_threadviews.MYI' where's it getting that from?

Yes that would be a problem with that table.

Note: I do have the option in the configuration to enable repair/optimize before the dump is initiated.

Trigunflame
04-04-2005, 10:46 PM
If this has been covered, sorry... I just don't have time right now to read through all the posts... It is working for me but I am getting the following error...


Failover was initiated for MySQLDump Binary, but dump completed successfully.


Any way I can fix this??

It just means that you dont have access to the mysqldump binary via shell commands, probably because of safemode.

In this case, it is failing over into a manual PHP dump mode. As long as it is saving your dumps correctly, all is ok. But you might want to get safemode turned off if possible.

rinkrat
04-05-2005, 02:26 AM
Installed in a heartbeat and spit out a perfectly compressed backup very quickly. Great stuff! Thanks for the great hack :)

Trigunflame
04-05-2005, 03:40 AM
Installed in a heartbeat and spit out a perfectly compressed backup very quickly. Great stuff! Thanks for the great hack :)

You're very welcome :), when I finish I hope for this to be integrated into Vbulletin permanently.

RichieBoy67
04-05-2005, 11:03 AM
Thanks for this :) Great Job!!

Lizard King
04-05-2005, 12:05 PM
When I am trying to run this i get an error Failover was initiated for MySQLDump Binary & An error occured during the MySQL backup. Details (Could not open Destination SQL file for writing.
Can you explain me what i am doing wrong ?

cybermomcafe
04-05-2005, 12:44 PM
Did you chmod the folder that the backup is being written into to 777?

Are you using an absolute path to the backup folder (i.e. /home/user/public_html/backup/) or wherever it is?

Also, if you are running it again on the same date, make sure that there is not a file in the backup folder with that date already... :) hth

Trigunflame
04-05-2005, 02:05 PM
Did you chmod the folder that the backup is being written into to 777?

Are you using an absolute path to the backup folder (i.e. /home/user/public_html/backup/) or wherever it is?

Also, if you are running it again on the same date, make sure that there is not a file in the backup folder with that date already... :) hth

The script auto removes a file of the same name before the backup process.

-------

But yes, cyber is right, make sure you are chmodding the directory you are wanting to save the dump to, 777.

Also make sure the path is valid. Either use a full path, or a path relative to the main forum directory.

For example I use:

$backup['DUMP_PATH'] = 'admincp/backups';

Which saves the dump to the backups folder inside my admincp folder.

mharmon
04-05-2005, 02:15 PM
When I ran the database backup via cron -- it worked perfectly -- but it did not compress the FILE -- and it left my forums closed with the default "backing up the database" message.

Any suggestions?

Trigunflame
04-05-2005, 02:34 PM
When I ran the database backup via cron -- it worked perfectly -- but it did not compress the FILE -- and it left my forums closed with the default "backing up the database" message.

Any suggestions?

Run the cron manually, and make sure you don't get any error messages.

------

What compression settings are you using, Mysqldump or PHPdump etc..

mharmon
04-05-2005, 02:57 PM
I ran the cron manually -- and it works perfectly. I didn't get any error messages on the screen.

These are the compression settings from the config file:

$backup['COMPRESS'] = 1;
$backup['BUFFER'] = 1048576;
$backup['COMPRESSION']= 'gz';
$backup['COMPRESSORS']= array(
'gzip' => 'gzip',
'bzip2' => 'bzip2',
'tar' => 'tar',
'zip' => 'zip'
);

Trigunflame
04-05-2005, 03:10 PM
I ran the cron manually -- and it works perfectly. I didn't get any error messages on the screen.

These are the compression settings from the config file:

$backup['COMPRESS'] = 1;
$backup['BUFFER'] = 1048576;
$backup['COMPRESSION']= 'gz';
$backup['COMPRESSORS']= array(
'gzip' => 'gzip',
'bzip2' => 'bzip2',
'tar' => 'tar',
'zip' => 'zip'
);

$backup['COMPRESSION']= 'gz';

to zip, or tgz..

As for the script not setting your forum back up.. that really doesnt make sense, unless for some reason something is causing premature termination of the script, sometimes calls to functions that don't exist will do that..

What php, mysql version are you using?

mharmon
04-05-2005, 03:24 PM
Mysql = 4.0.22

Php = 4.3.10

Lizard King
04-05-2005, 04:17 PM
I figured out what i was doing wrong :) Thanks great script.

Trigunflame
04-05-2005, 10:44 PM
I figured out what i was doing wrong :) Thanks great script.

Glad to hear.

EvilHawk
04-06-2005, 01:11 PM
Installed, works fine for 2 weeks now, i have forgot to say thanks for this great script! ;)

Trigunflame
04-07-2005, 02:09 AM
Welcome, still a bit more to add though. Will try to get it done over the week.

NxTek
04-07-2005, 06:36 PM
We want to set up a second copy of vB for development, writing and testing new mods, etc. Per the license, we can set it up in a directory called vbtest and not make it public (password protect it). We're looking for a script that can syncronize the database, templates, etc from the production copy to the development copy.

If we back up the production copy of using VBulletin Database Backup Pro, can we restore it to the development copy?

Trigunflame
04-07-2005, 11:17 PM
yes, just make sure to look over all the configuration options, and configure accordingly, make any changes needed.

neocorteqz
04-08-2005, 04:52 AM
Try using

tar xzvf MM_DD_YY.tar.gz it worked fine me in test, if not just change it to use straight .gz encoding.

then just use

gunzip MM_DD_YY.gz



-------------

As for it showing up in your log at multiple times, I can not yet explain that... it has not occured for me on any of the forums I have tried this on, make sure you dont have multiple cron entries for the backup either.

As for the CHMOD problem, I will add into the script to CHMOD the file after it is saved, that should fix the problem hopefully...
I just figured it was because of the Owner of the file, I should have exclusive access to ALL files in my hoting account, But it seems with that owner id (99 for me), It just doesn't wanna extract.
I did use the xzvf and it did uncompress it, but to a separate folder due to the path I used in the config. So it's extracting, but making a new folder.

No problem though, I can always use gzip compression like you suggested.

And Gzip -d works as well to uncompress. :)

NxTek
04-08-2005, 04:39 PM
yes, just make sure to look over all the configuration options, and configure accordingly, make any changes needed.

Before restoring to the testvb version?

Remi
04-08-2005, 04:56 PM
Great hack, thanks :smoke:

can't wait for the "Restore" addon.

:up:

greenhybrid
04-08-2005, 08:18 PM
Can anyone tell me about how long the forum would be down if the option to temporarily suspend is on? My database is currently at about 30MB

Trigunflame
04-08-2005, 08:51 PM
If you have shutdown option configured, it would shutdown before backup, and reopen immediately afterward.

30mb is a very small database, I would say total backup time, 5-20sec

neocorteqz
04-09-2005, 01:44 AM
If you have shutdown option configured, it would shutdown before backup, and reopen immediately afterward.

30mb is a very small database, I would say total backup time, 5-20secThat's about right, But then again I saw mine compress within say 20 seconds for a 130 Meg DB.. I could have timed it wrong though. :D :)

Trigunflame
04-09-2005, 09:44 AM
That's about right, But then again I saw mine compress within say 20 seconds for a 130 Meg DB.. I could have timed it wrong though. :D :)

Even better :)

XrayHead
04-09-2005, 10:37 AM
Top Hack, Installed with no probs at all.

One question though, Can I rename or change one of the phps so that I can also run a backup once a week and ftp my database? I have the cron set to run at midnight at present and dont want to FTP everynight.

Thanks

XraY

Trigunflame
04-10-2005, 12:34 AM
Top Hack, Installed with no probs at all.

One question though, Can I rename or change one of the phps so that I can also run a backup once a week and ftp my database? I have the cron set to run at midnight at present and dont want to FTP everynight.

Thanks

XraY

If I were just you, I would turn FTP off, and wait till I create the AdminCP based control panel that will allow various things to be done individually by the administrator regarding the backup script.

Other then that, just change your cron to backup once a week.. OR 1 more thing you could do.

-------------------

create a copy of cronbackup so that you have:

cronbackup.php
cronbackup2.php

create a copy of mysqlbackupconfig so that you have

mysqlbackupconfig.php
mysqlbackupconfig2.php

change cronbackup2.php to include mysqlbackupconfig2.php instead of mysqlbackupconfig.php

change mysqlbackupconfig.php, and turn FTP off
make sure mysqlbackupconfig2.php has FTP on

add the crontask as such:

cronbackup.php is daily, it will not use ftp
cronbackup2.php is weekly, it will use ftp

-------------------

Hope I didnt make that all to be too complicated

Remi
04-10-2005, 03:01 AM
Hi

Is there a way to encrypt the backup files so I can store my files safely on public servers knowing that no one can read them but me.

Or I could make a deal with someone to send me his backups and I send him min!

Thanks

Trigunflame
04-10-2005, 05:21 AM
Hi

Is there a way to encrypt the backup files so I can store my files safely on public servers knowing that no one can read them but me.

Or I could make a deal with someone to send me his backups and I send him min!

Thanks

Well this is not really a great idea, do to the problems with PHP and big files and the way block encyptions work.

1. I "Could" code in shell support for using ZIP's encryption, tar/gz and I believe bz also do not support encryption.

2. Use MCRYPT to encrpyt a backup archive into a solid mass of encryption, then it would be up to you to Decrypt using another PHP script or I could create one.

3. BEST Option. Set your backup directory to something, below your public_html directory, that way it can not be accessed via url by traditional means.

o0stephane0o
04-10-2005, 09:46 PM
I have aproblem that I cannot solve..
I put the rights passwords and login name etc but it gives me the following message...

Invalid SQL: LOCK TABLES `access` READ, `adminhelp` READ, `administrator` READ, `adminlog` READ, `adminutil` READ, `announcement` READ, `attachment` READ, `attachmenttype` READ, `attachmentviews` READ, `avatar` READ, `bbcode` READ, `calendar` READ, `calendarcustomfield` READ, `calendarmoderator` READ, `calendarpermission` READ, `cpsession` READ, `cron` READ, `cronlog` READ, `customavatar` READ, `customprofilepic` READ, `datastore` READ, `deletionlog` READ, `editlog` READ, `event` READ, `faq` READ, `forum` READ, `forumpermission` READ, `gamechallenges` READ, `gamenews` READ, `games` READ, `gamesessions` READ, `holiday` READ, `icon` READ, `imagecategory` READ, `imagecategorypermission` READ, `language` READ, `mailqueue` READ, `moderation` READ, `moderator` READ, `moderatorlog` READ, `passwordhistory` READ, `phrase` READ, `phrasetype` READ, `pm` READ, `pmreceipt` READ, `pmtext` READ, `poll` READ, `pollvote` READ, `post` READ, `post_parsed` READ, `posthash` READ, `postindex` READ, `profilefield` READ, `ranks` READ, `regimage` READ, `reminder` READ, `reputation` READ, `reputationlevel` READ, `search` READ, `session` READ, `setting` READ, `settinggroup` READ, `smilie` READ, `stats` READ, `strikes` READ, `style` READ, `subscribeevent` READ, `subscribeforum` READ, `subscribethread` READ, `subscription` READ, `subscriptionlog` READ, `template` READ, `thread` READ, `threadrate` READ, `threadviews` READ, `upgradelog` READ, `user` READ, `useractivation` READ, `userban` READ, `userfield` READ, `usergroup` READ, `usergroupleader` READ, `usergrouprequest` READ, `usernote` READ, `userpromotion` READ, `usertextfield` READ, `usertitle` READ, `word` READ;
mysql error: Access denied for user: 'o0oo0X@localhost' to database 'o0oo0X'

How can i solve it?

Trigunflame
04-11-2005, 12:17 AM
I have aproblem that I cannot solve..
I put the rights passwords and login name etc but it gives me the following message...

Invalid SQL: LOCK TABLES `access` READ, `adminhelp` READ, `administrator` READ, `adminlog` READ, `adminutil` READ, `announcement` READ, `attachment` READ, `attachmenttype` READ, `attachmentviews` READ, `avatar` READ, `bbcode` READ, `calendar` READ, `calendarcustomfield` READ, `calendarmoderator` READ, `calendarpermission` READ, `cpsession` READ, `cron` READ, `cronlog` READ, `customavatar` READ, `customprofilepic` READ, `datastore` READ, `deletionlog` READ, `editlog` READ, `event` READ, `faq` READ, `forum` READ, `forumpermission` READ, `gamechallenges` READ, `gamenews` READ, `games` READ, `gamesessions` READ, `holiday` READ, `icon` READ, `imagecategory` READ, `imagecategorypermission` READ, `language` READ, `mailqueue` READ, `moderation` READ, `moderator` READ, `moderatorlog` READ, `passwordhistory` READ, `phrase` READ, `phrasetype` READ, `pm` READ, `pmreceipt` READ, `pmtext` READ, `poll` READ, `pollvote` READ, `post` READ, `post_parsed` READ, `posthash` READ, `postindex` READ, `profilefield` READ, `ranks` READ, `regimage` READ, `reminder` READ, `reputation` READ, `reputationlevel` READ, `search` READ, `session` READ, `setting` READ, `settinggroup` READ, `smilie` READ, `stats` READ, `strikes` READ, `style` READ, `subscribeevent` READ, `subscribeforum` READ, `subscribethread` READ, `subscription` READ, `subscriptionlog` READ, `template` READ, `thread` READ, `threadrate` READ, `threadviews` READ, `upgradelog` READ, `user` READ, `useractivation` READ, `userban` READ, `userfield` READ, `usergroup` READ, `usergroupleader` READ, `usergrouprequest` READ, `usernote` READ, `userpromotion` READ, `usertextfield` READ, `usertitle` READ, `word` READ;
mysql error: Access denied for user: 'o0oo0X@localhost' to database 'o0oo0X'

How can i solve it?

Set

$backup['LOCK'] = 0;
$backup['LOCKTABLES'] = 0;

You most likely dont have the necessary permissions.

Also, make sure to read the configuration, there are lots of options to chose, and I have them explained; in detail.

o0stephane0o
04-12-2005, 09:24 AM
Thanks 4 answering, I guess I wont backup the locked tables if I set them to 0 right?
So my backup won´t be complete..
Tell me if Iám wrong.

greenhybrid
04-13-2005, 01:25 AM
Is there any way to have it backup other databases as well?

caislander
04-13-2005, 03:55 AM
This is a great hack, since it save my poor weary brain from forgetting to do backups on time but I am having a slight problem, each time it runs the backup, it turns the board off but it never turss it back on. So if II am not around when the bacckup runs the boards are shutdown till I check and turn them back on.. Any thoughts?

Thanks

Trigunflame
04-13-2005, 05:36 AM
This is a great hack, since it save my poor weary brain from forgetting to do backups on time but I am having a slight problem, each time it runs the backup, it turns the board off but it never turss it back on. So if II am not around when the bacckup runs the boards are shutdown till I check and turn them back on.. Any thoughts?

Thanks

Make sure cron logging is turned On for the backup. Check if you are getting any errors.

caislander
04-13-2005, 11:06 PM
Make sure cron logging is turned On for the backup. Check if you are getting any errors.
Trigunflame,

I have the error reporting option turned on, althought i don't see that it is creating any error files, I also have the scheduled task log reporting option set but I don't see that it is even creating a log entry :confused:

WhSox21
04-15-2005, 04:55 AM
I wrote something similiar to this without the amount of features. I can tell you are definately a programmer. I've never seen such a well documented clean written hack here yet. Other programmers and hack developers should follow your programming ways, including myself!

Great hack, documentation and beyond!

Trigunflame
04-15-2005, 06:35 AM
I wrote something similiar to this without the amount of features. I can tell you are definately a programmer. I've never seen such a well documented clean written hack here yet. Other programmers and hack developers should follow your programming ways, including myself!

Great hack, documentation and beyond!

Thanks, but it's actually not finished, lol..

For the last week, since I've recently purchased Zend Studio; I am completely writing the script phpDoc compatible. When that is finished, I will also upload the Zip file of it, which will completely document the class/function usage of the script in HTML format.

Trigunflame
04-15-2005, 06:36 AM
Trigunflame,

I have the error reporting option turned on, althought i don't see that it is creating any error files, I also have the scheduled task log reporting option set but I don't see that it is even creating a log entry :confused:

Contact me on AIM/MSN or my email, and I will try to help you.

Lionel
04-15-2005, 09:35 AM
I get this when I tried to do it manually

Warning: Invalid argument supplied for foreach() in /includes/mysqlbackup.php on line 1378
MySQL Backup was completed successfully with no errors.

Trigunflame
04-15-2005, 10:10 AM
I get this when I tried to do it manually

That line would be from

foreach ($tables AS $table)


That means you have set 2 things wrong in the configuration.

1. You should be doing a full table dump.
2. If you are doing a selective table dump, you have to fill the array of tables to be dumped.

Judging by that error you are getting.

You are doing a selective table dump, and you havnt filled the table array.

Please Read The Configuration, before asking questions here.

Lionel
04-15-2005, 10:51 AM
Note: If your using option 1 to dump all tables it will
ignore the TABLES array anyway.

NOte: If you are using prefixes, you must specify them
yourself in the tables array ie. forum_post, forum_user.
-----------------------------------------------------------
Combine Tables For Selective Dump
-----------------------------------------------------------
If you have selected option 0 to dump specific tables. This
option determines whether or not the dumps are Re-Combined
into 1 File for the purpose of keeping a cleaner dump
folder and ease of use, without having to re-import
multiple files again.

NOTE:
If you set COMBINE to 0 you MUST use the compression
handler (tar, tgz, tbz, zip) INSTEAD of (gz, bz2) as
they alone do not support multiple files in 1 compressed
archive.

For that reason, if you are not going to use the combine
option, use one of the tar variations or zip.
-----------------------------------------------------------
0 = Individual Tables SQL files are archived together.
1 = All Tables are re-joined back into 1 SQL File.
-----------------------------------------------------------
*/

$backup['TYPE'] = 1;
$backup['TABLES'] = array('post','user');
$backup['COMBINE'] = 1;

Trigunflame
04-15-2005, 11:01 AM
Reply...

Will contact you shortly.

Update:

Contact with Lionel, problem solved.

Pseudomizer
04-16-2005, 01:53 AM
Hi @LL,

installed the hack successful. Configured the config file, created the scheduled task, created a new backup directory and run the task.

Error not being able to write to path even with chmod 777. Then I changed the path to the full path with the / at the beginning and now it works fine.

Good job, install clicked and voted as well.

Cheers,

Lionel
04-16-2005, 01:55 AM
Will contact you shortly.

Update:

Contact with Lionel, problem solved.
Yes I can attest to that. Thanks again.

almqdad
04-16-2005, 08:20 AM
greeeeeeeeeeeeeeeeeeeeeeeeeeeeeaaaaaaaaaaaaaaaaaaa at

it works fine with 3.06 as well

Trigunflame
04-16-2005, 08:59 AM
greeeeeeeeeeeeeeeeeeeeeeeeeeeeeaaaaaaaaaaaaaaaaaaa at


it works fine with 3.06 as well

That's good to know :) Stick around for version 2 !

Razasharp
04-17-2005, 01:34 AM
How far are you from v2 Tri?

I'm holding off installing till then :-)

Trigunflame
04-17-2005, 01:53 AM
How far are you from v2 Tri?

I'm holding off installing till then :-)

You can go ahead and install this, version 2 will be a drop-in replacement anyway.

Just re-coding the internals, provide more debugging and a couple of new features.

Razasharp
04-17-2005, 02:23 AM
Hi Tri... I'm getting this error:

Forum Backups

Warning: main(./includes/cronbackup.php): failed to open stream: No such file or directory in /MY_Name_admin/cronadmin.php on line 56

Fatal error: main(): Failed opening required './includes/cronbackup.php' (include_path='.:/usr/local/lib/php') in /home/MYSITE/domains/NYSITE.com/public_html/MYforum/MY_Name_admin/cronadmin.php on line 56

Any ideas?

Trigunflame
04-17-2005, 02:27 AM
cronbackup.php should be in includes/cron/ folder, the rest of the files should be in the includes/ folder

Razasharp
04-17-2005, 02:31 AM
yep.. its there :-(

Trigunflame
04-17-2005, 02:42 AM
yep.. its there :-(

Take a look at your other crontabs, it needs to be the same path as whatever you them at.

Kihon Kata
04-17-2005, 06:19 AM
So what happens if PHP is in safe mode?

Mosh
04-17-2005, 09:30 AM
Hi Trigunflame,

I am getting this error when doing a manual test on my live server (works fine on test server box).

For $backup['DUMP_PATH'] I used the full path, then just 'backups', then '/backups', then './backups'.

Forum Database Backup

Could not fetch database options. Shutting backup down.

PHP: 4.3.10
MySQL: 3.23.49

Thanks in advance for the help,

jd.

Trigunflame
04-17-2005, 11:27 AM
So what happens if PHP is in safe mode?

Most likely you will not have access to the exec(), system(), passthru() commands to use mysqldump, instead it will failover to PHPdump.

Trigunflame
04-17-2005, 11:29 AM
Hi Trigunflame,

I am getting this error when doing a manual test on my live server (works fine on test server box).

For $backup['DUMP_PATH'] I used the full path, then just 'backups', then '/backups', then './backups'.



PHP: 4.3.10
MySQL: 3.23.49

Thanks in advance for the help,

jd.

It means it could not get datastore options from your database which is needed if you have configured the script to auto shutdown/reopen your forums.

M1th
04-17-2005, 02:01 PM
Has anyone tried installing this on large board? 2GB+ database size?

Trigunflame
04-17-2005, 02:26 PM
Has anyone tried installing this on large board? 2GB+ database size?

1 Person I know that is using it, has a 5gb+ database.

I recommend making absolute sure, you have access to mysqldump via php beforehand.

It is possible to dump via PHPdump that I have coded, however for a database of that size, it will be much better if you are able to use mysqldump.

1. SafeMode should be off
2. openbasedir should be off

The Piper
04-17-2005, 05:28 PM
Installed and tested, working perfectly!! Just what I needed... Thanks a lot for this amazing hack, Trigunflame. I already voted for it as Hack of the Month and hope to see it implemented in the upcoming vBulletin versions! :D

Trigunflame
04-17-2005, 06:07 PM
Installed and tested, working perfectly!! Just what I needed... Thanks a lot for this amazing hack, Trigunflame. I already voted for it as Hack of the Month and hope to see it implemented in the upcoming vBulletin versions! :D

Thanks a bunch :)

Mosh
04-18-2005, 08:00 AM
It means it could not get datastore options from your database which is needed if you have configured the script to auto shutdown/reopen your forums.
I am not so good with the mySQL part of things, how would I go about that? it is probably easy and I am just being a bit dense.

Thanks,

jd

Logan70
04-18-2005, 02:10 PM
This may been covered, but I could see it anywhere...

We have installed it and I tried to test it manually, but it dies after about 40mb of our 200mb backup.

Then the forums are still turned off and we need to manually start them again.

Anything else you need to know just ask..

Thanks

Trigunflame
04-18-2005, 02:12 PM
This may been covered, but I could see it anywhere...

We have installed it and I tried to test it manually, but it dies after about 40mb of our 200mb backup.

Then the forums are still turned off and we need to manually start them again.

Anything else you need to know just ask..

Thanks

What php version, what mysql version, do you have safemode on, do you have openbasedir on?

Logan70
04-18-2005, 02:28 PM
PHP - PHP Version 4.3.4
MySQL - Client API version - 3.23.58
safe_mode - On
open_basedir - has a value, but how can I tell if it is on?

Thanks

Trigunflame
04-18-2005, 02:34 PM
PHP - PHP Version 4.3.4
MySQL - Client API version - 3.23.58
safe_mode - On
open_basedir - has a value, but how can I tell if it is on?

Thanks

Get safemode turned off, most likely its not allowing the script to set an infinite script execution, and its timing out.

Logan70
04-18-2005, 02:48 PM
Ok,

Now, here comes a dumb question.... What is the easiest and safest way to do that?

Thanks

Logan70
04-18-2005, 02:54 PM
One other point... If I turn the safe mode option off, will that not open some security holes in the server?

Just a thought.

Thanks

Trigunflame
04-18-2005, 02:54 PM
Ok,

Now, here comes a dumb question.... What is the easiest and safest way to do that?

Thanks

On a shared host? Ask your administrator.
Run your own server? Use the CPANEL Administrator or modify your apache configuration for your virtual host.

Contact me on AIM: Trigunflame
MSN: Trigunflame@msn.com

if you need more help.

Pseudomizer
04-18-2005, 03:16 PM
One other point... If I turn the safe mode option off, will that not open some security holes in the server?

Just a thought.

Thanks

Why is it called safe mode? :speechless:

Cheers,

Trigunflame
04-18-2005, 03:20 PM
Why is it called safe mode? :speechless:

Cheers,

SafeMode prevents reading execution of some shell commands along with reading server depenendent files.

Insecure scripts do open holes to it yes, but most of the time its not a problem, thats why you turn them off selectively for certain sites.

skokarl
04-18-2005, 04:03 PM
Hallo,

excuse my bad english...

I have a local Server....( localhost under Windows )

It comes : Failover was initiated for mySQLDump Binary, but dump completed successfully

??

Can somebody help me ?

Logan70
04-18-2005, 04:52 PM
I just have to say one thing....

We had some problems. Got in AIM with Trigunflame helped us out works like a charm.

Cheers to you man.... This is great and your support is top notch.

Thanks for everything.

** Clicks Install **

Trigunflame
04-18-2005, 05:00 PM
Hallo,

excuse my bad english...

I have a local Server....( localhost under Windows )

It comes : Failover was initiated for mySQLDump Binary, but dump completed successfully

??

Can somebody help me ?

Thats a normal message, its not an "error"; its more of a "notification". It is just telling you that it is using phpDump instead of MySQLDump.

Are your backups still being saved?

Zoints
04-19-2005, 09:11 AM
Run your own server? Use the CPANEL Administrator or modify your apache configuration for your virtual host.

Could you please explain how to do this with the CPANEL Administrator? I looked for what setting to change but couldn't figure it out. Thank you!

skokarl
04-19-2005, 10:46 AM
Are your backups still being saved?

Yes, they are saved.

another question..Do you know if your backup is compatible with mysqldumper ?

When I need the backup..I don't know how can I restore it.

Trigunflame
04-19-2005, 12:00 PM
Could you please explain how to do this with the CPANEL Administrator? I looked for what setting to change but couldn't figure it out. Thank you!

Contacted:

Problem Solved

Update:

Lots of things need to be improved in version 2, working with this sites DB size; it will be quite a few things.

Trigunflame
04-19-2005, 12:01 PM
Yes, they are saved.

another question..Do you know if your backup is compatible with mysqldumper ?

When I need the backup..I don't know how can I restore it.

TO restore you would use

mysql -u user -ppass database < sqlfile.sql

Logan70
04-19-2005, 12:05 PM
Yes, they are saved.

another question..Do you know if your backup is compatible with mysqldumper ?

When I need the backup..I don't know how can I restore it.

You could also use 'bigdump' to restore large forums.

I staggers the restore procedure so the server will not time out. I have used it a few times to restore 150+mb databases and have never had a problem.

Just google 'bigdump'

skokarl
04-19-2005, 01:27 PM
many thanks.

Mosh
04-20-2005, 09:30 AM
It means it could not get datastore options from your database which is needed if you have configured the script to auto shutdown/reopen your forums.
Just bumping this, if you can point me in the right direction of where I am going wrong here that would be great.

Thanks,

jd.

The Realist
04-20-2005, 09:57 AM
What does this mean "An error occured during input of the source file for Compression"

Also if I turn compression off and run the cron I get this straight away?

"MySQL Backup was completed successfully with no errors"

And no files or error docs show in the backup dir?
Laters

Robert Basil
04-20-2005, 10:15 AM
Great script! Running fine on 3.03

I have 2 ideas for future options...

#1 How about an option to backup our httpdocs / DOCUMENT_ROOT directory as well? Would be nice to be able to backup all of our forums php pages and attachments for those of us who keep them in the file system and not the database. The backup file can be placed below the DOCUMENT_ROOT so it will not keep making copies of itself. ;)

#2 Since a lot of VB forums also have ad-ons (Gallery etc...) how about being able to include a 2nd, 3rd or even 4th database for backup as well?

Thanks again for a great script!

Installed and voted for HOTM.

Trigunflame
04-20-2005, 12:06 PM
Just bumping this, if you can point me in the right direction of where I am going wrong here that would be great.

Thanks,

jd.

I've noticed problems with this particular code on a couple systems yesterday, will be updated soon.

1. open mysqlbackup.php
2. find $this->OPTIONS = unserialize(stripslashes($this->OPTIONS['data']));
3. replace it with :
$this->OPTIONS = unserialize($this->OPTIONS['data']);

See if u still get the fetch database options problem anymore.

Trigunflame
04-20-2005, 12:07 PM
What does this mean "An error occured during input of the source file for Compression"

Also if I turn compression off and run the cron I get this straight away?

"MySQL Backup was completed successfully with no errors"

And no files or error docs show in the backup dir?
Laters

It means it couldn't open the sql dump file for compression.

Trigunflame
04-20-2005, 12:08 PM
Great script! Running fine on 3.03

I have 2 ideas for future options...

#1 How about an option to backup our httpdocs / DOCUMENT_ROOT directory as well? Would be nice to be able to backup all of our forums php pages and attachments for those of us who keep them in the file system and not the database. The backup file can be placed below the DOCUMENT_ROOT so it will not keep making copies of itself. ;)

#2 Since a lot of VB forums also have ad-ons (Gallery etc...) how about being able to include a 2nd, 3rd or even 4th database for backup as well?

Thanks again for a great script!

Installed and voted for HOTM.

Noted, :)

Greebo
04-20-2005, 01:25 PM
My backup file is getting to be over 100 MB, and unfortunately I can't use the shell option, I have to use the php option.

Last night, for the first time, the backup failed to return the forums to their online state. It did so again this morning when I ran it manually.

I suspect the size is an issue - first task on my part is probably going to be moving attachments out of the database (finally) and into the file system.

But - is there any way to guarantee that if the process fails, it doesn't keep my users locked out?

Trigunflame
04-20-2005, 01:31 PM
My backup file is getting to be over 100 MB, and unfortunately I can't use the shell option, I have to use the php option.

Last night, for the first time, the backup failed to return the forums to their online state. It did so again this morning when I ran it manually.

I suspect the size is an issue - first task on my part is probably going to be moving attachments out of the database (finally) and into the file system.

But - is there any way to guarantee that if the process fails, it doesn't keep my users locked out?

It's more of an issue on browser time, after helping a guy yesterday that had backup files of "1.1gb" timeouts were occuring during the backup, the problem being that for some reason you need to send data to the browser at some intervals to keep it from disconnecting.

Even though I have code that "supposedly" is supposed to ignore browser disconnection, I will need to make mods in version 2 that send updates to the browser..

possibly . . . . . etc..

May want to turn off forum shutdown for now, until the new version.

Greebo
04-20-2005, 01:43 PM
Perhaps - but I moved my attachments out of the DB (something I've been meaning to do for a while now) and now the backup is 30mb instead of 130, and it doesn't time out anymore. :)

By the time we get big enough to hit that limit again, we'll have moved to new servers and I'll be able to have it go via shell.

ncangler
04-21-2005, 08:35 PM
Trigunflame, This looks very impressive and may be very timely for me, so in advance...thank you!

My serverhost had indicated that they would do complete backups every night. Last night my site went down and they fessed up that they had not backed up for a week. I lost all posts and new users for the past week. Fortunately it is still a relatively new and small board (200+). My question is regarding optimum backup strategy. Since the server box and drives went down last night isn't it better to have a backup on another physical harddrive somewhere in addition to or in lieu of being on the same box that your site is on? Is there a data backup service that can be purchased where your script would ftp a complete site backup to another secure (hopefully mirrored) backup server?

I may be working through moving to another host soon (fear and dread) but want to choose the optimum backup strategy now. Any input is most appreciated. And I will install this on the first of three sites and let you know how it goes.

I continue to be amazed at the collaborative spirit of incredibly talented and gifted people like yourself. Thanks. :)

Robert Basil
04-21-2005, 08:40 PM
We use 1and1 Internet (Dedicated servers).

They include nightly backups for all shared accounts and an FTP server account for backups for dedicated servers.

http://www.1and1.com

Trigunflame
04-21-2005, 10:03 PM
We use 1and1 Internet (Dedicated servers).

They include nightly backups for all shared accounts and an FTP server account for backups for dedicated servers.

http://www.1and1.com

Never mention 1&1 again, they are close to being the worst webhost that exist in modern times.

I know, ive had em. :disappointed:

Trigunflame
04-21-2005, 10:06 PM
Trigunflame, This looks very impressive and may be very timely for me, so in advance...thank you!

My serverhost had indicated that they would do complete backups every night. Last night my site went down and they fessed up that they had not backed up for a week. I lost all posts and new users for the past week. Fortunately it is still a relatively new and small board (200+). My question is regarding optimum backup strategy. Since the server box and drives went down last night isn't it better to have a backup on another physical harddrive somewhere in addition to or in lieu of being on the same box that your site is on? Is there a data backup service that can be purchased where your script would ftp a complete site backup to another secure (hopefully mirrored) backup server?

I may be working through moving to another host soon (fear and dread) but want to choose the optimum backup strategy now. Any input is most appreciated. And I will install this on the first of three sites and let you know how it goes.

I continue to be amazed at the collaborative spirit of incredibly talented and gifted people like yourself. Thanks. :)

FTP Support is in the current script, but just for transferring the backup.

In version 2, I am transforming the script into a "Forum Backup Script"; so it's reach will include that of backing up the entire forum, not just the SQL File itself.

Robert Basil
04-21-2005, 10:09 PM
Never mention 1&1 again, they are close to being the worst webhost that exist in modern times.

I know, ive had em. :disappointed:

Bummer you had a bad expierence with them. We've been with them for about a year now and no problems. We don't really use them for much as we have a root server with our own OS installed on it. None of that Control Panel B.S. ;)

Robert Basil
04-21-2005, 10:10 PM
In version 2, I am transforming the script into a "Forum Backup Script"; so it's reach will include that of backing up the entire forum, not just the SQL File itself.

WOO! HOO! :up: :up: :up: :up:

ncangler
04-22-2005, 02:21 AM
I've installed and am able to run the backup successfully with no errors.

Question 1...does this backup only vBulletin MySQL tables or does it also backup vBadvanced, Photopost, etc. tables that populate a MySQL database? I'd love to have this back up the entire database if possible.

Question 2... $backup['TABLES'] = array('post','user'); If I use "vb_" as a vBulletin table prefix how do I designate that for all the tables? In your quote above is that I all I need to put a prefix in front of or do I have to list every table? Just want to make sure I do it right. :)

Thanks for a great addon!

ncangler
04-22-2005, 02:37 AM
Oh one more question...I am executing the cronbackup and it says it is completed successfully with no errors. How can I open it to make sure everything is good? Thanks again!

Trigunflame
04-22-2005, 03:01 AM
Oh one more question...I am executing the cronbackup and it says it is completed successfully with no errors. How can I open it to make sure everything is good? Thanks again!

Please read the configuration.

By default all tables are backed up. Only if you change the option to a selective backup, is the array needed to specify particular tables.

ncangler
04-22-2005, 03:21 AM
Sorry, I did read it but it makes sense now. As long as I chose "1" as the backup type, I'm backing up all tables. Very cool. Thank you!

Please read the configuration.
By default all tables are backed up. Only if you change the option to a selective backup, is the array needed to specify particular tables.

Mosh
04-22-2005, 08:54 AM
I've noticed problems with this particular code on a couple systems yesterday, will be updated soon.

1. open mysqlbackup.php
2. find $this->OPTIONS = unserialize(stripslashes($this->OPTIONS['data']));
3. replace it with :
$this->OPTIONS = unserialize($this->OPTIONS['data']);

See if u still get the fetch database options problem anymore.
Thank you Trigunflame, that did the trick splendedly :up:

Now clicks install, already voted for HOTM last week :)

Jagged Tooth
04-25-2005, 05:35 AM
Im getting an error message afer i try to run the scheduled task (Step 6 in the install file).

Error: Failover was initiated for MySQLDump Binary, but dump completed successfully.

MySQL Version: 4.1.10a-standard

PHP Version: 4.3.10

Any thing would be helpful

Mosh
04-25-2005, 08:28 AM
Im getting an error message afer i try to run the scheduled task (Step 6 in the install file).

Error: Failover was initiated for MySQLDump Binary, but dump completed successfully.

MySQL Version: 4.1.10a-standard

PHP Version: 4.3.10

Any thing would be helpful
Just quoting Trigunflame's answer below as I knew it was in this thread somewhere, hope this helps.
It just means that you dont have access to the mysqldump binary via shell commands, probably because of safemode.

In this case, it is failing over into a manual PHP dump mode. As long as it is saving your dumps correctly, all is ok. But you might want to get safemode turned off if possible.

Jagged Tooth
04-25-2005, 09:31 AM
Oh, ok thanks for that its working now.

Greebo
04-25-2005, 01:17 PM
Upon review of the code, it appears that LOG_PATH, set in the config file, is not actually being *used*?

ImportPassion
04-25-2005, 01:24 PM
the alert that shows at the top of the screen for admins that says the forum is turned off does not get removed even tho the site is on.

Greebo
04-25-2005, 01:26 PM
the alert that shows at the top of the screen for admins that says the forum is turned off does not get removed even tho the site is on.
Um, no - it is off - your users are getting the FOrum Closed full screen notice. As an admin, you can still log in.

Go turn your site back on. :)

Trigunflame
04-25-2005, 04:32 PM
Upon review of the code, it appears that LOG_PATH, set in the config file, is not actually being *used*?

Yes.. need to fix that.

freedom2support
04-26-2005, 02:22 PM
Hi, I keep getting this:

Failover was initiated for MySQLDump Binary but backup completed successfully.

What do I need to do? I had to take the lock option off as it was giving Invalid SQL when I did it.

Trigunflame
04-26-2005, 08:42 PM
Hi, I keep getting this:

Failover was initiated for MySQLDump Binary but backup completed successfully.

What do I need to do? I had to take the lock option off as it was giving Invalid SQL when I did it.

Sir. Read The Message.

Failover was initiated for MySQLDump Binary
but backup completed successfully.

If the dump exist in its path, then its ok. That is just informing you that the backup could not use MySQLDump, and had to resort to PHPDump.

Kirk Y
04-28-2005, 10:56 PM
Hey, I love this MOD, it's worked great...in fact I recently got a chance to test out its results. My forum was recently hacked and our database destroyed, we used a backup created by Database Backup Pro and restored the forum with no problems, except for an error when we tried to re-create a table named Hosted. It was used for an Image Uploader hack. We skipped the creation process of this table and the rest of the restore went fine, with no SQL Errors. After we'd completed the restore process, we opened up the forum and it appears to be stuck in a backup or something. We get the message saying that the forum is offline due to a backup in progress. The Admin Panel cannot be accessed, so I'm not sure what to do. Any help would be much appreciated.

Thanks in advance!
-acidburn0520

BTW: If you'd like to see the site it's URL is: http://www.socomcity.net/forum/index.php

Paul M
04-28-2005, 11:43 PM
In what way can you not access the ACP - I can get the login screen up ok.

Lizard King
04-28-2005, 11:47 PM
If he tries to login then it takes him again to database backup warning page ;) try to login with test test. P.s : i used it just for test it is probably not a regular forum user :P

By the way did you try to change settings in backup config file. Maybe if you change turn forum off setting and it may work then.

Razasharp
04-28-2005, 11:53 PM
Just a thought, but did you restore in exactly the same dirtectory? or have you changed it?

for eg: is your forum at the same address such as:

yourwebsite.com/forumname

or has it changed to

yourwebsite.com/newforumname

?

Kirk Y
04-29-2005, 01:06 AM
Just a thought, but did you restore in exactly the same dirtectory? or have you changed it?

for eg: is your forum at the same address such as:

yourwebsite.com/forumname

or has it changed to

yourwebsite.com/newforumname

?

Actually...you may have it there...I believe our previous directory was labeled "forums", would that have an adverse effect?

**Edit: I'm not sure if this problem is associated with the DB Backup or what...but whenever I try to login to the Admin Panel, I get a message saying: "This user has not registered and therefore does not have a profile to view."

Trigunflame
04-29-2005, 03:40 AM
Yes that is possible. Also the reason the forum is locked into "Forum is down", is due to it being shutdown before the backup was initiated, which is it's intended purpose :)

As for that error message to do with profile, not sure what that would be, I will look into it after the release of Backup Pro version 2.

Razasharp
04-29-2005, 11:08 AM
Yes it could well be.

It sounds like your cookie settings - if they were set for your old forum directory they will now stop working.

Try this:

> ftp to your site

> rename the directory of the forum to what it was AT THE TIME the backup was taken

> go to your site

> in your vB Admin Control Panel (ACP)

vBulletin Options > Cookies and HTTP Header Options > Path to Save Cookies

Make a note of the existing setting (for example mine is set to: /myforums/ )

Then change the setting to:

/

> click on SAVE

> ftp back to your site and name the directory to what you want it to be - for example you said it was: forums

> go back to your site and in your vB Admin Control Panel (ACP)

vBulletin Options > Cookies and HTTP Header Options > Path to Save Cookies

Then change the setting to:

/forums/ (or replace 'forums' with new directory name it is in)

> click on SAVE

----------------------

if that doesnt work let me know.

ImportPassion
04-29-2005, 02:14 PM
Um, no - it is off - your users are getting the FOrum Closed full screen notice. As an admin, you can still log in.

Go turn your site back on. :)

that's my point, it's not getting turned back on after the backup. this is killing my site. it was off from 4am-11am and the back up was long done. also, I even changed the option to NOT turn the forum off and it still turned it off.

ImportPassion
04-29-2005, 03:33 PM
It's more of an issue on browser time, after helping a guy yesterday that had backup files of "1.1gb" timeouts were occuring during the backup, the problem being that for some reason you need to send data to the browser at some intervals to keep it from disconnecting.

Even though I have code that "supposedly" is supposed to ignore browser disconnection, I will need to make mods in version 2 that send updates to the browser..

possibly . . . . . etc..

May want to turn off forum shutdown for now, until the new version.

this must be what is happening to me. too bad.

steadicamop
04-29-2005, 05:11 PM
This is a cracking addition, although I've noticed if I backup using this, my DB backup file (uncompressed from the .gz file) is 8.43mb, and if I do a backup directly through VBB it's 10.86mb!! Any reason why the file is smaller? I looked through the raw file using Wordpad and looked at the post table, and it doesn't look like theres 1500 posts in it!!!

Kirk Y
04-29-2005, 06:37 PM
Yes it could well be.

It sounds like your cookie settings - if they were set for your old forum directory they will now stop working.

Try this:

> ftp to your site

> rename the directory of the forum to what it was AT THE TIME the backup was taken

> go to your site

> in your vB Admin Control Panel (ACP)

vBulletin Options > Cookies and HTTP Header Options > Path to Save Cookies

Make a note of the existing setting (for example mine is set to: /myforums/ )

Then change the setting to:

/

> click on SAVE

> ftp back to your site and name the directory to what you want it to be - for example you said it was: forums

> go back to your site and in your vB Admin Control Panel (ACP)

vBulletin Options > Cookies and HTTP Header Options > Path to Save Cookies

Then change the setting to:

/forums/ (or replace 'forums' with new directory name it is in)

> click on SAVE


----------------------

if that doesnt work let me know.


After further investigation, I discovered that our directory name was "forum" before we were hacked and is "forum" now. As of this moment, I cannot login to the Administration Panel nor can I login as a User...I'm totally shut out. Whenever I attempt to login I receive an error message stating: "This user has not registered and therefore does not have a profile to view." Could this be associated with an improper vbulletin restore? This was my first time having to attempt to restore vBulletin, I assumed that simply putting all the tables back into the database would bring it back to normal...but it's been quite the opposite.

Razasharp
04-29-2005, 07:06 PM
how were you hacked? via your vB admin panel? via your host account?

I guess you'll need to ascertain whether or not any other damage was done and for how long it was going on (u might need an earlier restore)

Trigunflame
04-29-2005, 07:17 PM
This is a cracking addition, although I've noticed if I backup using this, my DB backup file (uncompressed from the .gz file) is 8.43mb, and if I do a backup directly through VBB it's 10.86mb!! Any reason why the file is smaller? I looked through the raw file using Wordpad and looked at the post table, and it doesn't look like theres 1500 posts in it!!!

Make sure you can use mysqldump from php, make sure safemode is off, wait just a bit until version 2.

steadicamop
04-29-2005, 08:36 PM
Thanks, safe mode is definately off, how would I find out about the mysqldump? I have phpMyAdmin installed if anything can be found there.....

Kirk Y
04-30-2005, 02:20 AM
how were you hacked? via your vB admin panel? via your host account?

I guess you'll need to ascertain whether or not any other damage was done and for how long it was going on (u might need an earlier restore)

As far as I can tell our cPanel password was discovered...not that I know how, trust me when I say it was long. In any event our entire FTP was erased, as were all our mySQL Databases.

ochlocracy
05-01-2005, 08:17 AM
It's more of an issue on browser time, after helping a guy yesterday that had backup files of "1.1gb" timeouts were occuring during the backup, the problem being that for some reason you need to send data to the browser at some intervals to keep it from disconnecting.

Even though I have code that "supposedly" is supposed to ignore browser disconnection, I will need to make mods in version 2 that send updates to the browser..

possibly . . . . . etc..

May want to turn off forum shutdown for now, until the new version.

I was beating myself up over this for days, the backup works fine when it's run by the schuduler, but when I manual do it, the forum wouldn't turn back on. It because the timeout for your web server to too low. I set mine to 300 and it worked on command.

greenhybrid
05-01-2005, 03:57 PM
My safemode is on. Is this a big deal?

The Realist
05-01-2005, 04:46 PM
It would be safer to turn it off.

greenhybrid
05-01-2005, 04:48 PM
Is that a play on words? Seriously, what does it do?

The Realist
05-01-2005, 05:10 PM
Read these:

http://uk.php.net/features.safe-mode
http://www.webhostgear.com/166.html

carwash
05-04-2005, 09:58 PM
I run this through the cron section of the admin panel, it says everything worked fine, but no file is created in the specified directory. Does anyone know why this is.

RichieBoy67
05-04-2005, 10:43 PM
Is your specified directory CHmoded to 777???

carwash
05-05-2005, 08:26 PM
Is your specified directory CHmoded to 777???
Yes it is. :surprised:

Kirk Y
05-07-2005, 04:51 AM
I still need help: After restoring my vBulletin Board, it's stuck at the Backup in Progress screen...what do I need to do??