vb.org Archive - Administrative and Maintenance Tools - Cron job for Files Backup (attachments, avatars, etc.)

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vBulletin 3.7 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=228)

- - Administrative and Maintenance Tools - Cron job for Files Backup (attachments, avatars, etc.) (https://vborg.vbsupport.ru/showthread.php?t=172630)

Lynne

03-09-2008 10:00 PM

Cron job for Files Backup (attachments, avatars, etc.)

This is a script meant to run as a Scheduled Task to backup your files if you have your attachments, avatars, etc. in the file system. I have been running it for a couple of years now and given it to a few other vb admins and figured I should share it here as my first Mod. I found the basic part of the script somewhere online a couple of years ago and modified it to work with vbulletin.

Installation:
Download the file and modify the variables at the top of the file. The $backupdir should be an absolute path to a folder you wish to backup. Make sure you have a trailing "/" in the path. The $backupto variable is the absolute path to the folder for your backups. It should also have a trailing "/" in the path. Once you fill in those variables, upload the file to your "includes/cron/" directory and go to your Admin CP > Scheduled Tasks > Add New Scheduled Task and create a new task to run the script. I run mine every night (as seen in the attachment).

Notes:
The script I have attached is made to backup three different directories. You may add more or take away some. But, make sure if you add more at the top ($backupdir4, $fileprefix4) that you add them to the backupsus call. I used to have these variables set as globals in the function, however that stopped working for me after my last upgrade and instead of figuring out if this was a server issue, I simply changed the script to accommodate the change.

I have heard that some servers do not allow the use of the passthru() function. I can't help with that as I really don't know anything about servers and their personalities. Also, I assume this will work with windows servers also, but I can't help with that either as I have absolutely no experience with windows servers. This works for me and I hope it works for you to. I'm big on having backups around as it helps me sleep better at night.

And, thanks to Boofo for encouraging me to release my first modification!

Update:
1.0.1 - I forgot a semi-colon at the end of the $backupdir lines
1.0.2 - was: $backupattach = "cd $backupdir2;
s/b : $backupattach = "cd $backupdir3;

Lynne

03-10-2008 04:10 PM

My test site is running 3.7 beta 2 and this script works fine. I don't see any reason why it won't work on the newer versions.

ChurchMedia

03-10-2008 04:42 PM

Nice! Thanks! *installed*

Jase2

04-21-2008 07:53 PM

I was thinking of something like this, too. Very handy! Thanks, Lynne. Will install tomorrow, as I currently have a lot on my plate hehe.

yinyang

04-21-2008 11:07 PM

does/can this also backup the database?

Jase2

04-22-2008 07:23 PM

No, it doesn't backup the database. You would need a mod such as this for that: https://vborg.vbsupport.ru/showthread.php?t=167276

Installed, Lynne! Thanks.

Jase2

04-22-2008 07:36 PM

Hi Lynne,

Getting the following:

Quote:

Parse error: syntax error, unexpected T_STRING in /home/user/public_html/forum/includes/cron/attachbackupscript.php on line 9

Can't figure it out at the moment. Need some sleep lol. I've added an extra directory to backup albums, but I've just added $backupdir4 and $fileprefix4 every where after $backupdir4. Any ideas ?

Lynne

04-22-2008 07:43 PM

Hmmm, error on line 9. My line 9 is blank.

Make sure your $backupdir4 ends with a trailing slash. Also, make sure when you call the function with "backupsus ($.....)" that the order of the variables in there is the same as the order of variables in the "function backupsus ($.....)"

Uh oh... I just noticed I'm missing a semi-colon at the end of the $backupdir2 and $backupdir3. The $backupdir lines should look like this:

$backupdir1 = "/my/path/to/avatars/";
$backupdir2 = "/my/path/to/customprofilepics/";
$backupdir3 = "/my/path/to/attachments/";

I'll update the first post with the corrected file.

Jase2

04-22-2008 08:04 PM

All becomes clear... :p

Thanks!

Lynne

04-22-2008 08:10 PM

Quote:

Originally Posted by Jase2 (Post 1496803)

All becomes clear... :p

Thanks!

Thanks for pointing that out. I guess I dropped the semi-colon when I made the file 'generic' to be released.

Jase2

04-22-2008 08:12 PM

Still getting the same error. Hmmm......

Jase2

04-22-2008 09:21 PM

Here's my attached file, Lynne -- as my email client is acting up lol.

Lynne

04-22-2008 09:28 PM

For some reason, the lines are getting all run together by your editor and that is why you are getting the error.

For instance, your file line 1 shows:

Code:

<?php/*======================================================================*\

Whereas it should only be:

Code:

<?php

I will email a fresh copy of the script to you. This is very strange! You are the first person to say this has happened to the file.

Jase2

04-22-2008 09:37 PM

I believe that is from:

Code:

/*======================================================================*\

|| #################################################################### ||

|| # Cron Job to backup all avatars, profiles, and attachments                        ||

|| # Last Modified: December 04, 2007                                   ||

|| #################################################################### ||

\*======================================================================*

I'm using CofeeCup HTML editor 2008, paid version.

Just ran it again, and it worked. However, files are not showing up. I've cmod the backups folder, still isn't working. This is really strange.

Lynne

04-22-2008 09:42 PM

Quote:

Originally Posted by Jase2 (Post 1496853)

I believe that is from:

Code:

/*======================================================================*\

|| #################################################################### ||

|| # Cron Job to backup all avatars, profiles, and attachments            ||

|| # Last Modified: December 04, 2007                                   ||

|| #################################################################### ||

\*======================================================================*

I'm using CofeeCup HTML editor 2008, paid version.

Just ran it again, and it worked. However, files are not showing up. I've cmod the backups folder, still isn't working. This is really strange.

I know what the code is from, however it is on line two of my script and if you look at what you attached, line 2 of my script is merged in which line 1 in your attached script. And, that happened with several lines. Very weird, as I said. I'm not sure why that would happen. I am just using a simple text editor, BBEdit, to write it and I don't think it should cause that problem. Who knows.

Is the action getting logged, just not producing any output? Are you getting any files output? Did you run it 'as-is' or with your modifications? Do you know if passthru() is an allowable function on your server?

Jase2

04-22-2008 09:49 PM

Quote:

Originally Posted by Lynne

Do you know if passthru() is an allowable function on your server?

That I'll have to find out. I'll report back tomorrow. I'm too tired, need some sleep now. See you tomorrow! Thanks for your help, too.

Jase2

04-23-2008 02:12 PM

Hello Lynne,

Haven't seen if passthru() is enabled on my server, yet (although I think it is) -- but I just noticed the attach, avatars, and profilepics in my forums directory as a .tar file. Any ideas as to why it is saving too my forums directory? I certainly don't have that set for the backup folder.

Lynne

04-23-2008 03:53 PM

Quote:

Originally Posted by Jase2 (Post 1497278)

If you are getting some files, then passthru is working fine.

It sounds like either you don't have the correct path set for the backup directory you want them to be moved to, or you don't have the correct permissions set for the directory. The script makes the backup and then moves the file. Looking at the file you attached above, you have this as your line:
$backupto = "/home/user/backups1";

You need to add the trailing slash. It should be:
$backupto = "/home/user/backups1/";

Jase2

04-23-2008 05:27 PM

Hi Lynne,

That is an old file, the new one has the trailing slash. I have also cmod the folder to 777. Any other suggestions..... ?

Lynne

04-23-2008 05:34 PM

Quote:

Originally Posted by Jase2 (Post 1497457)

Hi Lynne,

That is an old file, the new one has the trailing slash. I have also cmod the folder to 777. Any other suggestions..... ?

I'd have to look at the form to see if something else is wrong. You can try attaching it here or sending it via email (if you have that working again). It seems that if the permissions and ownership is correct, it should work.

Jase2

04-28-2008 08:51 PM

I've now fixed this, Lynne. Somehow, my HTML Editor decided it wanted to play up -- don't know what was wrong, as it was formatting correctly on my computer. Oh well, at least it's fixed now.

Thanks for this, anyway!

nerofix

10-27-2008 03:38 PM

Hey Lynne, thanks for sharing this, I'm very interested in this!

I got a question:

I'd like to backup user's profile albums and avatars only.

Could you assist me with this a bit please? At the moment, both folders are located under www.domain.com/useruploads ...

Now, how to modify the php that it just backup the complete useruploads folder with every subfolder and files?
And do I need the complete path /web/mydomain/vbforum/usersuploads or just /useruploads ?

Lynne

10-27-2008 03:44 PM

You need to put the full path in the file, so put in (no quotes, and note the trailing slash which I mentioned you need to have in my notes above) "/web/mydomain/vbforum/usersuploads/"

nerofix

10-27-2008 03:54 PM

Hey thanks a lot for your answer.. does it also save subfolders then? And can I remove not needed variables from the php file, like this:

PHP Code:


		
			
<?php





error_reporting(E_ALL & ~E_NOTICE);



if (!is_object($vbulletin->db))

{

    exit;

}



        $backupdate = date("Y-m-d");

        //"Y-m-d-H-i-s" to include time



        $backupdir1 = "/my/path/to/useruploads/";



        //Absolute path to where the files are located

        //ie.  "/home/sitename/public_html/forums/images/avatars/";



        $files = "*";

        //What file to backup? Use a * to backup all the files

        //inside the folder entered above.



        $backupto = "/my/path/to/backups/";

        //Absolute path to where to store the backups

        //Make sure to put this outside your root folder

        //Permissions should be 777

        //ie. "/home/sitename/backups/



        $fileprefix1 = "alluseruploads";



        //This is the prefix that will be added before the date:

        //bak2006-02-25.tar



        $tararg = "-cf";

        //Here goes the tar arguments.

        // -cf to archive - tar

        // -cjf to archive and bzip2

        // other notes: 

        // -tf to list the contents later on

        // -xf to extract them all later on



//Call the function

backupsus($backupdate,$backupdir1,$backupto,$fileprefix1,$tararg,$files);



function backupsus($backupdate,$backupdir1,$backupto,$fileprefix1,$tararg,$files) {



$backupalluseruploads = "cd $backupdir1;

    tar $tararg {$fileprefix1}{$backupdate}.tar $files;

    mv {$fileprefix1}{$backupdate}.tar $backupto";

    passthru ("$backupalluseruploads");



}



log_cron_action('Profile, Avatar and Attachment Backup Completed', $nextitem);



?>

Would I run into problems to just have 1 backup variable and 1 prefix variable?

compuminus

02-20-2009 04:29 PM

There's a big mistake in the provided mod php file. Currently, it reads:

Code:

$backupattach = "cd $backupdir2;

        tar $tararg {$fileprefix3}{$backupdate}.tar $files;

        mv {$fileprefix3}{$backupdate}.tar $backupto";

        passthru ("$backupattach");

}

When it should really be:

Code:

$backupattach = "cd $backupdir3;

        tar $tararg {$fileprefix3}{$backupdate}.tar $files;

        mv {$fileprefix3}{$backupdate}.tar $backupto";

        passthru ("$backupattach");

}

Otherwise all attachments will not be backed up! Instead, directory 2 (the profile pictures) are just backed up twice...

Lynne

02-20-2009 07:59 PM

Yikes! Good catch! Updated.

Sunka

06-07-2009 07:21 AM

One question:
Could use this script to backup forum folder itself?
I would like to backup several folders (pictures, attachments, thumbnails), they are all above forum folder (public_html and above) and to backup forum folder too.
Is it possible? I asking because, this script have to be in cron folder, and cron folder is in forum folder.

Lynne

06-07-2009 04:24 PM

I have never tested it to backup my forum folders. You actually don't have to put the cron job in the cron folder, you can put it wherever you want as long as you put the correct path in the Scheduled Task.

R-D	07-04-2009 12:58 AM

Thanks for the script, Lynne!

It works perfect on the smaller backups (avatars, profile pics, album pics, etc.). Didn't like near 300MB of attachments though but that's probably due to PHP limits on my server.

Still highly recommended to anyone else looking to backup their files. :)

Lynne

07-04-2009 02:38 PM

Quote:

Originally Posted by R-D (Post 1842452)

I get a "mysql server not found" error every time I do the backup of my attachments (over 12GB). It's a mysql timeout thing - the script does the backup and then the next step is to input a line in the Scheduled Task log and at that point the connection is no longer open and you get the error. It's no big deal - the attachments *are* backed up - there is just no line in the Scheduled Task log to say the script was run.

accludetuner

11-09-2009 12:16 AM

EDITED:

I had an issue with only 5 of my 8 dirs being backed up but I solved it by chmod 777 for the 3 dirs that weren't getting backed up.

Installed and working great on 3.8.4! Thanks!

All times are GMT. The time now is 05:34 PM.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.01612 seconds Memory Usage 1,830KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (6)bbcode_code_printable (1)bbcode_php_printable (7)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (31)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: