Cloud Backup Manager

[CvP]

This add-on enhances the vBulletin Cron Based DB Backup (https://vborg.vbsupport.ru/showthread.php?t=231481) by adding two key features:


Takes backup of your files.
Upload the file/db backups to cloud (OneDrive (http://onedrive.com)).


Please note that some of the options of "Corn Based DB Backup" are incompatible with this such as combine db backup files set to false.

Features to be implemented in future:

Cleanup older backup files. Done.
Email notifications on failure. Done.
Convert all texts to phrases and make this product ready for translation.


Project Sponsor: Steven Baron (https://vborg.vbsupport.ru/member.php?u=13602) [ HostBoard (www.hostboard.com) ]

Screenshots: http://imgur.com/a/ntkZV

FAQ

Does using this add-on gives you (developer) access to my (forum admin) OneDrive account?
No. You create your own app to access your own OneDrive. I have no control over it.
What do I put in OneDrive Folder URL in options?
You don't need to worry about it (generally). It will auto populate.
Where are the backups stored?
OneDrive/vB Cloud Backup Manager
Will this work on shared servers?
Mostly not. It requires shell_exec and "unlimited" execution time (transferring gigabytes of data takes a while) which are generally not available on shared hosting except a few (such as HostGator).


How to Install

Extract and upload the files in "upload" folder
Import XML file from "import XML" to vB AdminCP > Products > Add/Import
Create app (see below) and put "Client ID" and "Client Secret" in options.



How to Create the App
First you need to create an app and let the add-on know details about your app.

Go to https://account.live.com/developers/applications/create
Give a name and click accept.
Click API Settings.
Put yoursite.com/admincp/onedrive-callback.php in Redirect URLs.
Save.
Click API Settings
Get "Client ID" and Client Secret" and put them in vBulletin Admin CP > CBM > Options


Change Log:

v 1.3.1

Fixed a minor bug

v 1.3.0

Added email reporting
Added auto delete after upload
Added various toggle options
Added Modification date
Added confirmation for manual retrieve/delete/upload actions
Added logging
Fixed a MAJOR bug preventing auto backup/upload in some most cases

v 1.2.0

Added support for up to 10GB backup file
Added fragment size option (default 50MB)
Added lots of error checking
Fixed rename admincp issue.

v 1.0.0

Initial release.

[CvP]

Reserved.

[chikuru]

I hope it can also backup via scp to a vps or dedicated server on the next few updates. Really good plugin.

[Manoel J?nior]

Perfect!! I love this addon! Thanks

[CvP]

Perfect!! I love this addon! Thanks

Please let me know how it works for you. There are a couple of minor bugs and some pending features...

[Hostboard]

I hope it can also backup via scp to a vps or dedicated server on the next few updates. Really good plugin.

One of the reasons I choose to use OneDrive is because there are many options available to sync the data stored there to other services.

How to sync Microsoft OneDrive on Linux (http://xmodulo.com/sync-microsoft-onedrive-linux.html)

[CvP]

I hope it can also backup via scp to a vps or dedicated server on the next few updates. Really good plugin.

I actually have done this in the past. Will see if I can integrate that into this.

One of the key reason we are using OneDrive is because it is the cheapest (1TB basically for free) and supports largest files (10GB each). We think this is sufficient for our needs.


A few notes about the state of this add-on:
1. It will not work if your admincp is renamed. Minor issue; will be fixed in next update.
2. It will fail to upload large files to onedrive due to some API limitation. Fix will be provided in next update.

[chikuru]

I actually have done this in the past. Will see if I can integrate that into this.

One of the key reason we are using OneDrive is because it is the cheapest (1TB basically for free) and supports largest files (10GB each). We think this is sufficient for our needs.


A few notes about the state of this add-on:
1. It will not work if your admincp is renamed. Minor issue; will be fixed in next update.
2. It will fail to upload large files to onedrive due to some API limitation. Fix will be provided in next update.

Cool!
I will give plugin a try and will post feedback here.

[CvP]

it is the cheapest (1TB basically for free)
Correction: Not 1TB; unlimited.

Cool!
I will give plugin a try and will post feedback here.
Wait for the next update. Current version will fail for anything above 100MB.

[CvP]

Version 1.2.0 released.

Added support for up to 10GB backup file (separately for DB and FILES)
Added fragment size option (default 50MB)
Added lots of error checking
Fixed rename admincp issue


This is not considered beta anymore.

[davidg]

Are u making the same for vb3 ?
Thank You

[Hostboard]

Are u making the same for vb3 ?
Thank You

Sorry guys but this is an add-on to vBulletin Cron Based Database Backup (https://vborg.vbsupport.ru/showthread.php?t=231481)and is not a stand alone system. Paul's add-on supports 4.1.0 forward.

[chikuru]

Will you implement auto delete old backups in the future?

[Hostboard]

Will you implement auto delete old backups in the future?

Yes, there will be an option to auto-delete the old backup file from your server, once the upload to OneDrive has been confirmed/made. We are also looking at some sort of log file and or notification system.

[Fields]

I get the following error after logging in via "List Backup Files":

HTTP status code not expected - got - 400 on line 104 in /is/htdocs/************/trgcommunity/packages/cbm/includes/onedrive.php
#0 /is/htdocs/************/trgcommunity/admincp/cbm.php(352): skydrive->get_folder_properties('folder.484011be...')
#1 /is/htdocs/************/trgcommunity/admincp/cbm.php(79): validate_onedrive_url(Object(skydrive), 'https://onedriv...')
#2 {main}

[Delfi_r]

As soon as I manage to sign with One Drive, I'll install on my servers.

[Delfi_r]

I needed use IE to create the account. All fine, mod installed

One question? there will be no more backups in the server?

[TheLastSuperman]

I needed use IE to create the account. All fine, mod installed

One question? there will be no more backups in the server?

Possibly;

Features to be implemented in future:


Cleanup older backup files.
Email notifications on failure.
Convert all texts to phrases and make this product ready for translation.


So do this to check:
- Paul's mod is required, this is simply an add-on... so where does Pauls mod save your backups now? Exactly! All you do is check that location on your server before installing this. If the file is generated then removed it cleans itself up OTHERWISE you'll need to go in every so often and delete the old backups depending on disk space i.e. if you have limits or are close to limit.

Novel idea though imo!

[CvP]

Hi Guys.

I have not been able to update this mod and respond to your questions due to some issues on my side.

Anyways, I'm fixing up this mod and will release a new version soon with bug fixes and more features.

The end goal of this add-on is to completely automate the backup process including deleting backups once they are uploaded to OneDrive.

[CvP]

Updated to v 1.3.0

Added email reporting
Added auto delete after upload
Added various toggle options
Added Modification date
Added confirmation for manual retrieve/delete/upload actions
Added logging
Fixed a MAJOR bug preventing auto backup/upload in some most cases


This should now work perfectly for everyone.

Please let me know if you run in to any issues.

[tanzeelniazi]

I have altushost shared hosting and at this time i have 3gb backup data.
can i upoad or this mod auto upload backup in cloud server ?

[Hostboard]

I have altushost shared hosting and at this time i have 3gb backup data. can i upoad or this mod auto upload backup in cloud server ?

You will have to ask your hosting provider if they support, the following requirements:

Will this work on shared servers?
Mostly not. It requires shell_exec and "unlimited" execution time (transferring gigabytes of data takes a while) which are generally not available on shared hosting except a few (such as HostGator).

[Darkside2012]

CBM automatically uploads no files to Onedrive.

The E-Mail notificaten looks like this : Time: 2015-04-25 23:42:34 <br>File Backup: <a href="">Uploaded to Cloud.</a><br><br>DB Backup: <a href="">Uploaded to Cloud.</a><br>

Manual I can upload the backup to Onedrive

[Manoel J?nior]

You will have to ask your hosting provider if they support, the following requirements:

Will this work on shared servers?
Mostly not. It requires shell_exec and "unlimited" execution time (transferring gigabytes of data takes a while) which are generally not available on shared hosting except a few (such as HostGator).
how to increase shell_exec and unlimited?

[CvP]

CBM automatically uploads no files to Onedrive.

The E-Mail notificaten looks like this : Time: 2015-04-25 23:42:34 <br>File Backup: <a href="">Uploaded to Cloud.</a><br><br>DB Backup: <a href="">Uploaded to Cloud.</a><br>

Manual I can upload the backup to Onedrive
Do you receive any errors anywhere?

Manually works perfectly?


how to increase shell_exec and unlimited?
Unlimited execution time is set using ini set and shell_exec allows php to run Linux commands.

[SaN-DeeP]

Yet another great addon..

[Code Geass]

Good Job man !

I have a question, do we need to install this mod
vBulletin Cron Based DB Backup
https://vborg.vbsupport.ru/showthread.php?t=231481

to make your mod work ?

[dandanch]

When clicking on List Backup Files-->Login with OneDrive
I get this message below, how do I correct that?

Microsoft account
We're unable to complete your request
Microsoft account is experiencing technical problems. Please try again later.

EDIT: I got that part to work now.

[Manoel J?nior]

Already I activated the shell_exec but really this product does not work with large files.

The product generated a * .tar 3.5GB and not upload, I have over 15GB available onedrive.

The product only uploads files smaller.

[Hostboard]

Good Job man !

I have a question, do we need to install this mod
vBulletin Cron Based DB Backup
https://vborg.vbsupport.ru/showthread.php?t=231481

to make your mod work ?

Yes, this piggy backs on top of Paul's modification. Why re-invent the wheel :)

[Hostboard]

Already I activated the shell_exec but really this product does not work with large files.

The product generated a * .tar 3.5GB and not upload, I have over 15GB available onedrive.

The product only uploads files smaller.

Have you tried to manually upload the file? Does it complete?

[Hostboard]

For those that are using this modification and running it on multiple sites under a single hosting account, it is advisable that you edit the start time of the "vBulletin Cron Based Database Backup" modification and off set the times for each site by at least 2 hour increments.

This is found under the ACP > Scheduled Task Manager
Edit: Automatic Database Backup for vBulletin

[Manoel J?nior]

Have you tried to manually upload the file? Does it complete?
Yes, I've tried manually but does not start the upload.

[Hostboard]

Yes, I've tried manually but does not start the upload.

After you made the configuration changes did you restart your web server?
There will be another update to this modification in the near future.

[bridge2heyday]

Does this mod work fine with databases bigger than 1 GB ?

[Manoel J?nior]

Does this mod work fine with databases bigger than 1 GB ?
Not work fine!

:down:

[Manoel J?nior]

After you made the configuration changes did you restart your web server?
There will be another update to this modification in the near future.
Yes. I restarted.

[Hostboard]

Does this mod work fine with databases bigger than 1 GB ?

Yes it does. I run this on 3 different sites with off set times at which the backup system runs (2am, 4am, and 6am)

I GZip my Database files and they come down to roughly 625mb My File system is a standard TAR files and are roughly 1.6gb They all transfer to OneDrive.

Manoel J?nior, issues are possibly hosting related, although I am going to reserve that call until we release the next version and see if that clears his issues.

[SaN-DeeP]

This modification still deserves to be best on list (Voted myself), but it wont work on database more then 3+ GB in size *actually tested*

I will recommend, checking some domain level assurance (G does not allows new accounts to synchronize rapidly)
Setup some synchronization rules by default rather then hammering services :)

Regards,

[Hostboard]

This modification still deserves to be best on list (Voted myself), but it wont work on database more then 3+ GB in size *actually tested*

I will recommend, checking some domain level assurance (G does not allows new accounts to synchronize rapidly)
Setup some synchronization rules by default rather then hammering services :)

Regards,

Thank you for the kind words :)
Please remember that the database portion of the backup is handled completely by Paul's modification vBulletin Cron Based Database Backup. We are piggy backing on top of it to include the site files and OneDrive interfacing.

[SaN-DeeP]

Respected Sir,
I always appreciate voluntary work done by vb.org team/users, thanks for great modification..

developers/applications/create just dont hammer this URL with new accounts on this service I meant..
Oh replace G with O or M above.
Google/Onedrive/Microsoft

[CvP]

This modification still deserves to be best on list (Voted myself), but it wont work on database more then 3+ GB in size *actually tested*

I will recommend, checking some domain level assurance (G does not allows new accounts to synchronize rapidly)
Setup some synchronization rules by default rather then hammering services :)

Regards,

At which point does it fail for 3GB+ db size?

[CvP]

Does this mod work fine with databases bigger than 1 GB ?

It has been tested with 1.5GB data. Theoretically, it should work with up to 10GB.

The issue that can sometimes happen is that a portion of the upload might fail (due to network issue) and it gets retried. Right now there is a hard limit of 5 retry per upload instead of per chunk.

So if there are total of 5+ failure during upload (due to network issue), it will fail.

[CvP]

Already I activated the shell_exec but really this product does not work with large files.

The product generated a * .tar 3.5GB and not upload, I have over 15GB available onedrive.

The product only uploads files smaller.

You should go to backup list and upload that file manually. In this case, you will be able to see live progress of the upload and see what goes wrong.

[Manoel J?nior]

You should go to backup list and upload that file manually. In this case, you will be able to see live progress of the upload and see what goes wrong.
Already = Error

[Hostboard]

Please try changing the "OneDrive Upload Fragment Size" to a smaller number. I would suggest trying 5 or 10. I am wondering if the uploads are timing out.

[SaN-DeeP]

10 is not good... 3-5 is suggested as per above reply..

CvP.. databases mostly containing binary files (images/videos on 4.x) fails, others doing awesome..

[Hostboard]

We have discussed adding an option to specify certain directories not be backed up. These would be for more static based items such as arcade files which should reduce the overall file size and improve the upload.

[CvP]

Already = Error

Sorry. I think I know which issue you are facing and it doesn't have a good error msg. Will post the latest version which you can try.

[CvP]

Respected Sir,
I always appreciate voluntary work done by vb.org team/users, thanks for great modification..

just dont hammer this URL with new accounts on this service I meant..
Oh replace G with O or M above.
Google/Onedrive/Microsoft
We are using Microsoft OneDrive and their official API which allows up to 10GB single file upload.

10 is not good... 3-5 is suggested as per above reply..

CvP.. databases mostly containing binary files (images/videos on 4.x) fails, others doing awesome..
From CBM's side, it doesn't really matter as it is just reading your database backup file and sending it to OneDrive in binary format. Your file content should not affect anything unless OneDrive is refusing your content for some reason.

I will update a new version tomorrow. Please try that and if problem still persists, I'd be happy to have a copy of the "problematic" database for detailed testing/debugging.

OneDrive Upload Fragment Size should be set as large as possible for faster performance but you can try smaller values if the connection is dropping.

[torrentinvites]

I'm getting this on file upload

Fatal error: Call to undefined function curl_strerror() in public_html/packages/cbm/includes/onedrive.php on line 274

My database is 4 gigs.

The file backup works fine.

Any ideas?

I'm assuming it wants to give me an error message.

[Hostboard]

I'm getting this on file upload

Fatal error: Call to undefined function curl_strerror() in public_html/packages/cbm/includes/onedrive.php on line 274

My database is 4 gigs.

The file backup works fine.

Any ideas?

I'm assuming it wants to give me an error message.

This error indicates that you do not have CURL enabled in your PHP.

[Manoel J?nior]

Which is updated?

[friendlymela]

wow this is useful .. thanks to developer

[Hostboard]

Well, unfortunately I have lost contact with CvP so development has currently stopped. I am attaching the latest files which were due to be released. I do not believe you need to re-import the XML as the one attached is from a previous version but I wanted to include since this is what I a running across all my sites. Once I secure a new developer I will see about updating this add-on. Any suggestions as to what you would like to see?

153225

[Manoel J?nior]

Thank you. This release has been repaired uploading error databases/files larges?

[RichieBoy67]

Hey guys, I have tried using this but I do not see the section in the admincp where I can see what is or has been uploaded to onedrive. I also do not see anything at all on Onedrive yet.

Does anyone have this working?

Also, are there updated instructions anywhere for using this with Linux?

[Hostboard]

Thank you. This release has been repaired uploading error databases/files larges?

No it does not. however I do have a new developer and we will be revisiting CBM to see why this might be happening.

[LinksFreak]

really great mod is it possible to make vb 3.8.7 ?

[Hostboard]

Hey guys, I have tried using this but I do not see the section in the admincp where I can see what is or has been uploaded to onedrive. I also do not see anything at all on Onedrive yet.

Does anyone have this working?

Also, are there updated instructions anywhere for using this with Linux?

It is a pretty straight forward install as long as you follow the directions int he text file. if you want PM me your Skype and I will give you a hand. Only constraint is it will have to be an evening after 10pm EST.

Not sure I understand using with Linux but if you are referring to uploading the backups to a Linux box, no this will only work with the OneDrive service.

[Hostboard]

really great mod is it possible to make vb 3.8.7 ?

Unfortunately there are no plans at this time to make a 3.x version.

[RichieBoy67]

It is a pretty straight forward install as long as you follow the directions int he text file. if you want PM me your Skype and I will give you a hand. Only constraint is it will have to be an evening after 10pm EST.

Not sure I understand using with Linux but if you are referring to uploading the backups to a Linux box, no this will only work with the OneDrive service.

Yes, it is installed correctly and it is very simple. Perhaps my issue is a large database. I see the OneDrive token in my repository but not on OneDrive.

[Hostboard]

Yes, it is installed correctly and it is very simple. Perhaps my issue is a large database. I see the OneDrive token in my repository but not on OneDrive.

Is Paul's database backup add-on functioning properly? We are not doing anything database related other than uploading it.

[RichieBoy67]

Yes, I have been using it for years.

I do have some very large tables.

[Manoel J?nior]

Yes, I have been using it for years.

I do have some very large tables.
Me too

[Hostboard]

This is still being investigated and am waiting on a quote for an update.

[Manoel J?nior]

I gave this product.

I'm using vBulletin Cron Based Database Backup (https://vborg.vbsupport.ru/showthread.php?t=231481) and created a cron on the server to upload to my Dropbox account.

[friendlymela]

wow this is good one.. will try it soon

[Hostboard]

For those of you having trouble uploading "larger" files. How large is large? My understanding is OneDrive allows a maximum file size of 10GB. While I wait on the developer for an update, the ability to "split" the archive into files smaller than 10GB parts will be added to the immediate things to do.

[friendlymela]

where i can put this yoursite.com/admincp/onedrive-callback.php

https://vborg.vbsupport.ru/external/2015/10/14.png

[Hostboard]

where i can put this yoursite.com/admincp/onedrive-callback.php

https://vborg.vbsupport.ru/external/2015/10/14.png

You need to login to your OneDrive account and give CBM permission to access it by filling in the information.

[Dr.CustUmz]

- fixed issue, link needs to be placed in redirect url -_- my bad

confirmed working on 3.8.9 =)

EXCEPT the scheduled task, i tried manually creating one but still no luck.
anyone no a way i can get the task to run... its kind of the whole point in an auto backup.
https://vborg.vbsupport.ru/attachment.php?attachmentid=154409&stc=1&d=1457346038

[Hostboard]

There is already a CRON entry for CBM "Automatic Database Backup for vBulletin". If you are backing up and uploading multiple sites it is highly recommended you offset he times for each site.

[Dr.CustUmz]

yes, and after it runs it does not send the file to OneDrive...

the original scheduled task is the auto backup product, so how is this product linking to that scheduled task?

https://vborg.vbsupport.ru/attachment.php?attachmentid=154423&stc=1&d=1457381978

https://vborg.vbsupport.ru/attachment.php?attachmentid=154424&stc=1&d=1457381978

https://vborg.vbsupport.ru/attachment.php?attachmentid=154425&stc=1&d=1457381978

https://vborg.vbsupport.ru/attachment.php?attachmentid=154426&stc=1&d=1457383297

[Hostboard]

If you click on the upload button can you upload the file manually?

[Dr.CustUmz]

If you click on the upload button can you upload the file manually?

yes, just doesn't upload after the scheduled task

[Hostboard]

Please make sure you are using the files from my previous post:

https://vborg.vbsupport.ru/showpost.php?p=2553798&postcount=55

[Dr.CustUmz]

still not auto uploading =/

when i run the task manually what should the end result say? Anything about the upload to the cloud?

https://vborg.vbsupport.ru/attachment.php?attachmentid=154438&stc=1&d=1457484101

[charlesr]

The backup product works, but the OneDrive upload product did not for me (set to auto upload). I'm now trying to upload manually, and at that point, it created a folder on Onedrive. Now I see a screen saying "This might take a while. Uploading." The file I'm trying to upload is 600MB. It has been going about 20 mins. How long would you expect it to take? It's quite a fast server.

[tevbas]

i think apps now not work. :(

[Hostboard]

i think apps now not work. :(

The backup process works (running on 4.2.5),the upload to OneDrive is no longer functioning most likely due to changes to the OneDrive system.