Good Morning,
Like everyone else here, I think about what might happen if my site went down for some reason. How would I get it back up running again?
Currently, I back up my SMF sites through the "Maintenance" area of SMF; I use FTP to make a full copy of my folders. I also see several backup options in cPanel, but I am never sure which one to use. I also see that Charlottezweb does automatic backups. Are these good enough?
In short, I'm confused about backing up. I want to understand this now, not when I actually have an emergency!
OK, a simple question: what backup method should I use to make sure I can restore my site in the case of an emergency?
Let me take this opportunity to say again how terrific Jason is and what a great service he offers here!
Thanks,
Matthew
Hi Matthew,
I'll follow up on this thread tomorrow.
Jason,
Thanks, and I look forward to your reply. This is an important subject.
Matthew
There's not necessarily a "right or wrong" way to do this but I can tell you the way I look at it...
1. I *ALWAYS* backup any files and databases before I do any sort of edits, upgrades, modifications, etc. to them. ALWAYS.
2. I setup a cron job to automatically backup my databases daily to a folder in my account (outside of public_html). That assures you of having a backup to restore if something breaks one day without the loss of too much data.
3. I try to backup a full copy of all my webfiles and databases to my local pc about every week or so whenever I haven't backed them up from step 1. above.
Those are some best practices I personally follow but whatever you want to do is up to you and how important the data is.
That being said, the server backs up all data in your accounts daily to a secondary harddrive in the same server. Once or twice a week (depends on the server), a full backup is also copied to a remote storage server in case something catastophic happened to the server and both drives.
I provide these backups as a "value-add" but state throughout the forum a lot that they are not guaranteed options you should rely on rather than doing your own backups. In other words, they're there if we need them and they're generally reliable but they should not be a substitute for doing your own backups.
To comment on SMF's backup utility -- I'm not a big fan. I've had bad luck with it on several separate account moves I've done for clients so I wouldn't personally use that as my way of backing them up. Rather, I'd use phpmyadmin (if the database filesize is manageable) or I'd back them up from shell or a backup script.
To summarize what I think you're asking, I would recommend setting up a cron job in your cpanel to backup your database once a day to a folder somewhere *outside* of your public_html folder. Then whenever you wanted to back everything up, you could ftp down that folder along with your public_html folder and that would contain all your webfiles.
Here are a few threads that may help:
http://www.charlottezweb.com/forums/index.php?topic=455.0
http://www.charlottezweb.com/forums/index.php?topic=388.0
http://www.charlottezweb.com/forums/index.php?topic=405.0
Does that help?
Jason,
Thanks for the post, but I have to admit I am still really confused. I've been using the SMF feature, so I'm kind of upset to hear it's not reliable. I've looked at the cPanel options, but none of them make sense to me.
I see where to run a cron job in cPanel, but I don't get how to set it up. What "command" do I enter to back up my whole site?
Thanks,
Matthew
Quote from: MatthewSchenker on March 14, 2008, 06:53:55 PM
I've been using the SMF feature, so I'm kind of upset to hear it's not reliable. I've looked at the cPanel options, but none of them make sense to me.
phpMyAdmin probably won't make sense without reading a bit about it but it's pretty easy once you know how.
I'm not saying the smf utility isn't reliable it's just not the way I personally trust. I haven't had great luck with it a few times I tried it. I also have to manage sites with databases way to big too be handled by a http backup option.
Quote
I see where to run a cron job in cPanel, but I don't get how to set it up. What "command" do I enter to back up my whole site?
You could use the cron job to backup your database.
This thread discusses it: http://www.charlottezweb.com/forums/index.php?topic=405.0
You could have cpanel create a full backup under the "backups" link.
Jason,
I appreciate your help on this. I don't understand why backing up is such a confusing subject.
It sounds like the SMF option might be good enough, since I'm only talking about a relatively small site.
Quote from: Jason on March 14, 2008, 11:30:23 PM
You could use the cron job to backup your database.
This thread discusses it: http://www.charlottezweb.com/forums/index.php?topic=405.0
OK, I'm sure I'm missing something really obvious, but after reading that thread a couple of times I still don't get a sense of how to do a reliable backup with a cron job.
Quote from: Jason on March 14, 2008, 11:30:23 PM
You could have cpanel create a full backup under the "backups" link.
Yes, I like the way the cPanel option is designed, but again the options are really confusing. This is what led me to start this discussion:
- Should I do a "System Backup"? What is that?
- Doing a "Partial Backup" doesn't sound good enough.
- I like the sound of a "Full Backup," but cPanel displays this message: "Full backups can only be used for moving your account to another server or keeping a local copy of your account." I assume I need something that I can use to restore the site in the case of an emergency?
Still confused.
Matthew
Quote from: MatthewSchenker on March 15, 2008, 07:37:38 AM
OK, I'm sure I'm missing something really obvious, but after reading that thread a couple of times I still don't get a sense of how to do a reliable backup with a cron job.
1. Create a folder outside of public_html. In other words, right when you connect to your account, before you go into public_html, create a folder there.
2. Setup a cron job using the syntax in that thread above:
mysqldump -uDBUSERNAME -pDBUSERNAMEPASSWORD DB_NAME > /home/CPANELUSERNAME/BACKUPFOLDER/BACKUPFILENAME.sql
3. Based on how you set it, it will backup your database only automatically to the folder location. (You have to substitute the capitalized values above for your own specific database and folder settings. (You can obtain those values from SMF's Settings.php file if you don't know them)
Quote
Yes, I like the way the cPanel option is designed, but again the options are really confusing. This is what led me to start this discussion:
- Should I do a "System Backup"? What is that?
- Doing a "Partial Backup" doesn't sound good enough.
- I like the sound of a "Full Backup," but cPanel displays this message: "Full backups can only be used for moving your account to another server or keeping a local copy of your account." I assume I need something that I can use to restore the site in the case of an emergency?
A system backup should allow you to get to the automated backups of your account that are done each night. However, if you're logging into cpanel to do a backup, you might as well create a new one on the spot so it's the most recent.
A full backup will create a giant zipped file (.tar) in the root of you account space which you can then backup to your pc via ftp. As long as you have a program capable of opening a tarball, you're fine. Most programs like winrar, winzip (i believe), and even the windows built-in zip handler should open those files. That tarball will have *everything* for your account in it -- files, databases, email, logs, etc...
Jason,
So if I do a "full backup" through cPanel, then save that file to my local hard drive, I would be able to restore my forum in an emergency?
Should I ignore that stuff about only being used for moving the account and keeping a local copy?
Matthew
Yes, you could restore it. It contains your sql databases in a folder called "mysql" and your webfiles are located in "homedir."
Of course, the forum would only be as current as the backup date. That's why it's best to set a cron job to automated the backup process for your database.
Quote from: Jason on March 18, 2008, 05:37:26 PM
Yes, you could restore it. It contains your sql databases in a folder called "mysql" and your webfiles are located in "homedir."
Thanks for helping me understand this. From now on, I'll use the "full backup" feature in cPanel instead of the SMF function. It's fairly easy to do.
Quote from: Jason on March 18, 2008, 05:37:26 PM
Of course, the forum would only be as current as the backup date. That's why it's best to set a cron job to automated the backup process for your database.
Yes, but the commands still don't make sense to me. For now, I suppose I'll have to do a manual backup.
Matthew
Quote from: MatthewSchenker on March 19, 2008, 10:13:37 AM
Yes, but the commands still don't make sense to me. For now, I suppose I'll have to do a manual backup.
What do you need help with? You basically just cut/paste that command into the cron line and change the capitalized values to match what your SMF forum is using for connection settings. (You can get those settings from SMF's Settings.php file).
If you want to open a ticket I can set this up for you.
mysqldump -uDBUSERNAME -pDBUSERNAMEPASSWORD DB_NAME > /home/CPANELUSERNAME/BACKUPFOLDER/BACKUPFILENAME.sql
And I guess the "quote" feature on this new beta of SMF isn't working presently :P
Jason,
OK, I'm piecing this together.
My confusion can be summarized with these questions:
- I want the cron job to occur every day at midnight, so I set it for "0=12AM/Midnight" in the "Hour(s)" menu. Then I set it for "Every Day" in the "Day(s)" menu. But what am I supposed to set in the "Minute," "Month," and "Weekday" menus? These don't make much sense.
- Is there a way to output a backup of my entire domain? Do I have to back up each forum separately with its own cron job, since they each have their own database?
- It asks for an e-mail address "where output from commands run via crontab will be sent." Does this mean it will send the actual sql file(s) to my e-mail address, or just a summary of the event? These files are pretty large, so I'm not sure I want them clogging my e-mail.
Matthew
I ran a test of this function and received the following message in my e-mail in box:
/bin/sh: /Backups/SMFTestBackup.sql: No such file or directory
I created a folder called "Backups" and placed it in the home directory, so it looks like this:
/Backups/
The last part of the cron job command (/BACKUPFOLDER/BACKUPFILENAME.sql) looks like this:
/Backups/SMFTestBackup.sql
Looks the same to me. What am I doing wrong?
Matthew
Hi Matthew,
I fixed your paths to be server based. (I didn't have that complete in the examples above).
The paths based on your folder structure above should be:
/home/CPANELUSERNAME/Backups/BACKUPNAME.sql
Check after midnight tonight to confirm that one runs. I noticed you have one set for midnight and another set for 1pm.
I removed the email address but that should send you any errors that your cron produces like your example above.
To answer your questions above:
Quote
1 I want the cron job to occur every day at midnight, so I set it for "0=12AM/Midnight" in the "Hour(s)" menu. Then I set it for "Every Day" in the "Day(s)" menu. But what am I supposed to set in the "Minute," "Month," and "Weekday" menus? These don't make much sense.
2 Is there a way to output a backup of my entire domain? Do I have to back up each forum separately with its own cron job, since they each have their own database?
3 It asks for an e-mail address "where output from commands run via crontab will be sent." Does this mean it will send the actual sql file(s) to my e-mail address, or just a summary of the event? These files are pretty large, so I'm not sure I want them clogging my e-mail.
1. You can leave the minute, month and weekday fields blank if you want it to run daily. If you want it to run at more specific times, i.e. on a specific day of the week, week of the month or at a specific minute within an hour, you would populate those other fields. In other words, you could technically have the backups run at 9:28pm every Tuesday only if you wanted to get that specific.
2. I don't believe you have the access level to backup your full account by cron but it's something we could look into. I just backup my forums individually by database as you said.
3. See answer above.
Hope that helps!
Jason,
That all makes sense now, and the backups are working! Thanks for the help!
OK, one last question. Why is there such a huge difference between the file sizes when doing a manual "full backup" and doing an automatic backup?
Automatic backup using a cron job (total of both my SMF sites): 38,634,153
Manual "full backup" of my domain from "backup" in cPanel: 423,891,669
That's like a difference of 11X! Not only that, but the cron output is an sql file, and the "full" backup is a tar.gz file!!
Matthew
Each cron job is just a database backup. The full backup is everything in your account (including the databases).
Quote from: Jason on March 23, 2008, 08:12:51 AM
Each cron job is just a database backup. The full backup is everything in your account (including the databases).
OK, that makes sense. So if I had an emergency and my forum database was wrecked, I could restore it with the back up from the cron job, right?
Matthew
Quote from: MatthewSchenker on March 23, 2008, 08:21:02 AM
Quote from: Jason on March 23, 2008, 08:12:51 AM
Each cron job is just a database backup. The full backup is everything in your account (including the databases).
OK, that makes sense. So if I had an emergency and my forum database was wrecked, I could restore it with the back up from the cron job, right?
Matthew
Yup
Quote from: Jason on March 14, 2008, 02:52:23 PM
2. I setup a cron job to automatically backup my databases daily to a folder in my account (outside of public_html). That assures you of having a backup to restore if something breaks one day without the loss of too much data.
Jason, can i keep the last three backup databases in the backups directory? What is the code for this?
I'm not aware of how you can do that in a cron command. You'd need to have a script setup for something like that.
However, you could add separate cron jobs to accomplish this. For example, you could create a cron job that backs up the database to one file on Monday, Wed and Fridays. Then you could create a separate cron job that creates a backup of your database on the other days to a separate file. You could create a separate cron for each day of the week if you really wanted to although that would use a good bit of your account space.
Thanks
Is it possible to restore a database of 35 MB via CPanel?
or to restore my directory that is about 235MB via CPanel?
Are you talking about using phpmyadmin or using the actual cpanel backup/restore icon? I've never restored a database through that part of the site though if it was created there it should work in theory. That runs the command vs. using phpmyadmin (web-based) so it probably would be more successful.
A database of 35 MB is relatively big, i wonder if the script times out during the restoration.
Hi Steve, have you considered using the phpmysqladmin query dialog to run the mysql command "load data infile"?
Quote from: Steve on August 24, 2008, 12:35:38 AM
A database of 35 MB is relatively big, i wonder if the script times out during the restoration.
It depends on how you're doing it. However, I just looked in cPanel and RVSkin's word choice on the backup page is a bit misleading.
The button that should say "full backup" actually says "databases." I was hoping there was an option to create a real-time backup and then restore it all without having to use a copy from your local pc. Because if you have to browse to grab the file from your local pc, you'll likely timeout.
I would go the route of shell when your database(s) get to that size. Much easier.
Or, you could export and import by tables instead of doing the whole db at once. Of course, that can be a bit of a hassle and takes much more time than just uploading the db and restoring it from shell.
QuoteI would go the route of shell when your database(s) get to that size. Much easier.
Does this option available via CPanel? I think that we don't have this authority for security reasons.
No, it's granted upon special request.
What is the best way for backup
To use the 'full backup' via Cpanel or to transfer the directories of my account via ftp to my local pc?
Are these methods have the same result?
They are different --
The cPanel full backup packages *everything* (email, db's, files, etc) into a tarball which you can then download via ftp.
With the ftp only option, you just get your files.
I don't mean to transfer only the public_html directory but all the directories of my account such a mail, public_ftp, tmp, etc
You'd get that through the full backup.
If you do that through ftp-only, you may not get everything. Some things (like mysql) may be stored outside of your account directories.
I didn't know this. Thanks
And when i want to restore i use the Restore a Home Directory Backup option in the Cpanel?
In theory. I've never used it on an existing account so we'd need to confirm if it overwrites everything (which I'm thinking it would).
What are the other options in case of something serious happens? As i see the tar file contains only the files and their locations but not the folders of my account.
It does contain all the folders. Look inside the homedir folder.
You are right, thanks Jason.
No problem :)
When i try to download my 256MB full backup via Files Manager, the downloading procedure stops automatically at 55MB (approximately). Is this a kind of restriction?
To clarify, you're doing it via cpanel/filemanager?
I'd have to check if there's a restriction -- it's quite possible. On the other hand, that's a huge file size to be downloading via http (web browswer). You'd be way better off using ftp.
Yes, i confirm that i'm doingh it via cpanel/filemanager.
There is no any problem via ftp although the speed is slower.
Can you confirm which server you're referring to? Your accounts are on different boxes and I want to test something. Thanks!
Tsunami.
btw what is the difference between the ftp and http download? For ftp download i use an external ftp application and for htttp download i use my browser. Are there any other differences?
The ports used by http and ftp are different so the traffic is separate. FTP should be much more reliable for larger downloads (and less stress on the server).
http://www.webopedia.com/DidYouKnow/Internet/2002/FTP_HTTP.asp
That being said, I checked and there is no restriction in place and one of my techs was able to download that backup file from your account via http (port 80).
Do you have another pc or network you can try the download from?
I don't think that this is a pc issue but i will try it through other pc. Every time i try to download it, the download stops near the 55 - 60 MB. Strange :-\
The ftp downloading speed is always very slow. Is this normal? 8 - 15 KBps
It takes more than 4 hours to download the backup file! This happens almost every time.
That's why i have tried to download my backup file via http.
Are you downloading a cpanel backup file? (Just one file?) If so, you should get very fast speeds. I can generally download a several hundred mb backup file in several minutes.
However, if you try to download a forum folder (with hundreds or thousands of files), ftp will slow the transfer rate.
I try to download the 255MB backup file that is in the root directory. The rate of transfer is very low and it takes me hours until to get downloaded.
I can try it myself tonight if you'd like?
If it's fast for me, then my assumption would be that it's either related to the distance or network between you and the datacenter, or it's related to an ftp setting you may be using.
Have you tried a different (free) ftp program to see if the results are similar?
Would you like to create a backup file in order to try it?
Quote from: Steve on November 17, 2008, 12:29:40 PM
Would you like to create a backup file in order to try it?
Sure, or if the older one is still there, I can try to download it tonight.
Done. Thank you.
Well,I'm definitely not seeing an issue from where I am using WSFTP-Pro. I was able to download the full file (265 mb) in 5.5 minutes:
From my ftp connection log:
Quote
RETR backup-11.17.2008_14-06-47_******.tar.gz
150-Accepted data connection
150 271564.6 kbytes to download
226-File successfully transferred
226 334.212 seconds (measured here), 0.79 Mbytes per second
transferred 278082161 bytes in 334.313 seconds, 6498.461 Kbps ( 812.308 Kbps), transfer succeeded.
Transfer request completed with status: Finished
I use auto connect in wsftp. It used binary and the type was "passive."
It wouldn't be surprising that I'd get better speeds given the fact I'm only about 250 miles from the datacenter but you shouldn't see degradation like you are.
I use the same ftp software. I set it to binary transfer and the speed is now about 200 Kbps. Thanks
That's certainly a big improvement :)
Was it set to ascii (manual) before?
It was in the automatic mode. I don't know why it didn't work :-\
Hi Jason,
Thanks for granting me access to this part of the forum.
In the coming days I plan to set a cron job for our forum but before, I'll read this and other related threads well and try to absorb as much as possible.
Meantime, there is a question I would like to ask:
Does a manual full backup, downloaded via HTTP or FTP, count vis-a-vis the bandwidth usage?
Yes, all in/out traffic (files, email, etc) should count towards the monthly transfer quota.
Thanks, Jason.
I set the cron job some days ago, but yesterday received the following error:
Quotemysqldump: Got errno 122 on write
What's that?
I'll send you the code I use by PM
Your account exceeded its quota which is why it's giving the write error.
Quote from: Jason on May 24, 2010, 09:52:03 AM
Your account exceeded its quota which is why it's giving the write error.
Ah ok, let me check it.