Rsync backups being altered

[Jason]

As you all likely know, Charlottezweb performs nightly backups on all servers to offsite servers in a separate datacenter (several states apart) so that we can offer an insurance policy if needed should disaster occur.  This is a value-added feature not offered by most hosts though I aim for nothing less than exceptional service to you all so I gladly pay these charges to hopefully avoid issues down the road.  I know that for many of you who I've worked with, it's proven quite handy from time to time in retrieiving deleted files and such.

That being said, this feature is NOT meant to be your insurance policy.    I state this time and again in my emails, but I wanted to emphasize it again just so that everyone is clear.  cPanel offers backup features, in addition to manual options and I'd be more than happy to advise anyone interested on options available.

So, on to the point of this post...

Rsync is a fabulous idea and I love the technology.  It allows me to specify directories and files and it will copy -- at great speeds -- the files altered since the last transfer, thereby making it possible to keep a reasonable "live" backup in place of our files.

When it runs, however, it takes a pretty decent amount of resources which can raise our loads.  We've been doing this for over a year on some servers and from time to time, we do notice trends where the loads are regularly high, thereby restarting other services like pop, smtp, spamd, mysql and in some cases apache itself. 

I've been noticing that more and more lately and the time has come to alter our setup.

I am not abandoning remote backup storage as I think any "real" host needs to do this.  Sadly, the VAST majority do not.  I hope that is one of the traits that sets Charlottezweb apart from a fly-by-night provider.

Presently, we are running rsync to backup ALL files that clients could possibly change in a given day EVERY day.  And once a week, we are also transferring a weekly backup in addition to this.  All of this accounts for a significant load.  Not to mention, for those of you familiar with server specs, we're running on AMAZING equipment already.  For us to have issues means that we're really pushing the limits of present technology.

So my new setup will involve nightly backups from our primary harddrive to our secondary harddrive (this is already occuring) and a once-a-week backup via rsync to our remote servers.  In other words, we will continue to have nightly backups (to our second harddrive in the same server) but we will only be backing up to an offsite server once a week henceforth.

I wanted to announce and explain this because I have no secrets when it comes to your accounts and I want us all on the same page.  I am not abandoning remote (offsite) backup because I think that's an insurance policy that separates Charlottezweb from all of the startup companies out there that don't have their bases covered.  But I am going to limit this because at the end of the day, an insurance policy is fantastic, but one that hinders normal uptime does all of us no good.

As always, I'd be more than happy to hear your thoughts/concerns...this is your company too!

Regards,
Jason

[Jason]

FYI...Thunder's new rsync setup is in place and was manually run today.  We're back on track now for weekly backups on this box. Going forward, I expect to not see the extended load issues as we're backing up far less data remotely.

After the holiday weekend, I will configure Lightning as well for the same setup.  Will advise once completed.

[Jason]

Lightning is now backing up on the same schedule and format.