Crontab and backups

[tab:Why Cron?] Giggyup!  I love anything that can be automated.  I’m not a fan of having to manually perform mindless, brain numbing tasks.  This is post is just a summary of setting up a cron to perform an offsite backup.  The connection to the remote server, public keys and rsync have been covered in a more comprehensive post here, so we will just concentrate on setting up the cron itself.

Now in contemplating a backup regime you need to consider how many backups you want.  You have to keep in mind that if a server goes bad and continually writes those corruptions to your backup server, then at some point your backups will become polluted.   It can therefore be a good idea to create daily, weekly, and monthly backups – depending on the volume of data and the amount of activity the server actually sees.

If data is changed frequently then more frequent backups can be good.  If the server goes down today and your last backup was  a week ago, any data changes will be lost.  If its a webserver and changes only occur every now and again, then this isn’t so much of a problem.

I have to administer several client data bases that are used frequently and contain data that is operational critical, so ensuring it is backed up – and often – is very important.  I have scheduled those databases to back themselves up at the end of the day, mid morning, lunch time, and mid afternoon.  Should the server crash at lunch time, the most that would be lost would be the 2 hours of data modified/added since the mid morning backup.

In order to achieve this we can create a cron entry for every scenario.  ie:  One that occurs everyday, then one that occurs once a week and one that occurs once a month.  This WILL take up a lot of storage space, and given the constant writing and reading to disk, you will also have to consider hard drive failure rates.

[tab:crontab explained]

Crontab is a daemon that checks the crontab file for commands, and runs them based on the timing you indicate in that file. The method for adding a line of “timing” and its corresponding command is:

# m h dom mon dow user command

where..
m = minute
h = hour
dom = day of month
mon = month of year
dow = day of week
user = the user that will perform the command
command = the command you wish to automate

There are also some variables that you can use which crontab will recognise, if you do not wish to implicitly specify the “m h dom mon dow” section. Accepted variables include:

@reboot        Run once, at startup.
@yearly        Run once a year, "0 0 1 1 *".
@annually      (same as @yearly)
@monthly       Run once a month, "0 0 1 * *".
@weekly        Run once a week, "0 0 * * 0".
@daily         Run once a day, "0 0 * * *".
@midnight      (same as @daily)
@hourly        Run once an hour, "0 * * * *".

if we look at yearly it may help to explain the use of “m h dom mon dow”. the yearly variables given are:

 0 0 1 1 *

This indicates that at 0:0 or midnight, on the 1st day, of the first month crontab should execute whatever command we stipulate. The asterisk “*” at the end means “any dow (day of the week). This is because the 1st day of the first month will change; so for any day of the week that is the first day of the first month, run this cron.

[tab:Example cron]

now in this instance we are setting up a cron to automate an rsync backup from a remote server. I want my sync to occur daily, at midnight and I want the user “root” to run the command.

0 0 * * * root rsync -avv -e "ssh -i /root/cron/thishosts-rsync-key" remoteuser@xx.xxx.xxx.xx:/backup/cpbackup/daily/ /localbackup/daily/

and for those that want to actually see the command thats hidden under my annoyingly slender, site template… here tis.. (you would obviously have this all on one line 😉

rsync -avv -e “ssh -i /root/cron/thishosts-rsync-key” remoteuser@xx.xxx.xxx.xx:/backup/cpbackup/daily/ /localbackup/daily/

the remote directory I am ssh’ing to is the directory Cpanel/WHM dumps its backups to, and I am just grabbing the changes in those files.  It is usually just statistics files, mail, and logs, but occasionally someone has actually done some work and the downloads are slightly larger.