May 052018
 

We have all lost a hard drive at one time or another on a laptop or desktop computer and it always seems like it happens right after several weeks of not performing backups.  Last year, I lost about 15 years of research on an external drive that failed.  I had this system that has worked as long as I can remember where I simply swapped an external drive every two years with a new one after copying the data.  What failed on me though was I became over-confident in this system and wiped out the older drives in order to make room for something else, meanwhile the current drive decided to barf after only about 6 months of usage … literally within a couple weeks of me wiping the previous drives clean.  I was pretty pissed to say the least.  So, lesson learned, I decided to implement a better backup plan.  I wanted a way that would work and be simple.  Instead of a file server and transferring data over a wire, I wanted an external drive I could plug-in and leave plugged in while working or at home or in some motel.  I wanted full backups and I wanted it to be incremental to save space.  This was how I accomplished these tasks …

First thing you need to do is make sure you have rsnapshot available in your linux distributions releases.  Install it.  You should probably brush up on rsnapshot here

Second thing you will want to do is get you a brand new external usb, firewire, whatever you use and get it to auto mount in linux.  I did this by:

  • plugging the drive in and finding out what drive letter it was assigned (/dev/sdc for me)
  • deleting existing partitions using fdisk
  • recreating linux partition using fdisk
  • creating a xfs filesystem on the drive using mkfs.xfs /dev/sdc1
  • setting a drive label using xfs_admin to backupdrive
# xfs_admin -L backupdrive /dev/sdc1
  • making sure the label stuck
# xfs_admin -l /dev/sdc1
  • creating an mountpoint (I used /mnt/sdc for simplicity but this is very important because my wrapper script checks that mountpoint to make sure the drive is mounted)
  • creating an /etc/fstab entry so it would mount when plugged in
LABEL=backupdrive /mnt/sdc xfs defaults 0 0
  • I edited the /etc/rsnapshot.conf file and made a few changes, namely to create all the dirs I wanted backed up which was everything
# change the path at the top for snapshot_root to match your mountpoint
# adjust the backup locations at the bottom
# change hostname to your hostname at the bottom
 
config_version	1.2
 
# change this mountpoint path to match your external drive
snapshot_root	/mnt/sdc/rsnapshot/
 
no_create_root	1
cmd_cp		/bin/cp
cmd_rm		/bin/rm
cmd_rsync	/usr/bin/rsync
cmd_logger	/usr/bin/logger
 
# our backup schema
retain	alpha	6
retain	beta	7
retain	gamma	7
retain	delta	3
 
verbose		3
loglevel	4
logfile	/var/log/rsnapshot.log
lockfile	/var/run/rsnapshot.pid
link_dest	1
use_lazy_deletes	0
 
# what we are backing up
# make sure to use tabs and not spaces between each column
# replace hostname with your hostname
 
# this creates backups like:
# /mnt/sdc/rsnapshot/alpha.0/hostname/
 
backup	/home/		hostname/
backup	/etc/		hostname/
backup	/root/	hostname/
backup	/bin/	hostname/
backup	/boot/	hostname/
backup	/lib/	hostname/
backup	/lib32/	hostname/
backup	/opt/	hostname/
backup	/run/	hostname/
backup	/sbin/	hostname/
backup	/usr/	hostname/
backup	/var/	hostname/
  • i wrote a wrapper script which is very important.  This wrapper script makes sure the external drive is mounted before performing the backup.  Without this check, you would potentially fill your hard drive up.  Make sure you change the mountpoint in the script and make it executable, I used the path /usr/local/bin/rsnapshotwrapper.sh which will be shown in the cron entries below
#!/bin/bash
#===============================================================================
#
#          FILE:  /usr/local/bin/rsnapshotwrapper.sh
# 
#         USAGE:  /usr/local/bin/rsnapshotwrapper.sh <option>
# 
#   DESCRIPTION:  
# 
#       OPTIONS:  requires backup schema passed as option
#  REQUIREMENTS:  ---
#          BUGS:  ---
#         NOTES:  ---
#        AUTHOR:  Ed Wiget
#       COMPANY:  
#       VERSION:  1.0
#       CREATED:  02/05/2017 01:06:41 PM EDT
#      REVISION:  ---
#===============================================================================
 
# make sure we pass at least 1 option
if [ $# -ne 1 ]
then
  echo
  echo "    Usage: $0 <option>"
  echo
  exit 1
fi
 
# make sure we have our backup drive mounted
# change sdc1 to your drive letter
if [ "`mount | grep "/dev/sdc1" | wc -l`" = "1" ]; then
 
	# we have the drive mounted so we do whatever option
	case $1 in
		alpha ) /usr/bin/rsnapshot alpha;;
		beta ) /usr/bin/rsnapshot beta;;
		gamma ) /usr/bin/rsnapshot gamma;;
		delta ) /usr/bin/rsnapshot delta;;
	esac
else
	# the drive is not mounted, send error
	echo "please mount backup drive"
	exit 1
fi
  • finally I set up some crontab entries that calls the wrapper script using the option required.  The wrapper script then makes sure the drive is mounted.  If the drive is mounted, it then runs the correct rsnapshot command.  If the drive is not mounted, it simply exits with an error.
# our rsnapshot backup
# 6 hourly backups at 0,4,8,12,16,20
0 */4 * * * /usr/local/bin/rsnapshotwrapper.sh alpha
# 1 daily backup at 11:50 PM
50 23 * * * /usr/local/bin/rsnapshotwrapper.sh beta
# 1 weekly backup at 11:40 PM on Saturdays
40 23 * * 6 /usr/local/bin/rsnapshotwrapper.sh gamma
# 1 monthly at 11:30PM on the first day of the month
30 23 1 * * /usr/local/bin/rsnapshotwrapper.sh delta

Once you have everything set up, you should probably comment out the crontab entries until you have the first backup done.  On my laptop, it took roughly 2 hours to backup 465G of data to external drive.  Here is what I would suggest:

  • run ‘rsnapshot configtest’ and make sure it returns ok
  • run ‘rsnapshot -t alpha’ as root user and look over the commands
  • once you are satisfied with the above, run ‘rsnapshot -v alpha’ and wait for it to complete.  The v option just makes it more verbose.  If you run into problems, try again but replace the lowercase v with a capital V to make it even more verbose
  • Once you get the initial backup done, wait about 2-3 hours while doing normal work and then run this command to determine how long it took to complete the backup:  ‘time rsnapshot alpha’
  • Adjust the backup start times based on the above, notice that each starts about 10 minutes earlier than the daily, i.e. beta starts 10 minutes earlier than alpha, gamma starts 10 minutes earlier than beta, delta starts 10 minutes earlier than gamma.
  • Once you are satisfied with the start times, enable the crons and know you will have 3 months of backups over time and as long as you keep the external drive plugged in you will only ever lose at most 4 hours of work.
  • Over many months of using this, because it is an incremental backup and most files seldom change, the space required on the external disk has barely been larger than the used hard drive space