This last weekend I almost lost all the pictures. We have four kids and have tons of pictures. I had set up the user accounts on our Mac on an external mirrored drive, connected by Firewire. This was mostly good, but this weekend all the applications just refused to connect because of some kind of weird permissions issue. I went through all sorts of stuff to get it working, but at the end of the day I just created a new account and copied all the files over to internal drive on the Mac, then changed ownership of all the files, and thought I was good to go.
That was until my wife went to make a Christmas book for her grandmother. Some of the pictures were missing, and all were blurry. The full sized photos were gone. I ended up having to restore everything from Time Machine, which is good for those kinds of things, so everything turned out ok. But I have always been worried about what would happen if the Mac and the connected drives were stolen. I could use an online backup service… but I don’t like to do that. There had to be a better way.
I found that you could use the UNIX application sync to create a little script that would back up every file to a remote sync server. I did not have one of those, but I did have a USB drive connected to my home wireless router, so I figured I could set is up to copy the files to that. No, go. sync does not work very well connecting to a samba server on the other end. I had to set up a real sync server.
There was an old PC in the basement that I had set up a copy of Ubuntu Linux on last year. Setting up Ubuntu is ridiculously easy, especially on a PC that is a few years old. Just go to ubuntu.com, download the latest copy, put it on a USB stick, and boot to it. It sets up everything automatically.
As I researched this, I found out I could very easily set up a Rsync server on the Linux box and copy over files from the Mac, but it was too easy. It was too easy because sync is not secure. All the files were sent over my wireless unencrypted. I don’t like doing things unencrypted, not protected by passwords. When security is so easy to add into a system, it is worth taking a few extra minutes to do things right.
What I discovered was that you could set up sync to operate over secure shell (ssh), which is encrypted. Furthermore, there is a feature built into ssh that allows trusted computers to login without having to use a username or password by using public/private key encryption. So this is what I set up. There is a simple two line script I put on the Mac that I set to run once a week, and it copies over just the changed files from the Mac to the Linux box hidden in the basement. I have moved over to an 802.11n system, so the wireless runs nice and fast, good for transferring all those gymnastics pictures and videos.
So, the process is as follows:
Give your Linux box a static IP address so you can ssh to it. If you are fancy, give it a name on your internal DNS server. I am not that fancy.
Set up the Mac to be able to ssh to the Linux box with signatures by generating a key pair, doing a secure copy to the non-root user account on the Linux box (call it backup user or something like that), and putting it in the trusted keys file. See this website on ssh-keygen Mac to see how it is done.
Make a directory on the Linux box to store the backup files. I store mine on the external USB drive, so I created a directory called /media/Volume/USBdrive/mac_backup and gave the ownership of the directory to a backup user on the Linux box.
After you do this, you want to make sure that the USB drive mounts when the Linux box is rebooted. Mine didn’t, so I had to make an entry in the /etc/stab file to make sure it mounted on boot.
Create a sync.conf file in the /home/backup user directory. This is the tricky part – do not configure the main box’s sync server or its sync.conf file. This is going to be a mini-resync server that is kicked off when the Mac does an ssh to the backup user account on the Linux box. Here is what the sync.conf file should look like:
path = /media/Volume/mac_backup
read only = false
use chroot = false
Then create the script in a text file on the Mac and name it something like rsync_backup_script.txt. It should contain something like this (the \ is where I had to fit it onto this page, don’t actually type the \ and do put everything on the same line. Also substitute @ for (at character)):
Rsync -av –delete –exclude ‘.DS_Store’ –rsh=”ssh -l backup user”\
/Users/ backup user(at character)192.168.5.5::mac_backup
The sync command copies any new or changed files from the Mac to the Linux box. It also deletes any files on the backup that have been deleted on the Mac.
Make the script executable. There is some way to do this with the GUI, but I just open the terminal on the Mac, navigate to the directory with the file, and type:
do mod u+x rsync_backup_script.txt
Now test everything! You could just go for it and execute the script by entering on the command line:
If that does not work you then need to step through the parts of the process – make sure you can ssh without using a password, check the file permissions on the Linux box, check the sync.conf file, and even enter the command on the Mac terminal line by itself to make sure everything works.
It will take hours for the first backup if your system is anything like mine. The script is set up to compress any files it can, and in future backups, it will only transfer the changes.
This is not limited to backing up Mac’s. You can do the same thing with Windows boxes by installing Cygwin on the Windows machine and going through a similar process. Just create another directory on the backup drive, create a second profile in the sync.conf file that uses a different profile name and points to the new drive location, and you are off and running.
If you do have to use your backup, you can use the opposite command to copy the files back to the Mac. Hopefully, you won’t have to do that, though! Something like this should work:
sync -av –rsh=”ssh -l back up user” backup user(at character)192.168.5.5::mac_backup /Users/
That is the complete exercise. I have gone through the files on the Linux box and made sure they are there, so I am happy. The box has no screen or keyboard connected to it, and it looks like a chunky old PC that no one wants. The noisy 1TB mirrored hard drives in the external array are a little noisy, which is another good reason to keep it hidden away.
Of course, I still make a USB copy every once in a while and take it to work, but that is the backup to the backup to the backup. Hopefully, I will have good copies of my files available for many years to come.