Painless nightly server backups with Dropbox or SpiderOak

Update! I’ve posted an updated version of this script right here. Enjoy.

If you run a server, you need to be doing backups. There’s no ifs, ands or buts about it. I don’t care how small or big a site you run. It’s important. Whether it’s a hardware failure, a software bug that obliterates your database, a hacker or just negligence on your hosting provider’s side (all of which has happened to me) I assure you something will eventually go wrong. And you will wish to God you’d been doing them.

Over the years I’ve tried a lot of backup and sync solutions, but the three I’ve settled on using in my home, office and server environments are rsync, Dropbox and SpiderOak. Rsync I use mostly for my internal network sync or pushing code to my servers, so I’ll leave that one for a later post. In this post I’ll focus on the server end.

A while back I wrote a custom bash script for managing my backups, which I have setup to run as a cron task each night. It grabs all the user home directories, optimizes and exports the MySQL databases, and copies my MongoDB stores. It neatly compresses all of them and places them in a backup directory. It then fires up Dropbox or SpiderOak to push the backups to the cloud, and ultimately sync it all back to my local machines here in the office. It works flawlessly and gives me a lot of peace of mind.

Because Dropbox and SpiderOak both provide Linux CI binaries, this is all in all a pretty simple solution to setup and maintain. I will note that I switched to SpiderOak for my setup over Dropbox as I found the SpiderOak network to be more reliable, and their Linux binaries to be much more dependable. I also like that I can run SpiderOak on demand with the –batchmode flag rather than have it constantly taking up server memory when I’m not using it. SpiderOak’s attention to security and it’s encryption policies are also greatly appreciated, especially in light if tge recent discoveries about Dropbox’s own policies.

Step 1: Install Dropbox or SpiderOak

Both products offer Linux installation guides on their Wikis. These setup instructions have changed several times even in recent months, so I won’t go through the effort of echoing them here and likely leading you astray when they change down the road.

Dropbox instructions here, and SpiderOak here and here.

Step 2: Backup Script

Here is a modified version of my script. You can use it as a template to customize and build upon for your own needs.

#!/bin/bash

mysqlUser="MYSQL ROOT USER"
mysqlPass="MYSQL ROOT PASSWORD"

backup="/path/to/backups/"

folders[0]="/home/user1/"
folders[1]="/home/user2/"
folders[2]="/some/other/path/"

temp="$PWD/backup.tmp"

echo "Starting Backup..."

stamp=$(date -u  +%F)
backup=$backup$stamp

if [ ! -d "$backup" ]; then
  mkdir "$backup"
fi

echo "-----"
echo -ne "Preparing to backup MySQL tables... "
databases=( $(mysql -u"$mysqlUser" -p"$mysqlPass" --skip-column-names --batch -e "show databases;" 2>"$temp") );
echo "found ${#databases[@]} databases.";

for i in ${databases[@]}; do
  if [ $i != "information_schema" ] && [ $i != "mysql" ] && [ $i != "phpmyadmin" ]; then
    echo -ne "Optimizing and backing up database $i ... "
    mysql -u"$mysqlUser" -p"$mysqlPass" -D "$i" --skip-column-names --batch -e "optimize table $i" 2>"$temp" >/dev/null
    mysqldump -u"$mysqlUser" -p"$mysqlPass" --opt $i | bzip2 -c > "$backup/db_$i.sql.bz2"
    echo "Done."
  fi
done

echo "-----"

for f in "${folders[@]}"
do
  if [ -d "$f" ]; then
    echo -ne "Backing up directory $f ... "
    d=$(basename $f)
    d="$d.tar.bz2"
  elif [ -f "$f" ]; then
    echo -ne "Backing up file $f ... "
    d=${f##*/}
    d="$d.tar.bz2"
  else
    echo "Error! $f could not be found."
    exit 1
  fi

  tar -cjf "$backup/$d" -C / ${f:1}
  echo "Done."
done

if [ ! -s $temp ]; then
  rm -f "$temp"
fi

echo "-----"
echo "Pushing data to SpiderOak..."

SpiderOak --batchmode

echo "-----"
echo "Backup Complete!"
exit 0

Note that if you choose to use Dropbox or installed SpiderOak as a daemon, you’ll want to remove lines 62 through 65. I just prefer to use SpiderOak on-demand.

Step 3: Schedule It

Setup a Cron task to run the script. Be sure to think about user permissions for the script and run the task under the appropriate user/group.

* 3 * * * /path/to/backup.sh

This will run the backup every morning at 3am.

Step 4: Profit?

Aside from occasionally purging old backups you no longer need from your storage directory and SpiderOak, you should be golden. The script will run, the backups will be created and synced to your service of choice, and you’ll have one less thing to worry about.

If you’d like to give SpiderOak a try, you can sign up using this link for 3GB of free backup storage.
If you’d rather use Dropbox, you can sign up using this link for 2.25GB of free backup storage.

Hopes this helps someone out there. If you have any suggestions on improvements, please share them in the comments.

Evan Sims