Backup-Manager and S3

 

This article shows how to incremental backup  all sensible directories of an Ubuntu box and store it in a S3 bucket.

Install backup manager: 

[code language=”plain”]sudo apt-get install backup-manager[/code]

During initial setup you can use this configuration as reference (compatible with preseed sintax):

# Choices: tar, tar.gz, tar.bz2, tar.lz, zip, dar
backup-manager backup-manager/filetype select tar.gz

# Choices: long, short
backup-manager backup-manager/name-format select long

# Age of kept archives (days):
backup-manager backup-manager/time-to-live string 7

## Modify this to other value if you want to keep backups locally too. At least a week should keept


# Owner user of the repository:
backup-manager backup-manager/repo_user string root

–# Owner group of the repository:backup-manager  backup-manager/repo_group       string  root–

# Archives location:
backup-manager backup-manager/backup-repository string /var/archives


# Encrypt archives?
backup-manager backup-manager/enable_encryption boolean false

# Choices: never, daily, weekly, monthly
backup-manager backup-manager/cron_frequency select daily

## This install proper script in /etc/cron.daily

# Follow symlinks?
backup-manager backup-manager/dump_symlinks boolean false

## If set to yes huge archive could be created…

# Directories to backup:
backup-manager backup-manager/directories string /etc /home

## here is where you decide what to backup

# Directories to skip in archives:backup-manager  backup-manager/blacklist        string  /var/archives */tmp/* */cache/* ## Here you decide what to not backup. In this case  all contents of all directory named tmp and chache will be ignore by backup process. # Enable backup-manager’s uploading system?backup-manager  backup-manager/want_to_upload   boolean false ## do nou use standard s3 upload manager. 

Edit  /etc/backup-manager.conf  and substitute the  line:

export BM_ARCHIVE_METHOD="tarball"

with:

export BM_ARCHIVE_METHOD="tarball-incremental"

Verify that other configuration directive match your desiderata.

Test  backup manager configuratin just typing:

[code language=”plain”]sudo backup-manager[/code]

This will create a copuple of files in /var/archives directory.

Backup-manager is natively able to manage archive uploading to s3. Unfortunately its implementation contains some bug. In particular it seams that backup-manager-upload is not able to manage S3 redirect on http PUT verb  (see http://docs.amazonwebservices.com/AmazonS3/latest/dev/Redirects.html to understand how S3 protocol works).

To bypass this limitation you can install and configure the great s3cmd tool:

[code language=”plain”]sudo apt-get install s3cmd sudo s3md –configure[/code]

Create a bucket  in your AWS S3 account to store backup files. To dot you can use Amazon AWS Console or even s3cmd:

sudo s3cmb mb s3://MYBUCKET  

Please, substitute MYBUCKET with an unique name of your preference.

Using AWS console or S3 API configure your bucket lifecicle to store just the amounts of backups you want (I use 35 days, 5 weeks). 

Test transfering new archives to S3 with this command:

[code language=”plain”]/usr/bin/s3cmd -v put /var/archives/*.$(date “+%Y%m%d”).* s3://MYBUCKET/[/code]

Now just update the schedule ust adding previous line at the end of /etc/cron.daily/backup-manager

That’s it. Every day an incremental backup will be done and stored to your s3 bucket.