Database backup to Amazon S3

in site by Ian | 1 comment

mysql-logo

A while ago we posted a simple database dump and FTP backup script we use to backup all the site databases daily. Since jet lag is setting in pretty bad I added an extra option to upload the script to Amazon’s S3 storage system at 4:30 this morning.

The latest version will always be on the wiki.

You need a command line app to get the database dumps into S3. For our Debian server I installed s3cmd with the usual apt-get install s3cmd.

Next setup s3cmd with s3cmd –configure to create a profile. You’ll need the Access and Secret keys from your Amazon account (under security credentials). We made a new set just for backup. I didn’t bother setting the PGP or HTTPS stuff for now. S3cmd documentation is here.

Do a s3cmd ls to list all buckets and make sure the connection is working. If you haven’t already, create a bucket for your backups using the S3 web interface or the s3cmd utility as described in the documentation.

s3cmd put back_$i-$day-$hostname.zip s3://dpdbbackup/back_$i-$day-$hostname.zip

To upload to S3 just add a single line to the old backup script processing loop. Replace dpdbbackup with your bucket name (the version on the wiki now uses a variable to set this).

Run the script with ./db.sh (or whatever you named it) to test.

This entry was posted in site and tagged , , , .

Comments

  1. James says:

    I have managed a number of websites on EC2 instances, the EBS snapshot feature makes keeping full system incremental backups a doddle, my setup keeps one backup for each of the preceding 7 days, and one 1 backup for each of the preceding 12 months.

    Restoring a backup is as simple as creating a new EBS volume from the appropriate snapshot and mounting it.

    One of the sites I don’t mind if it goes down for an hour now and then so I run it on a spot priced instance for a ridiculously small amount of money, in the fairly rare occurance that it gets outbid and shuts down I get an email, and then when the price reduces amazon starts it up again.

    I also have the sites setup so I can quickly load balance in case of very high traffic, just spin up a “slave” instance or three and they register with the amazon load balancer.

    EC2 (and the other AWS services) are simply brilliant, so convenient, often I’ve spun up an instance for an hour or two for experimenting on something, or even spun up 50 or more systems to do some big job in parallel. Nothing really compares.

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.