This is my first serious attempt at a Python tool on github. I figured it was about time, as I’ve been leveraging Open Source tools for a long time, I might as well try to give a little bit back. Please check out the project and leave feedback by emailing, opening a github or issue or commenting here, I’d love to see what can be done with this tool, there are lots of bugs to shake out and things to improve. Even better if you have some code you’d like to contribute, this is very much a work in progress!
Here is the project – https://github.com/jmreicha/autosnap.
Essentially, this tool is designed to ease the management of the snapshot and volume lifecycle in an AWS environment. I have discovered that snapshots and volumes can be used together to form a simple backup management system, so by simplifying the management of these resources, by utilizing the power of the AWS API, you can easily manage backups of your AWS data.
While this obviously isn’t a full blown backup tool, it can do a few handy things like leverage tags to create and destroy backups based on custom expiration dates and create snapshots based on a few other criteria, all managed with tags. Another cool thing about handling backups this way is that you get amazing resiliency by storing snapshots to S3, as well as dirt cheap storage. Obviously if you have a huge number of servers and volumes your mileage will vary, but this solution should scale up in to the hundreds, if not thousands pretty easily. The last big bonus is that you can nice granularity for backups.
For example, if you wanted to keep a weeks worth of backups across all your servers in a region, you would simply use this tool to set an expiration tag of 7 days and voila. You will have rolling backups, based on snapshots for the previous seven days. You can get the backup schedule fairly granular, because the snapshots are tagged down to the hour. It would be easy to get them down to the second if that is something people would find useful, I could see DB snapshots being important enough but for now it is set to the hour.
The one drawback is that this needs to be run on a daily basis so you would need to add it to a cron job or some other tool that runs tasks periodically. Not a drawback really as much of a side note to be aware of.
There is a tiny bit of overhead to get started, so I will show you how to get going. You will need to either set up a config file or let autosnap build you one. By default, autosnap will help create one the first time you run it, so you can use this command to build it:
If you would like to provide your own config, create a file called ‘.config‘ in the base directory of this project. Check the README on the github page for the config variables and for any clarifications you may need.
Use the –help flag to get a feeling for some of the functions of this tool.
$ autosnap --help usage: autosnap [--config] [--list-vols] [--manage-vols] [--unmanage-vols] [--list-snaps] [--create-snaps] [--remove-snaps] [--dry-run] [--verbose] [--version] [--help] optional arguments: --config create or modify configuration file --list-vols list managed volumes --manage-vols manage all volumes --unmanage-vols unmanage all volumes --list-snaps list managed snapshots --create-snaps create a snapshot if it is managed --remove-snaps remove a snapshot if it is managed --version show program's version number and exit --help display this help and exit
The first thing you will need to do is let autosnap manage the volumes in a region:
This command will simply add some tags to help with the management of the volumes. Next, you can take a look and see what volumes got picked up and are now being managed by autosnap
To take a snapshot of all the volumes that are being managed:
And you can take a look at your snapshots:
Just as easily you can remove snapshots older than the specified expiration date:
There are some other useful features and flags but the above commands are pretty much the meat and potatoes of how to use this tool.
I know this is not going to be super useful for everybody but it is definitely a nice tool to have if you work with AWS volumes and snapshots on a semi regular basis. As I said, this can easily be improved so I’d love to hear what kinds of things to add or change to make this a great tool. I hope to start working on some more interesting projects and tools in the near future, so stay tuned.