Auto-Attach EBS-volume to a New Spot Instance?

I am experimenting with EC2 spot instances, and am needing some data to be retained between terminations. Now as I understand it, when the current price goes above my max. bid, it will be automatically terminated. I assume any init scripts I have will be run on shutdown so I can push data off to the EBS before unmounting.

My question is, how can I automatically mount the same EBS volume on the new spot instance once the price goes down, since it won't have any of my init scripts that I would've loaded onto the root volume the first time?

Do I have to create a custom AMI, or is there some other way to achieve this?

Solution 1:

If you simply want the instance to start out with a fixed starting point each time it runs, then you could:

1. Create your own AMI and run that as the instance each time, or

2. Specify a user-data script that installs and configures software to your specifications each time a new instance is started from a standard base AMI.

If you need to keep state between instance runs, then you'll need to save the data somewhere outside of the instance/AMI. For example:

1. The data could be kept up to date on S3, SimpleDB, DynamoDB, RDS, etc.

2. You could designate a permanent EBS volume that the instance dynamically attaches and mounts at startup. This could be done with a user-data script.

Solution 2:

Check out the ec2-spotter project on GitHub. It was created to solve the problem you describe.

Issues (#1 thru #4) described in @mcenzm's answer are real, and are handled correctly in ec2-spotter.

Solution 3:

I have been trying this for a while now, and there are still a number of issues with EC2 spot instances and attached storage.

1. The volume you want to attach may not be in the same zone as the instance ? Zones cannot be specified at launch.
2. Internally attaching with the init script will work, but is a little asynchronous so you will need to test for it or sleep 10 say.

3. I cannot understand why the AWS console shows a "partition" as the primary block device. Maybe to deny us pirate windows images on another partition ? (sda1 instead of sda). Good luck finding the partition table.

   

4. Currently hard to attach using CLI from external point of control, which makes it hard to use snapshots for versioning.

5. So in general I back-up ...tar.bz2 to another box, and can then populate the "work area" upon startup. This is only useful for small volumes of data, so baselining a new ami with both volumes is a good idea. Spot instances are really good for "work unit" or "restartable" work anyway so the concept of fetching work from a server is well established. Given you are waiting 9 minutes in the extreme for a launch, you may not mind formatting your (as small as possible) disk.

   6. EBS is still pretty flaky and you need to pay extra for "optimised". It is still faster than trying to upload your completed work in 90s or so when the instance is "priced out".

   7. I suspect this will all change with any "elastic file" offering.