How to migrate an EC2+EBS standard instance to spot instance?

Solution 1:

Firstly, allow me to clarify a few points. As you pointed out, spot instances are not meant to be run 24/7 - they are intended to provide extra compute capacity for a short time-span at a lower cost. Essentially, they are intended for tasks that can be broken into small pieces - so a terminated instance will not have a significant impact on the overall task.

That said, I have previously run an instance using the spot request model - its usual uptime was about 3 months - and I would typically overbid by about 15x the average market price. While this used to be a reasonably cost effective approach, I found that as more people employed the same technique, the price volatility increased to the point that it was no longer advantageous over a reserved instance.

The average spot price tends to hover around the hourly cost of a medium utilization reserved instance. Doing the math based on the current spot prices vs heavy utilization instances, you get the following:

+-------------+-----------------------------+------------------------------+----------+
|             |           1 year            |            3 year            |   Spot   |
+-------------+-----------------------------+------------------------------+----------+
| Small       | $0.016/h + $195 = $0.0383/h | $0.013/h +$300 = $0.0244/h   | $0.027/h |
| Medium      | $0.032/h + $390 = $0.0765/h | $0.026/h+$600 = $0.0488/h    | $0.038/h |
| Large       | $0.064/h + $780 = $0.153/h  | $0.052/h + $1200 = $0.0977/h | $0.108/h |
| Extra Large | $0.128/h + $1560 = $0.306/h | $0.104/h + $2400 = $0.195/h  | $0.216/h |
+-------------+-----------------------------+------------------------------+----------+

It is evident that in many cases, the 3 year heavy utilization reserved instance works out to less than the current spot price (which is subject to volatility). From my experience, the actual average spot price tends to be at least 50% higher than the baseline market value, since significant spikes are not uncommon.

Now, to actually try and answer your question:

You are correct that spot instances cannot be stopped - in some ways that does agree with the entire premise behind spot instances. Traditionally, an instance will create the EBS volume(s) it is using from snapshots, but it is possible to script the process of attaching an EBS volume. I used to use this script to setup 2 EBS volumes in a RAID0 (I am pretty sure I modified it from some script I found, but I can't find the original at the moment):

#! /bin/sh 
# 
# /etc/init.d/mountec2vol 
# 
# chkconfig: 234 20 50 
# description: Assigns an EC2 EBS Volume to a device and mounts the device 
# 
# To add this as a service run: 
# /sbin/chkconfig --add mountec2vol 
# 
# VARS 
# 
VOL1="vol-xxxxxxxa" 
VOL2="vol-xxxxxxxb" 
DEV1="/dev/sdh1" 
DEV2="/dev/sdh2" 
MOUNT_POINT="/raid" 

export PS1="[\T] [\W]# "
export JAVA_HOME=/usr/java/jre1.6.0_16
export EC2_HOME=/etc/ec2/apitools
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=/root/.ec2/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem 
export EC2_CERT=/root/.ec2/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem

MAX_TRIES=10 

# start/stop functions for OS 

start() { 
touch /var/lock/subsys/mountec2vol
INSTANCE=`curl http://169.254.169.254/latest/meta-data/instance-id 2> /dev/null` 
CTR=0 
/bin/echo "Mounting Elastic Block Store Volumes." 
ec2-attach-volume $VOL1 -i $INSTANCE -d $DEV1 
while [ ! -e "$DEV1" ]; do 
/bin/sleep 1 
CTR=`expr $CTR + 1` 
if [ $CTR -eq $MAX_TRIES ] 
then 
/bin/echo "WARNING: Cannot attach volume $VOL1 to $DEV1 -- Giving up after $MAX_TRIES attempts" 
exit 1 
fi 
done 
ec2-attach-volume $VOL2 -i $INSTANCE -d $DEV2
while [ ! -e "$DEV2" ]; do 
/bin/sleep 1 
CTR=`expr $CTR + 1` 
if [ $CTR -eq $MAX_TRIES ] 
then 
/bin/echo "WARNING: Cannot attach volume $VOL2 to $DEV2 -- Giving up after $MAX_TRIES attempts" 
exit 1 
fi 
done 
depmod -a
modprobe raid0
mdadm --assemble --verbose /dev/md0 /dev/sdh1 /dev/sdh2
if [ ! -d $MOUNT_POINT ]; then 
mkdir $MOUNT_POINT 
fi 
/bin/mount /dev/md0 $MOUNT_POINT 
} 

stop() { 
/bin/echo "Unmounting Elastic Block Store Volumes." 
rm -f /var/lock/subsys/mountec2vol 
/bin/umount $MOUNT_POINT
mdadm -S /dev/md0
ec2-detach-volume $VOL1
ec2-detach-volume $VOL2
} 


case "$1" in 

start) 
start 
;; 

stop) 
stop 
;; 
restart) 
stop 
sleep 5 
start 
;; 
*) 
echo "Usage: $0 {start|stop|restart}" 
exit 1 

esac 

exit 0 

This was for CentOS, but I imagine it would be easy enough to adapt to most other Linuxes. Change the Java version and remove the RAID attributes (and change the mount point) and you will be good to go. This would then be setup as an init script to run when the instance starts. An alternate way of accomplishing this would be to pass the EBS volume IDs as instance user-data, which would allow for a lot more flexibility.

It should be noted that this, of course, will not work for the root EBS volume - this setup presumes a mostly fixed root volume, with data kept on a separate mount-point. You can always set the EBS root volume from ec2-request-spot-instances using the --block-device-mapping parameter.

Solution 2:

What you're looking for can be accomplished through use of their API, I don't think it's possible to use Amazon-supplied automation to do it. You have to write it yourself. It is entirely possible to write a script to create a new spot instance, and associate a specific EBS volume to it.

What you won't be able to do is migrate the full machine-state, you'll still incur outages to spin up new spot instances.

Solution 3:

Check out https://github.com/atramos/ec2-spotter

From the README:

EC2-Spotter is a utility that brings together the best of both worlds -- Spot Instance pricing with the simplicity (persistent EBS filesystem) of On Demand & Reserved Instances. This sounds like cheating, but apparently is not forbidden by the Amazon Terms Of Service.