EC2 instance store cloning or to ebs via gui management console

I have found similar questions here but the answer are either outdated or are from the command line.

The case is this. I have an EC2 instance using instance store (this was the only AMI available for Debian 6 in Ireland). Now through the AWS GUI I can do a snapshot of the instance volume and/or even create a volume. But an image made from the snapshot doesn't boot.

What is the best solution to either clone an EC2 instance that uses instance store OR from the created snapshot of the instance store to launch a new EBS instance (identical clone) FROM the gui aws management console and not command line ? Before turning this down consider that there is not similar question on how to do it via the aws management console.

hint can't be done is not an appropriate answer. As you can create a snapshot of the instance store backed instance and/or a volume and create an AMI from that snapshot.

Solution 1:

You cannot do anything with the data on an instance-store volume from the AWS console. There is a good reason for that, which I will get into later.

First let me clarify a few points from your question:

The case is this. I have an EC2 instance using instance store (this was the only AMI available for Debian 6 in Ireland).

There are several EBS-root Debian 6 AMIs available in EU-West-1 (both 32-bit and 64-bit). Some of these can be found on the Debian Wiki.

Now through the AWS GUI I can do a snapshot of the instance volume and/or even create a volume. But an image made from the snapshot doesn't boot.

What AWS refers to as 'snapshots' can only be made from EBS volumes (see image). By their very nature, instance store volumes are ephemeral - temporary. The only way to capture the state of an instance-store volume is to bundle the files on it (almost like you would by making an archive) - these are not block-level. The snapshots that are mentioned in the AWS console are for EBS volumes - these are block level. In all likelihood, the snapshot you created was of an EBS volume that was attached to the instance, and, since this was not the root volume, trying to run an instance from it continually failed.

Additionally, you cannot add instance-store volumes to a running instance. Instance-storage is a characteristic of the instance type - it comes in fixed volume sizes (e.g. you cannot add a 15GB instance-store volume) and can only be added at the time the instance is launched (either as part of the ec2-run-instances command or if it is built into the AMI.

In general, instance-store instances should be avoided unless you have a very good reason to use them (typically high-io, transient tasks)

Now, to try and answer some of the questions you have asked:

What is the best solution to either clone an EC2 instance that uses instance store OR from the created snapshot of the instance store to launch a new EBS instance (identical clone) FROM the gui aws management console and not command line ? Before turning this down consider that there is not similar question on how to do it via the aws management console.

Use the command line. Before you think I am dismissing what you have asked, allow me to explain. AWS does not access your files - none of the actions that you can perform remotely require any knowledge of what is actually on your instance. So, creating an EBS snapshot occurs at the block level - it doesn't need an understanding of the file-system or permissions to read the files contained on the volume (imagine if AWS needed to be able to read every conceivable file-system on a linux operating system, it would be an impossible task). On the other hand, the process of creating a 'bundle' from an instance-store volume requires read access to the files on the system. You will note that the ec2-bundle-vol must be run from the instance in question (unlike the ec2-create-snapshot command, which can be run without access to the instance).

The command line and API exist for several reasons. Firstly, they are essential to automation - you can't script the 'AWS Console'. Just as important though, they offer a far wider range of options. While the AWS Console continues to add new features all the time, for one reason or another, there are still many things that can only be done from the command line. Choose what works best for you, but do not dismiss either the command line or the GUI interface as both have their uses.

So, since the console has been ruled out, you must use the command line to proceed.

Option 1:

Create a new instance-store AMI from your existing instance. You will need to run:

• ec2-bundle-vol to create the AMI

  ec2-bundle-vol -d /mnt -k $EC2_PRIVATE_KEY -c $EC2_CERT -u USERID -s SIZE

• ec2-upload-bundle to save it to S3

  ec2-upload-bundle -b BUCKETNAME -m MANIFESTFILE -a ACCESSKEY -s SECRETKEY

• ec2-register to register the AMI (this last one can be done from the AWS console).

This is definitely the easiest approach, but personally, I wouldn't go with it. If you are going to put in the effort, you might as well go all out and convert to EBS.

Option 2:

Copy the files you have to an EBS volume and create a new image based on that volume.

You need to first create a new EBS volume (in the same availability zone as your existing instance), and attach that volume to the instance.

Note: It is a good idea to remove any references to ephemeral storage from your /etc/fstab if you plan to use the AMI on different instance types, as the ephemeral storage available on each instance type varies (typically mounted on /mnt).

There are two approaches to this, depending on what you currently have.

1. If you have a bundled volume (e.g. from 'Option 1') you can unbundle the volume, and then use dd to copy it onto your EBS volume (you may need to download the bundle first with ec2-download-bundle):

   ec2-unbundle -m /local/path/to/manifest.xml -s SOURCE_DIRECTORY -d DESTINATION_DIRECTORY dd if=/path/to/image of=/dev/NAME

2. Otherwise, you can use rsync to copy the files on your root volume to the new EBS volume.

   • Format your EBS volume (e.g. mkfs.ext3 /dev/sdf)
   • Mount the EBS volume (e.g. mkdir /mnt/ebs && mount /dev/sdf /mnt/ebs)
   • Stop as many services running on the instance as you can (to minimize the risk of inconsistencies)
   • Flush the caches: (e.g. sync)
   • Copy the files over: (e.g. rsync -aHAXxSP --exclude /mnt/ebs / /mnt/ebs). You could instead use dd, but for a mounted source file system, rsync is better.
   • Flush the caches: (e.g. sync)
   • Unmount the EBS volume: (e.g. umount /mnt/ebs)
   • Check the file system: (e.g. fsck /dev/sdf)

   From here, the rest you can do in the AWS console: make a snapshot of the EBS volume and create an AMI from that snapshot. Launch your new AMI to ensure that everything works.