Clone kvm/qemu VM to a different server

And here is what I do where $VM is the VM name, and DEST is the destination hypervisor hostname. It does this on running VMs using dd and snapshot LVM disks (It is assuming that the LVM Group is called HypGroup00).

I just threw this together, so it isn't necessarily the prettiest, but does the job, I'm using it to migrate some VMs from a CentOS 5.9 hypervisor to CentOS 6 with minimal downtime.

This is for CentOS 5.9, tested on going to CentOS 5 and 6 as a destination.

VM=webserver
DEST=hyp5

Figure out which disks to copy, since most of our VM disks are referenced via the /dev/mapper/ path rather than the /dev/volgroup/volname path. So we need to translate between the two.

DISKS=`cat /etc/libvirt/qemu/$VM.xml |  grep HypGroup00 | sed "s|.*/\(HypGroup00-.*\)'/>|\1|"`

LVS=""

for disk in $DISKS; do
  echo VM Disk $disk
  LV=`lvdisplay /dev/mapper/$disk | grep "LV Name" | awk '{print $3}'`
  LVS="$LVS $LV"
done

Copy over the VM definition and register it

virsh dumpxml $VM > /tmp/$VM.xml
scp /tmp/$VM.xml $DEST:/tmp/
ssh $DEST virsh undefine $VM > /dev/null 2>&1
ssh $DEST virsh define /tmp/$VM.xml

Now create the LV on the remote server

for lv in $LVS; do
  ssh $DEST lvremove --force $lv
  SIZE=`lvdisplay $lv | grep "LV Size" | awk '{print $3}'`
  SHORTNAME=`basename $lv`
  ssh $DEST lvcreate -L"$SIZE"G -n$SHORTNAME HypGroup00
done

Now create the snapshot LVs and start copying over the data

virsh suspend $VM
for lv in $LVS; do
  lvcreate -L10G -s -n $lv-snapshot $lv
done
virsh resume $VM

Copy the disks across

for lv in $LVS; do
  echo Copying LV $lv, this will take a while...
  time dd bs=1M if=$lv-snapshot | gzip --fast | ssh $DEST "gzip -d | dd of=$lv"
done

A more complex version of the above that shows progress if required from dd, no good for multiple copies because of /tmp/pid, but you can change to to include $$ if you wish.

(dd bs=1M if=$lv-snapshot & echo $! >&3 ) 3>/tmp/pid  2> >(grep 'copied' 1>&2) | ssh $DEST "dd bs=1M of=$lv" &
# Need this sleep to give the above time to run
sleep 1
PID=$(</tmp/pid)

while kill -0 $PID; do
  kill -USR1 $PID
  sleep 5
done

Clean up

for lv in $LVS; do
  lvremove --force $lv-snapshot
done

Okay so the way I was doing it actually did work just fine. The problem is just that I didn't have enough resources to run that VM. So just to answer my own question... here are the details to how I did the VM duplication across different servers without shared disk.

Because you don't have a shared disk, you can't do a typical 'clone' and then 'migration'. Instead you do a typical clone

1. Here's the command to do the cloning (/local/vm/ is the path to your VM images, usually /var/something/):

   virt-clone --original=vm-to-clone --name=cloned-vm -f /local/vm/cloned-vm.img --mac=xx:xx:xx:xx:xx:xx

2. Now copy that img file from one server to the other... my servers can't talk directly to eachother, so I use this little SSH redirect to do the trick:

   ssh -n server1 '(cd /local/vm/; cat cloned-vm.img)' | ssh server2 '(cd /local/vm/; cat > cloned-vm.img)'

3. Then copy the config for that VM over:

   ssh -n server1 '(cd /etc/libvirt/qemu/; cat cloned-vm.xml)' | ssh server2 '(cd /etc/libvirt/qemu/; cat > cloned-vm.xml)'

4. Update the config with any new changes. In my case (and this was what was causing my problem), I needed to lower the "memory" and "currentMemory" attributes.

5. Add the new VM to libvirt:

   virsh define /etc/libvirt/qemu/cloned-vm.xml

6. Run it:

   virsh create /etc/libvirt/qemu/cloned-vm.xml