How to (properly) back up a live QEMU/KVM VM?

I'm currently engineering a backup solution for KVM VM's as an additional measure to traditional backups.

Unfortunately, all currently (August 2013) existing solutions I came across so far either:

• do not ensure a consistent backup of the VM (losing RAM state, creating a dirty image, or other things),
• or require lengthy downtime (complete VM shutdown while backing up).

I'm aware of QEMU/libvirt's functionality of taking snapshots, however, it's not yet usable since:

• image-internal snapshots present you with an ever-changing image file, resulting in a likely dirty backup (assuming one uses qcow2 images at all).
• one cannot yet merge a currently active external snapshot into the original backing image ("blockcommit").

Out of the above reasons, I'm now implementing a script that:

1. Saves the VM's state and halts it
2. Sets up a devicemapper snapshot(s) where the VM's disk images and state reside
3. Resumes the VM
4. Mount the snapshot(s) of step 2.
5. Backs up the VM's disk and state (configuration for convenience)
6. Merges back the snapshot(s).

If I got everything right, this will take consistent backups of VM's with only seconds (if at all, since 1-3 is fast, possibly sub-second) of downtime. Of course, when restoring, the VM will be way in the past, but at least giving me the option of an orderly shutdown/reboot.

Am I missing something with this solution? Or has someone indeed already implemented this?

Solution 1:

https://libvirt.org/kbase/live_full_disk_backup.html

This is possible with versions: QEMU 2.1 (and above), libvirt-1.2.9 (and above).

$ virsh domblklist vm1
Target     Source
------------------------------------------------
vda        /export/images/base.img 

$ virsh snapshot-create-as --domain vm1 guest-state1 \
--diskspec vda,file=/export/images/overlay1.qcow2 \
--disk-only --atomic --quiesce

$ rsync -avhW --progress /export/images/base.img \
/export/images/copy.img

$ virsh blockcommit vm1 vda --active --verbose --pivot

If you don't have QEmu Guest Agent inside, you can omit the --quiesce option, in which case when you restore from backup it would look like the system has crashed, but should be good anyway since the operating system inside should flush required data and maintain consistency of it's filesystems.

Solution 2:

Halting the VM will not flush the in flight IOs, you should use qemu-ga to quiesce the VM.

Since QEMU introduced block migration, you could try and use it to migrate a VM into a backup image, saving both the memory state and the disk