[libvirt-users] Backup a VM (using live external snapshot and blockcommit)

[Jérôme]

Hi.

I'm following here a conversation that was initiated on Kashyap's
website [1].

We have a server we use as a host for virtual machines using KVM
(virt-manager used for VM creation) and we would like to setup VM
backups. Basically, we're thinking of a backup schedule like "keep 7
daily and 4 weekly backups". We'd rather not shutdown the VM every day
so live backups would be nice.

I've been doing my best with documentation found on the Internet. It is
likely that the path I chose was not the best, so feel free to tell me
if I'm asking the wrong questions and I should be proceeding totally
differently.

AFAIU, backups can be done at filesystem level (using LVM) and at
virtualization level (using libvirt). We chose the libvirt way.

AFAIU, live backups using libvirt may be done thanks to blockcommit as
explained here on the wiki [2].

-> Considering our use case, is this the recommended way?

Assuming yes, here is the plan.

I wrote a script that does  

# Create snapshot
virsh snapshot-create-as --domain $VM_NAME snap --diskspec
vda,file=$VM_DIR/"$VM_NAME"-snap.qcow2 --disk-only --atomic
--no-metadata --quiesce

# Copy frozen backing file
cp $VM_DIR/"$VM_NAME".qcow2 $SNAP_FILEPATH

# Blockcommit snapshot back into backing file
virsh blockcommit $VM_NAME vda --active --pivot

# Remove snapshot file
rm $VM_DIR/"$VM_NAME"-snap.qcow2

Variables should be self-explanatory: 
- VM_DIR is the directory where the VM are stored
- VM_NAME is the name of the VM, and its qcow2 file is called
VM_NAME.qcow2
- SNAP_FILEPATH is the full path (including name) where the backup
should be created

Using this scheme, we only keep snapshots for the time of the VM file
copy, which is less than a minute. The backing chain is at most 'back <-
snap', and most of the time just 'back'.

If something ever happens to the VM (human error while being logged as
root, attack from the internet,...), we'll turn off the VM, replace its
qcow2 file and turn it back on.

I understand that this method only saves disk states, so the VM will be
started as if it had been powered-off suddenly while running (not quite:
thanks to the '--quiesce' option, at least the disks are in a sane
state). Not perfect but better than nothing. Those backups are meant to
be used only when all else failed, anyway, it's not daily use.

-> Does this make sense? Am I missing a feature or even a different
approach that would make things simpler or more secure? Am I using
libvirt snapshots for what they're not meant to?

-> Anything wrong about my snapshot-create-as and blockcommit command
lines? May I remove the snapshot with only a rm command?

Now, a few side questions, as I might have messed up with the VM I was
experimenting with.

I used the same command lines as described above, except I didn't pass
the '--no-metadata' option. Once the backing file was copied, I deleted
the snapshot qcow2 file and thought I was done with it, until I realized
the snapshot was still listed by virsh snapshot-list. And I couldn't
find a way to delete it. (For the record, I asked on serverfault about
that [3].)

Ultimately, I found the snapshot's .xml descriptor and deleted it (in
fact, moved it) while libvirtd was down. Now, the snapshot is not listed
anymore.

-> Am I getting away with it? Are there still some traces about that
snapshot? Is my VM in an unsafe state? Anything I should do about it?

-> What would be the proper way of dropping an external snapshot that
was created without the '--no-metadata' option, then blockcommitted? I
understand libvirt doesn't do it yet.

Thanks for any hint. I naively thought our use case was pretty usual,
and I must admit I didn't think I'd have to dive into this complexity,
which is why I'm thinking there might be a more "common" way...

[1]
http://kashyapc.com/2014/10/07/libvirt-blockcommit-shorten-disk-image-chain-by-live-merging-the-current-active-disk-content
[2]
http://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
[3]
http://serverfault.com/questions/721216/delete-orphan-libvirt-snapshot

-- 
Jérôme