[libvirt-users] how "safe" is blockcommit ?

[Lentes, Bernd]

----- On Sep 7, 2018, at 9:26 PM, Eric Blake eblake at redhat.com wrote:

> On 09/07/2018 12:06 PM, Lentes, Bernd wrote:
>> Hi,
>> 
>> currently i'm following
>> https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit. I 'm
>> playing around with it and it seems to be quite nice.
>> What i want is a daily consistent backup of my image file of the guest.
>> I have the idea of the following procedure:
>> 
>> - Shutdown the guest (i can live with a downtime of a few minutes, it will
>> happen in the night).
>>    And i think it's the only way to have a real clean snapshot
> 
> Not the only way - you can also use qemu-guest-agent with a trusted
> guest to quiesce all filesystem I/O after freezing database operations
> at a consistent point, for a clean snapshot of a live guest.  But
> shutting down is indeed safe, and easier to reason about than worrying
> whether your qga interaction is properly hooked into all necessary
> places for a live quiesce.
> 
>> - create a snapshot with snapshot-create-as: snapshot-create-as guest testsn
>> --disk-only
>> - start the guest again. Changes will now go into the overlay, as e.g. inserts
>> in a database
>> - rsync the base file to a cifs server. With rsync not the complete, likely big
>> file is transferred but just the delta
> 
> We're also trying to add support for incremental backups into a future
> version of libvirt on top of the qemu 3.0 feature of persistent bitmaps
> in qcow2 images, which could indeed guarantee that you transfer only the
> portions of the guest disk that were touched since the last backup.  But
> as that's still something I'm trying to code up, your solution of using
> rsync to pick out the deltas is as good as anything you can get right now.
> 
>> - blockcommit the overlay: blockcommit guest /path/to/testsn --active --wait
>> --verbose --pivot
>> - delete the snapshot: snapshot-delete guest --snapshotname testsn --metadata
>> - remove the overlay
>> 
>> Is that ok ? How "safe" is blockcommit on a running guest ?
> 
> Yep, that's the right way to do it!  It's perfectly safe - the guest
> doesn't care whether it is reading/writing from the backing file or the
> overlay, and even if the blockcommit action is aborted, you can restart
> it for the same effects.
> 
> (Okay, if you want to get technical, you need to know that committing
> from 'Base <- mid <- top' down to 'Base' leaves 'mid' in an inconsistent
> state - but that's not something the guest can see through 'top'; and
> your specific case is just committing to the immediate backing layer
> rather than skipping layers like that).
> 
>> It's possible that during the rsync, when the guest is running, some inserts are
>> done in a database.
> 
> As long as the backing file is read-only during the rsync (which it is,
> since all your guest writes are going to the overlay), then nothing the
> guest can do will interfere with what rsync can see.  Just be sure that
> you don't start the blockcommit until after rsync is done.
> 
>> Is it safe to copy the new sectors (i assume that's what blockcommit does) under
>> a running database ?
>> Or is it only safe doing blockcommit on a stopped guest ?
> 
> Live blockcommit is safe, and exists so you don't have to stop the guest.
> 
> For a bit more insight into what is going on under the hood, the slides
> from my KVM Forum talk from a few years back may give some nice insights:
> http://events17.linuxfoundation.org/sites/events/files/slides/2015-qcow2-expanded.pdf
> 
>> 
>> Thanks for any answer.
>> 
>> Bernd
>> 
>> P.S. Is the same procedure possible when the guest disk(s) reside directly in a
>> plain logical volume, without a file system in-between ?
> 
> Live blockcommit works onto any host storage protocol (whether
> filesystem, block device via LVM, or even remote access such as NBD or
> ceph).  The key is that your overlay is a qcow2 file that is tracking
> the deltas during the time in which you are capturing your backup of the
> backing file, and then blockcommit safely writes those deltas back into
> the backing file prior to no longer needing the overlay.
> 
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3266
> Virtualization:  qemu.org | libvirt.org

Hi Eric,

a big thanks to this outstanding clear and thorough answer.
It's a pleasure to get such information from the developers themselves, and so quick !

Bernd
 

Helmholtz Zentrum Muenchen
Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
Ingolstaedter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: MinDir'in Baerbel Brumme-Bothe
Geschaeftsfuehrer: Prof. Dr. med. Dr. h.c. Matthias H. Tschoep, Heinrich Bassler, Dr. rer. nat. Alfons Enhsen
Registergericht: Amtsgericht Muenchen HRB 6466
USt-IdNr: DE 129521671