[libvirt] [RFC v2] external (pull) backup API

[Vladimir Sementsov-Ogievskiy]

13.04.2018 12:16, Nikolay Shirokovskiy wrote:
>
> On 13.04.2018 00:16, John Snow wrote:
>>
>> On 04/03/2018 08:01 AM, Nikolay Shirokovskiy wrote:
>>> Hi, all.
>>>                                                                                                   
>>> This is another RFC on pull backup API. This API provides means to read domain
>>> disks in a snapshotted state so that client can back them up as well as means
>>> to write domain disks to revert them to backed up state. The previous version
>>> of RFC is [1]. I'll also describe the API implementation details to shed light
>>> on misc qemu dirty bitmap commands usage.
>>>                                                                                                   
>>> This API does not use existent disks snapshots. Instead it introduces snapshots
>>> provided by qemu's blockdev-backup command. The reason is we need snapshotted
>>> disk state only temporarily for duration of backup operation and newly
>>> introduced snapshots can be easily discarded at the end of operation without
>>> block commit operation. Technically difference is next. On usual snapshot we
>>> create new image backed by original and all new data goes to the new image thus
>>> original image stays in a snapshotted state. In temporary snapshots we create
>>> new image backed by original and all new data still goes to the original image
>>> but before new data is written old data to be overwritten is popped out to the new
>>> image thus we get snapshotted state thru new image.
>>>                                                                                                   
>>> Disks snapshots as well as disks itself are avaiable to read/write thru qemu
>>> NBD server.
>> [snip!]
>>
>> Do you think it's possible to characterize this API proposal as two
>> mechanisms:
>>
>> (1) A mechanism for creating and manipulating "checkpoints" -- which are
>> book-ended by bitmap objects in QEMU -- implemented by the creation,
>> deletion, 'disabling' and 'merging' of bitmaps, and
>>
>> (2) A mechanism for the consumption of said checkpoints via NBD / the
>> "fleecing" mechanisms that allow a live export of a static view of the
>> disk at that time (block snapshots + NBD exports)
> I can't share this view because checkpoints and snapshots are created
> in one transation in pull scheme so you can't not move these two to
> different mechs.
>
> I'll rather see 2 mechanism here at least for pull scheme.
>
> 1. create snapshots (and optionally checkpoints)
> 2. export snapshots
>
>> If this is the case, do you think it is possible to consider (1) and (2)
>> somewhat orthogonal items -- in so far as it might be possible to add
>> support to libvirt directly to add push-model support for writing out
>> these checkpoints?
>>
>> i.e.
>>
>> once you have created a temporary snapshot and merged the various
>> component bitmaps into it, instead of creating an ephemeral block
>> snapshot and exporting it via NBD, we request a `blockdev-backup` with a
>> libvirt-specified target instead?
>>
>> You don't have to add support for this right away, but I would really
>> enjoy if any API we check in here has the capacity to support both
>> push-and-pull paradigms without getting too ugly.
>>
>> Does that sound like it can easily fit in with your designs so far?
>>
> I think push scheme require 3rd (1st is fleece snapshots, 2nd is exporting
> snapshots) API. First push backup has nothing to do with exporting of course. Second
> contrary to fleece snapshots it will require additional parameter - a checkpoint
> in past in case of incremental backup. It also have quite different image parameter.
> In case of fleece snapshot fleece image will only store small delta even in case of
> full backups. In case of push backup image will store full disk in case of full
> backups.
>
> Nikolay

Hmm, to use checkpoints with push backups, we just need to start normal 
incremental backups in transaction with checkpoint creation.

So. Checkpoint is a separate thing, but it is useless, if we didn't 
create some kind of backup (push or pull) in same transaction with it.

If we in future, implement checkpoints support in qemu, libvirt api 
realization will become simpler. But anyway, we should not create 
checkpoints
separately in libvirt, as they will useless in this case.

-- 
Best regards,
Vladimir