[libvirt] [PATCH v4 00/20] Incremental Backup API additions

[Eric Blake]

On 2/19/19 1:30 AM, Peter Krempa wrote:

>> Somewhat correct - but we DO have to think about how we plan for the API
>> to grow in the future when we eventually DO fix snapshot
>> deletion/reversion.  Hence my question - would we rather have the
>> creation of a checkpoint at the same time as the creation of an external
>> snapshot (which we DO know we will want) to occur via the existing API
>> (by extending the snapshot XML to include the checkpoint XML as a
>> sub-element), or via a new API (by passing the checkpoint XML as a
>> second parameter)?  Once we've answered that question, it then
>> determines what signature we want for virDomainBackupBegin() (either two
>> separate XML parameters, one for the backup job and one for the
>> checkpoint creation, as presented in v4 of the series, OR as one single
>> XML call where the checkpoint XML is a sub-element of the backup XML).
> 
> This is an interesting point. Given that the snapshot creates new files
> (speaking of external snapshots obviously) you get an implicit
> "checkpoint" at that moment as basically all blocks changed since that
> moment are recorded in the overlay file. For any other snapshot you get
> the same if you apply the overlay file on top of it.
> 
> Now the question is whether we need to be able to track that checkpoint
> explicitly or whether it needs an explicit bitmap, but I'm not that
> familiar with qemu implementation.

The easiest way to perform a differential backup is to bitwise-or all
the bitmaps from the requested checkpoint to the present (where I've
implemented things that only one bitmap is active at a time, to avoid
qemu having to write to an ever-increasing number of bitmaps as more
checkpoints are created).  But unless I create a bitmap at the same time
an external snapshot is created, there is no easy way to make current
qemu expose that all clusters in the current active layer are dirty, to
OR that with the clusters that were marked dirty prior to creating the
snapshot.  So yes, it would be best if the creation of a snapshot can
ALSO be used as the creation of a bitmap, which in turn implies that
creating an explicit checkpoint associated with the snapshot is the way
to go.  Fortunately, with blockdev-add, it is possible to create the
qcow2 file, then create the bitmap, and only then perform the snapshot
operation, so we don't need any new qemu commands to create an explicit
bitmap, but we DO need to figure out where in the libvirt API to wire in
this extra step when external snapshots and checkpoints are both used in
the same image.

> 
> If we don't require any explicit bitmap or marking in qemu to be able to
> track a checkpoint from a snapshot point we can e.g. allow the backup
> API to take an external checkpoint as a reference  for the backup
> itself.
> 
> Creating both a snapshot and a checkpoint is certainly possible but I
> fear that making the snapshot more complex than it is will be an
> unexpected can of worms.

I'm afraid that when I do integrate bitmaps with external snapshots,
that I'll have to modify the creation of snapshots to create a bitmap,
whether or not I also expose the ability in API to create an explicit
Checkpoint object at the same time.  So as long as we have to manage
bitmaps, we might as well also manage the explicit creation of a
checkpoint for both a snapshot as well as for a virDomainBackupBegin call.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org