[libvirt] RFCv2: virDomainSnapshotCreateXML enhancements

[Stefan Hajnoczi]

On Thu, Aug 25, 2011 at 3:06 PM, Eric Blake <eblake at redhat.com> wrote:
> On 08/25/2011 05:54 AM, Stefan Hajnoczi wrote:
>>
>> On Tue, Aug 23, 2011 at 4:47 PM, Eric Blake<eblake at redhat.com>  wrote:
>>>
>>> On 08/23/2011 09:35 AM, Stefan Hajnoczi wrote:
>>>>
>>>> On Tue, Aug 23, 2011 at 4:18 PM, Eric Blake<eblake at redhat.com>    wrote:
>>>>>
>>>>> On 08/23/2011 09:12 AM, Stefan Hajnoczi wrote:
>>>>
>>>>  The
>>>> kinds of apps I am thinking about cannot make use of this API.
>>>
>>> What sort of APIs do you need for the apps you are thinking about?
>>> Without
>>> details of your needs, it's hard to say whether we can build on the
>>> framework we have to add the additional features you want.
>>
>> Take virt-manager and virsh, they should probably provide easy-to-use
>> snapshot functionality to users (i.e. snapshot create, snapshot
>> revert, snapshot delete, snapshot list).
>
> And to some degree, virsh already has those.  As I've been writing the
> series, I've been adding flags to those existing commands to expose the new
> functionality, but the core concepts remain the same.  The end goal is that
> 'virsh snapshot-revert dom snap' will properly revert for any type of
> snapshot, or else give a sensible error message of why it cannot revert by
> default and how to fix that (such as by acknowledging that the revert is
> risky because the snapshot was created prior to the point in time where
> libvirt stored full domain xml, so adding the --force flag is sufficient to
> promise that your current domain xml is compatible with the xml that was in
> use at the time the snapshot was created).
>
>>  These tools will need to
>>
>> understand the different storage scenarios and special flags required
>> depending on the type of snapshot, state of the VM, etc.  These code
>> paths need to be duplicated in all clients that want to  use the
>> libvirt API.
>
> Yes.  And the libvirt API is already achieving that.  'virsh
> snapshot-create' is a single wrapper that knows how to create system
> checkpoints (qemu savevm) and disk snapshots (qemu snapshot-blkdev), without
> the user having to know the difference between those commands. And as more
> snapshot patterns are added to the API, the user interface will still be
> 'virsh snapshot-create', with the only changes needed being to the xml used
> in <domainsnapshot> to specify whether the snapshot is done at the qemu
> qcow2 layer, or at the lvm layer, or so forth.
>
>>  This is what I'm pointing out.  In other APIs like
>> VMware's VIX, the snapshot functionality doesn't leak the special
>> cases so doing simple things is simple.
>
> The unadorned use of 'virsh snapshot-create' is already the simplest -
> create a full system checkpoint.  It is only when you want less than a full
> system checkpoint where you have to start being specific.
>
>>
>> The particular case I care about is a backup solution that wants to:
>> 1. Find out which VMs are running
>> 2. Snapshot a set of running or stopped VMs
>
> Yes, 'virsh snapshot-create' works on both running and stopped VMs.
>
>> 3. Copy the snapshot disk contents off-host
>
> Possible with offline internal snapshots using qemu-img, but not yet exposed
> by libvirt.  Not possible with online internal snapshots until qemu exposes
> more functionality.  Possible with offline or online external snapshots
> using cp, but not yet exposed by libvirt.  At any rate, yes, we will need to
> add new libvirt APIs to access this without the user having to know whether
> to use qemu-img, cp, or some other means, but we need qemu help for part of
> this task.
>
>> 4. Perform incremental snapshots and only copy dirty blocks off-host
>
> Not possible with either qemu-img (offline) or qemu (online); again, we will
> need new libvirt API, but we also need hypervisor functionality to expose
> this.
>
>> 5. Be able to completely restore a VM including its configuration and
>> disk contents
>
> My pending series just fixed 'virsh snapshot-restore' to properly do this.
>
>> The point I'm trying to make is that an API should provide a
>> vocabulary to handle tasks at a certain level of abstraction.  If the
>> API is just a pass-through of the underlying primitives, then it
>> doesn't provide much gain over doing things without the API.
>
> virDomainSnapshotCreateXML is indeed a higher layer than qemu's
> snapshot_blkdev, but it sounds like you want yet another layer on top of
> virDomainSnapshotCreateXML.

The virsh commands you have described sound good.  That's the level at
which I imagine third-party tools and custom scripts would want to
work with snapshots.  For special cases the low-level APIs are
necessary.

A virsh snapshot-create libvirt API would be good so that non-virsh
apps using libvirt do not need to duplicate its behavior.

Back to QEMU support, here's what I see missing:

1. merge_blkdev to flatten a COW file into its backing file snapshot.
This undoes the effect of snapshot_blkdev.  Can be used to "delete" a
snapshot.
2. Dirty block API
3. Reading snapshot contents of live image.  My focus is on the disk
snapshot case and not on savevm internal snapshots, but it could still
be useful.

A backup workflow involves taking consistent snapshots periodically
(e.g. once a day).  Therefore it is important to keep the backing file
chain at a fixed length and we need the merge_blkdev command in QEMU.
This looks like the next QEMU task to tackle, I'll try to propose
something next week in order to get start on merge_blkdev.

Stefan