[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] [RFC] externall (pull) backup API



ping

On 14.11.2017 18:38, Nikolay Shirokovskiy wrote:
>   Table of contents.
> 
>   I  Preface
> 
>   1. Fleece API
>   2. Export API
>   3. Incremental backups
>   4. Other hypervisors
> 
>   II Links
> 
> 
> 
> 
>   I Preface
> 
> This is a RFC for external (or pull) backup API in libvirt. There was a series [1]
> with more limited API scope and functionality for this kind of backup API.
> Besides other issues the series was abandoned as qemu blockdev-del command has
> experimental status at that time. There is also a long pending RFC series for
> internal (or push) backup API [2] which however has not much in comman with
> this RFC. Also there is RFC with overall agreement to having a backup API in
> libvirt [3].
> 
> The aim of external backup API is to provide means for 3d party application to
> read/write domain disks as block devices for the purpuse of backup. Disk is
> read on backup operation and in case of active domain is presented at some
> point in time (preferable in some guest consistent state). Disk is written on
> restore operation.
> 
> As to providing disk state at some point in time one can use existing disks
> snapshots for this purpose. However this RFC introduces API to leverage image
> fleecing (blockdev-backup command) instead. Image fleecing is somewhat inverse
> to snapshots. In case of snapshots writes go to top image thus backing image
> stays constant, in case of fleecing writes go to same image as before but old
> data is previously popped out to fleece image which have original image as
> backing. As a result fleece image became disk snapshot.
> 
> Another task of this API is to provide disks for read/write operations. One
> could try to leverage libvirt stream API for this purpose but AFAIK clients
> want random access to disks data which is not what stream API suitable for.
> I'm not sure what is costs of adding block API to libvirt, particularly what it
> costs to make it effective implementation at RPC level thus this RFC add means
> to export disks data thru existing block interfaces. For qemu it is NBD.
> 
> 
> 
>   1. Fleece API
> 
> So the below API is to provide means to start/stop/query disk image fleecing.
> I use BlockSnaphost name for this operation. Other options are Fleecing, BlockFleecing,
> TempBlockSnapshot etc.
> 
> /* Start fleecing */
> virDomainBlockSnapshotPtr
> virDomainBlockSnapshotCreateXML(virDomainPtr domain,
>                                 const char *xmlDesc,
>                                 unsigned int flags);
> 
> /* Stop fleecing */
> int
> virDomainBlockSnapshotDelete(virDomainBlockSnapshotPtr snapshot,
>                              unsigned int flags);
> 
> /* List active fleecings */
> virDomainBlockSnapshotList(virDomainPtr domain,
>                            virDomainBlockSnapshotPtr **snaps,
>                            unsigned int flags);
> 
> /* Get fleecing description */
> char*
> virDomainBlockSnapshotGetXMLDesc(virDomainBlockSnapshotPtr snapshot,
>                                  unsigned int flags);
> 
> /* Get fleecing by name */
> virDomainBlockSnapshotPtr
> virDomainBlockSnapshotLookupByName(virDomainPtr domain,
>                                    const char *name);
> 
> 
> Here is a minimal block snapshot xml description to feed creating function:
> 
> <domainblocksnapshot>
>   <snapshot disk='sda'>
>     <fleece file="/path/to/fleece-image-sda"/>
>   </snapshot>
>   <snapshot disk='sdb'>
>     <fleece file="/path/to/fleece-image-sdb"/>
>   </snapshot>
> </domainblocksnapshot>
> 
> Below is an example of what getting description function should provide upon
> successful block snaphost creation. The difference with the above xml is that
> name element (it can be specified on creation as well) and aliases are
> generated. Aliases will be useful later to identify block devices on exporting
> thru nbd.
> 
> <domainblocksnapshot>
>   <name>5768a388-c1c4-414c-ac4e-eab216ba7c0c</name>
>   <snapshot disk='sda'>
>     <fleece file="/path/to/fleece-image-sda"/>
>     <alias name="scsi0-0-0-0-backup"/>
>   </snapshot>
>   <snapshot disk='sdb'>
>     <fleece file="/path/to/fleece-image-sdb"/>
>     <alias name="scsi0-0-0-1-backup"/>
>   </snapshot>
> </domainblocksnapshot>
> 
> 
> 
>   2. Export API
> 
> During backup operation we need to provide read access to fleecing image. This
> is done thru qemu process nbd server. We just need to specify the disks to
> export.
> 
> /* start block export */
> int
> virDomainBlockExportStart(virDomainPtr domain,
>                           const char *xmlDesc,
>                           unsigned int flags);
> 
> /* stop block export */
> int
> virDomainBlockExportStop(virDomainPtr domain,
>                          const char *diskName,
>                          unsigned int flags);
> 
> Here is an example of xml for starting function:
> 
> <blockexport type="nbd" port="8001">
>   <listen type="address" address="10.0.2.10"/>
>   <disk name="scsi0-0-0-1-backup"/>
> </blockexport>
> 
> qemu nbd server is started upon first disk export start and shutted down upon
> last disk export stop. Another option is to control ndb server explicitly. One
> way to do it is to consider ndb server a new device so to start/stop/update ndb
> server we can use attach/detach/update device functions. Then in block export
> start we need to refer to this device somehow. This can be a generated
> name/uuid or type/address pair. Actually this approach to expose ndb server
> looks more natural to me even it includes more management from client side.
> I am not suggesting it in the first place mostly due to hesitations on how to
> refer to ndb server on block export.
> 
> In any case I'd like to provide export info in active domain config:
> 
> <devices>
>   <blockexport type="nbd" port="8001">
>     <listen type="address" address="10.0.2.10"/>
>     <disk name="scsi0-0-0-1-backup"/>
>     <disk name="scsi0-0-0-2-backup"/>
>   </blockexport>
> </devices>
> 
> This API is used in restore operation too. Domain is started in paused state,
> the disks to be restored are exported and backup client fills it with the
> backup data.
> 
> 
> 
>   3. Incremental backups
> 
> Qemu can track what disk parts are changed from from fleecing start. This is
> what typically called CBT (dirty bitmap in qemu community I guess). There are
> also experimental ndb support [4] and a bunch of merged/agreed/proposed bitmap
> operation that help to organize incremental backups.
> 
> Different hypervisors has different bitmap implementations with different
> costs thus it is up to hyperivsor whether to start CBT or not upon block snapshot
> create by default. Qemu implementations has memory and disk costs for every
> bitmap thus I suggest by default start fleecing without bitmap and add flag
> VIR_DOMAIN_BLOCK_SNAPSHOT_CREATE_CHECKPOINT to ask to start a bitmap.
> 
> Disks bitmaps are visible in active domain definition with the name
> of block snapshot for which bitmap was started.
> 
> <disk type='file' device='disk'>
>   ..
>   <target dev='sda' bus='scsi'/>
>   <alias name='scsi0-0-0-0'/>
>   <checkpoint name="93a5c045-6457-2c09-e56c-927cdf34e178">
>   <checkpoint name="5768a388-c1c4-414c-ac4e-eab216ba7c0c">
>   ..
> </disk>
> 
> The bitmap can be specified upon disk export like below (I guess there
> is no need to provide more then one bitmap per disk). Active domain
> config section for block export is expanded similarly.
> 
> <blockexport type="nbd" port="8001">
>   <listen type="address" address="10.0.2.10"/>
>   <disk name="scsi0-0-0-1-backup" checkpoint="5768a388-c1c4-414c-ac4e-eab216ba7c0c"/>
> </blockexport>
> 
> If bitmap was created on backup start but client failed to make a backup for some reason
> then it makes no sense to keep this checkpoint anymore. As having bitmap takes
> resources it is convinient to drop bitmap in this case. Also one may
> want to drop bitmap for pure resource managment issues. So we need API to remove bitmap:
> 
> virDomainBlockCheckpointRemove(virDomainPtr domain,
>                                const char *name,
>                                unsigned int flags);
> 
> 
> 
>   4. Other hypervisors
> 
> I took a somewhat considerable look only at vmware backup interface at [5] etc.
> Looks like they don't have fleecing like qemu has so for vmware snapshots one
> can use usual disks snapshots API. Also there is no nbd interface for snapshots
> expectedly thus to deal with vmware snapshot disks one eventually will have to
> add block API to libvirt. So the only point this RFC has to vmware backups is
> exporting checkpoints in disk xml. The vmware documentation does not say much
> about bitmap limitations but I guess they still can provide only a number of
> them which can be exposed as suggested for active domain disks.
> 
> 
> 
>   II Links:
> 
> [1] https://www.redhat.com/archives/libvir-list/2016-September/msg00192.html
> [2] https://www.redhat.com/archives/libvir-list/2017-May/msg00379.html
> [3] https://www.redhat.com/archives/libvir-list/2016-March/msg00937.html
> [4] https://github.com/NetworkBlockDevice/nbd/commit/cfa8ebfc354b2adbdf73b6e6c2520d1b48e43f7a
> [5] https://code.vmware.com/doc/preview?id=4076#/doc/vddkBkupVadp.9.3.html#1014717
> 
> --
> libvir-list mailing list
> libvir-list redhat com
> https://www.redhat.com/mailman/listinfo/libvir-list
> 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]