[libvirt] [PATCH v2 5/9] backup: Document new XML for backups
Peter Krempa
pkrempa at redhat.com
Fri Oct 12 09:38:10 UTC 2018
On Fri, Oct 12, 2018 at 00:10:07 -0500, Eric Blake wrote:
> Prepare for new checkpoint and backup APIs by describing the XML
> that will represent a checkpoint and backup. The checkpoint XML
> is modeled heavily after virDomainSnapshotPtr, since both represent
> a point in time of the guest (however, a snapshot exists with the
> intent to roll back to that point, while a checkpoint exists to
> facilitate later incremental backups). Meanwhile, the backup XML
> has enough information to represent both push model (the hypervisor
> writes the backup file to a location of the user's choice) and the
> pull model (the hypervisor needs local temporary storage, and also
> creates an NBD server that the user can use to read the backup via
> a third-party client).. But while a snapshot exists with the
> intent of rolling back to that state, a checkpoint instead makes it
> possible to create an incremental backup at a later time.
>
> Add testsuite coverage for some minimal uses of both XML.
>
> Ultimately, I'd love for push model backups to target a network
> driver rather than just a local file or block device; but doing
> that got hairy (while <domain> uses <source> as the description
> of a host or network resource, I picked <target> as the description
> of a push model backup target [defaults to qcow2 but can also be
> raw or any other format], and <scratch> as the description
> of a pull model backup scratch space [must be qcow2]). The ideal
> refactoring would be a way to parameterize RNG to accept
> <disk type='FOO'>...</disk> so that the name of the subelement
> can be <source> for domain, or <target> or <scratch> as needed for
> backups. Future patches may improve this area of code.
>
> Signed-off-by: Eric Blake <eblake at redhat.com>
[...]
> diff --git a/docs/formatcheckpoint.html.in b/docs/formatcheckpoint.html.in
> new file mode 100644
> index 0000000000..f32516894f
> --- /dev/null
> +++ b/docs/formatcheckpoint.html.in
> @@ -0,0 +1,285 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<!DOCTYPE html>
> +<html xmlns="http://www.w3.org/1999/xhtml">
> + <body>
> + <h1>Checkpoint and Backup XML format</h1>
> +
> + <ul id="toc"></ul>
> +
> + <h2><a id="CheckpointAttributes">Checkpoint XML</a></h2>
> +
> + <p>
> + One method of capturing domain disk backups is via the use of
> + incremental backups. Right now, incremental backups are only
> + supported for the qemu hypervisor when using qcow2 disks at the
> + active layer; if other disk formats are in use, capturing disk
> + backups requires different libvirt APIs
> + (see <a href="domainstatecapture.html">domain state capture</a>
> + for a comparison between APIs).
I'd drop the last sentence as anything else will not give the users the
ability to do incremental backups.
> + </p>
[...]
> + <p>
> + Checkpoints are maintained in a hierarchy. A domain can have a
> + current checkpoint, which is the most recent checkpoint compared to
> + the current state of the domain (although a domain might have
> + checkpoints without a current checkpoint, if checkpoints have been
> + deleted in the meantime). Creating or reverting to a checkpoint
> + sets that checkpoint as current, and the prior current checkpoint is
> + the parent of the new checkpoint. Branches in the hierarchy can
I think that too much of this paragraph was stolen from snapshots. It's
not entirely clear to me how the 'current checkpoint' is supposed to
work and since this is meant for completely external backups without
the way to revert the backup via libvirt the description of the setting
of the current checkpoint does not make much sense.
I'm not sure even whether we should copy the notion of the 'current'
checkpoint at this stage from snapshots.
> + be formed by reverting to a checkpoint with a child, then creating
> + another checkpoint.
> + </p>
> + <p>
> + The top-level <code>domaincheckpoint</code> element may contain
> + the following elements:
> + </p>
> + <dl>
> + <dt><code>name</code></dt>
> + <dd>The name for this checkpoint. If the name is specified when
> + initially creating the checkpoint, then the checkpoint will have
> + that particular name. If the name is omitted when initially
> + creating the checkpoint, then libvirt will make up a name for
> + the checkpoint, based on the time when it was created.
> + </dd>
> + <dt><code>description</code></dt>
> + <dd>A human-readable description of the checkpoint. If the
> + description is omitted when initially creating the checkpoint,
> + then this field will be empty.
> + </dd>
> + <dt><code>disks</code></dt>
> + <dd>On input, this is an optional listing of specific
> + instructions for disk checkpoints; it is needed when making a
> + checkpoint on only a subset of the disks associated with a
> + domain (in particular, since qemu checkpoints require qcow2
> + disks, this element may be needed on input for excluding guest
> + disks that are not in qcow2 format); if the entire element was
I'd avoid the part about 'qcow2' and just state that it can be used to
control which disks are part of the checkpoint.
> + omitted on input, then all disks participate in the
> + checkpoint, but if individual disks were omitted from the
> + element, they will not be part of the checkpoint. On output,
> + this is fully populated to show the state of each disk in the
> + checkpoint. This element has a list of <code>disk</code>
> + sub-elements, describing anywhere from one to all of the disks
> + associated with the domain.
> + <dl>
> + <dt><code>disk</code></dt>
> + <dd>This sub-element describes the checkpoint properties of
> + a specific disk. The attribute <code>name</code> is
> + mandatory, and must match either the <code><target
> + dev='name'/></code> or an unambiguous <code><source
> + file='name'/></code> of one of
Please don't allow the <source> part at all. <target> is unique and we
don't have to inflict more pain on us.
> + the <a href="formatdomain.html#elementsDisks">disk
> + devices</a> specified for the domain at the time of the
> + checkpoint. The attribute <code>checkpoint</code> is
> + optional on input; possible values are <code>no</code>
> + when the disk does not participate in this checkpoint;
> + or <code>bitmap</code> if the disk will track all changes
> + since the creation of this checkpoint via a bitmap, in
> + which case another attribute <code>bitmap</code> will be
> + the name of the tracking bitmap (defaulting to the
> + checkpoint name). On output, an additional
I'm afraid that this will lead to a mess if we allow naming the resource
differently than the snapshot.
> + attribute <code>size</code> may be present if
> + the <code>VIR_DOMAIN_CHECKPOINT_XML_SIZE</code> flag was
> + used to perform a dynamic query of the estimated size in
> + bytes of the changes made since the checkpoint was created.
> + </dd>
> + </dl>
> + </dd>
> + <dt><code>creationTime</code></dt>
> + <dd>The time this checkpoint was created. The time is specified
> + in seconds since the Epoch, UTC (i.e. Unix time). Readonly.
> + </dd>
> + <dt><code>parent</code></dt>
> + <dd>The parent of this checkpoint. If present, this element
> + contains exactly one child element, name. This specifies the
> + name of the parent checkpoint of this one, and is used to
> + represent trees of checkpoints. Readonly.
Is this possible by itself? Or only with combination with snapshots?
> + </dd>
> + <dt><code>domain</code></dt>
> + <dd>The inactive <a href="formatdomain.html">domain
> + configuration</a> at the time the checkpoint was created.
> + Readonly.
Umm, I don't see a purpose of this.
> + </dd>
> + </dl>
> +
> + <h2><a id="BackupAttributes">Backup XML</a></h2>
> +
> + <p>
> + Creating a backup, whether full or incremental, is done
> + via <code>virDomainBackupBegin()</code>, which takes an XML
> + description of the actions to perform. There are two general
> + modes for backups: a push mode (where the hypervisor writes out
> + the data to the destination file, which may be local or remote),
> + and a pull mode (where the hypervisor creates an NBD server that
The NBD part is a qemuism.
> + a third-party client can then read as needed, and which requires
> + the use of temporary storage, typically local, until the backup
> + is complete).
> + </p>
> + <p>
> + The instructions for beginning a backup job are provided as
> + attributes and elements of the
> + top-level <code>domainbackup</code> element. This element
> + includes an optional attribute <code>mode</code> which can be
> + either "push" or "pull" (default push). Where elements are
> + optional on creation, <code>virDomainBackupGetXMLDesc()</code>
> + can be used to see the actual values selected (for example,
> + learning which port the NBD server is using in the pull model,
> + or what file names libvirt generated when none were supplied).
> + The following child elements are supported:
> + </p>
> + <dl>
> + <dt><code>incremental</code></dt>
> + <dd>Optional. If this element is present, it must name an
> + existing checkpoint of the domain, which will be used to make
> + this backup an incremental one (in the push model, only
> + changes since the checkpoint are written to the destination;
> + in the pull model, the NBD server uses the
> + NBD_OPT_SET_META_CONTEXT extension to advertise to the client
> + which portions of the export contain changes since the
> + checkpoint). If omitted, a full backup is performed.
> + </dd>
> + <dt><code>server</code></dt>
> + <dd>Present only for a pull mode backup. Contains the same
> + attributes as the <code>protocol</code> element of a disk
> + attached via NBD in the domain (such as transport, socket,
> + name, port, or tls), necessary to set up an NBD server that
> + exposes the content of each disk at the time the backup
> + started.
> + </dd>
Is there a possibility that other hypervisors might use a per-disk
server?
> + <dt><code>disks</code></dt>
> + <dd>This is an optional listing of instructions for disks
> + participating in the backup (if omitted, all disks
> + participate, and libvirt attempts to generate filenames by
> + appending the current timestamp as a suffix). When provided on
> + input, disks omitted from the list do not participate in the
> + backup. On output, the list is present but contains only the
> + disks participating in the backup job. This element has a
> + list of <code>disk</code> sub-elements, describing anywhere
> + from one to all of the disks associated with the domain.
> + <dl>
> + <dt><code>disk</code></dt>
> + <dd>This sub-element describes the checkpoint properties of
> + a specific disk. The attribute <code>name</code> is
> + mandatory, and must match either the <code><target
> + dev='name'/></code> or an unambiguous <code><source
> + file='name'/></code> of one of
Again. Don't allow this or we will get bitten.
> + the <a href="formatdomain.html#elementsDisks">disk
> + devices</a> specified for the domain at the time of the
> + checkpoint. The optional attribute <code>type</code> can
> + be <code>file</code>, <code>block</code>,
> + or <code>network</code>, similar to a disk declaration for
> + a domain, controls what additional sub-elements are needed
> + to describe the destination (such as <code>protocol</code>
> + for a network destination). In push mode backups, the
> + primary sub-element is <code>target</code>; in pull mode,
> + the primary sub-element is <code>scratch</code>; but
> + either way, the primary sub-element describes the file
> + name to be used during the backup operation, similar to
> + the <code>source</code> sub-element of a domain disk. In
> + push mode, an optional sub-element <code>driver</code> can
> + also be used to specify a destination format different
> + from qcow2.
qcow2 is a qemuism. The docs should be general.
> + </dd>
> + </dl>
> + </dd>
> + </dl>
> +
> + <h2><a id="example">Examples</a></h2>
[...]
> diff --git a/docs/schemas/domainbackup.rng b/docs/schemas/domainbackup.rng
> new file mode 100644
> index 0000000000..edc68a37cf
> --- /dev/null
> +++ b/docs/schemas/domainbackup.rng
> @@ -0,0 +1,185 @@
> +<?xml version="1.0"?>
> +<!-- A Relax NG schema for the libvirt domain backup properties XML format -->
> +<grammar xmlns="http://relaxng.org/ns/structure/1.0">
> + <start>
> + <ref name='domainbackup'/>
> + </start>
> +
> + <include href='domaincommon.rng'/>
> +
> + <define name='domainbackup'>
> + <element name='domainbackup'>
> + <optional>
> + <attribute name='id'>
> + <ref name="unsignedInt"/>
This was not documented. What's the purpose?
> + </attribute>
> + </optional>
> + <interleave>
> + <optional>
> + <element name='incremental'>
> + <text/>
Shouldn't we use <empty/> ?
> + </element>
> + </optional>
> + <choice>
> + <group>
> + <optional>
> + <attribute name='mode'>
> + <value>push</value>
> + </attribute>
> + </optional>
> + <ref name='backupDisksPush'/>
> + </group>
> + <group>
> + <attribute name='mode'>
> + <value>pull</value>
> + </attribute>
Split this out similarly to name='backupDisksPush'. I see that the disk
portion is removed but the server specification should be as well.
> + <interleave>
> + <element name='server'>
> + <choice>
> + <group>
> + <optional>
> + <attribute name='transport'>
> + <value>tcp</value>
> + </attribute>
> + </optional>
> + <attribute name='name'>
> + <choice>
> + <ref name='dnsName'/>
> + <ref name='ipAddr'/>
> + </choice>
> + </attribute>
> + <optional>
> + <attribute name='port'>
> + <ref name='unsignedInt'/>
> + </attribute>
> + </optional>
> + <!-- add tls? -->
Yes
> + </group>
> + <group>
> + <attribute name='transport'>
> + <value>unix</value>
> + </attribute>
> + <attribute name='socket'>
> + <ref name='absFilePath'/>
> + </attribute>
> + </group>
> + </choice>
> + </element>
> + <ref name='backupDisksPull'/>
> + </interleave>
> + </group>
> + </choice>
> + </interleave>
> + </element>
> + </define>
> +
> + <define name='backupPushDriver'>
> + <optional>
> + <element name='driver'>
> + <attribute name='type'>
> + <ref name='storageFormat'/>
> + </attribute>
> + </element>
> + </optional>
> + </define>
> +
> + <define name='backupDisksPush'>
> + <optional>
> + <element name='disks'>
> + <oneOrMore>
> + <element name='disk'>
> + <attribute name='name'>
> + <choice>
> + <ref name='diskTarget'/>
> + <ref name='absFilePath'/>
The above needs to be deleted.
> + </choice>
> + </attribute>
> + <choice>
> + <!-- FIXME allow push to a network location, by
> + refactoring 'diskSource' to take element name by a
> + per-grammar ref -->
> + <group>
> + <optional>
> + <attribute name='type'>
> + <value>file</value>
> + </attribute>
> + </optional>
> + <interleave>
> + <optional>
> + <element name='target'>
> + <attribute name='file'>
> + <ref name='absFilePath'/>
> + </attribute>
> + </element>
> + </optional>
> + <ref name='backupPushDriver'/>
> + </interleave>
> + </group>
> + <group>
> + <attribute name='type'>
> + <value>disk</value>
> + </attribute>
> + <interleave>
> + <optional>
> + <element name='target'>
> + <attribute name='dev'>
> + <ref name='absFilePath'/>
> + </attribute>
> + </element>
> + </optional>
> + <ref name='backupPushDriver'/>
> + </interleave>
> + </group>
> + </choice>
> + </element>
> + </oneOrMore>
> + </element>
> + </optional>
> + </define>
> +
> + <define name='backupDisksPull'>
> + <optional>
> + <element name='disks'>
> + <oneOrMore>
> + <element name='disk'>
> + <attribute name='name'>
> + <choice>
> + <ref name='diskTarget'/>
> + <ref name='absFilePath'/>
The above should be deleted.
> + </choice>
> + </attribute>
> + <choice>
[...]
> diff --git a/docs/schemas/domaincheckpoint.rng b/docs/schemas/domaincheckpoint.rng
> new file mode 100644
> index 0000000000..d8dfda9f1c
> --- /dev/null
> +++ b/docs/schemas/domaincheckpoint.rng
> @@ -0,0 +1,94 @@
[...]
> +
> + <define name='diskcheckpoint'>
> + <element name='disk'>
> + <attribute name='name'>
> + <choice>
> + <ref name='diskTarget'/>
> + <ref name='absFilePath'/>
Nope.
> + </choice>
> + </attribute>
> + <choice>
> + <attribute name='checkpoint'>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20181012/8f0db9dc/attachment-0001.sig>
More information about the libvir-list
mailing list