[libvirt PATCH 1/2] docs: kbase: Add a doc on live full disk backup

[Michal Prívozník]

On 5/4/21 6:16 PM, Kashyap Chamarthy wrote:
> This is a rewrite of:
> 
>     https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
> 
> Once this commit merges, the above wiki should point to this kbase
> document.
> 
> Signed-off-by: Kashyap Chamarthy <kchamart at redhat.com>
> ---
>  docs/kbase/index.rst                 |   3 +
>  docs/kbase/live_full_disk_backup.rst | 111 +++++++++++++++++++++++++++
>  2 files changed, 114 insertions(+)
>  create mode 100644 docs/kbase/live_full_disk_backup.rst

Alright, first let me say that's it's difficult to chose which out of
libxml2 or rst2html gives less useful message on error. More on that below.

You will need to register this file in a list so that meson builds HTML:

diff --git i/docs/kbase/meson.build w/docs/kbase/meson.build
index 7b4e7abbd3..51d4bc7b90 100644
--- i/docs/kbase/meson.build
+++ w/docs/kbase/meson.build
@@ -6,6 +6,7 @@ docs_kbase_files = [
   'index',
   'kvm-realtime',
   'launch_security_sev',
+  'live_full_disk_backup',
   'locking-lockd',
   'locking',
   'locking-sanlock',


> 
> diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst
> index 532804fe05..5ee3b5fce8 100644
> --- a/docs/kbase/index.rst
> +++ b/docs/kbase/index.rst
> @@ -12,6 +12,9 @@ Usage
>     Explanation of how disk backing chain specification impacts libvirt's
>     behaviour and basic troubleshooting steps of disk problems.
>  
> +`Live full disk backup <live_full_disk_backup.html>`__
> +   A walkthrough of how to take effective live full disk backups.
> +
>  `Virtio-FS <virtiofs.html>`__
>     Share a filesystem between the guest and the host
>  
> diff --git a/docs/kbase/live_full_disk_backup.rst b/docs/kbase/live_full_disk_backup.rst
> new file mode 100644
> index 0000000000..f4ef7f2cc7
> --- /dev/null
> +++ b/docs/kbase/live_full_disk_backup.rst
> @@ -0,0 +1,111 @@
> +Efficient live full disk backup
> +===============================

AAand this is the place that rst2html identified as problematic. While
the true problem is below [1]. Anyway, how about making this h1 type of
header, and [2].
Also, you can generate table of contents here simply by putting:

.. contents::


> +
> +Overview
> +--------

2: this type h2? I mean putting "==.." at the beginning of the file, and
s/-/=/

> +
> +Live full disk backups are preferred in many scenarios, *despite* their
> +space requirements.  The following outlines an efficient method to do
> +that using libvirt's APIs.  This method involves concepts: the notion of
> +`backing chains <https://libvirt.org/kbase/backing_chains.html>`_,
> +`QCOW2 overlays
> +<https://qemu.readthedocs.io/en/latest/interop/live-block-operations.html#disk-image-backing-chain-notation>`_,
> +and a special operation called "active block-commit", which allows
> +live-merging an overlay disk image into its backing file.
> +
> +At a high-level, this is the approach to take a live ful disk backup:
> +Assuming a guest with a single disk image, create a temporary live QCOW2
> +overlay (commonly called as "external snapshot") to track the live guest
> +writes.  Then backup the original disk image while the guest (live QEMU)
> +keeps writing to the temporary overlay.  Finally, perform the "active
> +block-commit" opertion to live-merge the temporary overlay disk contents
> +into the original image — i.e. the backing file — and "pivot" the live
> +QEMU process to point to it.
> +
> +The below assumes you're using *at least* these versions (preferably
> +much newer): QEMU 2.1 and libvirt-1.2.9.
> +
> +Detailed procedure
> +------------------
> +
> +1. Start with a guest with a single disk image, ``base.raw``, which is

How about replacing "1. 2. 3. ..." with "#."? That way paragraphs will
get numbered automatically and we won't need to renumber them when
introducing a new one or shifting them around (unlikely, I know).

> +   where the live QEMU is pointing at, and recording the guest writes::
> +
> +     base.raw (live QEMU)
> +
> +2. List the current block device(s) in use::
> +
> +    $ virsh domblklist vm1
> +    Target     Source
> +    ------------------------------------------------
> +    vda        /var/lib/libvirt/images/base.raw
> +
> +3. Create the live "external disk snapshot" (or more correctly, "an
> +   overlay")::
> +
> +    $ virsh snapshot-create-as --domain vm1 overlay1 \
> +        --diskspec vda,file=/var/lib/libvirt/images/overlay1.qcow2 \
> +        --disk-only
> +
> +   The disk image chain looks as follows::
> +
> +    base.raw <-- overlay1.qcow2 (live QEMU)
> +
> +.. note::

1: this ^^ this is the problem, because it splits this numbered list
into two. I guess after I found out what the problem was I understand
the error message a bit:

../docs/kbase/live_full_disk_backup.rst:3: (INFO/1) Enumerated list
start value not ordinal-1: "4" (ordinal 4)

Anyway, putting two spaces at the beginning of the line fixes the
problem: "   .. note::"
> +    Above, if you have QEMU guest agent installed in your virtual
> +    machine, use the ``--quiesce`` option with ``virsh
> +    snapshot-create-as [...]`` to ensure you have a consistent disk
> +    state.
> +
> +    Optionally, you can also supply the ``--no-metadata`` option to
> +    ``virsh snapshot-create-as`` to tell libvirt not track the snapshot
> +    metadata.  Otherwise, when you decide to merge snapshot overlays,
> +    you have to explicitly clean the libvirt metadata using ``virsh
> +    snapshot-delete vm1 --metadata [name|--current]``.
> +
> +4. Now, take a backup the orignal image, ``base.raw``, to a different
> +   location using ``cp`` or ``rsync``::
> +
> +    $ cp /var/lib/libvirt/images/base.raw
> +        /export/backups/copy1_base.raw
> +
> +    # Or:
> +
> +    $ rsync -avhW --progress /var/lib/libvirt/images/base.raw \
> +        /export/backups/copy1_base.raw
> +
> +5. Enumerate the current block device(s) in use, again.  Notice that the
> +   current disk image in use is the above-created overlay,
> +   ``overlay1.qcow2``::
> +
> +    $ virsh domblklist vm1
> +    Target     Source
> +    ------------------------------------------------
> +    vda        vda,file=/var/lib/libvirt/images/overlay1.qcow2
> +
> +6. Once the backup of the original image completes, now perform the
> +   "active block-commit" to live-merge the contents of
> +   ``overlay1.qcow2`` into ``base.raw`` *and* pivot the live QEMU back
> +   to the original::
> +
> +    $ virsh blockcommit vm1 vda --active --verbose --pivot
> +
> +7. After the above operation completes, again list the current block
> +   device(s) in use.  And notice that the live QEMU is now writing to
> +   the original base image::
> +
> +    $ virsh domblklist vm1
> +    Target     Source
> +    ------------------------------------------------
> +    vda        /var/lib/libvirt/images/base.raw
> +
> +
> +The final updated disk image "chain" will be a single consolidated
> +disk::
> +
> +    [base.raw] (live QEMU)
> +
> +
> +Now you can safely **discard the overlay image**, ``overlay1.qcow2`` —
> +it is no longer valid; and its contents are now fully merged into the
> +base image.
> 

Michal