[libvirt] [PATCHv3 00/10] new API virDomainBlockSetWriteThreshold

[Eric Blake]

v2 was here:
https://www.redhat.com/archives/libvir-list/2015-June/msg00591.html

This series depends on my yajl/json cleanups:
https://www.redhat.com/archives/libvir-list/2015-June/msg01098.html

and can also be found here:
git clone git://repo.or.cz/libvirt/ericb.git threshold
http://repo.or.cz/w/libvirt/ericb.git/shortlog/refs/heads/threshold

Since then, I've rebased to master, addressed Peter's findings on v2,
and tested that things actually work when combined with a single
pending patch on qemu (I'll reply to this mail with the list id of
that patch, where that patch in turn points to this series as
justification).

Here's a transcript of how I tested:

I modified my 'f20' domain to include:
  <devices>
    <emulator>/home/eblake/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>
      <disk type='block' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source dev='/dev/sda6'/>
        <target dev='vda' bus='virtio'/>
      </disk>

then created a raw partition wrapper around the usual disk image:
# qemu-img create -f qcow2 -o backing_file=/var/lib/libvirt/images/f20.img,backing_fmt=qcow2 /dev/sda6
Formatting '/dev/sda6', fmt=qcow2 size=12884901888 backing_file='/var/lib/libvirt/images/f20.img' backing_fmt='qcow2' encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16

I then started the domain paused (as long as no guest I/O happens,
the threshold cannot be exceeded, which gives me time to register
a threshold and set up an event handler):
# virsh start --paused f20
Domain f20 started
# ./run tools/virsh -k0 domblkthreshold f20 vda 10000
threshold of vda set to 10000 bytes

I was then able to play with domstats to see different thresholds
as I registered them (such as playing with the parts-per-million
or even with virsh 'percentage sugar'):
# virsh -k0 domstats f20 --block | grep -C1 allo
  block.0.write-threshold=10000
  block.0.allocation=0
  block.0.capacity=12884901888

In another window, I set up my event loop:
$ ./run tools/virsh -k0 event f20 --loop --all

then in the first window, I resumed things:
# virsh resume f20

and got this message in the second window, after several seconds
had elapsed (the first few seconds in the guest is under control
of Grub, which doesn't write to disk; when it transitioned over
to the Linux boot, disk activity started happening and easily
causes the qcow2 wrapper to exceed the threshold I set earlier):

$ ./run tools/virsh -k0 -c qemu:///system event --loop --all
event 'lifecycle' for domain f20: Resumed Unpaused
event 'lifecycle' for domain f20: Resumed Unpaused
event 'write-threshold' for domain f20 disk vda (/dev/sda6): threshold 10000 exceeded by 186608 bytes

Sure enough, the registration had cleared, and the qcow2 wrapper
had grown (it grew more than once in the process of booting, so
by the time I re-queried, allocation had grown even further).

# virsh -k0 domstats f20 --block | grep -C1 allo
  block.0.write-threshold=0
  block.0.allocation=13434368
  block.0.capacity=12884901888

I'd really like to see this series in 1.3.0 to commit to the API;
there are still some further improvements we can make (in particular
'vda[1]' support, and also improving qemu event handling to re-grab
a job lock if the cache of node names is not populated after a
libvirtd restart), but as those improvements won't affect ABI, they
can be deferred if they don't make it into 1.3.0.

[I'm headed on vacation, so I will be really slow to respond in
the next 3 weeks; someone else may need to push this if it is
approved for 1.3.0]

Eric Blake (10):
  threshold: new API virDomainBlockSetWriteThreshold
  threshold: expose new API in virsh
  threshold: new event object for tracking write threshold
  threshold: wire up threshold event in RPC
  threshold: add qemu capability bits
  threshold: learn a node name for a given qcow2 disk
  threshold: track an allocation node name for a storage source
  threshold: scrape threshold data from QMP
  threshold: add threshold event handling in qemu
  threshold: add write threshold setting in qemu

 daemon/remote.c                                 |  51 +++
 include/libvirt/libvirt-domain.h                |  53 +++
 src/conf/domain_event.c                         | 101 ++++-
 src/conf/domain_event.h                         |  16 +-
 src/driver-hypervisor.h                         |   7 +
 src/libvirt-domain.c                            |  98 +++++
 src/libvirt_private.syms                        |   2 +
 src/libvirt_public.syms                         |   1 +
 src/qemu/qemu_capabilities.c                    |  12 +-
 src/qemu/qemu_capabilities.h                    |   4 +-
 src/qemu/qemu_domain.c                          | 105 ++++++
 src/qemu/qemu_domain.h                          |  10 +-
 src/qemu/qemu_driver.c                          | 108 +++++-
 src/qemu/qemu_migration.c                       |   2 +-
 src/qemu/qemu_monitor.c                         |  79 +++-
 src/qemu/qemu_monitor.h                         |  28 ++
 src/qemu/qemu_monitor_json.c                    | 305 ++++++++++++++-
 src/qemu/qemu_monitor_json.h                    |  11 +
 src/qemu/qemu_process.c                         |  44 +++
 src/remote/remote_driver.c                      |  35 ++
 src/remote/remote_protocol.x                    |  30 +-
 src/remote_protocol-structs                     |  16 +
 src/util/virstoragefile.c                       |   4 +-
 src/util/virstoragefile.h                       |   3 +-
 tests/qemucapabilitiesdata/caps_2.1.1-1.replies | 470 ++++++++++++------------
 tests/qemumonitortest.c                         |  13 +-
 tools/virsh-domain-monitor.c                    | 108 ++++++
 tools/virsh-domain.c                            |  24 ++
 tools/virsh.pod                                 |  26 ++
 29 files changed, 1508 insertions(+), 258 deletions(-)

-- 
2.4.3