[libvirt] [PATCHv5 0/8] bulk stats: QEMU implementation

[Li Wei]

Richard, 

Since this series has been applied, could you please update virt-top to use
these bulk APIs, this should fix the long update interval bug when domain
has a lot of disks.

Thanks.

On 09/15/2014 04:48 PM, Francesco Romani wrote:
> This patchset enhances the QEMU support
> for the new bulk stats API to include
> equivalents of these APIs:
> 
> virDomainBlockInfo
> virDomainGetInfo - for balloon stats
> virDomainGetCPUStats
> virDomainBlockStatsFlags
> virDomainInterfaceStats
> virDomainGetVcpusFlags
> virDomainGetVcpus
> 
> This subset of API is the one oVirt relies on.
> Scale/stress test on an oVirt test environment is in progress.
> 
> The patchset is organized as follows:
> - the first patch enhances the internal stats gathering API
>   to accomodate the needs of the groups which extract information
>   using QEMU monitor jobs.
> - the next five patches implement the bulk stats groups, extracting
>   helpers where do refactoring to extract internal helpers every time
>   it is feasible and convenient.
> - the seventh patch enhances the virsh domstats command with options
>   to use the new bulk stats.
> - the last patch enhances the block stats group adding the wr_highest_offset
>   information, needed by oVirt for thin provisioned disks.
> 
> ChangeLog
> 
> v5: address reviewer's comment
> - Eric pointed out a possible flaw in balloon stats if QEMU monitor needs
>   to be queried. A proper fix require further discussion and API changes
>   (possbily just a new flag); However, since the balloon event is available
>   in QEMU >= 1.2, I just dropped the query and relied on the event instead.
>   Support for older QEMUs will be reintroduced, if needed, with following
>   patches.
> - fix: per-domain monitor check and reporting. (pointed out by Peter)
> - reset last error when fail silently. (pointed out by Peter)
> 
> v4: address reviewer's comment
> - addressed reviewers comments (Peter, Wang Rui).
> - pushed domain check into group stats functions. This follows
>   the strategy to gather and report as much data as possible,
>   silently skipping errors along the way.
> - moved the block allocation patch to the end of the series.
> 
> v3: more polishing and fixes after first review
> - addressed Eric's comments.
> - squashed patches which extracts helpers with patches which
>   use them.
> - changed gathering strategy: now code tries to reap as much
>   information as possible instead to give up and bail out with
>   error. Only critical errors cause the bulk stats to fail.
> - moved away from the transfer semantics. I find it error-prone
>   and not flexible enough, I'd like to avoid as much as possible.
> - rearranged helpers to have one single QEMU query job with
>   many monitor jobs nested inside.
> - fixed docs.
> - implemented missing virsh domstats bits.
> 
> in v2: polishing and optimizations.
> - incorporated feedback from Li Wei (thanks).
> - added documentation.
> - optimized block group to gather all the information with just
>   one call to QEMU monitor.
> - stripped to bare bones merged the 'block info' group into the
>   'block' group - oVirt actually needs just one stat from there.
> - reorganized the keys to be more consistent and shorter.
> 
> 
> Francesco Romani (8):
>   qemu: bulk stats: extend internal collection API
>   qemu: bulk stats: implement CPU stats group
>   qemu: bulk stats: implement balloon group
>   qemu: bulk stats: implement VCPU group
>   qemu: bulk stats: implement interface group
>   qemu: bulk stats: implement block group
>   virsh: add options to query bulk stats group
>   qemu: bulk stats: add block allocation information
> 
>  include/libvirt/libvirt.h.in |   5 +
>  src/libvirt.c                |  61 +++++
>  src/qemu/qemu_driver.c       | 530 +++++++++++++++++++++++++++++++++++++------
>  src/qemu/qemu_monitor.c      |  26 +++
>  src/qemu/qemu_monitor.h      |  21 ++
>  src/qemu/qemu_monitor_json.c | 227 +++++++++++++-----
>  src/qemu/qemu_monitor_json.h |   4 +
>  tools/virsh-domain-monitor.c |  35 +++
>  tools/virsh.pod              |   4 +-
>  9 files changed, 777 insertions(+), 136 deletions(-)
>