[libvirt PATCH 00/11] qemu: introduce a per-VM event loop thread

Daniel P. Berrangé berrange at redhat.com
Fri Feb 14 12:51:58 UTC 2020


This series changes the way we manage the QEMU monitor and
QEMU agent, such that all I/O is processed by a dedicated
event loop thread.

Many times in the past years people are reported issues
where long running monitor event callbacks block the main
libvirtd event loop for an unacceptably long period of
time. In the best case, this delays other work being
completed, but in bad cases it leads to mgmt app failures
when keepalive times trigger a client disconnect.

With this series, when we spawn QEMU, we also spawn a
dedicated thread running a GMainLoop instance. Then QEMU
monitor and QEMU agent UNIX sockets are switched to use
GMainContext for events instead of the traditional libvirt
event loop APIs. We kill off the event thread when we see
EOF on the QEMU monitor during shutdown.

The cost of this approach is one extra thread per VM,
which incurs a new OS process and a new stack allocation.

The QEMU driver already delegates some QMP event handling
to a thread pool for certain types of event. This was a
previous hack to mitigate the impact on the main event
loop. It is likely that we can remove this thread pool
from the QEMU driver & rely on the per-VM event threads
to do all the work. This will, however, require careful
analysis of each handler we pushed into the thread pool
to make sure its work doesn't have a dependency on the
event loop running in parallel.

This should also eliminate the need to have the libvirt
event loop registered when using the embedded QEMU driver.
This has not yet been validated, however, so it is left
for a future patch to relax the constraint.

Daniel P. Berrangé (11):
  qemu: drop support for agent connections on PTYs
  qemu: drop ability to open monitor from FD
  src: set the OS level thread name
  src: improve thread naming with human targetted names
  src: introduce an abstraction for running event loops
  qemu: start/stop an event loop thread for domains
  qemu: start/stop an event thread for QMP probing
  tests: start/stop an event thread for QEMU monitor/agent tests
  qemu: convert monitor to use the per-VM event loop
  qemu: fix variable naming in agent code
  qemu: convert agent to use the per-VM event loop

 po/POTFILES.in                      |   1 +
 src/libvirt_private.syms            |   6 +
 src/libxl/libxl_domain.c            |  10 +-
 src/libxl/libxl_migration.c         |  23 +-
 src/lxc/lxc_fuse.c                  |   4 +-
 src/node_device/node_device_udev.c  |   7 +-
 src/nwfilter/nwfilter_dhcpsnoop.c   |  11 +-
 src/nwfilter/nwfilter_learnipaddr.c |  10 +-
 src/qemu/qemu_agent.c               | 634 ++++++++++++++--------------
 src/qemu/qemu_agent.h               |   1 +
 src/qemu/qemu_domain.c              |  33 ++
 src/qemu/qemu_domain.h              |   6 +
 src/qemu/qemu_driver.c              |   3 +-
 src/qemu/qemu_migration.c           |   8 +-
 src/qemu/qemu_monitor.c             | 155 +++----
 src/qemu/qemu_monitor.h             |   8 +-
 src/qemu/qemu_process.c             |  61 ++-
 src/qemu/qemu_process.h             |   2 +
 src/remote/remote_daemon.c          |   9 +-
 src/rpc/virnetserver.c              |   9 +-
 src/storage/storage_backend_scsi.c  |   4 +-
 src/storage/storage_driver.c        |   4 +-
 src/util/Makefile.inc.am            |   2 +
 src/util/vircommand.c               |   5 +-
 src/util/vireventthread.c           | 175 ++++++++
 src/util/vireventthread.h           |  31 ++
 src/util/virfdstream.c              |  10 +-
 src/util/virnodesuspend.c           |   8 +-
 src/util/virthread.c                |  44 +-
 src/util/virthread.h                |   4 +-
 src/util/virthreadpool.c            |  14 +-
 src/util/virthreadpool.h            |   2 +-
 tests/qemumonitortestutils.c        |  15 +
 33 files changed, 832 insertions(+), 487 deletions(-)
 create mode 100644 src/util/vireventthread.c
 create mode 100644 src/util/vireventthread.h

-- 
2.24.1




More information about the libvir-list mailing list