[libvirt] Ongoing work on lock contention in qemu driver?

[Peter Feiner]

Hello Daniel,

I've been working on improving scalability in OpenStack on libvirt+kvm
for the last couple of months. I'm particularly interested in reducing
the time it takes to create VMs when many VMs are requested in
parallel.

One apparent bottleneck during virtual machine creation is libvirt. As
more VMs are created in parallel, some libvirt calls (i.e.,
virConnectGetLibVersion and virDomainCreateWithFlags) take longer
without a commensurate increase in hardware utilization.

Thanks to your patches in libvirt-1.0.3, the situation has improved.
Some libvirt calls OpenStack makes during VM creation (i.e.,
virConnectDefineXML) have no measurable slowdown when many VMs are
created in parallel. In turn, parallel VM creation in OpenStack is
significantly faster with libvirt-1.0.3. On my standard benchmark
(create 20 VMs in parallel, wait until the VM is ACTIVE, which is
essentially after virDomainCreateWithFlags returns), libvirt-1.0.3
reduces the median creation time from 90s to 60s when compared to
libvirt-0.9.8.

I'd like to know if your concurrency work in the qemu driver is
ongoing. If it isn't, I'd like to pick the work up myself and work on
further improvements. Any advice or insight would be appreciated.

Thanks!
Peter Feiner