[libvirt-users] Lots of threads and increased IO load

[Gergely Horváth]

Hello guys,

We have a lot of small computers on top of two servers running QEMU/KVM
virtualisation with libvirt. In this case, most of the VMs are not doing
much work, some of them barely touches any "hardware" resources.

I have two problems, they might be connected.

When I do a bigger IO job on one of the VMs (like downloading files to
local disk) all other VMs' IO load increases. Even our LDAP server's IO
load is triggering an alert in our monitoring system - though that
server barely does any IO throughout the day.

The big IO job can NOT be considered big in the terms of the host
computers IO capabilities, as it has two really fast SSD drives in
hardware RAID 0 configuration. It can easily do hundreds of megabytes
per second random write. The file copy I was running barely touched 50
MB/s. So this should not be a problem for the host's disks and I also do
not understand why other computers IO load is increased.

The second problem came when I looked at `iotop`, to see how much IO
read and write is going on. The computer, that was doing the copying
easily had like 20-30 threads writing pieces to the disk. Some of the
threads are obviously "processor emulators" - they been running since
the birth of the VM, but what are these other threads?

TL;DR;

Two questions than, to sum up everything:
* How can I control and maybe decrease the number of threads spawned for
a virtual machine?
* How can I know what is the default IO caching for a VM, and what
caching mechanism/policy should I set to minimise IO latency (i.e. I do
not really care if the data is written to the disk later than the guest
thinks)

Thank you.

P.S: We have been running hypertable servers the whole time, and even
those did not generate such mess with IO...

--
Üdvözlettel / Best regards

Horváth Gergely | gergely.horvath at inepex.com

IneTrack - Nyomkövetés egyszerűen | Inepex Kft.
Ügyfélszolgálat: support at inetrack.hu | +36 30 825 7646 | support.inetrack.hu
Web: www.inetrack.hu | nyomkovetes-blog.hu | facebook.com/inetrack

Inepex - The White Label GPS fleet-tracking platform | www.inepex.com