[libvirt] [PATCH v2 0/4] Add cpu hotplug support to libvirt.

[Tang Chen]

It seems that libvirt is not cpu hotplug aware.
Please refer to the following problem.

1. At first, we have 2 cpus.
# cat /cgroup/cpuset/cpuset.cpus 
0-1
# cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 
0-1

2. And we have a vm1 with following configuration.
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <emulatorpin cpuset='1'/>
  </cputune>

3. Offline cpu1.
# echo 0 > /sys/devices/system/cpu/cpu1/online
# cat /sys/devices/system/cpu/cpu1/online 
0
# cat /cgroup/cpuset/cpuset.cpus 
0
# cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 
0
# cat /cgroup/cpuset/libvirt/lxc/cpuset.cpus 
0

4. Online cpu1.
# echo 1 > /sys/devices/system/cpu/cpu1/online
# cat /sys/devices/system/cpu/cpu1/online 
1
# cat /cgroup/cpuset/cpuset.cpus 
0-1
# cat /cgroup/cpuset/libvirt/cpuset.cpus 
0
# cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 
0
# cat /cgroup/cpuset/libvirt/lxc/cpuset.cpus 
0

Here，cgroup updated cpuset.cpus，but not for libvirt directory，and also qemu and lxc directory.
vm1 cannot be started again.
# virsh start vm1
error: Failed to start domain vm1
error: Unable to set cpuset.cpus: Permission denied

And libvird gave the following errors.
2012-07-17 07:30:22.478+0000: 3118: error : qemuSetupCgroupVcpuPin:498 : Unable to set cpuset.cpus: Permission denied


These patches resolves this problem by listening on the netlink for cpu hotplug event.
When the netlink service gets the cpu hotplug event, it will attract the cpuid in the message,
and add it into cpuset.cpus in:
    /cgroup/cpuset/libvirt
    /cgroup/cpuset/libvirt/qemu
    /cgroup/cpuset/libvirt/lxc


change log of v2:
    [PATCH v2 1/4]:
        * Move all the handlers from hotplug.c to modules who
          want to listen to cpu hotplug events.

    [PATCH v2 2/4]:
        * Set /libvirt/cpuset.cpus to the lastest value when
          libvirtd is being initialized.

    [PATCH v2 3/4]:
        * Set /libvirt/qemu/cpuset.cpus to the lastest value when
          qemu driver is being initialized.

    [PATCH v2 4/4]:
        * Set /libvirt/lxc/cpuset.cpus to the lastest value when
          lxc driver is being initialized.


Tang Chen (4):
  Add helpers to support cpu hotplug in libvirt.
  Register cpu hotplug netlink handler for libvirtd.
  Register cpu hotplug netlink handler for qemu driver.
  Register cpu hotplug netlink handler for lxc driver.

 daemon/libvirtd.c           |   62 +++++++++++
 include/libvirt/virterror.h |    2 +
 src/Makefile.am             |    1 +
 src/libvirt_private.syms    |    7 ++
 src/lxc/lxc_driver.c        |   67 +++++++++++-
 src/qemu/qemu_driver.c      |   64 ++++++++++++
 src/util/cgroup.c           |   10 +-
 src/util/cgroup.h           |    7 ++
 src/util/hotplug.c          |  240 +++++++++++++++++++++++++++++++++++++++++++
 src/util/hotplug.h          |   44 ++++++++
 src/util/virterror.c        |    3 +-
 11 files changed, 499 insertions(+), 8 deletions(-)
 create mode 100644 src/util/hotplug.c
 create mode 100644 src/util/hotplug.h

-- 
1.7.10.1