[libvirt] [RFC] Support cpu hotplug in libvirt.

[Viktor Mihajlovski]

On 07/18/2012 03:44 AM, Eric Blake wrote:
> On 07/17/2012 07:33 PM, tangchen wrote:
>> Hi~
>>
>> It seems that libvirt is not cpu hotplug aware.
> 
> Portions of libvirt are aware of host hotplug issues, but you are
> correct that there are still lingering bugs (I just found one today in
> nodeinfo.c).
> 
>> 4. Online cpu1.
>> # echo 1 > /sys/devices/system/cpu/cpu1/online
>> # cat /sys/devices/system/cpu/cpu1/online
>> 1
>> # cat /cgroup/cpuset/cpuset.cpus
>> 0-1
>> # cat /cgroup/cpuset/libvirt/cpuset.cpus
>> 0
> 
> I think this is related to (if not the same as) this known kernel bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=714271
> 
> Basically, when the kernel suspends and then resumes, it is not properly
> restoring descendant cgroup information.  Hot unplug of a host cpu is
> more or less a subset of suspending.

That really depends on the architecture, on s390 with it's multi-level
virtualization, it's common to hot unplug host (the host being a 1st level 
guest) CPUs during lower utilization periods. If these are part of a KVM 
guest's CPU set, then they are gone for good...

> 
>>
>>
>> I am trying to use netlink socket with NETLINK_KOBJECT_UEVENT protocol to
>> listen to cpu hotplug events. But I met a little problem here.
> 
> Interesting approach.  Is it also possible to use inotify?
> 
>>
>> virNetlinkEventServiceStart() only create a global variable server, and
>> create a NETLINK_ROUTE netlink socket for it.
>>
>> So if I want to create another different netlink socket, such as NETLINK_KOBJECT_UEVENT,
>> what should I do ?
> 
> I'm not sure here, having not really coded much with netlink sockets
> myself.  If you have a patch pending, then post it and we can discuss
> whether it can be improved.
> 

I was actually considering a workaround in libvirt for what I believe to be a 
kernel misbehavior, however it is possible to deconfigure the cpuset controller 
in the host (i.e. removing it from /etc/cgconfig.conf). The CPU pinning will 
still work, only that it using the "legacy" taskset mechanism. All other 
cgroup-related functionality continues to work as well.
One issue with monitoring the online state of cpus is that libvirtd might not 
be running all the time (crash, update) and thus can miss hotplug events.

>> Shall we make the server global variable a global array ?
>> (It seems there are a lot of work to do.)
>>
>> Thanks. :)
>>
> 
> 
> 
> --
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
> 


-- 

Mit freundlichen Grüßen/Kind Regards
   Viktor Mihajlovski

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Martin Jetter
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294