[libvirt] [RFC] Support cpu hotplug in libvirt.

[Srivatsa S. Bhat]

On 07/18/2012 05:17 PM, Viktor Mihajlovski wrote:
> On 07/18/2012 12:53 PM, tangchen wrote:
>> Hi Viktor:
>>
>> On 07/18/2012 05:06 PM, Viktor Mihajlovski wrote:
>>> I was actually considering a workaround in libvirt for what I believe to be a
>>> kernel misbehavior, however it is possible to deconfigure the cpuset controller
>>> in the host (i.e. removing it from /etc/cgconfig.conf). The CPU pinning will
>>> still work, only that it using the "legacy" taskset mechanism. All other
>>> cgroup-related functionality continues to work as well.
>>> One issue with monitoring the online state of cpus is that libvirtd might not
>>> be running all the time (crash, update) and thus can miss hotplug events.
>>>
>> I don't think remove cpuset controller from cgroup is a good idea.
>> Maybe other apps will need it.
> 
> Maybe ... but a workaround in libvirt will not solve the other apps' issues ;-)
> I still would maintain this has to be fixed in the kernel.
> 

Not too long ago, there was a long thread in the Linux kernel mailing list (see below)
where this issue was discussed in detail. And the conclusion (in short) was that, for
suspend/resume, its the kernel's responsibility to restore the cpusets to the same
state as to how it was before suspend. (Note that, suspend/resume invokes CPU hotplug
internally, and this is hidden from the user).

However, in the regular case of CPU hotplug, the user himself initiates CPU hotplug.
So it is expected that the user be aware of its consequences and take appropriate
actions. In short, there is nothing to fix in the kernel, for the case of regular
CPU hotplug. It is an expected behaviour.
[Note that, technically the kernel can be "fixed" for this case as well. But this
would break kernel semantics for CPU hotplug. Hence we have chosen not to do it.
The suspend/resume case was inevitable (as long as it depends on CPU hotplug).]

And the suspend/resume case has been fixed, in the 3.6 merge window by this commit:

commit d35be8bab9b0ce44bed4b9453f86ebf64062721e
Author: Srivatsa S. Bhat <srivatsa.bhat at linux.vnet.ibm.com>
Date:   Thu May 24 19:46:26 2012 +0530

    CPU hotplug, cpusets, suspend: Don't modify cpusets during suspend/resume


Links to the earlier discussions:
--------------------------------

[1]. http://thread.gmane.org/gmane.linux.documentation/4805
[2]. http://thread.gmane.org/gmane.linux.kernel/1296339
[3]. http://article.gmane.org/gmane.linux.kernel/1298967
     http://thread.gmane.org/gmane.linux.kernel/1298967/focus=1300380
[4]. http://thread.gmane.org/gmane.linux.kernel/1302893
     (this is the version which went upstream in the 3.6 merge window)

Regards,
Srivatsa S. Bhat