[libvirt] Globally Reserve Resources for Host

[Dusty Mabe]

On Wed, Nov 28, 2012 at 3:26 PM, Daniel P. Berrange <berrange at redhat.com> wrote:
> On Wed, Nov 28, 2012 at 02:27:53PM -0500, Dusty Mabe wrote:
>> On Wed, Nov 14, 2012 at 11:22 AM, Dusty Mabe <dustymabe at gmail.com> wrote:
>> > On Thu, Nov 1, 2012 at 11:32 PM, Dusty Mabe <dustymabe at gmail.com> wrote:


> Sorry for not replying before. I've been thinking about this today
> and am specifically wondering about the possible implications for
> a change we need to make to cgroups setup in libvirt

No worries, thank you for responding!!

>
> The core issue is that it has become apparent that nesting cgroups
> can cause some very significant performance / scalability problems
> for the kernel. They have requested / recommended that libvirt make
> its cgroup hierarchy as flat as possible to avoid this problem.

That makes sense. Does this affect how you feel about
https://www.redhat.com/archives/libvir-list/2011-March/msg01546.html ?

>
>  1. Match current hardcoded layout:
>
>     cgroup_layout="/::PROCESS::/libvirt/::DRIVER::/::VMNAME::"
>
>  2. Remove first level when used with systemd:
>
>     cgroup_layout="/::PROCESS::/::DRIVER::/::VMNAME::"
>
>  3. Combine 3rd/4th levels too
>
>     cgroup_layout="/::PROCESS::/::DRIVER::-::VMNAME::"
>
>  4. Ignore current libvirtd placement completely and create in the root:
>
>     cgroup_layout="/libvirt/::DRIVER::-::VMNAME::"
>
>  5. Use UUID instead of VM name
>
>     cgroup_layout="/libvirt/::DRIVER::-::VMUUID::"


A couple more options:

6.   cgroup_layout="/libvirt/::DRIVER::/VMNAME"
7.   cgroup_layout="/libvirt/::DRIVER::/VMUUID"


> I'm sure you've noticed that this plan doesn't leave much scope for the
> approach you outlined above, since the 'qemu' level will disappear by
> default.

It looks like you are right, by default there would be no driver level cgroup.

What if we still had settings in qemu.conf for something like
"reservedHostMem" and "reservedHostCpus" (maybe those names don't make
sense, but I think you know what i mean) that would only be enforced
if the cgroup_layout had a */::DRIVER::/* component (cases 1,2,6,7).
In this case there would be a  'qemu' level. We could explain this
dependency in the conf file and print out an error/warning in the logs
if it wasn't followed.

>
> It is on my plate to get these changes done for Fedora 19 / RHEL-7
> releases.
>
> Daniel


Thanks again for responding! I appreciate your input and it is helpful
to know the path that libvirt is going down. I know what I am
suggesting above may be a long shot but i figure it's worth at least
airing the idea. I like it because too often my hosts get
oversubscribed from other people creating guests and can cause wacky
behavior (my problem, not yours).


Dusty Mabe