[libvirt] [RFC] memory settings interface for containers

[Nikolay Shirokovskiy]

On 14.01.2016 16:31, Daniel P. Berrange wrote:
> On Thu, Jan 14, 2016 at 04:23:59PM +0300, Nikolay Shirokovskiy wrote:
>>
>>
>> On 14.01.2016 16:01, Daniel P. Berrange wrote:
>>> On Thu, Nov 12, 2015 at 11:11:31AM +0300, Nikolay Shirokovskiy wrote:
>>>> Hi, everyone.
>>>>
>>>> I plan to add means to configure vz containers memory setting and have trouble
>>>> getting it done thru libvirt interface. Looks like current interface fits good
>>>> for vm memory managment but its not clear how to use it with containers. First
>>>> let's take aside memory hotplugging which is obviously not suitable for
>>>> containers. Then memory interface is represented by 2 parameters: total_memory
>>>> and cur_balloon. For VMs total_memory can't be changed at runtime, cur_ballon
>>>> can't be greater than total_memory. But for containers memory model is
>>>> different. We have only one parameter and it can be changed for running 
>>>> domains. So question is how to map this model to existing interface (it is
>>>> unlikely to have a new interface for this case). I plan to make both parameters
>>>> to have same meaning and be equal for containers and update virsh, API and xml
>>>> model documentation accordingly.
>>>>
>>>> I'd be happy to hear core developers opinions on this topic.
>>>
>>> So from VM POV, the 'total_memory' represents the initial populated
>>> memory map. This is traditionally fixed at boot and cannot be changed
>>> while the VM is running.
>>>
>>> 'cur_balloon' represents the current memory after balloon adjustments
>>> and must be strictly less than or equal to total_memory.
>>>
>>> When the VM is shutoff, both values can be changed, but if you want
>>> to increase 'cur_balloon' you must first increase 'total_memory'.
>>>
>>> With LXC we essentially ignore 'cur_balloon' and just set the cgroups
>>> memory.limit_in_bytes to the 'total_memory' value.
>> [1]
>>>
>>> For reasons that escape me, we forbid changes to 'total_memory' in
>>> LXC driver, despite the fact that we could trivially allow them.
>>> We should fix that.
>>>
>>> In the virDomainInfo struct, things are a little different. For VMs
>>> we report 'current' as being the current balloon level. For LXC
>>> we report 'current' as being the current container usage, as reported
>>> by memory.usage_in_bytes cgroup field.
>>>
>>> I agree we should be more explicit about this all in the docs. For
>>> initial XML config, we should just raise an error if both
>>> <memory> and <currentMemory> are present and have different values,
>>> or possibly just clamp <currentMemory> to match <memory>.
>>>
>>> In virDomainSetMemoryFlags() we should document that with container
>>> based virt, you cannot change the CURRENT_MEMORY setting, and with
>>> machine based virt you cannot change the MAX_MEMORY setting but
>>> containers should allow it.
>> [2]
>>
>> Thanks for answer, Daniel.
>>
>> So actual[1] and proposed[2] behaviour are different if in [2] you
>> mean 'fail' when say 'cannot change'. So wouldn't fixing [1] to [2]
>> be a degradation? It is a case of a really lame libvirt usage but
>> nevertheless. Thes same question raises to XML config. If 
>> adding checks can break somebody?
> 
> Actually looking again, current impl is worse than I thought :-(
> 
> At initial startup, we set limit based on <memory> and ignore
> <currentMemory>. This is good. clamping <currentMemory> to
> <memory> would be fine in this respect.

Oh. Now it clear. Clamping is to ignore given value and take value
from different node.

So in short desicion you described later in relation
to API makes <memory> and <currentMemory> synonyms. But in 
case of XML we can be given both names with different values simultaneously.
So to be backward compatible instead of failing we let <memory>
silently overrule. Ok.

I plan to document it, fix in lxc and implement in vz.

Thanx.

> 
> For the virDomainSetMemory() API, it seems we deny the option
> to change max_memory, but allow setting current memory and
> use to set the cgroups. This is kind of crazy, as it is
> inconsistent with what we do during startup :-(
> 
> So based on that current impl, I think that for virDomainSetMemory
> we'll have to declare that MEMORY & MAX_MEMORY are considered to
> be identical, and make both have the same effect on cgroups. Otherwise
> you're right that we'd risk breaking apps. IOW, whenever we update one
> setting, we should update the other to match
> 
> Regards,
> Daniel
>