[libvirt] [RFC] Memory controller exploitation in libvirt

[Nikunj A. Dadhania]

On Tue, 24 Aug 2010 11:07:29 +0100, "Daniel P. Berrange" <berrange at redhat.com> wrote:
> On Tue, Aug 24, 2010 at 03:17:44PM +0530, Nikunj A. Dadhania wrote:
> > 
> > On Tue, 24 Aug 2010 11:02:49 +0200, Matthias Bolte <matthias.bolte at googlemail.com> wrote:
> > 
> > <snip>
> > 
> > > Yes the ESX driver allows to control ballooning through
> > > virDomainSetMemory and virDomainSetMaxMemory.
> > > 
> > > ESX itself also allows to set what's called memoryMinGaurantee in the
> > > thread, but this is not exposed in libvirt.
> > LXC driver is using virDomainSetMemory to set the memory hard limit while
> > QEmu/ESX use them to change the ballooning. And as you said, ESX does support
> > memoryMinGaurantee, we can get this exported in libvirt using this new API.
> > 
> > Here I am trying to group all the memory related parameters into one single
> > public API as we have in virDomainSetSchedulerParameters. Currently, the names
> > are not conveying what they modify in the below layer and are confusing.
> 
> For historical design record, I think it would be good to write a short
> description of what memory tunables are available for each hypervisor,
> covering VMWare, OpenVZ, Xen, KVM and LXC (the latter both cgroups based).
> I do recall that OpenVZ  in particular had a huge number of memory
> tunables.
> 
This is an attempt at covering the memory tunables supported by various
hypervisors in libvirt. Let me know if I have missed any memory
tunable. Moreover, inputs from the maintainers/key contributes of each HVs on
these parameters is appreciable. This would help in getting a complete
coverage of the memory tunables that libvirt can support.

1) OpenVZ
=========
vmguarpages: Memory allocation guarantee, in pages.
kmemsize: Size of unswappable kernel memory(in bytes), allocated for processes in this
	  container.
oomguarpages: The guaranteed amount of memory for the case the memory is
	      “over-booked” (out-of-memory kill guarantee), in pages. 
privvmpages: Memory allocation limit, in pages.


OpenVZ driver does not implement any of these functions:
domainSetMemory
domainSetMaxMemory
domainGetMaxMemory

Although, the driver has an internal implementation for setting memory:
openvzDomainSetMemoryInternal that is read from the domain xml file. 

2) VMWare
=========
ConfiuredSize:  Virtual memory the guest can have.

Shares: Priority of the VM, in case there is not enough memory or in case when
       	there is more memory. It has symbolic values like Low, Normal, High
       	and Custom

Reservation: Gauranteed lower bound on the amount of the physical memory that
	     the host reserves for the VM even in case of the overcommit. The
	     VM is allowed to allocate till this level and after it has hit
	     the reservation, those pages are not reclaimed. In case, if guest
	     is not using till the reservation, the host can use that portion
	     of memory.
 
Limit: This is the upper bound for the amount of physical memory that the host
       can allocate for the VM.

Memory Balloon

ESX driver uses following:
* domainSetMaxMemory to set the max virtual memory for the VM.
* domainSetMemory to inflate/deflate the balloon.
* ESX provides lower bound(Reservation), but is not being exploited currently.

3) Xen
======
maxmem_set: Maximum amount of memory reservation of the domain
mem_target_set: Set current memory usage of the domain


4) KVM & LXC
============
memory.limit_in_bytes: Memory hard limit
memory.soft_limit_in_bytes: Memory limit held during contention
memory.memsw_limit_in_bytes: Memory+Swap hard limit
memory.swapiness: Controls the tendency of moving the VM processes to the
		  swap. Value range is 0-100, where 0 means, avoid swapping as
		  long as possible and 100 means aggressively swap processes.

Statistics:
memory.usage_in_bytes: Current memory usage
memory.memsw_usage_in_bytes: Current memory+swap usage
memory.max_usage_in_bytes: Maximum memory usage recorded
memory.memsw_max_usage_in_bytes: Maximum memory+swap usage

Nikunj