[libvirt] 'dommemstat' is not working for LXC hypervisor.

Ryota Ozaki ozaki.ryota at gmail.com
Fri Apr 30 09:00:50 UTC 2010


On Fri, Apr 30, 2010 at 12:43 PM, Kumar L Srikanth-B22348
<B22348 at freescale.com> wrote:
> Thanks for your time Ozaki.
> But, I am new to libvirt. Can you please let me know how can we enable memory.use_hierarchy for every creation of libvirt/lxc/<domain>/?

Oh I'm sorry for confusing you, the solution is for developers. For users,
dropping ns subsystem of cgroups is a workaround at this time.

So please enable cgroups like this
  mount -t cgroup -o cpu,cpuacct,cpuset,memory,devices cgroup /cgroup
and then try running a domain.

Note that just doing umount and mount will fail because of cgroups limitation.
Cgroups doesn't allow dropping a subsystem containing groups. So before
umounting, delete every groups (subdirectories). Or just rebooting your
machine is an easier way. (reboot is always the last resort ;-)

Regards,
  ozaki-r

>
> Regards,
> Srikanth.
>
>
> -----Original Message-----
> From: Ryota Ozaki [mailto:ozaki.ryota at gmail.com]
> Sent: Thursday, April 29, 2010 9:51 PM
> To: Kumar L Srikanth-B22348
> Cc: Daniel P. Berrange; libvirt list; Ryota OZAKI
> Subject: Re: [libvirt] 'dommemstat' is not working for LXC hypervisor.
>
> On Thu, Apr 29, 2010 at 10:23 PM, Ryota Ozaki <ozaki.ryota at gmail.com> wrote:
>> On Thu, Apr 29, 2010 at 9:54 PM, Kumar L Srikanth-B22348
>> <B22348 at freescale.com> wrote:
>>> Thanks Ozaki.
>>> Please let me know the result, once you checked with 0.8.0.
>>
>> I may understand what happen.
>>
>> Could you check 'tree -d /cgroup' where running your domain and could
>> you find that a directory named your domain has another directory
>> named some numeric number?
>>
>> If so, could you try cgroup without ns subsystem like mount -t cgroup
>> -o cpu,cpuacct,cpuset,memory,devices cgroup /cgroup or something.
>>
>> I guess running lxc with ns subsystem of cgroup has still problems.
>
> OK, I've confirmed my expectation is correct. With 0.8.0, I can get proper used memory value without ns subsystem and cannot with ns enabled.
>
> The reason why this happens is as follows: ns creates a subgroup on every fork and put any child processes into them. So all processes in a container place under sub directories of libvirt/lxc/<domain>/.
> OTOH, cgroup.memory subsystem accounts memory statistics only its group by default, i.e., it isn't aware of sub groups (directories).
> On issuing dominfo, lxc driver just gets memory statistics of libvirt/lxc/<domain>/ that has no process, eventually we always get 0 as used memory.
>
> I think one solution that addresses this problem is to enable memory.use_hierarchy for every creation of libvirt/lxc/<domain>/.
> The option allows to account memory statistics with hierarchal manner and we can get total memory consumptions at the top of groups, i.e., libvirt/lxc/<domain>/.
>
> Thanks,
>  ozaki-r
>
>
>>
>>  ozaki-r
>>
>>
>>>
>>> Regards,
>>> Srikanth.
>>>
>>> -----Original Message-----
>>> From: Ryota Ozaki [mailto:ozaki.ryota at gmail.com]
>>> Sent: Thursday, April 29, 2010 6:14 PM
>>> To: Kumar L Srikanth-B22348
>>> Cc: Daniel P. Berrange; libvirt list; Ryota OZAKI
>>> Subject: Re: [libvirt] 'dommemstat' is not working for LXC hypervisor.
>>>
>>> On Thu, Apr 29, 2010 at 9:32 PM, Kumar L Srikanth-B22348 <B22348 at freescale.com> wrote:
>>>> Now it's showing Used Memory as 0. I don't know whether it's correct or not.
>>>> I wrote a small script in the domain to check the used memory, which will run continuously in the infinite loop.
>>>> dominfo 'domainname' is showing Used Memory as 0.
>>>> Is this right way to check the Used Memory of that domain?
>>>
>>> dominfo should be...but something wrong.
>>>
>>> In my machine, dominfo shows like:
>>>
>>> # virsh -c lxc:/// dominfo 31105
>>> Id:             31105
>>> Name:           cl_lxc_test
>>> UUID:           fd7b21d9-6cbb-17a8-53dd-233f0aed4b4a
>>> OS Type:        exe
>>> State:          running
>>> CPU(s):         1
>>> CPU time:       3.0s
>>> Max memory:     131072 kB
>>> Used memory:    23672 kB
>>> Autostart:      disable
>>>
>>> However, the version of libvirt is not 0.8.0. So I try to check what changed by recent updates.
>>>
>>> Thanks,
>>>  ozaki-r
>>>
>>>
>>>>
>>>> Regards,
>>>> Srikanth.
>>>>
>>>> -----Original Message-----
>>>> From: Ryota Ozaki [mailto:ozaki.ryota at gmail.com]
>>>> Sent: Thursday, April 29, 2010 5:50 PM
>>>> To: Kumar L Srikanth-B22348
>>>> Cc: Daniel P. Berrange; libvirt list; Ryota OZAKI
>>>> Subject: Re: [libvirt] 'dommemstat' is not working for LXC hypervisor.
>>>>
>>>> On Thu, Apr 29, 2010 at 9:14 PM, Kumar L Srikanth-B22348 <B22348 at freescale.com> wrote:
>>>>> Thanks ozaki.
>>>>> But can you please elaborate the process of getting memory.stat of a particular domain?
>>>>> I mounted cgroup filesystem prior to start libvirtd service using the below command:
>>>>> mount -t cgroup cgroup /cgroup
>>>>>
>>>>> Then started libvirtd service.
>>>>> Then started the domain.
>>>>>
>>>>> Here, can you let me know the process of viewing memory stats of that particular domain?
>>>>
>>>> Could you check output of dominfo again. Cannot you get proper 'Used memory'?
>>>>
>>>>> By the way, in the /cgroup directory lot of files are created. In those, which one can be memory stats?
>>>>
>>>> /cgroup/memory.stat is.
>>>>
>>>>  ozaki-r
>>>>
>>>>>
>>>>> Regards,
>>>>> Srikanth.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Ryota Ozaki [mailto:ozaki.ryota at gmail.com]
>>>>> Sent: Thursday, April 29, 2010 5:27 PM
>>>>> To: Kumar L Srikanth-B22348
>>>>> Cc: Daniel P. Berrange; libvirt list; Ryota OZAKI
>>>>> Subject: Re: [libvirt] 'dommemstat' is not working for LXC hypervisor.
>>>>>
>>>>> On Wed, Apr 28, 2010 at 8:19 PM, Kumar L Srikanth-B22348 <B22348 at freescale.com> wrote:
>>>>>> Daniel,
>>>>>> My requirement is to get how much memory is free in the guest domain.
>>>>>> Can you please let me know, how can I do that?
>>>>>
>>>>> Do you enable cgroup memory subsystem? If so, something wrong in lxc driver of 0.8.0.
>>>>>
>>>>> BTW, 'Used memory' includes both RSS and caches, so if you need total-RSS value, you should use dommemstat.
>>>>>
>>>>>> And also, as you said it's quite easy to implement dommemstats for
>>>>>> lxc driver, is there any possibility to implement that?
>>>>>
>>>>> We can get statistics of dommemstat easily through memory.stat of cgroup.memory subsystem except statistics of page faults.
>>>>>
>>>>>  ozaki-r
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Srikanth.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Daniel P. Berrange [mailto:berrange at redhat.com]
>>>>>> Sent: Wednesday, April 28, 2010 4:46 PM
>>>>>> To: Kumar L Srikanth-B22348
>>>>>> Cc: libvir-list at redhat.com
>>>>>> Subject: Re: 'dommemstat' is not working for LXC hypervisor.
>>>>>>
>>>>>> On Wed, Apr 28, 2010 at 04:43:41PM +0530, Kumar L Srikanth-B22348 wrote:
>>>>>>> Thanks for the reply Daniel.
>>>>>>> Virsh dominfo dommand showing following results:
>>>>>>>
>>>>>>> Id:             32473
>>>>>>> Name:           vm1
>>>>>>> UUID:           f13bc513-15f2-1dd1-74f6-18c47931ece9
>>>>>>> OS Type:        exe
>>>>>>> State:          running
>>>>>>> CPU(s):         1
>>>>>>> Max memory:     5000000 kB
>>>>>>> Used memory:    5000000 kB
>>>>>>> Autostart:      disable
>>>>>>>
>>>>>>>
>>>>>>> Though whatever services running on the domain, every time used
>>>>>>> memory
>>>>>>
>>>>>>> is showing as 5000000 kB. Is that result is correct?
>>>>>>
>>>>>> The 'Max memory' 5000000 KB is the hard limit that is allocated to
>>>>>> the container. I thought that the 'used memory' field would be
>>>>>> lower, but its possible we don't have the actual stats for used
>>>>>> memory so just set both the same. I'd have to research the code again to answer for sure...
>>>>>>
>>>>>> Daniel
>>>>>> --
>>>>>> |: Red Hat, Engineering, London    -o-
>>>>>> http://people.redhat.com/berrange/ :|
>>>>>> |: http://libvirt.org -o- http://virt-manager.org -o-
>>>>>> http://deltacloud.org :|
>>>>>> |: http://autobuild.org        -o-
>>>>>> http://search.cpan.org/~danberr/ :|
>>>>>> |: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742
>>>>>> |7D3B
>>>>>> 9505 :|
>>>>>>
>>>>>>
>>>>>> --
>>>>>> libvir-list mailing list
>>>>>> libvir-list at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/libvir-list
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>
>




More information about the libvir-list mailing list