[libvirt] Problem with setting up KVM guests to use HugePages

[Kashyap Chamarthy]

On Tue, Jun 09, 2015 at 05:52:04AM +0000, Vivi L wrote:
> Configure hugepages and then start virtual guest via "virsh start". However, 
> virtual guest failed to use hugepages although it's configured
> 
> The initial usage of system memory
> [root at local ~]# free
>             total        used        free      shared  buff/cache   available
> Mem:   263767352  1900628  261177892  9344    688832   261431872
> Swap:       4194300           0     4194300
> 
> After configuring hugepages, 
> [root at local ~]# cat /proc/meminfo | grep uge
> AnonHugePages:      2048 kB
> HugePages_Total:     140
> HugePages_Free:      140
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:    1048576 kB
> 
> [root at local ~]# mount | grep huge
> cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,
> relatime,hugetlb)hugetlbfs on /dev/hugepages type hugetlbfs (rw,realtime)
> 
> On the guest side, its config xml contains
> <domain type='kvm'>
>   <name>test-01</name>
>   <uuid>e1b72349-4a0b-4b91-aedc-fd34e92251e4</uuid>
>   <description>test-01</description>
>   <memory unit='KiB'>134217728</memory>
>   <currentMemory unit='KiB'>134217728</currentMemory>
>   <memoryBacking>
>     <hugepages/>

You might want re-test by explicitly setting the 'page' element and
'size' attribute? From my test, I had something like this:

    $ virsh dumpxml f21-vm | grep hugepages -B3 -A2 
      <memory unit='KiB'>2000896</memory>
      <currentMemory unit='KiB'>2000000</currentMemory>
      <memoryBacking>
        <hugepages>
          <page size='2048' unit='KiB' nodeset='0'/>
        </hugepages>
      </memoryBacking>
      <vcpu placement='static'>8</vcpu>

I haven't tested this exhaustively, but some basic test notes here:

    https://kashyapc.fedorapeople.org/virt/test-hugepages-with-libvirt.txt

>     <nosharepages/>
>   </memoryBacking>
> 
> 
> Tried to virsh start the guest, but it failed although the hugepages memory 
> was sufficient
> virsh # start test-01
> error: Failed to start domain test-01
> error: internal error: early end of file from monitor: possible problem:
> Cannot set up guest memory 'pc.ram': Cannot allocate memory
> 
> 
> however, if decrease the number of hugepages to smaller(say 100) that 
> makes the memory not used by hugepages more than required by guest, 
> then guest can start. But memory is not allocated from hugepages. 
> [root at local ~]# cat /proc/meminfo | grep uge
> AnonHugePages:  134254592 kB
> HugePages_Total:     100
> HugePages_Free:      100
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:    1048576 kB
> 
> In summary, although hugepages is configured, the guest seemed not 
> instructed to use hugepage, instead, the guest is allocated memory from 
> the memory pool leftover by hugepages. 
> 
> 
> 
> 
> --
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list

-- 
/kashyap