[vfio-users] Memory allocation with numatune

[Blair Bethwaite]

Hi Jan,

Looking back at your original pastebin I see you are not explicitly
specifying guest numa details. Perhaps your numatune needs the memnode
element, not sure if that is significant here but you could try:

<numatune>
  <memory mode='strict' nodeset='1'/>
  <memnode cellid='0' mode='strict' nodeset='1'/>
</numatune>

and then:

<cpu mode='host-model'>
  <model fallback='allow'/>
  <topology sockets='1' cores='6' threads='2'/>
  <numa>
    <cell id='0' cpus='0-11' memory='7340032' unit='KiB'/>
  </numa>
</cpu>

Cheers,

On 30 January 2017 at 07:02, Jan Wiele <jan at wiele.org> wrote:
> Am 2017-01-29 16:33, schrieb Alex Williamson:
>>
>> On Sat, 28 Jan 2017 12:58:41 +0100
>> Thomas Lindroth <thomas.lindroth at gmail.com> wrote:
>>
>>> On 01/27/2017 04:52 PM, Alex Williamson wrote:
>>> > vcpupin actually looks a little off though, just like on the host, the
>>> > VM is going to enumerate cores then threads
>>>
>>> Are you sure about that? Perhaps there is some numa trick I don't
>>> understand but it looks like the guest is a regular 6 cores
>>> with HT (<topology sockets='1' cores='6' threads='2'/>) and
>>> for me setups like that results in a layouts like this in the
>>> guest (qemu-2.8.0):
>>>
>>> cat /proc/cpuinfo |grep "core id"
>>> core id         : 0
>>> core id         : 0
>>> core id         : 1
>>> core id         : 1
>>> core id         : 2
>>> core id         : 2
>>>
>>> The ordering in the guest is threads then cores but for the
>>> host it's the other way around
>>>
>>> cat /proc/cpuinfo |grep "core id"
>>> core id         : 0
>>> core id         : 1
>>> core id         : 2
>>> core id         : 3
>>> core id         : 0
>>> core id         : 1
>>> core id         : 2
>>> core id         : 3
>>>
>>> Cores then threads. So to get a 1:1 match I have to use
>>> pinning like this which is similar to that numa setup.
>>>
>>> <vcpupin vcpu='0' cpuset='1'/>
>>> <vcpupin vcpu='1' cpuset='5'/>
>>> <vcpupin vcpu='2' cpuset='2'/>
>>> <vcpupin vcpu='3' cpuset='6'/>
>>> <vcpupin vcpu='4' cpuset='3'/>
>>> <vcpupin vcpu='5' cpuset='7'/>
>>> <topology sockets='1' cores='3' threads='2'/>
>>
>>
>> Oh wow, maybe I'm completely off.  I hadn't noticed the core-id
>> ordering in the VM, but indeed on my system the guest shows:
>>
>> core id         : 0
>> core id         : 0
>> core id         : 1
>> core id         : 1
>> core id         : 2
>> core id         : 2
>> core id         : 3
>> core id         : 3
>>
>> Any physical system I've seen has the ordering you show above,
>> enumerating cores then threads.  Sorry Jan, maybe the way you had it
>> originally was optimal.  Thanks,
>>
>> Alex
>>
>
>
> Ah, good to know! I will change my config back to the original one.
> I was curious how I could get this information on a Windows system. After
> some googling I found coreinfo [1], which reports this:
>
> Logical to Physical Processor Map:
> **----------  Physical Processor 0 (Hyperthreaded)
> --**--------  Physical Processor 1 (Hyperthreaded)
> ----**------  Physical Processor 2 (Hyperthreaded)
> ------**----  Physical Processor 3 (Hyperthreaded)
> --------**--  Physical Processor 4 (Hyperthreaded)
> ----------**  Physical Processor 5 (Hyperthreaded)
>
> But I'm still having my memory allocation problem: Can someone confirm that
> the numatune option actually works? I'm interested in the
> Kernel/Qemu/Libvirt version.
>
> Jan
>
> [1] https://technet.microsoft.com/en-us/sysinternals/cc835722.aspx
>
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users



-- 
Cheers,
~Blairo