[libvirt-users] Inconsistent behavior between x86_64 and ppc64 when creating guests with NUMA node placement

[Michal Privoznik]

On 12.09.2014 21:10, Michael Turek wrote:
> Hello all,
>
> I was recently trying out NUMA placement for my guests on both x86_64
> and ppc64 machines. When booting a guest on the x86_64 machine, the
> following specs were valid (obviously, just notable excepts from the xml):
>
> <memory unit='KiB'>8388608</memory>
> <currentMemory unit='KiB'>8388608</currentMemory>
>   <vcpu placement='static'>4</vcpu>
> ...
>    <cpu>
>      <topology sockets='4' cores='1' threads='1'/>
>      <numa>
>        <cell cpus='0-2' memory='6144'/>
>        <cell cpus='3' memory='2048'/>
>      </numa>
>    </cpu>


6144 + 2048 != 8388608.
Either you need to multiply those values by factor of 1024 (6144*1024 + 
2048*1024 == 8388608) or add @unit='MiB' attribute, so libvirt knows 
what units you mean.

>
> However, on ppc64 this causes the following error:
>
> error: Failed to create domain from sample_guest.xml
> error: internal error: early end of file from monitor: possible problem:
> 2014-09-11T18:44:25.502140Z qemu-system-ppc64: total memory for NUMA
> nodes (8388608) should equal RAM size (200000000)
>
> The 200000000 is actually 8192 MB in bytes and hexidecimal. This is
> apparently just an issue with the error message.
>
> The following specs work on ppc64:
>    <cpu>
>      <topology sockets='4' cores='1' threads='1'/>
>      <numa>
>        <cell cpus='0-2' memory='6291456'/>
>        <cell cpus='3' memory='2097152'/>
>      </numa>
>    </cpu>
>
> Note that the memory for each cell is 6144*1024 and 2048*1024
> respectively. The issue is that the memory size for each NUMA cell
> should be specified in KiB, not MB
> (http://libvirt.org/formatdomain.html#resPartition "|memory| specifies
> the node memory in kibibyte").
>
> In short, it seems that specifying NUMA cell memory in MB works on
> x86_64 but not on ppc64. Does anyone have any insight to what's causing
> this, or if I'm misunderstanding something? Any help is appreciated,
> thank you!

What does the corresponding part of qemu command line look like? In both 
cases. I don't see anything architecture dependent there.

What may happen here is that qemu on x86_64 does the checking while on 
ppc64 it does not.

Michal