On Tue, Oct 30, 2012 at 10:08 PM, Eric Blake <span dir="ltr"><<a href="mailto:eblake@redhat.com" target="_blank">eblake@redhat.com</a>></span> wrote:<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Both bitmaps show all 24 cores, so hwloc is able to read sysfs and<br>
determine the existence of 2 sockets with 12 nodes each, and where the<br>
12 nodes are numbered 0-5 twice according to which bank of cache they<br>
are tied to. Which version of libvirt is this tested on where libvirt<br>
was only reporting 12 cores, because I thought we already patched that<br>
with commit 80533ca in 0.10.0. That is, I think proc1.png should result in:<br>
<div class="im"><br>
$ virsh nodeinfo<br>
CPU model: x86_64<br>
CPU(s): 24<br>
CPU frequency: 2200 MHz<br>
</div> CPU socket(s): 2<br>
Core(s) per socket: 12<br>
<div class="im"> Thread(s) per core: 1<br>
NUMA cell(s): 1<br>
Memory size: 8047272 KiB<br>
<br></div></blockquote><div><br></div><div>(Just for clarity, I am the original reporter, sorry I couldn't answer earlier, I subscribed to the list too late and didn't want to break the thread)</div><div><br></div>
<div>On the host with 1 NUMA cell, with libvrit 0.10.2 I get:</div><div><br></div><div><div>[root@host29 ~]# virsh nodeinfo</div><div>CPU model: x86_64</div><div>CPU(s): 24</div><div>CPU frequency: 2200 MHz</div>
<div>CPU socket(s): 2</div><div>Core(s) per socket: 6</div><div>Thread(s) per core: 1</div><div>NUMA cell(s): 1</div><div>Memory size: 131971548 KiB</div></div><div><br></div><div>Furthermore, to answer the question at the bottom of your email, <a href="http://birzan.org/capabilities1.txt">http://birzan.org/capabilities1.txt</a> has this info.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
</div>and proc4.png would _ideally_ result in:<br>
<div class="im"><br>
$ virsh nodeinfo<br>
CPU model: x86_64<br>
CPU(s): 24<br>
CPU frequency: 2200 MHz<br>
</div> CPU socket(s): 2<br>
Core(s) per socket: 12<br>
Thread(s) per core: 1<br>
NUMA cell(s): 4<br>
Memory size: 8047272 KiB<br>
<br></blockquote><div><div><br></div><div>[root@host34 ~]# virsh nodeinfo</div><div>CPU model: x86_64</div><div>CPU(s): 24</div><div>CPU frequency: 2200 MHz</div><div>CPU socket(s): 2</div>
<div>Core(s) per socket: 6</div><div>Thread(s) per core: 1</div><div>NUMA cell(s): 1</div><div>Memory size: 131971020 kB</div></div><div><br></div><div>And <a href="http://birzan.org/capabilities4.txt">http://birzan.org/capabilities4.txt</a> </div>
<div><br></div><div>This is on 0.9.11.5, on 0.10.2:</div><div><br></div><div><div>[root@host34 libvirt]# virsh nodeinfo</div><div>CPU model: x86_64</div><div>CPU(s): 24</div><div>CPU frequency: 2200 MHz</div>
<div>CPU socket(s): 1</div><div>Core(s) per socket: 6</div><div>Thread(s) per core: 1</div><div>NUMA cell(s): 4</div><div>Memory size: 131971020 KiB</div></div><div><br></div><div>and <a href="http://birzan.org/capabilities4-newlibvirt.txt">http://birzan.org/capabilities4-newlibvirt.txt</a> </div>
<div><br></div><div>This has made the problem even worse, as we now can only use 6 cores out of 24 by default (libvirt pins qemus to the CPUs is sees available, so we have to manually taskset them after starting).</div><div>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I think the CPU _is_ reporting the complete NUMA topology through sysfs,<br>
but that we are probably consolidating information from the wrong files<br>
and therefore getting confused.<br>
<br></blockquote><div>The sysfs is tarred up in <a href="http://birzan.org/sysdevicessystem.tar.gz">http://birzan.org/sysdevicessystem.tar.gz</a> for host29 (the one with 1 numa cell in capabilities) and <a href="http://birzan.org/sysdevicessystem-34.tar.gz">http://birzan.org/sysdevicessystem-34.tar.gz</a></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Also, what does the 'virsh capabilities' report for the <topology><br>
section? Whereas 'virsh nodeinfo' is constrained by back-compat to give<br>
a lame answer for number of NUMA cells, at least 'virsh capabilities'<br>
should be showing a reasonable representation of the machine's topology.<br>
<div class="HOEnZb"><div class="h5"><br></div></div></blockquote><div><br></div><div>I answered this above. If you need any more info, feel free to ask.</div><div> </div></div><div><br></div>-- <br>George-Cristian Bîrzan<br>