[libvirt-users] VMs fail to start with NUMA configuration

[Doug Goldstein]

On Wed, Jan 23, 2013 at 11:02 PM, Osier Yang <jyang at redhat.com> wrote:
> On 2013年01月24日 12:11, Doug Goldstein wrote:
>>
>> On Wed, Jan 23, 2013 at 3:45 PM, Doug Goldstein<cardoe at gentoo.org>  wrote:
>>>
>>> I am using libvirt 0.10.2.2 and qemu-kvm 1.2.2 (qemu-kvm 1.2.0 + qemu
>>> 1.2.2 applied on top plus a number of stability patches). Having issue
>>> where my VMs fail to start with the following message:
>>>
>>> kvm_init_vcpu failed: Cannot allocate memory
>
>
> Smell likes we have problem on setting the NUMA policy (perhaps
> caused by the incorrect host NUMA topology), given that the system
> still has enough memory. Or numad (if it's installed) is doing
> something wrong.
>
> Can you see if there is something about the Nodeset used to set
> the policy in debug log?
>
> E.g.
>
> % cat libvirtd.debug | grep Nodeset

Well I don't see anything but its likely because I didn't do something
correct. I had LIBVIRT_DEBUG=1 exported and ran libvirtd --verbose
from the command line. My /etc/libvirt/libvirtd.conf had:
log_outputs="3:syslog:libvirtd 1:file:/tmp/libvirtd.log" But I didn't
get any debug messages.

> % cat /var/log/libvirt/qemu/$guest.log

Nothing in here. Just the command line:

2013-01-24 04:00:49.989+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/
bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.6.3 HOME=/root USER=root LOGNAME=root QEM
U_AUDIO_DRV=none /usr/bin/qemu-kvm -name bb-2.6.18-128.el5.x86_64 -S -M pc-1.2 -
cpu Opteron_G4,+perfctr_nb,+perfctr_core,+topoext,+nodeid_msr,+lwp,+wdt,+skinit,
+ibs,+osvw,+cr8legacy,+extapic,+cmp_legacy,+fxsr_opt,+mmxext,+osxsave,+monitor,+
ht,+vme -enable-kvm -m 2048 -smp 2,sockets=1,cores=2,threads=1 -uuid
2b5990f0-649f-ae25-99f0-dc4b05f682e1 -no-user-config -nodefaults
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/bb-2.6.18-128.el5.x86_64.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc
base=utc,driftfix=slew -no-shutdown -boot menu=off -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/dev/disk/by-path/ip-192.168.200.20:3260-iscsi-iqn.2011-07.lab.san-1:2.6.18-128.el5-x86_64-lun-0,if=none,id=drive-ide0-0-0,format=raw,cache=none,werror=stop,rerror=stop,aio=native
-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
-netdev tap,fd=31,id=hostnet0 -device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:7c:b2:81,bus=pci.0,addr=0x7
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:10,password
-vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device
i6300esb,id=watchdog0,bus=pci.0,addr=0x6 -watchdog-action reset
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
char device redirected to /dev/pts/12
CPU feature perfctr_nb not found
CPU feature perfctr_core not found
CPU feature topoext not found
CPU feature lwp not found
kvm_init_vcpu failed: Cannot allocate memory
2013-01-24 04:00:50.194+0000: shutting down

>
> On the other hand, I'm wondering if there is a chance to see the
> full error msg kernel throws out.
>

That's what I've been trying to figure out. Trying to get some more
info out of libvirt as well. If you've got any ideas I'll give them a
shot.

-- 
Doug Goldstein