[libvirt] crash in libvirtd when assigning PCI device (storage controller)

Daniel P. Berrange berrange at redhat.com
Wed Nov 25 11:23:07 UTC 2009


On Mon, Nov 23, 2009 at 10:08:17PM +0000, Terje Marthinussen wrote:
> [  987.286791] libvirtd[3193]: segfault at 10 ip 00007f921b4546b4 sp 
> 00007f9216e165e0 error 4 in libpthread-2.10.1.so[7f921b44a000+17000]

Ok, so this is the libvirtd daemon crash

> >From GDB
> 
> 06:45:32.434: error : qemuMonitorCommandWithHandler:290 : cannot send monitor command 'info cpus': Connection reset by peer

Interestingly this suggests that QEMU itself had quit/crashed during
startup. You can check /var/log/libvirt/qemu//$GUESTNAME.log to find
out what problem QEMU itself had.

For whatever reason, this seems to then trigger the libvirtd crash

> 06:45:32.434: error : qemuMonitorTextGetCPUInfo:436 : internal error cannot run monitor command to fetch CPU thread info
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7f485bfff910 (LWP 18120)]
> 0x00007f48657fb6b4 in pthread_mutex_unlock () from /lib/libpthread.so.0
> (gdb) bt
> #0  0x00007f48657fb6b4 in pthread_mutex_unlock () from /lib/libpthread.so.0
> #1  0x0000000000431101 in qemuDomainObjExitMonitorWithDriver (driver=0x11f0110, obj=0x12114f0) at qemu/qemu_driver.c:318
> #2  0x000000000043f436 in qemudStartVMDaemon (conn=<value optimized out>, driver=0x11f0110, vm=0x12114f0, 
>     migrateFrom=<value optimized out>, stdin_fd=<value optimized out>) at qemu/qemu_driver.c:2320
> #3  0x00000000004407c4 in qemudDomainStart (dom=0x11f0330) at qemu/qemu_driver.c:4370
> #4  0x00007f4865a621e7 in virDomainCreate (domain=0x11f0330) at libvirt.c:4509
> #5  0x0000000000420d68 in remoteDispatchDomainCreate (server=<value optimized out>, client=<value optimized out>, conn=0x1219da0, 
>     hdr=<value optimized out>, rerr=0x7f485bffedf0, args=<value optimized out>, ret=0x7f485bffeed0) at remote.c:853
> #6  0x00000000004228e1 in remoteDispatchClientCall (server=<value optimized out>, client=0x7f485c000d30, msg=0x7f485c080f00)
>     at dispatch.c:506
> #7  0x0000000000422c93 in remoteDispatchClientRequest (server=0x11e2790, client=0x7f485c000d30, msg=0x7f485c080f00) at dispatch.c:388
> #8  0x000000000041625c in qemudWorker (data=<value optimized out>) at libvirtd.c:1518
> #9  0x00007f48657f7a04 in start_thread () from /lib/libpthread.so.0
> #10 0x00007f48655617bd in clone () from /lib/libc.so.6
> #11 0x0000000000000000 in ?? ()
> (gdb) select 2
> (gdb) info locals
> argv = 0x0
> tmp = <value optimized out>
> progenv = 0x0
> i = 1
> ret = 1
> sb = {st_dev = 64512, st_ino = 140307, st_nlink = 1, st_mode = 33261, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, 
>   st_size = 2293664, st_blksize = 4096, st_blocks = 4480, st_atim = {tv_sec = 1258937087, tv_nsec = 967226499}, st_mtim = {
>     tv_sec = 1257180462, tv_nsec = 0}, st_ctim = {tv_sec = 1258886336, tv_nsec = 104908677}, __unused = {0, 0, 0}}
> tapfds = 0x0
> ntapfds = 1
> qemuCmdFlags = 9436542
> keepfd = {fds_bits = {262144, 0 <repeats 15 times>}}
> emulator = <value optimized out>
> child = 18145
> pos = <value optimized out>
> ebuf = "\220v\036\001\000\000\000\000\060|WeH\177\000\000(\000\000\000\060\000\000\000\000\354\377[H\177\000\000@\353\377[H\177\000\000`\351\377[H\177\000\000\377\377\377\377H\177\000\000\233\017\037\001\000\000\000\000\326\063G\000\000\000\000\000@\247~eH\177\000\000\200\307OeH\177\000\000\240\306OeH\177\000\000JST\000\071\000\000\000\300\352\377[H\177\000\000\342\000\000\000\000\000\000\000\342\000\000\000\000\000\000\000\273\340G\000\000\000\000\000\300\352\377[H\177\000\000K\000\000\000\000\000\000\000`\351\377[H\177\000\000@\354\377[H\177\000\000P\352\377[H\177\000\000\001\000\000\000\000\000\000\000\320\352\377[H\177\000\000$\000\000\000\000\000\000\000py\255eH\177\000\000@\354\377[H\177\000\000\060|WeH\177\000\000\001\200\255\373\000\000\000\000\260\351\377[H\177\000\000\220\354\377[H\177\000\000\240\352\377["...
> pidfile = 0x0
> logfile = 15
> hookData = {conn = 0x1219da0, vm = 0x12114f0, driver = 0x11f0110}
> __FUNCTION__ = "qemudStartVMDaemon"
> __func__ = "qemudStartVMDaemon"


If you are able to reproduce this crash reliably, can you edit the
/etc/libvirt/libvirtd.conf file and add


  log_filters="1:qemu"
  log_outputs="1:file:/var/log/libvirtd.log"

and then restart libvirtd, and try to make it crash again. Then send
us the log file

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|




More information about the libvir-list mailing list