[Libvirt-cim] [PATCH] DO NOT PUSH UPSTREAM

Tue Aug 30 13:48:38 UTC 2011

DV,

    Its not mirrored to RH yet, I am working on it. In the meantime, here
is the description of the defect.

********************************************
virInitialize leads to segfault when unloading libvirt-cim providers.

Contact Information = Sharad Mishra/snmishra at us.ibm.com,

---uname output---
Linux elm3b193.beaverton.ibm.com 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39
EDT
2010 x86_64 x86_64 x86_64 GNU/Linux

Machine Type = IBM system x blade, machine type - 8853, model - MC1

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 1. Install libvirt and libvirt-cim
2 start libvirtd
3. start tog-pegasus
4. run wbemcli command to enum instance on KVM_ComputerSystem class. This
is
just a way to load the provider to memory. All  we need to do is run
virConnectOpen() and virConnectClose() in this provider. So in this step
all
that is required is for virConnectOpen and virConnectClose to run.
5. Stop tog-pegasus.

A core is gererated.

---System Management Component Data---
Userspace tool common name: libvirt/libvirt-cim/tog-pegasus

System management type: IMM

The userspace tool has the following bit modes: 64

Userspace rpm: libvirt-0.6.3-33.el5,  libvirt-cim-0.5.14-1 and
tog-pegasus-2.9.2-2.el5

Userspace tool obtained from project website:  na

*Additional Instructions for Sharad Mishra/snmishra at us.ibm.com, :
-Post a private note with access information to the machine that the bug is
occuring on.
-Attach ltrace and strace of userspace application.
-Attach screen shoot of System Managment Utility

Here is the gist of investigation done so far ---

Here is the gist of his investigation -

I've looked into the libvirt source code and the reason for issues with the
unload of libvirt-cim providers is there (and cimprovider -r forces an
unload,
just as cimserver shutdown does).

virConnectOpen() does call virInitialize() which then will initialize the
gcrypt library after initializes the thread setup ->virThreadInitialize(),
creates the error reporting structure ->virErrorInitialize() and initialize
a
random number generator ->virRandomInitialize().

The issue is hidden in virErrorInitialize(). It calls virErrorInitialize()
which uses the wrapper function virThreadLocalInit(&virLastErr,
virLastErrFreeData);.
virThreadLocalInit() calls pthread_create() with a destructor function
(virLastErrFreeData()) which will be called on thread exits.

Trouble here is that pthread_key_delete() is never called in libvirt, but
next
time the system runs through the key list and libvirt has been unloaded the
system will jump to an already unloaded function (the pointer were
virLastErrFreeData() was loaded formerly) and consequently crash the
process.

Fixing this will not be easy, but a start would be a libvirt function on
its
externalize interface that allows cleanup short before library unload, or
alternatively libvirt does the thread key delete on library unload.

Trace output from gdb -

/var/log/messages:
Aug  6 11:35:58 SYX-e41f13fe5558 cimserver[21602]: OpenPegasus stopped.
Aug  6 11:35:59 SYX-e41f13fe5558 kernel: cimserver[21376]: segfault at
00002aaaab165f00 rip 00002aaaab165f00 rsp 00000000457d40b8 error 14

call stack trace:
command: gdb -batch -x /tmp/gdb.27236.cmd /usr/sbin/cimserver
/var/log/core/core.25567
time:    Sat Aug  6 11:35:59 UTC 2011
stdout/stderr:
-----------------------------------------------------------------
[New Thread 25567]
Core was generated by `/usr/sbin/cimserver'.
Program terminated with signal 11, Segmentation fault.
#0  0x00002aaaab4f6f00 in ?? ()
#0  0x00002aaaab4f6f00 in ?? ()
#1  0x00002ab4fb3c6ad9 in __nptl_deallocate_tsd ()
from /lib64/libpthread.so.0
#2  0x00002ab4fb3c774b in start_thread () from /lib64/libpthread.so.0
#3  0x00002ab4fc07af6d in clone () from /lib64/libc.so.6
-----------------------------------------------------------------
 exit_code=0
-----------------------------------------------------------------

Regards,
Sharad Mishra
Open Virtualization
Linux Technology Center
IBM

libvirt-cim-bounces at redhat.com wrote on 08/30/2011 01:43:06 AM:

> Daniel Veillard <veillard at redhat.com>
> Sent by: libvirt-cim-bounces at redhat.com
>
> 08/30/11 01:43 AM
>
> Please respond to
> veillard at redhat.com; Please respond to
> List for discussion and development of libvirt CIM
<libvirt-cim at redhat.com>
>
> To
>
> List for discussion and development of libvirt CIM
<libvirt-cim at redhat.com>
>
> cc
>
> Subject
>
> Re: [Libvirt-cim] [PATCH] DO NOT PUSH UPSTREAM
>
> On Fri, Aug 26, 2011 at 05:38:29PM -0000, Sharad Mishra wrote:
> > # HG changeset patch
> > # User Sharad Mishra <snmishra at us.ibm.com>
> > # Date 1314380301 25200
> > # Node ID 73e7bb11410f3c10d3b0b61678531314262f09b7
> > # Parent  2984ede9c082bab48b3494ef4b9f5561fadef2ad
> > DO NOT PUSH UPSTREAM
> >
> > This is the workaround that was given to zGryphon
> > team to prevent unloading of idle providers.
> > There is a bug in libvirt that needs to be fixed.
> > That fix will be the actual fix for this problem
> > that cause segfault when those providers are unloaded
> > that make a call to virConnectOpen.
>
>   I'm curious about the libvirt bug, which one is it ?
>
> Daniel
>
> --
> Daniel Veillard      | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
> daniel at veillard.com  | Rpmfind RPM search engine http://rpmfind.net/
> http://veillard.com/ | virtualization library  http://libvirt.org/
>
> _______________________________________________
> Libvirt-cim mailing list
> Libvirt-cim at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-cim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-cim/attachments/20110830/265e457b/attachment.htm>