[libvirt] increase number of libvirt threads by starting tansient guest doamin - is it a bug?

web2 ustermann78 at web.de
Tue Oct 7 19:04:10 UTC 2014


 Hi,

i have found the time and tested Xen-4.4.1 with libvirt-1.2.6 and the problem with the threads are disappered. But now i see after every restore of an xen-vm (windows-xp hvm) an openfile-handle (via lsof -p `pgrep libvirtd`) to an file in /var/lib/xen with the name qemu-resume.<vm-id> and what me real make confusion is that the entry is marked as deleted.If i look in the directory, the file is really no more their, but libvirt have still have an handle to that none existing file!?!

I know that "Xen" create such a file before they startet the vm, and i also know that xen delete that file , but i don´t understand while it is listed under the openfiles of libvirtd after the vm is started/destroyed???

all the best
max
 
----------------ursprüngliche Nachricht-----------------
Von: "web2" ustermann78 at web.de 
An: "Jim Fehlig" jfehlig at suse.com , "web2" ustermann78 at web.de 
Kopie: "Michal Privoznik" mprivozn at redhat.com , "libvirt-users redhat.com" 
, "libvirt-list redhat.com" 
Datum: Thu, 2 Oct 2014 07:49:03 +0000
-------------------------------------------------
 
 
> 
> 
> ----------------ursprüngliche Nachricht-----------------
> Von: "Jim Fehlig" jfehlig at suse.com 
> An: "web2" ustermann78 at web.de 
> Kopie: "Michal Privoznik" mprivozn at redhat.com , "libvirt-users redhat.com" 
> 
> , "libvirt-list redhat.com" 
> Datum: Wed, 01 Oct 2014 21:38:05 -0600
> -------------------------------------------------
> 
> 
>> web2 wrote:
>>> Hi
>>> 
>>> ----------------ursprüngliche Nachricht-----------------
>>> Von: "Michal Privoznik" mprivozn at redhat.com 
>>> An: "web2" ustermann78 at web.de , "libvirt-users redhat.com" 
>>> , "libvirt-list redhat.com" 
>>> Datum: Wed, 01 Oct 2014 18:12:45 +0200
>>> -------------------------------------------------
>>> 
>>> 
>>> 
>>>> On 01.10.2014 10:31, web2 wrote:
>>>> 
>>>>> Hello,
>>>>>
>>>>> sorry for my later answer.
>>>>>
>>>>> so, after i started libvirtd (no vm´s running) and attach gdb i get 
>>>>> the 
>>>>> following threads
>>>>>
>>>>> (gdb) info thread
>>>>> Id Target Id Frame
>>>>> 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 
>>>>> 0x00007f190677d7cd in 
>>>>> poll () at ../sysdeps/unix/syscall-template.S:81
>>>>>
>>>>> if i restore an persistent domain, i see the following in gdb:
>>>>>
>>>>> Detaching after fork from child process 20880.
>>>>> Detaching after fork from child process 20882.
>>>>> [New Thread 0x7f190893d700 (LWP 20883)]
>>>>> Detaching after fork from child process 20890.
>>>>> Detaching after fork from child process 20906.
>>>>>
>>>>> (gdb) info thread
>>>>> Id Target Id Frame
>>>>> 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 
>>>>> 0x00007f1906e6684d in 
>>>>> read () at ../sysdeps/unix/syscall-template.S:81
>>>>> 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 
>>>>> 0x00007f190677d7cd in 
>>>>> poll () at ../sysdeps/unix/syscall-template.S:81
>>>>>
>>>>> if i now destroy this vm i get the following:
>>>>>
>>>>>
>>>>> (gdb) info thread
>>>>> Id Target Id Frame
>>>>> 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 
>>>>> 0x00007f1906e6684d in 
>>>>> read () at ../sysdeps/unix/syscall-template.S:81
>>>>> 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" 
>>>>> pthread_cond_wait@@GLIBC_2.3.2 () at 
>>>>> 
>>>>> 
>>>>> ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1
>>>>> 85
>>>>> 
>>>>> * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 
>>>>> 0x00007f190677d7cd in 
>>>>> poll () at ../sysdeps/unix/syscall-template.S:81
>>>>>
>>>>> attaching thread Id 12 and bt:
>>>>> (gdb) thread 12
>>>>> [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))]
>>>>> #0 0x00007f1906e6684d in read () at 
>>>>> ../sysdeps/unix/syscall-template.S:81
>>>>> 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
>>>>> (gdb) bt
>>>>> #0 0x00007f1906e6684d in read () at 
>>>>> ../sysdeps/unix/syscall-template.S:81
>>>>> #1 0x00007f18f8c415be in read (__nbytes=16, 
>>>>> __buf=0x7f18d4000c50, 
>>>>> __fd=26) at /usr/include/bits/unistd.h:44
>>>>> #2 read_all (fd=26, data=data at entry =0x7f18d4000c50, 
>>>>> len=len at entry =16, 
>>>>> nonblocking=nonblocking at entry =0) at xs.c:374
>>>>> #3 0x00007f18f8c41675 in read_message (h=h at entry 
>>>>> =0x7f18e8001070, 
>>>>> nonblocking=nonblocking at entry =0) at xs.c:1139
>>>>> #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at 
>>>>> xs.c:1211
>>>>> #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at 
>>>>> 
>>>>> pthread_create.c:309
>>>>> #6 0x00007f1906787c3d in clone () at 
>>>>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
>>>>>
>>>>> and then bt full:
>>>>>
>>>>> (gdb) bt full
>>>>> #0 0x00007f1906e6684d in read () at 
>>>>> ../sysdeps/unix/syscall-template.S:81
>>>>> No locals.
>>>>> #1 0x00007f18f8c415be in read (__nbytes=16, 
>>>>> __buf=0x7f18d4000c50, 
>>>>> __fd=26) at /usr/include/bits/unistd.h:44
>>>>> No locals.
>>>>> #2 read_all (fd=26, data=data at entry =0x7f18d4000c50, 
>>>>> len=len at entry =16, 
>>>>> nonblocking=nonblocking at entry =0) at xs.c:374
>>>>> done = <optimized out>
>>>>> #3 0x00007f18f8c41675 in read_message (h=h at entry 
>>>>> =0x7f18e8001070, 
>>>>> nonblocking=nonblocking at entry =0) at xs.c:1139
>>>>> __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 
>>>>> 
>>>>> 0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>}
>>>>> msg = 0x7f18d4000c40
>>>>> body = 0x0
>>>>> saved_errno = 0
>>>>> ret = -1
>>>>> #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at 
>>>>> xs.c:1211
>>>>> h = 0x7f18e8001070
>>>>> fd = <optimized out>
>>>>> 
>>>> This stack doesn't come from libvirt. I suspect it comes from a library 
>>>> 
>>>> that libvirtd is linked with.
>> 
>> Yep, a thread created by libxl.
>> 
>>>> Maybe libxl? What's the connection uri 
>>>> you're using?
>>>> 
>>>
>>> i use xen:///. as i said early, i do tests with xen xm´s. xend is disabled, so 
>>> i 
>>> use the libxl.
>>> 
>> 
>> Earlier you mentioned using libvirt 1.1.3 for your Xen hosts. Have you
>> tried the freshly released 1.2.9? I didn't see the problem with libvirt
>> git master and Xen 4.4.1.
> 
> I use libvirt 1.1.3.5 and Xen 4.3.3 for the tests. I will try a newer version of Xen 
> and then a newer version of libvirt.
> 
> thanks
> max
> 
> 
>> Regards,
>> Jim
>> 
>> 
> 
> -- 
> 
> 
> 
> 






More information about the libvir-list mailing list