[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Buildsys strangeness



On Wed, 2006-06-07 at 13:20 +0200, Ralf Corsepius wrote:
> On Wed, 2006-06-07 at 12:06 +0200, Hans de Goede wrote:
> > Hi all,
> > 
> > I had a job which was building since yesterday evening, so it was 
> > clearly stuck hence i killed the job and requeued it now its stuck again 
> > in a very strange way:
> > 
> > 10586: dia (dia-0_95-4_fc6)  j w r degoede hhs nl   building/in-progress
> >          hammer3.fedora.redhat.com(i386): 
> > 3524044740cc821119c18f0b86d24a4a99e51884 done/building
> >          hammer3.fedora.redhat.com(i386): 
> > f70c9a2aea3719453f700a1ac04a1e632fd3b01f running/building
> >          hammer3.fedora.redhat.com(x86_64): 
> > da9431d21b048c528bcaff326544a1efd4d8a3b0 done/done
> >          ppc2.fedora.redhat.com(ppc): 
> > 653e112d8edb708fd7f730776e5cf5b6dba318ca done/done
> > 
> > Notice how its building for i386 twice (and on the same machine).
> 
> I am facing a similar issue:
> 
> # plague-client detail 10603
> 
> Detail for Job ID 10603 (OpenSceneGraph):
> --------------------------------------------------------------------------------
> Source: OpenSceneGraph-0_9_9-5_fc3
> Target: fedora-3-extras
> Submitter: rc040203 freenet de
> Status: building/
> Archjobs:
>     i386: hammer1.fedora.redhat.com    done/cleanup
>     x86_64: hammer2.fedora.redhat.com    running/prepping
>     i386: hammer1.fedora.redhat.com    running/prepping

Any ideas on how to further debug the following?

I looked at tmraz's vpnc build (10608) and the builder's mock buildroot
setup processes were waiting on in fcntl() to lock a file somewhere deep
in RPM:

strace:
Process 18882 attached - interrupt to quit
fcntl(15, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}
<unfinished ...>

backtrace:

#0  0x0000003fdab0b216 in fcntl () from /lib64/tls/libpthread.so.0
#1  0x00002aaaaacc91d4 in rpmcliVerify () from /usr/lib64/librpm-4.3.so
#2  0x00002aaaaacc927c in rpmtsAcquireLock () from /usr/lib64/librpm-4.3.so
#3  0x00002aaaaacc57cd in rpmtsRun () from /usr/lib64/librpm-4.3.so
#4  0x00002aaaaab7987e in rpmte_Wrap () from /usr/lib64/python2.3/site-packages/rpmmodule.so
#5  0x0000003fd988973f in _PyEval_SliceIndex () from /usr/lib64/libpython2.3.so.1.0
#6  0x0000003fd9889c01 in _PyEval_SliceIndex () from /usr/lib64/libpython2.3.so.1.0
#7  0x0000003fd988b2b0 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.3.so.1.0
#8  0x0000003fd9889a8a in _PyEval_SliceIndex () from /usr/lib64/libpython2.3.so.1.0
#9  0x0000003fd988b2b0 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.3.so.1.0
#10 0x0000003fd988b512 in PyEval_EvalCode () from /usr/lib64/libpython2.3.so.1.0
#11 0x0000003fd98a4129 in PyErr_Display () from /usr/lib64/libpython2.3.so.1.0
#12 0x0000003fd98a510d in PyRun_SimpleFileExFlags () from /usr/lib64/libpython2.3.so.1.0
#13 0x0000003fd98aa808 in Py_Main () from /usr/lib64/libpython2.3.so.1.0
#14 0x0000003fd911c4bb in __libc_start_main () from /lib64/tls/libc.so.6

And 'lsof' output (doesn't look abnormal to me, but I have no idea what to look for):

COMMAND    PID USER   FD   TYPE DEVICE    SIZE     NODE NAME
mock-yum 18882 root  cwd    DIR    9,2    4096        2 /
mock-yum 18882 root  rtd    DIR    9,2    4096        2 /
mock-yum 18882 root  txt    REG    9,2    8344    53479 /usr/bin/python
mock-yum 18882 root  mem    REG    0,0                0 [heap] (stat: No such file or directory)
mock-yum 18882 root  mem    REG    9,2  115168   336007 /lib64/ld-2.3.6.so
mock-yum 18882 root  mem    REG    9,2 1613968   336009 /lib64/tls/libc-2.3.6.so
mock-yum 18882 root  mem    REG    9,2   19880   336029 /lib64/libdl-2.3.6.so
mock-yum 18882 root  mem    REG    9,2  643408   336011 /lib64/tls/libm-2.3.6.so
mock-yum 18882 root  mem    REG    9,2 1138632    49241 /usr/lib64/libpython2.3.so.1.0
mock-yum 18882 root  mem    REG    9,2   61384   336039 /lib64/tls/librt-2.3.6.so
mock-yum 18882 root  mem    REG    9,2   69080    53421 /usr/lib64/libelf-0.96.so
mock-yum 18882 root  mem    REG    9,2   64192    49170 /usr/lib64/libbz2.so.1.0.2
mock-yum 18882 root  mem    REG    9,2   34112    49172 /usr/lib64/libpopt.so.0.0.0
mock-yum 18882 root  mem    REG    9,2 1031496    49174 /usr/lib64/librpmdb-4.3.so
mock-yum 18882 root  mem    REG    9,2  386472    49171 /usr/lib64/librpmio-4.3.so
mock-yum 18882 root  mem    REG    9,2  133424    49169 /usr/lib64/libbeecrypt.so.6.2.0
mock-yum 18882 root  mem    REG    9,2  130024   336033 /lib64/tls/libpthread-2.3.6.so
mock-yum 18882 root  mem    REG    9,2   19160   336070 /lib64/libutil-2.3.6.so
mock-yum 18882 root  mem    REG    9,2  118816    98887 /usr/lib64/python2.3/site-packages/rpmmodule.so
mock-yum 18882 root  mem    REG    9,2  350760    49175 /usr/lib64/librpm-4.3.so
mock-yum 18882 root  mem    REG    9,2   79240    49159 /usr/lib64/libz.so.1.2.1.2
mock-yum 18882 root  mem    REG    9,2   62504   336031 /lib64/libselinux.so.1
mock-yum 18882 root  mem    REG    9,2   24896    98150 /usr/lib64/python2.3/lib-dynload/strop.so
mock-yum 18882 root  mem    REG    9,2   20480    98148 /usr/lib64/python2.3/lib-dynload/timemodule.so
mock-yum 18882 root  mem    REG    9,2   24232    98155 /usr/lib64/python2.3/lib-dynload/structmodule.so
mock-yum 18882 root  mem    REG    9,2   21952    98153 /usr/lib64/python2.3/lib-dynload/zlibmodule.so
mock-yum 18882 root  mem    REG    9,2   21672    98030 /usr/lib64/python2.3/lib-dynload/_localemodule.so
mock-yum 18882 root  mem    REG    9,2  357184    98885 /usr/lib64/python2.3/site-packages/libxml2mod.so
mock-yum 18882 root  mem    REG    9,2 1073016    49167 /usr/lib64/libxml2.so.2.6.16
mock-yum 18882 root  mem    REG    9,2   51120    98032 /usr/lib64/python2.3/lib-dynload/_socketmodule.so
mock-yum 18882 root  mem    REG    9,2   17824    98033 /usr/lib64/python2.3/lib-dynload/_ssl.so
mock-yum 18882 root  mem    REG    9,2  244320   336040 /lib64/libssl.so.0.9.7a
mock-yum 18882 root  mem    REG    9,2 1221368   336030 /lib64/libcrypto.so.0.9.7a
mock-yum 18882 root  mem    REG    9,2   93832    53341 /usr/lib64/libgssapi_krb5.so.2.2
mock-yum 18882 root  mem    REG    9,2  464040    53340 /usr/lib64/libkrb5.so.3.2
mock-yum 18882 root  mem    REG    9,2   10384   336023 /lib64/libcom_err.so.2.1
mock-yum 18882 root  mem    REG    9,2  145424    49152 /usr/lib64/libk5crypto.so.3.0
mock-yum 18882 root  mem    REG    9,2   93624   336019 /lib64/libresolv-2.3.6.so
mock-yum 18882 root  mem    REG    9,2   21224    98042 /usr/lib64/python2.3/lib-dynload/binascii.so
mock-yum 18882 root  mem    REG    9,2   18352    98060 /usr/lib64/python2.3/lib-dynload/mathmodule.so
mock-yum 18882 root  mem    REG    9,2   11808    98031 /usr/lib64/python2.3/lib-dynload/_random.so
mock-yum 18882 root  mem    REG    9,2   15528    98053 /usr/lib64/python2.3/lib-dynload/fcntlmodule.so
mock-yum 18882 root  mem    REG    9,2   19096    98045 /usr/lib64/python2.3/lib-dynload/cStringIO.so
mock-yum 18882 root  mem    REG    9,2   13456    98127 /usr/lib64/python2.3/lib-dynload/md5module.so
mock-yum 18882 root  mem    REG    9,2   13008    98142 /usr/lib64/python2.3/lib-dynload/shamodule.so
mock-yum 18882 root  mem    REG    9,2   75544    98044 /usr/lib64/python2.3/lib-dynload/cPickle.so
mock-yum 18882 root  mem    REG    9,2   19936    98136 /usr/lib64/python2.3/lib-dynload/readline.so
mock-yum 18882 root  mem    REG    9,2  229824    49201 /usr/lib64/libreadline.so.4.3
mock-yum 18882 root  mem    REG    9,2   15744   336225 /lib64/libtermcap.so.2.0.8
mock-yum 18882 root  mem    REG    9,2  407640    98684 /usr/lib64/python2.3/lib-dynload/unicodedata.so
mock-yum 18882 root  mem    REG    9,2   27800    98058 /usr/lib64/python2.3/lib-dynload/itertools.so
mock-yum 18882 root  mem    REG    9,2   38592    98040 /usr/lib64/python2.3/lib-dynload/arraymodule.so
mock-yum 18882 root  mem    REG    9,2   58840   336105 /lib64/libnss_files-2.3.6.so
mock-yum 18882 root  mem    REG    9,2   24712   336102 /lib64/libnss_dns-2.3.6.so
mock-yum 18882 root  mem    REG    9,3   16384 16369286 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/__db.001
mock-yum 18882 root  mem    REG    9,3 1318912 16369287 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/__db.002
mock-yum 18882 root  mem    REG    9,3  663552 16369288 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/__db.003
mock-yum 18882 root    0u   CHR    1,3             2442 /dev/null
mock-yum 18882 root    1w  FIFO    0,6          2493184 pipe
mock-yum 18882 root    2w  FIFO    0,6          2493184 pipe
mock-yum 18882 root    3w   REG    9,2 7580121   160431 /var/log/plague-builder.log
mock-yum 18882 root    4u   REG    9,3    3939 16369251 /mnt/build/builder_work/839bf2f17e49475381b3254433701b5c7d5234c5/result/root.log
mock-yum 18882 root    5u  IPv4   7990              TCP *:8889 (LISTEN)
mock-yum 18882 root    6u  IPv4   8396              TCP *:8888 (LISTEN)
mock-yum 18882 root    7u   REG    9,3    1003 16369244 /mnt/build/builder_work/839bf2f17e49475381b3254433701b5c7d5234c5/result/job.log
mock-yum 18882 root    8u   REG    9,3    1017  7356641 /mnt/build/builder_work/9de1339d8bff65b824cf4258bc107452768e5d48/result/job.log
mock-yum 18882 root    9u   REG    9,3       0 16369252 /mnt/build/builder_work/839bf2f17e49475381b3254433701b5c7d5234c5/result/build.log
mock-yum 18882 root   10w   REG    9,3       0 16369282 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/log/yum.log
mock-yum 18882 root   11r   REG    9,3   12288 16369289 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/Packages
mock-yum 18882 root   12r   REG    9,3   12288 16369289 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/Packages
mock-yum 18882 root   13r   REG    9,3   12288 16369290 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/Providename
mock-yum 18882 root   14r   REG    9,3   12288 16369376 /var/lib/mock/fedora-development-x86_64-core-839bf2f17e49475381b3254433701b5c7d5234c5/root/var/lib/rpm/Conflictname
mock-yum 18882 root   15u   REG    9,2       0   160839 /var/lock/rpm/transaction



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]