buildsys woes

Wed Jan 18 17:31:46 UTC 2006

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings. 

Seems the buildsys has been locking up pretty regularly of late. 

ignacio on the #fedora-extras channel mentioned that he was seeing
mock builds also lock up. 

So, I pulled down fedora-extras cvs last night and fired off a job to
run a 'make mockbuild' for each package on my devel test machine. 

Indeed, it had locked up this morning. 

It looks like somehow mock-yum's groupinstall had installed the
net-tools rpm, but then tried to install it again (another thread?)
and they got stuck trying to lock the rpm db. (see the two fd's open
on the net-tools rpm?). Perhaps it got stuck reading from the rpm pipe
that it had already installed? I don't know enough about mock/yum
internals to say. 

it's worth noting all the jobs on the buildsys I have seen that are
causing it to stick are development jobs. This points to yum or mock
in development. Have they changed recently? Anything else on the build
systems changed recently? 

Hopefully this info will help someone fix the buildsys. ;) 

This is on a test box here, if anyone would like further info, catch
me on #fedora-extras. I can leave it in the stuck state, run further
commands, provide remote access for developers, etc. 

kevin
- --
extras   12760  0.0  0.1   3740   844 pts/3    S    08:16   0:00 make mockbuild
extras   12854  0.0  1.2  15360  6452 pts/3    S    08:16   0:00 /usr/bin/python -tt /usr/bin/mock -r fedora-5-i386-core.cfg --resultdir=/home/extras/extras/devel/autossh/autossh-1_3-2_fc5 /home/extras/extras/devel/autossh/autossh-1.3-2.fc5.src.rpm
extras   12884  0.0  0.2   4332  1060 pts/3    S    08:16   0:00 sh -c { /usr/sbin/mock-helper yum --installroot /var/lib/mock/fedora-development-i386-core/root groupinstall build; } 2>&1
root     12892  0.0  0.0      0     0 ?        S    08:18   0:00 [pdflush]
root     13495  0.0  0.2   4668  1184 pts/3    S    08:29   0:00 su
root     13497  0.0  0.2   4516  1516 pts/3    S    08:29   0:00 bash
root     12885  0.9 14.3  82312 74128 pts/3    S    08:16   1:13 /usr/bin/python /usr/libexec/mock-yum --installroot /var/lib/mock/fedora-development-i386-core/root groupinstall build

ls -lt /proc/12885/fd
total 31
lrwx------ 1 root extras 64 Jan 18 08:31 24 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Sigmd5
lrwx------ 1 root extras 64 Jan 18 08:31 25 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Sha1header
lrwx------ 1 root extras 64 Jan 18 08:31 26 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Filemd5s
lrwx------ 1 root extras 64 Jan 18 08:31 27 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Triggername
lrwx------ 1 root extras 64 Jan 18 08:31 28 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Pubkeys
lrwx------ 1 root extras 64 Jan 18 08:31 29 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Conflictname
l-wx------ 1 root extras 64 Jan 18 08:31 30 -> pipe:[2541345]
lrwx------ 1 root extras 64 Jan 18 08:31 10 -> /var/cache/yum/local/primary.xml.gz.sqlite
lrwx------ 1 root extras 64 Jan 18 08:31 11 -> /var/cache/yum/groups/primary.xml.gz.sqlite
lrwx------ 1 root extras 64 Jan 18 08:31 12 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Name
lrwx------ 1 root extras 64 Jan 18 08:31 13 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Basenames
lrwx------ 1 root extras 64 Jan 18 08:31 14 -> /var/lib/rpm/__db.000
lr-x------ 1 root extras 64 Jan 18 08:31 15 -> /var/cache/yum/core/packages/net-tools-1.60-60.i386.rpm
lr-x------ 1 root extras 64 Jan 18 08:31 16 -> /var/cache/yum/core/packages/net-tools-1.60-60.i386.rpm
lrwx------ 1 root extras 64 Jan 18 08:31 17 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Group
lrwx------ 1 root extras 64 Jan 18 08:31 18 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Requirename
lrwx------ 1 root extras 64 Jan 18 08:31 19 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Providename
lrwx------ 1 root extras 64 Jan 18 08:31 20 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Dirnames
lrwx------ 1 root extras 64 Jan 18 08:31 21 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Requireversion
lrwx------ 1 root extras 64 Jan 18 08:31 22 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Provideversion
lrwx------ 1 root extras 64 Jan 18 08:31 23 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Installtid
lrwx------ 1 root extras 64 Jan 18 08:31 3 -> /home/extras/extras/devel/autossh/autossh-1_3-2_fc5/root.log
lrwx------ 1 root extras 64 Jan 18 08:31 4 -> /home/extras/extras/devel/autossh/autossh-1_3-2_fc5/build.log
l-wx------ 1 root extras 64 Jan 18 08:31 5 -> /var/log/yum.log
lr-x------ 1 root extras 64 Jan 18 08:31 6 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Packages
lrwx------ 1 root extras 64 Jan 18 08:31 7 -> /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm/Packages
lrwx------ 1 root extras 64 Jan 18 08:31 8 -> /var/cache/yum/core/primary.xml.gz.sqlite
lrwx------ 1 root extras 64 Jan 18 08:31 9 -> /var/cache/yum/extras/primary.xml.gz.sqlite
lrwx------ 1 root extras 64 Jan 18 08:31 0 -> /dev/pts/3
l-wx------ 1 root extras 64 Jan 18 08:31 1 -> pipe:[2541345]
l-wx------ 1 root extras 64 Jan 18 08:31 2 -> pipe:[2541345]

rpm --root /var/lib/mock/fedora-development-i386-core/root -qi net-tools
Name        : net-tools                    Relocations: (not relocatable)
Version     : 1.60                              Vendor: Red Hat, Inc.
Release     : 60                            Build Date: Tue 17 Jan 2006 05:08:51 AM MST
Install Date: Wed 18 Jan 2006 08:20:58 AM MST      Build Host: ls20-bc1-13.build.redhat.com
Group       : System Environment/Base       Source RPM: net-tools-1.60-60.src.rpm
Size        : 745942                           License: GPL
Signature   : (none)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Summary     : Basic networking tools.
Description :
The net-tools package contains basic networking tools, including
ifconfig, netstat, route, and others.

cat root.log
ensuring dir /var/lib/mock/fedora-development-i386-core/state
Cleaning Root
ensuring dir /var/lib/mock/fedora-development-i386-core
ensuring dir /var/lib/mock/fedora-development-i386-core/root
ensuring dir /var/lib/mock/fedora-development-i386-core/state
ensuring dir /home/extras/extras/devel/autossh/autossh-1_3-2_fc5
ensuring dir /var/lib/mock/fedora-development-i386-core
ensuring dir /var/lib/mock/fedora-development-i386-core/root
ensuring dir /var/lib/mock/fedora-development-i386-core/state
ensuring dir /home/extras/extras/devel/autossh/autossh-1_3-2_fc5
ensuring dir /var/lib/mock/fedora-development-i386-core/root/var/lib/rpm
ensuring dir /var/lib/mock/fedora-development-i386-core/root/var/log
ensuring dir /var/lib/mock/fedora-development-i386-core/root/dev
ensuring dir /var/lib/mock/fedora-development-i386-core/root/etc/rpm
ensuring dir /var/lib/mock/fedora-development-i386-core/root/tmp
ensuring dir /var/lib/mock/fedora-development-i386-core/root/var/tmp
ensuring dir /var/lib/mock/fedora-development-i386-core/root/etc/yum.repos.d
ensuring dir /var/lib/mock/fedora-development-i386-core/root/proc
ensuring dir /var/lib/mock/fedora-development-i386-core/root/dev/pts
ensuring dir /var/lib/mock/fedora-development-i386-core/root/proc
ensuring dir /var/lib/mock/fedora-development-i386-core/root/dev/pts
/usr/sbin/mock-helper yum --installroot /var/lib/mock/fedora-development-i386-core/root groupinstall build

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFDznuF3imCezTjY0ERApU3AJ9ni+SPQ0PdrfyuAm8Ko5gP0ZIVXgCgjD/y
qJX0bbBjsXNjJa3SLZfCZ/Q=
=b2dx
-----END PGP SIGNATURE-----