From jiteshs at marvell.com Wed Nov 11 07:24:50 2009 From: jiteshs at marvell.com (Jitesh Shah) Date: Tue, 10 Nov 2009 23:24:50 -0800 Subject: Signing RPMs Message-ID: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> So, I picked up the sign_unsigned.py script from releng. I replaced the keys in there with our keys, tweaked some minor stuff here and there and managed to get it running. I use it as "./sign_unsigned.py --level " and it runs alright. I can see that the signatures are cached under the sigcache directory (but NOT embedded in the rpms themselves, which makes sense since the rpm can probably be a part of different tags and might be signed differently within each tag) So, I thought, well, mash would be the one which'll embed the keys in the rpms. So, I set strict_keys to True.. added my key to the keys list in my .mash file. mash has no problems with the rpms and it can verify the signatures alright. But, it still doesn't embed the signatures in the rpm (is it supposed to?). So, the created repository still has all rpms unsigned. What am I missing here? where to the rpms get signed actually? Regards, Jitesh From jwboyer at gmail.com Wed Nov 11 13:15:36 2009 From: jwboyer at gmail.com (Josh Boyer) Date: Wed, 11 Nov 2009 08:15:36 -0500 Subject: Signing RPMs In-Reply-To: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> Message-ID: <20091111131536.GC5699@hansolo.jdub.homelinux.org> On Tue, Nov 10, 2009 at 11:24:50PM -0800, Jitesh Shah wrote: >So, I picked up the sign_unsigned.py script from releng. I replaced the keys in there with our keys, tweaked some minor stuff here and there and managed to get it running. >I use it as >"./sign_unsigned.py --level " >and it runs alright. I can see that the signatures are cached under the sigcache directory (but NOT embedded in the rpms themselves, which makes sense since the rpm can probably be a part of different tags and might be signed differently within each tag) > >So, I thought, well, mash would be the one which'll embed the keys in the rpms. So, I set strict_keys to True.. added my key to the keys list in my .mash file. mash has no problems with the rpms and it can verify the signatures alright. But, it still doesn't embed the signatures in the rpm (is it supposed to?). So, the created repository still has all rpms unsigned. > >What am I missing here? where to the rpms get signed actually? The sign_unsigned script should eventually do a koji API call to do 'write-signed-rpm' on the packages you are signing. That will assemble signed RPMs in koji itself, which mash will download and used. Fedora Rel-Eng doesn't use sign_unsigned anymore because we have a signing server setup now. However, it should still work. josh From dennis at ausil.us Wed Nov 11 16:08:22 2009 From: dennis at ausil.us (Dennis Gilmore) Date: Wed, 11 Nov 2009 10:08:22 -0600 Subject: Signing RPMs In-Reply-To: <20091111131536.GC5699@hansolo.jdub.homelinux.org> References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> <20091111131536.GC5699@hansolo.jdub.homelinux.org> Message-ID: <200911111008.27590.dennis@ausil.us> On Wednesday 11 November 2009 07:15:36 am Josh Boyer wrote: > On Tue, Nov 10, 2009 at 11:24:50PM -0800, Jitesh Shah wrote: > >So, I picked up the sign_unsigned.py script from releng. I replaced the > > keys in there with our keys, tweaked some minor stuff here and there and > > managed to get it running. I use it as > >"./sign_unsigned.py --level " > >and it runs alright. I can see that the signatures are cached under the > > sigcache directory (but NOT embedded in the rpms themselves, which makes > > sense since the rpm can probably be a part of different tags and might be > > signed differently within each tag) > > > >So, I thought, well, mash would be the one which'll embed the keys in the > > rpms. So, I set strict_keys to True.. added my key to the keys list in my > > .mash file. mash has no problems with the rpms and it can verify the > > signatures alright. But, it still doesn't embed the signatures in the rpm > > (is it supposed to?). So, the created repository still has all rpms > > unsigned. > > > >What am I missing here? where to the rpms get signed actually? > > The sign_unsigned script should eventually do a koji API call to do > 'write-signed-rpm' on the packages you are signing. That will assemble > signed RPMs in koji itself, which mash will download and used. > > Fedora Rel-Eng doesn't use sign_unsigned anymore because we have a signing > server setup now. However, it should still work. it still works. EPEL releng still uses it. you need to make sure to add -- write-rpms to you command. the signed rpms will then get written. Dennis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From steve.traylen at cern.ch Wed Nov 11 16:11:13 2009 From: steve.traylen at cern.ch (Steve Traylen) Date: Wed, 11 Nov 2009 17:11:13 +0100 Subject: Signing RPMs In-Reply-To: <200911111008.27590.dennis@ausil.us> References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> <20091111131536.GC5699@hansolo.jdub.homelinux.org> <200911111008.27590.dennis@ausil.us> Message-ID: On Wed, Nov 11, 2009 at 5:08 PM, Dennis Gilmore wrote: > On Wednesday 11 November 2009 07:15:36 am Josh Boyer wrote: >> On Tue, Nov 10, 2009 at 11:24:50PM -0800, Jitesh Shah wrote: >> >So, I picked up the sign_unsigned.py script from releng. I replaced the >> > keys in there with our keys, tweaked some minor stuff here and there and >> > managed to get it running. I use it as >> >"./sign_unsigned.py --level " >> >and it runs alright. I can see that the signatures are cached under the >> > sigcache directory (but NOT embedded in the rpms themselves, which makes >> > sense since the rpm can probably be a part of different tags and might be >> > signed differently within each tag) >> > >> >So, I thought, well, mash would be the one which'll embed the keys in the >> > rpms. So, I set strict_keys to True.. added my key to the keys list in my >> > .mash file. mash has no problems with the rpms and it can verify the >> > signatures alright. But, it still doesn't embed the signatures in the rpm >> > (is it supposed to?). So, the created repository still has all rpms >> > unsigned. >> > >> >What am I missing here? where to the rpms get signed actually? >> >> The sign_unsigned script should eventually do a koji API call to do >> 'write-signed-rpm' on the packages you are signing. ?That will assemble >> ?signed RPMs in koji itself, which mash will download and used. >> >> Fedora Rel-Eng doesn't use sign_unsigned anymore because we have a signing >> server setup now. ?However, it should still work. > it still works. EPEL releng still uses it. you need to make sure to add -- > write-rpms to you command. the signed rpms will then get written. > I to have wanted to get this to work. I expect I have my key definition wrong, traceback below. I have, self.gpg_keys = { '89D891FB': { 'name': 'oatrelease', 'description': 'EGEE SA1 (Operations Automation Team) ', } } with $ gpg --list-keys /home/sign/.gnupg/pubring.gpg ----------------------------- pub 1024D/47EBAC2B 2009-11-11 [expires: 2019-11-09] uid EGEE SA1 (Operations Automation Team) sub 2048g/89D891FB 2009-11-11 [expires: 2019-11-09] Traceback (most recent call last): File "./sign_unsigned.py", line 734, in x.run_command() File "./sign_unsigned.py", line 285, in run_command cmd() File "./sign_unsigned.py", line 728, in cmd_default self.sign_to_cache(uncached, self.options.level) File "./sign_unsigned.py", line 638, in sign_to_cache self.do_signing(pkglist, level) File "./sign_unsigned.py", line 601, in do_signing cmd = self.get_signing_command(level, mypaths[:nlen], server=self.options.server) File "./sign_unsigned.py", line 587, in get_signing_command if self.gpg_keys[keyid]['size'] == 4096: KeyError: None > Dennis > > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list > -- Steve Traylen From jiteshs at marvell.com Thu Nov 12 05:20:49 2009 From: jiteshs at marvell.com (Jitesh Shah) Date: Thu, 12 Nov 2009 10:50:49 +0530 Subject: Signing RPMs In-Reply-To: <200911111008.27590.dennis@ausil.us> References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> <20091111131536.GC5699@hansolo.jdub.homelinux.org> <200911111008.27590.dennis@ausil.us> Message-ID: <1258003249.12045.6.camel@localhost.localdomain> ..snip.. > > > > The sign_unsigned script should eventually do a koji API call to do > > 'write-signed-rpm' on the packages you are signing. That will assemble > > signed RPMs in koji itself, which mash will download and used. > > > > Fedora Rel-Eng doesn't use sign_unsigned anymore because we have a signing > > server setup now. However, it should still work. > it still works. EPEL releng still uses it. you need to make sure to add -- > write-rpms to you command. the signed rpms will then get written. Nice! that was what I was missing! The signed rpms are now being written in the 'signed' directory. Thankyou Dennis and Josh. > > Dennis Jitesh From florian.laroche at gmx.net Thu Nov 12 06:40:16 2009 From: florian.laroche at gmx.net (Florian La Roche) Date: Thu, 12 Nov 2009 07:40:16 +0100 Subject: mock-0.9.19 on CentOS5/RHEL5 Message-ID: <20091112064016.GA7908@knorke.jur-linux.org> Hello, if you want to run the newest version of mock (0.9.19) with RHEL5/CentOS5, you can use a backported version from: http://www.jur-linux.org/rpms/el-updates/5/SRPMS/mock-0.9.19-1.el5.src.rpm http://www.jur-linux.org/rpms/el-updates/5/i386/mock-0.9.19-1.el5.noarch.rpm regards, Florian La Roche From jiteshs at marvell.com Thu Nov 12 07:10:44 2009 From: jiteshs at marvell.com (Jitesh Shah) Date: Thu, 12 Nov 2009 12:40:44 +0530 Subject: Signing RPMs In-Reply-To: References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> <20091111131536.GC5699@hansolo.jdub.homelinux.org> <200911111008.27590.dennis@ausil.us> Message-ID: <1258009844.12045.115.camel@localhost.localdomain> ..snip.. > > I to have wanted to get this to work. > > I expect I have my key definition wrong, traceback below. > > I have, > self.gpg_keys = { > '89D891FB': { 'name': 'oatrelease', > 'description': 'EGEE SA1 (Operations > Automation Team) ', > } } > > with > > $ gpg --list-keys > /home/sign/.gnupg/pubring.gpg > ----------------------------- > pub 1024D/47EBAC2B 2009-11-11 [expires: 2019-11-09] > uid EGEE SA1 (Operations Automation Team) > > sub 2048g/89D891FB 2009-11-11 [expires: 2019-11-09] Steve, you are using the subkey. You probably want to use the master signing key i.e. the one listed under "pub" ("47EBAC2B" in your case) Jitesh > > > > > Traceback (most recent call last): > File "./sign_unsigned.py", line 734, in > x.run_command() > File "./sign_unsigned.py", line 285, in run_command > cmd() > File "./sign_unsigned.py", line 728, in cmd_default > self.sign_to_cache(uncached, self.options.level) > File "./sign_unsigned.py", line 638, in sign_to_cache > self.do_signing(pkglist, level) > File "./sign_unsigned.py", line 601, in do_signing > cmd = self.get_signing_command(level, mypaths[:nlen], > server=self.options.server) > File "./sign_unsigned.py", line 587, in get_signing_command > if self.gpg_keys[keyid]['size'] == 4096: > KeyError: None > > > > > > > > > Dennis > > > > -- > > Fedora-buildsys-list mailing list > > Fedora-buildsys-list at redhat.com > > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list > > > > > From steve.traylen at cern.ch Thu Nov 12 08:38:33 2009 From: steve.traylen at cern.ch (Steve Traylen) Date: Thu, 12 Nov 2009 09:38:33 +0100 Subject: Signing RPMs In-Reply-To: <1258009844.12045.115.camel@localhost.localdomain> References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> <20091111131536.GC5699@hansolo.jdub.homelinux.org> <200911111008.27590.dennis@ausil.us> <1258009844.12045.115.camel@localhost.localdomain> Message-ID: On Thu, Nov 12, 2009 at 8:10 AM, Jitesh Shah wrote: > ..snip.. >> >> I to have wanted to get this to work. >> >> I expect I have my key definition wrong, traceback below. >> >> I have, >> ? ? ? ? self.gpg_keys = { >> ? ? ? ? ? ? '89D891FB': { 'name': 'oatrelease', >> ? ? ? ? ? ? ? ? ? ? ? ? ? 'description': 'EGEE SA1 (Operations >> Automation Team) ', >> ? ? ? ? ? ? ? ? ? ? ? ? ? } ? ? ? ? ? ? ? ? ? ? ? ?} >> >> with >> snip > > Steve, you are using the subkey. You probably want to use the master > signing key i.e. the one listed under "pub" ("47EBAC2B" in your case) > Hi Jitesh, Switching to the master key similar error as below. ./sign_unsigned.py --just-show dist-fc10 what is the level option? '47EBAC2B': { 'name': 'oatrelease', 'description': 'EGEE SA1 (Operations Automation Team) ', } Traceback (most recent call last): File "./sign_unsigned.py", line 734, in x.run_command() File "./sign_unsigned.py", line 284, in run_command cmd() File "./sign_unsigned.py", line 728, in cmd_default self.sign_to_cache(uncached, self.options.level) File "./sign_unsigned.py", line 638, in sign_to_cache self.do_signing(pkglist, level) File "./sign_unsigned.py", line 601, in do_signing cmd = self.get_signing_command(level, mypaths[:nlen], server=self.options.server) File "./sign_unsigned.py", line 586, in get_signing_command if self.gpg_keys[keyid]['size'] == 4096: KeyError: None The full edited script is here http://cern.ch/steve.traylen/tmp/oat-sign_unsigned.py is there something else I need to change? > Jitesh > >> >> >> >> >> Traceback (most recent call last): >> ? File "./sign_unsigned.py", line 734, in >> ? ? x.run_command() >> ? File "./sign_unsigned.py", line 285, in run_command >> ? ? cmd() >> ? File "./sign_unsigned.py", line 728, in cmd_default >> ? ? self.sign_to_cache(uncached, self.options.level) >> ? File "./sign_unsigned.py", line 638, in sign_to_cache >> ? ? self.do_signing(pkglist, level) >> ? File "./sign_unsigned.py", line 601, in do_signing >> ? ? cmd = self.get_signing_command(level, mypaths[:nlen], >> server=self.options.server) >> ? File "./sign_unsigned.py", line 587, in get_signing_command >> ? ? if self.gpg_keys[keyid]['size'] == 4096: >> KeyError: None >> >> >> >> >> >> >> >> > Dennis >> > >> > -- >> > Fedora-buildsys-list mailing list >> > Fedora-buildsys-list at redhat.com >> > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list >> > >> >> >> > > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list > -- Steve Traylen From mail-lists at karan.org Thu Nov 12 10:07:38 2009 From: mail-lists at karan.org (Karanbir Singh) Date: Thu, 12 Nov 2009 10:07:38 +0000 Subject: mock-0.9.19 on CentOS5/RHEL5 In-Reply-To: <20091112064016.GA7908@knorke.jur-linux.org> References: <20091112064016.GA7908@knorke.jur-linux.org> Message-ID: <4AFBDE6A.4000408@karan.org> On 11/12/2009 06:40 AM, Florian La Roche wrote: > Hello, > > if you want to run the newest version of mock (0.9.19) > with RHEL5/CentOS5, you can use a backported version from: > Have you done any tests to see how the output changes from what we have in extras/ now ? Are these test results public somewhere ? -- Karanbir Singh London, UK | http://www.karan.org/ | twitter.com/kbsingh ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax GnuPG Key : http://www.karan.org/publickey.asc From jkeating at redhat.com Thu Nov 12 16:30:36 2009 From: jkeating at redhat.com (Jesse Keating) Date: Thu, 12 Nov 2009 08:30:36 -0800 Subject: Signing RPMs In-Reply-To: References: <91A41E78C4F4444BBD1EB4198A7B9C4F5DDC23A2C9@SC-VEXCH2.marvell.com> <20091111131536.GC5699@hansolo.jdub.homelinux.org> <200911111008.27590.dennis@ausil.us> <1258009844.12045.115.camel@localhost.localdomain> Message-ID: <1258043436.2477.19.camel@localhost.localdomain> On Thu, 2009-11-12 at 09:38 +0100, Steve Traylen wrote: > The full edited script is here > > http://cern.ch/steve.traylen/tmp/oat-sign_unsigned.py > > is there something else I need to change? The traceback is looking in the dict of your key for a size, as gpg keys can come in many sizes and shapes, the script acts differently depending on the size of the key. -- Jesse Keating Fedora -- Freedom? is a feature! identi.ca: http://identi.ca/jkeating -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From mikem at redhat.com Fri Nov 13 18:09:14 2009 From: mikem at redhat.com (Mike McLean) Date: Fri, 13 Nov 2009 13:09:14 -0500 Subject: mock-0.9.19 on CentOS5/RHEL5 In-Reply-To: <20091112064016.GA7908@knorke.jur-linux.org> References: <20091112064016.GA7908@knorke.jur-linux.org> Message-ID: <4AFDA0CA.60504@redhat.com> On 11/12/2009 01:40 AM, Florian La Roche wrote: > if you want to run the newest version of mock (0.9.19) > with RHEL5/CentOS5, you can use a backported version from: > > http://www.jur-linux.org/rpms/el-updates/5/SRPMS/mock-0.9.19-1.el5.src.rpm > http://www.jur-linux.org/rpms/el-updates/5/i386/mock-0.9.19-1.el5.noarch.rpm So basically just reverting the dbus patch (2046136e)? Perhaps since RHEL5 is something of an important platform for mock we should consider making this codepath conditional, or otherwise fix this without forking. From williams at redhat.com Sun Nov 15 15:57:46 2009 From: williams at redhat.com (Clark Williams) Date: Sun, 15 Nov 2009 09:57:46 -0600 Subject: mock-0.9.19 on CentOS5/RHEL5 In-Reply-To: <4AFDA0CA.60504@redhat.com> References: <20091112064016.GA7908@knorke.jur-linux.org> <4AFDA0CA.60504@redhat.com> Message-ID: <20091115095746.1587f7ba@torg> On Fri, 13 Nov 2009 13:09:14 -0500 Mike McLean wrote: > On 11/12/2009 01:40 AM, Florian La Roche wrote: > > if you want to run the newest version of mock (0.9.19) > > with RHEL5/CentOS5, you can use a backported version from: > > > > http://www.jur-linux.org/rpms/el-updates/5/SRPMS/mock-0.9.19-1.el5.src.rpm > > http://www.jur-linux.org/rpms/el-updates/5/i386/mock-0.9.19-1.el5.noarch.rpm > > So basically just reverting the dbus patch (2046136e)? > > Perhaps since RHEL5 is something of an important platform for mock we > should consider making this codepath conditional, or otherwise fix this > without forking. > I've got a patch in my tree but I'm trying to figure out why my validation tests are failing on epel4. Should ahve a 0.9.20 out early next week though. Clark -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From florian.laroche at gmx.net Sun Nov 15 16:21:15 2009 From: florian.laroche at gmx.net (Florian La Roche) Date: Sun, 15 Nov 2009 17:21:15 +0100 Subject: mock-0.9.19 on CentOS5/RHEL5 In-Reply-To: <4AFDA0CA.60504@redhat.com> References: <20091112064016.GA7908@knorke.jur-linux.org> <4AFDA0CA.60504@redhat.com> Message-ID: <20091115162115.GA11701@knorke.jur-linux.org> On Fri, Nov 13, 2009 at 01:09:14PM -0500, Mike McLean wrote: > On 11/12/2009 01:40 AM, Florian La Roche wrote: > >if you want to run the newest version of mock (0.9.19) > >with RHEL5/CentOS5, you can use a backported version from: > > > >http://www.jur-linux.org/rpms/el-updates/5/SRPMS/mock-0.9.19-1.el5.src.rpm > >http://www.jur-linux.org/rpms/el-updates/5/i386/mock-0.9.19-1.el5.noarch.rpm > > So basically just reverting the dbus patch (2046136e)? > > Perhaps since RHEL5 is something of an important platform for mock we > should consider making this codepath conditional, or otherwise fix this > without forking. Hello Mike, this small change is all that is needed, at least for a first start. I've put this into bugzilla in August and Clark has done a clean patch within two days, but it is not commited into the public repo and also no new rpm has been pushed out. See bz 516355 for details. Karanbir: You've already emailed the current CentOS position to have one mock version across all of CentOS3/4/5 running and that the newest bits don't qualify for that. This could maybe change next year once CentOS3 does not get updates anymore. I could see similar decisions like e.g. keeping to a mock version starting at initial release of a new major CentOS push-out. (Then CentOS-5 would always stick to the mock release that was used for CentOS-5.0...) Another idea that Red Hat would probably support is to just use the same mock version that also Red Hat uses for their releases, just to have another data point that CentOS is really 100% compatibel to RHEL. (Keeping infrastructure the same should be extended to as many parts as possible.) regards, Florian La Roche From peng.chen22 at gmail.com Wed Nov 18 07:15:46 2009 From: peng.chen22 at gmail.com (peng chen) Date: Wed, 18 Nov 2009 15:15:46 +0800 Subject: Postgresql Database Error Message-ID: <7e200e7f0911172315q4b2fc425p8575bdd7e2b74765@mail.gmail.com> hello, fedora-buildsys-list: when I requset a build task for pakcage "anaconda" to koji, one errie error come out. It detailed as follow: pg.DatabaseError: error ' ERROR: new row for relation "task" violates check constraint "task_weight_check" ' in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' I'm sure that the source rpm of anaconda is OK,because I succed to build it in local mock environment. and the repo is the same as the koji build server. hope you do me a favor sincerely! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeb at redhat.com Wed Nov 18 13:51:54 2009 From: mikeb at redhat.com (Mike Bonnet) Date: Wed, 18 Nov 2009 08:51:54 -0500 Subject: Postgresql Database Error In-Reply-To: <7e200e7f0911172315q4b2fc425p8575bdd7e2b74765@mail.gmail.com> References: <7e200e7f0911172315q4b2fc425p8575bdd7e2b74765@mail.gmail.com> Message-ID: <4B03FBFA.2000001@redhat.com> On 11/18/2009 02:15 AM, peng chen wrote: > hello, fedora-buildsys-list: > > when I requset a build task for pakcage "anaconda" to koji, > one errie error come out. > It detailed as follow: > pg.DatabaseError: error ' ERROR: new row for relation "task" violates > check constraint "task_weight_check" ' > in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' > I'm sure that the source rpm of anaconda is OK,because I succed to build > it in local mock environment. and > the repo is the same as the koji build server. > hope you do me a favor sincerely! Does one of the builds have a completion_time earlier than its create_event time? Koji uses the build duration to dynamically calculate the weight of the task. It should probably be checking for a negative result, but it doesn't. From mikem at redhat.com Wed Nov 18 13:56:59 2009 From: mikem at redhat.com (Mike McLean) Date: Wed, 18 Nov 2009 08:56:59 -0500 Subject: Postgresql Database Error In-Reply-To: <7e200e7f0911172315q4b2fc425p8575bdd7e2b74765@mail.gmail.com> References: <7e200e7f0911172315q4b2fc425p8575bdd7e2b74765@mail.gmail.com> Message-ID: <4B03FD2B.5040909@redhat.com> On 11/18/2009 02:15 AM, peng chen wrote: > hello, fedora-buildsys-list: > > when I requset a build task for pakcage "anaconda" to koji, > one errie error come out. > It detailed as follow: > pg.DatabaseError: error ' ERROR: new row for relation "task" violates > check constraint "task_weight_check" ' > in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' > I'm sure that the source rpm of anaconda is OK,because I succed to build > it in local mock environment. and > the repo is the same as the koji build server. > hope you do me a favor sincerely! The system time on your hub must have been set back during an anaconda build and there must be sufficiently few anaconda builds for this to cause getAverageBuildDuration('anaconda') to return a negative number. We should of course fix this, but you should also avoid rolling back the time on your koji hosts, especially the hub and db hosts. In the meantime, this patch should help --- a/builder/kojid +++ b/builder/kojid @@ -2033,6 +2033,9 @@ class BuildArchTask(BaseTaskHandler): avg = session.getAverageBuildDuration(name) if not avg: return + if avg < 0: + self.logger.warn("Negative average build duration for %s: %s", name, avg) + return # increase the task weight by 0.75 for every hour of build duration adj = (avg / 4800.0) # cap the adjustment at +4.5 From peng.chen22 at gmail.com Thu Nov 19 05:32:52 2009 From: peng.chen22 at gmail.com (peng chen) Date: Thu, 19 Nov 2009 13:32:52 +0800 Subject: Postgresql Database Error Message-ID: <7e200e7f0911182132r583797b2i9f16fa24a76c3489@mail.gmail.com> 2009/11/19 > Send Fedora-buildsys-list mailing list submissions to > fedora-buildsys-list at redhat.com > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list > or, via email, send a message with subject or body 'help' to > fedora-buildsys-list-request at redhat.com > > You can reach the person managing the list at > fedora-buildsys-list-owner at redhat.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Fedora-buildsys-list digest..." > > > Today's Topics: > > 1. Postgresql Database Error (peng chen) > 2. Re: Postgresql Database Error (Mike Bonnet) > 3. Re: Postgresql Database Error (Mike McLean) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 18 Nov 2009 15:15:46 +0800 > From: peng chen > Subject: Postgresql Database Error > To: fedora-buildsys-list at redhat.com > Message-ID: > <7e200e7f0911172315q4b2fc425p8575bdd7e2b74765 at mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > hello, fedora-buildsys-list: > > when I requset a build task for pakcage "anaconda" to koji, > one errie error come out. > It detailed as follow: > pg.DatabaseError: error ' ERROR: new row for relation "task" violates > check constraint "task_weight_check" ' > in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' > I'm sure that the source rpm of anaconda is OK,because I succed to build > it in local mock environment. and > the repo is the same as the koji build server. > hope you do me a favor sincerely! > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > https://www.redhat.com/archives/fedora-buildsys-list/attachments/20091118/b463b957/attachment.html > > ------------------------------ > > Message: 2 > Date: Wed, 18 Nov 2009 08:51:54 -0500 > From: Mike Bonnet > Subject: Re: Postgresql Database Error > To: fedora-buildsys-list at redhat.com > Message-ID: <4B03FBFA.2000001 at redhat.com> > Content-Type: text/plain; charset=UTF-8 > > On 11/18/2009 02:15 AM, peng chen wrote: > > hello, fedora-buildsys-list: > > > > when I requset a build task for pakcage "anaconda" to koji, > > one errie error come out. > > It detailed as follow: > > pg.DatabaseError: error ' ERROR: new row for relation "task" violates > > check constraint "task_weight_check" ' > > in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' > > I'm sure that the source rpm of anaconda is OK,because I succed to > build > > it in local mock environment. and > > the repo is the same as the koji build server. > > hope you do me a favor sincerely! > > Does one of the builds have a completion_time earlier than its > create_event time? Koji uses the build duration to dynamically > calculate the weight of the task. It should probably be checking for a > negative result, but it doesn't. > > > > ------------------------------ > > Message: 3 > Date: Wed, 18 Nov 2009 08:56:59 -0500 > From: Mike McLean > Subject: Re: Postgresql Database Error > To: fedora-buildsys-list at redhat.com > Message-ID: <4B03FD2B.5040909 at redhat.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 11/18/2009 02:15 AM, peng chen wrote: > > hello, fedora-buildsys-list: > > > > when I requset a build task for pakcage "anaconda" to koji, > > one errie error come out. > > It detailed as follow: > > pg.DatabaseError: error ' ERROR: new row for relation "task" > violates > > check constraint "task_weight_check" ' > > in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' > > I'm sure that the source rpm of anaconda is OK,because I succed to > build > > it in local mock environment. and > > the repo is the same as the koji build server. > > hope you do me a favor sincerely! > > The system time on your hub must have been set back during an anaconda > build and there must be sufficiently few anaconda builds for this to > cause getAverageBuildDuration('anaconda') to return a negative number. > > We should of course fix this, but you should also avoid rolling back the > time on your koji hosts, especially the hub and db hosts. > > In the meantime, this patch should help > --- a/builder/kojid > +++ b/builder/kojid > @@ -2033,6 +2033,9 @@ class BuildArchTask(BaseTaskHandler): > avg = session.getAverageBuildDuration(name) > if not avg: > return > + if avg < 0: > + self.logger.warn("Negative average build duration for %s: > %s", name, avg) > + return > # increase the task weight by 0.75 for every hour of build > duration > adj = (avg / 4800.0) > # cap the adjustment at +4.5 > > > > ------------------------------ > As Mike Bonnet said in message 2, one of anaconda builds had a > completion_time earlier than creation_time, so I find the one and remember > the build ID, then update the table "build" to make the completion_time > later than creation_time. > Now the problem was solved. Thanks again! > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list > > End of Fedora-buildsys-list Digest, Vol 57, Issue 5 > *************************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at city-fan.org Thu Nov 19 16:06:14 2009 From: paul at city-fan.org (Paul Howarth) Date: Thu, 19 Nov 2009 16:06:14 +0000 Subject: Intermittent errors creating mock root cache tarball Message-ID: <4B056CF6.2070204@city-fan.org> I have a buildsystem that targets a number of different distribution releases, and so I get to rebuild a root cache quite often. Quite frequently, the creation of the root cache tarball fails and causes the package build that triggered the root cache creation to fail. However, simply repeating the build invariably succeeds, and mock uses the supposedly failed cache tarball from the previous build without problems. I've not looked at this in detail because the workaround has been so easy but yesterday I decided to take a look at it. I think there are two issues. Firstly, the cause of the tarball creation failure. Looking at the root log, it appeared to be a change in one of the files whilst it was being archived by tar. DEBUG util.py, Line: 234: tar: ./usr/lib/locale/locale-archive: file changed as we read it The same problem with the same file was menioned in a report dating back two years on fedora-devel-list: http://www.redhat.com/archives/fedora-devel-list/2007-November/msg02599.html More googling revealed a possible cause of the problem: http://www.mail-archive.com/linux-kernel at vger.kernel.org/msg190963.html So I tried forcing a "sync" before creating the tarball and lo and behold, the problem went away. I've created at least 20 root caches since making this change and all worked fine, which I'm very confident wouldn't have been the case without the "sync". So here's the change I made: --- /usr/lib/python2.6/site-packages/mock/plugins/root_cache.py.orig 2009-09-02 19:08:54.000000000 +0100 +++ /usr/lib/python2.6/site-packages/mock/plugins/root_cache.py 2009-11-18 15:20:04.353035160 +0000 @@ -110,6 +110,7 @@ # never rebuild cache unless it was a clean build. if self.rootObj.chrootWasCleaned: self.state("creating cache") + mock.util.do(["sync"], shell=False) mock.util.do( ["tar"] + self.compressArgs + ["-cf", self.rootCacheFile, "-C", self.rootObj.makeChrootPath(), "."], The second problem is I think that if the "tar" process to create the tarball fails (and hence causes the resulting build to fail), the cache should be invalidated so that the next build doesn't use that presumably-broken tarball. As it happens, a faulty copy of /usr/lib/locale/locale-archive doesn't seem to cause any problems during my builds but that may just be my good fortune. Cheers, Paul. From peng.chen22 at gmail.com Fri Nov 20 01:28:52 2009 From: peng.chen22 at gmail.com (peng chen) Date: Fri, 20 Nov 2009 09:28:52 +0800 Subject: How the pkglist of the repo associated with build tag is generated Message-ID: <7e200e7f0911191728p318e6aa9ia32bbdf3c23c0453@mail.gmail.com> for example,I have a target dist-test, it's detailed info as follow: Name Buildroot Destination ---------------------------------------------------------------------- dist-test dist-test-build dist-test I use command "koji list-tagged dist-test",and found the package in the list of dist-test,but when generating the repo associated with the build tag 'dist-test-build',the package doesn't exist in the file 'pkglist'.This results in Missing Dependency Error or Init mock buildroot Error. I reslove the problem this way that tag the missing package with command "koji call tagBuildBypass dist-test-build " ,and then regen-repo,It's OK! So, I wonder why and how the content of file 'pkglist' is written? Thanks, sincerely! -------------- next part -------------- An HTML attachment was scrubbed... URL: From dennis at ausil.us Fri Nov 20 14:31:05 2009 From: dennis at ausil.us (Dennis Gilmore) Date: Fri, 20 Nov 2009 08:31:05 -0600 Subject: How the pkglist of the repo associated with build tag is generated In-Reply-To: <7e200e7f0911191728p318e6aa9ia32bbdf3c23c0453@mail.gmail.com> References: <7e200e7f0911191728p318e6aa9ia32bbdf3c23c0453@mail.gmail.com> Message-ID: <200911200831.13564.dennis@ausil.us> On Thursday 19 November 2009 07:28:52 pm peng chen wrote: > for example,I have a target dist-test, it's detailed info as follow: > > Name Buildroot Destination > ---------------------------------------------------------------------- > dist-test dist-test-build dist-test > > I use command "koji list-tagged dist-test",and found the package in the > list of dist-test,but when generating the repo associated with the build > tag 'dist-test-build',the package doesn't exist in the file > 'pkglist'.This results in Missing Dependency Error or Init mock > buildroot Error. > > I reslove the problem this way that tag the missing > package with command "koji call tagBuildBypass dist-test-build missing package n-v-r>" ,and then regen-repo,It's OK! > So, I wonder why and how the content of file 'pkglist' is written? > > Thanks, sincerely! > does dist-test-build inherit from dist-test ? im guessing not koji list-tag-inheritance dist-f12-build dist-f12-build (86) ??dist-f12-override (113) ? ??dist-f12-updates (105) ? ??dist-f12 (85) ? ??dist-f11-updates (87) ? ??dist-f11 (62) ? ??dist-f10-updates (64) ? ??dist-f10 (45) ? ??f9-cutoff (110) ??dist-f11-build (63) ??dist-f11-override (89) ??dist-f10-build (46) ??dist-f10-override (68) ??f9-build-cutoff (111) in fedoras koji koji add-tag-inheritance dist-test-build dist-test should set you up you may need to use --priority and specify one if you have other tags already inherited. Dennis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From florian.laroche at gmx.net Sun Nov 22 21:49:10 2009 From: florian.laroche at gmx.net (Florian La Roche) Date: Sun, 22 Nov 2009 22:49:10 +0100 Subject: mock-0.9.19 on CentOS5/RHEL5 In-Reply-To: <4AFDA0CA.60504@redhat.com> References: <20091112064016.GA7908@knorke.jur-linux.org> <4AFDA0CA.60504@redhat.com> Message-ID: <20091122214910.GA26933@knorke.jur-linux.org> On Fri, Nov 13, 2009 at 01:09:14PM -0500, Mike McLean wrote: > Perhaps since RHEL5 is something of an important platform for mock we > should consider making this codepath conditional, or otherwise fix this > without forking. Hello Mike, thanks for getting the new release of mock out, also for the pykickstart commit to keep working with older python releases. I've moved to newest rpms and besides mock I've also tested koji a bit more now. Some recompile tests with kernel,glibc,memtest86+ and a dozen other rpms did all work ok, so the special koji config steps should be complete now... (Also Michal Hlavinka has merged in some bits for dovecot to have newest Fedora Rawhide rpms compile on RHEL5, so overall it's been a good week to reduce extra patches for me.) thanks, Florian La Roche From peng.chen22 at gmail.com Mon Nov 23 01:30:17 2009 From: peng.chen22 at gmail.com (peng chen) Date: Mon, 23 Nov 2009 09:30:17 +0800 Subject: About priority on tag-inheritance Message-ID: <7e200e7f0911221730j670b0ec1v4e57dbfb59839ca6@mail.gmail.com> I wonder how the priority on tag-inheritance works. whether the priority number which is large has priority, or not? In other words, the larger the priority number is ,the priorer . is it like this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dennis at ausil.us Mon Nov 23 05:58:18 2009 From: dennis at ausil.us (Dennis Gilmore) Date: Sun, 22 Nov 2009 23:58:18 -0600 Subject: About priority on tag-inheritance In-Reply-To: <7e200e7f0911221730j670b0ec1v4e57dbfb59839ca6@mail.gmail.com> References: <7e200e7f0911221730j670b0ec1v4e57dbfb59839ca6@mail.gmail.com> Message-ID: <200911222358.30329.dennis@ausil.us> On Sunday 22 November 2009 07:30:17 pm peng chen wrote: > I wonder how the priority on tag-inheritance works. whether the priority > number which is large has priority, or not? > In other words, the larger the priority number is ,the priorer . is it like > this? > the smaller the number the higher priority 0 is higest followed by 1 etc Dennis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From mikem at redhat.com Mon Nov 23 20:12:12 2009 From: mikem at redhat.com (Mike McLean) Date: Mon, 23 Nov 2009 15:12:12 -0500 Subject: About priority on tag-inheritance In-Reply-To: <200911222358.30329.dennis@ausil.us> References: <7e200e7f0911221730j670b0ec1v4e57dbfb59839ca6@mail.gmail.com> <200911222358.30329.dennis@ausil.us> Message-ID: <4B0AEC9C.7020608@redhat.com> On 11/23/2009 12:58 AM, Dennis Gilmore wrote: > On Sunday 22 November 2009 07:30:17 pm peng chen wrote: >> I wonder how the priority on tag-inheritance works. whether the priority >> number which is large has priority, or not? >> In other words, the larger the priority number is ,the priorer . is it like >> this? >> > the smaller the number the higher priority 0 is higest followed by 1 etc yes, ala *nix process priority. Also like English usage (e.g. 1st priority, 2nd priority, ...) Note that the other priority values in Koji (task priority, external repo priority) work the same way. From peng.chen22 at gmail.com Tue Nov 24 06:19:07 2009 From: peng.chen22 at gmail.com (peng chen) Date: Tue, 24 Nov 2009 14:19:07 +0800 Subject: how to cancel (stop) the build task which has strange state Message-ID: <7e200e7f0911232219x1115633cyc8ea8565aef39e8b@mail.gmail.com> hello, fedora-buildsys-list: I encounter with several strange build tasks, which build state is building,but task state is failed. resultly, these build tasks are always in the building state. I run command "koji cancel " ,but failed to cancel it ,it's still there(my koji website). So, how could I cancel (stop ) these strange build tasks. In addition, during these package building, the koji server and building host had poweroffed. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikem at redhat.com Tue Nov 24 13:47:55 2009 From: mikem at redhat.com (Mike McLean) Date: Tue, 24 Nov 2009 08:47:55 -0500 Subject: how to cancel (stop) the build task which has strange state In-Reply-To: <7e200e7f0911232219x1115633cyc8ea8565aef39e8b@mail.gmail.com> References: <7e200e7f0911232219x1115633cyc8ea8565aef39e8b@mail.gmail.com> Message-ID: <4B0BE40B.1020907@redhat.com> On 11/24/2009 01:19 AM, peng chen wrote: > hello, fedora-buildsys-list: > I encounter with several strange build tasks, which build state is > building,but task state is failed. > resultly, these build tasks are always in the building state. I run command > "koji cancel " ,but failed to > cancel it ,it's still there(my koji website). So, how could I cancel (stop > ) these strange build tasks. > In addition, during these package building, the koji server and building > host had poweroffed. > Thanks! Answered here: https://www.redhat.com/archives/fedora-buildsys-list/2008-December/msg00000.html From tibbs at math.uh.edu Tue Nov 24 18:54:14 2009 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Tue, 24 Nov 2009 12:54:14 -0600 Subject: Weird problem building zsh in local mock but not in koji Message-ID: I'm having a problem building the current zsh srpm (zsh-4.3.10-4.fc13.src.rpm) on my local builder, which runs F11-x86_64 and has mock-0.9.18-1.fc11.noarch. On IRC, a user reported the same problem, only his builder runs CentOS 5.4 and has mock-0.9.14-2.el5.noarch. Surprisingly, the same srpm will build fine in koji. I'm not sure how to describe the hang in enough detail without having to understand the zsh test framework. The bottom line is simply that one of the tests just hangs. It's doing some testing of the read builtin; I believe it's one of these two stanzas which hangs, although output buffering may make it difficult to be sure: read -d: <<foo print foo:bar|IFS=: read -A print $reply 0:use different, IFS separator to array >foo bar The last thing in the build log is "Running test: read up to delimiter". ps just shows: tibbs 9322 0.0 0.0 114684 1916 pts/3 TN 12:39 00:00:00 ../Src/zsh +Z -f ./ztst.zsh ./B04read.ztst wchan is signal_stop. The process doesn't seem to be consuming any CPU, but if I strace it, I get an endless stream of ioctl(11, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig -icanon echo...}) = ? ERESTARTSYS (To be restarted) --- SIGTTOU (Stopped (tty output)) @ 0 (0) --- --- SIGTTOU (Stopped (tty output)) @ 0 (0) --- In koji there doesn't seem to be any delay at all running this test. I'm stumped. - J< From roland at redhat.com Tue Nov 24 22:17:22 2009 From: roland at redhat.com (Roland McGrath) Date: Tue, 24 Nov 2009 14:17:22 -0800 (PST) Subject: Weird problem building zsh in local mock but not in koji In-Reply-To: Jason L Tibbitts III's message of Tuesday, 24 November 2009 12:54:14 -0600 References: Message-ID: <20091124221722.EF31FD37F@magilla.sf.frob.com> That is job control weirdness. Something about the state of tty magic is different in the two different contexts where you run the test. The things to look at are whether the tests are using a temporary pty and what 'ps j' says about all the processes involved in the build/test (including mock and its layers of whatever before the rpmbuild). From tibbs at math.uh.edu Tue Nov 24 22:56:13 2009 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Tue, 24 Nov 2009 16:56:13 -0600 Subject: Weird problem building zsh in local mock but not in koji In-Reply-To: <20091124221722.EF31FD37F@magilla.sf.frob.com> (Roland McGrath's message of "Tue, 24 Nov 2009 14:17:22 -0800 (PST)") References: <20091124221722.EF31FD37F@magilla.sf.frob.com> Message-ID: >>>>> "RM" == Roland McGrath writes: RM> That is job control weirdness. Something about the state of tty RM> magic is different in the two different contexts where you run the RM> test. Unfortunately that level of magic is mostly beyond me. RM> The things to look at are whether the tests are using a RM> temporary pty I do not really know how to determine that. However, it simplifies things to know that if there's any difference it's not in the spec but in the environment in which mock is called. RM> and what 'ps j' says about all the processes involved RM> in the build/test (including mock and its layers of whatever before RM> the rpmbuild). Here's the forest from px axjf (sorry for long lines): 23843 10479 10479 10479 ? -1 SNs 0 0:00 \_ sshd: tibbs [priv] 10479 10486 10479 10479 ? -1 SN 7225 0:00 | \_ sshd: tibbs at pts/3 10486 10487 10487 10487 pts/3 10527 SNs 7225 0:00 | \_ -zsh 10487 10527 10527 10487 pts/3 10527 SN+ 7225 0:00 | \_ /bin/bash /home/tibbs/bin/dobuild /home/tibbs/work/extras-cvs/zsh/devel/zsh-4.3.10-4.fc13.src.rpm 10527 10530 10527 10487 pts/3 10527 SN+ 7225 0:02 | \_ /usr/bin/python -tt /usr/sbin/mock -r fedora-rawhide-x86_64 -v --rebuild /home/tibbs/work/extras-cvs/zsh/devel/zsh-4.3.10-4.fc13.src.rpm 10530 13230 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/zsh.spec 13230 28410 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ /bin/sh -e /var/tmp/rpm-tmp.5VZZeU 28410 28414 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ make test 28414 28415 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ /bin/sh -c cd Test ; make check 28415 28416 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ make check 28416 28505 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ /bin/sh -c if ZTST_testlist="`for f in ./*.ztst; \? do echo $f; done`" \? ZTST_srcdir="." \? ZTST_exe=../Src/zsh \? ../Src/zsh +Z -f ./runtests.zsh; then \? sta 28505 28507 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ ../Src/zsh +Z -f ./runtests.zsh 28507 29888 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ ../Src/zsh +Z -f ./ztst.zsh ./B04read.ztst Is any of that remotely helpful? On the buildsys, a mock build (not for zsh, but they should all start the same) looks sort of like this: 1 3056 3055 3055 ? -1 S 0 391:44 /usr/bin/python /usr/sbin/kojid --force-lock --verbose 3056 17091 17091 3055 ? -1 S 0 0:07 \_ /usr/bin/python /usr/sbin/kojid --force-lock --verbose 17091 17272 17091 3055 ? -1 S 101 0:01 \_ /usr/bin/python -tt /usr/sbin/mock -r koji/dist-f13-build-653289-93821 --no-clean --target x86_64 --rebuild /mnt/koji/work/tasks/8377/1828377/kvirc-4.0.0-0.19.rc1.fc13.src.rpm 17272 25386 25386 3055 ? -1 S 101 0:00 \_ rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/kvirc.spec 25386 25413 25386 3055 ? -1 S 101 0:00 \_ /bin/sh -e /var/tmp/rpm-tmp.vzHree 25413 25917 25386 3055 ? -1 S 101 0:00 \_ make -j4 and so on. There are some obvious differences. - J< From roland at redhat.com Wed Nov 25 00:55:59 2009 From: roland at redhat.com (Roland McGrath) Date: Tue, 24 Nov 2009 16:55:59 -0800 (PST) Subject: Weird problem building zsh in local mock but not in koji In-Reply-To: Jason L Tibbitts III's message of Tuesday, 24 November 2009 16:56:13 -0600 References: <20091124221722.EF31FD37F@magilla.sf.frob.com> Message-ID: <20091125005559.A41BDD38F@magilla.sf.frob.com> > Unfortunately that level of magic is mostly beyond me. That's OK. We can translate for you. > RM> The things to look at are whether the tests are using a > RM> temporary pty > > I do not really know how to determine that. The TTY column in ps is a start. > However, it simplifies things to know that if there's any difference it's > not in the spec but in the environment in which mock is called. IMHO it is the fault of the package's test suite that it cares the ambient environment where 'make check' is run. If it's going to be affected by the tty state of the caller of "make", then it should run those tests inside a temporary pty (i.e. use expect or script or something). However, I also think that we should endeavor to make the multiple recommended ways of building Fedora rpms (i.e. "use koji" and "run mock yourself") run their builds in a consistent environment across all the recommended methods. > RM> and what 'ps j' says about all the processes involved > RM> in the build/test (including mock and its layers of whatever before > RM> the rpmbuild). > > Here's the forest from px axjf (sorry for long lines): > PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND > 23843 10479 10479 10479 ? -1 SNs 0 0:00 \_ sshd: tibbs [priv] > 10479 10486 10479 10479 ? -1 SN 7225 0:00 | \_ sshd: tibbs at pts/3 > 10486 10487 10487 10487 pts/3 10527 SNs 7225 0:00 | \_ -zsh > 10487 10527 10527 10487 pts/3 10527 SN+ 7225 0:00 | \_ /bin/bash /home/tibbs/bin/dobuild /home/tibbs/work/extras-cvs/zsh/devel/zsh-4.3.10-4.fc13.src.rpm > 10527 10530 10527 10487 pts/3 10527 SN+ 7225 0:02 | \_ /usr/bin/python -tt /usr/sbin/mock -r fedora-rawhide-x86_64 -v --rebuild /home/tibbs/work/extras-cvs/zsh/devel/zsh-4.3.10-4.fc13.src.rpm This shows mock running on your normal tty (TTY matches your command-line shell) in the foreground (PGID==TPGID means "foreground", and PGID!=TPGID, means "background"), as you would expect. > 10530 13230 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/zsh.spec This shows mock runs rpmbuild in the same session and in the background. This is the same is if you had done: $ rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/zsh.spec ^Z (before the test suite) [1]+ Stopped rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/zsh.spec $ bg [1]+ rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/zsh.spec & $ I think if you try it that way you will see the same problem. I suspect that if you run it in the foreground, you won't have any problem. PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND > 13230 28410 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ /bin/sh -e /var/tmp/rpm-tmp.5VZZeU > 28410 28414 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ make test > 28414 28415 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ /bin/sh -c cd Test ; make check > 28415 28416 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ make check > 28416 28505 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ /bin/sh -c if ZTST_testlist="`for f in ./*.ztst; \? do echo $f; done`" \? ZTST_srcdir="." \? ZTST_exe=../Src/zsh \? ../Src/zsh +Z -f ./runtests.zsh; then \? sta > 28505 28507 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ ../Src/zsh +Z -f ./runtests.zsh > 28507 29888 13230 10487 pts/3 10527 TN 7225 0:00 | | \_ ../Src/zsh +Z -f ./ztst.zsh ./B04read.ztst This all shows that rpmbuild on down did no job control fiddling, so it's all in the rpmbuild "job" (PGID == PID of rpmbuild). > Is any of that remotely helpful? On the buildsys, a mock build (not for > zsh, but they should all start the same) looks sort of like this: Yes, this is the information we needed to explain what you are seeing. PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND > 1 3056 3055 3055 ? -1 S 0 391:44 /usr/bin/python /usr/sbin/kojid --force-lock --verbose > 3056 17091 17091 3055 ? -1 S 0 0:07 \_ /usr/bin/python /usr/sbin/kojid --force-lock --verbose > 17091 17272 17091 3055 ? -1 S 101 0:01 \_ /usr/bin/python -tt /usr/sbin/mock -r koji/dist-f13-build-653289-93821 --no-clean --target x86_64 --rebuild /mnt/koji/work/tasks/8377/1828377/kvirc-4.0.0-0.19.rc1.fc13.src.rpm > 17272 25386 25386 3055 ? -1 S 101 0:00 \_ rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/kvirc.spec > 25386 25413 25386 3055 ? -1 S 101 0:00 \_ /bin/sh -e /var/tmp/rpm-tmp.vzHree > 25413 25917 25386 3055 ? -1 S 101 0:00 \_ make -j4 This shows that kojid runs mock in an orphan session (SID == PID of somebody dead) with no controlling terminal (TTY/TPGID shows ?/-1). From mock on down behaves the same, i.e. putting rpmbuild in its own process group. But with no tty, this behaves differently than being in a background process group. Off hand I think that mock should run rpmbuild either in a temporary pty/session or with no tty at all. The no tty option requires mock to catch the SIGHUP when its own tty goes away and know to go kill the rpmbuild that won't get its own SIGHUP, or else being in the middle of a mock command when your session dies (ssh drop, modem drop, terminal window killed, etc.) will leave the build still running. The temporary pty option seems better to me off hand. Then the whole rpmbuild would be in the foreground on that pty, so the bonehead packages like zsh won't be upset. Thanks, Roland From peng.chen22 at gmail.com Wed Nov 25 02:52:18 2009 From: peng.chen22 at gmail.com (peng chen) Date: Wed, 25 Nov 2009 10:52:18 +0800 Subject: everytime serveral packages built, then the package build tasks left alawys open or free Message-ID: <7e200e7f0911241852t5c777e83i44429031df9a2a56@mail.gmail.com> Recently , I encountered with a problem of building as follow : When I request 4 or 5 packages for building at a time , my buildsys run normally. but when I request 20 (for example), It just builds 4 or 5 of them and then the left always stay in open or free state. then, I cancel those then resubmit 4 or 5 packages of them , they are built normally and quickly . Additionally, I succeed to build hundreds of packages at a time before. Thus, I guess whether the koji-hub stop scheduling or build hosts outage? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rotru at br.ibm.com Wed Nov 25 13:41:45 2009 From: rotru at br.ibm.com (rotru at br.ibm.com) Date: Wed, 25 Nov 2009 11:41:45 -0200 Subject: Mergerepos command error: Cannot allocate memory/ RPMDB open failed Message-ID: Guy, I have had the following problem frequently. I'm not sure if I have a memory problem or a rmp db problem. Has anyone ever seen this ? I'm running koji on a Fedora 11, with createrepo-0.9.7-7.fc11.noarch yum-3.2.23-3.fc11.noarch rpm-4.7.1-3.fc11.x86_64 $ /usr/libexec/kojid/mergerepos -a i386 -b /mnt/koji/repos/oc-fedora12-build/742/i386/blocklist -o /tmp/koji/tasks/4653/4653/repo -g /mnt/koji/repos/oc-fedora12-build/742/groups/comps.xml -r file:///tmp/koji/tasks/4653/4653/repo_742_premerge/ -r http://c4ebdaily.ltc.br.ibm.com/fedora/releases/12/Everything/i386/os/ -r http://c4ebdaily.ltc.br.ibm.com/fedora/updates/12/i386/ error: db4 error(12) from dbenv->open: Cannot allocate memory error: db4 error(12) from dbenv->close: Cannot allocate memory error: cannot open Packages index using db3 - Cannot allocate memory (12) error: cannot open Packages database in /var/lib/rpm Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 241, in main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 232, in main merge = RepoMerge(opts.repos, opts.arches, opts.groupfile, blocked, opts.outputdir) File "/usr/libexec/kojid/mergerepos", line 106, in __init__ self.yumbase.conf.cachedir = tempfile.mkdtemp() File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 652, in conf = property(fget=lambda self: self._getConfig(), File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 239, in _getConfig self._conf = config.readMainConfig(startupconf) File "/usr/lib/python2.6/site-packages/yum/config.py", line 794, in readMainConfig yumvars['releasever'] = _getsysver(startupconf.installroot, startupconf.distroverpkg) File "/usr/lib/python2.6/site-packages/yum/config.py", line 867, in _getsysver idx = ts.dbMatch('provides', distroverpkg) TypeError: rpmdb open failed Exception AttributeError: "'YumBase' object has no attribute 'preconf'" in > ignored Exception AttributeError: "'YumBase' object has no attribute 'preconf'" in > ignored Regards Rodrigo Trujillo -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikem at redhat.com Wed Nov 25 16:13:38 2009 From: mikem at redhat.com (Mike McLean) Date: Wed, 25 Nov 2009 11:13:38 -0500 Subject: everytime serveral packages built, then the package build tasks left alawys open or free In-Reply-To: <7e200e7f0911241852t5c777e83i44429031df9a2a56@mail.gmail.com> References: <7e200e7f0911241852t5c777e83i44429031df9a2a56@mail.gmail.com> Message-ID: <4B0D57B2.6020402@redhat.com> On 11/24/2009 09:52 PM, peng chen wrote: > Recently , I encountered with a problem of building as follow : > When I request 4 or 5 packages for building at a time , my buildsys run > normally. > but when I request 20 (for example), It just builds 4 or 5 of them and then > the left > always stay in open or free state. then, I cancel those then resubmit 4 or > 5 packages > of them , they are built normally and quickly . > Additionally, I succeed to build hundreds of packages at a time before. > Thus, I guess whether the koji-hub stop scheduling or build hosts outage? That is unexpected. How long did the system sit idle with free jobs? You said 'open or free.' An open task is /still working/. Are you sure your builders weren't just at capacity? Did any tasks actually complete? Anyway, should this recur, - check that the kojid instances were running - check the kojid logs for errors - check the hub logs (/var/log/httpd/error_log) for errors From tim at richert.me Fri Nov 27 12:06:40 2009 From: tim at richert.me (Tim Richert) Date: Fri, 27 Nov 2009 13:06:40 +0100 Subject: php.spec -> "configure" needs to be removed before "buildconf" is called?! Message-ID: <4B0FC0D0.2010804@richert.me> I've tried building current PHP (5.3.1) with suhosin-patch. This fails with > checking for libedit readline replacement... yes > configure: error: Please reinstall libedit - I cannot find readline.h I figured out that the php-5.3.0-libedit.patch patches config.m4 and changes {/usr/local,/usr}/include/readline/readline.h into {/usr/local,/usr}/include/editline/readline.h The suhosin-patch patches ./configure directly but seems to also patch the according m4-files. If I've got i right > ./buildconf --force is called during the build to recreate ./configure to apply the patches made to the m4-files. It seems as if ./configure is only being recreated for sure, if it doesn't exist when buildconf is called. Can someone confirm that and if so wouldn't the spec-file then needed to be adjusted? If you delete ./configure before calling ./buildconf PHP builds fine with the suhosin-patch. I have changed the following > # Regenerate configure scripts (patches change config.m4's) > ./buildconf --force into > # Regenerate configure scripts (patches change config.m4's) > rm configure > ./buildconf --force in the spec-file. Can someone confirm that this wouldn't affect anything else and could so be applied to the current spec-file? Thanks! Greets, Tim