From jwboyer at jdub.homelinux.org Tue Aug 2 00:28:57 2005 From: jwboyer at jdub.homelinux.org (Josh Boyer) Date: Mon, 01 Aug 2005 19:28:57 -0500 Subject: [PATCH] Cleanup help message in plague-client Message-ID: <1122942537.4755.28.camel@yoda.jdub.homelinux.org> Hi all, This patch cleans up the Usage message in plague-client by listing all of the available commands and moving them into a function. It also adds a help command, which I think was asked for by Jef. thx, josh Index: extras-buildsys/client/client.py =================================================================== --- extras-buildsys.orig/client/client.py +++ extras-buildsys/client/client.py @@ -300,22 +300,27 @@ def getEmailAddress(): return config_email - +def Usage(): + print "Usage:\npackage-client.py \n" + print " is one of:" + print " build [package_name] [cvs_tag | srpm_path] [target]" + print " list" + print " list " + print " list " + print " kill " + print " update_builders" + print " list_builders" + print " pause" + print " unpause" + print " is_paused" + print " requeue " + print " detail " + print " help" + print "" if __name__ == '__main__': if len(sys.argv) < 2: - print "Usage:\npackage-builder.py \n" - print " is one of:" - print " build [package_name] [cvs_tag | srpm_path] [target]" - print " list" - print " list " - print " list " - print " kill " - print " update_builders" - print " list_builders" - print " pause" - print " unpause" - print "" + Usage() sys.exit(1) # Write out config file if it doesn't exist @@ -365,7 +370,11 @@ if __name__ == '__main__': pause(server, False) elif cmd == 'is_paused': is_paused(server) + elif cmd == 'help': + Usage() + sys.exit(1) else: print "Unknown command." + Usage() sys.exit(1) From jwboyer at jdub.homelinux.org Tue Aug 2 13:50:41 2005 From: jwboyer at jdub.homelinux.org (Josh Boyer) Date: Tue, 02 Aug 2005 08:50:41 -0500 Subject: [PATCH] Cleanup help message in plague-client In-Reply-To: <1122942537.4755.28.camel@yoda.jdub.homelinux.org> References: <1122942537.4755.28.camel@yoda.jdub.homelinux.org> Message-ID: <1122990641.4755.31.camel@yoda.jdub.homelinux.org> On Mon, 2005-08-01 at 19:28 -0500, Josh Boyer wrote: > Hi all, > > This patch cleans up the Usage message in plague-client by listing all > of the available commands and moving them into a function. It also adds > a help command, which I think was asked for by Jef. Here's an updated patch. Dan adds commands too quickly ;). It also fixes the name of the program in Usage from 'package-client' to 'plague-client'. josh Index: extras-buildsys/client/client.py =================================================================== --- extras-buildsys.orig/client/client.py +++ extras-buildsys/client/client.py @@ -313,22 +313,28 @@ def getEmailAddress(): return config_email - +def Usage(): + print "Usage:\nplague-client.py \n" + print " is one of:" + print " build [package_name] [cvs_tag | srpm_path] [target]" + print " list" + print " list " + print " list " + print " kill " + print " update_builders" + print " list_builders" + print " pause" + print " unpause" + print " is_paused" + print " requeue " + print " detail " + print " finish " + print " help" + print "" if __name__ == '__main__': if len(sys.argv) < 2: - print "Usage:\npackage-builder.py \n" - print " is one of:" - print " build [package_name] [cvs_tag | srpm_path] [target]" - print " list" - print " list " - print " list " - print " kill " - print " update_builders" - print " list_builders" - print " pause" - print " unpause" - print "" + Usage() sys.exit(1) # Write out config file if it doesn't exist @@ -380,7 +386,11 @@ if __name__ == '__main__': is_paused(server) elif cmd == 'finish': finish(server, sys.argv[2:]) + elif cmd == 'help': + Usage() + sys.exit(1) else: print "Unknown command." + Usage() sys.exit(1) From bryans at aspsys.com Tue Aug 2 18:28:57 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Tue, 2 Aug 2005 12:28:57 -0600 Subject: Plague web interface Message-ID: <20050802182857.GA23532@aspsys.com> I've set up my own internal plague server and I'm wondering where I can find the web interface to plague that powers this site: http://buildsys.fedoraproject.org/build-status/ Thanks, Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 From skvidal at phy.duke.edu Tue Aug 2 19:36:55 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 02 Aug 2005 15:36:55 -0400 Subject: Plague web interface In-Reply-To: <20050802182857.GA23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> Message-ID: <1123011415.12110.30.camel@cutter> On Tue, 2005-08-02 at 12:28 -0600, Bryan Stillwell wrote: > I've set up my own internal plague server and I'm wondering where I can > find the web interface to plague that powers this site: > > http://buildsys.fedoraproject.org/build-status/ > in the www directory in the plague tree. -sv From bryans at aspsys.com Tue Aug 2 21:18:39 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Tue, 2 Aug 2005 15:18:39 -0600 Subject: Plague web interface In-Reply-To: <1123011415.12110.30.camel@cutter> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> Message-ID: <20050802211839.GB23532@aspsys.com> Thanks Seth! It wasn't part of any of the rpms that were built, so I wrongly assumed it was somewhere else. It appears that the version in cvs must be older than what is currently used, because the links to "Failed Builds" and "Successful Builds" do not exist in the head.psp file: http://cvs.fedora.redhat.com/viewcvs/extras-buildsys/www/template/head.psp?root=fedora&rev=1.4&view=markup Also there seems to be a problem with the kill function in the cvs version. I'm getting this error: # plague-client kill 1 Traceback (most recent call last): File "/usr/bin/plague-client", line 376, in ? kill(server, email, jobid) File "/usr/bin/plague-client", line 176, in kill (err, msg) = server.kill_job(email, jobid) File "/usr/lib64/python2.3/xmlrpclib.py", line 1029, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.3/xmlrpclib.py", line 1316, in __request verbose=self.__verbose File "/usr/lib64/python2.3/xmlrpclib.py", line 1080, in request return self._parse_response(h.getfile(), sock) File "/usr/lib64/python2.3/xmlrpclib.py", line 1219, in _parse_response return u.close() File "/usr/lib64/python2.3/xmlrpclib.py", line 742, in close raise Fault(**self._stack[0]) xmlrpclib.Fault: Thanks, Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 On Tue, Aug 02, 2005 at 03:36:55PM -0400, seth vidal wrote: >On Tue, 2005-08-02 at 12:28 -0600, Bryan Stillwell wrote: >> I've set up my own internal plague server and I'm wondering where I can >> find the web interface to plague that powers this site: >> >> http://buildsys.fedoraproject.org/build-status/ >> > >in the www directory in the plague tree. > >-sv From dcbw at redhat.com Wed Aug 3 04:28:10 2005 From: dcbw at redhat.com (Dan Williams) Date: Wed, 3 Aug 2005 00:28:10 -0400 (EDT) Subject: Plague web interface In-Reply-To: <20050802211839.GB23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> Message-ID: On Tue, 2 Aug 2005, Bryan Stillwell wrote: > Thanks Seth! > > It wasn't part of any of the rpms that were built, so I wrongly assumed > it was somewhere else. > > It appears that the version in cvs must be older than what is currently > used, because the links to "Failed Builds" and "Successful Builds" do > not exist in the head.psp file: > > http://cvs.fedora.redhat.com/viewcvs/extras-buildsys/www/template/head.psp?root=fedora&rev=1.4&view=markup You're right, Seth hasn't committed that local modification yet... Bug him about it. > Also there seems to be a problem with the kill function in the cvs > version. I'm getting this error: Are you running SSL or non SSL between the client and the server? Either way, this is a bug. Dan From dcbw at redhat.com Wed Aug 3 04:32:42 2005 From: dcbw at redhat.com (Dan Williams) Date: Wed, 3 Aug 2005 00:32:42 -0400 (EDT) Subject: Plague web interface In-Reply-To: <20050802211839.GB23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> Message-ID: On Tue, 2 Aug 2005, Bryan Stillwell wrote: > Also there seems to be a problem with the kill function in the cvs > version. I'm getting this error: > > # plague-client kill 1 > Traceback (most recent call last): > File "/usr/bin/plague-client", line 376, in ? > kill(server, email, jobid) > File "/usr/bin/plague-client", line 176, in kill > (err, msg) = server.kill_job(email, jobid) > File "/usr/lib64/python2.3/xmlrpclib.py", line 1029, in __call__ > return self.__send(self.__name, args) > File "/usr/lib64/python2.3/xmlrpclib.py", line 1316, in __request > verbose=self.__verbose > File "/usr/lib64/python2.3/xmlrpclib.py", line 1080, in request > return self._parse_response(h.getfile(), sock) > File "/usr/lib64/python2.3/xmlrpclib.py", line 1219, in > _parse_response > return u.close() > File "/usr/lib64/python2.3/xmlrpclib.py", line 742, in close > raise Fault(**self._stack[0]) > xmlrpclib.Fault: Fixed in CVS. Thanks for the report, Dan From dcbw at redhat.com Wed Aug 3 04:37:13 2005 From: dcbw at redhat.com (Dan Williams) Date: Wed, 3 Aug 2005 00:37:13 -0400 (EDT) Subject: [PATCH] Cleanup help message in plague-client In-Reply-To: <1122990641.4755.31.camel@yoda.jdub.homelinux.org> References: <1122942537.4755.28.camel@yoda.jdub.homelinux.org> <1122990641.4755.31.camel@yoda.jdub.homelinux.org> Message-ID: On Tue, 2 Aug 2005, Josh Boyer wrote: > On Mon, 2005-08-01 at 19:28 -0500, Josh Boyer wrote: > > Hi all, > > > > This patch cleans up the Usage message in plague-client by listing all > > of the available commands and moving them into a function. It also adds > > a help command, which I think was asked for by Jef. > > Here's an updated patch. Dan adds commands too quickly ;). It also > fixes the name of the program in Usage from 'package-client' to > 'plague-client'. Committed, thanks. Dan From bryans at aspsys.com Wed Aug 3 18:12:40 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Wed, 3 Aug 2005 12:12:40 -0600 Subject: Plague web interface In-Reply-To: References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> Message-ID: <20050803181240.GC23532@aspsys.com> On Wed, Aug 03, 2005 at 12:32:42AM -0400, Dan Williams wrote: >Fixed in CVS. Thanks, that seemed to fix that problem! I ran into a couple other problems that I've attached one-line patches for. If I find anything else, I'll see if I can fix them first. Seth: any chance you can update the web code to match what's currently being used? Thanks, Bryan -------------- next part -------------- diff -ru extras-buildsys.orig/www/job.psp extras-buildsys/www/job.psp --- extras-buildsys.orig/www/job.psp 2005-07-28 23:27:16.000000000 -0600 +++ extras-buildsys/www/job.psp 2005-08-03 12:10:07.309463288 -0600 @@ -20,7 +20,7 @@ import socket, xmlrpclib try: - (e, msg, job) = server.detail_job('', uid) + (e, msg, job) = server.detail_job(uid) except socket.error, e: msg = str(e) %> -------------- next part -------------- diff -ru extras-buildsys.orig/server/BuildMaster.py extras-buildsys/server/BuildMaster.py --- extras-buildsys.orig/server/BuildMaster.py 2005-08-01 22:10:32.000000000 -0600 +++ extras-buildsys/server/BuildMaster.py 2005-08-03 12:08:26.816740504 -0600 @@ -189,7 +189,7 @@ self.curs.execute('DELETE FROM archjobs WHERE parent_uid=%d' % uid) self.dbcx.commit() - curs.execute('CREATE TABLE jobs (' \ + self.curs.execute('CREATE TABLE jobs (' \ 'uid INTEGER PRIMARY KEY, ' \ 'username VARCHAR(20), ' \ 'package VARCHAR(50), ' \ From bryans at aspsys.com Wed Aug 3 18:21:09 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Wed, 3 Aug 2005 12:21:09 -0600 Subject: Plague web interface In-Reply-To: <20050803181240.GC23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> <20050803181240.GC23532@aspsys.com> Message-ID: <20050803182109.GD23532@aspsys.com> So I fixed that problem, now I run into the following: Build Server accepting requests on localhost:8887. Exception in thread Thread-4: Traceback (most recent call last): File "/usr/src/build/394691-x86_64/install/usr/lib64/python2.3/threading.py", line 436, in __bootstrap self.run() File "/usr/share/plague/server/BuildMaster.py", line 496, in run self._start_requeued_jobs() File "/usr/share/plague/server/BuildMaster.py", line 192, in _start_requeued_jobs self.curs.execute('CREATE TABLE jobs (' \ File "/home/bryans/rpm/tmp/python-sqlite-root//usr/lib64/python2.3/site-packages/sqlite/main.py", line 244, in execute DatabaseError: table jobs already exists I'm doing everything without SSL btw... Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 On Wed, Aug 03, 2005 at 12:12:40PM -0600, Bryan Stillwell wrote: >On Wed, Aug 03, 2005 at 12:32:42AM -0400, Dan Williams wrote: >>Fixed in CVS. > >Thanks, that seemed to fix that problem! > >I ran into a couple other problems that I've attached one-line patches >for. If I find anything else, I'll see if I can fix them first. > >Seth: any chance you can update the web code to match what's currently >being used? > >Thanks, >Bryan From skvidal at phy.duke.edu Thu Aug 4 06:54:17 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Thu, 04 Aug 2005 02:54:17 -0400 Subject: Plague web interface In-Reply-To: <20050803181240.GC23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> <20050803181240.GC23532@aspsys.com> Message-ID: <1123138457.23426.19.camel@cutter> > Seth: any chance you can update the web code to match what's currently > being used? > done. -sv From jkeating at j2solutions.net Thu Aug 4 20:43:12 2005 From: jkeating at j2solutions.net (Jesse Keating) Date: Thu, 04 Aug 2005 13:43:12 -0700 Subject: Setting up plague server - Difficulty w/ SSL setup Message-ID: <1123188193.11730.22.camel@prometheus.gamehouse.com> I'm following the README in the plague package, and I'm having trouble w/ step 6 of Configuring SSL for your Build System. 6. Sign the build server certificate request with the BSCA certificate openssl ca -out server_cert.pem -infiles server_req.pem If I run this, I get: [root at plague server]# openssl ca -out server_cert.pem -infiles server_req.pem Using configuration from /usr/share/ssl/openssl.cnf Error opening CA private key ./demoCA/private/cakey.pem 18008:error:02001002:system library:fopen:No such file or directory:bss_file.c:259:fopen('./demoCA/private/cakey.pem','r') 18008:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:261: unable to load CA private key So it would seem I need to specify the CA private key so I try: [root at plague server]# openssl ca -keyfile /root/CERTS/CA/private/ca_key.pem -out server_cert.pem -infiles server_req.pem Using configuration from /usr/share/ssl/openssl.cnf Error opening CA certificate ./demoCA/cacert.pem 18085:error:02001002:system library:fopen:No such file or directory:bss_file.c:259:fopen('./demoCA/cacert.pem','r') 18085:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:261: unable to load certificate Now I'm missing the cert... one more step: [root at plague server]# openssl ca -keyfile /root/CERTS/CA/private/ca_key.pem -cert /root/CERTS/CA/ca_cert.pem -out server_cert.pem -infiles server_req.pem Using configuration from /usr/share/ssl/openssl.cnf I am unable to access the ./demoCA/newcerts directory ./demoCA/newcerts: No such file or directory Still no juice. Not sure why it is trying to access the newcerts file. Can anybody help me with what I'm doing wrong here, or help me generate a patch to the documentation that will lead people in the right direction? -- Jesse Keating RHCE (http://geek.j2solutions.net) Fedora Legacy Team (http://www.fedoralegacy.org) GPG Public Key (http://geek.j2solutions.net/jkeating.j2solutions.pub) Was I helpful? Let others know: http://svcs.affero.net/rm.php?r=jkeating From dcbw at redhat.com Fri Aug 5 15:02:45 2005 From: dcbw at redhat.com (Dan Williams) Date: Fri, 05 Aug 2005 11:02:45 -0400 Subject: Plague web interface In-Reply-To: <20050803182109.GD23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> <20050803181240.GC23532@aspsys.com> <20050803182109.GD23532@aspsys.com> Message-ID: <1123254165.4789.0.camel@dcbw.boston.redhat.com> On Wed, 2005-08-03 at 12:21 -0600, Bryan Stillwell wrote: > So I fixed that problem, now I run into the following: > > Build Server accepting requests on localhost:8887. > > Exception in thread Thread-4: > Traceback (most recent call last): > File "/usr/src/build/394691-x86_64/install/usr/lib64/python2.3/threading.py", line 436, in __bootstrap > self.run() > File "/usr/share/plague/server/BuildMaster.py", line 496, in run > self._start_requeued_jobs() > File "/usr/share/plague/server/BuildMaster.py", line 192, in _start_requeued_jobs > self.curs.execute('CREATE TABLE jobs (' \ > File "/home/bryans/rpm/tmp/python-sqlite-root//usr/lib64/python2.3/site-packages/sqlite/main.py", line 244, in execute > DatabaseError: table jobs already exists > > > I'm doing everything without SSL btw... Sorry, that was a copy & paste error that I fixed in CVS yesterday. Dan From dcbw at redhat.com Fri Aug 5 16:06:34 2005 From: dcbw at redhat.com (Dan Williams) Date: Fri, 05 Aug 2005 12:06:34 -0400 Subject: Plague web interface In-Reply-To: <20050803181240.GC23532@aspsys.com> References: <20050802182857.GA23532@aspsys.com> <1123011415.12110.30.camel@cutter> <20050802211839.GB23532@aspsys.com> <20050803181240.GC23532@aspsys.com> Message-ID: <1123257994.4789.8.camel@dcbw.boston.redhat.com> On Wed, 2005-08-03 at 12:12 -0600, Bryan Stillwell wrote: > On Wed, Aug 03, 2005 at 12:32:42AM -0400, Dan Williams wrote: > >Fixed in CVS. > > Thanks, that seemed to fix that problem! > > I ran into a couple other problems that I've attached one-line patches > for. If I find anything else, I'll see if I can fix them first. The issue with detail_job should be fixed in CVS soon. I'm not quite sure why I needed the email address for that function at all, so I've now removed code that provided it to detail job (job.psp, UserInterface.py, client.py). Dan From bryans at aspsys.com Fri Aug 5 23:14:20 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Fri, 5 Aug 2005 17:14:20 -0600 Subject: Killed jobs 'finished' not 'killed'? Message-ID: <20050805231420.GC3009@aspsys.com> I've run into another problem with my internal plague server. It appears that when you run `plague-client kill jobid` it sets the status to 'finished' and not 'killed'. I believe the proper behaviour should be 'killed', but because its currently using 'finished' the `plague-client requeue jobid` command doesn't work and gives this message: Error: Job 11 must be either 'failed' or 'killed' to requeue. Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 From bryans at aspsys.com Fri Aug 5 22:56:06 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Fri, 5 Aug 2005 16:56:06 -0600 Subject: Builds stuck in needsign/in-progress Message-ID: <20050805225606.GB3009@aspsys.com> Every once in a while on my internal plague server I get builds that get stuck in the state "needsign/in-progress" and never make it into "needsign/success". Has anyone else run into this problem? Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 From jwboyer at jdub.homelinux.org Sat Aug 6 02:18:30 2005 From: jwboyer at jdub.homelinux.org (Josh Boyer) Date: Fri, 05 Aug 2005 21:18:30 -0500 Subject: Setting up plague server - Difficulty w/ SSL setup In-Reply-To: <1123188193.11730.22.camel@prometheus.gamehouse.com> References: <1123188193.11730.22.camel@prometheus.gamehouse.com> Message-ID: <1123294710.23245.2.camel@yoda.jdub.homelinux.org> On Thu, 2005-08-04 at 13:43 -0700, Jesse Keating wrote: > I'm following the README in the plague package, and I'm having trouble > w/ step 6 of Configuring SSL for your Build System. > > > 6. Sign the build server certificate request with the BSCA certificate > > openssl ca -out server_cert.pem -infiles server_req.pem > > > If I run this, I get: > > [root at plague server]# openssl ca -out server_cert.pem -infiles server_req.pem > Using configuration from /usr/share/ssl/openssl.cnf > Error opening CA private key ./demoCA/private/cakey.pem > 18008:error:02001002:system library:fopen:No such file or directory:bss_file.c:259:fopen('./demoCA/private/cakey.pem','r') > 18008:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:261: > unable to load CA private key > > > So it would seem I need to specify the CA private key so I try: > > [root at plague server]# openssl ca -keyfile /root/CERTS/CA/private/ca_key.pem -out server_cert.pem -infiles server_req.pem > Using configuration from /usr/share/ssl/openssl.cnf > Error opening CA certificate ./demoCA/cacert.pem > 18085:error:02001002:system library:fopen:No such file or directory:bss_file.c:259:fopen('./demoCA/cacert.pem','r') > 18085:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:261: > unable to load certificate > > Now I'm missing the cert... one more step: > > [root at plague server]# openssl ca -keyfile /root/CERTS/CA/private/ca_key.pem -cert /root/CERTS/CA/ca_cert.pem -out server_cert.pem -infiles server_req.pem > Using configuration from /usr/share/ssl/openssl.cnf > I am unable to access the ./demoCA/newcerts directory > ./demoCA/newcerts: No such file or directory > > Still no juice. Not sure why it is trying to access the newcerts file. > Can anybody help me with what I'm doing wrong here, or help me generate > a patch to the documentation that will lead people in the right > direction? > I remember having the same problem. I believe i just renamed my ca_key.pem file to cakey.pem and the problem went away. I should say that I never really did get it all to work. Probably because the keys were screwed up. In any case, the README seems a bit out of date. At least on FC4 where I was running the steps. If anyone knows the correct steps, please post them. thx, josh From dcbw at redhat.com Sat Aug 6 17:51:22 2005 From: dcbw at redhat.com (Dan Williams) Date: Sat, 6 Aug 2005 13:51:22 -0400 (EDT) Subject: Killed jobs 'finished' not 'killed'? In-Reply-To: <20050805231420.GC3009@aspsys.com> References: <20050805231420.GC3009@aspsys.com> Message-ID: On Fri, 5 Aug 2005, Bryan Stillwell wrote: > I've run into another problem with my internal plague server. It > appears that when you run `plague-client kill jobid` it sets the status > to 'finished' and not 'killed'. I believe the proper behaviour should > be 'killed', but because its currently using 'finished' the > `plague-client requeue jobid` command doesn't work and gives this > message: > > Error: Job 11 must be either 'failed' or 'killed' to requeue. This may be fixed in current CVS. Each job actually has two values, a 'status' and a 'result'. When you kill the job (again, using current CVS) the status gets set directly to 'finished' but the result gets set to killed. Current CVS checks for a _result_ of 'killed' or 'failed', and disregards the status altogether. So it should currently work. I did just test this with current CVS, and it appears to requeue killed jobs correctly. That code has undergone some fairly large changes recently, so please let me know if current CVS still does not work for you. Dan From dcbw at redhat.com Sat Aug 6 17:53:29 2005 From: dcbw at redhat.com (Dan Williams) Date: Sat, 6 Aug 2005 13:53:29 -0400 (EDT) Subject: Builds stuck in needsign/in-progress In-Reply-To: <20050805225606.GB3009@aspsys.com> References: <20050805225606.GB3009@aspsys.com> Message-ID: On Fri, 5 Aug 2005, Bryan Stillwell wrote: > Every once in a while on my internal plague server I get builds that get > stuck in the state "needsign/in-progress" and never make it into > "needsign/success". Has anyone else run into this problem? I think this is an artifact of incorrect ordering of writing status updates to the database when the job is finished, I'll correct this in CVS shortly. Dan From dcbw at redhat.com Mon Aug 8 03:15:49 2005 From: dcbw at redhat.com (Dan Williams) Date: Sun, 7 Aug 2005 23:15:49 -0400 (EDT) Subject: plague 0.3 status Message-ID: Hi, I've committed the last of the code for what I believe will be plague 0.3. This includes changes to allow more than one builder instance on a single machine. If there are no other bugs found before tommorow afternoon, I'm going to tag plague 0.3, and add it to extras CVS and shove a package through the build system. Current CVS has built about 90 packages without problems on the Aurora build cluster so far. One thing that would be nice to have before then though: scripts or better README directions for SSL cert setup. Without using custom OpenSSL config files, the directions are quite confusing, but its an afternoon of work to hack together some scripts to do it all too. If any wants to volunteer :) Seth: I think when we switch over the Extras build system to 0.3, we should fire up a 2nd builder on ppc1 if only so we're actually using both processors. Sound OK? Dan From skvidal at phy.duke.edu Mon Aug 8 05:21:54 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Mon, 08 Aug 2005 01:21:54 -0400 Subject: plague 0.3 status In-Reply-To: References: Message-ID: <1123478514.29914.36.camel@cutter> On Sun, 2005-08-07 at 23:15 -0400, Dan Williams wrote: > Hi, > > I've committed the last of the code for what I believe will be plague 0.3. This > includes changes to allow more than one builder instance on a single machine. > If there are no other bugs found before tommorow afternoon, I'm going to tag > plague 0.3, and add it to extras CVS and shove a package through the build > system. Current CVS has built about 90 packages without problems on the Aurora > build cluster so far. > > One thing that would be nice to have before then though: scripts or better > README directions for SSL cert setup. Without using custom OpenSSL config > files, the directions are quite confusing, but its an afternoon of work to hack > together some scripts to do it all too. If any wants to volunteer :) > > Seth: I think when we switch over the Extras build system to 0.3, we should fire > up a 2nd builder on ppc1 if only so we're actually using both processors. Sound > OK? If we fire up another builder we need to get IS to open up more ports. that gets rather tricky. -sv From bryans at aspsys.com Mon Aug 8 22:46:45 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Mon, 8 Aug 2005 16:46:45 -0600 Subject: patch to add SuSE 9.3 support to mock Message-ID: <20050808224645.GD15252@aspsys.com> I see mock and plague being a great way for third parties to do builds of their software on many distributions. For example, our customers commonly request Fedora, RHEL, SuSE, or SLES to be installed on their Beowulf clusters. There's common clustering software (dsh, mpich, torque, etc.) that we install on top of these OSes and I've been working on packaging most of these into rpms. Since mock/plague appear to be mostly Fedora centric, I've had to patch a few things to get it working. Attached is a patch that I created to deal with some SuSE deficiencies I've run across. Also attached is the buildroots file and mock config file. Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 -------------- next part -------------- diff -ru mock-0.4.orig/mock.py mock-0.4/mock.py --- mock-0.4.orig/mock.py 2005-07-26 14:33:52.000000000 -0600 +++ mock-0.4/mock.py 2005-08-08 16:28:39.586256320 -0600 @@ -574,6 +574,12 @@ cmd = "chown %s.%s %s/%s" % (self.config['chrootuser'], self.config['chrootgroup'], self.builddir, subdir) self.do_chroot(cmd, fatal = True) + + # SuSE's useradd doesn't create home directory, so since we created + # it above we set the ownership here. + cmd = "chown %s.%s %s" % (self.config['chrootuser'], + self.config['chrootgroup'], self.homedir) + self.do_chroot(cmd, fatal = True) # rpmmacros default macrofile_out = '%s%s/.rpmmacros' % (self.rootdir, self.homedir) @@ -581,6 +587,10 @@ rpmmacros = open(macrofile_out, 'w') rpmmacros.write(self.config['macros']) rpmmacros.close() + + # SuSE's rpm version doesn't support the dist tag in 9.3 + cmd = "cat /etc/rpm/macros.disttag >>%s/.rpmmacros" % self.homedir + self.do_chroot(cmd, fatal = True) def _prep_build(self): -------------- next part -------------- A non-text attachment was scrubbed... Name: buildroots.xml Type: text/xml Size: 3244 bytes Desc: not available URL: -------------- next part -------------- #!/usr/bin/python -tt import os config_opts['root'] = 'suse-93-x86_64' config_opts['basedir'] = '/var/lib/mock/' config_opts['chroot'] = '/usr/sbin/mock-helper chroot' config_opts['mount'] = '/usr/sbin/mock-helper mount' config_opts['umount'] = '/usr/sbin/mock-helper umount' config_opts['rm'] = '/usr/sbin/mock-helper rm' config_opts['mknod'] = '/usr/sbin/mock-helper mknod' config_opts['yum'] = '/usr/sbin/mock-helper yum' config_opts['runuser'] = '/bin/su' config_opts['buildgroup'] = 'build' config_opts['chrootuser'] = 'mockbuild' config_opts['chrootgroup'] = 'users' config_opts['chrootuid'] = os.geteuid() config_opts['chrootgid'] = os.getegid() config_opts['chroothome'] = '/builddir' config_opts['clean'] = True config_opts['target_arch'] = 'x86_64' config_opts['yum.conf'] = """ [main] cachedir=/var/cache/yum debuglevel=1 logfile=/var/log/yum.log reposdir=/dev/null retries=20 obsoletes=1 gpgcheck=0 assumeyes=1 exclude=*.i386 *.i686 *.athlon *.i586 *.486 # repos [core] name=core baseurl=http://localhost/suse/9.3/x86_64/ #[updates-released] #name=updates #baseurl=http://localhost/suse/updates/9.3/x86_64/ [groups] name=groups baseurl=http://localhost/suse/buildgroups/9.3/x86_64/ [local] name=aspen baseurl=http://localhost/localrpms/suse93/ """ From bryans at aspsys.com Mon Aug 8 19:23:32 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Mon, 8 Aug 2005 13:23:32 -0600 Subject: Builds stuck in needsign/in-progress In-Reply-To: References: <20050805225606.GB3009@aspsys.com> Message-ID: <20050808192332.GB15252@aspsys.com> On Sat, Aug 06, 2005 at 01:53:29PM -0400, Dan Williams wrote: >On Fri, 5 Aug 2005, Bryan Stillwell wrote: > >> Every once in a while on my internal plague server I get builds that get >> stuck in the state "needsign/in-progress" and never make it into >> "needsign/success". Has anyone else run into this problem? > >I think this is an artifact of incorrect ordering of writing status updates to >the database when the job is finished, I'll correct this in CVS shortly. I've done about 15 builds in a row without running into this problem, so I consider this to be fixed now as well. Thanks again! Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 From bryans at aspsys.com Mon Aug 8 19:22:11 2005 From: bryans at aspsys.com (Bryan Stillwell) Date: Mon, 8 Aug 2005 13:22:11 -0600 Subject: Killed jobs 'finished' not 'killed'? In-Reply-To: References: <20050805231420.GC3009@aspsys.com> Message-ID: <20050808192210.GA15252@aspsys.com> On Sat, Aug 06, 2005 at 01:51:22PM -0400, Dan Williams wrote: >This may be fixed in current CVS. Each job actually has two values, a 'status' >and a 'result'. When you kill the job (again, using current CVS) the status >gets set directly to 'finished' but the result gets set to killed. Current CVS >checks for a _result_ of 'killed' or 'failed', and disregards the status >altogether. So it should currently work. > >I did just test this with current CVS, and it appears to requeue killed jobs >correctly. That code has undergone some fairly large changes recently, so >please let me know if current CVS still does not work for you. This seems to work great now. Thanks! Bryan -- Aspen Systems, Inc. | http://www.aspsys.com/ Production Engineer | Phone: (303)431-4606 bryans at aspsys.com | Fax: (303)431-7196 From dcbw at redhat.com Tue Aug 9 13:18:44 2005 From: dcbw at redhat.com (Dan Williams) Date: Tue, 09 Aug 2005 09:18:44 -0400 Subject: Setting up plague server - Difficulty w/ SSL setup In-Reply-To: <1123188193.11730.22.camel@prometheus.gamehouse.com> References: <1123188193.11730.22.camel@prometheus.gamehouse.com> Message-ID: <1123593524.11868.2.camel@dcbw.boston.redhat.com> On Thu, 2005-08-04 at 13:43 -0700, Jesse Keating wrote: > I'm following the README in the plague package, and I'm having trouble > w/ step 6 of Configuring SSL for your Build System. So, For plague 0.3 (packages for FC4/python2.4 at http://people.redhat.com/dcbw/plague ) I added a tool called plague-certhelper (in the plague-utils package) that should do what you need, and updated the README file to use the tool. If you had problems or were confused by the previous directions that used OpenSSL, use plague-certhelper. Let me know if anything seems unclear or buggy. Dan From skvidal at phy.duke.edu Tue Aug 9 13:37:29 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 09 Aug 2005 09:37:29 -0400 Subject: Setting up plague server - Difficulty w/ SSL setup In-Reply-To: <1123593524.11868.2.camel@dcbw.boston.redhat.com> References: <1123188193.11730.22.camel@prometheus.gamehouse.com> <1123593524.11868.2.camel@dcbw.boston.redhat.com> Message-ID: <1123594650.19308.3.camel@cutter> On Tue, 2005-08-09 at 09:18 -0400, Dan Williams wrote: > On Thu, 2005-08-04 at 13:43 -0700, Jesse Keating wrote: > > I'm following the README in the plague package, and I'm having trouble > > w/ step 6 of Configuring SSL for your Build System. > > So, > > For plague 0.3 (packages for FC4/python2.4 at > http://people.redhat.com/dcbw/plague ) I added a tool called > plague-certhelper (in the plague-utils package) that should do what you > need, and updated the README file to use the tool. If you had problems > or were confused by the previous directions that used OpenSSL, use > plague-certhelper. Let me know if anything seems unclear or buggy. > Dan, I made a wiki spot at: http://fedoraproject.org/wiki/Projects/Plague if you want to put plague items there and I made a release dir available at: http://fedoraproject.org/projects/plague/ if you want to put tarballs/rpms there. if you need access the non-wiki path email me. -sv From dcbw at redhat.com Tue Aug 9 13:48:25 2005 From: dcbw at redhat.com (Dan Williams) Date: Tue, 09 Aug 2005 09:48:25 -0400 Subject: Setting up plague server - Difficulty w/ SSL setup In-Reply-To: <1123594650.19308.3.camel@cutter> References: <1123188193.11730.22.camel@prometheus.gamehouse.com> <1123593524.11868.2.camel@dcbw.boston.redhat.com> <1123594650.19308.3.camel@cutter> Message-ID: <1123595305.11868.5.camel@dcbw.boston.redhat.com> On Tue, 2005-08-09 at 09:37 -0400, seth vidal wrote: > On Tue, 2005-08-09 at 09:18 -0400, Dan Williams wrote: > > On Thu, 2005-08-04 at 13:43 -0700, Jesse Keating wrote: > > > I'm following the README in the plague package, and I'm having trouble > > > w/ step 6 of Configuring SSL for your Build System. > > > > So, > > > > For plague 0.3 (packages for FC4/python2.4 at > > http://people.redhat.com/dcbw/plague ) I added a tool called > > plague-certhelper (in the plague-utils package) that should do what you > > need, and updated the README file to use the tool. If you had problems > > or were confused by the previous directions that used OpenSSL, use > > plague-certhelper. Let me know if anything seems unclear or buggy. > > > > Dan, > I made a wiki spot at: http://fedoraproject.org/wiki/Projects/Plague if > you want to put plague items there and I made a release dir available > at: http://fedoraproject.org/projects/plague/ if you want to put > tarballs/rpms there. > > if you need access the non-wiki path email me. BTW, I filed bug 165413 to submit plague packages to Extras. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=165413 Dan From dcbw at redhat.com Mon Aug 15 12:15:40 2005 From: dcbw at redhat.com (Dan Williams) Date: Mon, 15 Aug 2005 08:15:40 -0400 (EDT) Subject: Extras build system upgrade notice Message-ID: Hi, Seth, Jeremy, and I are upgrading the Extras build system today. We'll send out the announcement when the upgrade is complete, which is likely to take until late Monday afternoon, US EST, to get all the pieces together. Here are some things to expect: o You immediately see the Job ID that you just enqueued o The Web interface is easier to follow, the most relevant jobs are at the top of the page o We'll be running 2 PPC builders (hopefully), so PPC should keep up more easily o Users and their permissions can be changed on the fly, no more waiting to get added to the database What you need to do: o 'yum update plague' so that you have the latest plague-0.3 RPMs. Once the system is up, we'll push through FC-3 and FC-4 plague RPMs for those who aren't running Rawhide. Dan From dcbw at redhat.com Mon Aug 15 19:05:20 2005 From: dcbw at redhat.com (Dan Williams) Date: Mon, 15 Aug 2005 15:05:20 -0400 Subject: [UPGRADE COMPLETE] Re: Extras build system upgrade notice In-Reply-To: References: Message-ID: <1124132720.3145.11.camel@dcbw.boston.redhat.com> On Mon, 2005-08-15 at 08:15 -0400, Dan Williams wrote: > Hi, > > Seth, Jeremy, and I are upgrading the Extras build system today. We'll send out > the announcement when the upgrade is complete, which is likely to take until > late Monday afternoon, US EST, to get all the pieces together. The build system should be back up and accepting build jobs. You'll have to have plague-0.3.2-1 or later to talk to the build server. Rawhide RPMs are built already, I'll get some RPMs built for FC3 and FC4 here in a bit. Dan From dcbw at redhat.com Sat Aug 20 01:40:39 2005 From: dcbw at redhat.com (Dan Williams) Date: Fri, 19 Aug 2005 21:40:39 -0400 (EDT) Subject: Stuff to look for in plague 0.4 Message-ID: Quite a few things will change for plague 0.4: o Single builder, multiple jobs: you'll only have to run one builder instance on a machine to run multiple build jobs; each builder will be able to build a set number of jobs at a time (usually equal to # of processors). - Architectures are autodetected o New config file format: it's now using ConfigParser, which means more straightforward and text-friendly config files, for both the server and the builder. o Targets get their own config files: each target, on the server and on the builder, gets its own config file. This means a few things: - each target can have its own CVS settings - each target can be a different distribution o Scratch/Testing targets: targets that don't contribute packages tothe repository, so you can test out your packages before you actually push them through. Scratch targets also don't kill a job when one architecture fails. If anyone has more thoughts, file some RFEs in bugzilla for Fedora Extras Infrastructure. Dan From fedora at leemhuis.info Sat Aug 20 10:17:50 2005 From: fedora at leemhuis.info (Thorsten Leemhuis) Date: Sat, 20 Aug 2005 12:17:50 +0200 Subject: Stuff to look for in plague 0.4 In-Reply-To: References: Message-ID: <1124533070.8272.22.camel@localhost.localdomain> Am Freitag, den 19.08.2005, 21:40 -0400 schrieb Dan Williams: > Quite a few things will change for plague 0.4: > [...] > If anyone has more thoughts, file some RFEs in bugzilla for Fedora Extras > Infrastructure. What is the current status of passing special options to the actual rpmbuild call? We should have those for building kernel-modules because building kernel-modules for a lot of different kernels would be much easier if no changes to the actual kernel-module-specfile are needed. In the current KernelModuleProposal-Example2 (*1) this is realized by: %{!?kver: %define kver %(uname -r)} [...] make KVERS=%{kver} KSRC="%{_usrsrc}/kernels/%{kver}-%{_target_cpu}" -C driver So if someone builds the src.rpm at home it is rebuild for the current kernel. When build with mock the buildsystem should defined kver. Something like: for kernel in do mock build /path/to/srpm --define "kver $kernel" done AFACS we need support for this in both mock and plague (or is something like that already possible? mach can pass options to rpmbuild, mock can't iirc) CU thl *1 - Current kernel-module-proposal: http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal -- Thorsten Leemhuis From ivazquez at ivazquez.net Sat Aug 20 17:06:02 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Sat, 20 Aug 2005 13:06:02 -0400 Subject: Patch to put mockbuild results in $WORKDIR Message-ID: <1124557562.3414.8.camel@ignacio.lan> The attached patch makes the results of mockbuild end up under $WORKDIR. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: common-mockbuild.patch Type: text/x-patch Size: 1041 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From skvidal at phy.duke.edu Tue Aug 23 05:18:49 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 23 Aug 2005 01:18:49 -0400 Subject: Stuff to look for in plague 0.4 In-Reply-To: <1124533070.8272.22.camel@localhost.localdomain> References: <1124533070.8272.22.camel@localhost.localdomain> Message-ID: <1124774329.16972.75.camel@cutter> On Sat, 2005-08-20 at 12:17 +0200, Thorsten Leemhuis wrote: > Am Freitag, den 19.08.2005, 21:40 -0400 schrieb Dan Williams: > > Quite a few things will change for plague 0.4: > > [...] > > If anyone has more thoughts, file some RFEs in bugzilla for Fedora Extras > > Infrastructure. > > What is the current status of passing special options to the actual > rpmbuild call? We should have those for building kernel-modules because > building kernel-modules for a lot of different kernels would be much > easier if no changes to the actual kernel-module-specfile are needed. In > the current KernelModuleProposal-Example2 (*1) this is realized by: > > %{!?kver: %define kver %(uname -r)} > [...] > make KVERS=%{kver} KSRC="%{_usrsrc}/kernels/%{kver}-%{_target_cpu}" -C driver > > So if someone builds the src.rpm at home it is rebuild for the current > kernel. When build with mock the buildsystem should defined kver. > Something like: > > for kernel in > do > mock build /path/to/srpm --define "kver $kernel" > done > > AFACS we need support for this in both mock and plague (or is something > like that already possible? mach can pass options to rpmbuild, mock > can't iirc) > I do not think that adding this into mock or rpmbuild is a good idea, honestly. If the package won't built w/o arbitrary defines then we'll have a devil of a time figuring out how to REBUILD it later. We need builds to be replayable and if we have to count on what someone inputted into the cli then we can't replay them. -sv From fedora at leemhuis.info Tue Aug 23 05:47:53 2005 From: fedora at leemhuis.info (Thorsten Leemhuis) Date: Tue, 23 Aug 2005 07:47:53 +0200 Subject: kernel-modules in the buildsystem (Was: Re: Stuff to look for in plague 0.4) In-Reply-To: <1124774329.16972.75.camel@cutter> References: <1124533070.8272.22.camel@localhost.localdomain> <1124774329.16972.75.camel@cutter> Message-ID: <1124776073.30231.15.camel@thl.ct.heise.de> CC fedora-packaging because this concerns kernel-module-packaging Am Dienstag, den 23.08.2005, 01:18 -0400 schrieb seth vidal: > On Sat, 2005-08-20 at 12:17 +0200, Thorsten Leemhuis wrote: > > Am Freitag, den 19.08.2005, 21:40 -0400 schrieb Dan Williams: > > > Quite a few things will change for plague 0.4: > > > [...] > > > If anyone has more thoughts, file some RFEs in bugzilla for Fedora Extras > > > Infrastructure. > > > > What is the current status of passing special options to the actual > > rpmbuild call? We should have those for building kernel-modules because > > building kernel-modules for a lot of different kernels would be much > > easier if no changes to the actual kernel-module-specfile are needed. In > > the current KernelModuleProposal-Example2 (*1) this is realized by: > > > > %{!?kver: %define kver %(uname -r)} > > [...] > > make KVERS=%{kver} KSRC="%{_usrsrc}/kernels/%{kver}-%{_target_cpu}" -C driver > > > > So if someone builds the src.rpm at home it is rebuild for the current > > kernel. When build with mock the buildsystem should defined kver. > > Something like: > > > > for kernel in > > do > > mock build /path/to/srpm --define "kver $kernel" > > done > > > > AFACS we need support for this in both mock and plague (or is something > > like that already possible? mach can pass options to rpmbuild, mock > > can't iirc) > > I do not think that adding this into mock or rpmbuild is a good idea, > honestly. For other packages I agree. Kernel-modules should be a exception IMHO. > If the package won't built w/o arbitrary defines then we'll > have a devil of a time figuring out how to REBUILD it later. It builds for the normal user without arbitrary defines. And even better: It automatically builds what you probably want, because with the above it builds the kernel-module for the currently running kernel. If you get a random kernel-module.src.rpm where kver ist hardcoded you have to edit it each time to get compile against a new kernel. BTW, this is in all three proposals in http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal If we all agree that we don't like passing arguments to rpmbuild then we should restart the kernel-module-discussion on fedora-packaging. CU thl From skvidal at phy.duke.edu Tue Aug 23 05:59:27 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 23 Aug 2005 01:59:27 -0400 Subject: kernel-modules in the buildsystem (Was: Re: Stuff to look for in plague 0.4) In-Reply-To: <1124776073.30231.15.camel@thl.ct.heise.de> References: <1124533070.8272.22.camel@localhost.localdomain> <1124774329.16972.75.camel@cutter> <1124776073.30231.15.camel@thl.ct.heise.de> Message-ID: <1124776767.16972.88.camel@cutter> > For other packages I agree. Kernel-modules should be a exception IMHO. I worry that having them as the exception means we have to do the code anyway which makes it a slippery-damn-slope to all packages using arbitrary defines. > It builds for the normal user without arbitrary defines. And even > better: It automatically builds what you probably want, because with the > above it builds the kernel-module for the currently running kernel. If > you get a random kernel-module.src.rpm where kver ist hardcoded you have > to edit it each time to get compile against a new kernel. > why doesn't it automatically build for whatever the highest installed kernel-devel or highest installed kernel package is? -sv From fedora at leemhuis.info Tue Aug 23 06:27:09 2005 From: fedora at leemhuis.info (Thorsten Leemhuis) Date: Tue, 23 Aug 2005 08:27:09 +0200 Subject: kernel-modules in the buildsystem (Was: Re: Stuff to look for in plague 0.4) In-Reply-To: <1124776767.16972.88.camel@cutter> References: <1124533070.8272.22.camel@localhost.localdomain> <1124774329.16972.75.camel@cutter> <1124776073.30231.15.camel@thl.ct.heise.de> <1124776767.16972.88.camel@cutter> Message-ID: <1124778429.30231.40.camel@thl.ct.heise.de> Am Dienstag, den 23.08.2005, 01:59 -0400 schrieb seth vidal: > > For other packages I agree. Kernel-modules should be a exception IMHO. > > I worry that having them as the exception means we have to do the code > anyway which makes it a slippery-damn-slope to all packages using > arbitrary defines. :) > > It builds for the normal user without arbitrary defines. And even > > better: It automatically builds what you probably want, because with the > > above it builds the kernel-module for the currently running kernel. If > > you get a random kernel-module.src.rpm where kver ist hardcoded you have > > to edit it each time to get compile against a new kernel. > > > why doesn't it automatically build for whatever the highest installed > kernel-devel or highest installed kernel package is? And who decides when a module for a smp-kernel is build and when not? Doing this with %build buildfunc() { make something } %ifarch i686 x86_64 ppc build up build smp %else build up %endif in the spec is really ugly (and not possible with our current scheme afaics -- need to try, but i suspect it won't work). CU thl From wtogami at redhat.com Tue Aug 23 08:34:45 2005 From: wtogami at redhat.com (Warren Togami) Date: Mon, 22 Aug 2005 22:34:45 -1000 Subject: kernel-modules in the buildsystem (Was: Re: Stuff to look for in plague 0.4) In-Reply-To: <1124776767.16972.88.camel@cutter> References: <1124533070.8272.22.camel@localhost.localdomain> <1124774329.16972.75.camel@cutter> <1124776073.30231.15.camel@thl.ct.heise.de> <1124776767.16972.88.camel@cutter> Message-ID: <430ADFA5.1090308@redhat.com> seth vidal wrote: >>For other packages I agree. Kernel-modules should be a exception IMHO. > > > I worry that having them as the exception means we have to do the code > anyway which makes it a slippery-damn-slope to all packages using > arbitrary defines. > I don't buy the slippery slope argument here. Kernel is really the only messy case where you can have multiple versions installed simultaneously. There are also no other valid cases where what is installed does not necessarily match what is running. > > > why doesn't it automatically build for whatever the highest installed > kernel-devel or highest installed kernel package is? > The default spec "uname -r" makes it so a user can easily build it against their currently running kernel. This is desirable to easily support users who use custom kernels. For everyone else we provide packages against the official released kernels. It is very desirable to be able to build both of these cases from the same SRPM without modification. Defining only kver in the build environment is a simple and elegant way of supporting both of these requirements. It also makes it much possible to automate kernel module rebuilds when a new kernel is released. Warren Togami wtogami at redhat.com From wtogami at redhat.com Tue Aug 23 08:44:04 2005 From: wtogami at redhat.com (Warren Togami) Date: Mon, 22 Aug 2005 22:44:04 -1000 Subject: Traceback after package build Message-ID: <430AE1D4.401@redhat.com> http://buildsys.fedoraproject.org/logs//development/329-monotone-0.22-4.fc5/ppc/ http://buildsys.fedoraproject.org/logs//development/329-monotone-0.22-4.fc5/ppc/builder.log setup build ending Traceback (most recent call last): File "/usr/bin/mock", line 786, in ? main() File "/usr/bin/mock", line 781, in main my.close() File "/usr/bin/mock", line 343, in close self._umount_by_file() File "/usr/bin/mock", line 429, in _umount_by_file self._umount(item) File "/usr/bin/mock", line 406, in _umount (retval, output) = self.do(command) File "/usr/bin/mock", line 445, in do return (retval, output) UnboundLocalError: local variable 'retval' referenced before assignment Cleaning up the buildroot... /usr/bin/setarch ppc32 /usr/bin/mock clean --uniqueext=b3cd6c7b101ba3bd91f6b7550f776c7d486e316a -r fedora-development-ppc-core From symbiont at berlios.de Tue Aug 23 08:58:07 2005 From: symbiont at berlios.de (Jeff Pitman) Date: Tue, 23 Aug 2005 16:58:07 +0800 Subject: Traceback after package build In-Reply-To: <430AE1D4.401@redhat.com> References: <430AE1D4.401@redhat.com> Message-ID: <200508231658.07759.symbiont@berlios.de> On Tuesday 23 August 2005 16:44, Warren Togami wrote: > UnboundLocalError: local variable 'retval' referenced before > assignment Cleaning up the buildroot... > ? ? /usr/bin/setarch ppc32 /usr/bin/mock clean Somehow if os.WIFEXITED(status): resulted in FALSE. So, retval was never assigned. Currently, the do() function does not pre-init retval prior to the "if" with a default value. Something like this is needed: def do(self, command): """execute given command outside of chroot""" self.debug("Executing %s" % command) (status, output) = commands.getstatusoutput(command) retval = 0 if os.WIFEXITED(status): retval = os.WEXITSTATUS(status) return (retval, output) -- -jeff From Matt_Domsch at dell.com Tue Aug 23 19:55:38 2005 From: Matt_Domsch at dell.com (Matt Domsch) Date: Tue, 23 Aug 2005 14:55:38 -0500 Subject: [Fedora-packaging] Re: kernel-modules in the buildsystem (Was: Re: Stuff to look for in plague 0.4) In-Reply-To: <1124776767.16972.88.camel@cutter> References: <1124533070.8272.22.camel@localhost.localdomain> <1124774329.16972.75.camel@cutter> <1124776073.30231.15.camel@thl.ct.heise.de> <1124776767.16972.88.camel@cutter> Message-ID: <20050823195538.GA7861@lists.us.dell.com> On Tue, Aug 23, 2005 at 01:59:27AM -0400, seth vidal wrote: > why doesn't it automatically build for whatever the highest installed > kernel-devel or highest installed kernel package is? One of the problems is that the build system doesn't know what kernels an end user may be running. Therefore, you've got to a) be able to have the end user rebuild for whatever kernel they're running (easy, works that way today) b) if the driver is needed at install time, the build system needs to provide pre-builts for the installer (and CD media) kernel. (optionally) c) have the build system build binaries for each of the released errata kernels too, so you don't require a kernel upgrade in order to get a given module that matches. I'm not sure how optional c) really is though. Depends on how easy you want to make life for your system administrators to not have to do a) themselves all the time. -Matt -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com From skvidal at phy.duke.edu Tue Aug 23 21:12:51 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 23 Aug 2005 17:12:51 -0400 Subject: Traceback after package build In-Reply-To: <200508231658.07759.symbiont@berlios.de> References: <430AE1D4.401@redhat.com> <200508231658.07759.symbiont@berlios.de> Message-ID: <1124831571.25024.68.camel@cutter> On Tue, 2005-08-23 at 16:58 +0800, Jeff Pitman wrote: > On Tuesday 23 August 2005 16:44, Warren Togami wrote: > > UnboundLocalError: local variable 'retval' referenced before > > assignment Cleaning up the buildroot... > > /usr/bin/setarch ppc32 /usr/bin/mock clean > > Somehow if os.WIFEXITED(status): resulted in FALSE. So, retval > was never assigned. Currently, the do() function does not pre-init > retval prior to the "if" with a default value. > > Something like this is needed: > > def do(self, command): > """execute given command outside of chroot""" > > self.debug("Executing %s" % command) > (status, output) = commands.getstatusoutput(command) > > retval = 0 > > if os.WIFEXITED(status): > retval = os.WEXITSTATUS(status) > > return (retval, output) > thanks, applied - though its worth noting that people should only ever encounter this when their hardware is failing. :) -sv From wtogami at redhat.com Fri Aug 26 00:32:16 2005 From: wtogami at redhat.com (Warren Togami) Date: Thu, 25 Aug 2005 14:32:16 -1000 Subject: Sign & Push removal from needsign Message-ID: <430E6310.9070704@redhat.com> http://buildsys.fedoraproject.org/build-status/success.psp What event happens and how often does sign & push cause packages from needsign to disappear? About 4 hours after the last sign & push and nothing has disappeared. Warren Togami wtogami at redhat.com From skvidal at phy.duke.edu Fri Aug 26 00:49:26 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Thu, 25 Aug 2005 20:49:26 -0400 Subject: Sign & Push removal from needsign In-Reply-To: <430E6310.9070704@redhat.com> References: <430E6310.9070704@redhat.com> Message-ID: <1125017366.29456.2.camel@cutter> On Thu, 2005-08-25 at 14:32 -1000, Warren Togami wrote: > http://buildsys.fedoraproject.org/build-status/success.psp > > What event happens and how often does sign & push cause packages from > needsign to disappear? About 4 hours after the last sign & push and > nothing has disappeared. > The extras-push script needs to connect to the db or the plague-server and tell it to mark those jobids as 'finished'. the extras-push stuff is in the extras-buildsys cvs if anyone wants to work on it. I've be busy with start of school and closing out this job so I've not had any time. -sv From ivazquez at ivazquez.net Mon Aug 29 10:32:39 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Mon, 29 Aug 2005 06:32:39 -0400 Subject: Patch to put mockbuild results in $WORKDIR In-Reply-To: <1124557562.3414.8.camel@ignacio.lan> References: <1124557562.3414.8.camel@ignacio.lan> Message-ID: <1125311559.19545.3.camel@ignacio.lan> On Sat, 2005-08-20 at 13:06 -0400, Ignacio Vazquez-Abrams wrote: > The attached patch makes the results of mockbuild end up under $WORKDIR. Like? Dislike? Comments? -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From skvidal at phy.duke.edu Mon Aug 29 14:47:11 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Mon, 29 Aug 2005 10:47:11 -0400 Subject: Patch to put mockbuild results in $WORKDIR In-Reply-To: <1125311559.19545.3.camel@ignacio.lan> References: <1124557562.3414.8.camel@ignacio.lan> <1125311559.19545.3.camel@ignacio.lan> Message-ID: <1125326831.28267.2.camel@cutter> On Mon, 2005-08-29 at 06:32 -0400, Ignacio Vazquez-Abrams wrote: > On Sat, 2005-08-20 at 13:06 -0400, Ignacio Vazquez-Abrams wrote: > > The attached patch makes the results of mockbuild end up under $WORKDIR. > > Like? Dislike? Comments? Jeremy has been reminded of it in a round-the-bout way. :) -sv From katzj at redhat.com Mon Aug 29 14:45:28 2005 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 29 Aug 2005 10:45:28 -0400 Subject: Patch to put mockbuild results in $WORKDIR In-Reply-To: <1125311559.19545.3.camel@ignacio.lan> References: <1124557562.3414.8.camel@ignacio.lan> <1125311559.19545.3.camel@ignacio.lan> Message-ID: <1125326729.2576.10.camel@bree.local.net> On Mon, 2005-08-29 at 06:32 -0400, Ignacio Vazquez-Abrams wrote: > On Sat, 2005-08-20 at 13:06 -0400, Ignacio Vazquez-Abrams wrote: > > The attached patch makes the results of mockbuild end up under $WORKDIR. > > Like? Dislike? Comments? "oops, sorry". Thought it was a patch for mock itself and not Makefile.common so didn't look that closely at it. Looks good, applied and thanks. Jeremy From ivazquez at ivazquez.net Tue Aug 30 05:22:05 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Tue, 30 Aug 2005 01:22:05 -0400 Subject: Some mock ideas Message-ID: <1125379325.28096.27.camel@ignacio.lan> I've been running through some ideas in my head over the past couple of days, and here's what I've come up with. 1) Instead of including config files in the base mock package, put them in mock-config-- packages so that mock configs are easy to add via packages. (Corollary: use alternatives for default.cfg) 2) Include the buildgroups data in the config packages. There could also be a /etc/httpd/conf.d/buildgroups.conf file in the base package that would point to the buildgroup data in /usr/share/mock. 3) Make mock use yum as a module. That way you wouldn't need to invoke yum as an external program and you could have more-fine-grained control over it. 4) Have mock be a module that plague-builder could import. See the rationale for the previous point, but replace mock for yum. Any comments? -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From symbiont at berlios.de Tue Aug 30 07:14:20 2005 From: symbiont at berlios.de (Jeff Pitman) Date: Tue, 30 Aug 2005 15:14:20 +0800 Subject: Some mock ideas In-Reply-To: <1125379325.28096.27.camel@ignacio.lan> References: <1125379325.28096.27.camel@ignacio.lan> Message-ID: <200508301514.21320.symbiont@berlios.de> On Tuesday 30 August 2005 13:22, Ignacio Vazquez-Abrams wrote: > 1) Instead of including config files in the base mock package, put > them in mock-config-- packages so that mock > configs are easy to add via packages. (Corollary: use alternatives > for default.cfg) Also, some food for thought. I sent a patch called "flexible config for mock" in June. Since then, I have found execfile() is an excellent tool to organize configs a bit better. (Although, the ~/.mockrc is still a good one to allow the user to override things.) I have setup some configs for pyvault that you can checkout here: https://opensvn.csie.org/traccgi/pyvault/trac.cgi/browser/scripts/config/ Some key things that I did: 1. Take all the repetition out and put it in a config called general.cfg. Using the patch, this is the first to get sourced. You can get the same effect using execfile inside of the dist-specific configs. (The string concat operator is your friend!) 2. Created a pseudo target by execfile fedora-4-core-i386.cfg and renaming things. What it does is uses the same chroot, but changes some rpm macros necessary for the different environments. In my case, I can use it to switch library builds between python 2.2, 2.3, and 2.4. Overkill? maybe. Does it work? yes! :-D Just some ideas, if you need them... -- -jeff From katzj at redhat.com Tue Aug 30 15:51:04 2005 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 30 Aug 2005 11:51:04 -0400 Subject: Some mock ideas In-Reply-To: <1125379325.28096.27.camel@ignacio.lan> References: <1125379325.28096.27.camel@ignacio.lan> Message-ID: <1125417064.7285.13.camel@bree.local.net> On Tue, 2005-08-30 at 01:22 -0400, Ignacio Vazquez-Abrams wrote: > 1) Instead of including config files in the base mock package, put them > in mock-config-- packages so that mock configs > are easy to add via packages. Well, it's easy enough to add more via packages now, there's just an included base set that's considered "sane". The question of sticking them in foo-release came up, but doing it that way means lots of back ones being included. > (Corollary: use alternatives for default.cfg) Just ew. Realistically, I think that you almost always want to specify. default.cfg is mostly a crutch ;) > 2) Include the buildgroups data in the config packages. There could also > be a /etc/httpd/conf.d/buildgroups.conf file in the base package that > would point to the buildgroup data in /usr/share/mock. What's bought by this? Less dependency on the network? Although you still need to download the packages from somewhere. Note that if this was done, no need to use httpd -- just use a file:/// baseurl > 3) Make mock use yum as a module. That way you wouldn't need to invoke > yum as an external program and you could have more-fine-grained control > over it. You have to invoke as a separate program for fun with privilege escalation. Note that we do now exec mock-yum which imports yum directly, so we can make tweaks there. eg, I want to look at changing the db macros to what we use in anaconda as I think that could make chroot building faster. Jeremy From ivazquez at ivazquez.net Tue Aug 30 23:09:27 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Tue, 30 Aug 2005 19:09:27 -0400 Subject: Some mock ideas In-Reply-To: <1125417064.7285.13.camel@bree.local.net> References: <1125379325.28096.27.camel@ignacio.lan> <1125417064.7285.13.camel@bree.local.net> Message-ID: <1125443367.28096.33.camel@ignacio.lan> On Tue, 2005-08-30 at 11:51 -0400, Jeremy Katz wrote: > On Tue, 2005-08-30 at 01:22 -0400, Ignacio Vazquez-Abrams wrote: > > 2) Include the buildgroups data in the config packages. There could also > > be a /etc/httpd/conf.d/buildgroups.conf file in the base package that > > would point to the buildgroup data in /usr/share/mock. > > What's bought by this? Less dependency on the network? Although you > still need to download the packages from somewhere. Note that if this > was done, no need to use httpd -- just use a file:/// baseurl No, you can't. Chrooting causes anything above the chroot to become inaccessible. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From skvidal at phy.duke.edu Wed Aug 31 06:00:49 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Wed, 31 Aug 2005 02:00:49 -0400 Subject: Some mock ideas In-Reply-To: <1125443367.28096.33.camel@ignacio.lan> References: <1125379325.28096.27.camel@ignacio.lan> <1125417064.7285.13.camel@bree.local.net> <1125443367.28096.33.camel@ignacio.lan> Message-ID: <1125468049.9260.111.camel@cutter> On Tue, 2005-08-30 at 19:09 -0400, Ignacio Vazquez-Abrams wrote: > On Tue, 2005-08-30 at 11:51 -0400, Jeremy Katz wrote: > > On Tue, 2005-08-30 at 01:22 -0400, Ignacio Vazquez-Abrams wrote: > > > 2) Include the buildgroups data in the config packages. There could also > > > be a /etc/httpd/conf.d/buildgroups.conf file in the base package that > > > would point to the buildgroup data in /usr/share/mock. > > > > What's bought by this? Less dependency on the network? Although you > > still need to download the packages from somewhere. Note that if this > > was done, no need to use httpd -- just use a file:/// baseurl > > No, you can't. Chrooting causes anything above the chroot to become > inaccessible. > yum runs on the chroot from OUTSIDE the chroot - so it can use the file:/// urls. Nifty trick, that. :) -sv From ivazquez at ivazquez.net Wed Aug 31 06:20:31 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Wed, 31 Aug 2005 02:20:31 -0400 Subject: Some mock ideas In-Reply-To: <1125468049.9260.111.camel@cutter> References: <1125379325.28096.27.camel@ignacio.lan> <1125417064.7285.13.camel@bree.local.net> <1125443367.28096.33.camel@ignacio.lan> <1125468049.9260.111.camel@cutter> Message-ID: <1125469232.28096.39.camel@ignacio.lan> On Wed, 2005-08-31 at 02:00 -0400, seth vidal wrote: > yum runs on the chroot from OUTSIDE the chroot - so it can use the > file:/// urls. Nifty trick, that. :) So it does. Never mind then. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ivazquez at ivazquez.net Wed Aug 31 19:28:13 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Wed, 31 Aug 2005 15:28:13 -0400 Subject: Some mock ideas In-Reply-To: <1125379325.28096.27.camel@ignacio.lan> References: <1125379325.28096.27.camel@ignacio.lan> Message-ID: <1125516493.28096.49.camel@ignacio.lan> On Tue, 2005-08-30 at 01:22 -0400, Ignacio Vazquez-Abrams wrote: > I've been running through some ideas in my head over the past couple of > days, and here's what I've come up with. > > 1) Instead of including config files in the base mock package, put them > in mock-config-- packages so that mock configs > are easy to add via packages. (Corollary: use alternatives for > default.cfg) > > 2) Include the buildgroups data in the config packages. There could also > be a /etc/httpd/conf.d/buildgroups.conf file in the base package that > would point to the buildgroup data in /usr/share/mock. I've created this tarball that includes the saner ideas listed above: http://fedora.ivazquez.net/files/mock/mock-0.4-split.tar.gz -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: