From jstodaro at us.ibm.com Tue Jul 4 13:58:19 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Tue, 4 Jul 2006 09:58:19 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) Message-ID: Hi, I'm having a problem *not* being able to build 'i686' packages (i.e. optional_arches=i686) anymore *after* having upgraded our plague server/builder (Opteron x86_64) a couple of weeks ago from plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 (optional_arches=i686) -- not with i386 (base_arches=i386) which continues to work flawlessly. PLAGUE-0.4.3 / PLAGUE 0.5.0 ARCHES CONFIGURATION: Here's what the Arches section looks like in all the /etc/plague/server/targets/*.cfg files: [Arches] base_arches=i386 optional_arches=i686 noarch PLAGUE-0.4.3 SERVER LOG: Here's an example of how things used to work (snipped from /var/log/plague-server.log) whenever an i686 package-build request was submitted to our PLAGUE-0.4.3 server: Request to enqueue 'e1000' tag '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for target 'ao100' (user 'jstodaro at abc.com') 503 (e1000): Starting tag '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on target 'lnxaddons-100-install' 503 (e1000): Requesting depsolve... 503 (e1000): Starting depsolve for arches: ['i686']. 503 (e1000): Finished depsolve (successful), requesting archjobs. 503 (e1000/i686): https://lnxbuild1.abc.com:8888 - UID is d90078ec928db631ae8f590e6d5491d514cfe4a8 503 (e1000/i686): Build result files - [ 'mockconfig.log', 'build.log', 'root.log', 'kernel-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm', 'job.log', 'e1000-7.0.38-1_rhel4.src.rpm', 'kernel-smp-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm' ] Repo 'lnxaddons-100-install': updating repository metadata... 503 (e1000): Job finished. PLAGUE-0.5.0 SERVER LOG: But here's what happens (snipped from /var/log/plague-server.log) now, whenever the above i686 package-build request gets submitted to our "upgraded" plague server/builder running PLAGUE-0.5.0 (absolutely nothing!) Request to enqueue 'e1000' tag '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for target 'ao100' (user 'jstodaro at abc.com') 508 (e1000): Starting tag '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on target 'lnxaddons-100-install' 508 (e1000): Job finished. OBSERVATIONS: o The "last" function executed in the 'PackageJob.py' module (before it returned to 'BuildMaster.py') was 'arch_handling(self, ba, exclusive, exclude)'. o Adding the following section to /etc/plague/server/targets/*.cfg (including server/builder restart, request resubmit) did *not* help 'PackageJob.py' to progress any further than the 'arch_handling(self, ba, exclusive, exclude)' function. [Additional Package Arches] kernel=i686 o Moving 'i686' from the 'optional_arches' line up to the 'base_arches' line (including server/builder restart, request resubmit) *did* in fact cause 'i686' to be recognized by 'PackageJob.py' (but only as a "base arch" -- not as an "optional arch" like we need it to be) MY QUESTIONS: 1. Why is the *optional_arches* tag -- in the [Arches] section of our /etc/plague/server/*.cfg files -- *no longer* recognized *after* upgrading to plague-0.5.0? 2. What are some things I can try, that might help resolve the above (i.e. getting *plague-0.5.0* to recognize 'i686' as an *optional arch*?) Any help will be much appreciated! .. I have run out of ideas, and things to try... ;-( Thanks, --Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcbw at redhat.com Thu Jul 6 13:59:33 2006 From: dcbw at redhat.com (Dan Williams) Date: Thu, 06 Jul 2006 09:59:33 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1152194373.26411.9.camel@localhost.localdomain> On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > Hi, > I'm having a problem *not* being able to build 'i686' packages (i.e. > optional_arches=i686) anymore *after* having upgraded our plague > server/builder (Opteron x86_64) a couple of weeks ago from > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > (optional_arches=i686) -- not with i386 (base_arches=i386) which > continues to work flawlessly. Sounds like a bug; do the builder logs show anything, i.e. does the server farm the job out to builders? Note that as of Jun 21st, I committed a fix to the builders that was causing stuff that needed setarch to not work correctly, but that doesn't sound like this problem. If the job doesn't go to the builder, there are some known issues where plague will just not farm the job out to anyone if the builder config files aren't just right. The builder doesn't think it supports a particular target and so of course the server won't try to build on that target. Try double-checking your plague-builder targets config files and make _sure_ they support every architecture you want to build on (i386, i686, i586, etc). I'll see if I can look a bit more over the weekend. Dan > PLAGUE-0.4.3 / PLAGUE 0.5.0 ARCHES CONFIGURATION: > Here's what the Arches section looks like in all > the /etc/plague/server/targets/*.cfg files: > [Arches] > base_arches=i386 > optional_arches=i686 noarch > > PLAGUE-0.4.3 SERVER LOG: > Here's an example of how things used to work (snipped > from /var/log/plague-server.log) whenever an i686 package-build > request was submitted to our PLAGUE-0.4.3 server: > Request to enqueue 'e1000' tag > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for > target 'ao100' (user 'jstodaro at abc.com') > 503 (e1000): Starting tag > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on > target 'lnxaddons-100-install' > 503 (e1000): Requesting depsolve... > 503 (e1000): Starting depsolve for arches: ['i686']. > 503 (e1000): Finished depsolve (successful), requesting archjobs. > 503 (e1000/i686): https://lnxbuild1.abc.com:8888 - UID is > d90078ec928db631ae8f590e6d5491d514cfe4a8 > 503 (e1000/i686): Build result files - [ 'mockconfig.log', > 'build.log', 'root.log', > 'kernel-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm', 'job.log', > 'e1000-7.0.38-1_rhel4.src.rpm', > 'kernel-smp-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm' ] > Repo 'lnxaddons-100-install': updating repository metadata... > 503 (e1000): Job finished. > > PLAGUE-0.5.0 SERVER LOG: > But here's what happens (snipped from /var/log/plague-server.log) now, > whenever the above i686 package-build request gets submitted to our > "upgraded" plague server/builder running PLAGUE-0.5.0 (absolutely > nothing!) > Request to enqueue 'e1000' tag > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for > target 'ao100' (user 'jstodaro at abc.com') > 508 (e1000): Starting tag > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on > target 'lnxaddons-100-install' > 508 (e1000): Job finished. > > OBSERVATIONS: > o The "last" function executed in the 'PackageJob.py' module (before > it returned to 'BuildMaster.py') was 'arch_handling(self, ba, > exclusive, exclude)'. > o Adding the following section to /etc/plague/server/targets/*.cfg > (including server/builder restart, request resubmit) did *not* help > 'PackageJob.py' to progress any further than the 'arch_handling(self, > ba, exclusive, exclude)' function. > [Additional Package Arches] > kernel=i686 > o Moving 'i686' from the 'optional_arches' line up to the > 'base_arches' line (including server/builder restart, request > resubmit) *did* in fact cause 'i686' to be recognized by > 'PackageJob.py' (but only as a "base arch" -- not as an "optional > arch" like we need it to be) > > MY QUESTIONS: > 1. Why is the *optional_arches* tag -- in the [Arches] section of > our /etc/plague/server/*.cfg files -- *no longer* recognized *after* > upgrading to plague-0.5.0? > 2. What are some things I can try, that might help resolve the above > (i.e. getting *plague-0.5.0* to recognize 'i686' as an *optional > arch*?) > > Any help will be much appreciated! .. I have run out of ideas, and > things to try... ;-( > > Thanks, > --Joe > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list From jstodaro at us.ibm.com Thu Jul 6 18:59:18 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Thu, 6 Jul 2006 14:59:18 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: <1152194373.26411.9.camel@localhost.localdomain> Message-ID: Thank You, Dan. My answers are below... > Dan Williams wrote on 07/06/2006 09:59:33 AM: > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > > > Hi, > > I'm having a problem *not* being able to build 'i686' packages (i.e. > > optional_arches=i686) anymore *after* having upgraded our plague > > server/builder (Opteron x86_64) a couple of weeks ago from > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > > (optional_arches=i686) -- not with i386 (base_arches=i386) which > > continues to work flawlessly. > > Sounds like a bug; do the builder logs show anything, i.e. does the > server farm the job out to builders? The builder log shows nothing--the job is definitely *not* farmed out to the builder. > Note that as of Jun 21st, I > committed a fix to the builders that was causing stuff that needed > setarch to not work correctly, but that doesn't sound like this problem. I know--I installed your builder/BuilderMock.py updates on June 28th hoping that might help. Also, a few days ago I upgraded Mock to the latest version, mock-0.6-1, hoping that might help. I also forgot to mention sooner that we only use Passive Buliders -- not Active Builders. > > If the job doesn't go to the builder, there are some known issues where > plague will just not farm the job out to anyone if the builder config > files aren't just right. The builder doesn't think it supports a > particular target and so of course the server won't try to build on that > target. Try double-checking your plague-builder targets config files > and make _sure_ they support every architecture you want to build on > (i386, i686, i586, etc). Hopefully these next five sections will help shed more light... - - - - - (1 of 5) Here's what our /etc/plague/builder/plague-builder.cfg file looks like (with hostname/server changes to protect the innocent--that's me;): - - - - - [Active] fileserver_port = 8889 xmlrpc_port = 8888 [Passive] fileserver_port = 8889 xmlrpc_port = 8888 [Directories] target_configs_dir = /etc/plague/builder/targets builder_work_dir = /build/builder_work [SSL] builder_key_and_cert_dir = /etc/plague/builder/certs use_ssl = yes ca_cert = /etc/plague/builder/certs/ca_cert.pem [General] builder_cmd = /usr/bin/mock builder_user = plague-builder hostname = lnxbuild1.abc.com server = lnxbuild1.abc.com debug = yes comm_type = passive - - - - - (2 of 5) Here's what our /etc/plague/builder/targets/lnxaddons-100-i386-install.cfg file looks like (with distro change to protect the innocent): - - - - - [General] distro=lnxaddons target=100 basearch=i386 repo=install mock_config=lnxaddons-100-i386-install - - - - - (3 of 5) Here's what our /etc/mock/lnxaddons-100-i386-install.cfg file looks like (with baseurl changes to protect the innocent): - - - - - #!/usr/bin/python -tt import os config_opts['root'] = 'lnxaddons-100-i386' config_opts['basedir'] = '/var/lib/mock/' config_opts['chroot'] = '/usr/sbin/mock-helper chroot' config_opts['mount'] = '/usr/sbin/mock-helper mount' config_opts['umount'] = '/usr/sbin/mock-helper umount' config_opts['rm'] = '/usr/sbin/mock-helper rm' config_opts['mknod'] = '/usr/sbin/mock-helper mknod' config_opts['yum'] = '/usr/sbin/mock-helper yum' config_opts['runuser'] = '/sbin/runuser' config_opts['buildgroup'] = 'build' config_opts['chrootuser'] = 'aobuild' config_opts['chrootgroup'] = 'aobuild' config_opts['chrootuid'] = os.geteuid() config_opts['chrootgid'] = os.getegid() config_opts['chroothome'] = '/builddir' config_opts['clean'] = True config_opts['target_arch'] = 'i386' config_opts['yum.conf'] = """ [main] cachedir=/var/cache/yum debuglevel=1 logfile=/var/log/yum.log reposdir=/dev/null retries=20 obsoletes=1 gpgcheck=0 assumeyes=1 # repos [install] name=install baseurl=http://.../rhel4/i386/install/ [updates] name=updates baseurl=http://.../rhel4/updates/ [updates-rhel] name=updates-rhel baseurl=http://.../rhel4/updates/ [groups] name=groups baseurl=file:///build/buildgroups/i386/ [plague] name=plague baseurl=file:///build/yum/lnxaddons-100-install/i386/ """ - - - - - (4 of 5) Here's the e-mail that plague sends to my inbox each time I try to build the i686 (optional arch) e1000 package: - - - - - Subject: Prep Error (Job 618): /afs/.../SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm on lnxaddons-100-install Package /afs/.../SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm does not build on any architectures this build system supports. Package: ['i686'] Build System: ['i386', 'i386'] - - - - - (5 of 5) Here are all the arches our builders think they support--i got this info by clicking on the [Builder Status] button of the web interface: - - - - - lnxbuild1.abc.com (0/1) x86_64, amd64, ia32e, noarch, i386, i486, i586, i686, athlon lnxppc.abc.com (0/8) ppc64, noarch > > I'll see if I can look a bit more over the weekend. > > Dan > > > PLAGUE-0.4.3 / PLAGUE 0.5.0 ARCHES CONFIGURATION: > > Here's what the Arches section looks like in all > > the /etc/plague/server/targets/*.cfg files: > > [Arches] > > base_arches=i386 > > optional_arches=i686 noarch > > > > PLAGUE-0.4.3 SERVER LOG: > > Here's an example of how things used to work (snipped > > from /var/log/plague-server.log) whenever an i686 package-build > > request was submitted to our PLAGUE-0.4.3 server: > > Request to enqueue 'e1000' tag > > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for > > target 'ao100' (user 'jstodaro at abc.com') > > 503 (e1000): Starting tag > > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on > > target 'lnxaddons-100-install' > > 503 (e1000): Requesting depsolve... > > 503 (e1000): Starting depsolve for arches: ['i686']. > > 503 (e1000): Finished depsolve (successful), requesting archjobs. > > 503 (e1000/i686): https://lnxbuild1.abc.com:8888 - UID is > > d90078ec928db631ae8f590e6d5491d514cfe4a8 > > 503 (e1000/i686): Build result files - [ 'mockconfig.log', > > 'build.log', 'root.log', > > 'kernel-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm', 'job.log', > > 'e1000-7.0.38-1_rhel4.src.rpm', > > 'kernel-smp-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm' ] > > Repo 'lnxaddons-100-install': updating repository metadata... > > 503 (e1000): Job finished. > > > > PLAGUE-0.5.0 SERVER LOG: > > But here's what happens (snipped from /var/log/plague-server.log) now, > > whenever the above i686 package-build request gets submitted to our > > "upgraded" plague server/builder running PLAGUE-0.5.0 (absolutely > > nothing!) > > Request to enqueue 'e1000' tag > > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for > > target 'ao100' (user 'jstodaro at abc.com') > > 508 (e1000): Starting tag > > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on > > target 'lnxaddons-100-install' > > 508 (e1000): Job finished. > > > > OBSERVATIONS: > > o The "last" function executed in the 'PackageJob.py' module (before > > it returned to 'BuildMaster.py') was 'arch_handling(self, ba, > > exclusive, exclude)'. > > o Adding the following section to /etc/plague/server/targets/*.cfg > > (including server/builder restart, request resubmit) did *not* help > > 'PackageJob.py' to progress any further than the 'arch_handling(self, > > ba, exclusive, exclude)' function. > > [Additional Package Arches] > > kernel=i686 > > o Moving 'i686' from the 'optional_arches' line up to the > > 'base_arches' line (including server/builder restart, request > > resubmit) *did* in fact cause 'i686' to be recognized by > > 'PackageJob.py' (but only as a "base arch" -- not as an "optional > > arch" like we need it to be) > > > > MY QUESTIONS: > > 1. Why is the *optional_arches* tag -- in the [Arches] section of > > our /etc/plague/server/*.cfg files -- *no longer* recognized *after* > > upgrading to plague-0.5.0? > > 2. What are some things I can try, that might help resolve the above > > (i.e. getting *plague-0.5.0* to recognize 'i686' as an *optional > > arch*?) > > > > Any help will be much appreciated! .. I have run out of ideas, and > > things to try... ;-( > > > > Thanks, > > --Joe > > -- > > Fedora-buildsys-list mailing list > > Fedora-buildsys-list at redhat.com > > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list > -Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From tibbs at math.uh.edu Fri Jul 14 17:22:15 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 12:22:15 -0500 Subject: Running rpmlint within mock Message-ID: Christian Iseli and I were discussing the possibility of automatically running rpmlint somehow. It seems that the end of the mock build process is the ideal place for this. It has a chroot already set up with the package's build requirements already installed. (Obviously this doesn't include the runtime requirements, but generally there's quite some overlap.) It also has easy access to the freshly built binary and source RPMs. How difficult would it be to, at the end of the build process, install the freshly built package, install rpmlint, and run rpmlint on the source and any binary RPMs that were built? My python is poor but I'm willing to take a stab at it if someone could give me a few pointers. - J< From andreas at bawue.net Fri Jul 14 18:46:59 2006 From: andreas at bawue.net (Andreas Thienemann) Date: Fri, 14 Jul 2006 20:46:59 +0200 (CEST) Subject: Running rpmlint within mock In-Reply-To: References: Message-ID: Hi tibbs, On Fri, 14 Jul 2006, Jason L Tibbitts III wrote: > How difficult would it be to, at the end of the build process, install > the freshly built package, install rpmlint, and run rpmlint on the > source and any binary RPMs that were built? Installing the freshly built package is a big no. But it's not even necessary for running rpmlint. You could run rpmlint on the resulting rpms easily. regards, andreas From williams at redhat.com Fri Jul 14 18:46:52 2006 From: williams at redhat.com (Clark Williams) Date: Fri, 14 Jul 2006 13:46:52 -0500 Subject: Running rpmlint within mock In-Reply-To: References: Message-ID: <44B7E69C.6050601@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jason L Tibbitts III wrote: > Christian Iseli and I were discussing the possibility of automatically > running rpmlint somehow. It seems that the end of the mock build > process is the ideal place for this. It has a chroot already set up > with the package's build requirements already installed. (Obviously > this doesn't include the runtime requirements, but generally there's > quite some overlap.) It also has easy access to the freshly built > binary and source RPMs. > > How difficult would it be to, at the end of the build process, install > the freshly built package, install rpmlint, and run rpmlint on the > source and any binary RPMs that were built? Sounds like a good idea. Why do we need to install the just-generated binary RPMs in the chroot? Can't we get the same info by just running rpmlint against the binary and source RPMs? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEt+abHyuj/+TTEp0RAvtPAJ4iHlPJFP2ofetHfwFGrSSKuv1wogCcDn2u 8Rv7tZiv3JZkXadh9+mK/Es= =/0dA -----END PGP SIGNATURE----- From Christian.Iseli at licr.org Fri Jul 14 19:01:19 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Fri, 14 Jul 2006 21:01:19 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Fri, 14 Jul 2006 12:22:15 CDT." Message-ID: <200607141901.k6EJ1N3E016098@mx3.redhat.com> tibbs at math.uh.edu said: > Christian Iseli and I were discussing the possibility of automatically > running rpmlint somehow. Hey, thanks for posting this... I was just about to post such a message myself :-) I've tried to give some thought and get some feedback from the f-e-l list already. Basically for each built package, I'd like to - get rpmlint output, possibly comparing it against a reference output to see if unknown problems have crept in - analyze the compiler warnings - check the Provide'd things from the packages and compare against an expected set of things provided For starters, the whole bunch of tests would just generate a warning to the build submitter for things to verify. Down the road, we might like to fail the build, or put the resulting package in a quarantine, if bad things are detected. I had a first look at adding the checks in the plague builder, but after discussing this a bit with Jason, probably a good parts of the checks would be better performed while still in the mock buildroot... My python skills are pretty much nil ATM, but I'm willing to learn too... Any and all pieces of advice and/or code gladly accepted Cheers, Christian From jkeating at j2solutions.net Fri Jul 14 19:20:16 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 15:20:16 -0400 Subject: Running rpmlint within mock In-Reply-To: References: Message-ID: <200607141520.20634.jkeating@j2solutions.net> On Friday 14 July 2006 13:22, Jason L Tibbitts III wrote: > How difficult would it be to, at the end of the build process, install > the freshly built package, install rpmlint, and run rpmlint on the > source and any binary RPMs that were built? Why do you need to install the package? rpmlint works on the binary and srpm. I think we really need to separate the idea of Building and Testing. Mock is a build tool, not a test tool. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From williams at redhat.com Fri Jul 14 19:21:49 2006 From: williams at redhat.com (Clark Williams) Date: Fri, 14 Jul 2006 14:21:49 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141901.k6EJ1N3E016098@mx3.redhat.com> References: <200607141901.k6EJ1N3E016098@mx3.redhat.com> Message-ID: <44B7EECD.5020805@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christian.Iseli at licr.org wrote: > tibbs at math.uh.edu said: >> Christian Iseli and I were discussing the possibility of automatically >> running rpmlint somehow. > > Hey, thanks for posting this... I was just about to post such a message > myself :-) > > I've tried to give some thought and get some feedback from the f-e-l list > already. > > Basically for each built package, I'd like to > - get rpmlint output, possibly comparing it against a reference output to see > if unknown problems have crept in Getting the rpmlint output is easy enough. Doing anything more than that sounds like something out of the scope of mock. > - analyze the compiler warnings > - check the Provide'd things from the packages and compare against an > expected set of things provided > > For starters, the whole bunch of tests would just generate a warning to the > build submitter for things to verify. Ah, we're talking automated test suite here. That's not mock. > > Down the road, we might like to fail the build, or put the resulting package > in a quarantine, if bad things are detected. > > I had a first look at adding the checks in the plague builder, but after > discussing this a bit with Jason, probably a good parts of the checks would be > better performed while still in the mock buildroot... > > My python skills are pretty much nil ATM, but I'm willing to learn too... > > Any and all pieces of advice and/or code gladly accepted I applaud your desire to automate this stuff, but I don't think that mock is the right place for it. Mock is strictly a tool for managing a chroot and building SRPMs in those chroots. Anything else just introduces the potential for breaking builds and then Jeremy and Dan start sending me nasty emails :). That being said, I think it would be fairly straightforward to run rpmlint on the generated RPMs and on the SRPM and stash the output somewhere that's easily accessible. At that point you're talking about writing an analysis tool to classify the output of a mock build. Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEt+7NHyuj/+TTEp0RAuVPAKCKRg/kINpep85DA3F7OXFA+nUi6gCgnPsd AfZkfqe9NkWCDNzngIHLqZ0= =nvO5 -----END PGP SIGNATURE----- From notting at redhat.com Fri Jul 14 19:49:49 2006 From: notting at redhat.com (Bill Nottingham) Date: Fri, 14 Jul 2006 15:49:49 -0400 Subject: Running rpmlint within mock In-Reply-To: <200607141901.k6EJ1N3E016098@mx3.redhat.com> References: <200607141901.k6EJ1N3E016098@mx3.redhat.com> Message-ID: <20060714194949.GL19808@nostromo.devel.redhat.com> Christian.Iseli at licr.org (Christian.Iseli at licr.org) said: > My python skills are pretty much nil ATM, but I'm willing to learn too... > > Any and all pieces of advice and/or code gladly accepted Sounds like a good idea to me... I suggested some other things to check on this list a few months back. Bill From Christian.Iseli at licr.org Fri Jul 14 20:11:20 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Fri, 14 Jul 2006 22:11:20 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Fri, 14 Jul 2006 13:46:52 CDT." <44B7E69C.6050601@redhat.com> Message-ID: <200607142011.k6EKBKHC011621@mx1.redhat.com> williams at redhat.com said: > Why do we need to install the just-generated binary RPMs in the chroot? AFAIU, rpmlint can catch more/other things when a package is installed. So the full thing seems to be: 1 rpmlint the src.rpm 2 rpmlint the .rpm 3 install the rpm and then run rpmlint again Since mock builds a chroot anyway, Jason and I thought it might not cost much more just to install the built packages and do an rpmlint then Christian From Christian.Iseli at licr.org Fri Jul 14 20:12:13 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Fri, 14 Jul 2006 22:12:13 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Fri, 14 Jul 2006 20:46:59 +0200." Message-ID: <200607142012.k6EKCLBe012006@mx1.redhat.com> andreas at bawue.net said: > Installing the freshly built package is a big no. May I ask why ? Christian From Christian.Iseli at licr.org Fri Jul 14 20:16:56 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Fri, 14 Jul 2006 22:16:56 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Fri, 14 Jul 2006 14:21:49 CDT." <44B7EECD.5020805@redhat.com> Message-ID: <200607142016.k6EKGvJ4013336@mx1.redhat.com> williams at redhat.com said: > Getting the rpmlint output is easy enough. Doing anything more than that > sounds like something out of the scope of mock. Agreed. > Ah, we're talking automated test suite here. Yup. > That's not mock. Ok. > I applaud your desire to automate this stuff, but I don't think that mock is > the right place for it. Mock is strictly a tool for managing a chroot and > building SRPMs in those chroots. Anything else just introduces the potential > for breaking builds and then Jeremy and Dan start sending me nasty emails :). :) > That being said, I think it would be fairly straightforward to run rpmlint on > the generated RPMs and on the SRPM and stash the output somewhere that's > easily accessible. Yes, that'd be neat. > At that point you're talking about writing an analysis > tool to classify the output of a mock build. Yes. I'd like to add that to plague, or as a standalone thing that is called by plague after mock succeeds. Christian From Christian.Iseli at licr.org Fri Jul 14 20:18:50 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Fri, 14 Jul 2006 22:18:50 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Fri, 14 Jul 2006 15:49:49 EDT." <20060714194949.GL19808@nostromo.devel.redhat.com> Message-ID: <200607142018.k6EKIoVH013875@mx1.redhat.com> notting at redhat.com said: > Sounds like a good idea to me... I suggested some other things to check on > this list a few months back. Yes. I hope I put them all on the wiki here: http://fedoraproject.org/wiki/Extras/SIGs/QA Christian From jkeating at j2solutions.net Fri Jul 14 20:36:24 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 16:36:24 -0400 Subject: Running rpmlint within mock In-Reply-To: <200607142011.k6EKBKHC011621@mx1.redhat.com> References: <200607142011.k6EKBKHC011621@mx1.redhat.com> Message-ID: <200607141636.24337.jkeating@j2solutions.net> On Friday 14 July 2006 16:11, Christian.Iseli at licr.org wrote: > AFAIU, rpmlint can catch more/other things when a package is installed. > So the full thing seems to be: > ?1 rpmlint the src.rpm > ?2 rpmlint the .rpm > ?3 install the rpm and then run rpmlint again > > Since mock builds a chroot anyway, Jason and I thought it might not cost > much more just to install the built packages and do an rpmlint then This sounds like much more of a job of the Fedora Test Project software that Will Woods is talking about. Then you could do more tests like testing for conflicts, testing for scriptlet errors, testing for upgrade path / removal, etc.. A much more appropriate place for it. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tibbs at math.uh.edu Fri Jul 14 22:48:03 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 17:48:03 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141520.20634.jkeating@j2solutions.net> (Jesse Keating's message of "Fri, 14 Jul 2006 15:20:16 -0400") References: <200607141520.20634.jkeating@j2solutions.net> Message-ID: >>>>> "JK" == Jesse Keating writes: JK> Why do you need to install the package? rpmlint works on the JK> binary and srpm. Because I was told that is the only way rpmlint can run certain checks. The point is that in order to do a proper review, I should install the package but I just cannot install everything I review on my build host. JK> I think we really need to separate the idea of Building and JK> Testing. Mock is a build tool, not a test tool. The whole point is that mock already has everything necessary and pregenerated. It has the chroot just sitting there. It has the security infrastructure and the procedures for doing things within the chroot already done. It seems to me to be terribly foolish to duplicate that. How about this: does mock provide (or could it provide) a way for me to install a couple of packages and run a command within the chroot? Or is this relatively simple to do safely without using mock? Forget about rpmlint; maybe I need this for debugging or to figure out why something isn't building. - J< From toshio at tiki-lounge.com Fri Jul 14 22:57:27 2006 From: toshio at tiki-lounge.com (Toshio Kuratomi) Date: Fri, 14 Jul 2006 15:57:27 -0700 Subject: Running rpmlint within mock In-Reply-To: <200607142012.k6EKCLBe012006@mx1.redhat.com> References: <200607142012.k6EKCLBe012006@mx1.redhat.com> Message-ID: <1152917848.3010.220.camel@localhost> On Fri, 2006-07-14 at 22:12 +0200, Christian.Iseli at licr.org wrote: > andreas at bawue.net said: > > Installing the freshly built package is a big no. > > May I ask why ? I don't know what Andras had in mind but one potential problem is that BuildRequires might not pull in all of a package's Requires. Noarch packages would be the primary place you'll see this. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From tibbs at math.uh.edu Fri Jul 14 23:01:56 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 18:01:56 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141636.24337.jkeating@j2solutions.net> (Jesse Keating's message of "Fri, 14 Jul 2006 16:36:24 -0400") References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> Message-ID: >>>>> "JK" == Jesse Keating writes: JK> This sounds like much more of a job of the Fedora Test Project JK> software that Will Woods is talking about. I'm not sure how this would help with reviews of packages which haven't yet been accepted into Fedora. The idea is to lower the barrier to providing a good package review. - J< From tibbs at math.uh.edu Fri Jul 14 23:03:55 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 18:03:55 -0500 Subject: Running rpmlint within mock In-Reply-To: <1152917848.3010.220.camel@localhost> (Toshio Kuratomi's message of "Fri, 14 Jul 2006 15:57:27 -0700") References: <200607142012.k6EKCLBe012006@mx1.redhat.com> <1152917848.3010.220.camel@localhost> Message-ID: >>>>> "TK" == Toshio Kuratomi writes: TK> I don't know what Andras had in mind but one potential problem is TK> that BuildRequires might not pull in all of a package's Requires. TK> Noarch packages would be the primary place you'll see this. Obviously there's a working yum and a full set of package repositories available, else the chroot wouldn't have been populated in the first place. My point was only that the majority of package dependencies are likely to already be installed in the chroot. It should be trivial to get the rest with 'yum localinstall'. - J< From jkeating at j2solutions.net Fri Jul 14 23:04:02 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 19:04:02 -0400 Subject: Running rpmlint within mock In-Reply-To: References: <200607141520.20634.jkeating@j2solutions.net> Message-ID: <200607141904.05296.jkeating@j2solutions.net> On Friday 14 July 2006 18:48, Jason L Tibbitts III wrote: > The whole point is that mock already has everything necessary and > pregenerated. ?It has the chroot just sitting there. ?It has the > security infrastructure and the procedures for doing things within the > chroot already done. ?It seems to me to be terribly foolish to > duplicate that. > > How about this: does mock provide (or could it provide) a way for me > to install a couple of packages and run a command within the chroot? > Or is this relatively simple to do safely without using mock? ?Forget > about rpmlint; maybe I need this for debugging or to figure out why > something isn't building. The same software could be used. Mock can do it, it just isn't designed with this task in mind. More to the point, there are two distinct tasks here, one is building the package, the other is testing. They have some pretty different needs, and trying to do both of them in the same harness isn't the best of ideas. rpmlint is just a test, it leads to more tests, like I described. What about doing functional tests on things like php packages, where we want to spin up a local apache, browse to some pages, and make sure the content looks right. These are the types of things that can be done with dogtail in a real test environment that really really really shouldn't be done at build time. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tibbs at math.uh.edu Fri Jul 14 23:05:12 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 18:05:12 -0500 Subject: Running rpmlint within mock In-Reply-To: (Jason L. Tibbitts, III's message of "Fri, 14 Jul 2006 17:48:03 -0500") References: <200607141520.20634.jkeating@j2solutions.net> Message-ID: >>>>> "JLT" == Jason L Tibbitts, writes: JLT> Because I was told that is the only way rpmlint can run certain JLT> checks. For an example of a check that only works if you actually install the package, see: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=198835#c6 - J< From tibbs at math.uh.edu Fri Jul 14 23:14:27 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 18:14:27 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141904.05296.jkeating@j2solutions.net> (Jesse Keating's message of "Fri, 14 Jul 2006 19:04:02 -0400") References: <200607141520.20634.jkeating@j2solutions.net> <200607141904.05296.jkeating@j2solutions.net> Message-ID: >>>>> "JK" == Jesse Keating writes: JK> More to the point, there are two distinct tasks here, one is JK> building the package, the other is testing. Well, what I'm getting out of this is that you're telling me to ignore what is pretty much a perfect time-saver in the form of existing code and an existing populated chroot, and instead bring things up from scratch because there's a philosophical issue with using any of the "build" infrastructure for anything relating to testing. I guess I should never have mentioned rpmlint and just asked for a way to install a package and run a command inside the chroot via the mock infrastructure. >From my point it's all testing, because I'm testing the package submission by seeing if it builds. So perhaps that's why I'm just not seeing it the same as you. But OK, I'll just either rip out the bits of mock I think I can use, or just give up and write my own stuff that uses the chroot that mock leaves me. But it sure would have been nice to get a quick hint and perhaps a little help from someone who knows the insides of mock. - J< From dmalcolm at redhat.com Fri Jul 14 23:37:38 2006 From: dmalcolm at redhat.com (David Malcolm) Date: Fri, 14 Jul 2006 19:37:38 -0400 Subject: Running rpmlint within mock In-Reply-To: <200607141904.05296.jkeating@j2solutions.net> References: <200607141520.20634.jkeating@j2solutions.net> <200607141904.05296.jkeating@j2solutions.net> Message-ID: <1152920258.19346.34.camel@cassandra.boston.redhat.com> On Fri, 2006-07-14 at 19:04 -0400, Jesse Keating wrote: > On Friday 14 July 2006 18:48, Jason L Tibbitts III wrote: > > The whole point is that mock already has everything necessary and > > pregenerated. It has the chroot just sitting there. It has the > > security infrastructure and the procedures for doing things within the > > chroot already done. It seems to me to be terribly foolish to > > duplicate that. > > > > How about this: does mock provide (or could it provide) a way for me > > to install a couple of packages and run a command within the chroot? > > Or is this relatively simple to do safely without using mock? Forget > > about rpmlint; maybe I need this for debugging or to figure out why > > something isn't building. > > The same software could be used. Mock can do it, it just isn't designed with > this task in mind. > > More to the point, there are two distinct tasks here, one is building the > package, the other is testing. They have some pretty different needs, and > trying to do both of them in the same harness isn't the best of ideas. > rpmlint is just a test, it leads to more tests, like I described. What about > doing functional tests on things like php packages, where we want to spin up > a local apache, browse to some pages, and make sure the content looks right. > These are the types of things that can be done with dogtail in a real test > environment that really really really shouldn't be done at build time. I like to classify testing into two types: static vs dynamic When people refer to "static analysis" they're normally thinking of e.g. a new compiler plugin which applies some semantic rules to the parse tree and e.g. decides that you're mutex locking is dodgy (or whatever), which normally involves either a build-time test, or a rebuild. I like to generalize the term to any kind of test that doesn't require you to actually run the code (or install it, for that matter). This can include the tests that rpmlint already performs, or all kinds of other goodness (either on the metadata, or perhaps on the files in the package payload itself). So this can happen at some arbitrary time after the package is built - perhaps as an automatic postprocessing step within the build system (for both Core and Extras I hope). So, by contrast, tests that require you to actually run the code, I call "dynamic tests" - be it a functional test of the kind Jesse described, or a simple "does the app start without coredumping" test; do upgrades and downgrades work? (exercising the RPM scriptlets) etc We're working on opening up parts of our dynamic testing framework and our dynamic tests, as Will Woods has described in other emails. I'd love it if we could hook up a static testing framework as well - and I'm not sure exactly where it would sit (mock, plague, somewhere else). The ideas on the wiki page for a pluggable analysis framework look great. Hope this helps Dave From jkeating at j2solutions.net Fri Jul 14 23:43:12 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 19:43:12 -0400 Subject: Running rpmlint within mock In-Reply-To: References: <200607141904.05296.jkeating@j2solutions.net> Message-ID: <200607141943.17065.jkeating@j2solutions.net> On Friday 14 July 2006 19:14, Jason L Tibbitts III wrote: > >From my point it's all testing, because I'm testing the package > > submission by seeing if it builds. ?So perhaps that's why I'm just not > seeing it the same as you. ?But OK, I'll just either rip out the bits > of mock I think I can use, or just give up and write my own stuff that > uses the chroot that mock leaves me. ?But it sure would have been nice > to get a quick hint and perhaps a little help from someone who knows > the insides of mock. You should be testing your package in a local mock rather than abusing the buildsystem resources to just test a build. We deal with this internally too. I'm not saying you couldn't use the mock software as is, in a different harness. Internally we split the majority of testing away from building to not slog down our build system with running excessive tests which really belong in a QA phase. We do have some infrastructure to run some tests that are in the 'tests/' branch of a package module. However this is limited and has caused problems in our build environment when some packages like to bind to local ports (doesn't work too well when you have multiple builds happening at the same time, whoops that part is already in use.... However these are for limited tests. I also think that it is strange that we have tests here and also in our testing infrastructure. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From jkeating at j2solutions.net Fri Jul 14 23:48:34 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 19:48:34 -0400 Subject: Running rpmlint within mock In-Reply-To: References: <200607142012.k6EKCLBe012006@mx1.redhat.com> <1152917848.3010.220.camel@localhost> Message-ID: <200607141948.34580.jkeating@j2solutions.net> On Friday 14 July 2006 19:03, Jason L Tibbitts III wrote: > Obviously there's a working yum and a full set of package repositories > available, else the chroot wouldn't have been populated in the first > place. ?My point was only that the majority of package dependencies > are likely to already be installed in the chroot. ?It should be > trivial to get the rest with 'yum localinstall'. The working yum is outside the mockroot. Mock uses mockhelper to yum --installroot packages. So you'd have to leave the root, localinstall, and re-enter the chroot. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From jkeating at j2solutions.net Fri Jul 14 23:52:44 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 19:52:44 -0400 Subject: Running rpmlint within mock In-Reply-To: References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> Message-ID: <200607141952.44898.jkeating@j2solutions.net> On Friday 14 July 2006 19:01, Jason L Tibbitts III wrote: > I'm not sure how this would help with reviews of packages which > haven't yet been accepted into Fedora. ?The idea is to lower the > barrier to providing a good package review. Um, if the package isn't accepted into Fedora, it wouldn't be ran through the extras build system.... I guess thats part of where my argument is coming from. I incorrectly assumed you wanted this put into the Extras build system, not mock itself. Sorry about that (; mock chroot will allow you to run a comand in the chroot. mockhelper yum --installroot /root localinstall /rpm mock --noclean (needed?) -r config chroot rpmlint -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tibbs at math.uh.edu Sat Jul 15 01:52:16 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 20:52:16 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141943.17065.jkeating@j2solutions.net> (Jesse Keating's message of "Fri, 14 Jul 2006 19:43:12 -0400") References: <200607141904.05296.jkeating@j2solutions.net> <200607141943.17065.jkeating@j2solutions.net> Message-ID: >>>>> "JK" == Jesse Keating writes: JK> You should be testing your package in a local mock rather than JK> abusing the buildsystem resources to just test a build. We deal JK> with this internally too. I'm not sure where I gave the impression that this had anything to do with the buildsystem, but I certainly didn't mean to. I review lots of Fedora Extras package submissions; they aren't yet in CVS and can't possibly be built on the buildsystem machines. I'm building them myself, on my builder machines. The whole discussion is about doing something extra with mock itself, and has nothing to do with plague or with the actual Extras buildsys. It is for the purpose of doing more complete and accurate package reviews that I'd like to get at the mock chroot, install a couple of extra packages into it and run an arbitrary command or two. This is all running on my machines (or the local machines of the Extras reviewers, whose lives I would like to make just a bit easier). FESCo has charged me with getting more reviews done and improving the quality of reviews; this is one of the many things I'm looking at. - J< From tibbs at math.uh.edu Sat Jul 15 02:10:56 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Fri, 14 Jul 2006 21:10:56 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141952.44898.jkeating@j2solutions.net> (Jesse Keating's message of "Fri, 14 Jul 2006 19:52:44 -0400") References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> <200607141952.44898.jkeating@j2solutions.net> Message-ID: >>>>> "JK" == Jesse Keating writes: JK> mock chroot will allow you to run a comand in the chroot. Oh, awesome. Honestly, very many thanks. Might I be so bold as to suggest something like the following patch? Index: mock.py =================================================================== RCS file: /cvs/fedora/mock/mock.py,v retrieving revision 1.54 diff -u -r1.54 mock.py --- mock.py 28 Jun 2006 15:14:45 -0000 1.54 +++ mock.py 15 Jul 2006 02:08:57 -0000 @@ -791,6 +791,7 @@ usage = """ usage: mock [options] /path/to/srpm optional commands: + chroot - run the specified command within the chroot clean - clean out the specified chroot init - initialize the chroot, do not build anything""" parser = OptionParser(usage=usage, version=__VERSION__) cvs diff: Diffing docs Index: docs/mock.1 =================================================================== RCS file: /cvs/fedora/mock/docs/mock.1,v retrieving revision 1.2 diff -u -r1.2 mock.1 --- docs/mock.1 21 Aug 2005 16:29:32 -0000 1.2 +++ docs/mock.1 15 Jul 2006 02:08:57 -0000 @@ -37,6 +37,8 @@ .TP \fBinit\fR \- initialize a chroot (install packages, setup devices, etc.) .TP +\fBchroot\fR \- run the specified command within the chroot (which must already be initialized) +.TP \fBclean\fR \- purge the chroot tree .TP \fBrebuild\fR \- If no command is specified, rebuild is assumed. Rebuilds the specified SRPM - J< From jkeating at j2solutions.net Sat Jul 15 02:23:20 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Fri, 14 Jul 2006 22:23:20 -0400 Subject: Running rpmlint within mock In-Reply-To: References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141952.44898.jkeating@j2solutions.net> Message-ID: <200607142223.24253.jkeating@j2solutions.net> On Friday 14 July 2006 22:10, Jason L Tibbitts III wrote: > Oh, awesome. ?Honestly, very many thanks. ?Might I be so bold as to > suggest something like the following patch? Looks reasonable to me. Alas, I have no commit access (: (maybe that's a good thing...) -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tibbs at math.uh.edu Sat Jul 15 05:49:53 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Sat, 15 Jul 2006 00:49:53 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607141952.44898.jkeating@j2solutions.net> (Jesse Keating's message of "Fri, 14 Jul 2006 19:52:44 -0400") References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> <200607141952.44898.jkeating@j2solutions.net> Message-ID: OK, here's a first pass hack. This could be better written in Python and could extract some bits from the mock configuration, but this does at least work. It builds the package, installs rpmlint and all of the newly-built binary RPMs and (hopefully) their dependencies into the chroot. It then runs rpmlint on the SRPM, the binary RPMs and the installed package names. I have tested this against packages which show different warnings for each and it seems to actually work. You'll have to edit to taste, and try not to hurl as some of the quoting is nasty. - J< #!/bin/sh MOCKDIR=/var/lib/mock/fedora-development-x86_64-core MOCKCFG=fedora-6-x86_64-core.cfg time mock --autocache --debug -r $MOCKCFG $* echo ======= mock-helper yum --installroot $MOCKDIR/root install rpmlint echo ======= mock-helper yum --installroot $MOCKDIR/root localinstall $MOCKDIR/result/*{i386,x86_64,noarch}.rpm echo ======= echo rpmlint of SRPM: mock -r $MOCKCFG --debug -- chroot ls -l /builddir/build/SRPMS/\*.rpm \ | grep -E -v '^(init$|ending$|done$|DEBUG:)' \ | tee $MOCKDIR/result/rpmlint mock -r $MOCKCFG --debug -- chroot rpmlint /builddir/build/SRPMS/\*.rpm \ | grep -E -v '^(init$|ending$|done$|DEBUG:)' \ | tee -a $MOCKDIR/result/rpmlint echo ====== echo rpmlint of RPMs: mock -r $MOCKCFG --debug -- chroot ls -l /builddir/build/RPMS/\*.rpm \ | grep -E -v '^(init$|ending$|done$|DEBUG:)' \ | tee -a $MOCKDIR/result/rpmlint mock -r $MOCKCFG --debug -- chroot rpmlint /builddir/build/RPMS/\*.rpm \ | grep -E -v '^(init$|ending$|done$|DEBUG:)' \ | tee -a $MOCKDIR/result/rpmlint echo ====== echo rpmlint of installed RPMs: mock -r $MOCKCFG --debug -- chroot rpmlint \\\`rpm -qp /builddir/build/RPMS/\*.rpm --qf \\\"%{NAME} \\\"\\\` \ | grep -E -v '^(init$|ending$|done$|DEBUG:)' \ | tee -a $MOCKDIR/result/rpmlint echo ====== echo Results in $MOCKDIR/result From Christian.Iseli at licr.org Sat Jul 15 09:08:26 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Sat, 15 Jul 2006 11:08:26 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Fri, 14 Jul 2006 20:52:16 CDT." Message-ID: <200607150908.k6F98RnV016271@mx1.redhat.com> tibbs at math.uh.edu said: > The whole discussion is about doing something extra with mock itself, and has > nothing to do with plague or with the actual Extras buildsys. It was probably me who gave that impression. Sorry about that. I would actually like the buildsys to run rpmlint on built packages. But I'll try to ponder options a bit more, and start a new thread when I feel ready :-) Cheers, Christian From bugs.michael at gmx.net Sat Jul 15 10:21:57 2006 From: bugs.michael at gmx.net (Michael Schwendt) Date: Sat, 15 Jul 2006 12:21:57 +0200 Subject: Running rpmlint within mock In-Reply-To: References: <200607141904.05296.jkeating@j2solutions.net> <200607141943.17065.jkeating@j2solutions.net> Message-ID: <20060715122157.d22ac9c3.bugs.michael@gmx.net> On Fri, 14 Jul 2006 20:52:16 -0500, Jason L Tibbitts III wrote: > It is for the purpose of doing more complete and accurate package > reviews that I'd like to get at the mock chroot, install a couple of > extra packages into it and run an arbitrary command or two. This is > all running on my machines (or the local machines of the Extras > reviewers, whose lives I would like to make just a bit easier). FESCo > has charged me with getting more reviews done and improving the > quality of reviews; this is one of the many things I'm looking at. Sounds more like you want "yum install mach" and continue there. From andreas at bawue.net Sat Jul 15 11:28:21 2006 From: andreas at bawue.net (Andreas Thienemann) Date: Sat, 15 Jul 2006 13:28:21 +0200 (CEST) Subject: Running rpmlint within mock In-Reply-To: <200607142012.k6EKCLBe012006@mx1.redhat.com> References: <200607142012.k6EKCLBe012006@mx1.redhat.com> Message-ID: On Fri, 14 Jul 2006 Christian.Iseli at licr.org wrote: > > Installing the freshly built package is a big no. > May I ask why ? sure. mock is a chrooted buildtool. Installing the package outside of the chroot is completely senseless. We are building quite a large amount of packages on rhel4, installing a fc5 package is sure to break or host the system. Not gonna happen. Or even neater: I know that there are at least two people building rpms with mock on a debian host. regards, andreas From Christian.Iseli at licr.org Sat Jul 15 12:37:42 2006 From: Christian.Iseli at licr.org (Christian.Iseli at licr.org) Date: Sat, 15 Jul 2006 14:37:42 +0200 Subject: Running rpmlint within mock In-Reply-To: Your message of "Sat, 15 Jul 2006 13:28:21 +0200." Message-ID: <200607151237.k6FCbhue026333@mx1.redhat.com> andreas at bawue.net said: > Installing the package outside of the chroot is completely senseless. I completely agree, but I was thinking about installing the package *within* the chroot, not on the build host's root... Is that also a big no ? Christian From dcbw at redhat.com Sat Jul 15 17:20:54 2006 From: dcbw at redhat.com (Dan Williams) Date: Sat, 15 Jul 2006 13:20:54 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1152984054.3984.4.camel@localhost.localdomain> On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > Hi, > I'm having a problem *not* being able to build 'i686' packages (i.e. > optional_arches=i686) anymore *after* having upgraded our plague > server/builder (Opteron x86_64) a couple of weeks ago from > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > (optional_arches=i686) -- not with i386 (base_arches=i386) which > continues to work flawlessly. Found and fixed in CVS HEAD. Patch attached that should apply against an installed copy of plague-server as well. Thanks for the report, and sorry for the lag. Cheers, Dan -------------- next part -------------- A non-text attachment was scrubbed... Name: plague-opt-arches-fix.patch Type: text/x-patch Size: 535 bytes Desc: not available URL: From jstodaro at us.ibm.com Sun Jul 16 13:48:09 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Sun, 16 Jul 2006 09:48:09 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: <1152984054.3984.4.camel@localhost.localdomain> Message-ID: Thanks Dan. So I ran a quick regression test, just to make sure that *i386* is still working, but, it *failed* ... (also note that i686 gave basically the same errors..) SEE BELOW for *regression* test results, both BEFORE and AFTER applying patch 'plague-opt-arches-fix.patch' to server/Config.py. I repeated that test sequence several times, by the way. Also, I was surprised to see that 'WhoAmI' was trying to build as an 'i686' rpm rather than a 'noarch' which is what it really is. So, since I can't be exactly sure at this point how the "optional_arches" design is supposed to tie-in with the "base_arches" design, it doesn't make much sense for me (with my limited Python skills) to try and dig in any further.. Also, could you please explain/elaborate when you say: "Patch attached that should apply against an installed copy of plague-server as well." Um, your patch is clearly intended to update file server/Config.py file, so I really don't understand how I can also use *it* to update file *plague-server* (/usr/bin/plague-server?) as well, which appears to be a start-up script -- am I missing something here? For the good news (for me anyway), I'm also trying to learn Python (Diving Into Python) on my own time, so that hopefully one day I might be able help squash some of these bugs.. However, in the meantime, I/we very much appreciate your help and expertise.. REGRESSION TEST RESULTS ARE BELOW... Dan Williams wrote on 07/15/2006 01:20:54 PM: > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > > > Hi, > > I'm having a problem *not* being able to build 'i686' packages (i.e. > > optional_arches=i686) anymore *after* having upgraded our plague > > server/builder (Opteron x86_64) a couple of weeks ago from > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > > (optional_arches=i686) -- not with i386 (base_arches=i386) which > > continues to work flawlessly. > > Found and fixed in CVS HEAD. Patch attached that should apply against > an installed copy of plague-server as well. Thanks for the report, and > sorry for the lag. > > Cheers, > Dan > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe > Todaro/Poughkeepsie/IBM] REGRESSION TEST RESULTS: =========================================================================================== *WITHOUT* THE OPTIONAL-ARCH FIX TO CONFIG.PY: =========================================================================================== [root at lnxbuild1 ]$ cat /var/log/plague-server.log Using database engine mysql. Authorized Builders: ------------------------------------------------------------------------------------------ https://lnxbuild1.abc.com:8888 unavailable https://lnxppc.abc.com:8888 unavailable Build Server accepting requests on lnxbuild1.abc.com:8887. Re-activating builder 'https://lnxbuild1.abc.com:8888'. Re-activating builder 'https://lnxppc.abc.com:8888'. Request to enqueue 'WhoAmI' tag '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for target 'oc100' (user 'jstodaro at us.ibm.com') 625 (WhoAmI): Starting tag '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on target 'lnxaddons-100-install' 625 (WhoAmI): Requesting depsolve... 625 (WhoAmI): Starting depsolve for arches: ['i386']. 625 (WhoAmI): Finished depsolve (successful), requesting archjobs. 625 (WhoAmI/noarch): https://lnxbuild1.abc.com:8888 - UID is f2e2220b50c7707b7f224f18455f39fa088c5a22 625 (WhoAmI/noarch): Build result files - [ 'WhoAmI-4.00-9_rhel4.noarch.rpm', 'mockconfig.log', 'build.log', 'root.log', 'WhoAmI-4.00-9_rhel4.src.rpm', 'job.log' ] Repo 'lnxaddons-100-install': updating repository metadata... Repo 'lnxaddons-100-install': Done updating. 625 (WhoAmI): Job finished. Received SIGTERM, quitting... Shutting down... Repo (lnxaddons-100-install): shut down. Repo (lnxaddons-350-install): shut down. Repo (lnxaddons-100-unsupported): shut down. Done. =========================================================================================== *WITH* THE OPTIONAL-ARCH FIX TO CONFIG.PY: =========================================================================================== [root at lnxbuild1 ]$ cat /var/log/plague-server.log Using database engine mysql. Authorized Builders: ------------------------------------------------------------------------------------------ https://lnxbuild1.abc.com:8888 unavailable https://lnxppc.abc.com:8888 unavailable Build Server accepting requests on lnxbuild1.abc.com:8887. Re-activating builder 'https://lnxbuild1.abc.com:8888'. Re-activating builder 'https://lnxppc.abc.com:8888'. Request to enqueue 'WhoAmI' tag '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for target 'oc100' (user 'jstodaro at us.ibm.com') 626 (WhoAmI): Starting tag '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on target 'lnxaddons-100-install' 626 (WhoAmI): Requesting depsolve... 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686', 'noarch']. 626 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch i686. Exception in thread PackageJob: 626/WhoAmI: Traceback (most recent call last): File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap self.run() File "/usr/share/plague/server/PackageJob.py", line 86, in run self._pkg_job.process() File "/usr/share/plague/server/PackageJob.py", line 719, in process if func(): File "/usr/share/plague/server/PackageJob.py", line 587, in _stage_depsolve if self._arch_deps_solved(arch) == False: File "/usr/share/plague/server/PackageJob.py", line 540, in _arch_deps_solved if base: UnboundLocalError: local variable 'base' referenced before assignment Thanks again, -Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From williams at redhat.com Sun Jul 16 14:18:44 2006 From: williams at redhat.com (Clark Williams) Date: Sun, 16 Jul 2006 09:18:44 -0500 Subject: Running rpmlint within mock In-Reply-To: <200607142223.24253.jkeating@j2solutions.net> References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141952.44898.jkeating@j2solutions.net> <200607142223.24253.jkeating@j2solutions.net> Message-ID: <44BA4AC4.7090503@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jesse Keating wrote: > On Friday 14 July 2006 22:10, Jason L Tibbitts III wrote: >> Oh, awesome. Honestly, very many thanks. Might I be so bold as to >> suggest something like the following patch? > > Looks reasonable to me. Alas, I have no commit access (: (maybe that's a > good thing...) > Well, I have commit access and as soon as I get back to the house, this will go in. Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEukrDHyuj/+TTEp0RAh+cAKCLefB3+jvXIilFfMUALRV3keB3nQCeJ0AB ZYWrZiOmErdIYu66MGqKvks= =D+nL -----END PGP SIGNATURE----- From williams at redhat.com Sun Jul 16 14:26:33 2006 From: williams at redhat.com (Clark Williams) Date: Sun, 16 Jul 2006 09:26:33 -0500 Subject: Running rpmlint within mock In-Reply-To: References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> <200607141952.44898.jkeating@j2solutions.net> Message-ID: <44BA4C99.8040802@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jason L Tibbitts III wrote: > OK, here's a first pass hack. This could be better written in Python > and could extract some bits from the mock configuration, but this does > at least work. It builds the package, installs rpmlint and all of the > newly-built binary RPMs and (hopefully) their dependencies into the > chroot. It then runs rpmlint on the SRPM, the binary RPMs and the > installed package names. I have tested this against packages which > show different warnings for each and it seems to actually work. > > You'll have to edit to taste, and try not to hurl as some of the > quoting is nasty. > > - J< > Note that using mockhelper is deprecated, since we've re-architected so that mock becomes a chroot launcher for mock.py. Mockhelper will go away in the next release (0.7). Just so you know, I have a version of mock that takes a --rpmlint command line option which will add rpmlint to the chroot transaction set, install it in the chroot and then run rpmlint on the generated SRPM and RPMs, putting the output in an rpmlint.log results file. It wasn't very intrusive and won't affect build systems unless they specifically invoke it. As an aside, I wouldn't mind looking at mock as a chroot builder for things like the test project. It's mostly there, since it's main job is to build a chroot and run rpmbuild within that chroot. We should discuss. Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEukxNHyuj/+TTEp0RAgR2AKCOviuiwMw1apU/YXSXig7bzvmzggCeI+fB qJ6PTlrYp5XRxGhMbrGWiLA= =rQrE -----END PGP SIGNATURE----- From dcbw at redhat.com Sun Jul 16 14:41:55 2006 From: dcbw at redhat.com (Dan Williams) Date: Sun, 16 Jul 2006 10:41:55 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1153060915.2599.13.camel@localhost.localdomain> On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote: > > Thanks Dan. > > So I ran a quick regression test, just to make sure that *i386* is > still working, but, it *failed* ... (also note that i686 gave > basically the same errors..) > > SEE BELOW for *regression* test results, both BEFORE and AFTER > applying patch 'plague-opt-arches-fix.patch' to server/Config.py. I > repeated that test sequence several times, by the way. Also, I was > surprised to see that 'WhoAmI' was trying to build as an 'i686' rpm > rather than a 'noarch' which is what it really is. So, since I can't Hmm... Building an i386 package worked for me when I tested it with current CVS HEAD, as did an i686 package. > be exactly sure at this point how the "optional_arches" design is > supposed to tie-in with the "base_arches" design, it doesn't make > much sense for me (with my limited Python skills) to try and dig in > any further.. Ok; most distributions have a "base" architecture for any given architecture family. For RHEL & FC, that's 'i386' for x86 processors, and for POWER, that's 'ppc'. We only build specific RPMs as i686 since not every RPM requires it, and building many RPMs for i686 can actually _hurt_ performance. In the same vein, we don't build both 'em64t' and x86-64 packages for everything either, because the difference between EM64T and x86-64 only matters for stuff like the kernel. Given a set of packages, you _don't_ want to build every package on every architecture the buildsystem supports, unless the package explicitly asks for it. Given a specfile with _no_ ExcludeArch or IncludeArch, plague will build that package only for the "base" architecture, in this case i386. So what happens when you want a package to explicitly build on i686 or i586 or em64t? In this case, the specfile for the package must explicitly request to be built as such, using ExclusiveArch. Coming back to the original question, optional_arches gives buildsystem administrators a filter on arches to build. For example, Fedora Extras doesn't necessarily want to allow packages to be built for i486, even if the RPM requests it, because we don't support i486. So if you add i686 to 'optional_arches', then packages that request to be built for i686 with ExclusiveArch will be allowed to build for i686, otherwise not. > Also, could you please explain/elaborate when you say: "Patch attached > that should apply against an installed copy of plague-server as > well." > Um, your patch is clearly intended to update file server/Config.py > file, so I really don't understand how I can also use *it* to update > file *plague-server* (/usr/bin/plague-server?) as well, which appears > to be a start-up script -- am I missing something here? I had meant to change the paths such that you could apply the patch from / using patch -p0 or such; I did that and then re-diffed and likely forgot to update the patch bits again. Since the 'cvs diff' is rooted at a different directory than the installed plague server directory, I was trying to make life simpler for you. My bad. > For the good news (for me anyway), I'm also trying to learn Python > (Diving Into Python) on my own time, so that hopefully one day I might > be able help squash some of these bugs.. However, in the meantime, > I/we very much appreciate your help and expertise.. > > REGRESSION TEST RESULTS ARE BELOW... > > Dan Williams wrote on 07/15/2006 01:20:54 PM: > > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > > > > > Hi, > > > I'm having a problem *not* being able to build 'i686' packages > (i.e. > > > optional_arches=i686) anymore *after* having upgraded our plague > > > server/builder (Opteron x86_64) a couple of weeks ago from > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > > > (optional_arches=i686) -- not with i386 (base_arches=i386) which > > > continues to work flawlessly. > > > > Found and fixed in CVS HEAD. Patch attached that should apply > against > > an installed copy of plague-server as well. Thanks for the report, > and > > sorry for the lag. > > > > Cheers, > > Dan > > > > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe > > Todaro/Poughkeepsie/IBM] > > > 626 (WhoAmI): Requesting depsolve... > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686', > 'noarch']. Ok; what do you have in the 'optional_arches' for this target? What do the top bits of the specfile for WhoAmI look like? i.e., are there any of the following specfile tags, and what are their values? BuildArch ExclusiveArch ExcludeArch > 626 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch > i686. > Exception in thread PackageJob: 626/WhoAmI: > Traceback (most recent call last): > File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap > self.run() > File "/usr/share/plague/server/PackageJob.py", line 86, in run > self._pkg_job.process() > File "/usr/share/plague/server/PackageJob.py", line 719, in process > if func(): > File "/usr/share/plague/server/PackageJob.py", line 587, in > _stage_depsolve > if self._arch_deps_solved(arch) == False: > File "/usr/share/plague/server/PackageJob.py", line 540, in > _arch_deps_solved > if base: > UnboundLocalError: local variable 'base' referenced before assignment To get rid of this error, remove the marked line around line 490 of PackageJob.py: base = yum.YumBase() yum_config = self._write_yum_conf(arch) if not yum_config: - del base raise DepError("WARNING: bad yum config for arch %s." % arch) depsolve_root = os.path.dirname(yum_config) + '/' base.doConfigSetup(fn=yum_config, root=depsolve_root) That's a bug. Hope we can get this fixed. Dan > Thanks again, > -Joe From tibbs at math.uh.edu Sun Jul 16 15:25:17 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Sun, 16 Jul 2006 10:25:17 -0500 Subject: Running rpmlint within mock In-Reply-To: <44BA4C99.8040802@redhat.com> (Clark Williams's message of "Sun, 16 Jul 2006 09:26:33 -0500") References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> <200607141952.44898.jkeating@j2solutions.net> <44BA4C99.8040802@redhat.com> Message-ID: >>>>> "CW" == Clark Williams writes: CW> Just so you know, I have a version of mock that takes a --rpmlint CW> command line option which will add rpmlint to the chroot CW> transaction set, install it in the chroot and then run rpmlint on CW> the generated SRPM and RPMs, putting the output in an rpmlint.log CW> results file. Ah, very very cool. One important thing, and the whole reason for actually doing this, is that in addition to the SRPM and RPMs, you need to run rpmlint in the chroot on the package name itself. For example, build the aplus-fsf package under review. rpmlint will find nothing real to complain about with the SRPM or binary RPMs, but if you run "rpmlint aplus-fsf" in the chroot with that package installed you get 3800 warnings. - J< From jstodaro at us.ibm.com Sun Jul 16 16:18:23 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Sun, 16 Jul 2006 12:18:23 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: <1153060915.2599.13.camel@localhost.localdomain> Message-ID: Dan Williams wrote on 07/16/2006 10:41:55 AM: > On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote: > > > > Thanks Dan. > > > > So I ran a quick regression test, just to make sure that *i386* is > > still working, but, it *failed* ... (also note that i686 gave > > basically the same errors..) > > > > SEE BELOW for *regression* test results, both BEFORE and AFTER > > applying patch 'plague-opt-arches-fix.patch' to server/Config.py. I > > repeated that test sequence several times, by the way. Also, I was > > surprised to see that 'WhoAmI' was trying to build as an 'i686' rpm > > rather than a 'noarch' which is what it really is. So, since I can't > > Hmm... Building an i386 package worked for me when I tested it with > current CVS HEAD, as did an i686 package. Do you think that re-installing plague-0.5 from scratch (rpm -e/rpm -ivh) might help? If so, are there any "gotchas" I need to be aware of? But hopefully this won't be necessary. > > > be exactly sure at this point how the "optional_arches" design is > > supposed to tie-in with the "base_arches" design, it doesn't make > > much sense for me (with my limited Python skills) to try and dig in > > any further.. > > Ok; most distributions have a "base" architecture for any given > architecture family. For RHEL & FC, that's 'i386' for x86 processors, > and for POWER, that's 'ppc'. We only build specific RPMs as i686 since > not every RPM requires it, and building many RPMs for i686 can actually > _hurt_ performance. In the same vein, we don't build both 'em64t' and > x86-64 packages for everything either, because the difference between > EM64T and x86-64 only matters for stuff like the kernel. That was my understanding. > > Given a set of packages, you _don't_ want to build every package on > every architecture the buildsystem supports, unless the package > explicitly asks for it. Given a specfile with _no_ ExcludeArch or > IncludeArch, plague will build that package only for the "base" > architecture, in this case i386. > That was my understanding too. > So what happens when you want a package to explicitly build on i686 or > i586 or em64t? In this case, the specfile for the package must > explicitly request to be built as such, using ExclusiveArch. > Ok, now I'm clear on this. But also I have a question: Since i686 is a "subarch" of i386, do I need to create any target config files specicially for i686, such as in the following directories: - /etc/plague/server/targets - /etc/plague/builder/targets - /etc/mock ? > Coming back to the original question, optional_arches gives buildsystem > administrators a filter on arches to build. For example, Fedora Extras > doesn't necessarily want to allow packages to be built for i486, even if > the RPM requests it, because we don't support i486. So if you add i686 > to 'optional_arches', then packages that request to be built for i686 > with ExclusiveArch will be allowed to build for i686, otherwise not. > I'm also really clear on this, too, now. Well, at least this proves, to me anyway, that I've had a pretty decent understanding of how things work all along -- thanks to the great description you gave above, of course. > > Also, could you please explain/elaborate when you say: "Patch attached > > that should apply against an installed copy of plague-server as > > well." > > Um, your patch is clearly intended to update file server/Config.py > > file, so I really don't understand how I can also use *it* to update > > file *plague-server* (/usr/bin/plague-server?) as well, which appears > > to be a start-up script -- am I missing something here? > > I had meant to change the paths such that you could apply the patch > from / using patch -p0 or such; I did that and then re-diffed and likely > forgot to update the patch bits again. Since the 'cvs diff' is rooted > at a different directory than the installed plague server directory, I > was trying to make life simpler for you. My bad. > Oh, cool, I've never done that before - thanks. Regardless, though, I still don't see any variable in /usr/bin/plague-server that has *anything* to with "optional arches" or "base arches", which is what the patch was for right? Anyway, here's a snippet from /usr/bin/plague-server script, which is the one I've been assuming that you're referring to - is this the one? =================================== Excerpt from /usr/bin/plague-server =================================== ------------------------ # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. # # Copyright 2005 Dan Williams and Red Hat, Inc. import sys import os import socket import signal import time from plague import AuthedXMLRPCServer from plague import HTTPServer from plague import daemonize from plague import DebugUtils import SimpleXMLRPCServer from optparse import OptionParser import threading sys.path.append('/usr/share/plague/server') import User import BuildMaster import BuilderManager import DBManager import Config from UserInterface import UserInterfaceSSLAuth from UserInterface import UserInterfaceNoAuth ----------------------- > > For the good news (for me anyway), I'm also trying to learn Python > > (Diving Into Python) on my own time, so that hopefully one day I might > > be able help squash some of these bugs.. However, in the meantime, > > I/we very much appreciate your help and expertise.. > > > > REGRESSION TEST RESULTS ARE BELOW... > > > > Dan Williams wrote on 07/15/2006 01:20:54 PM: > > > > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > > > > > > > Hi, > > > > I'm having a problem *not* being able to build 'i686' packages > > (i.e. > > > > optional_arches=i686) anymore *after* having upgraded our plague > > > > server/builder (Opteron x86_64) a couple of weeks ago from > > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > > > > (optional_arches=i686) -- not with i386 (base_arches=i386) which > > > > continues to work flawlessly. > > > > > > Found and fixed in CVS HEAD. Patch attached that should apply > > against > > > an installed copy of plague-server as well. Thanks for the report, > > and > > > sorry for the lag. > > > > > > Cheers, > > > Dan > > > > > > > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe > > > Todaro/Poughkeepsie/IBM] > > > > > > 626 (WhoAmI): Requesting depsolve... > > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686', > > 'noarch']. > > Ok; what do you have in the 'optional_arches' for this target? ====================================== Excerpt from lnxaddons-100-install.cfg ====================================== ----------------------- [Arches] base_arches=i386 optional_arches=i686 noarch ---------------------- > > What do the top bits of the specfile for WhoAmI look like? i.e., are > there any of the following specfile tags, and what are their values? > > BuildArch > ExclusiveArch > ExcludeArch > ======================== Excerpt from WhoAmI.spec ======================== ----------------------- %define rversion 4.00 %define rel 9 %define pkgname WhoAmI Summary : %{__distribution_long} system information Name : %{pkgname} Version : %{rversion} Release : %{rel}_%{__build_distribution}%{__build_release} License : %__spec_internal_license Group : System Environment/Base Url : %__repository_url Packager : %{packager} Source0 : %{pkgname}.tar.gz %if %__build_rhel Requires : coreutils, bash, python, grep, rpm %endif %if %__build_suse Requires : coreutils, bash, aaa_base, python, grep, rpm %endif BuildArch : noarch BuildRoot : %{_tmppath}/%{name}-%{version}-root AutoReqProv : no %Description %{__distribution_long} system information %prep [ "%{buildroot}" != "/" ] && %{__rm} -rf %{buildroot} ----------------------- > > 626 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch > > i686. > > Exception in thread PackageJob: 626/WhoAmI: > > Traceback (most recent call last): > > File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap > > self.run() > > File "/usr/share/plague/server/PackageJob.py", line 86, in run > > self._pkg_job.process() > > File "/usr/share/plague/server/PackageJob.py", line 719, in process > > if func(): > > File "/usr/share/plague/server/PackageJob.py", line 587, in > > _stage_depsolve > > if self._arch_deps_solved(arch) == False: > > File "/usr/share/plague/server/PackageJob.py", line 540, in > > _arch_deps_solved > > if base: > > UnboundLocalError: local variable 'base' referenced before assignment > > To get rid of this error, remove the marked line around line 490 of > PackageJob.py: > > base = yum.YumBase() > yum_config = self._write_yum_conf(arch) > if not yum_config: > - del base > raise DepError("WARNING: bad yum config for arch %s." % arch) > > depsolve_root = os.path.dirname(yum_config) + '/' > base.doConfigSetup(fn=yum_config, root=depsolve_root) > > That's a bug. Hope we can get this fixed. Me too.. your help is really appreciated. > > Dan > > > Thanks again, > > -Joe > Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From williams at redhat.com Sun Jul 16 16:19:44 2006 From: williams at redhat.com (Clark Williams) Date: Sun, 16 Jul 2006 11:19:44 -0500 Subject: Running rpmlint within mock In-Reply-To: References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> <200607141952.44898.jkeating@j2solutions.net> <44BA4C99.8040802@redhat.com> Message-ID: <44BA6720.6080802@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jason L Tibbitts III wrote: >>>>>> "CW" == Clark Williams writes: > > CW> Just so you know, I have a version of mock that takes a > --rpmlint CW> command line option which will add rpmlint to the > chroot CW> transaction set, install it in the chroot and then run > rpmlint on CW> the generated SRPM and RPMs, putting the output in > an rpmlint.log CW> results file. > > Ah, very very cool. One important thing, and the whole reason for > actually doing this, is that in addition to the SRPM and RPMs, you > need to run rpmlint in the chroot on the package name itself. For > example, build the aplus-fsf package under review. rpmlint will > find nothing real to complain about with the SRPM or binary RPMs, > but if you run "rpmlint aplus-fsf" in the chroot with that package > installed you get 3800 warnings. Well dang, now you're going to make me work for it :). What I've got now just runs rpmlint inside the chroot on the source and binary packages. How about we go with that for a bit and let me think on how we could install into the chroot? I'm not really against it, since I think it goes with the "chroot manager" task of mock, but I'd like to look at how to do it without creating more problems than we solve. The way I see it, mock's primary mission is to be the tool that creates a pristine chroot for the purpose of buiding a package (or set of packages); everything we do here needs to keep that in mind. A related secondary mission though could be management of a longer lived chroot used for various purposes (testing, cross-building, etc.). Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEumcfHyuj/+TTEp0RAuuLAJ4x0mweM+j+yGwNBJ+mcKb2DA+XeQCeKrcs mjsmeV5HcgqdONQgUE151+k= =zNi5 -----END PGP SIGNATURE----- From jkeating at j2solutions.net Sun Jul 16 16:23:23 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Sun, 16 Jul 2006 12:23:23 -0400 Subject: Running rpmlint within mock In-Reply-To: <44BA6720.6080802@redhat.com> References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <44BA6720.6080802@redhat.com> Message-ID: <200607161223.27648.jkeating@j2solutions.net> On Sunday 16 July 2006 12:19, Clark Williams wrote: > What I've got now just runs rpmlint inside the chroot on the source > and binary packages. How about we go with that for a bit and let me > think on how we could install into the chroot? I'm not really against > it, since I think it goes with the "chroot manager" task of mock, but > I'd like to look at how to do it without creating more problems than > we solve. > > The way I see it, mock's primary mission is to be the tool that > creates a pristine chroot for the purpose of buiding a package (or set > of packages); everything we do here needs to keep that in mind. A > related secondary mission though could be management of a longer lived > chroot used for various purposes (testing, cross-building, etc.). How much of this usage falls under what Mach does? Mock has a pretty narrow purpose, and trying to add anything to it might be better through the mach project. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From dcbw at redhat.com Mon Jul 17 10:11:34 2006 From: dcbw at redhat.com (Dan Williams) Date: Mon, 17 Jul 2006 06:11:34 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1153131094.2545.17.camel@localhost.localdomain> On Sun, 2006-07-16 at 12:18 -0400, Joe Todaro wrote: > > Dan Williams wrote on 07/16/2006 10:41:55 AM: > > > On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote: > > > > > > Thanks Dan. > > > > > > So I ran a quick regression test, just to make sure that *i386* is > > > still working, but, it *failed* ... (also note that i686 gave > > > basically the same errors..) > > > > > > SEE BELOW for *regression* test results, both BEFORE and AFTER > > > applying patch 'plague-opt-arches-fix.patch' to server/Config.py. > I > > > repeated that test sequence several times, by the way. Also, I > was > > > surprised to see that 'WhoAmI' was trying to build as an 'i686' > rpm > > > rather than a 'noarch' which is what it really is. So, since I > can't > > > > Hmm... Building an i386 package worked for me when I tested it with > > current CVS HEAD, as did an i686 package. > > Do you think that re-installing plague-0.5 from scratch (rpm -e/rpm > -ivh) might help? > If so, are there any "gotchas" I need to be aware of? > But hopefully this won't be necessary. Hmm, this likely wouldn't change much. > But also I have a question: Since i686 is a "subarch" of i386, do I > need > to create any target config files specicially for i686, such as in > the > following directories: > - /etc/plague/server/targets > - /etc/plague/builder/targets > - /etc/mock No, you shouldn't need to create target configs for that. Only for 'base' architectures like i386, ppc, ppc64, x86_64, sparc, sparc64, etc. Sub-architectures like i686, em64t, sparcv9, etc, don't need their own target configs. For the builder, you need one target config for every "repository" that builder will be able to support. For example, if the machine were an x86-64-class machine able to build Fedora Core and Fedora Extras, you'd have in /etc/plague/builder/targets: fedora-devel-i386-core.cfg fedora-5-i386-core.cfg fedora-4-i386-core.cfg fedora-devel-i386-extras.cfg fedora-5-i386-extras.cfg fedora-4-i386-extras.cfg fedora-devel-x86_64-core.cfg fedora-5-x86_64-core.cfg ... and on the server, which has _no_ architecture at all, you'd have in /etc/plague/server/targets: fedora-devel-core.cfg fedora-5-core.cfg fedora-4-core.cfg fedora-devel-extras.cfg fedora-5-extras.cfg fedora-4-extras.cfg and on the server, if you care about depsolving (ie if you have the "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) then you actually do need mock target configs for everything too, because the server will use those to depsolve. > > Coming back to the original question, optional_arches gives > buildsystem > > administrators a filter on arches to build. For example, Fedora > Extras > > doesn't necessarily want to allow packages to be built for i486, > even if > > the RPM requests it, because we don't support i486. So if you add > i686 > > to 'optional_arches', then packages that request to be built for > i686 > > with ExclusiveArch will be allowed to build for i686, otherwise not. > > > > I'm also really clear on this, too, now. Well, at least this proves, > to > me anyway, that I've had a pretty decent understanding of how things > work > all along -- thanks to the great description you gave above, of > course. No problem. There's so many options to stuff like RPM and apt/dpkg that it's hard to keep straight. > > > Also, could you please explain/elaborate when you say: "Patch > attached > > > that should apply against an installed copy of plague-server as > > > well." > > > Um, your patch is clearly intended to update file server/Config.py > > > file, so I really don't understand how I can also use *it* to > update > > > file *plague-server* (/usr/bin/plague-server?) as well, which > appears > > > to be a start-up script -- am I missing something here? > > > > I had meant to change the paths such that you could apply the patch > > from / using patch -p0 or such; I did that and then re-diffed and > likely > > forgot to update the patch bits again. Since the 'cvs diff' is > rooted > > at a different directory than the installed plague server directory, > I > > was trying to make life simpler for you. My bad. > > > > Oh, cool, I've never done that before - thanks. > > Regardless, though, I still don't see any variable > in /usr/bin/plague-server > that has *anything* to with "optional arches" or "base arches", which > is > what the patch was for right? The patch would actually have been against /usr/share/plague/server/Config.py. That's where it gets somewhat confusing, since in the build directory everything is together, but during RPM creation and install the server's main.py file gets renamed to /usr/bin/plague-server. What I've said 'plague-server', I've generally been referring to the server process as a whole, not specific files. Sorry for that. > > > For the good news (for me anyway), I'm also trying to learn Python > > > (Diving Into Python) on my own time, so that hopefully one day I > might > > > be able help squash some of these bugs.. However, in the meantime, > > > I/we very much appreciate your help and expertise.. > > > > > > REGRESSION TEST RESULTS ARE BELOW... > > > > > > Dan Williams wrote on 07/15/2006 01:20:54 PM: > > > > > > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > > > > > > > > > Hi, > > > > > I'm having a problem *not* being able to build 'i686' packages > > > (i.e. > > > > > optional_arches=i686) anymore *after* having upgraded our > plague > > > > > server/builder (Opteron x86_64) a couple of weeks ago from > > > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with > i686 > > > > > (optional_arches=i686) -- not with i386 (base_arches=i386) > which > > > > > continues to work flawlessly. > > > > > > > > Found and fixed in CVS HEAD. Patch attached that should apply > > > against > > > > an installed copy of plague-server as well. Thanks for the > report, > > > and > > > > sorry for the lag. > > > > > > > > Cheers, > > > > Dan > > > > > > > > > > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe > > > > Todaro/Poughkeepsie/IBM] > > > > > > > > > 626 (WhoAmI): Requesting depsolve... > > > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686', > > > 'noarch']. > > > > Ok; what do you have in the 'optional_arches' for this target? > > ====================================== > Excerpt from lnxaddons-100-install.cfg > ====================================== > > ----------------------- > [Arches] > base_arches=i386 > optional_arches=i686 noarch Ok; I think this is the cause of the noarch problem. Since 'noarch' is a base architecture in itself, you don't need it in optional_arches. If you remove it, I think things will work as you expect. Every builder can build a 'noarch' package, so that architecture is essentially added by default and you don't need to do anything explicit for it. Yay for documentation (or complete lack of it). However, this does expose a bug. If an architecture is listed in optional_arches, it shouldn't actually be used by a package build unless the package was going to build for noarch already. > ---------------------- > > > > What do the top bits of the specfile for WhoAmI look like? i.e., > are > > there any of the following specfile tags, and what are their values? > > > > BuildArch > > ExclusiveArch > > ExcludeArch > > > > ======================== > Excerpt from WhoAmI.spec > ======================== > > ----------------------- > %define rversion 4.00 > %define rel 9 > %define pkgname WhoAmI > > Summary : %{__distribution_long} system information > Name : %{pkgname} > Version : %{rversion} > Release : %{rel}_%{__build_distribution}%{__build_release} > License : %__spec_internal_license > Group : System Environment/Base > Url : %__repository_url > > Packager : %{packager} > > Source0 : %{pkgname}.tar.gz > > %if %__build_rhel > Requires : coreutils, bash, python, grep, rpm > %endif > > %if %__build_suse > Requires : coreutils, bash, aaa_base, python, grep, rpm > %endif > > BuildArch : noarch I'm getting slightly confused here. Plague 0.4.3 was building this package correctly as an i686 package? Do you really _want_ this to build as an i686 package? If so, I think the "BuildArch: noarch" is wrong then. Typically, if you want a package to build noarch, you specify "BuildArch: noarch" and it will then only build on noarch. What arch is this package supposed to be for, exactly? Cheers, Dan From tibbs at math.uh.edu Mon Jul 17 15:13:31 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Mon, 17 Jul 2006 10:13:31 -0500 Subject: Running rpmlint within mock In-Reply-To: <44BA6720.6080802@redhat.com> (Clark Williams's message of "Sun, 16 Jul 2006 11:19:44 -0500") References: <200607142011.k6EKBKHC011621@mx1.redhat.com> <200607141636.24337.jkeating@j2solutions.net> <200607141952.44898.jkeating@j2solutions.net> <44BA4C99.8040802@redhat.com> <44BA6720.6080802@redhat.com> Message-ID: >>>>> "CW" == Clark Williams writes: CW> Well dang, now you're going to make me work for it :). Well, no big deal; I'm happy with what I have now, although I really hope that I'll be able to continue to make it work somehow after mockhelper goes away. I think some type of functionality like this is important; we (package reviewers, that is) need to make sure that the binary packages actually install, and then run the rpmlint checks on the installed packages. But it doesn't necessarily need to be built into mock; mock just needs to provide the hooks. - J< From williams at redhat.com Mon Jul 17 15:29:01 2006 From: williams at redhat.com (Clark Williams) Date: Mon, 17 Jul 2006 10:29:01 -0500 Subject: proposed mock changes (diff) Message-ID: <44BBACBD.6090900@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello all, I was poking around in the mock source last week and did some minor refactoring, a couple of name-changes and tried out the rpmlint request. Attached below is a CVS diff of my mock.py with the head of CVS. Please review and comment. A quick summary of the changes: 1. Changed version to 0.7. 2. Added code to avoid exec'ing mount for proc, sys, and dev/pts if we've already done it 3. Oh yeah, added /sys to chroot mount 4. Refactoring: renamed _mount to _mountall, created _mount routine that is called by _mountall 5. Renamed _umount_by_file to _umountall 6. Added code to run rpmlint 7. Added elevate/drop around raw chroot command I'd especially like some thought on #7, since any time you elevate and drop you can introduce a security hole and I freely admit that I'm not always thinking security first. If I don't get any push-back (or if I do and then get things resolved), I'll commit these later this week. Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEu6y9Hyuj/+TTEp0RAgumAJ9STO3Qc/7Ca4xYNdIAifcKs4oPvACgqpDD zOm5eNJ1Gwsgc4KqhS8WW0s= =0mBy -----END PGP SIGNATURE----- -------------- next part -------------- A non-text attachment was scrubbed... Name: mock.diff Type: text/x-patch Size: 8718 bytes Desc: not available URL: From Michael_E_Brown at Dell.com Mon Jul 17 16:52:35 2006 From: Michael_E_Brown at Dell.com (Michael_E_Brown at Dell.com) Date: Mon, 17 Jul 2006 11:52:35 -0500 Subject: proposed mock changes (diff) In-Reply-To: <44BBACBD.6090900@redhat.com> Message-ID: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> I am leaving for OLS 2006 and wont be able to do any review for the next week. I just caught up on the rpmlint discussion, and have a few concerns. -- Security of installing just-built RPM -- Can rpmlint just be done outside of mock (using mock chroot, for example)? Why do we have to extend mock for this? -- Michael > -----Original Message----- > From: fedora-buildsys-list-bounces at redhat.com > [mailto:fedora-buildsys-list-bounces at redhat.com] On Behalf Of > Clark Williams > Sent: Monday, July 17, 2006 10:29 AM > To: Discussion of Fedora build system > Subject: proposed mock changes (diff) > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hello all, > > I was poking around in the mock source last week and did some > minor refactoring, a couple of name-changes and tried out the > rpmlint request. Attached below is a CVS diff of my mock.py > with the head of CVS. Please review and comment. A quick > summary of the changes: > > 1. Changed version to 0.7. > 2. Added code to avoid exec'ing mount for proc, sys, and > dev/pts if we've already done it 3. Oh yeah, added /sys to > chroot mount 4. Refactoring: renamed _mount to _mountall, > created _mount routine that is called by _mountall 5. Renamed > _umount_by_file to _umountall 6. Added code to run rpmlint 7. > Added elevate/drop around raw chroot command > > I'd especially like some thought on #7, since any time you > elevate and drop you can introduce a security hole and I > freely admit that I'm not always thinking security first. > > If I don't get any push-back (or if I do and then get things > resolved), I'll commit these later this week. > > Clark > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.4 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org > > iD8DBQFEu6y9Hyuj/+TTEp0RAgumAJ9STO3Qc/7Ca4xYNdIAifcKs4oPvACgqpDD > zOm5eNJ1Gwsgc4KqhS8WW0s= > =0mBy > -----END PGP SIGNATURE----- > > From tibbs at math.uh.edu Mon Jul 17 19:43:57 2006 From: tibbs at math.uh.edu (Jason L Tibbitts III) Date: Mon, 17 Jul 2006 14:43:57 -0500 Subject: proposed mock changes (diff) In-Reply-To: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> (Michael E. Brown's message of "Mon, 17 Jul 2006 11:52:35 -0500") References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> Message-ID: >>>>> "MEB" == Michael E Brown writes: MEB> I just caught up on the rpmlint discussion, and have a few MEB> concerns. MEB> Security of installing just-built RPM Can't be any worse than actually building the package, can it? MEB> Can rpmlint just be done outside of mock (using mock chroot, for MEB> example)? Yes, as long as mock continues to give me a way to run a command in the chroot and some equivalent of "mock-helper yum". I was just asking for a way to do it, not that it actually be added as base functionality. - J< From Michael_E_Brown at Dell.com Mon Jul 17 19:49:30 2006 From: Michael_E_Brown at Dell.com (Michael_E_Brown at Dell.com) Date: Mon, 17 Jul 2006 14:49:30 -0500 Subject: proposed mock changes (diff) In-Reply-To: Message-ID: <35C9A9D68AB3FA4AB63692802656D9EC010F4508@ausx3mps303.aus.amer.dell.com> > -----Original Message----- > From: fedora-buildsys-list-bounces at redhat.com > [mailto:fedora-buildsys-list-bounces at redhat.com] On Behalf Of > Jason L Tibbitts III > Sent: Monday, July 17, 2006 2:44 PM > To: Discussion of Fedora build system > Subject: Re: proposed mock changes (diff) > > >>>>> "MEB" == Michael E Brown writes: > > MEB> I just caught up on the rpmlint discussion, and have a few > MEB> concerns. > > MEB> Security of installing just-built RPM > > Can't be any worse than actually building the package, can it? Building package happens as user running mock, not root. Package install happens as root. > > MEB> Can rpmlint just be done outside of mock (using mock chroot, for > MEB> example)? > > Yes, as long as mock continues to give me a way to run a > command in the chroot and some equivalent of "mock-helper > yum". I was just asking for a way to do it, not that it > actually be added as base functionality. Ok, cool. I think this is the way to go with this, then. -- Michael From williams at redhat.com Mon Jul 17 20:18:26 2006 From: williams at redhat.com (Clark Williams) Date: Mon, 17 Jul 2006 15:18:26 -0500 Subject: proposed mock changes (diff) In-Reply-To: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> Message-ID: <44BBF092.5090606@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael_E_Brown at Dell.com wrote: > I am leaving for OLS 2006 and wont be able to do any review for the next > week. > > I just caught up on the rpmlint discussion, and have a few concerns. > -- Security of installing just-built RPM > -- Can rpmlint just be done outside of mock (using mock chroot, > for example)? Why do we have to extend mock for this? > We don't really have to. I wondered if it could be done without a major upheaval in the code. Turns out it can if we're just talking about running it against the package files. But it would take quite a bit more to do it automagically against an installed package (i.e. install the just-built package into the chroot and run rpmlint on the installed package(s)). My thought is that running rpmlint against the just-generated RPM/SRPMs is a nice thing to have around, so that a developer could get a regular sanity check on the package (since the results are kept in an rpmlint.log file). Note that you'd still have to invoke mock with the --rpmlint command line option. I think that installing the package in a chroot and doing more extensive testing is job for a testing tool (not mock). So left to myself I'd probably add the --rpmlint option as is and let someone use 'mock --no-clean chroot ' to do more extensive testing. But I don't feel strongly enough about it that I'd argue for it in the face of determined resistance. Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEu/CRHyuj/+TTEp0RAsiIAKCJaWb5Oi8fFBLwPSPokx3jPGkJ4gCfcG9o mLa7aQdS89Wgq8t7OiXtv6w= =QV6u -----END PGP SIGNATURE----- From skvidal at linux.duke.edu Mon Jul 17 20:22:53 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Mon, 17 Jul 2006 16:22:53 -0400 Subject: proposed mock changes (diff) In-Reply-To: <44BBF092.5090606@redhat.com> References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> <44BBF092.5090606@redhat.com> Message-ID: <1153167773.1336.73.camel@cutter> On Mon, 2006-07-17 at 15:18 -0500, Clark Williams wrote: > My thought is that running rpmlint against the just-generated > RPM/SRPMs is a nice thing to have around, so that a developer could > get a regular sanity check on the package (since the results are kept > in an rpmlint.log file). Note that you'd still have to invoke mock > with the --rpmlint command line option. I think that installing the > package in a chroot and doing more extensive testing is job for a > testing tool (not mock). > +1 -sv From williams at redhat.com Mon Jul 17 20:57:46 2006 From: williams at redhat.com (Clark Williams) Date: Mon, 17 Jul 2006 15:57:46 -0500 Subject: proposed mock changes (diff) In-Reply-To: References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> Message-ID: <44BBF9CA.7090207@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jason L Tibbitts III wrote: >>>>>> "MEB" == Michael E Brown writes: > > MEB> Can rpmlint just be done outside of mock (using mock chroot, for > MEB> example)? > > Yes, as long as mock continues to give me a way to run a command in > the chroot and some equivalent of "mock-helper yum". I was just > asking for a way to do it, not that it actually be added as base > functionality. Well, there I was about to tell you to do this: $ mock --no-clean chroot yum install foo $ mock --no-clean chroot rpmlint -vi foo Then I thought, hey, I should try that first (what a concept, eh?). I did and about that time I realized that we don't install yum into the chroot (chroot manipulations occur outside the chroot). So what I did above won't work. Now, I really don't like the idea of adding *anything* to the default build chroot (other than what's required to build a package). It's slow enough as it is and for the most common usage (plague/mock builds) we don't want anything except what's absolutely needed. So, how would we go about managing a chroot for tasks like testing? Do we add an 'install' command to install stuff from the configured repositories into the chroot, like this: $ mock install rpmlint Or do we do something else? Do we make tester's put together their own cfg with a modified buildsys-build rpm that includes the packages they need? Is there a simpler mechanism for someone that wants a chroot for testing? Clark Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEu/nKHyuj/+TTEp0RAtWJAJsEfykRVC3RX/DXKibbf1zsBinEbACfYYDX Zd3ifqYv65iMf+zFvdDo1JM= =Clon -----END PGP SIGNATURE----- From sheltren at cs.ucsb.edu Mon Jul 17 22:22:35 2006 From: sheltren at cs.ucsb.edu (Jeff Sheltren) Date: Mon, 17 Jul 2006 18:22:35 -0400 Subject: proposed mock changes (diff) In-Reply-To: <44BBF9CA.7090207@redhat.com> References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> <44BBF9CA.7090207@redhat.com> Message-ID: <11DAAFDC-E917-43FC-96D0-C41718FAEC10@cs.ucsb.edu> On Jul 17, 2006, at 4:57 PM, Clark Williams wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jason L Tibbitts III wrote: >>>>>>> "MEB" == Michael E Brown writes: >> >> MEB> Can rpmlint just be done outside of mock (using mock chroot, for >> MEB> example)? >> >> Yes, as long as mock continues to give me a way to run a command in >> the chroot and some equivalent of "mock-helper yum". I was just >> asking for a way to do it, not that it actually be added as base >> functionality. > > Well, there I was about to tell you to do this: > > $ mock --no-clean chroot yum install foo > $ mock --no-clean chroot rpmlint -vi foo > > Then I thought, hey, I should try that first (what a concept, eh?). I > did and about that time I realized that we don't install yum into the > chroot (chroot manipulations occur outside the chroot). So what I did > above won't work. > > Now, I really don't like the idea of adding *anything* to the default > build chroot (other than what's required to build a package). It's > slow enough as it is and for the most common usage (plague/mock > builds) we don't want anything except what's absolutely needed. So, > how would we go about managing a chroot for tasks like testing? Do we > add an 'install' command to install stuff from the configured > repositories into the chroot, like this: > > $ mock install rpmlint > > Or do we do something else? Do we make tester's put together their own > cfg with a modified buildsys-build rpm that includes the packages they > need? Is there a simpler mechanism for someone that wants a chroot for > testing? > > Clark > I've always just used yum from the host system with the --installroot flag. You don't want to rely upon the RPM tools inside of the chroot because those won't always work - remember, we are doing all the package installs with the host system's yum in the first place. My vote would be to keep mock as simple/minimal as possible. -Jeff From jstodaro at us.ibm.com Mon Jul 17 22:49:59 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Mon, 17 Jul 2006 18:49:59 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: <1153131094.2545.17.camel@localhost.localdomain> Message-ID: DW> Dan Williams wrote on 07/17/2006 06:11:34 AM: JT> > On Sun, 2006-07-16 at 12:18 -0400, Joe Todaro wrote: JT> > Regardless, though, I still don't see any variable JT> > in /usr/bin/plague-server that has *anything* to JT> > with "optional arches" or "base arches", which JT> > is what the patch was for right? DW> DW> The patch would actually have been DW> against /usr/share/plague/server/Config.py. And indeed was successfully applied against /usr/share/plague/server/Config.py on July 15; one line got updated. DW> That's where it gets DW> somewhat confusing, since in the build directory everything is together, DW> but during RPM creation and install the server's main.py file gets DW> renamed to /usr/bin/plague-server. What I've said 'plague-server', I've DW> generally been referring to the server process as a whole, not specific DW> files. Sorry for that. DW> No apology necessary - my misunderstanding. Also, thanks for the excellent clarification/insight. DW> and on the server, which has _no_ architecture at all, you'd have DW> in /etc/plague/server/targets: DW> DW> fedora-devel-core.cfg DW> fedora-5-core.cfg DW> fedora-4-core.cfg DW> fedora-devel-extras.cfg DW> fedora-5-extras.cfg DW> fedora-4-extras.cfg DW> DW> and on the server, if you care about depsolving (ie if you have the DW> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) then DW> you actually do need mock target configs for everything too, DW> because the server will use those to depsolve. Absolutely, we do care about depsolving.. So, when you say "everything", does that also include (for example) 'i686' now, too, regardless of the fact that i686 is still only a sub-architecture? (Hmm, I wouldn't think so..but I just need to be clear on this.) JT> > ======================== JT> > Excerpt from WhoAmI.spec JT> > ======================== JT> > JT> > ----------------------- JT> > %define rversion 4.00 JT> > %define rel 9 JT> > %define pkgname WhoAmI JT> > JT> > Summary : %{__distribution_long} system information JT> > Name : %{pkgname} JT> > Version : %{rversion} JT> > Release : %{rel}_%{__build_distribution}%{__build_release} JT> > License : %__spec_internal_license JT> > Group : System Environment/Base JT> > Url : %__repository_url JT> > JT> > Packager : %{packager} JT> > JT> > Source0 : %{pkgname}.tar.gz JT> > JT> > %if %__build_rhel JT> > Requires : coreutils, bash, python, grep, rpm JT> > %endif JT> > JT> > %if %__build_suse JT> > Requires : coreutils, bash, aaa_base, python, grep, rpm JT> > %endif JT> > JT> > BuildArch : noarch JT> > DW> I'm getting slightly confused here. Sorry for that - my fault. DW> Plague 0.4.3 was building this DW> package correctly as an i686 package? No, plague 0.4.3 was building this package correctly as a 'noarch'; it never tried building as an 'i686' package until *after* I applied the one-liner patch to /usr/share/plague/server/Config.py on July 15; Moreover, it started doing so (building i686) automatically, because PackageJob.py is now setting "archlist = ['i386', 'i686']" rather than just "archlist = ['i386'] which it has always done in the past using plague-0.4. DW> Do you really _want_ this to DW> build as an i686 package? No, of course not - this is a noarch only package, and always has been. DW> If so, I think the "BuildArch: noarch" is DW> wrong then. Typically, if you want a package to build noarch, you DW> specify "BuildArch: noarch" and it will then only build on noarch. What DW> arch is this package supposed to be for, exactly? 'noarch' as is specified in the specfile. JT> > ====================================== JT> > Excerpt from lnxaddons-100-install.cfg JT> > ====================================== JT> > JT> > ----------------------- JT> > [Arches] JT> > base_arches=i386 JT> > optional_arches=i686 noarch JT> > GW> Ok; I think this is the cause of the noarch problem. Since 'noarch' is GW> a base architecture in itself, you don't need it in optional_arches. If GW> you remove it, I think things will work as you expect. Nope - no such luck; I have removed 'noarch' from 'optional_arches', re-tested, and am still seeing that same error.. (Jeez, I feel like we're soo close..I can taste it.) And now I have a design-related question if I may: Why is the PackageJob.py module appending 'optional_arches' onto the 'archlist' list (lines 574-576), and later trying to process those "optional" arches as is they were "base" arches? Hmm, *I* don't get it.. just when I thought things were starting to make sense too. (: CW> Every builder GW> can build a 'noarch' package, so that architecture is essentially added GW> by default and you don't need to do anything explicit for it. Yay for GW> documentation (or complete lack of it). GW> GW> However, this does expose a bug. If an architecture is listed in GW> optional_arches, it shouldn't actually be used by a package build unless GW> the package was going to build for noarch already. GW> I think we're *real* close.. Regards, Joe ====================== COMPLETE THREAD ============================= Dan Williams wrote on 07/17/2006 06:11:34 AM: > On Sun, 2006-07-16 at 12:18 -0400, Joe Todaro wrote: > > > > Dan Williams wrote on 07/16/2006 10:41:55 AM: > > > > > On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote: > > > > > > > > Thanks Dan. > > > > > > > > So I ran a quick regression test, just to make sure that *i386* is > > > > still working, but, it *failed* ... (also note that i686 gave > > > > basically the same errors..) > > > > > > > > SEE BELOW for *regression* test results, both BEFORE and AFTER > > > > applying patch 'plague-opt-arches-fix.patch' to server/Config.py. > > I > > > > repeated that test sequence several times, by the way. Also, I > > was > > > > surprised to see that 'WhoAmI' was trying to build as an 'i686' > > rpm > > > > rather than a 'noarch' which is what it really is. So, since I > > can't > > > > > > Hmm... Building an i386 package worked for me when I tested it with > > > current CVS HEAD, as did an i686 package. > > > > Do you think that re-installing plague-0.5 from scratch (rpm -e/rpm > > -ivh) might help? > > If so, are there any "gotchas" I need to be aware of? > > But hopefully this won't be necessary. > > Hmm, this likely wouldn't change much. > > > But also I have a question: Since i686 is a "subarch" of i386, do I > > need > > to create any target config files specicially for i686, such as in > > the > > following directories: > > - /etc/plague/server/targets > > - /etc/plague/builder/targets > > - /etc/mock > > No, you shouldn't need to create target configs for that. Only for > 'base' architectures like i386, ppc, ppc64, x86_64, sparc, sparc64, etc. > Sub-architectures like i686, em64t, sparcv9, etc, don't need their own > target configs. > > For the builder, you need one target config for every "repository" that > builder will be able to support. For example, if the machine were an > x86-64-class machine able to build Fedora Core and Fedora Extras, you'd > have in /etc/plague/builder/targets: > > fedora-devel-i386-core.cfg > fedora-5-i386-core.cfg > fedora-4-i386-core.cfg > fedora-devel-i386-extras.cfg > fedora-5-i386-extras.cfg > fedora-4-i386-extras.cfg > fedora-devel-x86_64-core.cfg > fedora-5-x86_64-core.cfg > ... > > and on the server, which has _no_ architecture at all, you'd have > in /etc/plague/server/targets: > > fedora-devel-core.cfg > fedora-5-core.cfg > fedora-4-core.cfg > fedora-devel-extras.cfg > fedora-5-extras.cfg > fedora-4-extras.cfg > > and on the server, if you care about depsolving (ie if you have the > "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) then > you actually do need mock target configs for everything too, because the > server will use those to depsolve. > > > > Coming back to the original question, optional_arches gives > > buildsystem > > > administrators a filter on arches to build. For example, Fedora > > Extras > > > doesn't necessarily want to allow packages to be built for i486, > > even if > > > the RPM requests it, because we don't support i486. So if you add > > i686 > > > to 'optional_arches', then packages that request to be built for > > i686 > > > with ExclusiveArch will be allowed to build for i686, otherwise not. > > > > > > > I'm also really clear on this, too, now. Well, at least this proves, > > to > > me anyway, that I've had a pretty decent understanding of how things > > work > > all along -- thanks to the great description you gave above, of > > course. > > No problem. There's so many options to stuff like RPM and apt/dpkg that > it's hard to keep straight. > > > > > Also, could you please explain/elaborate when you say: "Patch > > attached > > > > that should apply against an installed copy of plague-server as > > > > well." > > > > Um, your patch is clearly intended to update file server/Config.py > > > > file, so I really don't understand how I can also use *it* to > > update > > > > file *plague-server* (/usr/bin/plague-server?) as well, which > > appears > > > > to be a start-up script -- am I missing something here? > > > > > > I had meant to change the paths such that you could apply the patch > > > from / using patch -p0 or such; I did that and then re-diffed and > > likely > > > forgot to update the patch bits again. Since the 'cvs diff' is > > rooted > > > at a different directory than the installed plague server directory, > > I > > > was trying to make life simpler for you. My bad. > > > > > > > Oh, cool, I've never done that before - thanks. > > > > Regardless, though, I still don't see any variable > > in /usr/bin/plague-server > > that has *anything* to with "optional arches" or "base arches", which > > is > > what the patch was for right? > > The patch would actually have been > against /usr/share/plague/server/Config.py. That's where it gets > somewhat confusing, since in the build directory everything is together, > but during RPM creation and install the server's main.py file gets > renamed to /usr/bin/plague-server. What I've said 'plague-server', I've > generally been referring to the server process as a whole, not specific > files. Sorry for that. > > > > > For the good news (for me anyway), I'm also trying to learn Python > > > > (Diving Into Python) on my own time, so that hopefully one day I > > might > > > > be able help squash some of these bugs.. However, in the meantime, > > > > I/we very much appreciate your help and expertise.. > > > > > > > > REGRESSION TEST RESULTS ARE BELOW... > > > > > > > > Dan Williams wrote on 07/15/2006 01:20:54 PM: > > > > > > > > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > > > > > > > > > > > > Hi, > > > > > > I'm having a problem *not* being able to build 'i686' packages > > > > (i.e. > > > > > > optional_arches=i686) anymore *after* having upgraded our > > plague > > > > > > server/builder (Opteron x86_64) a couple of weeks ago from > > > > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with > > i686 > > > > > > (optional_arches=i686) -- not with i386 (base_arches=i386) > > which > > > > > > continues to work flawlessly. > > > > > > > > > > Found and fixed in CVS HEAD. Patch attached that should apply > > > > against > > > > > an installed copy of plague-server as well. Thanks for the > > report, > > > > and > > > > > sorry for the lag. > > > > > > > > > > Cheers, > > > > > Dan > > > > > > > > > > > > > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe > > > > > Todaro/Poughkeepsie/IBM] > > > > > > > > > > > > 626 (WhoAmI): Requesting depsolve... > > > > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686', > > > > 'noarch']. > > > > > > Ok; what do you have in the 'optional_arches' for this target? > > > > ====================================== > > Excerpt from lnxaddons-100-install.cfg > > ====================================== > > > > ----------------------- > > [Arches] > > base_arches=i386 > > optional_arches=i686 noarch > > Ok; I think this is the cause of the noarch problem. Since 'noarch' is > a base architecture in itself, you don't need it in optional_arches. If > you remove it, I think things will work as you expect. Every builder > can build a 'noarch' package, so that architecture is essentially added > by default and you don't need to do anything explicit for it. Yay for > documentation (or complete lack of it). > > However, this does expose a bug. If an architecture is listed in > optional_arches, it shouldn't actually be used by a package build unless > the package was going to build for noarch already. > > > ---------------------- > > > > > > What do the top bits of the specfile for WhoAmI look like? i.e., > > are > > > there any of the following specfile tags, and what are their values? > > > > > > BuildArch > > > ExclusiveArch > > > ExcludeArch > > > > > > > ======================== > > Excerpt from WhoAmI.spec > > ======================== > > > > ----------------------- > > %define rversion 4.00 > > %define rel 9 > > %define pkgname WhoAmI > > > > Summary : %{__distribution_long} system information > > Name : %{pkgname} > > Version : %{rversion} > > Release : %{rel}_%{__build_distribution}%{__build_release} > > License : %__spec_internal_license > > Group : System Environment/Base > > Url : %__repository_url > > > > Packager : %{packager} > > > > Source0 : %{pkgname}.tar.gz > > > > %if %__build_rhel > > Requires : coreutils, bash, python, grep, rpm > > %endif > > > > %if %__build_suse > > Requires : coreutils, bash, aaa_base, python, grep, rpm > > %endif > > > > BuildArch : noarch > > I'm getting slightly confused here. Plague 0.4.3 was building this > package correctly as an i686 package? Do you really _want_ this to > build as an i686 package? If so, I think the "BuildArch: noarch" is > wrong then. Typically, if you want a package to build noarch, you > specify "BuildArch: noarch" and it will then only build on noarch. What > arch is this package supposed to be for, exactly? > > Cheers, > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From williams at redhat.com Mon Jul 17 23:02:43 2006 From: williams at redhat.com (Clark Williams) Date: Mon, 17 Jul 2006 18:02:43 -0500 Subject: proposed mock changes (diff) In-Reply-To: <11DAAFDC-E917-43FC-96D0-C41718FAEC10@cs.ucsb.edu> References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> <44BBF9CA.7090207@redhat.com> <11DAAFDC-E917-43FC-96D0-C41718FAEC10@cs.ucsb.edu> Message-ID: <44BC1713.1080307@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Sheltren wrote: > > On Jul 17, 2006, at 4:57 PM, Clark Williams wrote: > >> >> Now, I really don't like the idea of adding *anything* to the default >> build chroot (other than what's required to build a package). It's >> slow enough as it is and for the most common usage (plague/mock >> builds) we don't want anything except what's absolutely needed. So, >> how would we go about managing a chroot for tasks like testing? Do we >> add an 'install' command to install stuff from the configured >> repositories into the chroot, like this: >> >> $ mock install rpmlint >> >> Or do we do something else? Do we make tester's put together their own >> cfg with a modified buildsys-build rpm that includes the packages they >> need? Is there a simpler mechanism for someone that wants a chroot for >> testing? >> >> Clark >> > > I've always just used yum from the host system with the > --installroot flag. You don't want to rely upon the RPM tools > inside of the chroot because those won't always work - remember, we > are doing all the package installs with the host system's yum in the > first place. My vote would be to keep mock as simple/minimal as > possible. > Ah, I hadn't thought about that. RPM and friends work fine in the FC5 chroot on my FC5 laptop, but I'm betting an FC2 chroot would yak all over the place. :) So someone wanting to put together a testing chroot would do something like this: $ mock -r fedora-5-i386-core init $ yum --installroot=/var/lib/mock/fedora-5-i386-core/root - --enablerepo= install $ mock -r fedora-5-i386-core chroot I probably don't have that yum command right, but you should get my intent. Is that the way you use it? Clark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEvBcTHyuj/+TTEp0RAgN2AJ0RF1KLvomfnbA8/g/9w2sDUZm6RQCeMvnB TxuUnKmqAkcAQJjeDCbF/iU= =174P -----END PGP SIGNATURE----- From jkeating at j2solutions.net Mon Jul 17 23:20:07 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Mon, 17 Jul 2006 19:20:07 -0400 Subject: proposed mock changes (diff) In-Reply-To: <44BC1713.1080307@redhat.com> References: <35C9A9D68AB3FA4AB63692802656D9EC010F44F9@ausx3mps303.aus.amer.dell.com> <11DAAFDC-E917-43FC-96D0-C41718FAEC10@cs.ucsb.edu> <44BC1713.1080307@redhat.com> Message-ID: <200607171920.10725.jkeating@j2solutions.net> On Monday 17 July 2006 19:02, Clark Williams wrote: > $ mock -r fedora-5-i386-core init > $ yum --installroot=/var/lib/mock/fedora-5-i386-core/root > --enablerepo= install > $ mock -r fedora-5-i386-core chroot > > I probably don't have that yum command right, but you should get my > intent. Is that the way you use it? Yes, that's how we run buildinstall for Fedora tree spinning. mock -r dist-fc6 init mockhelper yum --installroot=/var/lib/mock/dist-fc6-/root anaconda-runtime mock -r dist-fc6 chroot buildinstall Of course, in there we have a mount call so that we run buildinstall against an NFS mount and write back out to said NFS mount. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From dcbw at redhat.com Wed Jul 19 14:07:54 2006 From: dcbw at redhat.com (Dan Williams) Date: Wed, 19 Jul 2006 10:07:54 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1153318074.2537.9.camel@localhost.localdomain> On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote: > DW> and on the server, which has _no_ architecture at all, you'd have > DW> in /etc/plague/server/targets: > DW> > DW> fedora-devel-core.cfg > DW> fedora-5-core.cfg > DW> fedora-4-core.cfg > DW> fedora-devel-extras.cfg > DW> fedora-5-extras.cfg > DW> fedora-4-extras.cfg > DW> > DW> and on the server, if you care about depsolving (ie if you have > the > DW> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) > then > DW> you actually do need mock target configs for everything too, > DW> because the server will use those to depsolve. > > Absolutely, we do care about depsolving.. > > So, when you say "everything", does that also include (for example) > 'i686' now, too, regardless of the fact that i686 is still only a > sub-architecture? No; you only need mock and builder configs for base architectures like i386, x86-64, ppc, sparc, sparc64, ppc64, etc. > DW> Do you really _want_ this to > DW> build as an i686 package? > > No, of course not - this is a noarch only package, and always > has been. > > DW> If so, I think the "BuildArch: noarch" is > DW> wrong then. Typically, if you want a package to build noarch, you > DW> specify "BuildArch: noarch" and it will then only build on > noarch. What > DW> arch is this package supposed to be for, exactly? > > 'noarch' as is specified in the specfile. Ok, can you apply the attached patch? cd to /usr/share/plague/server and use 'patch -p0 < patchfile'. Enqueue the WhoAmI SRPM and let me know what it prints out. I tried running a noarch package through this morning and it seemed to go through fine; plague decided that 'noarch' was the only arch it was going to build for, even with i686 in the optional_arches for that target. > JT> > ====================================== > JT> > Excerpt from lnxaddons-100-install.cfg > JT> > ====================================== > JT> > > JT> > ----------------------- > JT> > [Arches] > JT> > base_arches=i386 > JT> > optional_arches=i686 noarch > JT> > > > GW> Ok; I think this is the cause of the noarch problem. Since > 'noarch' is > GW> a base architecture in itself, you don't need it in > optional_arches. If > GW> you remove it, I think things will work as you expect. > > Nope - no such luck; I have removed 'noarch' from 'optional_arches', > re-tested, and am still seeing that same error.. noarch shouldn't be in optional_arches, but if it's not working correctly when you remove it there's some other issue. Dan -------------- next part -------------- A non-text attachment was scrubbed... Name: print-archhandling.patch Type: text/x-patch Size: 2144 bytes Desc: not available URL: From jstodaro at us.ibm.com Wed Jul 19 17:09:47 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Wed, 19 Jul 2006 13:09:47 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: <1153318074.2537.9.camel@localhost.localdomain> Message-ID: Dan Williams wrote on 07/19/2006 10:07:54 AM: > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote: > > DW> and on the server, which has _no_ architecture at all, you'd have > > DW> in /etc/plague/server/targets: > > DW> > > DW> fedora-devel-core.cfg > > DW> fedora-5-core.cfg > > DW> fedora-4-core.cfg > > DW> fedora-devel-extras.cfg > > DW> fedora-5-extras.cfg > > DW> fedora-4-extras.cfg > > DW> > > DW> and on the server, if you care about depsolving (ie if you have > > the > > DW> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) > > then > > DW> you actually do need mock target configs for everything too, > > DW> because the server will use those to depsolve. > > > > Absolutely, we do care about depsolving.. > > > > So, when you say "everything", does that also include (for example) > > 'i686' now, too, regardless of the fact that i686 is still only a > > sub-architecture? > > No; you only need mock and builder configs for base architectures like > i386, x86-64, ppc, sparc, sparc64, ppc64, etc. > > > DW> Do you really _want_ this to > > DW> build as an i686 package? > > > > No, of course not - this is a noarch only package, and always > > has been. > > > > DW> If so, I think the "BuildArch: noarch" is > > DW> wrong then. Typically, if you want a package to build noarch, you > > DW> specify "BuildArch: noarch" and it will then only build on > > noarch. What > > DW> arch is this package supposed to be for, exactly? > > > > 'noarch' as is specified in the specfile. > > Ok, can you apply the attached patch? cd to /usr/share/plague/server > and use 'patch -p0 < patchfile'. Enqueue the WhoAmI SRPM and let me > know what it prints out. Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. Below are the results. Hope you find something - good luck.. And, thanks a lot for all the help you've been providing me, which is very much appreciated. ===================================== below from /var/log/plague-server.log ===================================== Request to enqueue 'WhoAmI' tag '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for target 'ao100' (user 'jstodaro at abc.com') 630 (WhoAmI): Starting tag '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on target 'lnxaddons-100-install' 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] 630 (WhoAmI): AH: addl_arches [] 630 (WhoAmI): AH: base_arches ['i386', 'i686'] 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, allowed: None 630 (WhoAmI): Requesting depsolve... 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch i686. Exception in thread PackageJob: 630/WhoAmI: Traceback (most recent call last): File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap self.run() File "/usr/share/plague/server/PackageJob.py", line 86, in run self._pkg_job.process() File "/usr/share/plague/server/PackageJob.py", line 725, in process if func(): File "/usr/share/plague/server/PackageJob.py", line 593, in _stage_depsolve if self._arch_deps_solved(arch) == False: File "/usr/share/plague/server/PackageJob.py", line 546, in _arch_deps_solved if base: UnboundLocalError: local variable 'base' referenced before assignment Re-activating builder 'https://lnxbuild1.abc.com:8888'. ===================================== above from /var/log/plague-server.log ===================================== > I tried running a noarch package through this > morning and it seemed to go through fine; plague decided that 'noarch' > was the only arch it was going to build for, even with i686 in the > optional_arches for that target. > > > JT> > ====================================== > > JT> > Excerpt from lnxaddons-100-install.cfg > > JT> > ====================================== > > JT> > > > JT> > ----------------------- > > JT> > [Arches] > > JT> > base_arches=i386 > > JT> > optional_arches=i686 noarch > > JT> > > > > > GW> Ok; I think this is the cause of the noarch problem. Since > > 'noarch' is > > GW> a base architecture in itself, you don't need it in > > optional_arches. If > > GW> you remove it, I think things will work as you expect. > > > > Nope - no such luck; I have removed 'noarch' from 'optional_arches', > > re-tested, and am still seeing that same error.. > > noarch shouldn't be in optional_arches, but if it's not working > correctly when you remove it there's some other issue. > > Dan > > [attachment "print-archhandling.patch" deleted by Joe > Todaro/Poughkeepsie/IBM] -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcbw at redhat.com Wed Jul 19 18:07:05 2006 From: dcbw at redhat.com (Dan Williams) Date: Wed, 19 Jul 2006 14:07:05 -0400 Subject: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1153332425.3642.1.camel@localhost.localdomain> On Wed, 2006-07-19 at 13:09 -0400, Joe Todaro wrote: > > Dan Williams wrote on 07/19/2006 10:07:54 AM: > > > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote: > > > DW> and on the server, which has _no_ architecture at all, you'd > have > > > DW> in /etc/plague/server/targets: > > > DW> > > > DW> fedora-devel-core.cfg > > > DW> fedora-5-core.cfg > > > DW> fedora-4-core.cfg > > > DW> fedora-devel-extras.cfg > > > DW> fedora-5-extras.cfg > > > DW> fedora-4-extras.cfg > > > DW> > > > DW> and on the server, if you care about depsolving (ie if you > have > > > the > > > DW> "depsolve_jobs = yes" set > in /etc/plague/server/plague-server.cfg) > > > then > > > DW> you actually do need mock target configs for everything too, > > > DW> because the server will use those to depsolve. > > > > > > Absolutely, we do care about depsolving.. > > > > > > So, when you say "everything", does that also include (for > example) > > > 'i686' now, too, regardless of the fact that i686 is still only a > > > > sub-architecture? > > > > No; you only need mock and builder configs for base architectures > like > > i386, x86-64, ppc, sparc, sparc64, ppc64, etc. > > > > > DW> Do you really _want_ this to > > > DW> build as an i686 package? > > > > > > No, of course not - this is a noarch only package, and always > > > has been. > > > > > > DW> If so, I think the "BuildArch: noarch" is > > > DW> wrong then. Typically, if you want a package to build noarch, > you > > > DW> specify "BuildArch: noarch" and it will then only build on > > > noarch. What > > > DW> arch is this package supposed to be for, exactly? > > > > > > 'noarch' as is specified in the specfile. > > > > Ok, can you apply the attached patch? cd > to /usr/share/plague/server > > and use 'patch -p0 < patchfile'. Enqueue the WhoAmI SRPM and let me > > know what it prints out. > > Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. > Below are the results. Hope you find something - good luck.. > And, thanks a lot for all the help you've been providing me, > which is very much appreciated. > > ===================================== > below from /var/log/plague-server.log > ===================================== > Request to enqueue 'WhoAmI' tag > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for > target 'ao100' (user 'jstodaro at abc.com') > 630 (WhoAmI): Starting tag > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on > target 'lnxaddons-100-install' > 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] > 630 (WhoAmI): AH: addl_arches [] > 630 (WhoAmI): AH: base_arches ['i386', 'i686'] > 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, allowed: > None Ok; this tells me that the PackageJob class' arch_handling() function is working as expected. It also tells me that the depsolve stuff is picking the wrong arches to depsolve. So we're closer in some senses to fixing the issue, but farther away in others :) Dan > 630 (WhoAmI): Requesting depsolve... > 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. > 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch > i686. > Exception in thread PackageJob: 630/WhoAmI: > Traceback (most recent call last): > File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap > self.run() > File "/usr/share/plague/server/PackageJob.py", line 86, in run > self._pkg_job.process() > File "/usr/share/plague/server/PackageJob.py", line 725, in process > if func(): > File "/usr/share/plague/server/PackageJob.py", line 593, in > _stage_depsolve > if self._arch_deps_solved(arch) == False: > File "/usr/share/plague/server/PackageJob.py", line 546, in > _arch_deps_solved > if base: > UnboundLocalError: local variable 'base' referenced before assignment > > Re-activating builder 'https://lnxbuild1.abc.com:8888'. > ===================================== > above from /var/log/plague-server.log > ===================================== > > > I tried running a noarch package through this > > morning and it seemed to go through fine; plague decided that > 'noarch' > > was the only arch it was going to build for, even with i686 in the > > optional_arches for that target. > > > > > JT> > ====================================== > > > JT> > Excerpt from lnxaddons-100-install.cfg > > > JT> > ====================================== > > > JT> > > > > JT> > ----------------------- > > > JT> > [Arches] > > > JT> > base_arches=i386 > > > JT> > optional_arches=i686 noarch > > > JT> > > > > > > > GW> Ok; I think this is the cause of the noarch problem. Since > > > 'noarch' is > > > GW> a base architecture in itself, you don't need it in > > > optional_arches. If > > > GW> you remove it, I think things will work as you expect. > > > > > > Nope - no such luck; I have removed 'noarch' from > 'optional_arches', > > > re-tested, and am still seeing that same error.. > > > > noarch shouldn't be in optional_arches, but if it's not working > > correctly when you remove it there's some other issue. > > > > Dan > > > > [attachment "print-archhandling.patch" deleted by Joe > > Todaro/Poughkeepsie/IBM] From jstodaro at us.ibm.com Fri Jul 21 20:48:38 2006 From: jstodaro at us.ibm.com (Joe Todaro) Date: Fri, 21 Jul 2006 16:48:38 -0400 Subject: (FIXED) Re: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: <1153332425.3642.1.camel@localhost.localdomain> Message-ID: Hey Dan, Great news. We *fixed* the plague-0.5 "depsolve" problem with a minor change to the PackageJob.py module. See below... >> On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: >> >> Hi, >> I'm having a problem *not* being able to build 'i686' packages (i.e. >> optional_arches=i686) anymore *after* having upgraded our plague >> server/builder (Opteron x86_64) a couple of weeks ago from >> plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 >> (optional_arches=i686) -- not with i386 (base_arches=i386) which >> continues to work flawlessly. > Dan Williams wrote on 07/19/2006 02:07:05 PM: > Ok, can you apply the attached patch? > cd to /usr/share/plague/server > and use 'patch -p0 < patchfile'. > Enqueue the WhoAmI SRPM and let me > know what it prints out. > ------- snip - snip ------ > > Ok; this tells me that the PackageJob class' arch_handling() function is > working as expected. It also tells me that the depsolve stuff is > picking the wrong arches to depsolve. So we're closer in some senses to > fixing the issue, but farther away in others :) > In _write_yum_conf() we changed: mock_config = self._target_cfg.mock_config_for_arch(arch) to: mock_config = self._target_cfg.mock_config_for_arch(ArchUtils.sub_arches[arch]) *that* did the trick! ... our new (plague-0.5 based) buildsys is up and running the same as it was *before* we upgraded from plague-0.4. So, please _confirm_ the above change - that it's *really* what you intended to do? If it is, I will be happy to re-test the code after you commit the changes to the CVS HEAD. Well, thanks again for all your help, especially for clarifying some of the key Plague buildsystem design points, without which it would have been impossible to debug/resolve the above "depsolve" issue. Also, this provided the opportunity to dig into some well-written Python code and do some serious debugging. Thanks again! Regards, Joe Todaro =============== HERE"S THE COMPLETE THREAD ============== Dan Williams wrote on 07/19/2006 02:07:05 PM: > On Wed, 2006-07-19 at 13:09 -0400, Joe Todaro wrote: > > > > Dan Williams wrote on 07/19/2006 10:07:54 AM: > > > > > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote: > > > > DW> and on the server, which has _no_ architecture at all, you'd > > have > > > > DW> in /etc/plague/server/targets: > > > > DW> > > > > DW> fedora-devel-core.cfg > > > > DW> fedora-5-core.cfg > > > > DW> fedora-4-core.cfg > > > > DW> fedora-devel-extras.cfg > > > > DW> fedora-5-extras.cfg > > > > DW> fedora-4-extras.cfg > > > > DW> > > > > DW> and on the server, if you care about depsolving (ie if you > > have > > > > the > > > > DW> "depsolve_jobs = yes" set > > in /etc/plague/server/plague-server.cfg) > > > > then > > > > DW> you actually do need mock target configs for everything too, > > > > DW> because the server will use those to depsolve. > > > > > > > > Absolutely, we do care about depsolving.. > > > > > > > > So, when you say "everything", does that also include (for > > example) > > > > 'i686' now, too, regardless of the fact that i686 is still only a > > > > > > sub-architecture? > > > > > > No; you only need mock and builder configs for base architectures > > like > > > i386, x86-64, ppc, sparc, sparc64, ppc64, etc. > > > > > > > DW> Do you really _want_ this to > > > > DW> build as an i686 package? > > > > > > > > No, of course not - this is a noarch only package, and always > > > > has been. > > > > > > > > DW> If so, I think the "BuildArch: noarch" is > > > > DW> wrong then. Typically, if you want a package to build noarch, > > you > > > > DW> specify "BuildArch: noarch" and it will then only build on > > > > noarch. What > > > > DW> arch is this package supposed to be for, exactly? > > > > > > > > 'noarch' as is specified in the specfile. > > > > > > Ok, can you apply the attached patch? cd > > to /usr/share/plague/server > > > and use 'patch -p0 < patchfile'. Enqueue the WhoAmI SRPM and let me > > > know what it prints out. > > > > Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. > > Below are the results. Hope you find something - good luck.. > > And, thanks a lot for all the help you've been providing me, > > which is very much appreciated. > > > > ===================================== > > below from /var/log/plague-server.log > > ===================================== > > Request to enqueue 'WhoAmI' tag > > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for > > target 'ao100' (user 'jstodaro at abc.com') > > 630 (WhoAmI): Starting tag > > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on > > target 'lnxaddons-100-install' > > 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] > > 630 (WhoAmI): AH: addl_arches [] > > 630 (WhoAmI): AH: base_arches ['i386', 'i686'] > > 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, allowed: > > None > > Ok; this tells me that the PackageJob class' arch_handling() function is > working as expected. It also tells me that the depsolve stuff is > picking the wrong arches to depsolve. So we're closer in some senses to > fixing the issue, but farther away in others :) > > Dan > > > 630 (WhoAmI): Requesting depsolve... > > 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. > > 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch > > i686. > > Exception in thread PackageJob: 630/WhoAmI: > > Traceback (most recent call last): > > File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap > > self.run() > > File "/usr/share/plague/server/PackageJob.py", line 86, in run > > self._pkg_job.process() > > File "/usr/share/plague/server/PackageJob.py", line 725, in process > > if func(): > > File "/usr/share/plague/server/PackageJob.py", line 593, in > > _stage_depsolve > > if self._arch_deps_solved(arch) == False: > > File "/usr/share/plague/server/PackageJob.py", line 546, in > > _arch_deps_solved > > if base: > > UnboundLocalError: local variable 'base' referenced before assignment > > > > Re-activating builder 'https://lnxbuild1.abc.com:8888'. > > ===================================== > > above from /var/log/plague-server.log > > ===================================== > > > > > I tried running a noarch package through this > > > morning and it seemed to go through fine; plague decided that > > 'noarch' > > > was the only arch it was going to build for, even with i686 in the > > > optional_arches for that target. > > > > > > > JT> > ====================================== > > > > JT> > Excerpt from lnxaddons-100-install.cfg > > > > JT> > ====================================== > > > > JT> > > > > > JT> > ----------------------- > > > > JT> > [Arches] > > > > JT> > base_arches=i386 > > > > JT> > optional_arches=i686 noarch > > > > JT> > > > > > > > > > GW> Ok; I think this is the cause of the noarch problem. Since > > > > 'noarch' is > > > > GW> a base architecture in itself, you don't need it in > > > > optional_arches. If > > > > GW> you remove it, I think things will work as you expect. > > > > > > > > Nope - no such luck; I have removed 'noarch' from > > 'optional_arches', > > > > re-tested, and am still seeing that same error.. > > > > > > noarch shouldn't be in optional_arches, but if it's not working > > > correctly when you remove it there's some other issue. > > > > > > Dan > > > > > > [attachment "print-archhandling.patch" deleted by Joe > > > Todaro/Poughkeepsie/IBM] > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcbw at redhat.com Sat Jul 22 15:11:43 2006 From: dcbw at redhat.com (Dan Williams) Date: Sat, 22 Jul 2006 11:11:43 -0400 Subject: (FIXED) Re: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3) In-Reply-To: References: Message-ID: <1153581103.2546.3.camel@localhost.localdomain> On Fri, 2006-07-21 at 16:48 -0400, Joe Todaro wrote: > > Hey Dan, > > Great news. We *fixed* the plague-0.5 "depsolve" problem with a minor > change to the PackageJob.py module. See below... > > >> On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote: > >> > >> Hi, > >> I'm having a problem *not* being able to build 'i686' packages > (i.e. > >> optional_arches=i686) anymore *after* having upgraded our plague > >> server/builder (Opteron x86_64) a couple of weeks ago from > >> plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686 > >> (optional_arches=i686) -- not with i386 (base_arches=i386) which > >> continues to work flawlessly. > > > Dan Williams wrote on 07/19/2006 02:07:05 PM: > > Ok, can you apply the attached patch? > > cd to /usr/share/plague/server > > and use 'patch -p0 < patchfile'. > > Enqueue the WhoAmI SRPM and let me > > know what it prints out. > > > ------- snip - snip ------ > > > > Ok; this tells me that the PackageJob class' arch_handling() > function is > > working as expected. It also tells me that the depsolve stuff is > > picking the wrong arches to depsolve. So we're closer in some > senses to > > fixing the issue, but farther away in others :) > > > > In _write_yum_conf() we changed: > > mock_config = self._target_cfg.mock_config_for_arch(arch) > > to: > > mock_config = > self._target_cfg.mock_config_for_arch(ArchUtils.sub_arches[arch]) > > *that* did the trick! ... our new (plague-0.5 based) buildsys is > up and running the same as it was *before* we upgraded from > plague-0.4. > > So, please _confirm_ the above change - that it's *really* what you > intended to do? If it is, I will be happy to re-test the code > after you commit the changes to the CVS HEAD. Yes, that looks correct. Usually we won't have a mock config file for arches like i686 or em64t, since it's assumed that if you have a builder for the sub-arch, you can build the whole list of arches, usually. That's a somewhat erroneous assumption, but in reality, if you have an i386-class machine, it's _not_ actually going to be an i386 from 1992 :) I've committed a patch to HEAD that will try the specified arch (just in case you do, in fact, have mock configs for the specific build arch) and then if there's no mock config for that arch, will fall back to the base architecture like your change above. Thanks for the pointer. Patch attached. Dan > Well, thanks again for all your help, especially for clarifying some > of the key Plague buildsystem design points, without which it would > have been impossible to debug/resolve the above "depsolve" issue. > > Also, this provided the opportunity to dig into some well-written > Python code and do some serious debugging. Thanks again! > > Regards, > Joe Todaro > > =============== HERE"S THE COMPLETE THREAD ============== > Dan Williams wrote on 07/19/2006 02:07:05 PM: > > > On Wed, 2006-07-19 at 13:09 -0400, Joe Todaro wrote: > > > > > > Dan Williams wrote on 07/19/2006 10:07:54 AM: > > > > > > > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote: > > > > > DW> and on the server, which has _no_ architecture at all, > you'd > > > have > > > > > DW> in /etc/plague/server/targets: > > > > > DW> > > > > > DW> fedora-devel-core.cfg > > > > > DW> fedora-5-core.cfg > > > > > DW> fedora-4-core.cfg > > > > > DW> fedora-devel-extras.cfg > > > > > DW> fedora-5-extras.cfg > > > > > DW> fedora-4-extras.cfg > > > > > DW> > > > > > DW> and on the server, if you care about depsolving (ie if you > > > have > > > > > the > > > > > DW> "depsolve_jobs = yes" set > > > in /etc/plague/server/plague-server.cfg) > > > > > then > > > > > DW> you actually do need mock target configs for everything > too, > > > > > DW> because the server will use those to depsolve. > > > > > > > > > > Absolutely, we do care about depsolving.. > > > > > > > > > > So, when you say "everything", does that also include (for > > > example) > > > > > 'i686' now, too, regardless of the fact that i686 is still > only a > > > > > > > > sub-architecture? > > > > > > > > No; you only need mock and builder configs for base > architectures > > > like > > > > i386, x86-64, ppc, sparc, sparc64, ppc64, etc. > > > > > > > > > DW> Do you really _want_ this to > > > > > DW> build as an i686 package? > > > > > > > > > > No, of course not - this is a noarch only package, and always > > > > > has been. > > > > > > > > > > DW> If so, I think the "BuildArch: noarch" is > > > > > DW> wrong then. Typically, if you want a package to build > noarch, > > > you > > > > > DW> specify "BuildArch: noarch" and it will then only build on > > > > > noarch. What > > > > > DW> arch is this package supposed to be for, exactly? > > > > > > > > > > 'noarch' as is specified in the specfile. > > > > > > > > Ok, can you apply the attached patch? cd > > > to /usr/share/plague/server > > > > and use 'patch -p0 < patchfile'. Enqueue the WhoAmI SRPM and > let me > > > > know what it prints out. > > > > > > Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. > > > Below are the results. Hope you find something - good luck.. > > > And, thanks a lot for all the help you've been providing me, > > > which is very much appreciated. > > > > > > ===================================== > > > below from /var/log/plague-server.log > > > ===================================== > > > Request to enqueue 'WhoAmI' tag > > > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' > for > > > target 'ao100' (user 'jstodaro at abc.com') > > > 630 (WhoAmI): Starting tag > > > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' > on > > > target 'lnxaddons-100-install' > > > 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] > > > 630 (WhoAmI): AH: addl_arches [] > > > 630 (WhoAmI): AH: base_arches ['i386', 'i686'] > > > 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, > allowed: > > > None > > > > Ok; this tells me that the PackageJob class' arch_handling() > function is > > working as expected. It also tells me that the depsolve stuff is > > picking the wrong arches to depsolve. So we're closer in some > senses to > > fixing the issue, but farther away in others :) > > > > Dan > > > > > 630 (WhoAmI): Requesting depsolve... > > > 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. > > > 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for > arch > > > i686. > > > Exception in thread PackageJob: 630/WhoAmI: > > > Traceback (most recent call last): > > > File "/usr/lib64/python2.3/threading.py", line 436, in > __bootstrap > > > self.run() > > > File "/usr/share/plague/server/PackageJob.py", line 86, in run > > > self._pkg_job.process() > > > File "/usr/share/plague/server/PackageJob.py", line 725, in > process > > > if func(): > > > File "/usr/share/plague/server/PackageJob.py", line 593, in > > > _stage_depsolve > > > if self._arch_deps_solved(arch) == False: > > > File "/usr/share/plague/server/PackageJob.py", line 546, in > > > _arch_deps_solved > > > if base: > > > UnboundLocalError: local variable 'base' referenced before > assignment > > > > > > Re-activating builder 'https://lnxbuild1.abc.com:8888'. > > > ===================================== > > > above from /var/log/plague-server.log > > > ===================================== > > > > > > > I tried running a noarch package through this > > > > morning and it seemed to go through fine; plague decided that > > > 'noarch' > > > > was the only arch it was going to build for, even with i686 in > the > > > > optional_arches for that target. > > > > > > > > > JT> > ====================================== > > > > > JT> > Excerpt from lnxaddons-100-install.cfg > > > > > JT> > ====================================== > > > > > JT> > > > > > > JT> > ----------------------- > > > > > JT> > [Arches] > > > > > JT> > base_arches=i386 > > > > > JT> > optional_arches=i686 noarch > > > > > JT> > > > > > > > > > > > GW> Ok; I think this is the cause of the noarch problem. > Since > > > > > 'noarch' is > > > > > GW> a base architecture in itself, you don't need it in > > > > > optional_arches. If > > > > > GW> you remove it, I think things will work as you expect. > > > > > > > > > > Nope - no such luck; I have removed 'noarch' from > > > 'optional_arches', > > > > > re-tested, and am still seeing that same error.. > > > > > > > > noarch shouldn't be in optional_arches, but if it's not working > > > > correctly when you remove it there's some other issue. > > > > > > > > Dan > > > > > > > > [attachment "print-archhandling.patch" deleted by Joe > > > > Todaro/Poughkeepsie/IBM] > > -------------- next part -------------- A non-text attachment was scrubbed... Name: depsolve-arch.patch Type: text/x-patch Size: 1288 bytes Desc: not available URL: From dennis at ausil.us Sat Jul 22 17:00:58 2006 From: dennis at ausil.us (Dennis Gilmore) Date: Sat, 22 Jul 2006 12:00:58 -0500 Subject: PATCH: niagara support for plague Message-ID: <200607221200.58784.dennis@ausil.us> Hey all the attached patch adds support for Suns niagara arch to plague. -- Dennis Gilmore, RHCE Proud Australian -------------- next part -------------- A non-text attachment was scrubbed... Name: niagara.patch Type: text/x-diff Size: 2032 bytes Desc: not available URL: From dcbw at redhat.com Wed Jul 26 00:34:15 2006 From: dcbw at redhat.com (Dan Williams) Date: Tue, 25 Jul 2006 19:34:15 -0500 Subject: PATCH: niagara support for plague In-Reply-To: <200607221200.58784.dennis@ausil.us> References: <200607221200.58784.dennis@ausil.us> Message-ID: <1153874055.1154.2.camel@localhost.localdomain> On Sat, 2006-07-22 at 12:00 -0500, Dennis Gilmore wrote: > Hey all > > the attached patch adds support for Suns niagara arch to plague. Committed to HEAD, thanks. How's 0.5 working out for you so far? Dan From dennis at ausil.us Wed Jul 26 01:03:12 2006 From: dennis at ausil.us (Dennis Gilmore) Date: Tue, 25 Jul 2006 20:03:12 -0500 Subject: PATCH: niagara support for plague In-Reply-To: <1153874055.1154.2.camel@localhost.localdomain> References: <200607221200.58784.dennis@ausil.us> <1153874055.1154.2.camel@localhost.localdomain> Message-ID: <200607252003.14197.dennis@ausil.us> On Tuesday 25 July 2006 7:34 pm, Dan Williams wrote: > On Sat, 2006-07-22 at 12:00 -0500, Dennis Gilmore wrote: > > Hey all > > > > the attached patch adds support for Suns niagara arch to plague. > > Committed to HEAD, thanks. How's 0.5 working out for you so far? > > Dan Seems to be really good so far. Im using active builders, I havent turned depsolving on yet. that will be my next job. spot has added niagara support to rpm. it still needs to be added to gcc and glibc but its coming along. -- Dennis Gilmore, RHCE Proud Australian From michael at knox.net.nz Thu Jul 27 03:13:38 2006 From: michael at knox.net.nz (Michael J. Knox) Date: Thu, 27 Jul 2006 15:13:38 +1200 Subject: manpage patch. Message-ID: <44C82F62.2070302@knox.net.nz> Patch adds missing options to the manpage: --autocache Turn on build-root caching --rebuildcache Force rebuild of build-root cache Michael -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: missing_options.patch URL: