From williams at redhat.com Thu Mar 2 16:09:05 2006 From: williams at redhat.com (Clark Williams) Date: Thu, 02 Mar 2006 10:09:05 -0600 Subject: Add a chroot command to mock? Message-ID: <1141315745.16196.94.camel@localhost.localdomain> When cross-building a root filesystem, you need to bootstrap something we call a "sysroot", which is essentially a software development environment for the target system (headers and libraries mostly). The way we do that is we create an empty "root" and as we build RPMS (starting with GLIBC), we install the -devel packages into that target root (the sysroot). Our cross toolchains know how to access the sysroot, so when you compile hello.c, it gets the correct headers and libraries. To be able to install rpms into the sysroot (which lives in the mock chroot) I've added a chroot command to my local copy of mock (patch below). Anyone object to committing this? Clark Index: mock.py =================================================================== RCS file: /cvs/fedora/mock/mock.py,v retrieving revision 1.36 diff -u -r1.36 mock.py --- mock.py 24 Feb 2006 19:31:57 -0000 1.36 +++ mock.py 2 Mar 2006 15:48:01 -0000 @@ -781,6 +781,12 @@ my.close() print 'Finished initializing root' + elif args[0] == 'chroot': + config_opts['clean'] = False + my= Root(config_opts) + my.do_chroot(' '.join(args[1:]), True) + print 'Finished chroot command' + else: if args[0] == 'rebuild': if len(args) > 1: -- Clark Williams -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From enrico.scholz at informatik.tu-chemnitz.de Mon Mar 6 13:12:17 2006 From: enrico.scholz at informatik.tu-chemnitz.de (Enrico Scholz) Date: Mon, 06 Mar 2006 14:12:17 +0100 Subject: [ANN] plague-client++ (with proxy support) Message-ID: <87bqwjra32.fsf@kosh.bigo.ensc.de> Hello, this program is lingering around on my harddisk for some while already and I found now time for some cleanups to announce it officially. 'plague-client++' is an alternative plague-client implementation which uses a professional HTTP transport library (curl) instead of botching around with python's urllib resp. with direct sockets. So, 'plague-client++' works behind HTTP proxies. Currently, it was written completely for my purposes so it has the following cons: * no configuration files; it uses the default paths and urls for the certificates; when you need to change them, you have to edit src/PlagueOptions.cc * no support for src.rpm or local plague operations A snapshot is available at http://ensc.de/fedora/ or you can checkout it from the GNU Arch/Bazaar repository: tla register-archive ensc at ensc.de--fedora http://ensc.de/tla/{archives}/fedora baz get ensc at ensc.de--fedora/plague-clientxx--mainline--0.1 Enrico -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 482 bytes Desc: not available URL: From Matt_Domsch at dell.com Tue Mar 7 04:13:14 2006 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 6 Mar 2006 22:13:14 -0600 Subject: OT: using mock to build for SLES Message-ID: <20060307041314.GA21949@lists.us.dell.com> I've been looking at using mock to build srpms for Novell/SuSE SLES product, rather than using their build.rpm script. With one little change to mock (below), seems to work pretty well. opensuse-10.x is still a little weird, but SLES9 x86 and x86_64 seem to be building fine. Here's the mock patch to pass -m to useradd, and I'll follow up with the buildroots.xml and mock cfg file example too. Thanks, Matt -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com --- mock.orig 2006-03-05 20:58:49.000000000 -0600 +++ mock 2006-03-06 08:30:51.000000000 -0600 @@ -556,8 +556,8 @@ if not os.path.exists(self.rootdir + self.homedir): if not os.path.exists(os.path.join(self.rootdir, 'usr/sbin/useradd')): raise RootError, "Could not find useradd in chroot, maybe the install failed?" - cmd = '/usr/sbin/useradd -u %s -d %s %s' % (self.config['chrootuid'], - self.homedir, self.config['chrootuser']) + cmd = '/usr/sbin/useradd -m -u %s -d %s %s' % (self.config['chrootuid'], + self.homedir, self.config['chrootuser']) self.do_chroot(cmd, fatal = True) def _build_dir_setup(self): @@ -571,9 +571,9 @@ for subdir in ('RPMS', 'SRPMS', 'SOURCES', 'SPECS', 'BUILD', 'originals'): cmd = "mkdir -p %s/%s" % (self.builddir, subdir) self.do_chroot(cmd, fatal = True) - cmd = "chown %s.%s %s/%s" % (self.config['chrootuser'], - self.config['chrootgroup'], self.builddir, subdir) - self.do_chroot(cmd, fatal = True) + cmd = "chown -R %s.%s %s" % (self.config['chrootuser'], + self.config['chrootgroup'], self.builddir) + self.do_chroot(cmd, fatal = True) # rpmmacros default macrofile_out = '%s%s/.rpmmacros' % (self.rootdir, self.homedir) From Matt_Domsch at dell.com Tue Mar 7 04:13:53 2006 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 6 Mar 2006 22:13:53 -0600 Subject: OT: using mock to build for SLES In-Reply-To: <20060307041314.GA21949@lists.us.dell.com> References: <20060307041314.GA21949@lists.us.dell.com> Message-ID: <20060307041353.GB21949@lists.us.dell.com> On Mon, Mar 06, 2006 at 10:13:14PM -0600, Matt Domsch wrote: > I've been looking at using mock to build srpms for Novell/SuSE SLES > product, rather than using their build.rpm script. With one little > change to mock (below), seems to work pretty well. opensuse-10.x is > still a little weird, but SLES9 x86 and x86_64 seem to be building > fine. > > Here's the mock patch to pass -m to useradd, and I'll follow up with > the buildroots.xml and mock cfg file example too. buildroots.xml -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com build-minimal true Minimal Install aaa_base bash glibc glibc-devel glibc-locale sles-release buildsys-macros build-base true Basic Install build-minimal coreutils findutils build true Build Install build-base build-minimal make gcc tar gzip patch unzip bzip2 diffutils cpio gcc-c++ autoconf gettext automake flex libtool strace bison binutils acl attr bind-utils bzip2 cpp cracklib cvs cyrus-sasl db devs e2fsprogs file filesystem fillup gawk gdbm gdbm-devel gpm grep groff info insserv less libacl libattr libgcc libselinux libstdc++ libxcrypt m4 man mktemp module-init-tools ncurses ncurses-devel netcfg net-tools openldap2-client openssl pam pam-modules perl permissions popt procinfo procps psmisc pwdutils rcs readline sed sysvinit tcpd texinfo timezone util-linux vim zlib zlib-devel rpm suse-build-key From Matt_Domsch at dell.com Tue Mar 7 04:15:54 2006 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 6 Mar 2006 22:15:54 -0600 Subject: OT: using mock to build for SLES In-Reply-To: <20060307041314.GA21949@lists.us.dell.com> References: <20060307041314.GA21949@lists.us.dell.com> Message-ID: <20060307041554.GC21949@lists.us.dell.com> On Mon, Mar 06, 2006 at 10:13:14PM -0600, Matt Domsch wrote: > I've been looking at using mock to build srpms for Novell/SuSE SLES > product, rather than using their build.rpm script. With one little > change to mock (below), seems to work pretty well. opensuse-10.x is > still a little weird, but SLES9 x86 and x86_64 seem to be building > fine. > > Here's the mock patch to pass -m to useradd, and I'll follow up with > the buildroots.xml and mock cfg file example too. sles-9-x86_64.cfg. SLES doesn't automatically create a group for each user with useradd, all the users get put into group 'users' by default. And no runuser, use su instead. I manually created the i386 and x86_64 [base] repos. -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com #!/usr/bin/python -tt import os config_opts['root'] = 'sles-9-x86_64' config_opts['basedir'] = '/var/lib/mock/' config_opts['chroot'] = '/usr/sbin/mock-helper chroot' config_opts['mount'] = '/usr/sbin/mock-helper mount' config_opts['umount'] = '/usr/sbin/mock-helper umount' config_opts['rm'] = '/usr/sbin/mock-helper rm' config_opts['mknod'] = '/usr/sbin/mock-helper mknod' config_opts['yum'] = '/usr/sbin/mock-helper yum' config_opts['runuser'] = '/bin/su' config_opts['buildgroup'] = 'build' config_opts['chrootuser'] = 'mockbuild' config_opts['chrootgroup'] = 'users' config_opts['chrootuid'] = os.geteuid() config_opts['chrootgid'] = os.getegid() config_opts['chroothome'] = '/builddir' config_opts['clean'] = True config_opts['target_arch'] = 'x86_64' config_opts['yum.conf'] = """ [main] cachedir=/var/cache/yum debuglevel=1 logfile=/var/log/yum.log reposdir=/dev/null retries=20 obsoletes=1 gpgcheck=0 assumeyes=1 # repos [base] name=base baseurl=http://my-server/pub/yum/sles9/base/x86_64/ [groups] name=groups baseurl=http://my-server/pub/yum/mock/buildgroups/sles9/x86_64/ """ From Matt_Domsch at dell.com Tue Mar 7 04:19:53 2006 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 6 Mar 2006 22:19:53 -0600 Subject: OT: using mock to build for SLES In-Reply-To: <20060307041353.GB21949@lists.us.dell.com> References: <20060307041314.GA21949@lists.us.dell.com> <20060307041353.GB21949@lists.us.dell.com> Message-ID: <20060307041953.GD21949@lists.us.dell.com> On Mon, Mar 06, 2006 at 10:13:53PM -0600, Matt Domsch wrote: > On Mon, Mar 06, 2006 at 10:13:14PM -0600, Matt Domsch wrote: > > I've been looking at using mock to build srpms for Novell/SuSE SLES > > product, rather than using their build.rpm script. With one little > > change to mock (below), seems to work pretty well. opensuse-10.x is > > still a little weird, but SLES9 x86 and x86_64 seem to be building > > fine. > > > > Here's the mock patch to pass -m to useradd, and I'll follow up with > > the buildroots.xml and mock cfg file example too. > > buildroots.xml I'll note that the list of packages is quite large, but it's just the list that the SuSE build.rpm package itself uses. They do it a little different: if you specify a BuildRequires, then you have to specify *all* BuildRequires. If you don't specify, then you get the list below. I prefer the fedora (debian equivalent) of the base set of packages you don't have to specify in BuildRequires all the time, and that's what I'm doing here. Another strangeness: items from build-minimal and build-base aren't necessarily being pulled in. I had to move the rpm package out from build-minimal to build in order for it to get installed; the other items in build-minimal are getting pulled via the depsolver rather than being explicitly installed into the chroot. Thanks, Matt -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com From jkeating at j2solutions.net Mon Mar 13 00:01:23 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Sun, 12 Mar 2006 19:01:23 -0500 Subject: having a weird issue building emacs w/ plague Message-ID: <1142208083.14782.17.camel@ender> So we've got a plague+mock setup for Legacy, and I"m trying to build a new FC1 emacs. Here is the fun thing, when using plague-client and passing it a srpm to build, I get this error: creating src/Makefile + setarch i386 make i386: Success error: Bad exit status from /var/tmp/rpm-tmp.65918 (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.65918 (%build) Not very helpful. So I go to hand build it w/ mock so that I can see where it is actually failing and fix it, but mock itself builds just fine. mock -r fedora-1-i386-core emacs.src.rpm completes w/out blinking. We've been able to reproduce it many times on our build system, and even other plague+mock setups. I'm really at a loss as to how plague can get it wrong w/ mock gets it right. Help? When it finishes uploading, you can find the package in question at http://geek.j2solutions.net/emacs-21.3-9.legacy.src.rpm Note, all Legacy did beyond the stock FC package was add patch16, add autoconf213 as a BuildReq, and added the changelog. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From jkeating at j2solutions.net Mon Mar 13 00:24:51 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Sun, 12 Mar 2006 19:24:51 -0500 Subject: having a weird issue building emacs w/ plague In-Reply-To: <1142208083.14782.17.camel@ender> References: <1142208083.14782.17.camel@ender> Message-ID: <1142209491.14782.21.camel@ender> On Sun, 2006-03-12 at 19:01 -0500, Jesse Keating wrote: > creating src/Makefile > + setarch i386 make > i386: Success > error: Bad exit status from /var/tmp/rpm-tmp.65918 (%build) > > > RPM build errors: > Bad exit status from /var/tmp/rpm-tmp.65918 (%build) > > > Not very helpful. So I go to hand build it w/ mock so that I can see > where it is actually failing and fix it, but mock itself builds just > fine. mock -r fedora-1-i386-core emacs.src.rpm completes w/out > blinking. We've been able to reproduce it many times on our build > system, and even other plague+mock setups. I'm really at a loss as to > how plague can get it wrong w/ mock gets it right. Help? > > When it finishes uploading, you can find the package in question at > http://geek.j2solutions.net/emacs-21.3-9.legacy.src.rpm Ok, so we did more digging, and we find that plague calls setarch before the mock call. So when I duplicate this style of call: /usr/bin/setarch i686 /usr/bin/mock -r fedora-1-i386-core --arch i386 /home/jkeating/rpmbuild/SRPMS/emacs-21.3-9.legacy.src.rpm I do indeed get the vague failure. Upon running the rpm script inside the setarch chrooted buildroot, it does indeed error out trying to remove some files that it can't find. I have no idea why it can't find these, I'm trying to figure it out. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From jkeating at j2solutions.net Mon Mar 13 00:36:07 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Sun, 12 Mar 2006 19:36:07 -0500 Subject: having a weird issue building emacs w/ plague In-Reply-To: <1142209491.14782.21.camel@ender> References: <1142208083.14782.17.camel@ender> <1142209491.14782.21.camel@ender> Message-ID: <1142210167.14782.23.camel@ender> On Sun, 2006-03-12 at 19:24 -0500, Jesse Keating wrote: > > Upon running the rpm script inside the setarch chrooted buildroot, it > does indeed error out trying to remove some files that it can't find. I > have no idea why it can't find these, I'm trying to figure it out. Ok, more discovery. With FC1's setarch, if I do: setarch i386 make All I get is a success. It completely ignores the make call. However if I enter the chroot w/ setarch x86_64, then call setarch i386 make, it does the right thing. Wow, this is a really weird one. I mean, the problem IS setarch, but other than fixing it, is there a way you can see to work around this? -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From dcbw at redhat.com Mon Mar 13 03:26:41 2006 From: dcbw at redhat.com (Dan Williams) Date: Sun, 12 Mar 2006 22:26:41 -0500 Subject: OT: using mock to build for SLES In-Reply-To: <20060307041314.GA21949@lists.us.dell.com> References: <20060307041314.GA21949@lists.us.dell.com> Message-ID: <1142220401.13182.12.camel@localhost.localdomain> On Mon, 2006-03-06 at 22:13 -0600, Matt Domsch wrote: > I've been looking at using mock to build srpms for Novell/SuSE SLES > product, rather than using their build.rpm script. With one little > change to mock (below), seems to work pretty well. opensuse-10.x is > still a little weird, but SLES9 x86 and x86_64 seem to be building > fine. > > Here's the mock patch to pass -m to useradd, and I'll follow up with > the buildroots.xml and mock cfg file example too. Neat; does Fedora deal with the -m on useradd? If not, perhaps we could make the exact useradd arguments configurable in the mock config file for the target to allow this. Doesn't seem like a bad thing. Dan From Matt_Domsch at dell.com Tue Mar 14 21:46:51 2006 From: Matt_Domsch at dell.com (Matt Domsch) Date: Tue, 14 Mar 2006 15:46:51 -0600 Subject: OT: using mock to build for SLES In-Reply-To: <1142220401.13182.12.camel@localhost.localdomain> References: <20060307041314.GA21949@lists.us.dell.com> <1142220401.13182.12.camel@localhost.localdomain> Message-ID: <20060314214651.GB13670@lists.us.dell.com> On Sun, Mar 12, 2006 at 10:26:41PM -0500, Dan Williams wrote: > On Mon, 2006-03-06 at 22:13 -0600, Matt Domsch wrote: > > I've been looking at using mock to build srpms for Novell/SuSE SLES > > product, rather than using their build.rpm script. With one little > > change to mock (below), seems to work pretty well. opensuse-10.x is > > still a little weird, but SLES9 x86 and x86_64 seem to be building > > fine. > > > > Here's the mock patch to pass -m to useradd, and I'll follow up with > > the buildroots.xml and mock cfg file example too. > > Neat; does Fedora deal with the -m on useradd? If not, perhaps we could > make the exact useradd arguments configurable in the mock config file > for the target to allow this. Doesn't seem like a bad thing. Yes. The drawback is that useradd -m also populates the dotfiles from /etc/skel ino the users's homedir. Not that big a problem. -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com From notting at redhat.com Wed Mar 15 03:24:55 2006 From: notting at redhat.com (Bill Nottingham) Date: Tue, 14 Mar 2006 22:24:55 -0500 Subject: rfe: simple framework for post-build checks Message-ID: <20060315032455.GA15285@devserv.devel.redhat.com> The build system should have a framework where simple post-build checks can run. Simple design would be: - tests are collected in a package of scripts (or in CVS) - packagers can add extra ones for their packages - tests have access to: - the build logs - the name of the user who submitted the build - the owner of the package (if not the same) - the previous version of the package - the repodata for the repo being built for - tests have a defined return status, such as 0 - everythings OK. Carry on, sir! 1 - some information here 7 - someone should review this and approve before pushing the package 11 - the package is bad, please throw away These tests could then be: - e-mailed to the builder - e-mailed to the owner - posted on the web Doing this allows for simple, automated, QA; it can check that things that are caught in the package review for initial import are then caught later if they regress, and it can catch other things. Examples of tests: - test for files owned by the compile user - test for files with executable stack - look for specific compiler warnings in the build output, such as: ... will always overflow destination buffer ... ... warning: format argument XX unused before used argument XX ... *** buffer overflow detected *** ... warning: ignoring return value of 'realloc' ... - check for introduced multilib conflicts - check the scriplets for: - correct use of chkconfig - correct use of useradd or equivalents - excess complication (warning: package %post is 150 lines) - using the repo data: - check the package for broken deps - check the repository for induced breakage - using previous version of the same package: - check for added/removed files - check for added/removed/changed dependencies - check for introduction of new setuid files - check that symbols changed in libraries without changing soname - heck, just run the package through rpmlint and mail it out (or check it against a known list of exceptions for the package) I'm sure people could come up with a lot more. By making it a simple framework, it allows for easier contribution. Bill From toshio at tiki-lounge.com Tue Mar 21 18:31:26 2006 From: toshio at tiki-lounge.com (Toshio Kuratomi) Date: Tue, 21 Mar 2006 10:31:26 -0800 Subject: /dev/std* missing in mock Message-ID: <1142965887.3437.6.camel@localhost> Hi all, Seth is a bit tapped for time right now, could someone with mock checkin rights take a look at this bug? https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=179852 The current understanding in the bug report is that the buildsystem should have /dev/std{in,out,err} added. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From skvidal at linux.duke.edu Tue Mar 21 18:33:14 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Tue, 21 Mar 2006 13:33:14 -0500 Subject: /dev/std* missing in mock In-Reply-To: <1142965887.3437.6.camel@localhost> References: <1142965887.3437.6.camel@localhost> Message-ID: <1142965994.18716.24.camel@cutter> On Tue, 2006-03-21 at 10:31 -0800, Toshio Kuratomi wrote: > Hi all, > > Seth is a bit tapped for time right now, could someone with mock checkin > rights take a look at this bug? > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=179852 > > The current understanding in the bug report is that the buildsystem > should have /dev/std{in,out,err} added. > Thank You, Toshio. -sv From williams at redhat.com Tue Mar 21 19:14:55 2006 From: williams at redhat.com (Clark Williams) Date: Tue, 21 Mar 2006 13:14:55 -0600 Subject: /dev/std* missing in mock In-Reply-To: <1142965887.3437.6.camel@localhost> References: <1142965887.3437.6.camel@localhost> Message-ID: <442050AF.2080806@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Toshio Kuratomi wrote: >Hi all, > >Seth is a bit tapped for time right now, could someone with mock checkin >rights take a look at this bug? > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=179852 > >The current understanding in the bug report is that the buildsystem >should have /dev/std{in,out,err} added. > Toshio, The attached patch seems to work on my system. Can you try it to confirm and if it fixes your problem I'll check it in (unless someone objects strenuously). Clark Index: mock.py =================================================================== RCS file: /cvs/fedora/mock/mock.py,v retrieving revision 1.37 diff -u -r1.37 mock.py - --- mock.py 15 Mar 2006 22:14:38 -0000 1.37 +++ mock.py 21 Mar 2006 19:12:40 -0000 @@ -555,6 +555,14 @@ if not os.path.exists(devpath): os.symlink('../proc/self/fd', devpath) + fd = 0 + for item in ('stdin', 'stdout', 'stderr'): + devpath = os.path.join(self.rootdir, 'dev', item) + if not os.path.exists(devpath): + fdpath = os.path.join('../proc/self/fd', str(fd)) + os.symlink(fdpath, devpath) + fd += 1 + for item in [os.path.join(self.rootdir, 'etc', 'mtab'), os.path.join(self.rootdir, 'etc', 'fstab'), os.path.join(self.rootdir, 'var', 'log', 'yum.log')]: -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFEIFCvHyuj/+TTEp0RAk2mAKCId7zoU+2rflmbPgDSXpUbmezxpQCeIf/n H1b/VAVtfIK1HQTvLdnr7xE= =FckP -----END PGP SIGNATURE----- From dcbw at redhat.com Thu Mar 23 18:52:04 2006 From: dcbw at redhat.com (Dan Williams) Date: Thu, 23 Mar 2006 13:52:04 -0500 Subject: rfe: simple framework for post-build checks In-Reply-To: <20060315032455.GA15285@devserv.devel.redhat.com> References: <20060315032455.GA15285@devserv.devel.redhat.com> Message-ID: <1143139925.14689.0.camel@dhcp83-5.boston.redhat.com> On Tue, 2006-03-14 at 22:24 -0500, Bill Nottingham wrote: > The build system should have a framework where simple post-build > checks can run. Simple design would be: > > - tests are collected in a package of scripts (or in CVS) > - packagers can add extra ones for their packages > - tests have access to: > - the build logs > - the name of the user who submitted the build > - the owner of the package (if not the same) > - the previous version of the package > - the repodata for the repo being built for > - tests have a defined return status, such as > 0 - everythings OK. Carry on, sir! > 1 - some information here > 7 - someone should review this and approve before pushing > the package > 11 - the package is bad, please throw away > > These tests could then be: > - e-mailed to the builder > - e-mailed to the owner > - posted on the web > > Doing this allows for simple, automated, QA; it can check that > things that are caught in the package review for initial import > are then caught later if they regress, and it can catch other > things. > > Examples of tests: > - test for files owned by the compile user > - test for files with executable stack > - look for specific compiler warnings in the build output, such as: > ... will always overflow destination buffer ... > ... warning: format argument XX unused before used argument XX ... > *** buffer overflow detected *** > ... warning: ignoring return value of 'realloc' ... > - check for introduced multilib conflicts > - check the scriplets for: > - correct use of chkconfig > - correct use of useradd or equivalents > - excess complication (warning: package %post is 150 lines) > - using the repo data: > - check the package for broken deps > - check the repository for induced breakage > - using previous version of the same package: > - check for added/removed files > - check for added/removed/changed dependencies > - check for introduction of new setuid files > - check that symbols changed in libraries without changing soname > - heck, just run the package through rpmlint and mail it out > (or check it against a known list of exceptions for the package) > > I'm sure people could come up with a lot more. By making it > a simple framework, it allows for easier contribution. Sounds like a plan. We should do this. Dan From skvidal at linux.duke.edu Tue Mar 28 15:05:41 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Tue, 28 Mar 2006 10:05:41 -0500 Subject: mock buildroot definitions Message-ID: <1143558341.29933.13.camel@cutter> Hey, So it was brought up a while ago that due to the changing nature of the comps.xml format that we should not rely on it for the buildroot installations in mock. The idea a while back was to just have a 'buildroot' rpm that required all the stuff that would normally be in the comps.xml group. Then we could just install that rpm and it pulls in the rest of the buildroot components. What do folks think? Worth going through the process to do it? -sv From timp at redhat.com Tue Mar 28 15:27:40 2006 From: timp at redhat.com (Tim Powers) Date: Tue, 28 Mar 2006 10:27:40 -0500 Subject: mock buildroot definitions In-Reply-To: <1143558341.29933.13.camel@cutter> References: <1143558341.29933.13.camel@cutter> Message-ID: On Mar 28, 2006, at 10:05 AM, seth vidal wrote: > Hey, > So it was brought up a while ago that due to the changing nature > of the > comps.xml format that we should not rely on it for the buildroot > installations in mock. The idea a while back was to just have a > 'buildroot' rpm that required all the stuff that would normally be in > the comps.xml group. Then we could just install that rpm and it > pulls in > the rest of the buildroot components. > > What do folks think? Worth going through the process to do it? Out of curiosity, why can't mock just use the same format it has been using? Why is it dependent on what's being used elsewhere? Tim From cweyl at alumni.drew.edu Tue Mar 28 16:09:30 2006 From: cweyl at alumni.drew.edu (Chris Weyl) Date: Tue, 28 Mar 2006 08:09:30 -0800 Subject: mock buildroot definitions In-Reply-To: References: <1143558341.29933.13.camel@cutter> Message-ID: <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> On 3/28/06, Tim Powers wrote: > Out of curiosity, why can't mock just use the same format it has been > using? > > Why is it dependent on what's being used elsewhere? I think the idea is that it's not so much mock as yum, and every time yum changes how it interacts with comps.xml that would force a change as to how mock interacts with it (and a corresponding update of it). (right?) I like the idea, if it makes things more stable for a plague implementation. How would it be handled tho? Different rpms for the different groups? (e.g., buildsys-base, buildsys-minimal, etc, etc) Or one big rpm? -Chris From skvidal at linux.duke.edu Tue Mar 28 16:21:29 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Tue, 28 Mar 2006 11:21:29 -0500 Subject: mock buildroot definitions In-Reply-To: <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> Message-ID: <1143562889.29933.24.camel@cutter> On Tue, 2006-03-28 at 08:09 -0800, Chris Weyl wrote: > On 3/28/06, Tim Powers wrote: > > Out of curiosity, why can't mock just use the same format it has been > > using? > > > > Why is it dependent on what's being used elsewhere? > > I think the idea is that it's not so much mock as yum, and every time > yum changes how it interacts with comps.xml that would force a change > as to how mock interacts with it (and a corresponding update of it). > (right?) > > I like the idea, if it makes things more stable for a plague > implementation. How would it be handled tho? Different rpms for the > different groups? (e.g., buildsys-base, buildsys-minimal, etc, etc) > Or one big rpm? I like the 3 different pkgs idea. -sv From jkeating at j2solutions.net Tue Mar 28 16:36:46 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Tue, 28 Mar 2006 08:36:46 -0800 Subject: mock buildroot definitions In-Reply-To: <1143562889.29933.24.camel@cutter> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <1143562889.29933.24.camel@cutter> Message-ID: <1143563806.3752.58.camel@yoda.loki.me> On Tue, 2006-03-28 at 11:21 -0500, seth vidal wrote: > > I like the idea, if it makes things more stable for a plague > > implementation. How would it be handled tho? Different rpms for the > > different groups? (e.g., buildsys-base, buildsys-minimal, etc, etc) > > Or one big rpm? > > I like the 3 different pkgs idea. I like this too. It would allow for easy customization, just adding another package or so for different package groups. I don't think this is any heavier or harder than comps was, and since its just an rpm, yum's interaction with it shouldn't change overtime. +1 -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From mikem at redhat.com Tue Mar 28 16:51:40 2006 From: mikem at redhat.com (Mike McLean) Date: Tue, 28 Mar 2006 11:51:40 -0500 Subject: mock buildroot definitions In-Reply-To: <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> Message-ID: <4429699C.2020304@redhat.com> Chris Weyl wrote: > I like the idea, if it makes things more stable for a plague > implementation. How would it be handled tho? Different rpms for the > different groups? (e.g., buildsys-base, buildsys-minimal, etc, etc) > Or one big rpm? If we pursue this I recommend we use a more isolated naming scheme. Someone might conceivably want to build a real package named buildsys-base. Perhaps mock-abstract-group-{base,minimal,...} I would like to have the original behavior available via configuration. Surely yum will continue to have a groupinstall command. From jkeating at j2solutions.net Tue Mar 28 16:58:52 2006 From: jkeating at j2solutions.net (Jesse Keating) Date: Tue, 28 Mar 2006 08:58:52 -0800 Subject: mock buildroot definitions In-Reply-To: <4429699C.2020304@redhat.com> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <4429699C.2020304@redhat.com> Message-ID: <1143565132.3752.64.camel@yoda.loki.me> On Tue, 2006-03-28 at 11:51 -0500, Mike McLean wrote: > I would like to have the original behavior available via configuration. > Surely yum will continue to have a groupinstall command. > groupinstall will still exist, but how the groups are resolved may change. Today it's a comps file, which is a bit of a mess to maintain. In the future there may be a much better way of defining groups. Since this is something of a moving target, I like the idea of a package itself providing the start of the build root, less moving, and easier to hand out so that people can duplicate our build root exactly. -- Jesse Keating RHCE (geek.j2solutions.net) Fedora Legacy Team (www.fedoralegacy.org) GPG Public Key (geek.j2solutions.net/jkeating.j2solutions.pub) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part URL: From wtogami at redhat.com Tue Mar 28 21:15:16 2006 From: wtogami at redhat.com (Warren Togami) Date: Tue, 28 Mar 2006 16:15:16 -0500 Subject: mock buildroot definitions In-Reply-To: <1143565132.3752.64.camel@yoda.loki.me> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <4429699C.2020304@redhat.com> <1143565132.3752.64.camel@yoda.loki.me> Message-ID: <4429A764.7000809@redhat.com> Jesse Keating wrote: > On Tue, 2006-03-28 at 11:51 -0500, Mike McLean wrote: >> I would like to have the original behavior available via configuration. >> Surely yum will continue to have a groupinstall command. >> > > groupinstall will still exist, but how the groups are resolved may > change. Today it's a comps file, which is a bit of a mess to maintain. > In the future there may be a much better way of defining groups. Since > this is something of a moving target, I like the idea of a package > itself providing the start of the build root, less moving, and easier to > hand out so that people can duplicate our build root exactly. > Is the concern about yum groupinstall that it is now "conditional" depending on other things that are installed, thus it is not guaranteed what is actually in the buildroot. For this reason it is safer to explicitly list it somewhere? Warren Togami wtogami at redhat.com From skvidal at linux.duke.edu Tue Mar 28 21:21:34 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Tue, 28 Mar 2006 16:21:34 -0500 Subject: mock buildroot definitions In-Reply-To: <4429A764.7000809@redhat.com> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <4429699C.2020304@redhat.com> <1143565132.3752.64.camel@yoda.loki.me> <4429A764.7000809@redhat.com> Message-ID: <1143580894.29933.55.camel@cutter> > Is the concern about yum groupinstall that it is now "conditional" > depending on other things that are installed, thus it is not guaranteed > what is actually in the buildroot. For this reason it is safer to > explicitly list it somewhere? > no the concern is that format for the comps.xml has changed and may change in the future and it matters what version of yum you use with what comps.xml format. -sv From skvidal at linux.duke.edu Wed Mar 29 00:19:06 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Tue, 28 Mar 2006 19:19:06 -0500 Subject: mock buildroot definitions In-Reply-To: <4429699C.2020304@redhat.com> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <4429699C.2020304@redhat.com> Message-ID: <1143591546.422.2.camel@cutter> On Tue, 2006-03-28 at 11:51 -0500, Mike McLean wrote: > Chris Weyl wrote: > > I like the idea, if it makes things more stable for a plague > > implementation. How would it be handled tho? Different rpms for the > > different groups? (e.g., buildsys-base, buildsys-minimal, etc, etc) > > Or one big rpm? > > If we pursue this I recommend we use a more isolated naming scheme. > Someone might conceivably want to build a real package named > buildsys-base. Perhaps mock-abstract-group-{base,minimal,...} just easier to have them as variables of: base, minimal, build > I would like to have the original behavior available via configuration. > Surely yum will continue to have a groupinstall command. why would you want to continue to support the buildgroups.xml file format in mock? What does it benefit to maintain both sets of the code? -sv From sheltren at cs.ucsb.edu Wed Mar 29 01:41:11 2006 From: sheltren at cs.ucsb.edu (Jeff Sheltren) Date: Tue, 28 Mar 2006 21:41:11 -0400 Subject: mock buildroot definitions In-Reply-To: <1143558341.29933.13.camel@cutter> References: <1143558341.29933.13.camel@cutter> Message-ID: On Mar 28, 2006, at 11:05 AM, seth vidal wrote: > Hey, > So it was brought up a while ago that due to the changing nature > of the > comps.xml format that we should not rely on it for the buildroot > installations in mock. The idea a while back was to just have a > 'buildroot' rpm that required all the stuff that would normally be in > the comps.xml group. Then we could just install that rpm and it > pulls in > the rest of the buildroot components. > > What do folks think? Worth going through the process to do it? > > -sv > I still don't like the idea of using an RPM to accomplish this. Having the buildroot packages in an editable text file makes it extremely easy to see what is being installed, and also it simple to add or remove packages if you have the desire/need to do so. If this information were stored in an RPM, you'd have to install the SRPM to get at the spec file which would contain this information (I guess... depending on how the package were to be implemented). To add/remove a package to/from the buildroot group would involve rebuilding the RPM and then uploading it to the server. So, ideally (for me, anyway), the group package lists should be stored in some sort of editable file, be it xml, plain text, or whatever. Even if the comps format is changing, wouldn't it be possible to keep the current group code in place within yum? It could even be called --mockinstall if you need a way to separate it from the "new" groups format. Another idea would be to include these package lists inside of the mock package. For example, a config setting in /etc/mock/ whatever.cfg would allow you to list a file which contains a list of the build group. Of course, if you have a large number of machines running mock (as in a plague setup) you'd probably want this config stored in a central location so there is no need to modify each builder individually. -Jeff From skvidal at linux.duke.edu Wed Mar 29 06:09:34 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Wed, 29 Mar 2006 01:09:34 -0500 Subject: mock buildroot definitions In-Reply-To: References: <1143558341.29933.13.camel@cutter> Message-ID: <1143612574.422.66.camel@cutter> > I still don't like the idea of using an RPM to accomplish this. > > Having the buildroot packages in an editable text file makes it > extremely easy to see what is being installed, and also it simple to > add or remove packages if you have the desire/need to do so. If this > information were stored in an RPM, you'd have to install the SRPM to > get at the spec file which would contain this information (I guess... > depending on how the package were to be implemented). To add/remove > a package to/from the buildroot group would involve rebuilding the > RPM and then uploading it to the server. not really - just have the specfile in the tree, edit the spec file then: rpmbuild --a-bunch-of-path-definitions -bb foo.spec then createrepo the dir. I do it now for something other than mock and it's not really much more work than a cvs commit. > So, ideally (for me, anyway), the group package lists should be > stored in some sort of editable file, be it xml, plain text, or > whatever. Even if the comps format is changing, wouldn't it be > possible to keep the current group code in place within yum? It > could even be called --mockinstall if you need a way to separate it > from the "new" groups format. not nicely. If we were to do that we'd have 3 different versions of the group code in place right now and I'm sure we'll have more as we go forward. It'd just make for cumbersome, unmaintained code in yum and I'd rather not have that if we can avoid it. > Another idea would be to include these package lists inside of the > mock package. For example, a config setting in /etc/mock/ > whatever.cfg would allow you to list a file which contains a list of > the build group. Of course, if you have a large number of machines > running mock (as in a plague setup) you'd probably want this config > stored in a central location so there is no need to modify each > builder individually. and the reason we like having it centrally stored is to help all the extras packagers who are testing their builds using mock on their home machines. This way if we change what is in the base of the buildroot we don't have to propagate the files and they have to get to the repo in order to use buildsys-macro to get the %{dist} tag. I'm open to other suggestions, though. In fact - to accommodate some of these ideas I've added a MockTodo page in the fedoraproject.org wiki. Please put down some things you want to see in there. http://fedoraproject.org/wiki/Projects/MockTodo thanks, -sv From enrico.scholz at informatik.tu-chemnitz.de Wed Mar 29 06:22:31 2006 From: enrico.scholz at informatik.tu-chemnitz.de (Enrico Scholz) Date: Wed, 29 Mar 2006 08:22:31 +0200 Subject: mock buildroot definitions In-Reply-To: (Jeff Sheltren's message of "Tue, 28 Mar 2006 21:41:11 -0400") References: <1143558341.29933.13.camel@cutter> Message-ID: <87bqvplqhk.fsf@kosh.bigo.ensc.de> sheltren at cs.ucsb.edu (Jeff Sheltren) writes: > Having the buildroot packages in an editable text file makes it > extremely easy to see what is being installed, and also it simple to > add or remove packages if you have the desire/need to do so. I use http://ensc.de/mock/mock-0.4-buildpkg.patch which allows things like | config_opts['buildpkg'] = [ prefix + '-filesystem', | prefix + '-binutils', | prefix + '-gcc41', | prefix + '-libgcc41', | prefix + '-setup', | ] in the mock-configfile. 'prefix' differs between several cross-build targets (e.g. 'arm-xscale-linux-gnu' or 'arm-xscale-linux-uclibc' or 'mips-linux-gnu' or ...). Enrico -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 482 bytes Desc: not available URL: From laroche at redhat.com Wed Mar 29 07:58:53 2006 From: laroche at redhat.com (Florian La Roche) Date: Wed, 29 Mar 2006 09:58:53 +0200 Subject: mock buildroot definitions In-Reply-To: <1143580894.29933.55.camel@cutter> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <4429699C.2020304@redhat.com> <1143565132.3752.64.camel@yoda.loki.me> <4429A764.7000809@redhat.com> <1143580894.29933.55.camel@cutter> Message-ID: <20060329075853.GA12782@dudweiler.stuttgart.redhat.com> > no the concern is that format for the comps.xml has changed and may > change in the future and it matters what version of yum you use with > what comps.xml format. You might want to just extend the code to also read newer versions, but keep also reading the old ones. Not a real good solution to comps format changes, but at least we would get new things in without moving to yet another file/format. regards, Florian La Roche From skvidal at linux.duke.edu Wed Mar 29 12:47:20 2006 From: skvidal at linux.duke.edu (seth vidal) Date: Wed, 29 Mar 2006 07:47:20 -0500 Subject: mock buildroot definitions In-Reply-To: <20060329075853.GA12782@dudweiler.stuttgart.redhat.com> References: <1143558341.29933.13.camel@cutter> <7dd7ab490603280809g621e7668hf346b606af1212f9@mail.gmail.com> <4429699C.2020304@redhat.com> <1143565132.3752.64.camel@yoda.loki.me> <4429A764.7000809@redhat.com> <1143580894.29933.55.camel@cutter> <20060329075853.GA12782@dudweiler.stuttgart.redhat.com> Message-ID: <1143636440.422.99.camel@cutter> On Wed, 2006-03-29 at 09:58 +0200, Florian La Roche wrote: > > no the concern is that format for the comps.xml has changed and may > > change in the future and it matters what version of yum you use with > > what comps.xml format. > > You might want to just extend the code to also read newer versions, > but keep also reading the old ones. Not a real good solution to > comps format changes, but at least we would get new things in without > moving to yet another file/format. > except that the comps format in 2.6.X removed some of the complexity from the code that was there before. For FC5 we removed things, really, in order to make the groups selection work for the ui. -sv From bugs.michael at gmx.net Tue Mar 21 18:12:48 2006 From: bugs.michael at gmx.net (Michael Schwendt) Date: Tue, 21 Mar 2006 18:12:48 -0000 Subject: Fw: [Bug 179852] /dev/std* missing in mock - FE buildsystem Message-ID: <20060321191205.d0976866.bugs.michael@gmx.net> Begin forwarded message: Date: Tue, 21 Mar 2006 13:04:20 -0500 From: bugzilla at redhat.com To: bugs.michael at gmx.net Subject: [Bug 179852] /dev/std* missing in mock - FE buildsystem Please do not reply directly to this email. All additional comments should be made in the comments box of this bug report. Summary: /dev/std* missing in mock - FE buildsystem https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=179852 ------- Additional Comments From bugs.michael at gmx.net 2006-03-21 13:04 EST ------- I'm not subscribed to that list, and Cc from within bugzilla doesn't work, please don't create additional hurdles inside the Fedora Project. I call upon the FE buildsys maintainers to please process this bugzilla ticket. wine-docs package for FC-4 continues to fail in the FE buildsys with an strace pointing to missing /dev/std{in,out,err} device links in the mock chroot: [pid 27154] open("/dev/stdin", O_RDONLY|O_NOCTTY|O_LARGEFILE) = -1 ENOENT (No su ch file or directory) [pid 27154] write(2, "ELinks: ", 8ELinks: ) = 8 [pid 27154] write(2, "No such file or directory", 25No such file or directory) = 25 It has not been examined why elinks searches for these devices or why it doesn't appear to do the same for FC-3 and FC-5. Anyway, wine-docs for FC-4 can't be built due to this issue with the build system. -- Configure bugmail: https://bugzilla.redhat.com/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You reported the bug, or are watching the reporter. You are on the CC list for the bug, or are watching someone who is.