From skvidal at phy.duke.edu Fri Dec 2 06:29:11 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Fri, 02 Dec 2005 01:29:11 -0500 Subject: mock update Message-ID: <1133504951.17520.37.camel@cutter> hey, we need a mock release. Who has patches in process for mock and would like to get things in for it? We need to fix two items: 1. make log files dump out on the fly rather than only at the end. 2. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=163576 I know the second one is fixed in cvs - anyone know about the first one? -sv From oliver at linux-kernel.at Fri Dec 2 07:45:41 2005 From: oliver at linux-kernel.at (Oliver Falk) Date: Fri, 02 Dec 2005 08:45:41 +0100 Subject: mock update In-Reply-To: <1133504951.17520.37.camel@cutter> References: <1133504951.17520.37.camel@cutter> Message-ID: <438FFBA5.3090604@linux-kernel.at> On 12/02/2005 07:29 AM, seth vidal wrote: > hey, > we need a mock release. Who has patches in process for mock and would > like to get things in for it? > > We need to fix two items: > 1. make log files dump out on the fly rather than only at the end. > 2. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=163576 > > > I know the second one is fixed in cvs - anyone know about the first one? What about my suggestion to add a kill command to kill mock builds!? Best, Oliver From jeff.pitman at gmail.com Fri Dec 2 07:48:14 2005 From: jeff.pitman at gmail.com (Jeff Pitman) Date: Fri, 2 Dec 2005 07:48:14 +0000 Subject: mock update In-Reply-To: <1133504951.17520.37.camel@cutter> References: <1133504951.17520.37.camel@cutter> Message-ID: <6b9c17630512012348p6d04b966y2b8202b1b8872f45@mail.gmail.com> On 12/2/05, seth vidal wrote: > > We need to fix two items: > 1. make log files dump out on the fly rather than only at the end. > I committed this to CVS. dcbw was gonna test it, but I haven't heard anything about it. I've tested it pretty good on my side, but, it'd be good to get a second set of eyeballs on it prior to release. -- -jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcbw at redhat.com Fri Dec 2 12:42:07 2005 From: dcbw at redhat.com (Dan Williams) Date: Fri, 02 Dec 2005 07:42:07 -0500 Subject: mock update In-Reply-To: <6b9c17630512012348p6d04b966y2b8202b1b8872f45@mail.gmail.com> References: <1133504951.17520.37.camel@cutter> <6b9c17630512012348p6d04b966y2b8202b1b8872f45@mail.gmail.com> Message-ID: <1133527327.2650.0.camel@localhost.localdomain> On Fri, 2005-12-02 at 07:48 +0000, Jeff Pitman wrote: > On 12/2/05, seth vidal wrote: > We need to fix two items: > 1. make log files dump out on the fly rather than only at the > end. > > I committed this to CVS. dcbw was gonna test it, but I haven't heard > anything about it. I've tested it pretty good on my side, but, it'd be > good to get a second set of eyeballs on it prior to release. Hmm, yeah, I did say that didn't I. I'll stick the new code on one of the builders today and see what blows up. Dan From skvidal at phy.duke.edu Sat Dec 3 08:18:03 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Sat, 03 Dec 2005 03:18:03 -0500 Subject: mock update In-Reply-To: <438FFBA5.3090604@linux-kernel.at> References: <1133504951.17520.37.camel@cutter> <438FFBA5.3090604@linux-kernel.at> Message-ID: <1133597883.26893.27.camel@cutter> On Fri, 2005-12-02 at 08:45 +0100, Oliver Falk wrote: > On 12/02/2005 07:29 AM, seth vidal wrote: > > hey, > > we need a mock release. Who has patches in process for mock and would > > like to get things in for it? > > > > We need to fix two items: > > 1. make log files dump out on the fly rather than only at the end. > > 2. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=163576 > > > > > > I know the second one is fixed in cvs - anyone know about the first one? > > What about my suggestion to add a kill command to kill mock builds!? Maybe I missed it - did you send a patch in? -sv From chris.weyl at gmail.com Sun Dec 4 21:28:18 2005 From: chris.weyl at gmail.com (Chris Weyl) Date: Sun, 4 Dec 2005 16:28:18 -0500 Subject: Best way to sign packages before adding to the repos? In-Reply-To: <1132331173.5377.7.camel@dhcp83-87.boston.redhat.com> References: <7dd7ab490511161443h65fa7875v265a157ce6f18fc7@mail.gmail.com> <1132197166.2636.16.camel@localhost.localdomain> <7dd7ab490511180708n45fa32bese78f7c823d8b923e@mail.gmail.com> <1132331173.5377.7.camel@dhcp83-87.boston.redhat.com> Message-ID: <7dd7ab490512041328h59ac873bh7e105db00fc6e6c4@mail.gmail.com> Ok, so after a brief pause to gather my thoughts.... ;) On 11/18/05, Dan Williams wrote: > On Fri, 2005-11-18 at 10:08 -0500, Chris Weyl wrote: > > On 11/16/05, Dan Williams wrote: > Hmm, it actually runs the scripts _after_ packages have been copied to > the repository. So maybe it's not exactly what you want. Ahhh... gotcha. I thought they were being run exactly once after each time a builder finished a given package... My suggestions may be slightly out of whack then :) [...snip...] > The problem here is that if packages are held until manually finished, > then they are not available to builders. So if I want to build package > B that depends on a new package A, I have to wait for you to sign > package A and add it to the repo before I can build package B. That's > the main reason to add packages to the repo ASAP, and also to run the > repo scripts _after_ copying packages to the repodir and running > createrepo. It's also probably why Fedora Extras doesn't use the build > server's repository but requires manual runs of a push-script to sign & > copy to the "official" mirrored repo. > > It's a balance of package maintainer's needs (quick building of deps) > versus administrator needs (making sure repos are up to date and > signed). Suggestions on how to better balance these two issues would be > welcomed. > > In the longer term (plague 0.5) I'd like to put automatic depsolving > into the build server, so that the server will wait like 8 hours for > dependencies of a particular SRPM to be solved before failing the > package. If during those 8 or 12 hours, the deps get solved, then the > build server will allow the package to be built. This way, we let > package maintainers queue up however many packages they want, and then > walk away and not have to worry about stuff until they get mails about > failures or successes. That's the end goal here. And it should solve > your problem too, since it fixes the dependency issue that forces repo > scripts to be run last. Ok, so here's some revised thoughts :) (And I like depsolving in the buildserver... would make life easier.) I think my initial attempt to have one repo to serve both the buildservers and the Masses is probably a bit optimistic. Creating a repo, with the cache option in createrepo, is a fairly low-impact activity, so having 2 would help both manage what is released publicly without compromising the buildservers' access to newly built rpms as soon as possible. So assuming I use 2 repos, one for the buildsys' own purposes and one "public" one (with packages signed, blessed, etc, etc), is there a way to leverage the needsign/finished push to signal/initiate/queue/whatever that the package being finished needs to be pushed into the public repo? (Along with whatever attendant steps need to happen concomitant with that push.) Two ideas came to me on this: 1. Create a "pushtopublic" stage, where the buildsys essentially redoes "addtorepo" but with the public repos. Hooks could be put in place (a la #2) to permit signing, etc. 2. Allow a script (along the lines of repo_script) to be specified and executed against a particular package whenever it was "finish"ed. This script could take care of the copying, signing, and redoing the public repo's metadata. Or queuing the package for those steps to be done by an external process. Or... I can see advantages and disadvantages to both. If a new "pushtorepo" stage is created, then we have the advantage of having everything handled under the buildsys by default; and simply adding a trigger script would allow for people to handle this differently, according to their tastes/desires. Anyways, just off the top of my head :) -Chris From skvidal at phy.duke.edu Tue Dec 13 05:42:20 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 13 Dec 2005 00:42:20 -0500 Subject: mock and fc5/fe5 and comps pain Message-ID: <1134452540.6158.37.camel@cutter> Hi, As some of you are aware the comps changes that anaconda wanted changed how comps works in yum which nicely BROKE Fedora Extras rawhide builds once the comps.xml changes landed in rawhide. So what do we do to fix it? Jeremy and I tossed around a couple of ideas - none of them terribly attractive but we settled on this, I'd like to hear some input: - make three packages and put them in the buildgroups repo: http://fedoraproject.org/buildgroups/ The packages would have no real payload but would have only a requires list that matches the 3 groups listed in: http://fedoraproject.org/buildgroups/4/i386/buildroots.xml - modify mock to act on those packages instead of on group names. Possibly making that act be something you configure so we don't break everyone with the mock update. - no longer have to worry about groups-file-format-changes screwing up mock and extras builds So what do y'all think? That sound like a plan? Would someone like to make the packages for us while I go futz with mock? thanks, -sv From wtogami at redhat.com Tue Dec 13 15:41:36 2005 From: wtogami at redhat.com (Warren Togami) Date: Tue, 13 Dec 2005 10:41:36 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134452540.6158.37.camel@cutter> References: <1134452540.6158.37.camel@cutter> Message-ID: <439EEBB0.9050507@redhat.com> seth vidal wrote: > Hi, > As some of you are aware the comps changes that anaconda wanted changed > how comps works in yum which nicely BROKE Fedora Extras rawhide builds > once the comps.xml changes landed in rawhide. So what do we do to fix > it? > > Jeremy and I tossed around a couple of ideas - none of them terribly > attractive but we settled on this, I'd like to hear some input: > > - make three packages and put them in the buildgroups repo: > http://fedoraproject.org/buildgroups/ > The packages would have no real payload but would have only > a requires list that matches the 3 groups listed in: > http://fedoraproject.org/buildgroups/4/i386/buildroots.xml > > - modify mock to act on those packages instead of on group names. > Possibly > making that act be something you configure so we don't break > everyone with > the mock update. > > - no longer have to worry about groups-file-format-changes screwing up > mock and > extras builds > > So what do y'all think? That sound like a plan? > Would someone like to make the packages for us while I go futz with > mock? Old fedora.us and our current Extras packaging guidelines tell folks to test their builds against the minimum build environment pulled in by the artifical deps of the fedora-rpmdevtools package. You could just use fedora-rpmdevtools as the base package instead of creating yet another package to pull in minimum buildroot deps. Note that we made an explicit decision NOT to include auto* as base deps in fedora-rpmdevtools. This is because different packages may require specific versions of autoconf or automake. Warren Togami wtogami at redhat.com From skvidal at phy.duke.edu Tue Dec 13 16:19:50 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 13 Dec 2005 11:19:50 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <439EEBB0.9050507@redhat.com> References: <1134452540.6158.37.camel@cutter> <439EEBB0.9050507@redhat.com> Message-ID: <1134490791.12376.4.camel@cutter> > Old fedora.us and our current Extras packaging guidelines tell folks to > test their builds against the minimum build environment pulled in by the > artifical deps of the fedora-rpmdevtools package. You could just use > fedora-rpmdevtools as the base package instead of creating yet another > package to pull in minimum buildroot deps. We'd need a fair bit more stuff than is in fedora-rpmdevtools: - createrepo, buildsys-macros, python, rpm-python, take a look at the contents of this file: http://fedoraproject.org/buildgroups/4/i386/buildroots.xml that's what we need and that's a bit of a problem for fedora-rpmdevtools. > > Note that we made an explicit decision NOT to include auto* as base deps > in fedora-rpmdevtools. This is because different packages may require > specific versions of autoconf or automake. right and in the buildroots.xml we include all the automakes and autoconf. I think we'd be better off with NOT confusing the purposes of fedora-rpmdevtools package and the mock build packages, not to mention that I'd like to make sure this is written so it can be used for !fedora distributions, too. I know I'm not using mock just for building fedora-based things; I'm sure others aren't either. -sv From wtogami at redhat.com Tue Dec 13 16:24:16 2005 From: wtogami at redhat.com (Warren Togami) Date: Tue, 13 Dec 2005 11:24:16 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134490791.12376.4.camel@cutter> References: <1134452540.6158.37.camel@cutter> <439EEBB0.9050507@redhat.com> <1134490791.12376.4.camel@cutter> Message-ID: <439EF5B0.5070909@redhat.com> seth vidal wrote: >> Old fedora.us and our current Extras packaging guidelines tell folks to >> test their builds against the minimum build environment pulled in by the >> artifical deps of the fedora-rpmdevtools package. You could just use >> fedora-rpmdevtools as the base package instead of creating yet another >> package to pull in minimum buildroot deps. > > We'd need a fair bit more stuff than is in fedora-rpmdevtools: > - createrepo, buildsys-macros, python, rpm-python, > > take a look at the contents of this file: > http://fedoraproject.org/buildgroups/4/i386/buildroots.xml > > that's what we need and that's a bit of a problem for > fedora-rpmdevtools. > OK no problem. So do you think this package should exist in the Extras tree, or a special case outside? Warren Togami wtogami at redhat.com From skvidal at phy.duke.edu Tue Dec 13 16:32:51 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 13 Dec 2005 11:32:51 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <439EF5B0.5070909@redhat.com> References: <1134452540.6158.37.camel@cutter> <439EEBB0.9050507@redhat.com> <1134490791.12376.4.camel@cutter> <439EF5B0.5070909@redhat.com> Message-ID: <1134491571.12376.6.camel@cutter> > OK no problem. So do you think this package should exist in the Extras > tree, or a special case outside? > I'm thinking it should be like buildsys-macros, only really existing in the buildgroups repo. -sv From sheltren at cs.ucsb.edu Tue Dec 13 21:14:43 2005 From: sheltren at cs.ucsb.edu (Jeff Sheltren) Date: Tue, 13 Dec 2005 17:14:43 -0400 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134491571.12376.6.camel@cutter> References: <1134452540.6158.37.camel@cutter> <439EEBB0.9050507@redhat.com> <1134490791.12376.4.camel@cutter> <439EF5B0.5070909@redhat.com> <1134491571.12376.6.camel@cutter> Message-ID: <1779AFF8-2CF9-4655-94E2-B6D9DAF13526@cs.ucsb.edu> On Dec 13, 2005, at 12:32 PM, seth vidal wrote: > >> OK no problem. So do you think this package should exist in the >> Extras >> tree, or a special case outside? >> > > I'm thinking it should be like buildsys-macros, only really > existing in > the buildgroups repo. > > -sv > That's what I'm picturing, too. Although I don't really like the idea of creating extra packages just to fake some dependencies... By the way, is there a place where the changes to comps.xml are documented? I'd like to know more about what's changing before I add (subtract?) any more from this conversation. -Jeff From skvidal at phy.duke.edu Tue Dec 13 21:21:51 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Tue, 13 Dec 2005 16:21:51 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1779AFF8-2CF9-4655-94E2-B6D9DAF13526@cs.ucsb.edu> References: <1134452540.6158.37.camel@cutter> <439EEBB0.9050507@redhat.com> <1134490791.12376.4.camel@cutter> <439EF5B0.5070909@redhat.com> <1134491571.12376.6.camel@cutter> <1779AFF8-2CF9-4655-94E2-B6D9DAF13526@cs.ucsb.edu> Message-ID: <1134508911.13311.24.camel@cutter> On Tue, 2005-12-13 at 17:14 -0400, Jeff Sheltren wrote: > On Dec 13, 2005, at 12:32 PM, seth vidal wrote: > > > > >> OK no problem. So do you think this package should exist in the > >> Extras > >> tree, or a special case outside? > >> > > > > I'm thinking it should be like buildsys-macros, only really > > existing in > > the buildgroups repo. > > > > -sv > > > > That's what I'm picturing, too. Although I don't really like the > idea of creating extra packages just to fake some dependencies... By > the way, is there a place where the changes to comps.xml are > documented? I'd like to know more about what's changing before I add > (subtract?) any more from this conversation. > right now it's all in comps.py in yum cvs-HEAD. however the short version, so far, is this. There are 2 main elements: - category - group groups can contain packages in modes 'mandatory, default, optional'. groups CANNOT contain other groups. categories can contain other groups. Things still up in the air: 1. whether or not categories can or should contain single package entries. 2. How/what yum will do with the categories at all. Right now the only thing using the categories is anaconda. Both categories and groups are exposed via yum.comps right now. The comps/groupInfo object in yum is signficantly simpler than it was before. hope this helps. -sv From mikem at redhat.com Wed Dec 14 20:10:31 2005 From: mikem at redhat.com (Mike McLean) Date: Wed, 14 Dec 2005 15:10:31 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134452540.6158.37.camel@cutter> References: <1134452540.6158.37.camel@cutter> Message-ID: <43A07C37.9000807@redhat.com> seth vidal wrote: > Hi, > As some of you are aware the comps changes that anaconda wanted changed > how comps works in yum which nicely BROKE Fedora Extras rawhide builds > once the comps.xml changes landed in rawhide. So what do we do to fix > it? I realize this might be an odd time for me to jump into the discussion, but a few concerns and questions leap to mind. Perhaps some of these stem from my imperfect knowledge of the situation; if so, I apologize. First, isn't this an odd sort of incompatibility to introduce in yum? Even if this solution is applied in mock, that is no insurance against further incompatible yum changes in other areas. It seems to me that this problem could be resolved at the repo level. Couldn't plague maintain its own copy of the rawhide repo sans comps? I guess my points are: 1) not all mock users will be impacted by this issue (I won't, since I generate my own repos). 2) If this change is applied in mock, it will impact mock users unless the new behavior is optional. Let me throw out another possibility. Add a mock config option named buildcomps containing a url. If this option is present, have mock parse that comps and issue the appropriate yum install commands instead of a groupinstall. From katzj at redhat.com Wed Dec 14 21:35:41 2005 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 14 Dec 2005 16:35:41 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A07C37.9000807@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> Message-ID: <1134596141.2708.65.camel@bree.local.net> On Wed, 2005-12-14 at 15:10 -0500, Mike McLean wrote: > seth vidal wrote: > > As some of you are aware the comps changes that anaconda wanted changed > > how comps works in yum which nicely BROKE Fedora Extras rawhide builds > > once the comps.xml changes landed in rawhide. So what do we do to fix > > it? > > I realize this might be an odd time for me to jump into the discussion, > but a few concerns and questions leap to mind. Perhaps some of these > stem from my imperfect knowledge of the situation; if so, I apologize. > > First, isn't this an odd sort of incompatibility to introduce in yum? > Even if this solution is applied in mock, that is no insurance against > further incompatible yum changes in other areas. The incompatibility is more in the repo metadata than in yum -- yum 2.4.x (which is what's on the build hosts) doesn't understand the new comps format. And that's partly because the comps side of things has had a lot less thought and effort put into making sure that it covers the needs of the userbase. And I'm going to go out on a limb and say that I expect this _not_ to be the last time it changes... we're actually starting to use the information and so I'm all but certain that we'll find things that we hadn't considered. > It seems to me that this problem could be resolved at the repo level. > Couldn't plague maintain its own copy of the rawhide repo sans comps? It's more than just the devel repo -- it'll then be the fc5 repo and the devel repo. Plus the extras counterparts. Plus everyone on older releases trying to build against fc5/devel using mock. > I guess my points are: > 1) not all mock users will be impacted by this issue (I won't, since I > generate my own repos). All mock users will be impacted because you're not guaranteed that the version of yum you have can handle the group information in the repository. > 2) If this change is applied in mock, it will impact mock users unless > the new behavior is optional. > > Let me throw out another possibility. Add a mock config option named > buildcomps containing a url. If this option is present, have mock parse > that comps and issue the appropriate yum install commands instead of a > groupinstall. Then mock has to grow how to parse a comps file. And which format should be preferred? If we were to go that route, we're better off just putting a list of packages in the mock config file, but that's just painful from a maintenance perspective, IMHO Jeremy From laroche at redhat.com Wed Dec 14 22:11:53 2005 From: laroche at redhat.com (Florian La Roche) Date: Wed, 14 Dec 2005 23:11:53 +0100 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134596141.2708.65.camel@bree.local.net> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> Message-ID: <20051214221153.GB4946@dudweiler.stuttgart.redhat.com> > The incompatibility is more in the repo metadata than in yum -- yum > 2.4.x (which is what's on the build hosts) doesn't understand the new > comps format. And that's partly because the comps side of things has > had a lot less thought and effort put into making sure that it covers > the needs of the userbase. And I'm going to go out on a limb and say > that I expect this _not_ to be the last time it changes... we're > actually starting to use the information and so I'm all but certain that > we'll find things that we hadn't considered. While the new format is in certain ways better for the installer, you only need to change very few lines to make other apps use both the old format and the new format. I think we should do more testing and put in more thoughts before pushing out changes for the comps.xml file. Especially if now further changes might get done over the next weeks. Shouldn't we then start a completely new file and only then add it to the other tools once its implementation is more complete? I haven't chekd, but is the comps.xml file part of the repodata with defined content or is it just more "riding along" in the same directory? > Then mock has to grow how to parse a comps file. And which format > should be preferred? If we were to go that route, we're better off just > putting a list of packages in the mock config file, but that's just > painful from a maintenance perspective, IMHO Or just allow both ways. Doesn't make a too big difference until things get more standardized at some point. regards, Florian La Roche From mikem at redhat.com Wed Dec 14 22:40:42 2005 From: mikem at redhat.com (Mike McLean) Date: Wed, 14 Dec 2005 17:40:42 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134596141.2708.65.camel@bree.local.net> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> Message-ID: <43A09F6A.2040101@redhat.com> Jeremy Katz wrote: > The incompatibility is more in the repo metadata than in yum -- yum > 2.4.x (which is what's on the build hosts) doesn't understand the new > comps format. And that's partly because the comps side of things has > had a lot less thought and effort put into making sure that it covers > the needs of the userbase. And I'm going to go out on a limb and say > that I expect this _not_ to be the last time it changes... we're > actually starting to use the information and so I'm all but certain that > we'll find things that we hadn't considered. I was thinking more along the lines of the "new yum can't parse old comps" issue. If it could, it seems like we could just migrate to the new yum in the buildsystem. For that matter, if the old yum could be told to ignore comps on a per-repo basis, we could work around it that way. >>Let me throw out another possibility. Add a mock config option named >>buildcomps containing a url. If this option is present, have mock parse >>that comps and issue the appropriate yum install commands instead of a >>groupinstall. > > Then mock has to grow how to parse a comps file. And which format > should be preferred? If we were to go that route, we're better off just > putting a list of packages in the mock config file, but that's just > painful from a maintenance perspective, IMHO Well, there's pain all around. Personally I think having to rebuild packages every time I make a buildgroups change is painful. I also think that being required to have fake packages in my buildroots is annoying. At least with my suggestion you can point mock at a comps file you are confident it will be able to parse. By making it a url, you can still centralize that portion of the config. From katzj at redhat.com Wed Dec 14 23:13:08 2005 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 14 Dec 2005 18:13:08 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A09F6A.2040101@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> Message-ID: <1134601988.2708.78.camel@bree.local.net> On Wed, 2005-12-14 at 17:40 -0500, Mike McLean wrote: > Jeremy Katz wrote: > > The incompatibility is more in the repo metadata than in yum -- yum > > 2.4.x (which is what's on the build hosts) doesn't understand the new > > comps format. And that's partly because the comps side of things has > > had a lot less thought and effort put into making sure that it covers > > the needs of the userbase. And I'm going to go out on a limb and say > > that I expect this _not_ to be the last time it changes... we're > > actually starting to use the information and so I'm all but certain that > > we'll find things that we hadn't considered. > > I was thinking more along the lines of the "new yum can't parse old > comps" issue. If it could, it seems like we could just migrate to the > new yum in the buildsystem. Only if the new yum doesn't require pulling in lots of other stuff. And it's far better to just be able to use _any_ yum. Otherwise, eg, people won't be able to build for the devel tree on old releases without upgrading their yum. > For that matter, if the old yum could be told to ignore comps on a > per-repo basis, we could work around it that way. That should work -- set enablegroups=0 in your repo config file. > >>Let me throw out another possibility. Add a mock config option named > >>buildcomps containing a url. If this option is present, have mock parse > >>that comps and issue the appropriate yum install commands instead of a > >>groupinstall. > > > > Then mock has to grow how to parse a comps file. And which format > > should be preferred? If we were to go that route, we're better off just > > putting a list of packages in the mock config file, but that's just > > painful from a maintenance perspective, IMHO > > Well, there's pain all around. Personally I think having to rebuild > packages every time I make a buildgroups change is painful. I also think > that being required to have fake packages in my buildroots is annoying. > > At least with my suggestion you can point mock at a comps file you are > confident it will be able to parse. By making it a url, you can still > centralize that portion of the config. mock doesn't do the parsing, though (or any downloading of files at all) -- all of that is in yum. To make mock do the parsing would need a) mock to be able to download files b) mock to be able to parse the comps file That seems somewhat less than ideal to me Jeremy From mikem at redhat.com Thu Dec 15 03:38:56 2005 From: mikem at redhat.com (Mike McLean) Date: Wed, 14 Dec 2005 22:38:56 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134601988.2708.78.camel@bree.local.net> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> Message-ID: <43A0E550.6050605@redhat.com> Jeremy Katz wrote: > On Wed, 2005-12-14 at 17:40 -0500, Mike McLean wrote: >>For that matter, if the old yum could be told to ignore comps on a >>per-repo basis, we could work around it that way. > > That should work -- set enablegroups=0 in your repo config file. Is this not a viable solution, then? Mock only needs the groups from the (fairly stable) groups repo. We could set enablegroups=0 for all the other repos and make sure that the comps in the groups repo is safe. From skvidal at phy.duke.edu Thu Dec 15 04:21:02 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Wed, 14 Dec 2005 23:21:02 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A0E550.6050605@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> Message-ID: <1134620462.22019.22.camel@cutter> > Is this not a viable solution, then? Mock only needs the groups from the > (fairly stable) groups repo. We could set enablegroups=0 for all the > other repos and make sure that the comps in the groups repo is safe. > safe in what regard? We still need to add a bunch of intelligence to mock, then. It needs to know how to fetch it and parse it and that's just one more file format to deal with. I do agree with your point of having to build new packages if you change the packages in your buildgroups but one nice thing you get from this is sign-able buildgroup specification. b/c, right now, packages can be signed, but the groups file cannot. -sv From mikem at redhat.com Thu Dec 15 17:49:10 2005 From: mikem at redhat.com (Mike McLean) Date: Thu, 15 Dec 2005 12:49:10 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134620462.22019.22.camel@cutter> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> Message-ID: <43A1AC96.7040005@redhat.com> seth vidal wrote: >>Is this not a viable solution, then? Mock only needs the groups from the >>(fairly stable) groups repo. We could set enablegroups=0 for all the >>other repos and make sure that the comps in the groups repo is safe. > > safe in what regard? We still need to add a bunch of intelligence to > mock, then. It needs to know how to fetch it and parse it and that's > just one more file format to deal with. The enablegroups=0 approach is separate from my suggestion of a buildcomps config option. Unless I am misunderstanding (which I admit is possible), you could: - set enablegroups=0 for all repos except [groups] in the yum config that is embedded in the mock config. - make sure that the buildgroups.xml in the groups repo is understandable by the yum version installed on your mock system You still have yum parsing the comps, but it only parses one (and that one is easier to control). Since the groups repo is so directly tied to mock, it seems reasonable to expect that buildgroups.xml should match your mock/yum setup. > I do agree with your point of having to build new packages if you change > the packages in your buildgroups but one nice thing you get from this is > sign-able buildgroup specification. b/c, right now, packages can be > signed, but the groups file cannot. Good points. I'm not 100% opposed to the fake packages, but the idea still kinda bugs me, especially if it becomes the only way. From skvidal at phy.duke.edu Thu Dec 15 18:20:21 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Thu, 15 Dec 2005 13:20:21 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A1AC96.7040005@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> Message-ID: <1134670821.25232.14.camel@cutter> > The enablegroups=0 approach is separate from my suggestion of a > buildcomps config option. Unless I am misunderstanding (which I admit is > possible), you could: > > - set enablegroups=0 for all repos except [groups] in the yum config > that is embedded in the mock config. > - make sure that the buildgroups.xml in the groups repo is > understandable by the yum version installed on your mock system > So then depending on what release the user is running mock and yum on then the buildgroups.xml file they are using will change? So then we'll need more than one buildgroups repo for misc fedora building based on what distro the user is using? That seems to scale less well. > You still have yum parsing the comps, but it only parses one (and that > one is easier to control). Since the groups repo is so directly tied to > mock, it seems reasonable to expect that buildgroups.xml should match > your mock/yum setup. yes - but now you have to maintain 2 of these files. > Good points. I'm not 100% opposed to the fake packages, but the idea > still kinda bugs me, especially if it becomes the only way. wrt to rebuilding the package. What if I could give you a script that just took a spec file and spat out the noarch.rpm? Would that make your life easier and less irritated by the package-full-of-deps, thing? -sv From mikem at redhat.com Thu Dec 15 20:50:47 2005 From: mikem at redhat.com (Mike McLean) Date: Thu, 15 Dec 2005 15:50:47 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134670821.25232.14.camel@cutter> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> <1134670821.25232.14.camel@cutter> Message-ID: <43A1D727.9000406@redhat.com> seth vidal wrote: >>You still have yum parsing the comps, but it only parses one (and that >>one is easier to control). Since the groups repo is so directly tied to >>mock, it seems reasonable to expect that buildgroups.xml should match >>your mock/yum setup. > > yes - but now you have to maintain 2 of these files. not if new yum could handle old comps. > wrt to rebuilding the package. What if I could give you a script that > just took a spec file and spat out the noarch.rpm? Would that make your > life easier and less irritated by the package-full-of-deps, thing? The rebuilding isn't that big a deal, more of an annoyance. My main worry is the fake packages sitting in the repos. Using fake packages like this has always seemed like a hack to me. Sometimes it is necessary, but I hate to see it used as a central mechanism in the build system. In my case the fake packages introduce a problem in bookkeeping. I can work around it, of course, but I'd rather avoid the mess. Anyway, how about a compromise? See attached patch. -------------- next part -------------- A non-text attachment was scrubbed... Name: mock-compromise.patch Type: text/x-patch Size: 1237 bytes Desc: not available URL: From skvidal at phy.duke.edu Thu Dec 15 21:02:16 2005 From: skvidal at phy.duke.edu (seth vidal) Date: Thu, 15 Dec 2005 16:02:16 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A1D727.9000406@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> <1134670821.25232.14.camel@cutter> <43A1D727.9000406@redhat.com> Message-ID: <1134680536.26513.4.camel@cutter> On Thu, 2005-12-15 at 15:50 -0500, Mike McLean wrote: > seth vidal wrote: > >>You still have yum parsing the comps, but it only parses one (and that > >>one is easier to control). Since the groups repo is so directly tied to > >>mock, it seems reasonable to expect that buildgroups.xml should match > >>your mock/yum setup. > > > > yes - but now you have to maintain 2 of these files. > > not if new yum could handle old comps. I do not think we'll be putting the old comps support into yum 2.5.X or later. That'd be like adding 1.x headers support into yum 2.1 and higher. in short, it'd be silly. The whole point of changing the api is to be able to CHANGE it. > The rebuilding isn't that big a deal, more of an annoyance. My main > worry is the fake packages sitting in the repos. Using fake packages > like this has always seemed like a hack to me. Sometimes it is > necessary, but I hate to see it used as a central mechanism in the build > system. Sure - it is a hack. All of building inside a chroot is a hack, too. What's your point? > In my case the fake packages introduce a problem in bookkeeping. I can > work around it, of course, but I'd rather avoid the mess. what bookkeeping problem do they introduce? > Anyway, how about a compromise? See attached patch. How is this a compromise? -sv From mikem at redhat.com Fri Dec 16 00:09:50 2005 From: mikem at redhat.com (Mike McLean) Date: Thu, 15 Dec 2005 19:09:50 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134680536.26513.4.camel@cutter> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> <1134670821.25232.14.camel@cutter> <43A1D727.9000406@redhat.com> <1134680536.26513.4.camel@cutter> Message-ID: <43A205CE.7030703@redhat.com> seth vidal wrote: > How is this a compromise? Your proposal was to have mock perform the prep install from a set of specially named fake packages rather than from comps groups. I believe this patch allows that. My current thinking is that I'd like to keep using the comps groups and just make sure that my yum and my repos and in alignment. This patch allows that. ...or perhaps this is more or less what you had in mind all along, but I go the impression that mock support for using groups was in danger of disappearing. From enrico.scholz at informatik.tu-chemnitz.de Fri Dec 16 16:34:09 2005 From: enrico.scholz at informatik.tu-chemnitz.de (Enrico Scholz) Date: Fri, 16 Dec 2005 17:34:09 +0100 Subject: [PATCH] Allow to customize the used plague-client Message-ID: <87r78d57cu.fsf@kosh.bigo.ensc.de> Hello, please apply the attached patch why allows to use an alternative plague-client in the Fedora Extras CVS make-system. Thx Enrico -------------- next part -------------- A non-text attachment was scrubbed... Name: custom-plague-client.patch Type: text/x-patch Size: 1802 bytes Desc: adds $(PLAGUE_CLIENT) variable to allow a custom plague-client URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 480 bytes Desc: not available URL: From katzj at redhat.com Fri Dec 16 16:57:38 2005 From: katzj at redhat.com (Jeremy Katz) Date: Fri, 16 Dec 2005 11:57:38 -0500 Subject: [PATCH] Allow to customize the used plague-client In-Reply-To: <87r78d57cu.fsf@kosh.bigo.ensc.de> References: <87r78d57cu.fsf@kosh.bigo.ensc.de> Message-ID: <1134752259.18062.20.camel@bree.local.net> On Fri, 2005-12-16 at 17:34 +0100, Enrico Scholz wrote: > please apply the attached patch why allows to use an alternative > plague-client in the Fedora Extras CVS make-system. Applied, thanks Jeremy From skvidal at linux.duke.edu Mon Dec 19 06:05:58 2005 From: skvidal at linux.duke.edu (seth vidal) Date: Mon, 19 Dec 2005 01:05:58 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A205CE.7030703@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> <1134670821.25232.14.camel@cutter> <43A1D727.9000406@redhat.com> <1134680536.26513.4.camel@cutter> <43A205CE.7030703@redhat.com> Message-ID: <1134972358.24389.60.camel@cutter> > Your proposal was to have mock perform the prep install from a set of > specially named fake packages rather than from comps groups. I believe > this patch allows that. > > My current thinking is that I'd like to keep using the comps groups and > just make sure that my yum and my repos and in alignment. This patch > allows that. > two questions: 1. what does giving this option buy us? Other than more code to maintain, I mean. 2. why wouldn't you just go on using the older mock? > ...or perhaps this is more or less what you had in mind all along, but I > go the impression that mock support for using groups was in danger of > disappearing. very much so. -sv From mikem at redhat.com Mon Dec 19 21:37:55 2005 From: mikem at redhat.com (Mike McLean) Date: Mon, 19 Dec 2005 16:37:55 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <1134971811.24389.57.camel@cutter> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> <1134670821.25232.14.camel@cutter> <43A1D727.9000406@redhat.com> <1134680536.26513.4.camel@cutter> <43A205CE.7030703@redhat.com> <1134971811.24389.57.camel@cutter> Message-ID: <43A72833.3030500@redhat.com> seth vidal wrote: > two questions: > 1. what does giving this option buy us? Other than more code to > maintain, I mean. It seems like a reasonable config option to me, and it's hardly much more code to maintain. It would make it somewhat easier for someone to rapidly adjust their buildroot config while tinkering with builds. > 2. why wouldn't you just go on using the older mock? I could, or I could run a patched mock, although I had hoped to use what the community is using. Look, I don't seem to be winning anyone ever here. I think I'll just deal with the fallout of the proposed change. Ultimately, staying in tune with the rest of the mock community is more important to me than my previously stated objections. From skvidal at linux.duke.edu Tue Dec 20 08:33:55 2005 From: skvidal at linux.duke.edu (seth vidal) Date: Tue, 20 Dec 2005 03:33:55 -0500 Subject: mock and fc5/fe5 and comps pain In-Reply-To: <43A72833.3030500@redhat.com> References: <1134452540.6158.37.camel@cutter> <43A07C37.9000807@redhat.com> <1134596141.2708.65.camel@bree.local.net> <43A09F6A.2040101@redhat.com> <1134601988.2708.78.camel@bree.local.net> <43A0E550.6050605@redhat.com> <1134620462.22019.22.camel@cutter> <43A1AC96.7040005@redhat.com> <1134670821.25232.14.camel@cutter> <43A1D727.9000406@redhat.com> <1134680536.26513.4.camel@cutter> <43A205CE.7030703@redhat.com> <1134971811.24389.57.camel@cutter> <43A72833.3030500@redhat.com> Message-ID: <1135067635.31481.22.camel@cutter> On Mon, 2005-12-19 at 16:37 -0500, Mike McLean wrote: > It seems like a reasonable config option to me, and it's hardly much > more code to maintain. It would make it somewhat easier for someone to > rapidly adjust their buildroot config while tinkering with builds. And it confuses new users as which to use, doesn't it? I'm not really all that wed to this - I'm just trying to find an easy solution to this issue - which was the point of mock to begin with. whatever, I'm too tired for this argument anymore.. -sv From ivazquez at ivazquez.net Wed Dec 28 05:41:54 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Wed, 28 Dec 2005 00:41:54 -0500 Subject: [PATCH] Allow creation of new branches via cvs-import.sh Message-ID: <1135748514.4522.4.camel@ignacio.lan> The attached patch (hopefully) allows cvs-import.sh to create new branches. Please check it extra-super-thoroughly for correctness before applying. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: cvs-import.patch Type: text/x-patch Size: 7588 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ivazquez at ivazquez.net Wed Dec 28 07:51:10 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Wed, 28 Dec 2005 02:51:10 -0500 Subject: [PATCH] Allow creation of new branches via cvs-import.sh In-Reply-To: <1135748514.4522.4.camel@ignacio.lan> References: <1135748514.4522.4.camel@ignacio.lan> Message-ID: <1135756271.15008.0.camel@ignacio.lan> On Wed, 2005-12-28 at 00:41 -0500, Ignacio Vazquez-Abrams wrote: > The attached patch (hopefully) allows cvs-import.sh to create new > branches. Please check it extra-super-thoroughly for correctness before > applying. Silly bugs. Silly me. New patch. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: cvs-import.patch Type: text/x-patch Size: 7578 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ivazquez at ivazquez.net Thu Dec 29 02:44:48 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Wed, 28 Dec 2005 21:44:48 -0500 Subject: [PATCH] Allow creation of new branches via cvs-import.sh In-Reply-To: <1135756271.15008.0.camel@ignacio.lan> References: <1135748514.4522.4.camel@ignacio.lan> <1135756271.15008.0.camel@ignacio.lan> Message-ID: <1135824288.15008.16.camel@ignacio.lan> Okay, never mind the patches for now. I have a mostly-tested script on my machine now, but I still need to test it live. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ivazquez at ivazquez.net Thu Dec 29 05:33:27 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Thu, 29 Dec 2005 00:33:27 -0500 Subject: [PATCH] Allow creation of new branches via cvs-import.sh In-Reply-To: <1135824288.15008.16.camel@ignacio.lan> References: <1135748514.4522.4.camel@ignacio.lan> <1135756271.15008.0.camel@ignacio.lan> <1135824288.15008.16.camel@ignacio.lan> Message-ID: <1135834407.15008.21.camel@ignacio.lan> Okay, this one has been tested and should work properly. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: cvs-import.patch Type: text/x-patch Size: 8918 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ivazquez at ivazquez.net Thu Dec 29 05:39:27 2005 From: ivazquez at ivazquez.net (Ignacio Vazquez-Abrams) Date: Thu, 29 Dec 2005 00:39:27 -0500 Subject: [PATCH] Allow creation of new branches via cvs-import.sh In-Reply-To: <1135834407.15008.21.camel@ignacio.lan> References: <1135748514.4522.4.camel@ignacio.lan> <1135756271.15008.0.camel@ignacio.lan> <1135824288.15008.16.camel@ignacio.lan> <1135834407.15008.21.camel@ignacio.lan> Message-ID: <1135834767.15008.23.camel@ignacio.lan> I fail. -- Ignacio Vazquez-Abrams http://fedora.ivazquez.net/ gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72 -------------- next part -------------- A non-text attachment was scrubbed... Name: cvs-import.patch Type: text/x-patch Size: 8956 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: