Packaging review guidelines clarification

[Dan Williams]

On Sat, 2006-02-18 at 09:59 -0500, Josh Boyer wrote:
> On Thu, 2006-02-16 at 18:12 +0100, Thorsten Leemhuis wrote:
> > Am Donnerstag, den 16.02.2006, 17:45 +0100 schrieb Nicolas Mailhot:
> > > Le jeudi 16 février 2006 à 09:30 -0500, Josh Boyer a écrit :
> > > > > On 2/16/06, Paul Howarth <paul at city-fan.org> wrote:
> > > > > there was a discussion at somepoint about scratch build trees in the
> > > > > buildsystem to help with update builds.
> > > > +1
> > > +1
> > 
> > Enough votes ;-) I like the idea also and added it already to 
> > http://www.fedoraproject.org/wiki/Extras/Schedule/IdeasContainer
> > 
> > I put things like this there where I agree that they should be done for
> > fedora-extras sooner or later.
> > 
> > I'm also willing to put it on the FESCo Schedule -- but first I'd like
> > to have someone who drives this whole thing forward. Means: Ask dcbw and
> > the other buildsys people (skvidal,?) if this is is possible in general,
> > what is needed to get this running, help with doing the work that might
> > be needed, report and discuss details with FESCo and fedora-extras-list
> > and organize all other stuff that might show up to realize this idea. 
> > 
> > Any volunteers? No, it does not have to be someone from FESCo. Everybody
> > interested in this task can work on this.
> 
> Ok, I did some digging in the plague code this morning.  It seems that
> plague 0.4 already has support built into it for a testing/scratch repo.
> Namely, if the target is said repo, it changes the status of the build
> job to done after building and skips the addtorepo step. (Unless I read
> some code wrong).

(note that I renamed "scratch" to "testing" to reduce confusion.  If
"scratch" is still in the 0.5 code I should fix that.)

No, you read the code correctly.  However, there's a big gaping hole for
scratch/testing builds; dependencies.  Since the packages don't get
added to the distro repository or the buildsystem repository, you can't
build a testing package B that depends on a testing package A.  You can
only build single packages into a testing repo.  Oversight on my part
perhaps.  We figured this out right before we were going to enable
testing targets last year.

To fix this, we have to make testing targets full-fledged repositories
in their own right, that inherit from the normal distro repo too, and
then periodically purge packages over a day or two old.

> So... it's more a question of can we enable this on the FE buildsys and
> if so what method should we use?  Some options:
> 
> 1)
> 
> Plague has the ability to allow building from an SRPM itself.  We could
> allow folks to specify the SRPM for a package which will then be
> uploaded and built on the scratch repo.  I believe this option is
> disabled at the moment on the server config side of things.

Correct, you can either do SRPM _or_ CVS builds, but not both.  In the
context of testing targets/repos, perhaps SRPMs would be a good thing.
But I'm not entirely sure of that.  At least if all the files are in
CVS, there's a slightly higher accountability so that people don't just
shove crap or any old SRPM through the buildsys.  It's more "open" and
transparent what people are trying to build from a security perspective
too.  I guess I'd discourage SRPM building for these reasons in the
context of Fedora.

Remember, you can create arbitrary tags and branches with package CVS,
and the buildsys doesn't really care what tag the package comes from as
long as the tag exists.

> Doing this would help people ensure that new packages build correctly on
> all arches, etc before actually being imported into CVS.  The idea is
> that it'll catch some things that a simple review wouldn't and help
> fixup the package that much earlier.

You bring up a good point here.  But how do we gate this?  We probably
shouldn't be handing out buildsys accounts some new person 5 minutes
after they submit a package just so they can build some random SRPM they
found online.

> 2)
> 
> Enable the scratch repo and add a 'make scratch' target to the
> makefiles.  Same advantages as above, just that it requires the package
> to be in CVS already.

I think this is the more likely scenario.  We fix the testing/scratch
targets by making them real targets that just don't get their packages
moved over to the "official" repository ever.  We add cronjobs to kill
packages in the testing targets after two days.

> The issues with both of these options are: how do we make sure scratch
> builds aren't holding up real builds (or if that will even be a problem)
> and if option 1) is used, are there security issues at all with allowing
> people to upload SRPMs directly.

Priorities are going to take a little work in the buildsys.  Currently
every job has the same importance, and testing targets shouldn't.

So I propose that at some point in March we do a major buildsys upgrade
to grab plague 0.5 and give us:

1) depsolving (done)
2) real testing targets (needs a bit more work)
3) job priorities (to be done)
4) build preference (to be done)

Dan