build system glue scripts and requirements

[Jeremy Katz]

On Fri, 2005-03-25 at 05:09 -0500, seth vidal wrote:
> > Yep, has seemed pretty reasonable to me in the poking at it I've done so
> > far.  mschwendt's idea of running a lot of already built packages
> > through to see how they fare isn't a bad one.  I'll try to set up one of
> > my test boxes to do this over the course of this week (hopefully
> > tomorrow).  You don't have to worry about doing that right now :-)
> 
> You did this and it did reasonably okay, right?

Yeah, almost everything succeeded.  Just quick checking some of the
failure logs showed missing BuildRequires being the reason, which seems
sane.  Thanks for poking to remind me.

> > One other thing that springs to mind is the question of what arches to
> > build packages for and whether a specific arch failing should block the
> > build.  My opinion, which matches how Core gets built, is that 
> >   * Packages get built for all Extras arches.  ExcludeArch/ExclusiveArch
> > can be used for the (rare) things which need otherwise
> >   * Build failures on one arch block all arches.  Otherwise, some arches
> > will fall far behind and things like prereqs get really painful
> 
> This might be tricky. I'm worried about notifications b/t the
> buildsystems. Or are you thinking about having levels like:
> 
> buildsystem -> pops out rpm in some known location
> releasesystem -> determines if all arches have built and therefore if
> the pkg ever gets moved/copied/linked to the release tree it's targeted
> for.

That's the easiest way to do it.  A file gets dropped in specific
location (BUILD-SUCCESS, BUILD-FAIL) that can just be watched for.  We
don't have to have instantaneous knowledge of build success. 

> > And the bit that we talked about during LinuxWorld on how we want to
> > make it possible and easy for developers to download the buildsystem and
> > get it going on their own workstation for test builds.  That then also
> > enables other third party repositories (there won't be only one :-)  to
> > use it and get some consistency.
> 
> Here's what I'm thinking right now:
> If we can get a good working interface for bugzilla for package
> tracking. And we can define some new fields/tags for items in this
> interface then we should be able to let it work for us.

This could work.  Although more "fields" in bugzilla always makes me a
little bit wary.  Even though I know dkl would end up just overloading
existing fields in some cases and making them look different with
template magic.

> 4. build system puts the finalized packages + other stuff in some path 
>    that's web accessible as you described above.(CAVEAT: some special 
>    casing for embargo'd builds will need to be put in place)

Yeah, embargo'd stuff probably requires thought.  There's not really a
way to do it right now with the CVS repo either, though, so it's further
out as a question

>  3. Having one build master system scan and kick off builds on other 
>     machines/arches is not crazy OR having multiple build systems scan, 
>     mark the package as 'being built for arch foo on system bar' is 
>     also not outside the realm of possibility (though there might be 
>     race conditions there)

I think there is provision for having a build master machine?
Cristian? 

> Disadvantages:
>  1. might be overstating the functionality available in the xml-rpc 
>     interface to bugzilla

It just sounds like query is mostly needed.  And for the nice, easy to
use build target from the makefile, open bug/add comments.  Those should
both be present.

>  2. Users cannot directly kick off builds, they have to wait (waaaaaaah)

So long as the polling is done frequently enough, this isn't a huge
deal.

>  3. Dealing with Embargo'd builds - gonna be a pain no matter what

See above.

>  4. Ordering of builds based on dependency

FIFO.  You need to request your builds in dep order.  Sucks a little,
but is easy to implement :)


Also, another thing I've thought about (and it's run through my head a
few times, this is just the first time I've remembered to type it out).
One thing that would be nice would be able to have multiple roots for
the same release in mach easily (ie, without having to make copies of
the config files with tweaked paths :-).  At the same time, that's
probably not something that matters in the short term as it could easily
be added as an optimization later.

Jeremy