Plan for tomorrow's (20070604) Release Engineering meeting

[Dominik 'Rathann' Mierzejewski]

On Friday, 08 June 2007 at 01:32, Axel Thimm wrote:
> On Thu, Jun 07, 2007 at 11:18:46PM +0200, Nils Philippsen wrote:
> > On Mon, 2007-06-04 at 14:15 -0400, Jesse Keating wrote:
> > > On Monday 04 June 2007 13:45:07 Tom "spot" Callaway wrote:
> > > > I thought about this, but it wouldn't ensure proper ordering between
> > > > packages. Packager A rebuilds after Package B is done, but changes to
> > > > Package A affect Package B's build.
> > > 
> > > How would a mass rebuild be any different?  A mass rebuild is likely to go 
> > > through in either ls ordering or python hash ordering....
> > 
> > That needn't be the case. Packages could be built in a "from the ground
> > up" order beginning with what's by default in the buildroots (i.e. what
> > doesn't need to be build-required). This gets only ambiguous with cyclic
> > build-dependencies in which case we'd have to fall back to something
> > else (ls ordering, python hash ordering or even "bug the release
> > engineers and let them decide" ;-)).
> 
> But Jesse rightfully argues that doing so requires a createrepo
> running after each build [1], which takes 20-30 minutes. So for 4000
> packages you would need 55-83 *days* just for createrepo.
> 
> Of course this can be sped up, 20-30 minutes sounds like a lot even
> for a blank non-cached createrepo run. And the repomd data could be
> optimized in both creating and downloading to only take a tiny
> fraction of time & bandwidth of what it takes today, so the createrepo
> command is of the order of seconds instead of minutes. This would also
> benefit any yum user as a side-effect.
> 
> But a two-phase rebuild would already cater for most ordering issues
> anyway, especially when the tree is considered in a good shape.
> 
> [1] This isn't completely true however, since package N+1 will not
>     always depend on package N, so an intelligent mass-rebuilder could
>     skip a lot of createrepo steps by only issuing a createrepo when
>     really needed. But even if we kill say 3/4 of all createrepo
>     commands we would still spend 2 weeks in createrepo alone.

We could split the tree into self-contained subtrees and let the build
process "see" a repository containing only those packages it needs to
build. That'd speed up the metadata regeneration after rebuilds
considerably and we'd need only one (or a couple) of full tree metadata
rebuilds.

Regards,
R.

-- 
Fedora Extras contributor  http://fedoraproject.org/wiki/DominikMierzejewski
Livna contributor http://rpm.livna.org MPlayer developer http://mplayerhq.hu
"Faith manages."
        -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"