the mechanics of pushing updates

[Luke Macken]

On Thu, May 24, 2007 at 01:02:35AM -0400, Bill Nottingham wrote:
> Mmm, plumbing. bodhi is heading for production soon. To push updates, what
> bodhi currently does is, for any update:
> 
> - sign the package

Nope, neither bodhi or the current update system sign any packages.
The current system mails releng with the proper command to sign the
packages, but it has always been done by hand as far as I know.
You and Jesse are the only people I know of that have signed updates.
By the looks of the fedora-release-tools module, there are two scripts
that have been used to sign packages, ftsign and fedorasign, both of
which call /usr/local/bin/rpm-4.1-sign, which is a symlink to
/usr/lib/rpm/rpmk.

I started implementing an XMLRPC server for bodhi so we can eventually
do everything from the command line as well as from the browser.
Hopefully we can streamline the signing process as much as possible in
a command-line sign/push tool, until it can be fully automated with a
signing server (when might this happen, btw?).

Koji keeps a sigcache for each package in pkg/ver/rel/data/sigcache/arch/,
although I have no idea at what point in the build process this gets
created.  I'm also under the impression that just having this detached
signature isn't enough, and that there still must be some manual
intervention?  Is there anything else bodhi needs to do other than make
sure the corresponding .sig exists for each package?

> - copy the package to a 'staging' tree of the entirety of updates
> - read a static list of packages that should be multilib, act on that

This isn't as bad as the previous biarch-list-of-doom[0] anymore.  Bodhi
imports it into its Multilib database table[1] during initialization, and
doesn't deal with it again.  Upon submission of an update, bodhi builds
the list of associated packages, taking care of multilib based on what's
in the db.  The multilib table can then be modified with ease using the
TurboGears CatWalk database editor, or a simple command-line tool.

> - run createrepo
> - check deps on the repo

FIXME: I need to track down some false positives in bodhi's
closure.py[2] (or rewrite it).  Mash would obviously resolve this for
us.

> - rsync the whole repo out

TODO.  At the moment bodhi stages to /mnt/koji/updates-stage -- where
are we going to sync this to? wallace still?

> Older updates are cleaned by a cron script later.

TODO.  We need something similar to the fedora-updates-clean script that
is currently in place (but less hackish), or RepoPrune / repomanage.
The TurboGears scheduler[3] is probably a good place for this.   I'm
going to try and find some time tonight to throw one together.

> Advantages of this approach:
> - it's simple
> - it's easy to clean upthings that Go Wrong (just manually remove them
>   from the repo and re-sync)

This also gives bodhi a LOT more control over the repos, as it maintains
the extended updateinfo.xml.gz in the repodata as well.  If we use mash
we will have to maintain this file outside of the tree and re-insert it
post-compose.

> Disadvantages:
> - multilib. In a world where we continually add new packages, this
>   *will not scale*.

Random idea.  Since multilib is handled by mash, which pushed out
rawhide nightly, couldn't we just have mash keep the Multilib table up to
date?  Would this solve the scalability issue wrt new packages?

> So, we need at least *some* sort of better workflow.
> 
> One alternative - using mash (what we're using to build rawhide.) It
> would go something like this:
> 
> - sign the package
> - tag the package (for updates-testing, or updates)
> - run mash to create a repo of updates/updates-testing, solve it for
>   multilib
> - rsync it out
> 
> Advantages:
> - solves multilib
> - doesn't require continually keeping a staging tree around
> - depcheck is built in when solving multilib
> - builds on koji tags to let anyone easily query what updates are
>   released
> 
> Disadvantages:
> - by rebuilding the repo each time, it's going to be slow once
>   the repo gets large
> - harder to clear out other strangeness
> - will only have one version of each updated package
> 
> The last of these isn't as *big* of a concern now, as all builds
> will be available through the koji web site, space permitting.
> 
> Other ideas for better workflow? What do the extras push scripts do?
> Do we want to add a modified version of mash's multilib solver into
> bodhi?

I think the mash idea is interesting.  Although, due to it's overhead,
we would probaby have to resort to pushing out a single batch of
updates a day, and maybe some smaller batches of security updates.  This
might become a pain.

I'm going to look around at the multilib solver for mash and extrsa
tonight and see if bodhi can steal any of it.  Michael Schwendt would
probably be the expert in the extras world.

luke

[0]:
https://hosted.fedoraproject.org/projects/bodhi/browser/bodhi/deprecated/biarch.py
[1]:
https://hosted.fedoraproject.org/projects/bodhi/browser/bodhi/model.py#L63
[2]:
https://hosted.fedoraproject.org/projects/bodhi/browser/bodhi/closure.py
[3]: http://docs.turbogears.org/1.0/Scheduler