yum-deltarpm (Was Thread Hijack - Our package management GUI tools need improvement)

[Ahmed Kamal]

The idea of using metadata to conclude whether/not drpm reconstruction will
be successful, is possible. This metadata is called a sequence number "SEQ"
for some reason. However, at this time, I have not integrated code for that.
The current code blindly tries downloading and fails with a 404. Integrating
metadata is listed on the TODO on presto page, and it shouldn't be too
difficult anyway. However, since drpms are usually too small compared to
their comparative rpm, and since in almost all cases on disk files are not
going to be corrupt, I did not see much value in using this kind of
metadata.

The drpm generator script does keep drpms on the server only if they are
worth keeping. The worthfulness numbers, of course can be tuned later. I
even think it might be a good idea to make worthfulness depend linearly on
the new rpm size. i.e. keep drpms for large rpms, even if savings are not
that great. We are however getting very good savings on large packages
anyway

My main focus now is on testing and making sure the "base" system is working
as it should. Any ideas about that regression system? Do you think it's a
good idea? I'm not primarily into coding, so I'll need help making sound
decisions. I'm thinking of having a full FC6 install, then using drpms to
upgrade that into *-testing, that should give us some nice reports for how
many upgrades/reconstructions are failing. We'll probably need some server
to host the drpms on, plus the test client.

On 3/9/07, Warren Togami <wtogami at redhat.com> wrote:
>
> Jonathan Dieter wrote:
> > On Fri, 2007-03-09 at 12:35 -0500, Warren Togami wrote:
> >> 2) Ahmed Kamal has been working on a potentially sane implementation of
> >> deltarpm for Fedora's yum.  Theoretically, it would work as an optional
> >> yum plugin.  If the deltarpm is substantially smaller than an RPM
> >> update, then the deltarpm is provided on a mirror.  If the deltarpm is
> >> not provided, then yum downloads the original RPM instead.  If it
> >> downloaded a deltarpm, it reconstructs the original RPM and uses GPG to
> >> verify integrity just like yum would verify plain RPM downloads.
> >>
> >> Ahmed probably could use some developer and testing help.  I've been
> >> encouraging him to be more communicative about his project in order to
> >> get more help, but I haven't seen any further outreach lately.
> >>
> >> Warren Togami
> >> wtogami at redhat.com
> >>
> > I've been working with Ahmed and Michael Schroeder (the upstream
> > maintainer of deltarpm) to track down some long-standing bugs in
> > deltarpm, especially as it relates to prelinked binaries.  These bugs
> > were causing very odd problems while working with the yum-deltarpm
> > plugin.
>
> Interesting!  I had not considered prelinking.  I'm glad you found this
> problem.
>
> Does it check the integrity of all other files (except %config) before
> deciding to attempt to download the deltarpm?  (If some other file is
> modified, abort and download the regular RPM?)
>
> >
> > We *think* we've found them all (the patches are in the latest Rawhide
> > version of deltarpm in Extras), so I think we're at the point where what
> > we really need is someone who would be willing to create drpms of all
> > new packages in Core and Extras (there's a modified version of prunerepo
> > that does all the work for you), and host them for us.
> >
> > To give you an idea of the savings we're looking at:
> > * kdebase-3.5.5-0.4.fc6  => kdebase-3.5.6-0.1.fc6 =  3.5MB vs. 30.2MB
> > * kdegames-3.5.5-0.1.fc6 => kdegames-3.5.6-0.1.fc6 = 740KB vs. 11.1MB
> > * OO.o-core-2.0.4-5.5.3  => OO.o-core-2.0.4-5.5.10 = 8.7MB vs. 92.4 MB
> > There won't be that kind of savings for all the packages, but a general
> > rule of the thumb is that the bigger the package, the better the chance
> > that we'll get a good compression ratio on the drpm.
>
> Generated drpms are only to be kept on mirrors if the download savings
> are substantial.  Some threshold level has to be decided.
>
> For example (actual numbers can be decided later):
> Only push a drpm to mirrors if the download savings are greater than 40%
> total size, and greater than 5MB.
>
> Warren Togami
> wtogami at redhat.com
>
> --
> fedora-devel-list mailing list
> fedora-devel-list at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-devel-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20070309/eee865cd/attachment.htm>