why doesn't yum cache anything?

[Konstantin Ryabitsev]

On Sun, 02 Jan 2005 21:59:58 +0100, Farkas Levente <lfarkas at bppiac.hu> wrote:
> just try out a:
> time yum -C list mtr
> it has nothing to do with the network and the speed is almost the same!

In my experience it isn't, but then again, doing "yum list \*mtr\*"
only takes 8 seconds on my laptop. If it takes longer on other
machines, considering mine is certainly not the fastest of the bunch,
my suspicion is that the slowdown is not dependent on the processor.
Memory and network would be my suspects.

> it seems there are two place where yum is very slow:
> - loading the local metadata (even if using the pickle files)
> - the dependencie resolution
> probably it needs to investigate further what is the real reason and how
> can be solve. if yum be so slow for a long time, there will be someone
> who create another package using a better/more clever local cache file
> format and and may be reimplement it in a faster language (may be a
> better/faster server side metadata format).

I'm not sure such a tool can ever be achieved with RPM. The reason
Apt-RPM is fast is because it does not involve RPM libs to do
anything: as you said yourself, the only time rpm is invoked is during
the final system call to "/bin/rpm --force --nodeps". This is
inherently unsafe, but if you go the way of using RPM libs for this,
things slow down significantly, and this is not in any way the fault
of yum, but of the underlying libraries. All yum does, effectively, is
pass data to rpmlib and wait for it. If any slowness is to blame on
yum, it's XML parsing, which takes a very long time, and even then
it's the problem of how C strings become Python strings.

With pickling in place things speed up significantly, as all the
system has to do is read the (very large) .pickle file into memory and
then convert it into python objects. However, even then these objects
must be converted into package classes, as it is very tricky to pickle
live package objects due to their underlying connection to librpm and
the RPM database. XML parsing occurs when you see "MD Read" as the
identification string, and it is relatively slow. When you see
"updates-released" or somesuch in the identification field, with ###'s
moving faster, that's Yum converting simple package metadata classes
into live rpm package objects. On my machine, a speed-stepped P4M
1200, loading primary metadata for base, updates-released, and
freshrpms takes about 3 seconds, and that's about 3500 packages. If
someone can speed this up further, they will be a better person than I
am, since I'm to blame for the XML parsing and pickling code.

However, like I said, I do not believe this is where we should look
for to speed up the performance of the package management system. The
slowness is not in yum, it's in the systems it relies on to function.
Let me be the one who voices a popular opinion among anyone who has to
deal with RPM (beyond simply packaging) -- it's not the best
imaginable packaging system out there. In fact, it lacks in many ways.
That is the reason why in the past year or two we saw the emergence of
two new alternatives to the existing implementation -- the one
proposed by Specifix, and now by SmartPM. And, of course, there is
Debian with its promising offshoot, the Ubuntu. (</plug>)

Feel free to encourage anyone to come up with a system better than yum
that doesn't do things the apt way, i.e. doing everything outside of
RPM and then just making a system call to the binary. I believe anyone
attempting to re-implement yum will soon run into exactly the same
problems as those faced by Seth et al. If someone does come up with an
application that functions speedily and covers the same robust set of
features that yum provides, then I'm sure Seth will have no problem
stepping aside and letting the poor bastard who replaces him to take
all the abuse. In fact, as someone for whom Seth is a direct
supervisor, I'll downright encourage that development. :)

However, I believe that until RPM is replaced with an able enough
alternative, things will not dramatically improve, however much you
desire them to, and however many times you rewrite yum. Feel free to
prove me incorrect.

Regards,
-- 
Konstantin Ryabitsev
Zlotniks, INC