Some ideas/questions about yum

Fri Aug 28 17:05:08 UTC 2009

Hi again,

On ۰۹/۰۸/۲۸  04:50, Seth Vidal wrote:
>
>
> On Fri, 28 Aug 2009, Hedayat Vatnakhah wrote:
>
>> Hi all,
>> Currently, Fedora package management is one of the most annoying part
>> of Fedora experience for new desktop users (at least for those
>> without fast, always available internet connection). For such users,
>> Fedora package management "Just Doesn't Work"! I've almost never been
>> able to demonstrate using fedora package management tools for a fresh
>> Fedora install without the need to use command line, editing yum
>> configuration file(s), killing current running yum/package kit
>> instance(s), installing some small rpm packages using rpm command
>> instead of using yum or graphically, etc. And sometimes, I found it
>> better to download yum metadata using another application and copying
>> the downloaded file to yum cache; or even completely skip yum and use
>> rpm and manually resolve dependencies when they are not too much!
>
>
> Have you filed bugs on any of these issues or are they strictly due to
> unreliable network connections?
I've reported bugs for some of them (e.g. the bug you know about 
resuming downloding metadata files), I'll report some more, and some of 
them happened on systems of some users and I wasn't able to investigate 
them to file useful reports for them.
And yes, many of them happen with poor (or no) network connection only.

>> First, I've some suggestions/requests which doesn't seem to need much
>> work, and then some ideas which I'd like to know your opinions about.
>> 1. Since Fedora 4, Fedora doesn't support installing software from
>> DVD "out of the box". Fedora 8 is an exception here. Currently, it
>> seems that the work is almost done (99% completed as in [1]), and
>> fixing the remaining 1% doesn't seem to need much work but
>> unfortunately it seems that it is stopped. I think requesting a small
>> collaboration between the feature owner, PolicyKit and/or GIO people
>> is not too much.
>
> Alsadi has done a great deal of work to make this happen and I believe
> it is likely it will go in for F13.
I hope that happens.

>> 2. Maybe yum could be a bit more forgiving about inaccessible
>> repositories when running. Consider this case: a new offline user
>> installs Fedora, and then runs "Add/Remove Software". Currently, if
>> he clicks on "all packages", he'll see an error message that yum is
>> unable to contact fedora repository. I think it is better to show a
>> warning to user about being unable to contact online repositories and
>> then show all installed packages + the packages from all accessible
>> repositories. IMHO it is much more reasonable than expecting the user
>> to disable all such repositories in such cases (yum/packagekit can be
>> a bit more intelligent and do it itslef).
>
>
> Disabling repos which are unavailable/inaccessible is a pretty
> dangerous behavior. If you're doing a 'yum install foo' and the only
> 'foo' you have access to is insecure - but a secure 'foo' is in the
> updates dir - it would be better for you to not install 'foo' at all
> than install a bad one.
And this is why the warning should be shown! The warning message could 
state that there might be security concerns too. But what if the user 
just wants to remove a package (maybe a separate section called 
"installed packages" in PackageKit will do the job in this case) or he 
just wants to install a package from a local repository (DVD) or 
rpmfusion?! I think the current frustration for the end user is not 
better than what you mentioned.

>> Now, some ideas:
>> 3. AFAIK, currently yum's primary database file contains information
>> about packages, and all of the files in directories such as /usr/bin
>> and /usr/lib, so that it can resolve package and file level
>> dependencies. Isn't it possible to move file level information
>> outside primary db (e.g. to primary_file_deps.db) and translate
>> internal dependencies from file level dependencies to package level
>> dependencies when creating repositories? (So that provides and
>> requires tables in primary db only contain package references rather
>> than file references?). It might be even possible to do it for
>> dependencies outside repository; for example when creating updates
>> repository, you can introduce fedora repository to createrepo, so it
>> can translate all of the file level dependencies of updates packages
>> also.
>
> Except none of this will work for dependencies for 3rd party repos at
> all. Nor will it adequately handle any of the cases where we have
> multiple pkgs which provide the same file.
For the first case, it'll work. Yum will download the file I mentioned 
(primary_file_deps.db) when it needs to resolve file dependencies. I do 
not know what do you do now about the latter case and what is that at 
all, so I'll be silent here!

>> 4. Even if the above solution is possible and can reduce the size of
>> primary db, it won't solve the main problem: for large repositories,
>> you'll need to download large database files. You'll need to download
>> extra database files on some use cases anyway. So, it can be said
>> that currently yum doesn't scale well.
>
> no - it can be said that yum if you have a lot of pkgs you have a lot
> of metadata. That's not a function of yum scaling that's a function of
> network connectivity.
OK, maybe I use bad wording. Yes, as the number of package grow, the 
experience become undesirable because of larger metadata. It's so 
annoying to be forced to download 10Mb for installing a 200K package 
specially somewhere like a dial-up connection!

>> What do you think about it: we can implement parts of yum at the
>> server side (e.g. a web service), and do queries online. The client
>> can submit queries to online repositories, aggregate the results
>> (+using local repositories by itself) and do appropriate actions. It
>> can also store received data to be used when offline or while they
>> are valid. It'll be completely backward compatible with the current
>> clients: those who use the old method can download repositories
>> themselves, like what they do now.
>> It is possible to think about further details and design it
>> completely, but I want to know about your opinions about the whole idea.
>
> That's what up2date and rhn did in rhl6->rhel3. There are multiple
> problems:
>
> 1. we need a single web service that many systems would access to
> query for depsolving If you want to talk about things NOT scaling....
Yes, you might be true. But is requesting some information such as: 
"What does foo provide?" or "Who provides bar?" more expensive than 
serving a 10M file?! Or searching for one word in package 
names/descriptions is much more problematic than searching a word in 
Fedora wiki? (BTW, the GUI should provide separate facilities for 
querying package names (yum list) and searching in package 
summary/description (yum search)).
Specially that all/most mirrors provide this service, not just one system.
Also, depsolving could be done by the client, but it'll download only 
the data it needs and not lots of extra data.

>
> 2. it would mean anyone wanting to do a respin couldn't just setup a
> website or an ftp mirror or a file:/// repo - they'd have to setup a
> web service and populate it. That's some pretty serious overhead.
Oh, no! First, the word web service was just an example, and the first 
word which came in my mind. This could be done using XML/RPC, a single 
php/python/... file can provide that service AFAIK. Doesn't need 
anything but copying a single file. Also, I didn't propose to drop 
support for local repositories (file://) or the current type of 
repositories. That service will be an add-on to the current system. 
Also, I clearly stated: "+using local repositories by itself". So, I 
don't see any problem here.

>
> 3. 3rd party repos would have to go through a whole other level of
> hoop jumping.
Yum would fall back to the old method if the repository doesn't provide 
the service (or it might be configured explicitly to use that). Also, 
all what 3rd party repositories will need to do is to add the single 
file (or some files!) I mentioned earlier.

>
>
> Now, I don't think anyone is opposed to seeing some of these things
> but given what you've said on the yum-devel mailing list (where much
> of this discussion should be anyway) you don't seem to be interested
> in actually working on it or have the time. Did that change?
Oops, sorry! Maybe it is because of my poor English (sorry for that). I 
am certainly interested in actually working on yum; but I mentioned that 
I do not have much free time and I'll be slow. So, I am interested, and 
I'm planning to put some time on it, but I'll be slow. If I find that 
this idea will work, I'll certainly start working on it. And this is 
exactly why I asked this question here, since I know that there are many 
people here that know a lot about such things and can let me know about 
the issues.

Thanks for the answer,
Hedayat

>
> -sv
>