sane dependencies -- a positive look at 'fix your packages'

[Sean Middleditch]

On Sat, 2003-10-11 at 18:22, Mike Hearn wrote:

> Another simple example - statically linking the Jamboree gnome music
> player would change the size of the binary from 600k to 24mb, which is
> clearly insane.

Statically linking isn't the answer proposed at all.  Anyone trying to
do that will make the situation *much* worse.

I'm ASSuming Havoc meant bundling with as how many Windows apps do it -
they come (metaphorically speaking) as a bundle of RPMs, and the
dependencies are then installed as needed.

With our world of up2date/yum/apt we don't need to bundle the
dependencies.  We just need to not keep breaking them so they stop
working unless you recompile half the machine.  i.e., if mydep 1.1 isn't
API/ABI compatable with mydep 1.0, they must be coinstallable, and the
library versions updated.

> 
> Now, you can say "Linux is a platform, which consists of libraries A, B
> and C - do not depend on anything else". That approach basically works,
> but is kind of difficult to force through. One mans essential library is

LSB is just this, and many distros are LSB compliant these days.

> anothers bloat, it may be that some libraries cannot commit to the ABI
> guarantees needed etc. I have 151 packages installed with "lib" in their
> name, it'd be pretty hard to get them all into even a tiered platform,
> though it could be done.

Those that can't keep a stable ABI simply need to reflect this in some
meaningful way to both developers and users - soversions and
co-installable data, big warning labels, etc.

> 
> > It's also what an LSB-compliant package _must_ do. But of course, nobody
> > seems to make LSB-compliant packages, instead they just complain about
> > how packages are distribution-dependent. ;-) People, this is the whole
> > point of the LSB...
> 
> Well, have you tried to make an LSB compliant package? It is really
> quite hard. For instance, a standard compile of any non-trivial C app on
> RH9 will pull in the thread-local locale model stuff from glibc 2.3. 

This sounds like it needs to be fixed; i.e., an easy way to disable
that.  I know apps linked against older libcs work on RH9 (usually, the
occasional odd app fails, which is quite unfortunate).

> 
> You can use the lsb tools to statically link everything needed, but it
> really does have to be *everything*, you cannot say "well everybody has
> GTK, i'll skip that" - the way the tools work mean the build will fail
> if you link against libraries that were not LSB compiled themselves.
> That's not really intentional, it's just because ld wants to completely
> close the symbol set. We figured out a way around that with apbuild, but
> the LSB weren't interested of course - they *want* people to be forced
> to only use their base set (they need to be able to guarantee what an
> LSB app is and isn't).

Which is good.  The LSB just needs to define more packages.  But that's
just LSB.  LSB is, again, most useful for commercial software.  When
talking open-source apps, we have a lot more leeway.  Namely, the
upstream devs can provide the "standard".  Example, so long as the GTK
devs don't break their excellent track record of sane API/ABI
versioning, and so long as people package it properly (which, so far, is
the case on RH/Fedora), software can rely on GTK.

The same goes for any other dependency in which the upstream authors
apply proper release techniques.

> 
> Not to mention that the LSB headers are not the same as the glibc ones,
> for instance I once tried to compile DBUS (i think v0.1 or something
> quite old now) as an LSB app to see what it was like, but the build
> failed due to obscure bugs in the LSB headers. Something to do with
> sunrpc iirc, which was wierd but there you go. I should try again
> sometime, as I reported the bug.

LSB, I should note, also isn't for low-level system stuff.  It's really
for monolithic apps.  I don't even see the point in installing something
like, say, apache, as an LSB package - its not really the target
audience.

> 
> Then you have to be able to make an RPMv3 package that meets their
> stringent restrictions etc...
> 
> So back to my original example, an LSB package of Jamboree would have
> taken most of the day to download on a 56kbit connection, as opposed to
> approx 5 mins for a non-LSB package that simply depended on the right
> libraries.
> 
> > Unfortunately, for moving around and launching you're using a .desktop
> > file. For install/uninstall you're using the r-c-p "comps group" concept
> > sometimes, and "foo.rpm" files sometimes. 
> 
> Yes, this is not ideal. The way I was planning to tackle this (when
> someday autopackage doesn't suck) was to embed package information into
> the .desktop file, then teach Nautilus/Konqueror about that. They could
> read the .desktop file, see that the relevant program was not installed
> and perhaps gray it out to show that it'd take some time to launch
> (because it'd need to be downloaded). The launcher, when clicked, takes
> care of download, verification, installation, dep resolution and so on.

That would be interesting.  The problem is, then, every single app that
possibly exists in the application/dependency archive would be in the
menues.  Ick.

While I understand the use cases, as I've said on the autopackage lists,
and real Application Manager is needed (which I've some new ideas for,
i'll post those to the ap list).

> 
> You could also use the launcher to uninstall, though the idea of garbage
> collecting packages appeals to me in some vague way.

Which isn't really possible.  The best you could do is inform the user
that the system doesn't *appear* to be using the app anymore (no users
have a launcher for it, if you can even determine that easily) and let
them optionally uninstall it.

The same is true to libraries/dependencies - sure, normal users might
not need them if no other packages are using them, but hackers/admins
probably have a lot of stuff installed from source.  So automatic
dependency "garbage" collection isn't really feasible, at least not
without lots and lots of user interaction and such.

> 
> In this scenario you have only one concept as a user, that of "the
> application" (or launcher) - you can click it, email it to friends, drop
> it on a disk etc, and the details of what CPU arch and deps you have etc
> are sorted out by the system.
> 
> It's kind of like the MacOS X AppFolders system UI wise, except without
> the massive bloat and suckyness that accompanies a pure NeXT style
> system.
> 
> > So ways to improve this might include:
> > 
> >  - a way to bundle a set of RPMs into a single user-visible file, 
> >    so you can have an "uninstalled app" visible in the file manager
> 
> Too large for dialup users I think :(

It could be useful for on-disk management, corporate deployment, CDs,
etc.  But probably not useful enough if the tools just become more
intelligent; i.e., if I click on a package in Nautilus and
redhat-install-packages sees I'm missing a dependency, it should try
looking in the folder the main package is in to find them.  That would
make a lot of "off the 'net" RPM installations easier, for one.  If the
package could point to URLs that have a list of packages for dependency
resolution, it would get even cooler (tho then GPG security and such
would probably be more important - I can just see the "Trust content
from Macromedia" dialogs and such one's accustomed to in Windows...  but
then, blindly trusting an entire distro which is community based and in
which nearly anyone can eventually get access to upload packages, those
incessant dialogs are probably a better approach...)

> 
> >  - have ways to install/uninstall a package by manipulating the thing 
> >    implemented now as a .desktop file (the menu/filemanager entry)
> 
> Seems the easiest way forward, though it'd be easier still if there were
> standards for packaging.

yay standards!  ;-)

> 
> >  - try to ensure packages for "end user apps" always have names that 
> >    match the primary desktop file in the package; so e.g. the Gnumeric 
> >    .rpm package would show up in file manager as "Gnumeric Spreadsheet"
> >    and once installed you get same in menu
> 
> Kind of icky. What happens if you click an installed RPM? It'd look like
> it should run the app, but then how do you remove it? It also relies on
> something that could be broken quite easily (the name mapping).

Fortunately, the file name of an RPM is completely arbitrary.  In large
collections with comps file (or whatever), "specific" naming is
possible; but if I want to put an RPM of some single piece of software
on my site, I'm quite free to call it anything.

> > Those are just some possibilities, not well-researched proposals. The
> > general idea is to fix the "leakiness" of the current abstraction;
> > ideally, a naive end user doesn't have to learn 3 implementation details
> > when a single concept of "an application" should be sufficient.
> 
> Right. I quite like the idea of a non-technical user browsing to
> frozen-bubble.org, and seeing the icon embedded in the web page. They
> drag it to the panel, or perhaps even just click it in the webpage
> directly. The button starts glowing or something to indicate that the
> computer is working, and the process of installation gets going in the
> background. When it's done, the icon goes coloured, maybe with some
> pretty sparkly effect to indicate "newness", and the program launches.

Hmm, a bit farther than I had in mind, but it would certainly work; one
can imagine the "RPM Installer" Mozilla plugin.  ;-)

If we even get to the point tho where the user can click the website
icon, and the Browser pops up a "Install Application?" dialog that
downloads and installs without the glitz, we'd be a long way ahead of
where we are now (click icon, get RPM, watch it not work, realize some
goof made the libinsane1 on OS release 2 completely incompatible with
libinsane1 on OS release 1, spend time figuring out whether the lib or
the app is easier to rebuild, find out the rebuild needs tons of build
dependencies which the tools don't bother to grab for you, etc. etc.).

> 
> The app is now available from the menu, the web page, can be dragged to
> the panel/desktop/whatever. User decides frozen bubble is really cute,
> and opens up their chat program and drops the icon (the application, in
> effect) into the chat window, where it appears the other end grayed out
> and the process begins again. If that other user is on PowerPC and lives
> the opposite end of the planet, no problems, the right download is
> selected from the right mirror.

That would certainly be nifty.  Really, all that would require a small
.desktop-like file describing the package, and the mirror-net its on. 
The install app just finds a mirror, looks up the real package (and
dependencies, possibly querying not only the app's mirror but also the
distro's repository), etc.  Heck, I think there are a couple (not nearly
done) apps that do just this, minus the high-level of desktop
integration.

> 
> A lot of work, obviously, but all doable. First of all you need an easy
> way to build distro-neutral packages. From my experiences, I'd say the
> LSB is not it....

Agreed.  Most of the responsibility has to lay with the upstream
developers.  If the authors of libfoobar are goofs and refuse to put the
3 seconds of effort forth to make their software at least co-installable
with other versions, than any upstream dev who relies on libfoobar is
just as guilty of bad practices.

A good guide on packaging might help resolve a number of such
situation.  One of the biggest problems with RPMs isn't even that the
software is actually incompatible, just that the packaging *makes* it
incompatible; packages putting things in different locations, packages
that conflict with each other in order to provide different optional
featuresets (why in all the world should a user have to unisntall an app
to add features?  sounds dumb, but yet we have to do it many times,
because make packages like someapp-nofeatures, someapp-feature1,
someapp-feature2, someapp-allfeatures, etc.  not necessary!)

Evangelism on the importance of proper development practices could be
useful, too.  Probably the only place I've seen describing library
versioning in detail is the innards of the libtool manual.  No wonder
lots of devs don't do it right, eh?  Something like the war on drugs is
needed: DARP, Developers Against Ramshackle Packages.  ;-)

> 
> thanks -mike
> 
> 
> --
> fedora-devel-list mailing list
> fedora-devel-list at redhat.com
> http://www.redhat.com/mailman/listinfo/fedora-devel-list
-- 
Sean Middleditch <elanthis at awesomeplay.com>
AwesomePlay Productions, Inc.