[olpc-software] Package manager stuff

[Mike Hearn]

On Wed, 2006-03-15 at 14:22 -0500, David Zeuthen wrote:
> I'm reiterating all this to give some kind of insight to what kind of
> "external applications" we can expect.

I think this would be a wonderful platform to start with. Miles better
than Win32 at any rate, which is the competition.

> Btw, I was wondering whether we really need union mounts to make your
> proposal work? Won't playing tricks with PATH and LD_LIBRARY_PATH work
> instead? If so, what are the constraints of this? Things like IPC and
> other stuff comes to mind.

Union mounts are not totally essential. I proposed them because a _lot_
of software assumes everything is installed under /usr, /etc and so on.

You don't realise how deep this assumption goes until you try and let
people install software elsewhere - as both autopackage and CodeWeavers
products let users do this to some extent I've experienced this pain
directly.

For instance, consider the following things that have to be adjusted to
enable a new prefix to work correctly:

* Shell PATHs. For every shell, mind, not just bash.
* DBUS config file needs to merge paths
* Fontconfig config file needs to merge paths
* $XDG_DATA_DIRS
* Menu XML definitions sometimes need to merge in extra .desktop
  directories
* Bonobo activation paths
* Linker paths
* man/info paths
* screensaver paths

The list never seems to end. Certainly adjusting all these paths at
runtime would require serious patching action.

Essentially this is a problem of namespace design: without union mounts
the filing system can only represent one kind of "query". On Windows/OS
X this query is "what files are logically related by software package?",
and on UNIX this query is "what files are logically related by
purpose?". 

Both choices are fine, but because every modern OS needs to be able to
do both kinds of query another separate database is maintained. On
Windows this is the registry and this is what is queried to get file
association information, COM activation paths, menu definition data etc.
On Linux this is the package manager database. And on OS X this is a
database held in memory by "LaunchServices".

Union mounts let you drop the separate database because now you can use
the filing system to define both groups. Keeping /usr around makes it
the 'central file type database' and that is great for backwards
compatibility and convenience. Keeping stuff in /applications means you
don't need a separate db to find out which files are related by package
(file hash lists etc can always be stored
in /applications/whatever/.hashes).

> (yes, I really wish myself that we had union mounts, don't Darwin or Mac
> OS X support this btw?)

There is unionfs which works OK and is in widespread deployment on Live
CDs, and actually I use it myself a bit. But it's not a part of the
mainline kernel. And as pointed out by Alan the code doesn't seem to
mention file notifications anywhere (I've asked on the unionfs list
about this).

I don't know if OS X supports union mounts but I'd doubt it. They don't
use the filing system to do things like "Give me a list of every help
document on the system", instead they do RPCs to LaunchServices which
builds a database at startup and then registers watches to keep it up to
date.

> Oh - our image build scripts will make this somewhat easy for you to
> experiment with [1] as the whole OS is prepared there. 

Yes, I saw that. It looks nice and easy to play with. For extra bonus
points this summer I'll be exiled to Malvern again for two months, which
means I'll be using a 600mhz laptop with 128mb of ram and no internet
access in the evenings. Apart from resembling the specs of an OLPC
laptop the lack of internet is great for productivity :)

Hopefully I'll get time to hack on some of this stuff then. It's still a
few months off but I want a binary ABI lint tool for autopackage anyway,
so our goals are aligned here.

> Personally I
> think it would be really awesome if you/we could come up with a solution
> (based on autopackage? modified?) that fits both the requirements we
> come up with (cf. you mail about use cases and design which is good) and
> is feasible to implement kernel, user and desktop-space wise.

Yeah, where/if autopackage fits into this is still something I'm looking
at. If you assume a closed universe then there's no point to it, as
autopackage is at heart an abstraction of Linux distributions. If a goal
is to let the wider Linux community package software for the OLPC
without necessarily being a part of the project (or even knowing about
it) then it could prove handy. Autopackage also currently lacks a good
online update system, but it's not too hard to add and I hope to reach
it this summer.

The rest of it is feasible IMHO. Bringing unionfs up to scratch is
probably the biggest chunk of work, there's a difference between a live
CD and a full time OS. Writing the little programs to watch network
mounts and manage the unions etc is easy. So there's some work to be
done. On the other hand, the requirements are sufficiently different
that I don't think "zero coding" is realistic for software management
here.

One nice thing is that personally if I can find time and energy, I
wanted to try these ideas out anyway. As they're quite experimental they
could be tried in parallel and the OLPC project gets them "for free" in
effect.

> [1] : I'm not sure if our Mercurial repository for these tools are
> publically available yet, but if it's not it will be as soon when FC5 is
> out - the people controlling servers in our DMZ have been / are busy
> with Fedora and RHEL update releases...

That's cool. Why Mercurial though?

thanks -mike