Default Groups, Default Packages, CD Sets and DVD Size

[Jeroen van Meeuwen]

Jesse Keating wrote:
> On Mon, 2008-08-11 at 17:40 +0200, Jeroen van Meeuwen wrote:
>> However, the default installation requires all 7 CDs, because of how the 
>> compose tools resolve dependencies (inclusive) during the compose of the 
>> media, and pull in more then is minimally required to complete the 
>> actual transaction of a default installation. The compose tools do so 
>> for good reason:
>>
>> - one cannot know what package is the user-preferred package for any 
>> given required capability (eg. for fictive capability 'web-client', 
>> there's firefox, iceweasel, elinks, wget, curl, emacs, emacs, emacs, 
>> foo, bar and baz)
>>
>> - one cannot predict on what installed system one is performing an 
>> upgrade, and to be able to close the transaction certain considerations 
>> must be met justifying the need for inclusive dependency resolving when 
>> composing the media (set) or installation tree.
>>
>> - it makes the released media apply to N+X use-cases where the package 
>> set or transaction payload during the installation is controlled in a 
>> more granular fashion then the selection dialogs allow (by means of a 
>> kickstart package manifest maybe?), which I guess applies more to 
>> businesses or advanced users using Fedora then it does to Joe Average users.
>>
>> Basically what I'm saying is that the 2.88 GB is spread over more then 
>> the minimal amount of discs it would fit on, since the complete package 
>> payload when using inclusive dependency resolving grows to 3.66 GB, or 7 
>> discs. So, the compose process spits out 7 discs, each of which contain 
>> a part of the 2.88 GB sized RPM payload needed for a default installation.
> 
> I've been playing a bit in this area, and noticed that if I make one
> small change to pkgorder, at least on i386, a default 'next next next'
> install based on a compose done with the Fedora config and it only needs
> the first 3 media.  This isn't bad really, given the growth of the
> distro.
> 

I'm extremely curious what the change is as I've been digging through 
this for hours.

>> == Next Generation of Package Ordering ==
>>
>> So, what package ordering is going to do -instead of having a static 
>> list of groups to add to a transaction, resolve, spit out the packages- 
>> is use the "default" parameter to groups in comps.xml as well (and then 
>> instead of exclusive dependency resolving like it does now, move to 
>> inclusive dependency resolving as well). This makes the "which packages 
>> and groups are in a default installation" a little less hard to maintain 
>> and the package ordering will almost automagically match up with 
>> comps.xml (of which the installation procedure also uses the default 
>> parameter to groups!)
> 
> Hrm, I hadn't thought about applying inclusive depsolving (or
> depresolving as I call it) here.  That could have some interesting
> effects and warrants some investigation.
> 

I've done this investigation and I found the following:

Put to extremes, just to get the point across without nitpicking 
percentages, in principle the inclusive dependency resolving during 
package ordering results in a "tighter" set of packages on, say, disc 
1-3, including the packages not used by a default install, but 
potentially used by a relatively large number of non-default installs 
and upgrades, whereas using exclusive dependency resolving during 
package ordering supposedly stuffs the default install onto a minimal 
number of discs (say, 1 & 2), but you might need disc 5, 6 & 7 if you 
make a small change in what is going to be installed.

>> == Advise needed ==
>>
>> There's several ways this can be solved, but I'm not sure what is the 
>> most advisable (some are not feasible I'm sure, I'm just brainstorming 
>> here):
>>
>> 1) reduce the number of mandatory and default packages per group in 
>> comps.xml
> 
> This is a great start, something that hasn't really been done outside of
> core/base recently that I'm aware of.
> 
>> 2) reduce the number of groups in comps.xml that have "default" set to True
> 
> This one is going to be a bit tougher to get people to agree upon I'd
> imagine, but I'm willing to give it a go.
> 

Do we know, per group or overall, who can do this?

>> 3) Revisit how comps is formatted; Example: Keep the "default" for 
>> compose decisions, but add an "install" attribute for installation 
>> decision making. Install set to True may require default set to True as 
>> well for the group to even be included on the media.
> 
> What exactly does this accomplish?
> 

Having "defaults" included on the media but having "installs" being 
selected by the installation procedure so that the compose process can 
put them on the first few discs. Does that make sense? It still requires 
reducing the groups installed by default though.

>> 4) Split the packages that are mandatory or default in comps groups, 
>> into smaller packages providing what the group needs and another set of 
>> smaller packages belonging to the group as to reduce the number of 
>> dependencies needing to be met when the compose or installation 
>> procedure selects a group. See also PS2.
>>
>> 5) Or, compared to 4, revisit the Requires in mandatory and default 
>> packages and the Provides in the packages that provide the required 
>> capabilities so that it abstracts from the requires/provides matching 
>> with too many other packages (related to the inclusive dependency 
>> resolving which will then make for a thinner RPM payload on the composed 
>> media)
> 
> This is probably a more interesting case, something that we want to do
> for folks like OLPC as well.
> 
>> 6) Have the compose tools as well as the installation procedures not 
>> depend on the default attribute to groups anymore, at all.
> 
> What does this accomplish?
> 

Not so much it'd sorta obsolete the default attribute (iirc the 
compose/ordering/install is the very purpose of the default attr. isn't 
it?).

-Jeroen