RFD: kernel-devel package improvements

Tue Mar 15 18:03:45 UTC 2005

Warning: lengthy mail.

Once again, here's a request for discussion about the current kernel-
devel packages, and improving them to better meet the requirements of
module packagers.  As suggested by wtogami in private mail (and agreed
by riel, if I understand correctly), let's try on this list in order to
avoid too much noise which has been the problem with earlier kernel-
source/devel discussions on fedora-devel.

The kernel-devel packages currently in Rawhide are a good step towards
general usefulness, but they partially fail to implement what they've
been designed for, and there is room for improvements.  Rik has agreed
to follow this discussion and assist in evaluating and getting the
improvements applied if we can come up with something sane here.  Based
on Bugzilla traffic, I see he's already working on some of the issues
below.

Because I'm starting this discussion, I'll present my own biased,
lengthy POV first.  Be encouraged to comment, disagree and fill in the
blanks.  I'm focusing on the kernel-devel packages, not things like
module package naming; let's get the framework ready first.  I'm more
describing what I would consider "ideal", with less focus on backwards
compatibility issues or build system constraints.

Short version: I would like to more use of "uname -r" output in the
provisions and naming in general, and a single big kernel-devel package
for each archtecture "group", using (sym|hard) links in order to save
space for common files.  One example of one (fictious) arch group would
be eg. {i586,i686}{<up>,smp,xen0,xenU}, all of the above in a single
kernel-devel rpm, possibly noarch or i386 (although that might be tricky
to implement).  One example of the space-saving linking is in the
fedora.us/livna.org kernel-module-devel packages (<= FC3 only), see
http://cvs.fedora.us/cgi-bin/cvsweb.cgi/pkg/kernel-module-devel/
The end of this message contains a pseudo-module-specfile snippet
describing one approach how a module specfile could depend on stuff and
locate dirs, as well as a kernel specfile patch for
documentation/clarification purposes.  For the long version, read on.

Requirement categories: in order to be really useful for module
packagers and module package users, kernel-devel (and kernel) packages
should 1) provide means for robust build and install time dependencies,
and 2) be easy enough to use, and 3) not require module packagers too
much knowledge/hardcoding of the set of variants (think custom kernels),
and 4) honor the documented upstream way of building modules for the
currently running kernel while keeping it possible to build for a given
arbitrary kernel.

Definitions: for the purpose of this discussion, $version is the "base
version" of the kernel (without the possible "smp" etc variants; usually
equivalent to %{version}-%{release} in rpm terms), $variant is the
"smp", "xen0", "custom" etc part, $uname is the output of "uname -r" for
that kernel (ie. $version and $variant concatenated), and $arch is the
architecture.  For example, in the current Rawhide kernel-
smp-2.6.11-1.1177_FC4.i686 package: $variant=smp,
$version=2.6.11-1.1177_FC4, $arch=i686, $uname=2.6.11-1.1177_FC4smp.

Item 1: naming in general (categories 1, 2, and 3 above).

Module packages need to be able to robustly identify their build and
install time dependencies.  The components required for this AFAICS are
$version, $variant, and $arch.  I and several others have used $uname in
the past with pretty good results, and the kernel packages themselves
already use $uname in various places.  Here's a summary of how the above
"components" are currently being used in Rawhide kernel(-devel)
packages:

  a) rpm package names:
      kernel(-$variant)(-devel) = $version(.$arch)
  b) kernel*-devel provision:
      kernel(-$variant)-devel-$arch = $version
  c) Provision in all kernel packages:
      kernel = $version
  d) /usr/src/kernels dir names:
      $version-$variant-$arch
  e) Additional provision rumoured to be in the kernel packages or 
     coming soon, not in Rawhide yet though:
      kernel(-$variant)-$arch = $version
  f) Module dirs:
      /lib/modules/$uname
  g) Kernel binaries (note: not always in /boot):
      vmlinuz-$uname

rpm does not currently allow architecture qualified dependencies, so
such provisions are being and have been added to the kernel and -devel
packages in various forms, which is ok.

Of the above, b) and d) are the only suitable ones to be used as module
package build dependencies, and e) is the only one suitable for an
install time dependency.  However, IMO inventing another naming scheme
is not (or would not have been) necessary: $uname could have been used
in b), d) and e):
  b) kernel-devel-$arch = $uname
  d) $uname-$arch
  e) kernel-$arch = $uname
The good things about $uname over various other %{name}, $variant and
$version combinations are for example: it has always been there in the
kernel packages in various places, there's a lot of documentation out
there referring to it, people are familiar with it, using it avoids
having to know the global set of $variants.  Related Bugzilla entries:
https://bugzilla.redhat.com/145914
https://bugzilla.redhat.com/149249

Short version: IMO $uname should be used more, instead of inventing new
name-version-variant combinations.

Item 2: kernel header tree location (categories 1, 2, 3, and 4).

/lib/modules/$uname/build is AFAIK the documented, official upstream way
to point to the headers tree for a particular kernel.  This is mostly
relevant for people building modules (not necessarily packages)
theirselves, and probably eg. DKMS.  The current Rawhide packages sort
of break this "contract": https://bugzilla.redhat.com/149210
Keeping the "build" symlink in the devel packages as it is now results
in the possibility of $arch mismatch with the installed kernel package.
Moving the symlink away from the -devel packages to the kernel packages
would look like an obvious solution.

The header tree location is of course also relevant for buildsystem
produced module packages, or IOW ones to be built for a given kernel
version.  Item 1 and its Bugzilla entries above already discussed this
to some extent; in this case it's necessary to be able to find the
headers tree without having the target kernel installed, and to depend
on the package providing it.  I think /usr/src/kernels/$uname-$arch
would work for both.  /usr/src/kernels/$version-$variant-$arch works
too, but does not have the good properties of using $uname.

Note: using /lib/modules/$uname/build as the header tree location in
module packages does not look like a good idea.  If the build symlink
stays as is, there's possibility of arch mismatch as discussed above.
If it is moved to the kernel packages, the module package builds would
require the target kernel package being installed during the build,
which is bad.  Better to stick with the /usr/src/kernels/$whatever dir
when packaging modules and leave use of the build symlink to non-
packaged module builders.

Item 3: installing -devel packages for multiple archs (categories 2, 4).

For mass production of kernel modules in automated build environments,
it'd be good to be able to install the headers for all supported (and
buildable in the target system) archs in parallel.  The file/dir naming
part (to avoid conflicts) of this is AFAICS ok in Rawhide, but the
requirement is not met due to other implementation issues:
https://bugzilla.redhat.com/147553  I don't have good ideas how to fix
this apart from the vague suggestion of combining more archs and
variants in "bigger" and fewer kernel-devel packages (possibly noarch
ones).

Item 4: install time dependencies (category 1).

Currently there's no sane way to place arch-qualified install time
dependencies in module packages.  https://bugzilla.redhat.com/149249
There have been reports of "kernel(-$variant)-$arch = $version" being
The Way, but no FC kernel yet provides that.  Since it's not there yet,
I'd suggest considering using $uname instead (see end of Item 1 above,
as well as below).

Summary:

Yes, I am a $uname fan.  Anyway, to summarize, here's how a module
package could depend on the necessary packages and locate the necessary
dirs in its specfile, assuming the target $uname would be available at
build time as %{uname} (eg. by passing something like "--define 'uname
2.6.10-1.770_FC3smp'" to rpmbuild).  %{_target_cpu} is already
available, its value is determined by --target to rpmbuild.  People
building this example package for their running kernel would need to
specify the correct --target manually, but %{uname} could be "auto-
detected".

  %{!?uname: %define uname %(uname -r)}
  ...
  BuildRequires: kernel-devel-%{_target_cpu} = %{uname}
  Requires: kernel-%{_target_cpu} = %{uname}
  ...
  %build
  KSRCDIR=/usr/src/kernels/%{uname}-%{_target_cpu}
  ...
  %install
  DESTDIR=$RPM_BUILD_ROOT/lib/modules/%{uname}/...

Attached is also a patch against the current Rawhide kernel specfile
that would implement the stuff required for the above specfile snippet.
For brevity (in a mail like this, ha!) it doesn't take backwards
compatibility into account, but that'd be trivial to provide if
necessary.  It's untested, and included here for documentation purposes.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kernel-2.6.spec.patch
Type: text/x-patch
Size: 3984 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-maintainers/attachments/20050315/146d1348/attachment.bin>