Optimizing kernel-devel

[Sam Varshavchik]

Arjan van de Ven writes:

> On Sat, 2006-05-13 at 12:53 -0400, Sam Varshavchik wrote:
>> After drinking yet another half a dozen cups of tea while waiting for both 
>> kernel-devel and kernel-devel-smp to install, I have an idea how to fix this 
>> mess:
>> 
>> 1) Add one more file to kernel-devel and kernel-devel-smp packages, whose 
>> contents are the output of:
>> 
>> cd $RPM_BUILD_ROOT/usr/src/kernels/<<version>>
>> find . -type f -print | sort | xargs md5sum >../<<version>>.md5sum
>> 
>> 2) %post would then simply read all .md5sum files, reconcile them, and know 
>> exactly what to hard link.
>> 
>> I'd be surprised if this will not speed things up by at least a factor of 
>> 10.
> 
> another one would be to make a kernel-devel-base, which works for all
> versions, and which just has filenames that are the md5sums of all the
> headers. The kernel-devel package would then only contain symlinks to
> the md5sum filenames ;)
> 
> (yes it's a hack, and yes it's grossly horrible :)

You will also need to figure out how to create one that will hold all files 
that will go into every kernel release in the foreseeable future.


This one's really a no-brainer.  You're now forking/execing about seven 
thousand times, one for each file in the kernel-devel package, and reading 
and computing the checksum, every time you install a -devel package.

Compare that against computing the checksums only once, when the package 
gets built, using only a fraction of the fork/exec pairs.  Then you can 
install the package on a thousand machines, and each install merely reads 
all .md5sum files, finds the dupes, and creates the links.

The way this is done now is very inefficient.  I could see how it happened 
-- oh, yeah, no big deal, just slap on a find+exec of hardlink.  No sweat. 
But, the consequences of doing so were never considered.

The Perl script to reconcile the .md5 files, and create the hard links, is 
laughably trivial.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20060513/82c3d497/attachment.sig>