[Fedora-livecd-list] Woohoo! 31% boot speedup (*so far*), and 4.5MB free...

[Douglas McClendon]

Disclaimer: take these claims with the proverbial grain of salt, as I 
can on occasion be a bit over-optimistic or over-committal...

Second Disclaimer: read the first disclaimer again.

BUT THAT SAID...

The Bad News: my "SuperDeviceMapperCaching" technique does not (yet[1]) 
appear to actually significantly improve LiveCD boot speed.

The Good News: Some of the things that I did which were prerequisites of 
SDMC, appear to have the potential to speed up LiveCD boot by at least 
31%, and even shrink the LiveCD by 4.5MB.

The Better News: those prerequisite things are actually far less 
invasive (non-invasive really) than SDMC.

Review: The SDMC idea was to use a virtualized boot of a LiveCD to 
determine which files were accessed during boot, then remaster the root 
filesystem so that those files would be grouped together, so that they 
could be read into a portion of the rootfs that would temporarily live 
in ram, via a devicemapper linear combination of the 'cached/preloaded' 
portion, and the normal rest of the rootfs on CDROM.

The Reality: For expediency, I opted to gather the list of files via a 
simple find atime command after a virtual boot settled.  I then sorted 
the files (and perhaps more importantly, directories and symlinks) by 
size, and then populated a filesystem, starting with the smallest 
boot-accessed file, and working my way up to the largest boot accessed 
file, and then just doing the rest.

The realization: At some point I realized that this was actually 
gathering the files at inner part of the cdrom, which is read half as 
fast as the outter part.

The sick future: I will get those files pushed to the outter edge...  no 
matter what sick devicemapper magic it takes ;)

The already optimistic numbers, that I really hope I'm not 
misinterpreting or overhyping-

I timed a fedora-8 livecd boot on my laptop.

I got about 2:30s to gdm, 3:30s to gnome desktop fully rendered, and 
3:50s to firefox launched and fully rendered.

I then (slight lie, did this last), timed a build of my very fedora-like 
livecd, built with my VirOS tools, but for all practical purposes, in no 
way radically different than the fedora livecd.

What I got was 2:06s to the first post rhgb cursor (I'm using gdm 
autologin, so this is either a gdm mouse arrow, or the gnome mouse 
arrow).  Then 3:19s to gnome fully rendered, and 3:23s to firefox fully 
rendered.  (firefox was autostarted and fighting with gnome a bit).

Then, I tried my SDMC livecd, but without SDMC, just the file 
reordering- and got 1:48s to first gdm mouse arrow, 2:29s to gnome fully 
rendered, and 2:34s to firefox fully rendered.

I think I was able to get my SDMC mechanism to shave 2-5 seconds off of 
that, but not really a terribly significant difference.

Another interesting fact, is that the reordered filesystem was 4.5MB 
smaller than unordered.  This is no doubt because when I sort those 
files by size, a lot of very similar files end up spatially closer 
together, and are thus better compressed by squashfs.

Thus, what I tenatively see, beyond the 4.5M of free space, is a 31% 
speedup (203s/154s)

So, going forward, I plan to

[1]A) do some sick stuff to get those early files on the outter rim of 
the cdrom, which I think may get further significant gains by itself, 
and perhaps even push my SDMC method into the significantly profitable 
area (though I don't much care at this point).

B) perform all of this semi-manually on the official F8-livecd, and see 
how much I can really improve 'the real deal'.

C) do boot file access data gathering on a pair of dissimilar physical 
hosts, and use the file diffs to determine classes of files (hw drivers 
mostly) which should get added to the list.  Probably /lib/modules and a 
few other obvious directories should just get thrown into the list.

D) I don't really care enough at this point to do a full systemtap or 
similar boot profile.  atime seems sufficient for now (though I wonder, 
do I miss a lot of files that get touched when root is still mounted 
readonly?)

E) do all of this for ubuntu, where I think my SDMC techniques, combined 
with their native squashfs usage, will yield as much, or perhaps even 
more benefit.

-dmc/jdog

Disclaimer: I really hope this all doesn't turn out to be fruitless... 
but I'm doing a pretty good job convincing myself that it all makes 
sense...  (I'm just disappointed that I failed to really eliminate 95% 
of all seeks during boot...  I still think that might be possible... but 
I need to go write a little C program to visualize the population of a 
sparse file... or instrument some filesystems....   Nah... I'm pretty 
sure I've got lower hanging fruit within my grasp...)

P.S.- When I say I 'did' all of this, I of course mean completely 
scripted from a single VirOS commandline that just takes nearly half a 
day to run :)