adventures in booting
David Malcolm
dmalcolm at redhat.com
Tue Nov 30 20:20:46 UTC 2004
On Sun, 2004-11-28 at 20:49 -0500, David Zeuthen wrote:
Awesome! So... is no-one else going to reply to this email?
Various comments inline below...
> Hey,
>
> So, I've looked a bit more into the booting process and how to optimize
> it. Mostly based on the discussion triggered by Owen's boot poster
> challenge, here
>
> http://www.redhat.com/archives/fedora-devel-list/2004-November/msg00447.html
>
> and also some experiments that I did - basically replacing rhgb with gdm
> as described here
>
> http://www.redhat.com/archives/fedora-desktop-list/2004-November/msg00066.html
>
> What I've done is a bit crude - I've replaced init(1) with a shell
> script based on /etc/rc.d/rc.sysinit and tried to optimize specifically
> for my system: IBM Thinkpad T41 laptop with a Pentium M processor at
> 1600MHz along with 512MB of RAM.
If I understand, you're also optimising for a specific user of that
system. Is it worth splitting the readahead into a system-wide list of
files (enough to get to the login screen), followed by a per-user list
for logging in as a user? To what extent will the files needed to get
to a usable desktop vary between Alice and Bob?
>
> The results are pretty good I think, here is the general time line made
> with a wallclock
>
> 00: exit grub; start booting the kernel
> 04: kernel prints audit()
> 11: initrd is mounted; Red Hat nash visible
> mount / ro (normal initrd procedure)
> 13: start bootchart logging; start readahead of approx 193MB files
> sleep until readahead is complete
> 24: readahead done; now
> create /dev and modprobe (in background)
> mount / rw, enable swap
> start xfs
> startx as user davidz in background
> start messagebus
> start hald
> start acpid
> start NetworkManager
> 32: X claims the display
> 34: GNOME desktop banner
> 40: GNOME desktop is usable (Nautilus desktop, panel fully populated)
>
> Here is a bootchart made with the bootchart software from Ziga Mahkovec:
>
> http://people.redhat.com/davidz/bootchart.png
> You may notice that I also start firefox after login and it starts very
> very fast - that's because readahead loads all files used by Firefox -
> in earlier experiments I've also added files from OpenOffice.org to
> readahead and that meant I could start up OpenOffice.org Writer in about
> three seconds. More below.
>
> I've made the following observations
>
> 1. The kernel patch, linux-2.6.3-printopen.patch, wasn't really working
> well for me - it reported far to few files - instead I added a
> printk() to fs/namei.c:link_path_walk()
> (disclaimer: I don't know much about the kernel so there may be a
> better solution than this).
>
> 2. The data captured from link_path_walk() was massaged into a list
> of unique files to preload and sorted on sectors.
>
> 3. While capturing the data link_path_walk() and before processing
> I went through all the menus in the GNOME desktop (to make sure
> their icon and desktop files would be added to the list) as well as
> loading Firefox. The list contains 5189 unique files - 231 of these
> from my home directory - 103 of these from gconf in my home
> directory and 302 from gconf in /etc. 2267 were .png files and
> 814 of them were .desktop files. 488 files had ".so" in their name.
> There was a total of 193MB of files (which says something about
> the footprint of the GNOME desktop on Fedora :-/)
>
> 4. Doing the readahead really helped the time from startx till a
> usable desktop - less than 10 seconds!
>
> 5. Doing readahead on the 5189 files took about 45 seconds on my
> system, mostly because the files were scattered around the disk.
> Since I had a spare partition 17GB partition, I did this:
> a. format spare partition as ext3
> b. copy all readahead files to spare partition (193MB)
> c. copy rest of files from main partition to spare partition
> (about 9GB)
> Now the readahead is down to 11 seconds which averages out to
> be 18MB/s. On the other hand, I can still see (using fileblock)
> that the files in the readahead is still scattered out and hdparm
> says I should be able to get 33.87 MB/sec with no seeks.
>
> 6. I made a hack to cache /dev (a dev.tar file) and the list of modules
> to load. This could be used in production if the kernel could give
> us basically a hash value for the kobject hierarchy representing
> the hardware (perhaps even a 'tree /sys |md5sum' would suffice).
> This shaved some seconds of as well.
>
> 7. A number of things was started in parallel - I found that doing
> readahead while running modprobe wasn't helping anything; in fact
> it contributed negatively to performance (a bit to my surprise, I
> guess because the kernel was busy).
>
> 8. readahead on the right files is A Good Thing(tm). Booting my system
> without readahead on the partition with the readahead files scattered
> took 58 seconds (compared to 39 with readahead on the optimized
> partition)
>
> http://people.redhat.com/davidz/bootchart-without-readahead-scattered.png
>
> and without readahead on on the optimized partition it took 43
> seconds
>
> http://people.redhat.com/davidz/bootchart-without-readahead-nonscattered.png
>
> again compared to 39 seconds. As an added bonus, the readahead
> makes sure that e.g Firefox loads fast; all .png and .desktop files
> are in place for when using the menus. As mentioned, one could put
> very big apps like e.g. OO.o in the readahead set.
>
> So, I think these numbers are good and there's still some room for
> improvement; e.g. it takes ten seconds from grub to when the initrd is
> mounted - surely the kernel can boot faster? It's after all 25% of the
> time spent from grub until I have usable desktop.
>
> The bad thing is that this approach is highly specific to my system (and
> thus why I'm not posting an RPM with it :-), however I think it clearly
> shows where improvements should be made; here are some random thoughts
>
> a. We should keep track of files being loaded and maintain the
> readahead fileset as appropriate. printk() doesn't seem like the
> right solution; perhaps a system daemon using inotify or the
> kernel events layer is the road ahead? This would enable us to
> readahead the KDE stuff if the user is e.g. using KDE a lot.
>
> b. ext3 should support operations for moving blocks around; e.g.
> optimize around the readahead fileset - when idle the system should
> rearrange the files to facilitate faster booting
>
> c. the start_udev and kmodule process could be cached as I did above
>
> d. The whole init(1) procedure seems dated; perhaps something more
> modern built on top of D-BUS is the right choice - SystemServices
> by Seth Nickell comes to mind [1]. Ideally services to be started
> would have dependencies such as 1) don't start the gdm service
> before /usr/bin/gdm is available; 2) the SSH service would only
> be active when NetworkManager says there is a network connection;
> /usr from LABEL=/usr would only be mounted when there is a volume
> with that label and so forth. Also, such a system would of course
> have support for LSB init scripts.
> (This is probably a whole project on it's own so I'm omitting
> detailed thinking on it for now)
Hopefully this could also allow us to make system-config-services look a
lot slicker. I've never liked the way it has random text spewage for
each service's status - some kind of widgetry would satisfy my eye-candy
cravings.
>
> Thanks a lot to Ziga Mahkovec for the bootchart software - it's been
> very useful.
>
> Have fun,
> David
>
> [1] : http://www.osnews.com/story.php?news_id=4711
> http://www.gnome.org/~seth/blog/2003/Sep/27
>
More information about the fedora-devel-list
mailing list