[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: More upd-instroot speedups

Jeremy Katz wrote:
On Tue, 2007-10-09 at 16:00 -0600, Orion Poplawski wrote:
- Remove and old, unneeded duplication of the file lists adding "./" to
the paths. We no longer use cpio to copy files, so this just slows us down (a lot).

Indeed.  Although I'm surprised it makes a big difference

Well, running "find" is a large component of the time upd-instroot takes. Halving the number of times it runs helps. Running find "smarter" (with the find patch) also helps.

 >> anaconda-ldso.patch:
- This doesn't actually work yet, but it tries to do the following:
- Fixes the code to find and install shared library dependencies
- removes those libraries from being explicitly listed in the file lists.
- Other list cleanup
- This might actually slow us back down, but it does fix what appears to have been completely broken and makes adding new binaries easier.

Woohoo!  Someone else sat down to do this :-)  I noticed it was broken
right before test2, but we needed to get the test release out the door
and I said "I'll get back to it afterwards".  Which, of course, I
haven't.  If you get this, I definitely owe you beer.

Glad my effort wasn't in vain then. Will need to be more careful about what's in the filelists though as we're picking up lots of extra stuff now that we probably don't need. Also, perhaps it makes sense to only look for so deps if the file is in a "bin" has an ".so" in the name, or at least not if in "share"? Could speed things up again.

- Use "tsflags=nodocs noscripts notriggers" to avoid extra rpm transactions.

This one I'm less sure about -- I'd like to actually be able to get rid
of a lot of the other junk in upd-instroot that duplicates the working
being done by package scriptlets.  While it might cost us in build time,
the savings in maintenance seem like they're worth it.

Yeah, makes sense. And I'm less worried about yum times running on tmpfs and with the yumdir patch.

New timing runs with buildinstall tmpdir on tmpfs filesystem (which is how I usually run). Each test adds a new patch. I've split the find patch into to parts. findpath just limits the find to the path specified. findtest adds the non-wildcard case. The "bin" patch only looks for so deps of files in a "bin/" directory or with a ".so" in the filename, but apparently doesn't make a difference in this case. YMMV.

               orig   nocpio yumdir findpath findtest ldso   bin  noldso
yum text:       2:48s  2:29s  2:54s   2:27s    2:37s  2:36s 2:53s 2:27s
install text:  11:37s  9:23s  9:20s   7:09s    7:07s  1:46s 1:45s 0:57s
yum graph:      3:27s  3:55s  1:17s   1:15s    1:13s  1:15s 1:13s 1:13s
install graph: 29:07s 20:46s 20:41s  13:13s   12:48s  3:02s 3:03s 1:51s

So not only is the current so dep code broken, it's horribly slow. You might want to just remove the following:

--- anaconda- 2007-10-10 13:39:29.000000000 -0600 +++ anaconda- 2007-10-10 13:39:33.000000000 -0600
@@ -150,12 +150,6 @@
        cp -aL $FILE $DESTROOT/`dirname $FILE`
-    get_dso_deps $(pwd) "$FILE"
-    local DEPS="$DSO_DEPS"
-    for x in $DEPS ; do
-      instFile $x $DESTROOT
-    done

 expandPackageSet() {

from instFile() for the time being, since it makes no difference to the files installed in minstg2.img or stage2.img. That's the "noldso" column above.

Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion cora nwra com
Boulder, CO 80301              http://www.cora.nwra.com
--- anaconda-	2007-10-10 10:52:53.000000000 -0600
+++ anaconda-	2007-10-10 11:00:21.000000000 -0600
@@ -153,11 +153,13 @@
 	cp -aL $FILE $DESTROOT/`dirname $FILE`
-    get_dso_deps $(pwd) "$FILE"
-    local DEPS="$DSO_DEPS"
-    for x in $DEPS ; do 
-      instFile ./$x $DESTROOT
-    done
+    if [ "$FILE" != "${FILE/bin\//}" -o "${FILE} != "${FILE/.so/} ]; then
+      get_dso_deps $(pwd) "$FILE"
+      local DEPS="$DSO_DEPS"
+      for x in $DEPS ; do 
+        instFile ./$x $DESTROOT
+      done
+    fi
 expandPackageSet() {

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]