Sugg: Improve for mock (caching)

[Enrico Scholz]

skvidal at phy.duke.edu (seth vidal) writes:

>> >> $ mock createcache
>> >> 
>> >> will create the chroot tree with yum groupinstall build and save it in a
>> >> gzipped tar file.
>> >> ...
>> > The network environment is incredibly fast where the buildsystems are.
>> 
>> The speedup gained by caching is not (primarily) related to the network
>> connection but to the rpm installation and disk-IO. On the fedora.us
>> server we used caching also and the initial buildroot was created in
>> <1min (afair).
>> 
>> Local tests at that time showed, that the initial buildroot can be
>> created in 20 seconds when a tmpfs will be used instead of a real disk
>> with ext3.
>
> So the numbers I'm curious about that you may have:

fedora.us does not exist anymore, so I will try to estimate the numbers
below with other methods. The times for a real buildsystem will be
higher because of additional operations.


> - remote repos - but lan-like connection times

The LAN-like connection is 100Mb/s ethernet to an ftp server; machine is
a P4 2.6GHz with 1GB RAM and a SAMSUNG SP1614C 7200rpm SATA disk.

I repeated the operations several times to avoid false results due to
caching effects.


| # rpm --root `pwd`/tree-1 -Uvh
| # ~mirror/fedora/core/4/i386/fedora-release-4-2.noarch.rpm --nodeps rpm
| # --root `pwd`/tree-1 --import /etc/pki/rpm-gpg/RPM-GPG-KEY*
| 
| # time ftp_proxy= yum -d0 -y -c `pwd`/yum.conf  --installroot=`pwd`/tree-1 install redhat-rpm-config gcc-c++ make tar
| 
| real    1m42.072s     ## 1st time
| user    0m51.955s
| sys     0m7.256s
| 
| real    1m39.572s     ## 2nd time
| user    0m51.263s
| sys     0m6.960s



> - local-disk repos - chroot creation - all on ext3

sorry, makes too much work to setup and configure the local repo...


> - tmpfs targets for the chroot creation and repos

My comments regarding 'tmpfs' are related to caching, so let me compare
caching on ext3 and on tmpfs:

| # tar cf tree.tar tree-1
| # ll -h tree.tar
| -rw-r--r--  1 root root 416M  3. Nov 22:23 tree.tar

ext3:
| # mkdir tree-2
| # time tar xf tree.tar -C tree-2
| real    0m37.369s
| user    0m0.248s
| sys     0m3.372s
|
| real    0m32.805s
| user    0m0.260s
| sys     0m3.456s
| 
| real    0m34.126s
| user    0m0.212s
| sys     0m3.492s

tmpfs:
| # mkdir tree-3
| # mount -t tmpfs -o size=500m none tree-3
| # time tar xf tree.tar -C tree-3
| 
| real    0m14.976s
| user    0m0.156s
| sys     0m1.628s
| 
| real    0m14.253s
| user    0m0.176s
| sys     0m1.632s
| 
| real    0m14.700s
| user    0m0.196s
| sys     0m1.464s


The caching numbers do NOT include things like 'yum upgrade' operations
but with yum-2.4 and up-to-date cache-images, this will add perhaps 5-10
seconds.




Enrico