interesting timings.. specially interesting to see that pyrpm is alot faster than the yum for fc7 and the yum used in PRE fc7.. yum (pre fc7) vs yum fc7 doesn't realy seem to be much of a speed improvement.. some parts are faster but some parts are slower. funny to see how those 3 versions differ in speed i still thinks that a pure c or c++ implementation in combination with mysql will be the fastest.. downside of that is that python and the repository data itself will have to be rebuilded.. and RPM itself will most likely have to get some vital updates.. like bringing in logic in versioning. o well.. that will take time till it gets done i guess. <div>2007/5/15, Florian Festi <<a href="mailto:ffesti@redhat.com">ffesti@redhat.com</a>>:<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Hi! Yum traditionally used the rpmlib to resolve the dependecies of the packages being installed/removed/updated. This has always restricted yum to very simple algorithms as the interface of the rpmlib doesn't allow much control. The current yum - that will be shipped with F7 - has now replaced the rpmlib by a Python only resolver. I did some test runs to find out the current state of development and to see if the new resolver is an improvement over the rpmlib. The candidates: =============== * yum-3.0.3 as used in FC6 * yum from CVS head (2007/05/07), which is very similar to the yum in F7 * pyrpmyum CVS head (2007/05/07) - a pure Python implementation developed for testing purposed [1]. The test cases: =============== Tests are based in FC-5 run on FC-6.x86_64. All operations are aborted by "pressing n". Installs: Try to install into an empty build root. There is a big number of excluded pkgs to make the install '*' work. * Normal install: several groups, 856 Pkgs * Full install: "*", 6057 Pkgs * Install "[a-k]*", 3052 Pkgs - needs to drag in a lot of pkgs to satisfy dependencies Updates: Before the Updates the "Normal Install" is run and the Pkgs are installed into the build root. * Empty update * Update libgnome: 1 Pkg * Big update: 346u+12i Pkgs * Dist upgrade: Enable FC6 repos and do an upgrade; fails for both yum versions,so don't take results too serious Erase: * glibc: 828 Pkgs Warm ups: Do a install '*'/upgrade to build the caches. Each program has different issues with that. yum-3.0.3 needs to download the rpm headers (Test was run with repos in a NFS mount). All need to build the sqlite databases. pyrpmyum currently uses a Python only implementation that uses a lot of time. Warm up operations fail for all programs. This all make the results of the warm ups a bit unreliable and questionable. They are included for completeness. Measured Values: ================ All data is from just one test run - so don't expect too high precision. I put 3 different test results into tables: * Real time passed during the test * Heap peak - maximum of Memory used * Heap total - amount of heap memory allocated The last one gives an idea of how much data has been moved around. Conclusions: ============ Please read the data tables first to get your own impression. Memory usage: The new yum saves a lot of memory for big operations (about 50%). The comparison to pyrpmyum shows that it is possible to save even more memory and to drop memory usage below 100MB for all cases. In pyrpmyum this is achieved with dynamic loading of rpm tags. Some are not even kept in the pkgs but just cached in a least recently used list. For really small updates the old 3.0.3 yum wins. I guess this is because it does not do any obsolete handling. Runtime: Although the new yum save the very costly rpm header downloads (which take nearly 3 minutes for the whole repository) it is still significantly slower for big operations. Things may look a bit different in real world scenarios where downloading the headers is even more expensive. Nevertheless the performance of the current yum is poor. Runtime behavior has dropped from O(#prco * #rpmlib_calls) to O(#prco^2) (prco = Provides, Requires, Conflicts, Obsoletes). The results of pyrpmyum show that is it possible to do the depsolving in O(#prco) (Probably just O(#prco * log #prco) but that doesn't matter much). There are already plans how to get rid of the quadratic behavior and I hope we can fix these issues before Fedora 8. Have fun Florian Festi [1] <a href="https://hosted.fedoraproject.org/projects/pyrpm">https://hosted.fedoraproject.org/projects/pyrpm</a> Heap peak (MB) Operation | 3.0.3 | 05/07 | pyrpm ------------------------------------------------ Warmup: Full install | 475.3 | 226.6 | 128.6 Resolve normal install | 94.6 | 74.1 | 27.0 Resolve full install | 462.4 | 218.5 | 59.8 Resolve install [a-k]* | 278.5 | 137.8 | 52.0 Resolve empty update | 25.9 | 52.9 | 45.2 Resolve update libgnome | 30.3 | 57.0 | 48.8 Resolve big update | 112.5 | 108.2 | 60.3 Resolve erase glibc | 210.0 | 99.8 | 39.6 Warmup: Dist upgrade | 162.7 | 107.3 | 112.0 Resolve dist upgrade | 162.4 | 107.3 | 62.9 Heap total (MB) This is the total amount of memory requested. The amount of memory being allocated at one time is much less! Operation | 3.0.3 | 05/07 | pyrpm ----------------------------------------------- Warmup: Full install | 20003 | 90859 | 10829 Resolve normal install | 2088 | 9448 | 639 Resolve full install | 15402 | 84951 | 4987 Resolve install [a-k]* | 17542 | 24805 | 4121 Resolve empty update | 167 | 80 | 747 Resolve update libgnome | 282 | 136 | 1247 Resolve big update | 3497 | 2699 | 2574 Resolve erase glibc | 1732 | 2192 | 3369 Warmup: Dist upgrade | 6203 | 2644 | 7395 Resolve dist upgrade | 5869 | 2644 | 2154 Real time (s) Operation | 3.0.3 | 07/05/07| pyrpm ---------------------------------------------------- Warmup: Full install | 653.51 | 1064.35 | 147.23 Resolve normal install | 40.59 | 134.11 | 8.04 Resolve full install | 477.67 | 1093.57 | 51.01 Resolve install [a-k]* | 313.56 | 887.40 | 33.52 Resolve empty update | 6.91 | 6.57 | 13.36 Resolve update libgnome | 3.34 | 2.57 | 7.51 Resolve big update | 33.04 | 45.56 | 24.81 Resolve erase glibc | 29.83 | 37.72 | 30.95 Warmup: Dist upgrade | 81.42 | 40.67 | 108.50 Resolve dist upgrade | 52.07 | 40.29 | 27.20 -- fedora-devel-list mailing list <a href="mailto:fedora-devel-list@redhat.com">fedora-devel-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/fedora-devel-list">https://www.redhat.com/mailman/listinfo/fedora-devel-list</a> </blockquote> </div>