Buildsys issues

[Ville Skyttä]

On Sun, 2005-11-06 at 08:47 -0500, seth vidal wrote:
> On Sun, 2005-11-06 at 15:27 +0200, Ville Skyttä wrote:
>
> > Repository generation takes much too long.  It was a bit under 30
> > minutes for FC-4 and devel each, and 15ish for FC-3.  repoview seems to
> > be the culprit, taking 25+ minutes of the 30 mentioned above.  So,
> > totally, one full push takes roughly 1.5 hours, during which human input
> > is needed at start, between each of FC-3, FC-4, devel generation, and at
> > the end.  Very tedious.
> 
> So if we take repoview out of the process it only takes 5 minutes for
> the process?

I doubt it could be squeezed down to 5 doing only that, but something
like 10-15.  But that would be much better than ~90.

> If so then maybe we should have repoview run on
> fedoraproject.org at regular intervals to limit the need for a person to
> be in front of it.

Yep.  Also, IIRC the way repoview is currently set up on the buildsys
(or due to repoview's internals, dunno), it's unable to use any caching
and regenerates everything from scratch every time even if only one
package would be changed in a repo.

> Another option is I could look through the repoview code and see why it
> is taking so long.

I've had the intention to do that too, but have so far failed to find
time for it.  Thanks in advance ;).  One thing that could be worth
looking into is whether something in repoview or the templating engine
uses a lot of string concatenations (AFAIK strings are immutable in
Python).

A couple of years ago I was using htmltmpl for something, and changing
it to append strings into a list and joining at the end instead of
concatenating all the time made a huge difference in performance, in
particular for large pages.  See the htmltmpl package in Extras and the
-performance patch included in it if interested.