[Pulp-list] pulp v1 vs pulp v2 rpm repo sync times

[Mike McCune]

On 02/25/2013 12:09 PM, Bryan Kearney wrote:
> On 02/25/2013 03:03 PM, Jay Dobies wrote:
>>> To give an example of a larger repo with latency, the RHEL 6Server
>>> x86_64 repo:
>>>
>>> Pulp V1 (i believe 4 threads):  108 minutes
>>> Pulp V2 with 4 threads: 192 minutes
>>> Pulp V2 with 1 thread:  300 minutes
>>>
>>> -Justin
>>
>> Few points...
>>
>> Looks like the analysis for 4 threads v. 1 was incorrect. We'll address
>> this in some capacity for 2.1.
>>
>> The simple fact is that it's just plain going to be slower than v1 for a
>> bit. In v1, Pulp was acting largely as a web interface to a yum repo on
>> disk. In v2, the paradigm has significantly changed. Grinder, however,
>> did not change. So what we're dealing with is grinder functioning one
>> way, v2 advocating a different model, and a bunch of glue in between them.
>>
>> We've been investigating this since really the start of the year. Part
>> one was us measuring different download approaches in Python. We settled
>> on one and implemented a chunk that will handle concurrent downloads
>> smoothly. Part two is taking place this sprint as we start to use this
>> new approach to handle a yum repository instead of yum itself, which
>> admittedly has always been a bit of a square peg, circle hole situation.
>> I'm not saying the decision was wrong, but at the end of the day the new
>> approach should fit cleaner.
>>
>> Since you guys are set up to run these tests easily, can I ask you to
>> take it past 4 threads and see what you find? There has to be a place
>> where the increases are less significant than you saw from 1 to 4, but I
>> am curious just how far we should take it.
>>
>>
>
> If I do a "yum install foo" am I still hitting only the disk, and not a
> db? I want to make sure we have not introduced an known
> $PREVIOUS_PRODUCT bugs.
>
>

AFAIK, still just disk -> http server -> network -> client. That said, 
what does this have todo with syncing remote repos?  I'm guessing you 
are talking client installing packages ...

Mike