[Pulp-dev] Pulp3 Concurrency testing

[Brian Bouterse]

On Tue, Jul 28, 2020 at 12:17 PM David Davis <daviddavis at redhat.com> wrote:

> On Tue, Jul 28, 2020 at 12:03 PM Justin Sherrill <jsherril at redhat.com>
> wrote:
>
>>
>> On 7/28/20 11:44 AM, David Davis wrote:
>>
>> Today we discussed this at triage. We're leaning towards changing the
>> default from 20 to 10 as it seems like 10 only incurs an extra 30% penalty
>> in time while seeming to fix the problem[0].
>>
>> One question though is how we should treat existing data because most
>> Remotes at this point probably have a value of 20 for download_concurrency.
>> We came up with two options that we would like some feedback on.
>>
>> It seems kinda strange that the 'default' value is recorded in the db on
>> the remote at creation time.  Any reason to just leave it 'nil' and use the
>> 'app default' ?   This doesn't really help with past values, but seems like
>> a better model going forward.
>>
>
> I think this maybe makes sense given that we'll potentially have to change
> it again if we find out that 10 doesn't work for some reason. Perhaps we
> could set a class constant on Remote that subclasses could override.
>
One benefit of what we have now is that if you get a db dump, you can see
all the data right there. One downside of what we have now is that we can't
disambiguate a user intending to use the default from just receiving the
default. Overall I value the simplicity of recording actual values in the
db because it's simple, explicit, and consistent with how all other
defaults work in Pulp.


>>
>> # Option 1: Migrate 20 to 10
>>
>> This would be a migration in pulpcore that would update
>> download_concurrency to 10 for all Remotes whose download_concurrency is
>> set to 10. Something like:
>>
>>
>>   Remote.objects.all().filter(download_concurrency=20).update(download_concurrency=10)
>>
>> My vote is for #1 because:
>>
>> a) I imagine ~99% of people want the recommended default, so 99% of
>> people will either need to update this manually (if doing #2) or face sync
>> failures
>>
>> b) i'd rather 99% of people not have to do anything than the 1% that for
>> some reason actually wanted 20 and weren't just going with the 'recommended
>> default' at that time.  The number of people in this case could very well
>> be zero.
>>
>>
>>
>> # Option 2: Documentation
>>
>> This would be similar to the migration approach but instead of modifying
>> our users' data, we'd document how they could do it themselves. So
>> something like:
>>
>>     pulpcore-manager shell_plus -c
>> "Remote.objects.all().filter(download_concurrency=20).update(download_concurrency=10)
>>
>>
>> Any feedback is welcome.
>>
>> [0] https://pulp.plan.io/issues/7186#note-2
>>
>> David
>>
>>
>> On Mon, Jul 27, 2020 at 2:57 PM Grant Gainey <ggainey at redhat.com> wrote:
>>
>>> Hey folks,
>>>
>>> Looking into issue 7212 <https://pulp.plan.io/issues/7212> , over the
>>> weekend I did some ad-hoc evaluations of sync-performance at various
>>> concurrency settings. I wrote up my observations here:
>>>
>>> https://hackmd.io/@ggainey/pulp3_sync_concurrency
>>>
>>> Just thought folk might be interested.
>>>
>>> G
>>> --
>>> Grant Gainey
>>> Principal Software Engineer, Red Hat System Management Engineering
>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
>> _______________________________________________
>> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200728/029367e6/attachment.htm>