[Pulp-dev] Pulp3 Concurrency testing

[David Davis]

On Tue, Jul 28, 2020 at 12:03 PM Justin Sherrill <jsherril at redhat.com>
wrote:

>
> On 7/28/20 11:44 AM, David Davis wrote:
>
> Today we discussed this at triage. We're leaning towards changing the
> default from 20 to 10 as it seems like 10 only incurs an extra 30% penalty
> in time while seeming to fix the problem[0].
>
> One question though is how we should treat existing data because most
> Remotes at this point probably have a value of 20 for download_concurrency.
> We came up with two options that we would like some feedback on.
>
> It seems kinda strange that the 'default' value is recorded in the db on
> the remote at creation time.  Any reason to just leave it 'nil' and use the
> 'app default' ?   This doesn't really help with past values, but seems like
> a better model going forward.
>

I think this maybe makes sense given that we'll potentially have to change
it again if we find out that 10 doesn't work for some reason. Perhaps we
could set a class constant on Remote that subclasses could override.

>
>
> # Option 1: Migrate 20 to 10
>
> This would be a migration in pulpcore that would update
> download_concurrency to 10 for all Remotes whose download_concurrency is
> set to 10. Something like:
>
>
>   Remote.objects.all().filter(download_concurrency=20).update(download_concurrency=10)
>
> My vote is for #1 because:
>
> a) I imagine ~99% of people want the recommended default, so 99% of people
> will either need to update this manually (if doing #2) or face sync failures
>
> b) i'd rather 99% of people not have to do anything than the 1% that for
> some reason actually wanted 20 and weren't just going with the 'recommended
> default' at that time.  The number of people in this case could very well
> be zero.
>
>
>
> # Option 2: Documentation
>
> This would be similar to the migration approach but instead of modifying
> our users' data, we'd document how they could do it themselves. So
> something like:
>
>     pulpcore-manager shell_plus -c
> "Remote.objects.all().filter(download_concurrency=20).update(download_concurrency=10)
>
>
> Any feedback is welcome.
>
> [0] https://pulp.plan.io/issues/7186#note-2
>
> David
>
>
> On Mon, Jul 27, 2020 at 2:57 PM Grant Gainey <ggainey at redhat.com> wrote:
>
>> Hey folks,
>>
>> Looking into issue 7212 <https://pulp.plan.io/issues/7212> , over the
>> weekend I did some ad-hoc evaluations of sync-performance at various
>> concurrency settings. I wrote up my observations here:
>>
>> https://hackmd.io/@ggainey/pulp3_sync_concurrency
>>
>> Just thought folk might be interested.
>>
>> G
>> --
>> Grant Gainey
>> Principal Software Engineer, Red Hat System Management Engineering
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
> _______________________________________________
> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200728/7732f10d/attachment.htm>