[Pulp-dev] Pulp3 Concurrency testing

[Justin Sherrill]

On 7/28/20 11:44 AM, David Davis wrote:
> Today we discussed this at triage. We're leaning towards changing the 
> default from 20 to 10 as it seems like 10 only incurs an extra 30% 
> penalty in time while seeming to fix the problem[0].
>
> One question though is how we should treat existing data because most 
> Remotes at this point probably have a value of 20 for 
> download_concurrency. We came up with two options that we would like 
> some feedback on.

It seems kinda strange that the 'default' value is recorded in the db on 
the remote at creation time.  Any reason to just leave it 'nil' and use 
the 'app default' ?   This doesn't really help with past values, but 
seems like a better model going forward.

>
>
> # Option 1: Migrate 20 to 10
>
> This would be a migration in pulpcore that would update 
> download_concurrency to 10 for all Remotes whose download_concurrency 
> is set to 10. Something like:
>
>   Remote.objects.all().filter(download_concurrency=20).update(download_concurrency=10)
>
My vote is for #1 because:

a) I imagine ~99% of people want the recommended default, so 99% of 
people will either need to update this manually (if doing #2) or face 
sync failures

b) i'd rather 99% of people not have to do anything than the 1% that for 
some reason actually wanted 20 and weren't just going with the 
'recommended default' at that time.  The number of people in this case 
could very well be zero.


>
> # Option 2: Documentation
>
> This would be similar to the migration approach but instead of 
> modifying our users' data, we'd document how they could do it 
> themselves. So something like:
>
>     pulpcore-manager shell_plus -c 
> "Remote.objects.all().filter(download_concurrency=20).update(download_concurrency=10)
>
>
> Any feedback is welcome.
>
> [0] https://pulp.plan.io/issues/7186#note-2
>
> David
>
>
> On Mon, Jul 27, 2020 at 2:57 PM Grant Gainey <ggainey at redhat.com 
> <mailto:ggainey at redhat.com>> wrote:
>
>     Hey folks,
>
>     Looking into issue 7212 <https://pulp.plan.io/issues/7212> , over
>     the weekend I did some ad-hoc evaluations of sync-performance at
>     various concurrency settings. I wrote up my observations here:
>
>     https://hackmd.io/@ggainey/pulp3_sync_concurrency
>
>     Just thought folk might be interested.
>
>     G
>     -- 
>     Grant Gainey
>     Principal Software Engineer, Red Hat System Management Engineering
>
>
>     _______________________________________________
>     Pulp-dev mailing list
>     Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>     https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200728/0d228e8a/attachment.htm>