[Pulp-list] Stress Testing Script to Create Hundreds of Repos

[Brian Bouterse]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

The tasking system was improved significantly with 2.6.0, and I expect
the improvements you perceive after upgrading are due to that.

We're always looking to improve Pulp so bug reports are welcome [0].
Can you put any observations of stuck tasks with 2.6.x into a bug
report? Also, it would be good to have specific steps for an
investigator to attempt to reproduce with.

[0]:  https://pulp.plan.io/projects/pulp/issues/new

- -Brian


On 05/04/2015 08:54 AM, Ben Stanley wrote:
> 
> On 4 May 2015 10:26:53 PM Brian Bouterse <bbouters at redhat.com>
> wrote:
> 
>> Regarding the tasks getting stuck, what version of Pulp are you
>> running?
> 
> Getting stuck was on 2.5.1.
> 
> I upgraded to pulp 2.6.0 today. I had to bypass my local epel repo
> to do the upgrade, due to an issue with the errata on epel.
> 
> Pulp behaviour has improved, but that may also be due to clearing
> out the task list. I have also cleared out the task list again
> today, after starting, aborting and restarting the rhel script a
> few times due to problems. So it seems that 2.6.0 can have the
> getting stuck problem too, when there is lots in the task list.
> 
> I saw it run most of the way through that script before I left.
> However, rhel-6.7 isn't out yet, so those fail. I also observed
> that it downloaded metadata and a few rpms, even though I didn't do
> a sync. The repos were already up to date, or nearly up to date,
> but would it have done a full download just by creating or updating
> the repo? My understanding was that it should not.
> 
> When you run the script, there is a big disconnect between where
> the script is up to and what is going on in the log file. The
> script stops and waits at various times, which I don't understand.
> I thought it should just create the repo, not sync it. However, the
> log file shows syncs happening, which may explain the delay.
> 
> If you want to cause trouble, try to delete the rhel-6.7 repos
> (after they have been created) while the other repos are still
> being created. That causes a long wait. I haven't seen the
> resolution of that yet.
> 
> So, in summary, I think pulp 2.5.1 and 2.6.0 both get stuck under
> the right conditions. Perhaps the waiting message should say what
> it is waiting for, and give some real indicator of progress
> (position on the task queue, blocked resources, etc).
> 
> You may notice that I added a few entries on pulp.plan.io over the
> weekend.
> 
> Ben.
> 
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJVR3RSAAoJEK48cdELyEfyq1YIAIZq4sp/G+Op822h51Wi3erY
29W18Pm88zx62gA8JZu9+WNhddMwvz1a1ozqdf34j1zQCWgvZ3ng/WWIhI/zECOJ
WESbBwFOO4jR43mPV1T4lw7BjNZWVOcB2Fg4r4oCifBRRUROZMZS3cgEjRELABpO
bUwMoRSpUPn5IVdgJV9BMLyQnrQ8H34/cl+i/h6MUGeXlVl/YyYCTm9xvhCqomSY
uxmpATljpRPS1fSY2rbZ+Aw43Puk0RmRrO4aiEpFd+fT+z45dcW29admWBBwlb1N
oNsPvdnGwGzraHv9AOu6uLUBTKMSN7wdCk+toCWVFP5eyuYP7RC4uwpRLc0Egj0=
=7ox/
-----END PGP SIGNATURE-----