[Pulp-list] Sometimes 'purge_duplicates' takes an abnormal amount of time
ehelms at redhat.com
Tue Nov 22 01:02:11 UTC 2016
I'm not sure if this is related, but this error popped up:
error: 'Cursor not found, cursor id: 156504476941'
On Mon, Nov 21, 2016 at 6:00 PM, Sean Myers <sean.myers at redhat.com> wrote:
> On 11/21/2016 05:52 PM, Eric Helms wrote:
> > On Mon, Nov 21, 2016 at 4:03 PM, Sean Myers <sean.myers at redhat.com>
> >> On 11/21/2016 03:29 PM, Eric Helms wrote:
> >>> Hi,
> >>> I've noticed that at times, during a sync, the 'purge_duplicates' step
> >>> seems to take what I would consider an abnormal amount of time. A sync
> >> that
> >>> usually takes 5 or so minutes can suddenly take an hour and it seems to
> >>> just sit at this particular step. Since Pulp progress reports do not
> >>> contain timing data  I cannot give hard numbers on this. Is there
> >>> anything that might explain this behavior? Is this a valid or known
> >> I
> >>> am happy to provide as much data and information as I can because.
> >> I don't know of issues where this step just pauses.
> >> What version(s) of pulp/mongodb is this running on? What
> > 2.8.7 on mongodb 2.6.11
> >> Is there a particular feed that reliably causes this?
> > I wish. This does not happen all of the time which makes it hard to zero
> > the problem. We experience high IO time at some points, and I have
> > that during these times when we do large queries to Pulp and thus mongodb
> > we run into all things slowing down. I am working to get some better hard
> > evidence by adding cProfiles.
> cProfile stuff would be gravy, but I suspect that a pulp developer with a
> fresh take on this would be able to take my work from the original effort
> to speed up the purging of duplicate nevra and find some obvious things to
> optimization. This is a new Redmine issue that should be filed in Pulp's
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pulp-list