[Pulp-list] Task Failed:Importer indicated a failed response
Tatiana Tereshchenko
ttereshc at redhat.com
Wed Jan 2 12:25:28 UTC 2019
Hi Venkataramana,
It seems like you are running into this issue
https://pulp.plan.io/issues/2540.
You probably have a very large repo and you probably run your Pulp 2.11 on
RHEL 6 with MongoDB 2.4 which is very old and has certain issues.
See the suggested way of upgrading MongoDB to 2.6 (if you are on RHEL 6)
https://pulp.plan.io/issues/2540#note-14.
This should solve your problem if my assumptions above are correct.
Though I encourage you to upgrade to the latest RHEL 7 and to the latest
Pulp 2.18 and MongoDB 3.4.
Let me know if your problem is solved or if you have any questions,
Tanya
On Tue, Jan 1, 2019 at 8:50 PM Venkataramana Bora <venkbora at in.ibm.com>
wrote:
> Team, here is /var/log/messages error getting while trying to sync .
> Please help.
>
>
>
> Jan 1 10:10:07 dpa-pulpslv-01 puppet-agent[3657]: Caught TERM; exiting
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.beat:INFO: Scheduler: Sending
> due task download_deferred_content
> (pulp.server.controllers.repository.queue_download_deferred)
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.strategy:INFO: Received
> task:
> pulp.server.controllers.repository.queue_download_deferred[c2f818fd-c871-4538-a091-5451e32885cf]
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: kombu.transport.qpid:INFO: Connected
> to qpid with SASL mechanism ANONYMOUS
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.strategy:INFO: Received
> task:
> pulp.server.controllers.repository.download_deferred[1f04ce4f-c093-43a3-80bb-4f71b6ab8695]
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.job:INFO: Task
> pulp.server.controllers.repository.queue_download_deferred[c2f818fd-c871-4538-a091-5451e32885cf]
> succeeded in 0.060901439s: None
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.job:INFO: Task
> pulp.server.controllers.repository.download_deferred[1f04ce4f-c093-43a3-80bb-4f71b6ab8695]
> succeeded in 0.130910374s: None
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) command
> SON([('mapreduce', u'units_rpm'), ('map', Code("\n function () {\n var
> key_fields = [this.name, this.epoch, this.version, this.release,
> this.arch]\n emit(key_fields.join('-'), {ids: [this._id]});\n }\n ", {})),
> ('reduce', Code("\n function (key, values) {\n // collect mapped values
> into the first value to build the list of ids for this key/nevra\n var
> collector = values[0]\n // since collector is values[0] start this loop at
> index 1\n // reduce isn't called if map only emits one result for key,\n //
> so there is at least one value to collect\n for (var i = 1; i <
> values.length; i++) {\n collector.ids =
> collector.ids.concat(values[i].ids)\n }\n return collector\n }\n ", {})),
> ('out', {'inline': 1}), ('query', {}), ('finalize', Code("\n function (key,
> reduced) {\n if (reduced.ids.length > 1) {\n return reduced;\n }\n // if
> there's only one value after reduction, this key is useless\n // undefined
> is implicitly returned here, which saves space\n }\n ", {}))]) on namespace
> pulp_database.$cmd failed: exception: BSONObj size: 17162604 (0x6CE10501)
> is invalid. Size must be between 0 and 16793600(16MB) First element: 0: {
> _id: "0ad-0-0.0.22-1.el7-x86_64", value: null }
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) Traceback (most
> recent call last):
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py",
> line 296, in run
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528)
> purge.remove_repo_duplicate_nevra(self.conduit.repo_id)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/purge.py",
> line 304, in remove_repo_duplicate_nevra
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) for unit_ids in
> _duplicate_key_id_generator(unit_type):
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/purge.py",
> line 493, in _duplicate_key_id_generator_mapreduce
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) for reduced in
> reduced_units:
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py", line 1157,
> in map_reduce
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) map_f, reduce_f,
> **mr_args)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 1903, in
> inline_map_reduce
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) res =
> self._command(sock_info, cmd, slave_ok)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 205, in
> _command
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528)
> read_concern=read_concern)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/pool.py", line 211, in command
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) read_concern)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/network.py", line 100, in
> command
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528)
> helpers._check_command_response(response_doc, msg, allowable_errors)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/helpers.py", line 196, in
> _check_command_response
>
>
>
>
>
> Sincerely,
> Venkataramana Bora
> IBM Visakha Hills
> Visakhapatnam – 530 045, India
> Direct:+91.891-4499606 / VoIP:92-29207
> Mob:+91.9949358804
>
> [image: Inactive hide details for pulp-list-request---01/01/2019 10:02:18
> PM---Send Pulp-list mailing list submissions to pulp-list at re]pulp-list-request---01/01/2019
> 10:02:18 PM---Send Pulp-list mailing list submissions to
> pulp-list at redhat.com
>
> From: pulp-list-request at redhat.com
> To: pulp-list at redhat.com
> Date: 01/01/2019 10:02 PM
> Subject: Pulp-list Digest, Vol 110, Issue 1
> Sent by: pulp-list-bounces at redhat.com
> ------------------------------
>
>
>
> Send Pulp-list mailing list submissions to
> pulp-list at redhat.com
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://www.redhat.com/mailman/listinfo/pulp-list
> or, via email, send a message with subject or body 'help' to
> pulp-list-request at redhat.com
>
> You can reach the person managing the list at
> pulp-list-owner at redhat.com
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Pulp-list digest..."
>
>
> Today's Topics:
>
> 1. Task Failed:Importer indicated a failed response
> (Venkataramana Bora)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 1 Jan 2019 22:00:58 +0530
> From: "Venkataramana Bora" <venkbora at in.ibm.com>
> To: pulp-list at redhat.com
> Subject: [Pulp-list] Task Failed:Importer indicated a failed response
> Message-ID:
> <
> OFD49419D1.B0C5E372-ON65258375.00586335-65258375.005ABA25 at notes.na.collabserv.com
> >
>
> Content-Type: text/plain; charset="utf-8"
>
>
> Hi Team, can some some one kindly help me with Pulp sync error "Task
> Failed Importer indicated a failed response ? Also pasted here some lines
> of /var/log/messages. Please let me know if you need any further info to
> trouble shoot on this.
> This is Pulp slave server with version : pulp-server-2.11.2-1.el6. syncs
> with master Pulp server 2.16. It was working till few days ago with out any
> issues.
>
> Pulp sync error:
>
> [root at dpa-pulpslv-01 ~]# pulp-admin rpm repo sync run
> --repo-id=rhel-6-server-rpms-x86_64
> +----------------------------------------------------------------------+
> Synchronizing Repository [rhel-6-server-rpms-x86_64]
> +----------------------------------------------------------------------+
>
> This command may be exited via ctrl+c without affecting the request.
>
>
> Downloading metadata...
> [/]
> ... completed
>
> Downloading repository content...
> [-]
> [==================================================] 100%
> RPMs: 0/0 items
> Delta RPMs: 0/0 items
>
> ... completed
>
> Downloading distribution files...
> [==================================================] 100%
> Distributions: 0/0 items
> ... completed
>
> Importing package groups/categories...
> [|]
> ... completed
>
> Cleaning duplicate packages...
> [\]
> ... failed
>
>
> Task Failed
>
> Importer indicated a failed response
>
> [root at dpa-pulpslv-01 ~]#
>
>
> /var/log/messages:
>
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.beat:INFO: Scheduler: Sending
> due task download_deferred_content
> (pulp.server.controllers.repository.queue_download_deferred)
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.strategy:INFO: Received
> task: pulp.server.controllers.repository.queue_download_deferred
> [c2f818fd-c871-4538-a091-5451e32885cf]
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: kombu.transport.qpid:INFO: Connected
> to qpid with SASL mechanism ANONYMOUS
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.strategy:INFO: Received
> task: pulp.server.controllers.repository.download_deferred
> [1f04ce4f-c093-43a3-80bb-4f71b6ab8695]
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.job:INFO: Task
> pulp.server.controllers.repository.queue_download_deferred
> [c2f818fd-c871-4538-a091-5451e32885cf] succeeded in 0.060901439s: None
> Jan 1 10:11:27 dpa-pulpslv-01 pulp: celery.worker.job:INFO: Task
> pulp.server.controllers.repository.download_deferred
> [1f04ce4f-c093-43a3-80bb-4f71b6ab8695] succeeded in 0.130910374s: None
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) command SON
> ([('mapreduce', u'units_rpm'), ('map', Code("\n function () {\n
> var key_fields = [this.name, this.epoch, this.version, this.release,
> this.arch]\n emit(key_fields.join('-'), {ids: [this._id]});\n }\n
> ", {})), ('reduce', Code("\n function (key, values) {\n // collect
> mapped values into the first value to build the list of ids for this
> key/nevra\n var collector = values[0]\n // since collector is
> values[0] start this loop at index 1\n // reduce isn't called if map
> only emits one result for key,\n // so there is at least one value to
> collect\n for (var i = 1; i < values.length; i++) {\n
> collector.ids = collector.ids.concat(values[i].ids)\n }\n return
> collector\n }\n ", {})), ('out', {'inline': 1}), ('query', {}),
> ('finalize', Code("\n function (key, reduced) {\n if
> (reduced.ids.length > 1) {\n return
> reduced;\n }\n // if there's only one value after reduction,
> this key is useless\n // undefined is implicitly returned here,
> which saves space\n }\n ", {}))]) on namespace pulp_database.$cmd
> failed: exception: BSONObj size: 17162604 (0x6CE10501) is invalid. Size
> must be between 0 and 16793600(16MB) First element: 0: { _id:
> "0ad-0-0.0.22-1.el7-x86_64", value: null }
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) Traceback (most
> recent call last):
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py",
> line 296, in run
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528)
> purge.remove_repo_duplicate_nevra(self.conduit.repo_id)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/purge.py",
> line 304, in remove_repo_duplicate_nevra
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) for unit_ids in
> _duplicate_key_id_generator(unit_type):
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/purge.py",
> line 493, in _duplicate_key_id_generator_mapreduce
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) for reduced in
> reduced_units:
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py", line 1157,
> in map_reduce
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) map_f,
> reduce_f, **mr_args)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 1903, in
> inline_map_reduce
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) res =
> self._command(sock_info, cmd, slave_ok)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 205, in
> _command
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528)
> read_concern=read_concern)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/pool.py", line 211, in command
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) read_concern)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/network.py", line 100, in
> command
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528)
> helpers._check_command_response(response_doc, msg, allowable_errors)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) File
> "/usr/lib64/python2.6/site-packages/pymongo/helpers.py", line 196, in
> _check_command_response
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) raise
> OperationFailure(msg %% errmsg, code, response)
> Jan 1 10:18:48 dpa-pulpslv-01 pulp:
> pulp_rpm.plugins.importers.yum.sync:ERROR: (3304-06528) OperationFailure:
> command SON([('mapreduce', u'units_rpm'), ('map', Code("\n function ()
> {\n var key_fields = [this.name, this.epoch, this.version,
> this.release, this.arch]\n emit(key_fields.join('-'), {ids:
> [this._id]});\n }\n ", {})), ('reduce', Code("\n function (key,
> values) {\n // collect mapped values into the first value to build the
> list of ids for this key/nevra\n var collector = values[0]\n //
> since collector is values[0] start this loop at index 1\n // reduce
> isn't called if map only emits one result for key,\n // so there is at
> least one value to collect\n for (var i = 1; i < values.length; i++)
> {\n collector.ids = collector.ids.concat(values[i].ids)\n }\n
> return collector\n }\n ", {})), ('out', {'inline': 1}), ('query',
> {}), ('finalize', Code("\n function (key, reduced) {\n if
> (reduced.ids.length > 1) {\n return
> reduced;\n }\n // if there's only one value after reduction,
> this key is useless\n // undefined is implicitly returned here,
> which saves space\n }\n ", {}))]) on namespace pulp_database.$cmd
> failed: exception: BSONObj size: 17162604 (0x6CE10501) is invalid. Size
> must be between 0 and 16793600(16MB) First element: 0: { _id:
> "0ad-0-0.0.22-1.el7-x86_64", value: null }
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: pulp.server.async.tasks:INFO: Task
> failed : [6daefad5-b4a2-4a30-acbc-e6a2cd4dd697]
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> Task pulp.server.managers.repo.sync.sync
> [6daefad5-b4a2-4a30-acbc-e6a2cd4dd697] raised unexpected:
> PulpExecutionException('Importer indicated a failed response',)
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> Traceback (most recent call last):
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in
> trace_task
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> R = retval = fun(*args, **kwargs)
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line
> 491, in __call__
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> return super(Task, self).__call__(*args, **kwargs)
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line
> 103, in __call__
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> return super(PulpTask, self).__call__(*args, **kwargs)
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in
> __protected_call__
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> return self.run(*args, **kwargs)
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> File
> "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py",
> line 820, in sync
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed
> response'))
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:ERROR: (3221-06528)
> PulpExecutionException: Importer indicated a failed response
> Jan 1 10:18:49 dpa-pulpslv-01 pulp: celery.worker.job:INFO: Task
> pulp.server.async.tasks._release_resource
> [1850458c-9817-40d2-b295-d1ad1198269d] succeeded in 0.00577054099995s: None
> ~
>
>
>
>
>
>
> Sincerely,
> Venkataramana Bora
> IBM Visakha Hills
> Visakhapatnam ? 530 045, India
> Direct:+91.891-4499606 / VoIP:92-29207
> Mob:+91.9949358804
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://www.redhat.com/archives/pulp-list/attachments/20190101/fa070532/attachment.html
> >
>
> ------------------------------
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
> End of Pulp-list Digest, Vol 110, Issue 1
> *****************************************
>
>
>
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190102/338076b8/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190102/338076b8/attachment.gif>
More information about the Pulp-list
mailing list