[Pulp-list] yum importer failed after upgrade to 2.16 OR missing .treeinfo for EPEL

Jim Davis JDavis at douglas.co.us
Wed Apr 11 14:28:27 UTC 2018 

The error may not be related - just a coincidence - but the failures I see happened after I upgraded to 2.16 and forced a sync.
The two errors for a missing .treeinfo or treeinfo may be the cause - I'm not sure. This particular problem seems to have happened in the past (surfing pulp.plan.io).
I've deleted and rebuilt the repo using the same config as I have for a long time - same error. Has anyone else had problems mirroring EPEL reciently?

Simple setup - mirror CentOS 6 and 7 repos. Both EPEL repos fail sync with the same error. Repo built with:

pulp-admin rpm repo create \
        --repo-id=epel-7-x86_64 \
        --description 'CentOS 7 EPEL Repo' \
        --display-name 'CentOS 7 EPEL Repo' \
        --feed=https://dl.fedoraproject.org/pub/epel/7/x86_64/ \
        --gpg-key=/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

# pulp-admin -v rpm repo sync run --repo-id epel-7-x86_64
+----------------------------------------------------------------------+
                Synchronizing Repository [epel-7-x86_64]
+----------------------------------------------------------------------+

A sync task is already in progress for this repository. Its progress will be
tracked below.

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items

Task Failed

Importer indicated a failed response

====================================================================================
Detail on repo:

# pulp-admin rpm repo list --details  --repo-id=epel-7-x86_64
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+

Id:                   epel-7-x86_64
Display Name:         CentOS 7 EPEL Repo
Description:          CentOS 7 EPEL Repo
Content Unit Counts:
  Drpm: 38
  Rpm:  12469
Notes:
Scratchpad:
  Checksum Type: sha256
Importers:
  Config:
    Feed: https://dl.fedoraproject.org/pub/epel/7/x86_64/
  Id:                   yum_importer
  Importer Type Id:     yum_importer
  Last Override Config:
  Last Sync:            2018-04-11T14:01:47Z
  Last Updated:         2018-04-10T21:40:42Z
  Repo Id:              epel-7-x86_64
  Scratchpad:           None
Distributors:
  Auto Publish:         True
  Config:
    Gpgkey:       -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.11
                  (GNU/Linux)
                  mQINBFKuaIQBEAC1UphXwMqCAarPUH/ZsOFslabeTVO2pDk5YnO96f+rgZB7xA
                  rB
                  OSeQk7B90iqSJ85/c72OAn4OXYvT63gfCeXpJs5M7emXkPsNQWWSju99lW+AqS
                  Nm
                  jYWhmRlLRGl0OO7gIwj776dIXvcMNFlzSPj00N2xAqjMbjlnV2n2abAE5gq6Vp
                  qP
                  vFXVyfrVa/ualogDVmf6h2t4Rdpifq8qTHsHFU3xpCz+T6/dGWKGQ42ZQfTaLn
                  DM
                  jToAsmY0AyevkIbX6iZVtzGvanYpPcWW4X0RDPcpqfFNZk643xI4lsZ+Y2Er9Y
                  u5
                  S/8x0ly+tmmIokaE0wwbdUu740YTZjCesroYWiRg5zuQ2xfKxJoV5E+Eh+tYwG
                  DJ
                  n6HfWhRgnudRRwvuJ45ztYVtKulKw8QQpd2STWrcQQDJaRWmnMooX/PATTjCBE
                  xB
                  9dkz38Druvk7IkHMtsIqlkAOQMdsX1d3Tov6BE2XDjIG0zFxLduJGbVwc/6rIc
                  95
                  T055j36Ez0HrjxdpTGOOHxRqMK5m9flFbaxxtDnS7w77WqzW7HjFrD0VeTx2vn
                  jj
                  GqchHEQpfDpFOzb8LTFhgYidyRNUflQY35WLOzLNV+pV3eQ3Jg11UFwelSNLqf
                  Qf
                  uFRGc+zcwkNjHh5yPvm9odR1BIfqJ6sKGPGbtPNXo7ERMRypWyRz0zi0twARAQ
                  AB
                  tChGZWRvcmEgRVBFTCAoNykgPGVwZWxAZmVkb3JhcHJvamVjdC5vcmc+iQI4BB
                  MB
                  AgAiBQJSrmiEAhsPBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRBqL66iNS
                  xk
                  5cfGD/4spqpsTjtDM7qpytKLHKruZtvuWiqt5RfvT9ww9GUUFMZ4ZZGX4nUXg4
                  9q
                  ixDLayWR8ddG/s5kyOi3C0uX/6inzaYyRg+Bh70brqKUK14F1BrrPi29eaKfG+
                  Gu
                  MFtXdBG2a7OtPmw3yuKmq9Epv6B0mP6E5KSdvSRSqJWtGcA6wRS/wDzXJENHp5
                  re
                  9Ism3CYydpy0GLRA5wo4fPB5uLdUhLEUDvh2KK//fMjja3o0L+SNz8N0aDZyn5
                  Ax
                  CU9RB3EHcTecFgoy5umRj99BZrebR1NO+4gBrivIfdvD4fJNfNBHXwhSH9ACGC
                  Nv
                  HnXVjHQF9iHWApKkRIeh8Fr2n5dtfJEF7SEX8GbX7FbsWo29kXMrVgNqHNyDnf
                  AB
                  VoPubgQdtJZJkVZAkaHrMu8AytwT62Q4eNqmJI1aWbZQNI5jWYqc6RKuCK6/F9
                  9q
                  thFT9gJO17+yRuL6Uv2/vgzVR1RGdwVLKwlUjGPAjYflpCQwWMAASxiv9uPyYP
                  Hc
                  ErSrbRG0wjIfAR3vus1OSOx3xZHZpXFfmQTsDP7zVROLzV98R3JwFAxJ4/xqeO
                  N4
                  vCPFU6OsT3lWQ8w7il5ohY95wmujfr6lk89kEzJdOTzcn7DBbUru33CQMGKZ3E
                  vt RjsC7FDbL017qxS+ZVA/HGkyfiu4cpgV8VUnbql5eAZ+1Ll6Dw== =hdPa
                  -----END PGP PUBLIC KEY BLOCK-----
    Http:         False
    Https:        True
    Relative URL: pub/epel/7/x86_64/
  Distributor Type Id:  yum_distributor
  Id:                   yum_distributor
  Last Override Config:
  Last Publish:         None
  Last Updated:         2018-04-10T21:40:42Z
  Repo Id:              epel-7-x86_64
  Scratchpad:
  Auto Publish:         False
  Config:
    Http:         False
    Https:        True
    Relative URL: pub/epel/7/x86_64/
  Distributor Type Id:  export_distributor
  Id:                   export_distributor
  Last Override Config:
  Last Publish:         None
  Last Updated:         2018-04-10T21:40:42Z
  Repo Id:              epel-7-x86_64
  Scratchpad:
====================================================================================
Log:
Apr 11 08:07:29 prpulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[22de3934-a360-4160-9402-76b5bb7947a0]
Apr 11 08:07:29 prpulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[b4829e41-257b-46f5-8624-d17b9077621a]
Apr 11 08:07:29 prpulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[b95d49af-f70c-40a7-9406-6db61cec2327]
Apr 11 08:07:29 prpulp pulp: celery.app.trace:INFO: [22de3934] Task pulp.server.async.tasks._queue_reserved_task[22de3934-a360-4160-9402-76b5bb7947a0] succeeded in 0.0248043539468s: None
Apr 11 08:07:29 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading metadata from https://dl.fedoraproject.org/pub/epel/7/x86_64/.
Apr 11 08:07:29 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org
Apr 11 08:07:44 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/repomd.xml.
Apr 11 08:07:44 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Parsing metadata.
Apr 11 08:07:44 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading metadata from https://dl.fedoraproject.org/pub/epel/7/x86_64/.
Apr 11 08:07:44 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org
Apr 11 08:07:44 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/repomd.xml.
Apr 11 08:07:45 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Parsing metadata.
Apr 11 08:07:45 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading metadata from https://dl.fedoraproject.org/pub/epel/7/x86_64/.
Apr 11 08:07:45 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org
Apr 11 08:07:45 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/repomd.xml.
Apr 11 08:07:46 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Parsing metadata.
Apr 11 08:07:46 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading metadata from https://dl.fedoraproject.org/pub/epel/7/x86_64/.
Apr 11 08:07:46 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org
Apr 11 08:07:46 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/repomd.xml.
Apr 11 08:07:47 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Parsing metadata.
Apr 11 08:07:47 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading metadata files.
Apr 11 08:07:47 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): dl.fedoraproject.org
Apr 11 08:07:47 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (3): dl.fedoraproject.org
Apr 11 08:07:47 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (4): dl.fedoraproject.org
Apr 11 08:07:47 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (5): dl.fedoraproject.org
Apr 11 08:07:48 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/3fe3f147762ba75e292e52f3a9cf163cb5c3ee2ede52bfb0caa2fdbb0a8114bc-comps-Everything.x86_64.xml.
Apr 11 08:07:48 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/3e3bf72827ce3cbe2381f1cac087f6ff23b8bb8c2bdab06598f18057209e423e-updateinfo.xml.bz2.
Apr 11 08:07:48 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/a2117dc14c5dc990849f0a2c97535a0e51f9d1fd3bbe1680fc17691ec75ea53e-primary.xml.gz.
Apr 11 08:07:49 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/f0375c4a152043f649e8209c7c1ee514671910af7c5ca8a0f83523b4c76e91c5-other.xml.gz.
Apr 11 08:08:02 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/c1dccb1092bc1e5bae18f863318d29abbee1e5cbe63fe978e46cd3fe289ee702-prestodelta.xml.gz.
Apr 11 08:08:02 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/f2f5ddd4101a588a7c6eb60e20c45756b87e8123994fe1b3f0116a57ef74ce89-comps-Everything.x86_64.xml.gz.
Apr 11 08:08:03 prpulp pulp: nectar.downloaders.threaded:INFO: Download succeeded: https://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/68edb06b76563042026f82da873a639c216007baaadfa8e7e743757c1d452116-filelists.xml.gz.
Apr 11 08:08:03 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Generating metadata databases.
Apr 11 08:08:36 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Determining which units need to be downloaded.
Apr 11 08:10:07 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading 0 RPMs.
Apr 11 08:10:27 prpulp pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: [b4829e41] The content container reported: {'downloads': {}, 'total_sources': 0} for base URL: https://dl.fedoraproject.org/pub/epel/7/x86_64/
Apr 11 08:10:27 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading 0 DRPMs.
Apr 11 08:10:28 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [b4829e41] Downloading additional units.
Apr 11 08:10:28 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org
Apr 11 08:10:28 prpulp pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://dl.fedoraproject.org/pub/epel/7/x86_64/.treeinfo failed with code 404: Not Found
Apr 11 08:10:29 prpulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org
Apr 11 08:10:29 prpulp pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://dl.fedoraproject.org/pub/epel/7/x86_64/treeinfo failed with code 404: Not Found
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248) invalid literal for int() with base 10: '2.0'
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248) Traceback (most recent call last):
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 283, in run
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)     self.get_errata(metadata_files)
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 867, in get_errata
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)     updateinfo.process_package_element, additive_type=True)
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 951, in save_fileless_units
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)     errata_controller.create_or_update_pkglist(model, self.repo.repo_id)
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/controllers/errata.py", line 25, in create_or_update_pkglist
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)     if not existing_erratum or existing_erratum.update_needed(new_erratum):
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 1247, in update_needed
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)     return util.is_version_newer(other.version, self.version)
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp_rpm/yum_plugin/util.py", line 208, in is_version_newer
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248)     new = int(new_version)
Apr 11 08:10:30 prpulp pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [b4829e41] (1039-41248) ValueError: invalid literal for int() with base 10: '2.0'
Apr 11 08:10:30 prpulp pulp: pulp.server.async.tasks:INFO: [b4829e41] Task failed : [b4829e41-257b-46f5-8624-d17b9077621a]
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248) Task pulp.server.managers.repo.sync.sync[b4829e41-257b-46f5-8624-d17b9077621a] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248) Traceback (most recent call last):
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)     R = retval = fun(*args, **kwargs)
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)     return super(Task, self).__call__(*args, **kwargs)
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)     return super(PulpTask, self).__call__(*args, **kwargs)
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)     return self.run(*args, **kwargs)
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 827, in sync
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Apr 11 08:10:30 prpulp pulp: celery.app.trace:ERROR: [b4829e41] (1039-41248) PulpExecutionException: Importer indicated a failed response
Apr 11 08:10:30 prpulp pulp: celery.app.trace:INFO: [b95d49af] Task pulp.server.async.tasks._release_resource[b95d49af-f70c-40a7-9406-6db61cec2327] succeeded in 0.00261277402751s: None

Thanks

Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20180411/bd7ebb9c/attachment.htm>

More information about the Pulp-list mailing list