<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style></style></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1>So I upgraded to 2.10.1 and managed to get past the failing db migration, but now I am finding I can't even publish a repo, and I suspect other issues might exist. I have documented my troubleshooting steps below, and I verified these issues don't exist on another pulp server running 2.10.0. Ultimately, I am wondering if I can revert to 2.10.0 cleanly, or if I need to upgrade to a beta version and whether that should be 2.10.2 or 2.11.0 Beta 2?<o:p></o:p><o:p> </o:p>[root@pulp-server]# pulp-admin rpm repo publish run --repo-id redhat-6-x86_64-os<o:p></o:p>+----------------------------------------------------------------------+<o:p></o:p> Publishing Repository [redhat-6-x86_64-os]<o:p></o:p>+----------------------------------------------------------------------+<o:p></o:p><o:p> </o:p>This command may be exited via ctrl+c without affecting the request.<o:p></o:p><o:p> </o:p><o:p> </o:p>Initializing repo metadata<o:p></o:p>[-]<o:p></o:p>... completed<o:p></o:p><o:p> </o:p>Publishing Distribution files<o:p></o:p>[-]<o:p></o:p>... completed<o:p></o:p><o:p> </o:p>Publishing RPMs<o:p></o:p>[==================================================] 100%<o:p></o:p>4023 of 4023 items<o:p></o:p>... completed<o:p></o:p><o:p> </o:p>Publishing Delta RPMs<o:p></o:p>... skipped<o:p></o:p><o:p> </o:p>Publishing Errata<o:p></o:p>[==================================================] 100%<o:p></o:p>3712 of 3712 items<o:p></o:p>... completed<o:p></o:p><o:p> </o:p>Publishing Comps file<o:p></o:p>[==================================================] 100%<o:p></o:p>213 of 213 items<o:p></o:p><o:p> </o:p>Task Failed<o:p></o:p><o:p> </o:p>[Errno 1] Operation not permitted<o:p></o:p><o:p> </o:p><o:p> </o:p>Nov 8 15:56:02 pulp-server pulp: pulp.server.async.tasks:INFO: Task failed : [17ecfada-b3a3-42cc-a2b1-74aec4fc9231]<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) Task pulp.server.managers.repo.publish.publish[id] raised unexpected: OSError(1, 'Operation not permitted')<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) Traceback (most recent call last):<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) R = retval = fun(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 488, in __call__<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return super(Task, self).__call__(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in __call__<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return super(PulpTask, self).__call__(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return self.run(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1095, in publish<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1187, in check_publish<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1239, in _do_publish<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) publish_report = publish_repo(transfer_repo, conduit, call_config)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 673, in wrap_f<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return f(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return self._publisher.process_lifecycle()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 566, in process_lifecycle<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) super(PluginStep, self).process_lifecycle()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) step.process()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 253, in process<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) self._process_block()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 297, in _process_block<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) self.process_main()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 905, in process_main<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) selinux.restorecon(timestamp_master_dir.encode('utf-8'), recursive=True)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib64/python2.6/site-packages/selinux/__init__.py", line 83, in restorecon<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) lsetfilecon(path, context)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) OSError: [Errno 1] Operation not permitted<o:p></o:p><o:p> </o:p># I confirmed that selinux is disabled on this test server<o:p></o:p>[root@pulp-server:~]# sestatus<o:p></o:p>SELinux status: disabled<o:p></o:p><o:p> </o:p># I tried running restorecon recursively as root as described here<o:p></o:p>https://www.redhat.com/archives/pulp-list/2016-May/msg00054.html<o:p></o:p><o:p> </o:p># There was no stdout or stderr, and it had no affect on the problem.<o:p></o:p>[root@pulp-server:pulp]# restorecon -R /etc/pki/pulp<o:p></o:p><o:p> </o:p># Running pulp 2.10.1 I tried the fix documented here<o:p></o:p>https://pulp.plan.io/issues/2326<o:p></o:p><o:p> </o:p># I got this error and publish still fails with same error.<o:p></o:p><o:p> </o:p>[root@pulp-server:~]# semodule -R<o:p></o:p>SELinux: Could not downgrade policy file /etc/selinux/targeted/policy/policy.24, searching for an older version.<o:p></o:p>SELinux: Could not open policy file <= /etc/selinux/targeted/policy/policy.24: No such file or directory<o:p></o:p>libsemanage.semanage_reload_policy: load_policy returned error code 2. (No such file or directory).<o:p></o:p><o:p> </o:p># The policy.24 files does in fact exist despite the error to the contrary<o:p></o:p>[root@pulp-server:~]# ls -al /etc/selinux/targeted/policy/policy.24<o:p></o:p>-rw-r--r-- 1 root root 8424080 Oct 28 19:31 /etc/selinux/targeted/policy/policy.24<o:p></o:p>Dustin<o:p></o:p></div></body></html>