<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style></style></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1>Brian –<o:p></o:p><o:p> </o:p>Thanks for your response. You are right that I don’t seem to have the selinux-policy-devel package. Although I do have a number of seemingly related packages, but I don’t need nor intend to use selinux on this particular test server. I tried getenforce again with the same result. I also provided the instructions from 2326 that I followed below. Another interesting aspect of this issue: I rebooted the box to ensure that selinux was in fact disabled, which it was, but I noticed that my first publish attempt is actually partially successful. All subsequent attempts fail immediately. This also happens by just restarting the pulp services. You don't have to reboot. In discussing this with Sean Meyers it seems I have a variant of <a href="https://pulp.plan.io/issues/2387">https://pulp.plan.io/issues/2387</a> that doesn’t depend on selinux being enabled. I have been told they are working on a hotfix for that. At this point I am just wondering if it is possible to revert to 2.10 while I wait for the full solution in 2.10.2. Thanks.<o:p></o:p><o:p> </o:p>[root@pulp-server:~]# getenforce<o:p></o:p>Disabled<o:p></o:p>[root@pulp-server:~]# rpm -qa |grep -i selinux<o:p></o:p>selinux-policy-targeted-3.7.19-292.el6.noarch<o:p></o:p>libselinux-2.0.94-7.el6.i686<o:p></o:p>libselinux-ruby-2.0.94-7.el6.x86_64<o:p></o:p>libselinux-devel-2.0.94-7.el6.x86_64<o:p></o:p>selinux-policy-3.7.19-292.el6.noarch<o:p></o:p>libselinux-python-2.0.94-7.el6.x86_64<o:p></o:p>libselinux-2.0.94-7.el6.x86_64<o:p></o:p>pulp-selinux-2.10.1-1.el6.noarch<o:p></o:p>libselinux-utils-2.0.94-7.el6.x86_64<o:p></o:p><o:p> </o:p># Instructions I followed from 2326. I also rebooted this test machine for good measure.<o:p></o:p>$ getenforce<o:p></o:p>Enforcing<o:p></o:p>$ setenforce 0<o:p></o:p>$ echo > /var/log/audit/audit.log<o:p></o:p>$ semodule -R<o:p></o:p><o:p> </o:p>From: Brian Bouterse [mailto:bbouters@redhat.com] Sent: Wednesday, November 09, 2016 11:41 AM To: Mcnabb, Dustin Cc: pulp-list@redhat.com Subject: [E] Re: [Pulp-list] Receiving error when trying to publish any rpm repo.<o:p></o:p><o:p> </o:p><div><div>Hi Dustin,<o:p></o:p></div><div>I believe the 2.10.2 and 2.11 GA releases allow you to upgrade from 2.10.1. If you can't restore from backups onto a fresh 2.10.0 GA install, I would recommend upgrading to the 2.10.2 once that is GA. You could also try the 2.10.2 beta or 2.11 beta.<o:p></o:p></div><div>You're showing what looks like an SELinux error, but sestatus shows disabled. Does audit.log also show errors when your publish fails? Also what does `sudo getenforce` show? It would be good to confirm that SELinux is really not set to "Enforcing" on your system.<o:p></o:p></div><div>You mention you applied the fix from 2326. What changes did you make in that area?<o:p></o:p></div><div>Also regarding the /etc/selinux/targeted/policy/policy.24 stuff, maybe you are missing the selinux-policy-devel package?<o:p></o:p></div><div>-Brian<o:p></o:p></div><div><o:p> </o:p></div></div><div><o:p> </o:p><div>On Tue, Nov 8, 2016 at 1:53 PM, Mcnabb, Dustin <<a href="mailto:dustin.mcnabb@verizon.com" target="_blank">dustin.mcnabb@verizon.com</a>> wrote:<o:p></o:p><div><div>So I upgraded to 2.10.1 and managed to get past the failing db migration, but now I am finding I can't even publish a repo, and I suspect other issues might exist. I have documented my troubleshooting steps below, and I verified these issues don't exist on another pulp server running 2.10.0. Ultimately, I am wondering if I can revert to 2.10.0 cleanly, or if I need to upgrade to a beta version and whether that should be 2.10.2 or 2.11.0 Beta 2?<o:p></o:p> <o:p></o:p>[root@pulp-server]# pulp-admin rpm repo publish run --repo-id redhat-6-x86_64-os<o:p></o:p>+----------------------------------------------------------------------+<o:p></o:p> Publishing Repository [redhat-6-x86_64-os]<o:p></o:p>+----------------------------------------------------------------------+<o:p></o:p> <o:p></o:p>This command may be exited via ctrl+c without affecting the request.<o:p></o:p> <o:p></o:p> <o:p></o:p>Initializing repo metadata<o:p></o:p>[-]<o:p></o:p>... completed<o:p></o:p> <o:p></o:p>Publishing Distribution files<o:p></o:p>[-]<o:p></o:p>... completed<o:p></o:p> <o:p></o:p>Publishing RPMs<o:p></o:p>[==================================================] 100%<o:p></o:p>4023 of 4023 items<o:p></o:p>... completed<o:p></o:p> <o:p></o:p>Publishing Delta RPMs<o:p></o:p>... skipped<o:p></o:p> <o:p></o:p>Publishing Errata<o:p></o:p>[==================================================] 100%<o:p></o:p>3712 of 3712 items<o:p></o:p>... completed<o:p></o:p> <o:p></o:p>Publishing Comps file<o:p></o:p>[==================================================] 100%<o:p></o:p>213 of 213 items<o:p></o:p> <o:p></o:p>Task Failed<o:p></o:p> <o:p></o:p>[Errno 1] Operation not permitted<o:p></o:p> <o:p></o:p> <o:p></o:p>Nov 8 15:56:02 pulp-server pulp: pulp.server.async.tasks:INFO: Task failed : [17ecfada-b3a3-42cc-a2b1-74aec4fc9231]<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) Task pulp.server.managers.repo.publish.publish[id] raised unexpected: OSError(1, 'Operation not permitted')<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) Traceback (most recent call last):<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) R = retval = fun(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 488, in __call__<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return super(Task, self).__call__(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in __call__<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return super(PulpTask, self).__call__(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return self.run(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1095, in publish<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1187, in check_publish<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1239, in _do_publish<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) publish_report = publish_repo(transfer_repo, conduit, call_config)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 673, in wrap_f<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return f(*args, **kwargs)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return self._publisher.process_lifecycle()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 566, in process_lifecycle<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) super(PluginStep, self).process_lifecycle()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) step.process()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 253, in process<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) self._process_block()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 297, in _process_block<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) self.process_main()<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 905, in process_main<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) selinux.restorecon(timestamp_master_dir.encode('utf-8'), recursive=True)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib64/python2.6/site-packages/selinux/__init__.py", line 83, in restorecon<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) lsetfilecon(path, context)<o:p></o:p>Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) OSError: [Errno 1] Operation not permitted<o:p></o:p> <o:p></o:p># I confirmed that selinux is disabled on this test server<o:p></o:p>[root@pulp-server:~]# sestatus<o:p></o:p>SELinux status: disabled<o:p></o:p> <o:p></o:p># I tried running restorecon recursively as root as described here<o:p></o:p><a href="https://www.redhat.com/archives/pulp-list/2016-May/msg00054.html" target="_blank">https://www.redhat.com/archives/pulp-list/2016-May/msg00054.html</a><o:p></o:p> <o:p></o:p># There was no stdout or stderr, and it had no affect on the problem.<o:p></o:p>[root@pulp-server:pulp]# restorecon -R /etc/pki/pulp<o:p></o:p> <o:p></o:p># Running pulp 2.10.1 I tried the fix documented here<o:p></o:p><a href="https://pulp.plan.io/issues/2326" target="_blank">https://pulp.plan.io/issues/2326</a><o:p></o:p> <o:p></o:p># I got this error and publish still fails with same error.<o:p></o:p> <o:p></o:p>[root@pulp-server:~]# semodule -R<o:p></o:p>SELinux: Could not downgrade policy file /etc/selinux/targeted/policy/policy.24, searching for an older version.<o:p></o:p>SELinux: Could not open policy file <= /etc/selinux/targeted/policy/policy.24: No such file or directory<o:p></o:p>libsemanage.semanage_reload_policy: load_policy returned error code 2. (No such file or directory).<o:p></o:p> <o:p></o:p># The policy.24 files does in fact exist despite the error to the contrary<o:p></o:p>[root@pulp-server:~]# ls -al /etc/selinux/targeted/policy/policy.24<o:p></o:p>-rw-r--r-- 1 root root 8424080 Oct 28 19:31 /etc/selinux/targeted/policy/policy.24<o:p></o:p>Dustin<o:p></o:p></div></div> _______________________________________________ Pulp-list mailing list <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a><o:p></o:p></div><o:p> </o:p></div></div></body></html>