[Pulp-list] Issue upgrading from 3.5.0 to 3.12.0
David Davis
daviddavis at redhat.com
Thu Apr 15 12:49:24 UTC 2021
When you say you run the playbook again and got the same error, which error
do you mean? The "No such file or directory:
'/var/lib/pulp/assets/rest_framework/js/jquery-3.4.1.min.js'" one?
David
On Thu, Apr 15, 2021 at 4:43 AM Eric VS <vs.eric at gmail.com> wrote:
> Hey Matthias,
>
> I ran the pulpcore-manager collectstatic with success, then the playbook
> again but got the same error. This is the full list of commands for
> pulpcore-manager, any others I would need to run?
>
> Last login: Thu Apr 15 16:23:00 CST 2021 on pts/0
> [root at tw-stg-man01 ~]# export PULP_SETTINGS=/etc/pulp/settings.py &&
> /usr/local/lib/pulp/bin/pulpcore-manager
>
> Type 'pulpcore-manager help <subcommand>' for help on a specific
> subcommand.
>
> Available subcommands:
>
> [app]
> handle-artifact-checksums
> reset-admin-password
> stage-profile-summary
>
> [auth]
> changepassword
> createsuperuser
>
> [contenttypes]
> remove_stale_contenttypes
>
> [django]
> check
> compilemessages
> createcachetable
> dbshell
> diffsettings
> dumpdata
> flush
> inspectdb
> loaddata
> makemessages
> makemigrations
> migrate
> sendtestemail
> shell
> showmigrations
> sqlflush
> sqlmigrate
> sqlsequencereset
> squashmigrations
> startapp
> startproject
> test
> testserver
>
> [drf_spectacular]
> spectacular
>
> [guardian]
> clean_orphan_obj_perms
>
> [rest_framework]
> generateschema
>
> [sessions]
> clearsessions
>
> [staticfiles]
> collectstatic
> findstatic
> runserver
>
> *Eric Van Steenbergen*
>
> *E-mail: vs.eric at gmail.com <vs.eric at gmail.com>*
> *Skype: ericvs2014*
> *LinkedIn: Eric Van Steenbergen
> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>*
>
>
> On Thu, Apr 15, 2021 at 3:13 AM Matthias Dellweg <mdellweg at redhat.com>
> wrote:
>
>> Yeah, i can see how that happened. There may be some triggers from the
>> previous run that aren't triggered now, because "nothing changed" according
>> to ansible.
>> This one should be: pulpcore-manager collect-static (or very similar)
>> When you just run pulpcore-manager you get a list where you should be
>> able to spot that subcommand.
>>
>> On Thu, Apr 15, 2021 at 10:06 AM Eric VS <vs.eric at gmail.com> wrote:
>>
>>> Hi Matthias,
>>>
>>> I did try to run that command but thought it was failing due to the last
>>> error.
>>> Unfortunately, when I just tried running the playbook again I ran into
>>> the next error :-)
>>>
>>> TASK [pulp.pulp_installer.pulp_health_check : Checking Pulp services]
>>> ***********************************************************************************************************************************************************
>>> skipping: [localhost] => (item={'key': 'pulpcore-worker at 2.service',
>>> 'value': {'status': 'unknown', 'source': 'systemd', 'state': 'running',
>>> 'name': 'pulpcore-worker at 2.service'}})
>>> skipping: [localhost] => (item={'key':
>>> 'pulpcore-resource-manager.service', 'value': {'status': 'enabled',
>>> 'source': 'systemd', 'state': 'running', 'name':
>>> 'pulpcore-resource-manager.service'}})
>>> skipping: [localhost] => (item={'key': 'pulpcore-worker at .service',
>>> 'value': {'status': 'enabled', 'source': 'systemd', 'state': 'unknown',
>>> 'name': 'pulpcore-worker at .service'}})
>>> skipping: [localhost] => (item={'key': 'pulpcore-worker at 1.service',
>>> 'value': {'status': 'unknown', 'source': 'systemd', 'state': 'running',
>>> 'name': 'pulpcore-worker at 1.service'}})
>>> failed: [localhost] (item={'key': 'pulpcore-api.service', 'value':
>>> {'status': 'enabled', 'source': 'systemd', 'state': 'stopped', 'name':
>>> 'pulpcore-api.service'}}) => {"ansible_loop_var": "item", "changed": false,
>>> "item": {"key": "pulpcore-api.service", "value": {"name":
>>> "pulpcore-api.service", "source": "systemd", "state": "stopped", "status":
>>> "enabled"}}, "msg": "pulpcore-api.service state: stopped"}
>>> skipping: [localhost] => (item={'key': 'pulpcore-content.service',
>>> 'value': {'status': 'enabled', 'source': 'systemd', 'state': 'running',
>>> 'name': 'pulpcore-content.service'}})
>>>
>>> PLAY RECAP
>>> **********************************************************************************************************************************************************************************************************************
>>> localhost : ok=115 changed=19 unreachable=0
>>> failed=1 skipped=50 rescued=0 ignored=0
>>>
>>> (ansible-venv) ╭─ericv at tw-stg-man01 ~
>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) sudo systemctl start
>>> pulpcore-api
>>>
>>> 2 ↵
>>> (ansible-venv) ╭─ericv at tw-stg-man01 ~
>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) ^start^status
>>> (ansible-venv) ╭─ericv at tw-stg-man01 ~
>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) sudo systemctl status
>>> pulpcore-api
>>> ● pulpcore-api.service - Pulp WSGI Server
>>> Loaded: loaded (/usr/lib/systemd/system/pulpcore-api.service;
>>> enabled; vendor preset: disabled)
>>> Active: activating (auto-restart) (Result: exit-code) since Thu
>>> 2021-04-15 16:03:12 CST; 1s ago
>>> Process: 4875 ExecStart=/usr/local/lib/pulp/bin/gunicorn
>>> pulpcore.app.wsgi:application --bind 127.0.0.1:24817 --workers 1
>>> --timeout 90 --access-logfile - --access-logformat pulp
>>> [%({correlation-id}o)s]: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s
>>> "%(f)s" "%(a)s" (code=exited, status=3)
>>> Main PID: 4875 (code=exited, status=3)
>>>
>>> Apr 15 16:03:12 tw-stg-man01.stg01.activeinfra.net systemd[1]: Unit
>>> pulpcore-api.service entered failed state.
>>> Apr 15 16:03:12 tw-stg-man01.stg01.activeinfra.net systemd[1]:
>>> pulpcore-api.service failed.
>>> (ansible-venv) ╭─ericv at tw-stg-man01 ~
>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) sudo journalctl -u
>>> pulpcore-api
>>>
>>> 3 ↵
>>> -- Logs begin at Mon 2020-07-20 11:44:39 CST, end at Thu 2021-04-15
>>> 16:03:38 CST. --
>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Started Pulp WSGI Server.
>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 10:24:09 +0800] [815] [INFO] Starting gunicorn 20.0.4
>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 10:24:09 +0800] [815] [INFO] Listening at:
>>> http://127.0.0.1:24817 (815)
>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 10:24:09 +0800] [815] [INFO] Using worker: sync
>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 10:24:09 +0800] [923] [INFO] Booting worker with pid: 923
>>> Aug 05 10:24:27 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Stopping Pulp WSGI Server...
>>> Aug 05 10:24:27 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 10:24:27 +0800] [815] [INFO] Handling signal: term
>>> Aug 05 10:24:27 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 02:24:27 +0000] [923] [INFO] Worker exiting (pid: 923)
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]:
>>> [2020-08-05 10:24:28 +0800] [815] [INFO] Shutting down: Master
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Stopped Pulp WSGI Server.
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Started Pulp WSGI Server.
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 10:24:28 +0800] [3794] [INFO] Starting gunicorn 20.0.4
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 10:24:28 +0800] [3794] [INFO] Listening at:
>>> http://127.0.0.1:24817 (3794)
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 10:24:28 +0800] [3794] [INFO] Using worker: sync
>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 10:24:28 +0800] [3873] [INFO] Booting worker with pid: 3873
>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 11:16:03 +0800] [3794] [INFO] Handling signal: term
>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Stopping Pulp WSGI Server...
>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 03:16:03 +0000] [3873] [INFO] Worker exiting (pid: 3873)
>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]:
>>> [2020-08-05 11:16:03 +0800] [3794] [INFO] Shutting down: Master
>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Stopped Pulp WSGI Server.
>>> -- Reboot --
>>> Aug 05 11:19:03 tw-stg-man01.app-stg01.activeinfra.net systemd[1]:
>>> Started Pulp WSGI Server.
>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> [2020-08-05 11:19:05 +0800] [1858] [INFO] Starting gunicorn 20.0.4
>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> [2020-08-05 11:19:05 +0800] [1858] [INFO] Listening at:
>>> http://127.0.0.1:24817 (1858)
>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> [2020-08-05 11:19:05 +0800] [1858] [INFO] Using worker: sync
>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> [2020-08-05 11:19:05 +0800] [2329] [INFO] Booting worker with pid: 2329
>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> pulp: django.request:WARNING: Not Found: /
>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [05/Aug/2020:03:38:18 +0000] "GET / HTTP/1.0" 404 77 "-"
>>> "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:81.0) Gecko/20100101
>>> Firefox/81.0
>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> pulp: django.request:WARNING: Not Found: /favicon.ico
>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [05/Aug/2020:03:38:18 +0000] "GET /favicon.ico HTTP/1.0" 404
>>> 77 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:81.0)
>>> Gecko/20100101 F
>>> Aug 07 13:40:58 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [07/Aug/2020:05:40:58 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0"
>>> Aug 07 13:41:07 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [07/Aug/2020:05:41:07 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0"
>>> Aug 07 13:41:43 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [07/Aug/2020:05:41:43 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0"
>>> Aug 07 13:44:48 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [07/Aug/2020:05:44:48 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0"
>>> Aug 07 13:44:59 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [07/Aug/2020:05:44:59 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0"
>>> Aug 07 13:48:11 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [07/Aug/2020:05:48:11 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "HTTPie/0.9.4"
>>> Aug 10 10:34:07 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [10/Aug/2020:02:34:07 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "HTTPie/0.9.4"
>>> Aug 10 12:55:34 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - - [10/Aug/2020:04:55:34 +0000] "GET /pulp/api/v3/status/
>>> HTTP/1.0" 200 1129 "-" "HTTPie/0.9.4"
>>> Aug 10 13:01:47 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> pulp: django.request:WARNING: Not Found: /
>>> Aug 10 13:01:47 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:01:47 +0000] "GET / HTTP/1.1" 404 77 "-"
>>> "HTTPie/0.9.4"
>>> Aug 10 13:02:01 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> pulp: django.request:WARNING: Not Found: /
>>> Aug 10 13:02:01 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:02:01 +0000] "GET / HTTP/1.1" 404 77 "-"
>>> "HTTPie/0.9.4"
>>> Aug 10 13:03:16 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:16 +0000] "POST
>>> /pulp/api/v3/repositories/rpm/rpm/ HTTP/1.1" 201 441 "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:16 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:16 +0000] "GET
>>> /pulp/api/v3/repositories/rpm/rpm/46d2f3e3-904e-448c-ae36-8d0662788f8e/
>>> HTTP/1.1" 200 441 "-" "HTTPie/0.9.
>>> Aug 10 13:03:16 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:16 +0000] "POST
>>> /pulp/api/v3/remotes/rpm/rpm/ HTTP/1.1" 201 462 "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "GET
>>> /pulp/api/v3/remotes/rpm/rpm/ HTTP/1.1" 200 514 "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "GET
>>> /pulp/api/v3/remotes/rpm/rpm/a4e0901e-5091-4a3e-ad6d-41f1f0c88d1e/
>>> HTTP/1.1" 200 462 "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "POST
>>> /pulp/api/v3/repositories/rpm/rpm/46d2f3e3-904e-448c-ae36-8d0662788f8e/sync/
>>> HTTP/1.1" 202 67 "-" "HTTPie
>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 606
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:19 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:19 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1094
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:20 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:20 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:22 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:22 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:23 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:23 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:24 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:24 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:26 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:26 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:27 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:27 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:28 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:28 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:30 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:30 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:31 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:31 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> Aug 10 13:03:33 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]:
>>> 127.0.0.1 - admin [10/Aug/2020:05:03:33 +0000] "GET
>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201
>>> "-" "HTTPie/0.9.4"
>>> ...skipping...
>>> Apr 15 16:03:34 tw-stg-man01.stg01.activeinfra.net systemd[1]:
>>> pulpcore-api.service: main process exited, code=exited,
>>> status=3/NOTIMPLEMENTED
>>> Apr 15 16:03:34 tw-stg-man01.stg01.activeinfra.net systemd[1]: Unit
>>> pulpcore-api.service entered failed state.
>>> Apr 15 16:03:34 tw-stg-man01.stg01.activeinfra.net systemd[1]:
>>> pulpcore-api.service failed.
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net systemd[1]:
>>> pulpcore-api.service holdoff time over, scheduling restart.
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net systemd[1]: Stopped
>>> Pulp WSGI Server.
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net systemd[1]: Started
>>> Pulp WSGI Server.
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 16:03:37 +0800] [4938] [INFO] Starting gunicorn 20.1.0
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 16:03:37 +0800] [4938] [INFO] Listening at:
>>> http://127.0.0.1:24817 (4938)
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 16:03:37 +0800] [4938] [INFO] Using worker: sync
>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 16:03:37 +0800] [4941] [INFO] Booting worker with pid: 4941
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 08:03:39 +0000] [4941] [ERROR] Exception in worker process
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> Traceback (most recent call last):
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/arbiter.py",
>>> line 589, in spawn_worker
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> worker.init_process()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py",
>>> line 134, in init_process
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> self.load_wsgi()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py",
>>> line 146, in load_wsgi
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> self.wsgi = self.app.wsgi()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/base.py",
>>> line 67, in wsgi
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> self.callable = self.load()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py",
>>> line 58, in load
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> return self.load_wsgiapp()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py",
>>> line 48, in load_wsgiapp
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> return util.import_app(self.app_uri)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/util.py", line
>>> 359, in import_app
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: mod
>>> = importlib.import_module(module)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> return _bootstrap._gcd_import(name[level:], package, level)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "<frozen importlib._bootstrap>", line 994, in _gcd_import
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "<frozen importlib._bootstrap>", line 971, in _find_and_load
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "<frozen importlib._bootstrap>", line 665, in _load_unlocked
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "<frozen importlib._bootstrap_external>", line 678, in exec_module
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/wsgi.py",
>>> line 17, in <module>
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> application = get_wsgi_application()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/wsgi.py",
>>> line 13, in get_wsgi_application
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> return WSGIHandler()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/wsgi.py",
>>> line 135, in __init__
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> self.load_middleware()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py",
>>> line 37, in load_middleware
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> mw_instance = middleware(handler)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/middleware.py",
>>> line 49, in __init__
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> self.add_files(self.static_root, prefix=self.static_prefix)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py",
>>> line 113, in add_files
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> self.update_files_dictionary(root, prefix)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py",
>>> line 120, in update_files_dictionary
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> stat_cache = dict(scantree(root))
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py",
>>> line 271, in scantree
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> yield from scantree(entry.path)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py",
>>> line 271, in scantree
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> yield from scantree(entry.path)
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: File
>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py",
>>> line 273, in scantree
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> yield entry.path, entry.stat()
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> FileNotFoundError: [Errno 2] No such file or directory:
>>> '/var/lib/pulp/assets/rest_framework/js/jquery-3.4.1.min.js'
>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 08:03:39 +0000] [4941] [INFO] Worker exiting (pid: 4941)
>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 16:03:40 +0800] [4938] [INFO] Shutting down: Master
>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]:
>>> [2021-04-15 16:03:40 +0800] [4938] [INFO] Reason: Worker failed to boot.
>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net systemd[1]:
>>> pulpcore-api.service: main process exited, code=exited,
>>> status=3/NOTIMPLEMENTED
>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net systemd[1]: Unit
>>> pulpcore-api.service entered failed state.
>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net systemd[1]:
>>> pulpcore-api.service failed.
>>>
>>> Kind regards,
>>>
>>> *Eric Van Steenbergen*
>>>
>>> *E-mail: vs.eric at gmail.com <vs.eric at gmail.com>*
>>> *Skype: ericvs2014*
>>> *LinkedIn: Eric Van Steenbergen
>>> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>*
>>>
>>>
>>> On Thu, Apr 15, 2021 at 2:56 AM Matthias Dellweg <mdellweg at redhat.com>
>>> wrote:
>>>
>>>>
>>>> There is the clue: Run 'pulpcore-manager handle-artifact-checksums'
>>>> first to unset forbidden checksums.
>>>> Pulp dropped to support md5 and sha1 by default. And there is a manual
>>>> step that should bring the database in the right place.
>>>> Try rerunning the ansible playbook afterwards.
>>>>
>>>> However i thought this cleanup routine should have been called
>>>> automatically (as a migration) in the upgrade process. Can anyone spot why
>>>> it didn't?
>>>>
>>>> On Thu, Apr 15, 2021 at 1:48 AM Eric VS <vs.eric at gmail.com> wrote:
>>>>
>>>>> Hey Matthias,
>>>>>
>>>>> The only plugin I am currently using, at least until I'm familiar
>>>>> enough with Pulp, is the RPM plugin. That's our primary use case for now to
>>>>> make sure we are in control of what version of packages we are using across
>>>>> our environments and avoid discrepancies between servers.
>>>>>
>>>>> I have never added or removed any other plugin. After also upgrading
>>>>> the Postgres ansible-galaxy role I don't get the version mismatch error any
>>>>> more but now get the below error and 'command to run'. Running the command
>>>>> fails however also....
>>>>>
>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Collect static
>>>>> content]
>>>>> ******************************************************************************************************************************************************
>>>>> fatal: [localhost]: FAILED! => {"changed": true, "cmd":
>>>>> ["/usr/local/lib/pulp/bin/pulpcore-manager", "collectstatic", "--noinput",
>>>>> "--link"], "delta": "0:00:02.481870", "end": "2021-04-15 07:34:39.205489",
>>>>> "msg": "non-zero return code", "rc": 1, "start": "2021-04-15
>>>>> 07:34:36.723619", "stderr": "Traceback (most recent call last):\n File
>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>\n
>>>>> sys.exit(manage())\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\",
>>>>> line 11, in manage\n execute_from_command_line(sys.argv)\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>> line 381, in execute_from_command_line\n utility.execute()\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>> line 357, in execute\n django.setup()\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py\",
>>>>> line 24, in setup\n apps.populate(settings.INSTALLED_APPS)\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py\",
>>>>> line 114, in populate\n app_config.import_models()\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py\",
>>>>> line 211, in import_models\n self.models_module =
>>>>> import_module(models_module_name)\n File
>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in
>>>>> import_module\n return _bootstrap._gcd_import(name[level:], package,
>>>>> level)\n File \"<frozen importlib._bootstrap>\", line 994, in
>>>>> _gcd_import\n File \"<frozen importlib._bootstrap>\", line 971, in
>>>>> _find_and_load\n File \"<frozen importlib._bootstrap>\", line 955, in
>>>>> _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line
>>>>> 665, in _load_unlocked\n File \"<frozen importlib._bootstrap_external>\",
>>>>> line 678, in exec_module\n File \"<frozen importlib._bootstrap>\", line
>>>>> 219, in _call_with_frames_removed\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/__init__.py\",
>>>>> line 22, in <module>\n from .exporter import ( # noqa\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/exporter.py\",
>>>>> line 15, in <module>\n from .task import CreatedResource, Task\n
>>>>> File\"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/task.py\",
>>>>> line 14, in <module>\n from pulpcore.app.settings import WORKER_TTL\n
>>>>> File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\",
>>>>> line 366, in <module>\n raise e\n File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\",
>>>>> line 344, in <module>\n
>>>>> ).format(checksum)\ndjango.core.exceptions.ImproperlyConfigured: There
>>>>> have been identified artifacts with forbidden checksum 'sha1'. Run
>>>>> 'pulpcore-manager handle-artifact-checksums' first to unset forbidden
>>>>> checksums.", "stderr_lines": ["Traceback (most recent call last):", " File
>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>", "
>>>>> sys.exit(manage())", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\",
>>>>> line 11, in manage", " execute_from_command_line(sys.argv)", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>> line 381, in execute_from_command_line", " utility.execute()", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>> line 357, in execute", " django.setup()", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py\",
>>>>> line 24, in setup", " apps.populate(settings.INSTALLED_APPS)", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py\",
>>>>> line 114, in populate", " app_config.import_models()", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py\",
>>>>> line 211, in import_models", " self.models_module =
>>>>> import_module(models_module_name)", " File
>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in
>>>>> import_module", " return _bootstrap._gcd_import(name[level:], package,
>>>>> level)", " File \"<frozen importlib._bootstrap>\", line 994, in
>>>>> _gcd_import", " File \"<frozen importlib._bootstrap>\", line 971, in
>>>>> _find_and_load", " File \"<frozen importlib._bootstrap>\", line 955, in
>>>>> _find_and_load_unlocked", " File \"<frozen importlib._bootstrap>\", line
>>>>> 665, in _load_unlocked", " File \"<frozen
>>>>> importlib._bootstrap_external>\", line 678, in exec_module", " File
>>>>> \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed",
>>>>> " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/__init__.py\",
>>>>> line 22, in <module>", " from .exporter import ( # noqa", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/exporter.py\",
>>>>> line 15, in <module>", " from .task import CreatedResource, Task", "
>>>>> File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/task.py\",
>>>>> line 14, in <module>", " from pulpcore.app.settings import WORKER_TTL",
>>>>> " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\",
>>>>> line 366, in <module>", " raise e", " File
>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\",
>>>>> line 344, in <module>", " ).format(checksum)",
>>>>> "django.core.exceptions.ImproperlyConfigured: There have been identified
>>>>> artifacts withforbidden checksum 'sha1'. Run 'pulpcore-manager
>>>>> handle-artifact-checksums' first to unset forbidden checksums."], "stdout":
>>>>> "", "stdout_lines": []}
>>>>>
>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Restart all Pulp
>>>>> services]
>>>>> ***************************************************************************************************************************************************
>>>>>
>>>>> NO MORE HOSTS LEFT
>>>>> **************************************************************************************************************************************************************************************************************
>>>>>
>>>>> PLAY RECAP
>>>>> **********************************************************************************************************************************************************************************************************************
>>>>> localhost : ok=87 changed=3 unreachable=0
>>>>> failed=1 skipped=48 rescued=0 ignored=0
>>>>>
>>>>> (ansible-venv) ╭─ericv at tw-stg-man01 ~
>>>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) export
>>>>> PULP_SETTINGS=/etc/pulp/settings.py &&
>>>>> /usr/local/lib/pulp/bin/pulpcore-manager handle-artifact-checksums
>>>>> 2 ↵
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py",
>>>>> line 283, in <module>
>>>>> CONTENT_ORIGIN
>>>>> NameError: name 'CONTENT_ORIGIN' is not defined
>>>>>
>>>>> During handling of the above exception, another exception occurred:
>>>>>
>>>>> Traceback (most recent call last):
>>>>> File "/usr/local/lib/pulp/bin/pulpcore-manager", line 8, in <module>
>>>>> sys.exit(manage())
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py",
>>>>> line 11, in manage
>>>>> execute_from_command_line(sys.argv)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py",
>>>>> line 381, in execute_from_command_line
>>>>> utility.execute()
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py",
>>>>> line 357, in execute
>>>>> django.setup()
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py",
>>>>> line 24, in setup
>>>>> apps.populate(settings.INSTALLED_APPS)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py",
>>>>> line 114, in populate
>>>>> app_config.import_models()
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py",
>>>>> line 211, in import_models
>>>>> self.models_module = import_module(models_module_name)
>>>>> File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in
>>>>> import_module
>>>>> return _bootstrap._gcd_import(name[level:], package, level)
>>>>> File "<frozen importlib._bootstrap>", line 994, in _gcd_import
>>>>> File "<frozen importlib._bootstrap>", line 971, in _find_and_load
>>>>> File "<frozen importlib._bootstrap>", line 955, in
>>>>> _find_and_load_unlocked
>>>>> File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
>>>>> File "<frozen importlib._bootstrap_external>", line 678, in
>>>>> exec_module
>>>>> File "<frozen importlib._bootstrap>", line 219, in
>>>>> _call_with_frames_removed
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/__init__.py",
>>>>> line 22, in <module>
>>>>> from .exporter import ( # noqa
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/exporter.py",
>>>>> line 15, in <module>
>>>>> from .task import CreatedResource, Task
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/task.py",
>>>>> line 14, in <module>
>>>>> from pulpcore.app.settings import WORKER_TTL
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py",
>>>>> line 287, in <module>
>>>>> "CONTENT_ORIGIN is a required setting but it was not configured.
>>>>> This may be caused "
>>>>> django.core.exceptions.ImproperlyConfigured: CONTENT_ORIGIN is a
>>>>> required setting but it was not configured. This may be caused by invalid
>>>>> read permissions of the settings file. Note that CONTENT_ORIGIN is set by
>>>>> the installer automatically.
>>>>> (ansible-venv) ╭─ericv at tw-stg-man01 ~
>>>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd)
>>>>>
>>>>> I finally had some time to dig deeper and played around with the
>>>>> permissions on the file to no result, until I ran the pulpcore-manager
>>>>> command as root. Then this popped up as error indicating there is some
>>>>> integrity error:
>>>>>
>>>>> [root at tw-stg-man01 ~]# export PULP_SETTINGS=/etc/pulp/settings.py &&
>>>>> /usr/local/lib/pulp/bin/pulpcore-manager handle-artifact-checksums
>>>>> Removing forbidden checksum sha1 from database
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py",
>>>>> line 84, in _execute
>>>>> return self.cursor.execute(sql, params)
>>>>> psycopg2.errors.NotNullViolation: null value in column "sha1" violates
>>>>> not-null constraint
>>>>> DETAIL: Failing row contains (1883479d-9e2d-45e1-971f-37246fc3c358,
>>>>> 2020-08-13 05:50:33.543549+00, 2020-08-13 05:50:33.543572+00,
>>>>> artifact/52/e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac...,
>>>>> 286400, 51b3d1a3ff20a0f88ccb084eca86073c, null,
>>>>> bd036226130d705e95bf4ad9acaa5d8d6d4df7460943062c08154bf1,
>>>>> 52e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac0f16a27789,
>>>>> 53964efdef715dd8f8ff96588e7f1ab184a3747d39bb3c9c28fc339659014274...,
>>>>> 0b0d8a7d5d5816705016a5a59489f032fddc31afc438a482549091b730ca4f6b...).
>>>>>
>>>>>
>>>>> The above exception was the direct cause of the following exception:
>>>>>
>>>>> Traceback (most recent call last):
>>>>> File "/usr/local/lib/pulp/bin/pulpcore-manager", line 8, in <module>
>>>>> sys.exit(manage())
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py",
>>>>> line 11, in manage
>>>>> execute_from_command_line(sys.argv)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py",
>>>>> line 381, in execute_from_command_line
>>>>> utility.execute()
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py",
>>>>> line 375, in execute
>>>>> self.fetch_command(subcommand).run_from_argv(self.argv)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/base.py",
>>>>> line 323, in run_from_argv
>>>>> self.execute(*args, **cmd_options)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/base.py",
>>>>> line 364, in execute
>>>>> output = self.handle(*args, **options)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/management/commands/handle-artifact-checksums.py",
>>>>> line 212, in handle
>>>>> artifacts_qs.update(**update_params)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py",
>>>>> line 741, in update
>>>>> rows = query.get_compiler(self.db).execute_sql(CURSOR)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>>>>> line 1471, in execute_sql
>>>>> cursor = super().execute_sql(result_type)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py",
>>>>> line 1142, in execute_sql
>>>>> cursor.execute(sql, params)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py",
>>>>> line 67, in execute
>>>>> return self._execute_with_wrappers(sql, params, many=False,
>>>>> executor=self._execute)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py",
>>>>> line 76, in _execute_with_wrappers
>>>>> return executor(sql, params, many, context)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py",
>>>>> line 84, in _execute
>>>>> return self.cursor.execute(sql, params)
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py",
>>>>> line 89, in __exit__
>>>>> raise dj_exc_value.with_traceback(traceback) from exc_value
>>>>> File
>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py",
>>>>> line 84, in _execute
>>>>> return self.cursor.execute(sql, params)
>>>>> django.db.utils.IntegrityError: null value in column "sha1" violates
>>>>> not-null constraint
>>>>> DETAIL: Failing row contains (1883479d-9e2d-45e1-971f-37246fc3c358,
>>>>> 2020-08-13 05:50:33.543549+00, 2020-08-13 05:50:33.543572+00,
>>>>> artifact/52/e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac...,
>>>>> 286400, 51b3d1a3ff20a0f88ccb084eca86073c, null,
>>>>> bd036226130d705e95bf4ad9acaa5d8d6d4df7460943062c08154bf1,
>>>>> 52e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac0f16a27789,
>>>>> 53964efdef715dd8f8ff96588e7f1ab184a3747d39bb3c9c28fc339659014274...,
>>>>> 0b0d8a7d5d5816705016a5a59489f032fddc31afc438a482549091b730ca4f6b...).
>>>>>
>>>>> I could just delete the whole bunch and reinstall but that wouldn't
>>>>> teach me anything useful I guess so I'm sticking to my guns hoping you guys
>>>>> can help me out in understanding the root cause.
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> *Eric Van Steenbergen*
>>>>>
>>>>> *E-mail: vs.eric at gmail.com <vs.eric at gmail.com>*
>>>>> *Skype: ericvs2014*
>>>>> *LinkedIn: Eric Van Steenbergen
>>>>> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>*
>>>>>
>>>>>
>>>>> On Wed, Apr 14, 2021 at 3:53 AM Matthias Dellweg <mdellweg at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> In your playbook, where there is the list of plugins to install, did
>>>>>> you ever remove one from that list?
>>>>>> Also can you try adding "upgrade: true" to all plugins in that list?
>>>>>>
>>>>>> On Wed, Apr 14, 2021 at 5:27 AM Eric VS <vs.eric at gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Me again :-).
>>>>>>>
>>>>>>> I had a Pulp installation from several weeks ago (on our staging
>>>>>>> environment) and wanted to align it today with my main Pulp installation
>>>>>>> running 3.12.0.
>>>>>>> The staging one was still running on 3.5.0, installed using the
>>>>>>> Ansible Pulp Installer so I followed the upgrade process for the collection
>>>>>>> and ran the playbook. This is the error I got, due to version mismatch but
>>>>>>> not sure on how to fix this one.
>>>>>>>
>>>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Restore SELinux
>>>>>>> contexts on Pulp dirs that may exist]
>>>>>>> ************************************************************************************************************************
>>>>>>> changed: [localhost]
>>>>>>>
>>>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Collect static
>>>>>>> content]
>>>>>>> ******************************************************************************************************************************************************
>>>>>>> fatal: [localhost]: FAILED! => {"changed": true, "cmd":
>>>>>>> ["/usr/local/lib/pulp/bin/pulpcore-manager", "collectstatic", "--noinput",
>>>>>>> "--link"], "delta": "0:00:00.297250", "end": "2021-04-14 11:20:11.886134",
>>>>>>> "msg": "non-zero return code", "rc": 1, "start": "2021-04-14
>>>>>>> 11:20:11.588884", "stderr": "Traceback (most recent call last):\n File
>>>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>\n
>>>>>>> sys.exit(manage())\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\",
>>>>>>> line 11, in manage\n execute_from_command_line(sys.argv)\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>>>> line 381, in execute_from_command_line\n utility.execute()\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>>>> line 325, in execute\n settings.INSTALLED_APPS\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\",
>>>>>>> line 79, in __getattr__\n self._setup(name)\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\",
>>>>>>> line 66, in _setup\n self._wrapped = Settings(settings_module)\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\",
>>>>>>> line 157, in __init__\n mod =
>>>>>>> importlib.import_module(self.SETTINGS_MODULE)\n File
>>>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in
>>>>>>> import_module\n return _bootstrap._gcd_import(name[level:], package,
>>>>>>> level)\n File \"<frozen importlib._bootstrap>\", line 994, in
>>>>>>> _gcd_import\n File \"<frozen importlib._bootstrap>\", line 971, in
>>>>>>> _find_and_load\n File \"<frozen importlib._bootstrap>\", line 955, in
>>>>>>> _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line
>>>>>>> 665, in _load_unlocked\n File \"<frozen importlib._bootstrap_external>\",
>>>>>>> line 678, in exec_module\n File \"<frozen importlib._bootstrap>\", line
>>>>>>> 219, in _call_with_frames_removed\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\",
>>>>>>> line 83, in <module>\n plugin_app_config = entry_point.load()\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\",
>>>>>>> line 2317, in load\n self.require(*args, **kwargs)\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\",
>>>>>>> line 2340, in require\n items = working_set.resolve(reqs, env,
>>>>>>> installer, extras=self.extras)\n File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\",
>>>>>>> line 779, in resolve\n raise VersionConflict(dist,
>>>>>>> req).with_context(dependent_req)\npkg_resources.VersionConflict: (pulpcore
>>>>>>> 3.12.0 (/usr/local/lib/pulp/lib/python3.6/site-packages),
>>>>>>> Requirement.parse('pulpcore<3.6,>=3.4'))", "stderr_lines": ["Traceback
>>>>>>> (most recent call last):", " File
>>>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>", "
>>>>>>> sys.exit(manage())", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\",
>>>>>>> line 11, in manage", " execute_from_command_line(sys.argv)", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>>>> line 381, in execute_from_command_line", " utility.execute()", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\",
>>>>>>> line 325, in execute", " settings.INSTALLED_APPS", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\",
>>>>>>> line 79, in __getattr__", " self._setup(name)", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\",
>>>>>>> line 66, in _setup", " self._wrapped = Settings(settings_module)", "
>>>>>>> File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\",
>>>>>>> line 157, in __init__", " mod =
>>>>>>> importlib.import_module(self.SETTINGS_MODULE)", "File
>>>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in
>>>>>>> import_module", " return _bootstrap._gcd_import(name[level:], package,
>>>>>>> level)", " File \"<frozen importlib._bootstrap>\", line 994, in
>>>>>>> _gcd_import", " File \"<frozen importlib._bootstrap>\", line 971, in
>>>>>>> _find_and_load", " File \"<frozen importlib._bootstrap>\", line 955, in
>>>>>>> _find_and_load_unlocked", " File \"<frozen importlib._bootstrap>\", line
>>>>>>> 665, in _load_unlocked", "File \"<frozen importlib._bootstrap_external>\",
>>>>>>> line 678, in exec_module", " File \"<frozen importlib._bootstrap>\", line
>>>>>>> 219, in _call_with_frames_removed", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\",
>>>>>>> line 83, in <module>", " plugin_app_config = entry_point.load()", "
>>>>>>> File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\",
>>>>>>> line 2317, in load", " self.require(*args, **kwargs)", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\",
>>>>>>> line 2340, in require", " items = working_set.resolve(reqs, env,
>>>>>>> installer, extras=self.extras)", " File
>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\",
>>>>>>> line 779, in resolve", " raise VersionConflict(dist,
>>>>>>> req).with_context(dependent_req)", "pkg_resources.VersionConflict:
>>>>>>> (pulpcore 3.12.0 (/usr/local/lib/pulp/lib/python3.6/site-packages),
>>>>>>> Requirement.parse('pulpcore<3.6,>=3.4'))"], "stdout": "", "stdout_lines":
>>>>>>> []}
>>>>>>>
>>>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Restart all Pulp
>>>>>>> services]
>>>>>>> ***************************************************************************************************************************************************
>>>>>>>
>>>>>>> RUNNING HANDLER [geerlingguy.postgresql : restart postgresql]
>>>>>>> *******************************************************************************************************************************************************************
>>>>>>>
>>>>>>> NO MORE HOSTS LEFT
>>>>>>> **************************************************************************************************************************************************************************************************************
>>>>>>>
>>>>>>> PLAY RECAP
>>>>>>> **********************************************************************************************************************************************************************************************************************
>>>>>>> localhost : ok=91 changed=33 unreachable=0
>>>>>>> failed=1 skipped=46 rescued=0 ignored=0
>>>>>>>
>>>>>>> Any help is greatly appreciated.
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> *Eric Van Steenbergen*
>>>>>>>
>>>>>>> *E-mail: vs.eric at gmail.com <vs.eric at gmail.com>*
>>>>>>> *Skype: ericvs2014*
>>>>>>> *LinkedIn: Eric Van Steenbergen
>>>>>>> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>*
>>>>>>> _______________________________________________
>>>>>>> Pulp-list mailing list
>>>>>>> Pulp-list at redhat.com
>>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-list
>>>>>>
>>>>>> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210415/39864cfc/attachment.htm>
More information about the Pulp-list
mailing list