[Pulp-list] Pulp-list Digest, Vol 141, Issue 4
Bin Li (BLOOMBERG/ 120 PARK)
bli111 at bloomberg.net
Wed Aug 18 19:15:46 UTC 2021
I was able to migrate to the 3.14.4 with updated pulp settings. However, I got this error when tried to use the pulp_rpm api:
AttributeError: module 'pulpcore.client.pulp_rpm.models' has no attribute 'OneOf MetadataChecksumTypeEnumNullEnum'
The error went away if I erase and reset the db completely. I think this error is still somehow related to the migration from 3.7. Please advise.
The below is the entire error messages:
# ./test.py
Traceback (most recent call last):
File "./test.py", line 17, in <module>
RepositoriesRpmApi(api_client=rpm_client).list().results
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api/repositories_rpm_api.py", line 305, in list
return self.list_with_http_info(**kwargs) # noqa: E501
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api/repositories_rpm_api.py", line 417, in list_with_http_info
return self.api_client.call_api(
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 365, in call_api
return self.__call_api(resource_path, method,
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 209, in __call_api
return_data = self.deserialize(response_data, response_type)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 281, in deserialize
return self.__deserialize(data, response_type)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 320, in __deserialize
return self.__deserialize_model(data, klass)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 659, in __deserialize_model
kwargs[attr] = self.__deserialize(value, attr_type)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 297, in __deserialize
return [self.__deserialize(sub_data, sub_kls)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 297, in <listcomp>
return [self.__deserialize(sub_data, sub_kls)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 320, in __deserialize
return self.__deserialize_model(data, klass)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 659, in __deserialize_model
kwargs[attr] = self.__deserialize(value, attr_type)
File "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client /pulp_rpm/api_client.py", line 309, in __deserialize
klass = getattr(pulpcore.client.pulp_rpm.models, klass)
AttributeError: module 'pulpcore.client.pulp_rpm.models' has no attribute 'OneOf MetadataChecksumTypeEnumNullEnum'
The test.py is as following:
#!/opt/utils/venv/pulp/3.8.8/bin/python
from pulpcore.client import pulpcore
from pulpcore.client import pulp_rpm
from pulpcore.client.pulp_rpm import (
RepositoriesRpmApi
)
# Configure HTTP basic authorization: basic
configuration = pulpcore.Configuration()
configuration.host = 'http://localhost'
configuration.username = 'pulp'
configuration.password = 'pulp'
configuration.safe_chars_for_path_param = '/'
rpm_client = pulp_rpm.ApiClient(configuration)
#RepositoriesRpmApi(api_client=rpm_client).list().results
RepositoriesRpmApi(api_client=rpm_client).list()
I am running:
{
"versions": [
{
"component": "core",
"version": "3.14.4"
},
{
"component": "rpm",
"version": "3.14.1"
},
{
"component": "file",
"version": "1.8.2"
}
# pip list |grep pulp
pulp-file 1.8.2
pulp-rpm 3.14.1
pulp-rpm-client 3.14.1
pulpcore 3.14.4
pulpcore-client 3.14.4
From: pulp-list at redhat.com At: 08/06/21 02:44:17 UTC-4:00To: pulp-list at redhat.com
Subject: Pulp-list Digest, Vol 141, Issue 4
Send Pulp-list mailing list submissions to
pulp-list at redhat.com
To subscribe or unsubscribe via the World Wide Web, visit
https://listman.redhat.com/mailman/listinfo/pulp-list
or, via email, send a message with subject or body 'help' to
pulp-list-request at redhat.com
You can reach the person managing the list at
pulp-list-owner at redhat.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Pulp-list digest..."
Today's Topics:
1. Re: pulp 3.14 migration (Daniel Alley)
----------------------------------------------------------------------
Message: 1
Date: Thu, 5 Aug 2021 17:38:56 -0400
From: Daniel Alley <dalley at redhat.com>
To: Fabricio Aguiar <faguiard at redhat.com>
Cc: pulp-list <pulp-list at redhat.com>, Bin Li <bli111 at bloomberg.net>
Subject: Re: [Pulp-list] pulp 3.14 migration
Message-ID:
<CAK1D4mAcs7Fa_wM=zhHMkzXH6+Gzf0_3wFnaSaXowY+1_YeBbg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Here's a hypothesis for what happened.
* System is at pulpcore 3.7, it has artifacts saved with checksums that are
disabled on later versions of Pulp
* 3.14 is installed, migration to 3.14 database schema fails due to the
disallowed checksums (IIRC we have a special case to avoid this happening,
but maybe we missed a scenario. There is a question mark here.)
* Bin attempts to run "handle-artifact-checksums" but is unable to do so
because of the incompleted migrations
I think the correct workaround is to update the settings file to allow all
checksum types prior to the upgrade. And presumably if you're still
managing content with such checksums, you want to keep them enabled, anyway.
@Bin Li, On your test system please edit /etc/pulp/settings.py to include
the following line
ALLOWED_CONTENT_CHECKSUMS = ["md5", "sha1", "sha224", "sha256", "sha384",
> "sha512"]
>
And then restart the upgrade process / database migrations.
On Thu, Aug 5, 2021 at 5:26 PM Daniel Alley <dalley at redhat.com> wrote:
> Bin is using Katello's RPM builds, I'm not sure if Katello has RPMs built
> for 3.10.
>
> On Thu, Aug 5, 2021 at 4:22 PM Fabricio Aguiar <faguiard at redhat.com>
> wrote:
>
>> I tried but I couldn't reproduce the error,
>> I know pulpcore 3.11 introduced some breaking changes [1]
>> I would try to migrate from 3.7 to 3.10, and then from 3.10 to 3.14
>>
>> [1] https://docs.pulpproject.org/pulpcore/changes.html#id198
>>
>> Best regards,
>> Fabricio Aguiar
>> Software Engineer, Pulp Project
>> Red Hat Brazil - Latam <https://www.redhat.com/>
>> +55 22 999000595
>>
>>
>>
>> On Thu, Jul 29, 2021 at 1:17 PM Bin Li (BLOOMBERG/ 120 PARK) <
>> bli111 at bloomberg.net> wrote:
>>
>>> Thanks Faguiard. We are using pulp_installer. The error originally from
>>> pulpcore 3.14 and pulp_rpm 3.13.
>>>
>>> I just tried the latest pulp_installer. I still see the same errors.
>>> pulp-file 1.8.2
>>> pulp-rpm 3.14.0
>>> pulpcore 3.14.3
>>>
>>> I noticed a interesting thing. The migration from 3.7.3 works fine if I
>>> migrate a fresh installed 3.7.3 to 3.14.3. The error only happened after I
>>> syncd a few repos to the 3.7.3 instance and then migrate to 3.14.3.
>>>
>>>
>>> From: faguiard at redhat.com At: 07/27/21 10:37:36 UTC-4:00
>>> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
>>> Cc: pulp-list at redhat.com
>>> Subject: Re: [Pulp-list] pulp 3.14 migration
>>>
>>> Hi Bin,
>>>
>>> We currently test migrating from 3.11 to 3.14, so probably something is
>>> happening from 3.7 to 3.11
>>> I'll be trying to reproduce the error. Are you using pulp_installer? If
>>> so, which version? Which plugins do you use?
>>>
>>> Best regards,
>>> Fabricio Aguiar
>>> Software Engineer, Pulp Project
>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>> +55 22 999000595
>>>
>>>
>>>
>>> On Fri, Jul 23, 2021 at 11:29 AM Bin Li (BLOOMBERG/ 120 PARK) <
>>> bli111 at bloomberg.net> wrote:
>>>
>>>> The installation didn't pass the health check because the following
>>>> errors:
>>>> pulpcore-worker: django.core.exceptions.ImproperlyConfigured: There
>>>> have been identified artifacts with forbidden checksum 'sha1'. Run
>>>> 'pulpcore-manager handle-artifact-checksums' first to unset forbidden
>>>> checksums.
>>>>
>>>> Tried to run pulpcore-manager. It also failed. This is a migration from
>>>> existing 3.7.2 to 3.14. Please advise.
>>>>
>>>> $ pulpcore-manager handle-artifact-checksums
>>>> System check identified some issues:
>>>>
>>>> WARNINGS:
>>>> ?: (guardian.W001) Guardian authentication backend is not hooked. You
>>>> can add this in settings as eg: `AUTHENTICATION_BACKENDS =
>>>> ('django.contrib.auth.backends.ModelBackend',
>>>> 'guardian.backends.ObjectPermissionBackend')`.
>>>> Traceback (most recent call last):
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py",
>>>> line 84, in _execute
>>>> return self.cursor.execute(sql, params)
>>>> psycopg2.errors.UndefinedColumn: column
>>>> core_artifact.timestamp_of_interest does not exist
>>>> LINE 1: ...ore_artifact"."sha384", "core_artifact"."sha512",
>>>> "core_arti...
>>>> ^
>>>>
>>>>
>>>> The above exception was the direct cause of the following exception:
>>>>
>>>> Traceback (most recent call last):
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/sql/compiler.py",
>>>> line 1142, in execute_sql
>>>> cursor.execute(sql, params)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py",
>>>> line 67, in execute
>>>> return self._execute_with_wrappers(sql, params, many=False,
>>>> executor=self._execute)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py",
>>>> line 76, in _execute_with_wrappers
>>>> return executor(sql, params, many, context)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py",
>>>> line 84, in _execute
>>>> return self.cursor.execute(sql, params)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/utils.py",
>>>> line 89, in __exit__
>>>> raise dj_exc_value.with_traceback(traceback) from exc_value
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py",
>>>> line 84, in _execute
>>>> return self.cursor.execute(sql, params)
>>>> django.db.utils.ProgrammingError: column
>>>> core_artifact.timestamp_of_interest does not exist
>>>> LINE 1: ...ore_artifact"."sha384", "core_artifact"."sha512",
>>>> "core_arti...
>>>> ^
>>>>
>>>>
>>>> During handling of the above exception, another exception occurred:
>>>>
>>>> Traceback (most recent call last):
>>>> File "/opt/utils/venv/pulp/3.8.8/bin/pulpcore-manager", line 8, in
>>>> <module>
>>>> sys.exit(manage())
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/app/manage.py",
>>>> line 11, in manage
>>>> execute_from_command_line(sys.argv)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/__init__.py",
>>>> line 381, in execute_from_command_line
>>>> utility.execute()
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/__init__.py",
>>>> line 375, in execute
>>>> self.fetch_command(subcommand).run_from_argv(self.argv)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/base.py",
>>>> line 323, in run_from_argv
>>>> self.execute(*args, **cmd_options)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/base.py",
>>>> line 364, in execute
>>>> output = self.handle(*args, **options)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/app/management/commands/handle-artifact-checksums.py",
>>>> line 177, in handle
>>>> for a in artifacts_qs.iterator():
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/query.py",
>>>> line 341, in _iterator
>>>> yield from self._iterable_class(self, chunked_fetch=use_chunked_fetch,
>>>> chunk_size=chunk_size)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/query.py",
>>>> line 55, in __iter__
>>>> results = compiler.execute_sql(chunked_fetch=self.chunked_fetch,
>>>> chunk_size=self.chunk_size)
>>>> File
>>>>
"/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/sql/compiler.py",
>>>> line 1145, in execute_sql
>>>> cursor.close()
>>>> psycopg2.errors.InvalidCursorName: cursor
>>>> "_django_curs_139896608020288_1" does not exist
>>>>
>>>>
>>>> _______________________________________________
>>>> Pulp-list mailing list
>>>> Pulp-list at redhat.com
>>>> https://listman.redhat.com/mailman/listinfo/pulp-list
>>>
>>>
>>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://listman.redhat.com/archives/pulp-list/attachments/20210805/41d782be/
attachment.html>
------------------------------
_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list
End of Pulp-list Digest, Vol 141, Issue 4
*****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210818/dd5f606c/attachment.htm>
More information about the Pulp-list
mailing list