[Pulp-list] Upgrade from v1 to v2 failed
Michael Hrivnak
mhrivnak at redhat.com
Tue Mar 18 12:40:48 UTC 2014
Andreas,
Thanks for being in touch. We're beyond the point of accepting code changes for 2.1, but we would be happy to receive documentation improvements, including workarounds such as yours. Please just make sure your pull request is against the pulp-2.1 branch.
Thanks,
Michael
----- Original Message -----
From: "Andreas Piesk" <a.piesk at gmx.net>
To: pulp-list at redhat.com
Sent: Sunday, March 16, 2014 5:27:16 PM
Subject: [Pulp-list] Upgrade from v1 to v2 failed
Hello list,
i tried to upgrade from v1 (1.1.15) to v2 (2.1.3), unfortunately it
failed upgrading the database. these are the issue i encountered:
1. missing package attributes
some packages miss some attributes which pulp depends on.
# pulp-v1-upgrade
= Upgrading Database =
..
Upgrading: Packages, Errata, and Distributions
Traceback (most recent call last):
File "/usr/bin/pulp-v1-upgrade", line 50, in <module>
upgrader.upgrade()
File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py",
line 187, in upgrade
self._upgrade_database()
File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py",
line 228, in _upgrade_database
report = db_call(v1_database, tmp_database)
File
"/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line
121, in upgrade
rpms_success = _rpms(v1_database, v2_database, report)
File
"/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line
217, in _rpms
return _packages(v1_database, v2_database, rpm_coll, all_rpms,
'rpm', report)
File
"/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line
247, in _packages
'description' : v1_rpm['description'],
KeyError: 'description'
this is one of the the culprits:
SON([(u'repoids', [u'oracle-jdk-x86_64']), (u'checksum',
SON([(u'sha256',
u'338a454a36b034bc1373dbdda00d99a3eeba184fb2b690d3a81adb3825812727')])),
(u'filename', u'jdk-1.6.0_33-fcs.x86_64.rpm'), (u'epoch', u'2000'),
(u'version', u'1.6.0_33'), (u'_ns', u'packages'), (u'release', u'fcs'),
(u'_id', u'f516b651-f958-4d22-bd1d-e7d8ee66bcc4'), (u'arch', u'x86_64'),
(u'id', u'f516b651-f958-4d22-bd1d-e7d8ee66bcc4'), (u'name', u'jdk')])
i fixed it by assigning defaults:
# diff -Purp
/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py.org
/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py
--- /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py.org
2013-08-18 21:42:25.723204539 +0200
+++ /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py
2013-08-18 22:02:27.581580546 +0200
@@ -244,13 +244,13 @@ def _packages(v1_database, v2_database,
'version' : v1_rpm['version'],
'release' : v1_rpm['release'],
'arch' : v1_rpm['arch'],
- 'description' : v1_rpm['description'],
- 'vendor' : v1_rpm['vendor'],
+ 'description' : v1_rpm.get('description',None),
+ 'vendor' : v1_rpm.get('vendor', None),
'filename' : v1_rpm['filename'],
- 'requires' : v1_rpm['requires'],
- 'provides' : v1_rpm['provides'],
- 'buildhost' : v1_rpm['buildhost'],
- 'license' : v1_rpm['license'],
+ 'requires' : v1_rpm.get('requires', None),
+ 'provides' : v1_rpm.get('provides', None),
+ 'buildhost' : v1_rpm.get('buildhost', None),
+ 'license' : v1_rpm.get('license', None),
'_id' : new_rpm_id,
'_content_type_id' : unit_type_id
2. package names with '.'
# pulp-v1-upgrade
..
Upgrading: Packages, Errata, and Distributions
Traceback (most recent call last):
File "/usr/bin/pulp-v1-upgrade", line 50, in <module>
upgrader.upgrade()
File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py",
line 187, in upgrade
self._upgrade_database()
File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py",
line 228, in _upgrade_database
report = db_call(v1_database, tmp_database)
File
"/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line
125, in upgrade
groups_success = _package_groups(v1_database, v2_database, report)
File
"/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line
622, in _package_groups
v2_coll.insert(new_group, safe=True)
File "/usr/lib64/python2.6/site-packages/pymongo/collection.py",
line 312, in insert
continue_on_error, self.__uuid_subtype), safe)
bson.errors.InvalidDocument: key 'openoffice.org-langpack-pt_PT' must
not contain '.'
I believe, this was introduced in pymongo >2.0.1 and pulp v2 requires 2.1.
RHEL5 contains these openoffice packages and i believe (but i might be
wrong), rpm allows dots.
Funny thing, this issue was mentioned in
http://www.pulpproject.org/ug/UGFAQ.html. workaround for this is to
downgrade.
The issue has been discovered some time ago, see
http://osdir.com/ml/pulp-list/2013-06/msg00014.html.
To sucessfully upgrade i have to set check_keys=False (w=1 is the same
as safe=True on recent versions of pymongo):
---
/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py.org
2013-08-18 21:42:25.723204539 +0200
+++ /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py
2013-08-20 21:57:16.201263427 +0200
@@ -619,7 +619,7 @@ def _package_groups(v1_database, v2_data
'translated_name' : v1_group['translated_name'],
'user_visible' : v1_group['user_visible'],
}
- v2_coll.insert(new_group, safe=True)
+ v2_coll.insert(new_group, check_keys=False, w=1)
new_association = {
'_id' : ObjectId(),
Unfortunately "check_keys=False" must be set on every insert, save,
update, upsert.
3. upgrade documentation
the upgrade documentation
https://pulp-user-guide.readthedocs.org/en/pulp-2.1/v1_upgrade.html is
missing a important step: running pulp-manage-db after db upgrade and
updating pulp to the last stable version.
Do you want pull requests for issue 1 and/or issue 2?
Regards,
-ap
_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list
More information about the Pulp-list
mailing list