[Freeipa-devel] [PATCH] 0020..0022 pki-related upgrade fixes
Fraser Tweedale
ftweedal at redhat.com
Tue Jun 30 16:02:33 UTC 2015
On Mon, Jun 29, 2015 at 05:56:11PM +0200, Martin Basti wrote:
> On 29/06/15 16:03, Fraser Tweedale wrote:
> >On Thu, Jun 25, 2015 at 11:23:01AM +0200, Martin Basti wrote:
> >>On 19/06/15 09:28, Fraser Tweedale wrote:
> >>>The attached patches fix upgrade issues when pki is also updated
> >>>from pre 10.2.4.
> >>>pki dependency is bumped to 10.2.5 - the official builds should be
> >>>done Friday (US time) but it is available from my copr[1]. If
> >>>someone wants to add to official freeipa COPR in meantime the SRPM
> >>>is here[2].
> >>>
> >>>[1] https://copr.fedoraproject.org/coprs/ftweedal/freeipa/
> >>>[2] https://ftweedal.fedorapeople.org/pki-core-10.2.5-0.2.fc21.src.rpm
> >>>
> >>>Thanks,
> >>>Fraser
> >>>
> >>>
> >>Thank you.
> >>
> >>1)
> >>I cannot apply patches.
> >>
> >Rebased patches attached.
> >
> >>2)
> >>IMO patch 0020 was fixed with my patch 266
> >>
> >It seems we are hitting another case of LDAP disconnection during
> >upgrade; without 0020 the upgrade fails. There might be a better
> >way so let me know if you have ideas.
> >
> >>3)
> >>This print should not be there
> >>+
> >>+ print cs_cfg
> >>+ for profile_id in profile_ids:
> >>
> >Thakns; removed.
> >
> >>4)
> >>This is unused variable, it is defined later
> >>+ cs_cfg = None
> >>
> >Thanks; removed.
> >
> >>5)
> >>Can you add there log.error or log.debug instead of pass please?
> >>+ # enable the profile
> >>+ try:
> >>+ profile_api.enable_profile(profile_id)
> >>+ except errors.RemoteRetrieveError:
> >>+ pass
> >>
> >You've got it. Also did this a few lines up where the profile is
> >disabled.
> >
> >>I will test it later.
> >>
> >>--
> >>Martin Basti
> >>
> >Thank you,
> >Fraser
> PATCH 0020 - NACK see my patch 269, it fixes root cause. (IMO with reworked
> patch 21 it is not needed)
>
> PATCH 0021 - NACK, it runs whole upgrade machinery again. Patch how to fix
> it is attached. Sorry I didn't notice it last time.
>
> PATCH 0022 - LGTM
>
>
> --
> Martin Basti
>
Thank you very much!
Your patch to my patch works perfectly. I squashed it into 0021.
Patch 0020 rescinded. Rebased patches attached.
Cheers,
Fraser
-------------- next part --------------
From 8daaed33cf06b5f940195d08038dbaadc562f880 Mon Sep 17 00:00:00 2001
From: Fraser Tweedale <frase at frase.id.au>
Date: Tue, 16 Jun 2015 07:40:36 -0400
Subject: [PATCH 21/22] Upgrade CA schema during upgrade
New schema (for LDAP-based profiles) was introduced in Dogtag, but
Dogtag does not yet have a reliable method for upgrading its schema.
Use FreeIPA's schema update machinery to add the new attributeTypes
and objectClasses defined by Dogtag.
Also update the pki dependencies to 10.2.5, which provides the
schema update file.
---
freeipa.spec.in | 6 +++---
ipaserver/install/server/upgrade.py | 23 +++++++++++++++++++++++
2 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/freeipa.spec.in b/freeipa.spec.in
index 4f08db9f693318c6f4bfaf5e634ccffa78a4a28c..de250d8843506acd6109525c0630132fe60e2268 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -96,7 +96,7 @@ BuildRequires: python-backports-ssl_match_hostname
BuildRequires: softhsm-devel >= 2.0.0rc1-1
BuildRequires: openssl-devel
BuildRequires: p11-kit-devel
-BuildRequires: pki-base >= 10.2.4-1
+BuildRequires: pki-base >= 10.2.5
BuildRequires: python-pytest-multihost >= 0.5
BuildRequires: python-pytest-sourceorder
BuildRequires: python-kdcproxy >= 0.3
@@ -141,8 +141,8 @@ Requires(post): systemd-units
Requires: selinux-policy >= %{selinux_policy_version}
Requires(post): selinux-policy-base
Requires: slapi-nis >= 0.54.2-1
-Requires: pki-ca >= 10.2.4-1
-Requires: pki-kra >= 10.2.4-1
+Requires: pki-ca >= 10.2.5
+Requires: pki-kra >= 10.2.5
Requires(preun): python systemd-units
Requires(postun): python systemd-units
Requires: python-dns >= 1.11.1
diff --git a/ipaserver/install/server/upgrade.py b/ipaserver/install/server/upgrade.py
index 822f746222bd3cb491901205af862a68ec464bbb..4a9f0128aed901e21a1fb57d3f72aecf954df478 100644
--- a/ipaserver/install/server/upgrade.py
+++ b/ipaserver/install/server/upgrade.py
@@ -31,6 +31,7 @@ from ipaserver.install import service
from ipaserver.install import cainstance
from ipaserver.install import certs
from ipaserver.install import otpdinstance
+from ipaserver.install import schemaupdate
from ipaserver.install import sysupgrade
from ipaserver.install import dnskeysyncinstance
from ipaserver.install.upgradeinstance import IPAUpgrade
@@ -1254,6 +1255,27 @@ def update_mod_nss_protocol(http):
sysupgrade.set_upgrade_state('nss.conf', 'protocol_updated_tls12', True)
+def ca_upgrade_schema(ca):
+ root_logger.info('[Upgrading CA schema]')
+ if not ca.is_configured():
+ root_logger.info('CA is not configured')
+ return False
+
+ schema_files=['/usr/share/pki/server/conf/schema-certProfile.ldif']
+ try:
+ modified = schemaupdate.update_schema(schema_files, ldapi=True)
+ except Exception as e:
+ root_logger.error("%s", e)
+ raise RuntimeError('CA schema upgrade failed.', 1)
+ else:
+ if modified:
+ root_logger.info('CA schema update complete')
+ return True
+ else:
+ root_logger.info('CA schema update complete (no changes)')
+ return False
+
+
def add_default_caacl(ca):
root_logger.info('[Add default CA ACL]')
@@ -1452,6 +1474,7 @@ def upgrade_configuration():
ca_restart = any([
ca_restart,
+ ca_upgrade_schema(ca),
upgrade_ca_audit_cert_validity(ca),
certificate_renewal_update(ca),
ca_enable_pkix(ca),
--
2.1.0
-------------- next part --------------
From 56182c88a2853b6d4833ff4500ea56edd2f6653b Mon Sep 17 00:00:00 2001
From: Fraser Tweedale <ftweedal at redhat.com>
Date: Tue, 16 Jun 2015 07:38:06 -0400
Subject: [PATCH 22/22] Migrate CA profiles after enabling LDAPProfileSubsystem
After enabling LDAPProfileSubsystem in Dogtag, migrate the
file-based profiles into the LDAP database.
---
ipaserver/install/cainstance.py | 136 +++++++++++++++++++++++++++++-------
ipaserver/install/server/upgrade.py | 7 +-
2 files changed, 113 insertions(+), 30 deletions(-)
diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py
index 563a198ab472a58cc6fbeeceb7731486ce7ca6b5..884d27a9115bcac6a50d147bfd1f73a0f1d3b72d 100644
--- a/ipaserver/install/cainstance.py
+++ b/ipaserver/install/cainstance.py
@@ -1639,6 +1639,29 @@ def update_people_entry(dercert):
return True
+def ensure_ldap_profiles_container():
+ server_id = installutils.realm_to_serverid(api.env.realm)
+ dogtag_uri = 'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % server_id
+
+ conn = ldap2.ldap2(shared_instance=False, ldap_uri=dogtag_uri)
+ if not conn.isconnected():
+ conn.connect(autobind=True)
+
+ dn = DN(('ou', 'certificateProfiles'), ('ou', 'ca'), ('o', 'ipaca'))
+ try:
+ conn.get_entry(dn)
+ except errors.NotFound:
+ # entry doesn't exist; add it
+ entry = conn.make_entry(
+ dn,
+ objectclass=['top', 'organizationalUnit'],
+ ou=['certificateProfiles'],
+ )
+ conn.add_entry(entry)
+
+ conn.disconnect()
+
+
def configure_profiles_acl():
server_id = installutils.realm_to_serverid(api.env.realm)
dogtag_uri = 'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % server_id
@@ -1677,6 +1700,9 @@ def import_included_profiles():
if not conn.isconnected():
conn.connect(autobind=True)
+ api.Backend.ra_certprofile._read_password()
+ api.Backend.ra_certprofile.override_port = 8443
+
for (profile_id, desc, store_issued) in dogtag.INCLUDED_PROFILES:
dn = DN(('cn', profile_id),
api.env.container_certprofile, api.env.basedn)
@@ -1685,9 +1711,6 @@ def import_included_profiles():
continue # the profile is present
except errors.NotFound:
# profile not found; add it
- profile_data = ipautil.template_file(
- '/usr/share/ipa/profiles/{}.cfg'.format(profile_id), sub_dict)
-
entry = conn.make_entry(
dn,
objectclass=['ipacertprofile'],
@@ -1696,34 +1719,93 @@ def import_included_profiles():
ipacertprofilestoreissued=['TRUE' if store_issued else 'FALSE'],
)
conn.add_entry(entry)
- api.Backend.ra_certprofile._read_password()
- api.Backend.ra_certprofile.override_port = 8443
- with api.Backend.ra_certprofile as profile_api:
- # import the profile
- try:
- profile_api.create_profile(profile_data)
- except errors.RemoteRetrieveError:
- # conflicting profile; replace it if we are
- # installing IPA, but keep it for upgrades
- if api.env.context == 'installer':
- try:
- profile_api.disable_profile(profile_id)
- except errors.RemoteRetrieveError:
- pass
- profile_api.delete_profile(profile_id)
- profile_api.create_profile(profile_data)
-
- # enable the profile
- try:
- profile_api.enable_profile(profile_id)
- except errors.RemoteRetrieveError:
- pass
-
- api.Backend.ra_certprofile.override_port = None
+ profile_data = ipautil.template_file(
+ '/usr/share/ipa/profiles/{}.cfg'.format(profile_id), sub_dict)
+ _create_dogtag_profile(profile_id, profile_data)
root_logger.info("Imported profile '%s'", profile_id)
+ api.Backend.ra_certprofile.override_port = None
conn.disconnect()
+
+def migrate_profiles_to_ldap():
+ """Migrate profiles from filesystem to LDAP.
+
+ This must be run *after* switching to the LDAPProfileSubsystem
+ and restarting the CA.
+
+ The profile might already exist, e.g. if a replica was already
+ upgraded, so this case is ignored.
+
+ """
+ ensure_ldap_profiles_container()
+
+ api.Backend.ra_certprofile._read_password()
+ api.Backend.ra_certprofile.override_port = 8443
+
+ with open(dogtag.configured_constants().CS_CFG_PATH) as f:
+ cs_cfg = f.read()
+ match = re.search(r'^profile\.list=(\S*)', cs_cfg, re.MULTILINE)
+ profile_ids = match.group(1).split(',')
+
+ for profile_id in profile_ids:
+ match = re.search(
+ r'^profile\.{}\.config=(\S*)'.format(profile_id),
+ cs_cfg, re.MULTILINE
+ )
+ if match is None:
+ root_logger.info("No file for profile '%s'; skipping", profile_id)
+ continue
+ filename = match.group(1)
+
+ match = re.search(
+ r'^profile\.{}\.class_id=(\S*)'.format(profile_id),
+ cs_cfg, re.MULTILINE
+ )
+ if match is None:
+ root_logger.info("No class_id for profile '%s'; skipping", profile_id)
+ continue
+ class_id = match.group(1)
+
+ root_logger.info("Migrating profile '%s' to LDAP", profile_id)
+ with open(filename) as f:
+ profile_data = f.read()
+ if profile_data[-1] != '\n':
+ profile_data += '\n'
+ profile_data += 'profileId={}\n'.format(profile_id)
+ profile_data += 'classId={}\n'.format(class_id)
+ _create_dogtag_profile(profile_id, profile_data)
+
+ api.Backend.ra_certprofile.override_port = None
+
+
+def _create_dogtag_profile(profile_id, profile_data):
+ with api.Backend.ra_certprofile as profile_api:
+ # import the profile
+ try:
+ profile_api.create_profile(profile_data)
+ except errors.RemoteRetrieveError:
+ # conflicting profile; replace it if we are
+ # installing IPA, but keep it for upgrades
+ if api.env.context == 'installer':
+ try:
+ profile_api.disable_profile(profile_id)
+ except errors.RemoteRetrieveError:
+ root_logger.debug(
+ "Failed to disable profile '%s' "
+ "(it is probably already disabled)")
+ profile_api.delete_profile(profile_id)
+ profile_api.create_profile(profile_data)
+
+ # enable the profile
+ try:
+ profile_api.enable_profile(profile_id)
+ except errors.RemoteRetrieveError:
+ root_logger.debug(
+ "Failed to enable profile '%s' "
+ "(it is probably already enabled)")
+
+
if __name__ == "__main__":
standard_logging_setup("install.log")
ds = dsinstance.DsInstance()
diff --git a/ipaserver/install/server/upgrade.py b/ipaserver/install/server/upgrade.py
index 4a9f0128aed901e21a1fb57d3f72aecf954df478..740f04634283cf7673af069812232ebbb10dc82c 100644
--- a/ipaserver/install/server/upgrade.py
+++ b/ipaserver/install/server/upgrade.py
@@ -333,8 +333,8 @@ def ca_enable_ldap_profile_subsystem(ca):
quotes=False,
separator='=')
- # TODO import file-based profiles into Dogtag
- # More code needed on Dogtag side for this.
+ ca.restart(dogtag.configured_constants().PKI_INSTANCE_NAME)
+ cainstance.migrate_profiles_to_ldap()
return needs_update
@@ -1479,7 +1479,6 @@ def upgrade_configuration():
certificate_renewal_update(ca),
ca_enable_pkix(ca),
ca_configure_profiles_acl(ca),
- ca_enable_ldap_profile_subsystem(ca),
])
if ca_restart:
@@ -1489,6 +1488,8 @@ def upgrade_configuration():
except ipautil.CalledProcessError as e:
root_logger.error("Failed to restart %s: %s", ca.service_name, e)
+ ca_enable_ldap_profile_subsystem(ca)
+
# This step MUST be done after ca_enable_ldap_profile_subsystem and
# ca_configure_profiles_acl, and the consequent restart, but does not
# itself require a restart.
--
2.1.0
More information about the Freeipa-devel
mailing list