From abokovoy at redhat.com Wed Aug 1 08:25:08 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 1 Aug 2012 11:25:08 +0300 Subject: [Freeipa-devel] [PATCH] 0067 Handle exceptions when establishing trusts Message-ID: <20120801082508.GA11348@redhat.com> Hi, I'm almost on vacation ... after this patch. :) Translate exceptions produced by DCERPC bindings when establishing trusts. There are two types of errors that may be produced by DCERPC bindings: - RuntimeError with a text (RuntimeError('NT_STATUS_OBJECT_NAME_NOT_FOUND') - RuntimeError with a numeric code and 'friendly' message Error codes could have two prefixes: - NT error codes, start with NT_STATUS_ prefix - Windows error codes, start with WERR_ prefix Full list of errors is available in Samba source code: libcli/util/ntstatus.h: NT_STATUS error codes libcli/util/werror.h: Windows error codes Majority of errors returned when dealing with trusts are of NT_STATUS type, these also include all typical POSIX errors mapped to corresponding NT errors. Unfortunately, in the textual RuntimeError case very little can be done to get better clarification of the error. More error paths will need to be added as they will be discovered -- DCERPC error messaging is complex. https://fedorahosted.org/freeipa/ticket/2868 -- / Alexander Bokovoy -------------- next part -------------- >From b83dd85543e86c8825243c062c5a5ed2a3e72b75 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 1 Aug 2012 10:14:09 +0300 Subject: [PATCH 3/3] Handle exceptions when establishing trusts Translate exceptions produced by DCERPC bindings when establishing trusts. There are two types of errors that may be produced by DCERPC bindings: - RuntimeError with a text (RuntimeError('NT_STATUS_OBJECT_NAME_NOT_FOUND') - RuntimeError with a numeric code and 'friendly' message Error codes could have two prefixes: - NT error codes, start with NT_STATUS_ prefix - Windows error codes, start with WERR_ prefix Full list of errors is available in Samba source code: libcli/util/ntstatus.h: NT_STATUS error codes libcli/util/werror.h: Windows error codes Majority of errors returned when dealing with trusts are of NT_STATUS type, these also include all typical POSIX errors mapped to corresponding NT errors. Unfortunately, in the textual RuntimeError case very little can be done to get better clarification of the error. More error paths will need to be added as they will be discovered -- DCERPC error messaging is complex. https://fedorahosted.org/freeipa/ticket/2868 --- ipaserver/dcerpc.py | 75 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 15 deletions(-) diff --git a/ipaserver/dcerpc.py b/ipaserver/dcerpc.py index 6b830f65b854b74fcf080b071212e7658f334adf..a2a8e9814c8ba6d676ceda4e20b1b5aa8530573a 100644 --- a/ipaserver/dcerpc.py +++ b/ipaserver/dcerpc.py @@ -28,6 +28,7 @@ from ipalib.parameters import Enum from ipalib import Command from ipalib import errors from ipapython import ipautil +from ipapython.ipa_log_manager import * from ipaserver.install import installutils import os, string, struct, copy @@ -49,6 +50,31 @@ The code in this module relies heavily on samba4-python package and Samba4 python bindings. """) +access_denied_error = errors.ACIError(info='CIFS server denied your credentials') +dcerpc_error_codes = { + -1073741823 : errors.RemoteRetrieveError(reason='communication with CIFS server was unsuccessful'), + -1073741790 : access_denied_error, + -1073741715 : access_denied_error, + -1073741614 : access_denied_error, + -1073741603 : errors.ValidationError(name='AD domain controller', error='unsupported functional level'), +} + +dcerpc_error_messages = { + "NT_STATUS_OBJECT_NAME_NOT_FOUND" : errors.NotFound(reason='Cannot find specified domain or server name'), + "NT_STATUS_INVALID_PARAMETER_MIX" : errors.RequirementError(name='At least the domain or IP address should be specified'), +} + +def assess_dcerpc_exception(num=None,message=None): + """ + Takes error returned by Samba bindings and converts it into + an IPA error class. + """ + if num and num in dcerpc_error_codes: + return dcerpc_error_codes[num] + if message and message in dcerpc_error_messages: + return dcerpc_error_messages[message] + return errors.RemoteRetrieveError(reason='CIFS server communication error: code "%s", message "%s" (both may be "None")' % (num, message)) + class ExtendedDNControl(_ldap.controls.RequestControl): def __init__(self): self.controlType = "1.2.840.113556.1.4.529" @@ -151,8 +177,8 @@ class TrustDomainInstance(object): try: result = lsa.lsarpc(binding, self.parm, self.creds) return result - except: - return None + except RuntimeError, (num, message): + raise assess_dcerpc_exception(num=num, message=message) def __init_lsa_pipe(self, remote_host): """ @@ -168,13 +194,21 @@ class TrustDomainInstance(object): if self._pipe: return + attempts = 0 bindings = self.__gen_lsa_bindings(remote_host) for binding in bindings: - self._pipe = self.__gen_lsa_connection(binding) - if self._pipe: - break + try: + self._pipe = self.__gen_lsa_connection(binding) + if self._pipe: + break + except errors.ACIError, e: + attempts = attempts + 1 + + if self._pipe is None and attempts == len(bindings): + raise errors.ACIError(info='CIFS server %s denied your credentials' % (remote_host)) + if self._pipe is None: - raise errors.RequirementError(name='Working LSA pipe') + raise errors.RemoteRetrieveError(reason='Cannot establish LSA connection to %s. Is CIFS server running?' % (remote_host)) def __gen_lsa_bindings(self, remote_host): """ @@ -195,10 +229,14 @@ class TrustDomainInstance(object): When retrieving DC information anonymously, we can't get SID of the domain """ netrc = net.Net(creds=self.creds, lp=self.parm) - if discover_srv: - result = netrc.finddc(domain=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) - else: - result = netrc.finddc(address=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) + try: + if discover_srv: + result = netrc.finddc(domain=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) + else: + result = netrc.finddc(address=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) + except RuntimeError, e: + raise assess_dcerpc_exception(message=str(e)) + if not result: return False self.info['name'] = unicode(result.domain_name) @@ -217,7 +255,7 @@ class TrustDomainInstance(object): result = res['defaultNamingContext'][0] self.info['dns_hostname'] = res['dnsHostName'][0] except _ldap.LDAPError, e: - print "LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e)) + root_logger.error("LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e))) if result: self.info['sid'] = self.parse_naming_context(result) @@ -232,8 +270,12 @@ class TrustDomainInstance(object): objectAttribute = lsa.ObjectAttribute() objectAttribute.sec_qos = lsa.QosInfo() - self._policy_handle = self._pipe.OpenPolicy2(u"", objectAttribute, security.SEC_FLAG_MAXIMUM_ALLOWED) - result = self._pipe.QueryInfoPolicy2(self._policy_handle, lsa.LSA_POLICY_INFO_DNS) + try: + self._policy_handle = self._pipe.OpenPolicy2(u"", objectAttribute, security.SEC_FLAG_MAXIMUM_ALLOWED) + result = self._pipe.QueryInfoPolicy2(self._policy_handle, lsa.LSA_POLICY_INFO_DNS) + except RuntimeError, e: + raise assess_dcerpc_exception(message=str(e)) + self.info['name'] = unicode(result.name.string) self.info['dns_domain'] = unicode(result.dns_domain.string) self.info['dns_forest'] = unicode(result.dns_forest.string) @@ -315,9 +357,12 @@ class TrustDomainInstance(object): dname.string = another_domain.info['dns_domain'] res = self._pipe.QueryTrustedDomainInfoByName(self._policy_handle, dname, lsa.LSA_TRUSTED_DOMAIN_INFO_FULL_INFO) self._pipe.DeleteTrustedDomain(self._policy_handle, res.info_ex.sid) - except: + except RuntimeError, e: pass - self._pipe.CreateTrustedDomainEx2(self._policy_handle, info, self.auth_info, security.SEC_STD_DELETE) + try: + self._pipe.CreateTrustedDomainEx2(self._policy_handle, info, self.auth_info, security.SEC_STD_DELETE) + except RuntimeError, (num, message): + raise assess_dcerpc_exception(num=num, message=message) class TrustDomainJoins(object): def __init__(self, api): -- 1.7.11.2 From abokovoy at redhat.com Wed Aug 1 08:33:50 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 1 Aug 2012 11:33:50 +0300 Subject: [Freeipa-devel] [PATCH] 0067 Handle exceptions when establishing trusts In-Reply-To: <20120801082508.GA11348@redhat.com> References: <20120801082508.GA11348@redhat.com> Message-ID: <20120801083350.GA12189@redhat.com> On Wed, 01 Aug 2012, Alexander Bokovoy wrote: > Hi, > > I'm almost on vacation ... after this patch. :) > > Translate exceptions produced by DCERPC bindings when establishing > trusts. > > There are two types of errors that may be produced by DCERPC bindings: > - RuntimeError with a text > (RuntimeError('NT_STATUS_OBJECT_NAME_NOT_FOUND') > - RuntimeError with a numeric code and 'friendly' message > > Error codes could have two prefixes: > - NT error codes, start with NT_STATUS_ prefix > - Windows error codes, start with WERR_ prefix > > Full list of errors is available in Samba source code: > libcli/util/ntstatus.h: NT_STATUS error codes > libcli/util/werror.h: Windows error codes > > Majority of errors returned when dealing with trusts are of NT_STATUS > type, these also include all typical POSIX errors mapped to corresponding NT > errors. > > Unfortunately, in the textual RuntimeError case very little can be done > to get better clarification of the error. More error paths will need to be > added as they will be discovered -- DCERPC error messaging is complex. > > https://fedorahosted.org/freeipa/ticket/2868 Too fast :) Updated patch, fixed one copy paste place where different type of RuntimeError is used (all LSA exceptions are of (num, message) type). -- / Alexander Bokovoy -------------- next part -------------- >From ea71b19a028df0a101c7214ed670153ca1bdd6d6 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 1 Aug 2012 10:14:09 +0300 Subject: [PATCH 3/3] Handle exceptions when establishing trusts Translate exceptions produced by DCERPC bindings when establishing trusts. There are two types of errors that may be produced by DCERPC bindings: - RuntimeError with a text (RuntimeError('NT_STATUS_OBJECT_NAME_NOT_FOUND') - RuntimeError with a numeric code and 'friendly' message Error codes could have two prefixes: - NT error codes, start with NT_STATUS_ prefix - Windows error codes, start with WERR_ prefix Full list of errors is available in Samba source code: libcli/util/ntstatus.h: NT_STATUS error codes libcli/util/werror.h: Windows error codes Majority of errors returned when dealing with trusts are of NT_STATUS type, these also include all typical POSIX errors mapped to corresponding NT errors. Unfortunately, in the textual RuntimeError case very little can be done to get better clarification of the error. More error paths will need to be added as they will be discovered -- DCERPC error messaging is complex. https://fedorahosted.org/freeipa/ticket/2868 --- ipaserver/dcerpc.py | 75 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 15 deletions(-) diff --git a/ipaserver/dcerpc.py b/ipaserver/dcerpc.py index 6b830f65b854b74fcf080b071212e7658f334adf..136ed96bf3ad04c04f837a10a2c3253a8253b83c 100644 --- a/ipaserver/dcerpc.py +++ b/ipaserver/dcerpc.py @@ -28,6 +28,7 @@ from ipalib.parameters import Enum from ipalib import Command from ipalib import errors from ipapython import ipautil +from ipapython.ipa_log_manager import * from ipaserver.install import installutils import os, string, struct, copy @@ -49,6 +50,31 @@ The code in this module relies heavily on samba4-python package and Samba4 python bindings. """) +access_denied_error = errors.ACIError(info='CIFS server denied your credentials') +dcerpc_error_codes = { + -1073741823 : errors.RemoteRetrieveError(reason='communication with CIFS server was unsuccessful'), + -1073741790 : access_denied_error, + -1073741715 : access_denied_error, + -1073741614 : access_denied_error, + -1073741603 : errors.ValidationError(name='AD domain controller', error='unsupported functional level'), +} + +dcerpc_error_messages = { + "NT_STATUS_OBJECT_NAME_NOT_FOUND" : errors.NotFound(reason='Cannot find specified domain or server name'), + "NT_STATUS_INVALID_PARAMETER_MIX" : errors.RequirementError(name='At least the domain or IP address should be specified'), +} + +def assess_dcerpc_exception(num=None,message=None): + """ + Takes error returned by Samba bindings and converts it into + an IPA error class. + """ + if num and num in dcerpc_error_codes: + return dcerpc_error_codes[num] + if message and message in dcerpc_error_messages: + return dcerpc_error_messages[message] + return errors.RemoteRetrieveError(reason='CIFS server communication error: code "%s", message "%s" (both may be "None")' % (num, message)) + class ExtendedDNControl(_ldap.controls.RequestControl): def __init__(self): self.controlType = "1.2.840.113556.1.4.529" @@ -151,8 +177,8 @@ class TrustDomainInstance(object): try: result = lsa.lsarpc(binding, self.parm, self.creds) return result - except: - return None + except RuntimeError, (num, message): + raise assess_dcerpc_exception(num=num, message=message) def __init_lsa_pipe(self, remote_host): """ @@ -168,13 +194,21 @@ class TrustDomainInstance(object): if self._pipe: return + attempts = 0 bindings = self.__gen_lsa_bindings(remote_host) for binding in bindings: - self._pipe = self.__gen_lsa_connection(binding) - if self._pipe: - break + try: + self._pipe = self.__gen_lsa_connection(binding) + if self._pipe: + break + except errors.ACIError, e: + attempts = attempts + 1 + + if self._pipe is None and attempts == len(bindings): + raise errors.ACIError(info='CIFS server %s denied your credentials' % (remote_host)) + if self._pipe is None: - raise errors.RequirementError(name='Working LSA pipe') + raise errors.RemoteRetrieveError(reason='Cannot establish LSA connection to %s. Is CIFS server running?' % (remote_host)) def __gen_lsa_bindings(self, remote_host): """ @@ -195,10 +229,14 @@ class TrustDomainInstance(object): When retrieving DC information anonymously, we can't get SID of the domain """ netrc = net.Net(creds=self.creds, lp=self.parm) - if discover_srv: - result = netrc.finddc(domain=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) - else: - result = netrc.finddc(address=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) + try: + if discover_srv: + result = netrc.finddc(domain=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) + else: + result = netrc.finddc(address=remote_host, flags=nbt.NBT_SERVER_LDAP | nbt.NBT_SERVER_DS) + except RuntimeError, e: + raise assess_dcerpc_exception(message=str(e)) + if not result: return False self.info['name'] = unicode(result.domain_name) @@ -217,7 +255,7 @@ class TrustDomainInstance(object): result = res['defaultNamingContext'][0] self.info['dns_hostname'] = res['dnsHostName'][0] except _ldap.LDAPError, e: - print "LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e)) + root_logger.error("LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e))) if result: self.info['sid'] = self.parse_naming_context(result) @@ -232,8 +270,12 @@ class TrustDomainInstance(object): objectAttribute = lsa.ObjectAttribute() objectAttribute.sec_qos = lsa.QosInfo() - self._policy_handle = self._pipe.OpenPolicy2(u"", objectAttribute, security.SEC_FLAG_MAXIMUM_ALLOWED) - result = self._pipe.QueryInfoPolicy2(self._policy_handle, lsa.LSA_POLICY_INFO_DNS) + try: + self._policy_handle = self._pipe.OpenPolicy2(u"", objectAttribute, security.SEC_FLAG_MAXIMUM_ALLOWED) + result = self._pipe.QueryInfoPolicy2(self._policy_handle, lsa.LSA_POLICY_INFO_DNS) + except RuntimeError, (num, message): + raise assess_dcerpc_exception(num=num, message=message) + self.info['name'] = unicode(result.name.string) self.info['dns_domain'] = unicode(result.dns_domain.string) self.info['dns_forest'] = unicode(result.dns_forest.string) @@ -315,9 +357,12 @@ class TrustDomainInstance(object): dname.string = another_domain.info['dns_domain'] res = self._pipe.QueryTrustedDomainInfoByName(self._policy_handle, dname, lsa.LSA_TRUSTED_DOMAIN_INFO_FULL_INFO) self._pipe.DeleteTrustedDomain(self._policy_handle, res.info_ex.sid) - except: + except RuntimeError, e: pass - self._pipe.CreateTrustedDomainEx2(self._policy_handle, info, self.auth_info, security.SEC_STD_DELETE) + try: + self._pipe.CreateTrustedDomainEx2(self._policy_handle, info, self.auth_info, security.SEC_STD_DELETE) + except RuntimeError, (num, message): + raise assess_dcerpc_exception(num=num, message=message) class TrustDomainJoins(object): def __init__(self, api): -- 1.7.11.2 From pvoborni at redhat.com Wed Aug 1 11:19:51 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 01 Aug 2012 13:19:51 +0200 Subject: [Freeipa-devel] [PATCH] 175, 176 Fixed: Unable to select option in combobox in IE and Chrome In-Reply-To: <50183317.90009@redhat.com> References: <5012B9EB.8010304@redhat.com> <50183317.90009@redhat.com> Message-ID: <501910D7.1030401@redhat.com> On 07/31/2012 09:33 PM, Endi Sukma Dewata wrote: > On 7/27/2012 10:55 AM, Petr Vobornik wrote: >> [PATCH] 175 Fixed: Unable to select option in combobox in IE and Chrome >> >> There's probably a bug regarding z-index stacking in Chrome and IE. It >> appears when combobox is used in dialog. Combobox's select area had >> z-index=1010. When first jquery dialog is open it has z-index=1000. >> Further dialogs have higher z-index. When dialog's z-index exceeds 1010 >> option in select control can't be selected. IMO it is a browser bug >> because select control lies in dialog content's stacking context so it >> should be functional even with z-index=1. >> >> This patch raises select area's z-index to 9000000 which should prevent >> the issue for some time. Also it make's combobox's z-index configurable >> so we can solve combobox stacking (ie in service-add dialog). >> >> Second part of: >> https://fedorahosted.org/freeipa/ticket/2834 >> >> [PATCH] 176 Fixed: combobox stacking in service adder dialog >> >> First select's content is displayed under second comboxes content when >> select is opened when second combobox is opened >> >> Bonus for: >> https://fedorahosted.org/freeipa/ticket/2834 > > ACK. > > Pushed to master. -- Petr Vobornik From pvoborni at redhat.com Wed Aug 1 13:43:15 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 01 Aug 2012 15:43:15 +0200 Subject: [Freeipa-devel] [PATCHES] 177-184 Rebase of jquery and jquery-ui libs In-Reply-To: <50183338.7060809@redhat.com> References: <5012C8DE.9090105@redhat.com> <50183338.7060809@redhat.com> Message-ID: <50193273.6060105@redhat.com> On 07/31/2012 09:34 PM, Endi Sukma Dewata wrote: > On 7/27/2012 11:59 AM, Petr Vobornik wrote: >> This effort was driven by ticket >> https://fedorahosted.org/freeipa/ticket/2817 which needed update of >> jqury-ui library. When I was at it I also updated core jquery lib. All >> patches are associated with ticket #2817 which might be bad. Should I >> split it to more tickets? >> >> This effort makes Web UI look consistent in Firefox, Chrome, Opera and >> IE. I didn't try Safari. Also it fixes some visual issues (mostly in IE >> and Opera) and makes buttons prettier. >> >> Patches 177,178 are rebases of jquery and jquery-ui libraries. Upgrade >> introduced two new functional bugs and 2 test bugs which are fixed by >> patches 179, 182 and 183. Update of jquery-ui.css made some parts of >> ipa.css redundant so the redundancy is removed in patch 180. Patch 180 >> also removes some styles which make buttons in association dialog look >> bad. Patch 181 utilizes jquery.button and so makes buttons in >> association dialogs consistent with the rest of the UI. Patch 184 is >> just tiny refactoring. >> >> >> Patch notes: >> [PATCH] 177 Update to jquery.1.7.2.min >> >> jquery library wasn't updated for a long time. >> >> >> [PATCH] 178 Update to jquery-ui-1.8.21.custom >> >> jquery-ui was regenerated to up to date version. >> >> Border radius and IPA custom colors were added to theme so we don't have >> to override them in ipa.css. >> >> [PATCH] 179 Fix for incorrect event handler definition >> >> Clicks events should be better defined by jquery calls (usually >> addEventListener) not as elements attributes. Definition as element >> attribute causes problems after upgrade to jquery 1.7.2. Two occurances >> were removed. >> >> [PATCH] 180 Removal of unnecessary overrides of jquery-ui styles >> >> ipa.css had to be updated to work with updated jquery-ui. This patch >> removes several duplicate styles. >> >> Following issues were fixed: >> * dialogs titles in IE and Opera were black instead of green >> * no black line in first navigation level in IE and Opera >> * all browsers (FF, IE, Chrome, Opera) have the same style for buttons >> and headers >> * dialogs has borders again (should we remove its shadow?) >> >> Known issues: >> * selected tab-1 in Chrome and Opera doesn't overlaps background line >> as in IE and FF. Not sure how to fix without breaking (there are border >> overlaps) the latter ones. I think it looks good enough. >> * some buttons are missing padding. Will be fixed in next patch. >> >> [PATCH] 181 Unified buttons >> >> Buttons in association dialog and action list have different style and >> behavior than buttons in dialogs. This patch unifies it by using >> jquery.button widget. >> >> [PATCH] 182 Web UI tests fix >> >> ACI tests were crashing because of misconfigured facet. >> Entity link test were crashing because of incorrect jquery selector. >> >> [PATCH] 183 Fixed incorrect use of jQuery.attr for setting disabled >> attribute >> >> Occurance: select_widget >> >> Update to latest version of jQuery uncovered this issue. >> >> [PATCH] 184 Replace use of attr with prop for booleans >> >> Recommened way of setting boolean HTML attributes is by $.prop(boolean) >> method not $.attr(boolean) because it sets DOM object property not an >> attribute. Latter works because of jquery's backward compatibility. This >> patch makes things clearer. >> >> Some info about prop and attr: http://stackoverflow.com/a/5876747 > > The build fails, it looks like the Makefile in install/ui/images needs > to be updated: > > make[5]: Entering directory > `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install/ui/images' > > make[5]: *** No rule to make target `ui-bg_glass_40_111111_1x400.png', > needed by `all-am'. Stop. > make[5]: Leaving directory > `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install/ui/images' > > make[4]: *** [all-recursive] Error 1 > make[4]: Leaving directory > `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install/ui' > > make[3]: *** [all-recursive] Error 1 > make[3]: Leaving directory > `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install' > make[2]: *** [all] Error 2 > make[2]: Leaving directory > `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install' > make[1]: *** [all] Error 1 > make[1]: Leaving directory > `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff' > error: Bad exit status from /var/tmp/rpm-tmp.thVPHP (%build) > > > RPM build errors: > Bad exit status from /var/tmp/rpm-tmp.thVPHP (%build) > make: *** [rpms] Error 1 > Silly mistake. I should test builds more often when playing with files... Updated patch 178 attached. Build and install should work. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0178-1-Update-to-jquery-ui-1.8.21.custom.patch Type: text/x-patch Size: 457554 bytes Desc: not available URL: From pspacek at redhat.com Wed Aug 1 14:19:11 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 01 Aug 2012 16:19:11 +0200 Subject: [Freeipa-devel] [PATCH 0046] Separate RR data parsing from LDAP connections Message-ID: <50193ADF.8020609@redhat.com> Hello, this patch finishes LDAP connection vs. LDAP result separation. It is first step necessary for: https://fedorahosted.org/bind-dyndb-ldap/ticket/68 Avoid manual connection management outside ldap_query() It should prevent deadlocks in future. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0046-Separate-RR-data-parsing-from-LDAP-connections.patch Type: text/x-patch Size: 9503 bytes Desc: not available URL: From simo at redhat.com Wed Aug 1 14:35:02 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 01 Aug 2012 10:35:02 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> Message-ID: <1343831702.20530.79.camel@willson.li.ssimo.org> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: > On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: > > On 07/31/2012 12:27 AM, John Dennis wrote: > >> > >> > >> What is taking so long with session bookkeeping? I don't know yet. I would > >> need more timing instrumentation. I will say when I looked at the > >> python-krb5 > >> code (which we use to populate the ccache from the session and read back > >> to > >> store in the session) seemed to be remarkably inefficient. We also elected > >> to > >> use file based ccache rather than in-memory ccache (that means there is a > >> bit > >> of file-IO occurring). > > > > > > A note regarding python-krbV: > > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV > > can obtain several thousands of TGTs per second (even with ccache in a > > file). AFAIK VFS calls are not done synchronously. But others parts of > > python-krbV were left uncovered, so it can contain some surprises. > > > > === Wild speculation follows === > > 1.5 second is incredibly long time, it sounds like some kind of timeout. > > Krb5 libs have usual timeout = 1 second per request. > > > > Are all KDCs in /etc/krb5.conf alive and reachable? > > In this case, as I'm referring to the extreme slowness of the Web UI, > the KDC is on the same system (the ipa server) that is making the > request, correct? > > > Is SSSD running on problematic server? > > Yes. Again, I'm guessing the problematic server is the IPA server itself. > > > Is proper KDC selected by SSSD KDC auto-locator plugin? > > (See /var/lib/sss/pubconf/) > > Yes, I checked that file and it is the IP address of the IPA server on > the same server. Perhaps should this be 127.0.0.1 instead? > > I also have checked the resolv.conf file, and indeed the IP points to > the IPA server itself (same machine) as expected. Both forward and > reverse DNS work. I'm not really sure what else could be setup > incorrectly to cause any KDC slowness. > > Due to the extreme UI slowness issue, I have not created any replicas > so this system is it. I'm not so sure I would be able to see the 1.5 s > delay if it weren't compounded by the overall slowness of the Web UI, > however, the KDC seems to perform well for other systems in the realm. > I'm certainly not taxing it with a huge load, but tickets seem to be > issued without delay. Stephen, another user sent me a wireshark trace for a similar performance issue. So far I see a pause when doing the first leg of a SASL authentication. This may well explain also your issue. Can you test connecting to the ldap server using ldapsearch -Y GSSAPI (you need a kerberos ticket) and telling me if you experience any delay ? If you could run a bunch of searches in a loop and take a wireshark trace that may help analyzing the timings and seeing if there is a correlation. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Aug 1 14:37:23 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 01 Aug 2012 10:37:23 -0400 Subject: [Freeipa-devel] Strange issue I keep hitting with invalid tickets In-Reply-To: <5018530E.70105@redhat.com> References: <5018530E.70105@redhat.com> Message-ID: <1343831843.20530.81.camel@willson.li.ssimo.org> On Tue, 2012-07-31 at 14:50 -0700, Michael Gregg wrote: > I am not sure why, but when I let my ipa machines sit around for a > while(overnight-24hours), and then kinit. When I try to run IPA commands > I get output like this: > > [root at zippyvm12 ~]# ipa host-find > ipa: ERROR: Local error: SASL(-1): generic failure: GSSAPI Error: > Unspecified GSS failure. Minor code may provide more information > (Ticket not yet valid) > > This issue seems to be addressed here: > > https://access.redhat.com/knowledge/solutions/133433 > > It's strange, because when I kinit again, I seem to have a valid > credentials, like here: > > [root at zippyvm12 ~]# klist > Ticket cache: FILE:/tmp/krb5cc_0 > Default principal: admin at TESTRELM.COM > Valid starting Expires Service principal > 07/31/12 17:31:16 08/01/12 17:31:14 krbtgt/TESTRELM.COM at TESTRELM.COM > 07/31/12 17:32:39 08/01/12 17:31:14 > HTTP/zippyvm12.testrelm.com at TESTRELM.COM > > > The work around for me seems to be deleting /tmp/krb5* > Then, I kinit again, and it all starts to work again. > > My question is, why is this happening? Any ideas? On what distro/krb5 libs version ? We fixed a bug where krb5 was badly using the timestamp in the cache and thus sometimes failing to properly set the clock skew in the request. You may be falling for the same bug (normally you'd see this with krb5-auth-dialog when it tried to renew tickets). Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Wed Aug 1 14:53:56 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 01 Aug 2012 16:53:56 +0200 Subject: [Freeipa-devel] [PATCH] 1028 service pac types In-Reply-To: <4FF6F3BA.6030702@redhat.com> References: <5251d02c-d545-4dcc-a1aa-61934d312bb9@zmail17.collab.prod.int.phx2.redhat.com> <4FE8C8C4.8040600@redhat.com> <1340659322.32038.560.camel@willson.li.ssimo.org> <4FE8DA50.9050403@redhat.com> <1340661350.32038.574.camel@willson.li.ssimo.org> <4FF6F3BA.6030702@redhat.com> Message-ID: <50194304.8050507@redhat.com> On 07/06/2012 04:18 PM, Rob Crittenden wrote: > Simo Sorce wrote: >> On Mon, 2012-06-25 at 17:38 -0400, Rob Crittenden wrote: >>> Simo Sorce wrote: >>>> On Mon, 2012-06-25 at 16:23 -0400, Rob Crittenden wrote: >>>>> Simo Sorce wrote: >>>>>> ----- Original Message ----- >>>>>>> This patch is more a WIP than anything. I want to see if I'm on the >>>>>>> right track. >>>>>> >>>>>> Hi Rob, >>>>>> I don't think we need ipaDefaultKrbAuthzData, we can use the same >>>>>> attribute both in ipaGuiConfig and ipaService, where it is placed makes >>>>>> the difference. >>>>>> >>>>>> You haven't changed ipaService in the base ldif. >>>>> >>>>> On new installs the updates are still applied, gets added. >>>> >>>> Sure it 'works' but the ldif files are now incomplete and slightly >>>> misleading, is there a good reason to not update them ? >>> >>> It is because it is in a file 60basev2.ldif. This is a v3 schema >>> addition. It is one confusing element over another. >> >> My concern is that if you pick the ipa schema files to install somewhere >> else you will not have the full schema. >> >> If we do not provide the full schema in our installable ldif files then >> we also need to publish a separate set of documents with the official >> schema. >> >> If that's what we decide to do, then please open a ticket to address >> publication of this separate set of ldif file, although it will become >> yet another thing to maintain and make sure it doesn't get >> de-synchronized with the actual data in the git tree. > > Ok, moved some things around. > > rob The patch works fine. I just had to do 2 things before I could ACK&push it. 1) I added the new attribute OID to our IOD assignment list: https://home.corp.redhat.com/wiki/ldapschemaoids 2) I had to modify the ipaKrbAuthzData update line in 60-trusts.update from: +add: ipaKrbAuthzData: MS-PAC to: +addifnew: ipaKrbAuthzData: MS-PAC Otherwise it would add the MS-PAC type to the config when user changed it to just "PAD" type. I also created a ticket to address a wiring of the new setting also in ipa_kdb driver: https://fedorahosted.org/freeipa/ticket/2960 ACK. Pushed to master. Martin From loris at lgs.com.ve Wed Aug 1 15:20:10 2012 From: loris at lgs.com.ve (Loris Santamaria) Date: Wed, 01 Aug 2012 10:50:10 -0430 Subject: [Freeipa-devel] slow response In-Reply-To: <1343831702.20530.79.camel@willson.li.ssimo.org> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> Message-ID: <1343834410.5050.0.camel@toron.pzo.lgs.com.ve> El mi?, 01-08-2012 a las 10:35 -0400, Simo Sorce escribi?: > On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: > > On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: > > > On 07/31/2012 12:27 AM, John Dennis wrote: > > >> > > >> > > >> What is taking so long with session bookkeeping? I don't know yet. I would > > >> need more timing instrumentation. I will say when I looked at the > > >> python-krb5 > > >> code (which we use to populate the ccache from the session and read back > > >> to > > >> store in the session) seemed to be remarkably inefficient. We also elected > > >> to > > >> use file based ccache rather than in-memory ccache (that means there is a > > >> bit > > >> of file-IO occurring). > > > > > > > > > A note regarding python-krbV: > > > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV > > > can obtain several thousands of TGTs per second (even with ccache in a > > > file). AFAIK VFS calls are not done synchronously. But others parts of > > > python-krbV were left uncovered, so it can contain some surprises. > > > > > > === Wild speculation follows === > > > 1.5 second is incredibly long time, it sounds like some kind of timeout. > > > Krb5 libs have usual timeout = 1 second per request. > > > > > > Are all KDCs in /etc/krb5.conf alive and reachable? > > > > In this case, as I'm referring to the extreme slowness of the Web UI, > > the KDC is on the same system (the ipa server) that is making the > > request, correct? > > > > > Is SSSD running on problematic server? > > > > Yes. Again, I'm guessing the problematic server is the IPA server itself. > > > > > Is proper KDC selected by SSSD KDC auto-locator plugin? > > > (See /var/lib/sss/pubconf/) > > > > Yes, I checked that file and it is the IP address of the IPA server on > > the same server. Perhaps should this be 127.0.0.1 instead? > > > > I also have checked the resolv.conf file, and indeed the IP points to > > the IPA server itself (same machine) as expected. Both forward and > > reverse DNS work. I'm not really sure what else could be setup > > incorrectly to cause any KDC slowness. > > > > Due to the extreme UI slowness issue, I have not created any replicas > > so this system is it. I'm not so sure I would be able to see the 1.5 s > > delay if it weren't compounded by the overall slowness of the Web UI, > > however, the KDC seems to perform well for other systems in the realm. > > I'm certainly not taxing it with a huge load, but tickets seem to be > > issued without delay. > > Stephen, > another user sent me a wireshark trace for a similar performance issue. > So far I see a pause when doing the first leg of a SASL authentication. > This may well explain also your issue. Hi, I experience the same delay in SASL authentication. The number I posted on freeipa-users, show a 1-2 second delay with SASL authentication: # time ldapsearch -x uid=bdteg01662 dn # extended LDIF # # LDAPv3 # base (default) with scope subtree # filter: uid=bdteg01662 # requesting: dn # # bdteg01662, users, accounts, xxx.gob.ve dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 real 0m0.006s user 0m0.001s sys 0m0.003s # time ldapsearch -Y GSSAPI uid=bdteg01662 dn SASL/GSSAPI authentication started SASL username: admin at XXX.GOB.VE SASL SSF: 56 SASL data security layer installed. # extended LDIF # # LDAPv3 # base (default) with scope subtree # filter: uid=bdteg01662 # requesting: dn # # bdteg01662, users, accounts, xxx.gob.ve dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve # search result search: 4 result: 0 Success # numResponses: 2 # numEntries: 1 real 0m2.344s user 0m0.007s sys 0m0.005s > Can you test connecting to the ldap server using ldapsearch -Y GSSAPI > (you need a kerberos ticket) and telling me if you experience any > delay ? > If you could run a bunch of searches in a loop and take a wireshark > trace that may help analyzing the timings and seeing if there is a > correlation. > > Simo. > -- Loris Santamaria linux user #70506 xmpp:loris at lgs.com.ve Links Global Services, C.A. http://www.lgs.com.ve Tel: 0286 952.06.87 Cel: 0414 095.00.10 sip:103 at lgs.com.ve ------------------------------------------------------------ "If I'd asked my customers what they wanted, they'd have said a faster horse" - Henry Ford -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6187 bytes Desc: not available URL: From rmeggins at redhat.com Wed Aug 1 15:58:33 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 01 Aug 2012 09:58:33 -0600 Subject: [Freeipa-devel] slow response In-Reply-To: <1343834410.5050.0.camel@toron.pzo.lgs.com.ve> References: <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343834410.5050.0.camel@toron.pzo.lgs.! com.ve> Message-ID: <50195229.3090505@redhat.com> On 08/01/2012 09:20 AM, Loris Santamaria wrote: > El mi?, 01-08-2012 a las 10:35 -0400, Simo Sorce escribi?: >> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>> >>>>> What is taking so long with session bookkeeping? I don't know yet. I would >>>>> need more timing instrumentation. I will say when I looked at the >>>>> python-krb5 >>>>> code (which we use to populate the ccache from the session and read back >>>>> to >>>>> store in the session) seemed to be remarkably inefficient. We also elected >>>>> to >>>>> use file based ccache rather than in-memory ccache (that means there is a >>>>> bit >>>>> of file-IO occurring). >>>> >>>> A note regarding python-krbV: >>>> I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >>>> can obtain several thousands of TGTs per second (even with ccache in a >>>> file). AFAIK VFS calls are not done synchronously. But others parts of >>>> python-krbV were left uncovered, so it can contain some surprises. >>>> >>>> === Wild speculation follows === >>>> 1.5 second is incredibly long time, it sounds like some kind of timeout. >>>> Krb5 libs have usual timeout = 1 second per request. >>>> >>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>> In this case, as I'm referring to the extreme slowness of the Web UI, >>> the KDC is on the same system (the ipa server) that is making the >>> request, correct? >>> >>>> Is SSSD running on problematic server? >>> Yes. Again, I'm guessing the problematic server is the IPA server itself. >>> >>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>> (See /var/lib/sss/pubconf/) >>> Yes, I checked that file and it is the IP address of the IPA server on >>> the same server. Perhaps should this be 127.0.0.1 instead? >>> >>> I also have checked the resolv.conf file, and indeed the IP points to >>> the IPA server itself (same machine) as expected. Both forward and >>> reverse DNS work. I'm not really sure what else could be setup >>> incorrectly to cause any KDC slowness. >>> >>> Due to the extreme UI slowness issue, I have not created any replicas >>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>> delay if it weren't compounded by the overall slowness of the Web UI, >>> however, the KDC seems to perform well for other systems in the realm. >>> I'm certainly not taxing it with a huge load, but tickets seem to be >>> issued without delay. >> Stephen, >> another user sent me a wireshark trace for a similar performance issue. >> So far I see a pause when doing the first leg of a SASL authentication. >> This may well explain also your issue. > Hi, I experience the same delay in SASL authentication. The number I > posted on freeipa-users, show a 1-2 second delay with SASL > authentication: > > # time ldapsearch -x uid=bdteg01662 dn > # extended LDIF > # > # LDAPv3 > # base (default) with scope subtree > # filter: uid=bdteg01662 > # requesting: dn > # > > # bdteg01662, users, accounts, xxx.gob.ve > dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve > > # search result > search: 2 > result: 0 Success > > # numResponses: 2 > # numEntries: 1 > > real 0m0.006s > user 0m0.001s > sys 0m0.003s > > # time ldapsearch -Y GSSAPI uid=bdteg01662 dn > SASL/GSSAPI authentication started > SASL username: admin at XXX.GOB.VE > SASL SSF: 56 > SASL data security layer installed. > # extended LDIF > # > # LDAPv3 > # base (default) with scope subtree > # filter: uid=bdteg01662 > # requesting: dn > # > > # bdteg01662, users, accounts, xxx.gob.ve > dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve > > # search result > search: 4 > result: 0 Success > > # numResponses: 2 > # numEntries: 1 > > real 0m2.344s > user 0m0.007s > sys 0m0.005s Can you post excerpts from your 389 access log showing the sequence of operations for this connection, bind and search? > > > > >> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >> (you need a kerberos ticket) and telling me if you experience any >> delay ? >> If you could run a bunch of searches in a loop and take a wireshark >> trace that may help analyzing the timings and seeing if there is a >> correlation. >> >> Simo. >> > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvoborni at redhat.com Wed Aug 1 16:04:55 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 01 Aug 2012 18:04:55 +0200 Subject: [Freeipa-devel] [PATCH] 185 PAC Type options for services in Web UI Message-ID: <501953A7.9080405@redhat.com> Following options were added to Web UI * PAC Type in service * PAC Type in configuration Testing metadata for objects and commands were regenerated. https://fedorahosted.org/freeipa/ticket/2958 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0185-PAC-Type-options-for-services-in-Web-UI.patch Type: text/x-patch Size: 52134 bytes Desc: not available URL: From mgregg at redhat.com Wed Aug 1 18:50:10 2012 From: mgregg at redhat.com (Michael Gregg) Date: Wed, 01 Aug 2012 11:50:10 -0700 Subject: [Freeipa-devel] Strange issue I keep hitting with invalid tickets In-Reply-To: <1343831843.20530.81.camel@willson.li.ssimo.org> References: <5018530E.70105@redhat.com> <1343831843.20530.81.camel@willson.li.ssimo.org> Message-ID: <50197A62.9080007@redhat.com> On 08/01/2012 07:37 AM, Simo Sorce wrote: > On Tue, 2012-07-31 at 14:50 -0700, Michael Gregg wrote: >> I am not sure why, but when I let my ipa machines sit around for a >> while(overnight-24hours), and then kinit. When I try to run IPA commands >> I get output like this: >> >> [root at zippyvm12 ~]# ipa host-find >> ipa: ERROR: Local error: SASL(-1): generic failure: GSSAPI Error: >> Unspecified GSS failure. Minor code may provide more information >> (Ticket not yet valid) >> >> This issue seems to be addressed here: >> >> https://access.redhat.com/knowledge/solutions/133433 >> >> It's strange, because when I kinit again, I seem to have a valid >> credentials, like here: >> >> [root at zippyvm12 ~]# klist >> Ticket cache: FILE:/tmp/krb5cc_0 >> Default principal: admin at TESTRELM.COM >> Valid starting Expires Service principal >> 07/31/12 17:31:16 08/01/12 17:31:14 krbtgt/TESTRELM.COM at TESTRELM.COM >> 07/31/12 17:32:39 08/01/12 17:31:14 >> HTTP/zippyvm12.testrelm.com at TESTRELM.COM >> >> >> The work around for me seems to be deleting /tmp/krb5* >> Then, I kinit again, and it all starts to work again. >> >> My question is, why is this happening? Any ideas? > On what distro/krb5 libs version ? > > We fixed a bug where krb5 was badly using the timestamp in the cache and > thus sometimes failing to properly set the clock skew in the request. > > You may be falling for the same bug (normally you'd see this with > krb5-auth-dialog when it tried to renew tickets). > > Simo. > distro is rhel6.3 krb5-libs-1.9-33.el6.x86_64 Was the fix included later than this version? Michael- From loris at lgs.com.ve Wed Aug 1 19:34:27 2012 From: loris at lgs.com.ve (Loris Santamaria) Date: Wed, 01 Aug 2012 15:04:27 -0430 Subject: [Freeipa-devel] slow response In-Reply-To: <50195229.3090505@redhat.com> References: <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343834410.5050.0.camel@toron.pzo.lgs.! com.ve> <50195229.3090505@redhat.com> Message-ID: <1343849667.5050.4.camel@toron.pzo.lgs.com.ve> El mi?, 01-08-2012 a las 09:58 -0600, Rich Megginson escribi?: > On 08/01/2012 09:20 AM, Loris Santamaria wrote: > > El mi?, 01-08-2012 a las 10:35 -0400, Simo Sorce escribi?: > > > On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: > > > > On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: > > > > > On 07/31/2012 12:27 AM, John Dennis wrote: > > > > > > > > > > > > What is taking so long with session bookkeeping? I don't know yet. I would > > > > > > need more timing instrumentation. I will say when I looked at the > > > > > > python-krb5 > > > > > > code (which we use to populate the ccache from the session and read back > > > > > > to > > > > > > store in the session) seemed to be remarkably inefficient. We also elected > > > > > > to > > > > > > use file based ccache rather than in-memory ccache (that means there is a > > > > > > bit > > > > > > of file-IO occurring). > > > > > > > > > > A note regarding python-krbV: > > > > > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV > > > > > can obtain several thousands of TGTs per second (even with ccache in a > > > > > file). AFAIK VFS calls are not done synchronously. But others parts of > > > > > python-krbV were left uncovered, so it can contain some surprises. > > > > > > > > > > === Wild speculation follows === > > > > > 1.5 second is incredibly long time, it sounds like some kind of timeout. > > > > > Krb5 libs have usual timeout = 1 second per request. > > > > > > > > > > Are all KDCs in /etc/krb5.conf alive and reachable? > > > > In this case, as I'm referring to the extreme slowness of the Web UI, > > > > the KDC is on the same system (the ipa server) that is making the > > > > request, correct? > > > > > > > > > Is SSSD running on problematic server? > > > > Yes. Again, I'm guessing the problematic server is the IPA server itself. > > > > > > > > > Is proper KDC selected by SSSD KDC auto-locator plugin? > > > > > (See /var/lib/sss/pubconf/) > > > > Yes, I checked that file and it is the IP address of the IPA server on > > > > the same server. Perhaps should this be 127.0.0.1 instead? > > > > > > > > I also have checked the resolv.conf file, and indeed the IP points to > > > > the IPA server itself (same machine) as expected. Both forward and > > > > reverse DNS work. I'm not really sure what else could be setup > > > > incorrectly to cause any KDC slowness. > > > > > > > > Due to the extreme UI slowness issue, I have not created any replicas > > > > so this system is it. I'm not so sure I would be able to see the 1.5 s > > > > delay if it weren't compounded by the overall slowness of the Web UI, > > > > however, the KDC seems to perform well for other systems in the realm. > > > > I'm certainly not taxing it with a huge load, but tickets seem to be > > > > issued without delay. > > > Stephen, > > > another user sent me a wireshark trace for a similar performance issue. > > > So far I see a pause when doing the first leg of a SASL authentication. > > > This may well explain also your issue. > > Hi, I experience the same delay in SASL authentication. The number I > > posted on freeipa-users, show a 1-2 second delay with SASL > > authentication: > > > > # time ldapsearch -x uid=bdteg01662 dn > > # extended LDIF > > # > > # LDAPv3 > > # base (default) with scope subtree > > # filter: uid=bdteg01662 > > # requesting: dn > > # > > > > # bdteg01662, users, accounts, xxx.gob.ve > > dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve > > > > # search result > > search: 2 > > result: 0 Success > > > > # numResponses: 2 > > # numEntries: 1 > > > > real 0m0.006s > > user 0m0.001s > > sys 0m0.003s > > > > # time ldapsearch -Y GSSAPI uid=bdteg01662 dn > > SASL/GSSAPI authentication started > > SASL username: admin at XXX.GOB.VE > > SASL SSF: 56 > > SASL data security layer installed. > > # extended LDIF > > # > > # LDAPv3 > > # base (default) with scope subtree > > # filter: uid=bdteg01662 > > # requesting: dn > > # > > > > # bdteg01662, users, accounts, xxx.gob.ve > > dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve > > > > # search result > > search: 4 > > result: 0 Success > > > > # numResponses: 2 > > # numEntries: 1 > > > > real 0m2.344s > > user 0m0.007s > > sys 0m0.005s > > Can you post excerpts from your 389 access log showing the sequence of > operations for this connection, bind and search? Here they are: [01/Aug/2012:10:39:40 -041800] conn=33 fd=70 slot=70 connection from 172.18.32.246 to 172.18.32.246 [01/Aug/2012:10:39:40 -041800] conn=33 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI [01/Aug/2012:10:39:42 -041800] conn=33 op=0 RESULT err=14 tag=97 nentries=0 etime=2, SASL bind in progress [01/Aug/2012:10:39:42 -041800] conn=33 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI [01/Aug/2012:10:39:42 -041800] conn=33 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress [01/Aug/2012:10:39:42 -041800] conn=33 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI [01/Aug/2012:10:39:42 -041800] conn=33 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" [01/Aug/2012:10:39:42 -041800] conn=33 op=3 SRCH base="dc=XXXX,dc=XXXX,dc=ve" scope=2 filter="(uid=bdteg01662)" attrs=ALL [01/Aug/2012:10:39:42 -041800] conn=33 op=3 RESULT err=0 tag=101 nentries=1 etime=0 > > > > > > > > > > > Can you test connecting to the ldap server using ldapsearch -Y GSSAPI > > > (you need a kerberos ticket) and telling me if you experience any > > > delay ? > > > If you could run a bunch of searches in a loop and take a wireshark > > > trace that may help analyzing the timings and seeing if there is a > > > correlation. > > > > > > Simo. > > > > > > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > -- Loris Santamaria linux user #70506 xmpp:loris at lgs.com.ve Links Global Services, C.A. http://www.lgs.com.ve Tel: 0286 952.06.87 Cel: 0414 095.00.10 sip:103 at lgs.com.ve ------------------------------------------------------------ "If I'd asked my customers what they wanted, they'd have said a faster horse" - Henry Ford -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6187 bytes Desc: not available URL: From rmeggins at redhat.com Wed Aug 1 19:50:37 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 01 Aug 2012 13:50:37 -0600 Subject: [Freeipa-devel] slow response In-Reply-To: <1343849667.5050.4.camel@toron.pzo.lgs.com.ve> References: <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343834410.5050.0.camel@toron.pzo.lgs.! com.ve> <50195229.3090505@redhat.com> <1343849667.5050.4.camel@toron.pzo.lgs.com! .ve> Message-ID: <5019888D.1000308@redhat.com> On 08/01/2012 01:34 PM, Loris Santamaria wrote: > El mi?, 01-08-2012 a las 09:58 -0600, Rich Megginson escribi?: >> On 08/01/2012 09:20 AM, Loris Santamaria wrote: >>> El mi?, 01-08-2012 a las 10:35 -0400, Simo Sorce escribi?: >>>> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>>>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >>>>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>>>> What is taking so long with session bookkeeping? I don't know yet. I would >>>>>>> need more timing instrumentation. I will say when I looked at the >>>>>>> python-krb5 >>>>>>> code (which we use to populate the ccache from the session and read back >>>>>>> to >>>>>>> store in the session) seemed to be remarkably inefficient. We also elected >>>>>>> to >>>>>>> use file based ccache rather than in-memory ccache (that means there is a >>>>>>> bit >>>>>>> of file-IO occurring). >>>>>> A note regarding python-krbV: >>>>>> I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >>>>>> can obtain several thousands of TGTs per second (even with ccache in a >>>>>> file). AFAIK VFS calls are not done synchronously. But others parts of >>>>>> python-krbV were left uncovered, so it can contain some surprises. >>>>>> >>>>>> === Wild speculation follows === >>>>>> 1.5 second is incredibly long time, it sounds like some kind of timeout. >>>>>> Krb5 libs have usual timeout = 1 second per request. >>>>>> >>>>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>>>> In this case, as I'm referring to the extreme slowness of the Web UI, >>>>> the KDC is on the same system (the ipa server) that is making the >>>>> request, correct? >>>>> >>>>>> Is SSSD running on problematic server? >>>>> Yes. Again, I'm guessing the problematic server is the IPA server itself. >>>>> >>>>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>>>> (See /var/lib/sss/pubconf/) >>>>> Yes, I checked that file and it is the IP address of the IPA server on >>>>> the same server. Perhaps should this be 127.0.0.1 instead? >>>>> >>>>> I also have checked the resolv.conf file, and indeed the IP points to >>>>> the IPA server itself (same machine) as expected. Both forward and >>>>> reverse DNS work. I'm not really sure what else could be setup >>>>> incorrectly to cause any KDC slowness. >>>>> >>>>> Due to the extreme UI slowness issue, I have not created any replicas >>>>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>>>> delay if it weren't compounded by the overall slowness of the Web UI, >>>>> however, the KDC seems to perform well for other systems in the realm. >>>>> I'm certainly not taxing it with a huge load, but tickets seem to be >>>>> issued without delay. >>>> Stephen, >>>> another user sent me a wireshark trace for a similar performance issue. >>>> So far I see a pause when doing the first leg of a SASL authentication. >>>> This may well explain also your issue. >>> Hi, I experience the same delay in SASL authentication. The number I >>> posted on freeipa-users, show a 1-2 second delay with SASL >>> authentication: >>> >>> # time ldapsearch -x uid=bdteg01662 dn >>> # extended LDIF >>> # >>> # LDAPv3 >>> # base (default) with scope subtree >>> # filter: uid=bdteg01662 >>> # requesting: dn >>> # >>> >>> # bdteg01662, users, accounts, xxx.gob.ve >>> dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve >>> >>> # search result >>> search: 2 >>> result: 0 Success >>> >>> # numResponses: 2 >>> # numEntries: 1 >>> >>> real 0m0.006s >>> user 0m0.001s >>> sys 0m0.003s >>> >>> # time ldapsearch -Y GSSAPI uid=bdteg01662 dn >>> SASL/GSSAPI authentication started >>> SASL username: admin at XXX.GOB.VE >>> SASL SSF: 56 >>> SASL data security layer installed. >>> # extended LDIF >>> # >>> # LDAPv3 >>> # base (default) with scope subtree >>> # filter: uid=bdteg01662 >>> # requesting: dn >>> # >>> >>> # bdteg01662, users, accounts, xxx.gob.ve >>> dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve >>> >>> # search result >>> search: 4 >>> result: 0 Success >>> >>> # numResponses: 2 >>> # numEntries: 1 >>> >>> real 0m2.344s >>> user 0m0.007s >>> sys 0m0.005s >> Can you post excerpts from your 389 access log showing the sequence of >> operations for this connection, bind and search? > Here they are: > > [01/Aug/2012:10:39:40 -041800] conn=33 fd=70 slot=70 connection from 172.18.32.246 to 172.18.32.246 > [01/Aug/2012:10:39:40 -041800] conn=33 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI > [01/Aug/2012:10:39:42 -041800] conn=33 op=0 RESULT err=14 tag=97 nentries=0 etime=2, SASL bind in progress Yep, this is it - this should not be taking 2 seconds. I'd like to see what internal operations are going on in this time. Try this - follow the directions at http://port389.org/wiki/FAQ#Troubleshooting but for the access log, to turn on Heavy trace output debugging: dn: cn=config changetype: modify replace: nsslapd-accesslog-level nsslapd-accesslog-level: 4 Then turn the access log level back to the default (256) after reproducing the problem. We should then be able to see the sequence of internal operations triggered by the BIND dn="" method=sasl version=3 mech=GSSAPI > [01/Aug/2012:10:39:42 -041800] conn=33 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI > [01/Aug/2012:10:39:42 -041800] conn=33 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress > [01/Aug/2012:10:39:42 -041800] conn=33 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI > [01/Aug/2012:10:39:42 -041800] conn=33 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" > [01/Aug/2012:10:39:42 -041800] conn=33 op=3 SRCH base="dc=XXXX,dc=XXXX,dc=ve" scope=2 filter="(uid=bdteg01662)" attrs=ALL > [01/Aug/2012:10:39:42 -041800] conn=33 op=3 RESULT err=0 tag=101 nentries=1 etime=0 > > >>> >>> >>> >>>> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >>>> (you need a kerberos ticket) and telling me if you experience any >>>> delay ? >>>> If you could run a bunch of searches in a loop and take a wireshark >>>> trace that may help analyzing the timings and seeing if there is a >>>> correlation. >>>> >>>> Simo. >>>> >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel From loris at lgs.com.ve Wed Aug 1 20:49:13 2012 From: loris at lgs.com.ve (Loris Santamaria) Date: Wed, 01 Aug 2012 16:19:13 -0430 Subject: [Freeipa-devel] slow response In-Reply-To: <5019888D.1000308@redhat.com> References: <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343834410.5050.0.camel@toron.pzo.lgs.! com.ve> <50195229.3090505@redhat.com> <1343849667.5050.4.camel@toron.pzo.lgs.com! .ve> <5019888D.1000308@redhat.com> Message-ID: <1343854153.5050.23.camel@toron.pzo.lgs.com.ve> El mi?, 01-08-2012 a las 13:50 -0600, Rich Megginson escribi?: > On 08/01/2012 01:34 PM, Loris Santamaria wrote: > > El mi?, 01-08-2012 a las 09:58 -0600, Rich Megginson escribi?: > >> On 08/01/2012 09:20 AM, Loris Santamaria wrote: > >>> El mi?, 01-08-2012 a las 10:35 -0400, Simo Sorce escribi?: > >>>> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: > >>>>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: > >>>>>> On 07/31/2012 12:27 AM, John Dennis wrote: > >>>>>>> What is taking so long with session bookkeeping? I don't know yet. I would > >>>>>>> need more timing instrumentation. I will say when I looked at the > >>>>>>> python-krb5 > >>>>>>> code (which we use to populate the ccache from the session and read back > >>>>>>> to > >>>>>>> store in the session) seemed to be remarkably inefficient. We also elected > >>>>>>> to > >>>>>>> use file based ccache rather than in-memory ccache (that means there is a > >>>>>>> bit > >>>>>>> of file-IO occurring). > >>>>>> A note regarding python-krbV: > >>>>>> I used python-krbV extensively in my thesis for KDC stress test. Python-krbV > >>>>>> can obtain several thousands of TGTs per second (even with ccache in a > >>>>>> file). AFAIK VFS calls are not done synchronously. But others parts of > >>>>>> python-krbV were left uncovered, so it can contain some surprises. > >>>>>> > >>>>>> === Wild speculation follows === > >>>>>> 1.5 second is incredibly long time, it sounds like some kind of timeout. > >>>>>> Krb5 libs have usual timeout = 1 second per request. > >>>>>> > >>>>>> Are all KDCs in /etc/krb5.conf alive and reachable? > >>>>> In this case, as I'm referring to the extreme slowness of the Web UI, > >>>>> the KDC is on the same system (the ipa server) that is making the > >>>>> request, correct? > >>>>> > >>>>>> Is SSSD running on problematic server? > >>>>> Yes. Again, I'm guessing the problematic server is the IPA server itself. > >>>>> > >>>>>> Is proper KDC selected by SSSD KDC auto-locator plugin? > >>>>>> (See /var/lib/sss/pubconf/) > >>>>> Yes, I checked that file and it is the IP address of the IPA server on > >>>>> the same server. Perhaps should this be 127.0.0.1 instead? > >>>>> > >>>>> I also have checked the resolv.conf file, and indeed the IP points to > >>>>> the IPA server itself (same machine) as expected. Both forward and > >>>>> reverse DNS work. I'm not really sure what else could be setup > >>>>> incorrectly to cause any KDC slowness. > >>>>> > >>>>> Due to the extreme UI slowness issue, I have not created any replicas > >>>>> so this system is it. I'm not so sure I would be able to see the 1.5 s > >>>>> delay if it weren't compounded by the overall slowness of the Web UI, > >>>>> however, the KDC seems to perform well for other systems in the realm. > >>>>> I'm certainly not taxing it with a huge load, but tickets seem to be > >>>>> issued without delay. > >>>> Stephen, > >>>> another user sent me a wireshark trace for a similar performance issue. > >>>> So far I see a pause when doing the first leg of a SASL authentication. > >>>> This may well explain also your issue. > >>> Hi, I experience the same delay in SASL authentication. The number I > >>> posted on freeipa-users, show a 1-2 second delay with SASL > >>> authentication: > >>> > >>> # time ldapsearch -x uid=bdteg01662 dn > >>> # extended LDIF > >>> # > >>> # LDAPv3 > >>> # base (default) with scope subtree > >>> # filter: uid=bdteg01662 > >>> # requesting: dn > >>> # > >>> > >>> # bdteg01662, users, accounts, xxx.gob.ve > >>> dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve > >>> > >>> # search result > >>> search: 2 > >>> result: 0 Success > >>> > >>> # numResponses: 2 > >>> # numEntries: 1 > >>> > >>> real 0m0.006s > >>> user 0m0.001s > >>> sys 0m0.003s > >>> > >>> # time ldapsearch -Y GSSAPI uid=bdteg01662 dn > >>> SASL/GSSAPI authentication started > >>> SASL username: admin at XXX.GOB.VE > >>> SASL SSF: 56 > >>> SASL data security layer installed. > >>> # extended LDIF > >>> # > >>> # LDAPv3 > >>> # base (default) with scope subtree > >>> # filter: uid=bdteg01662 > >>> # requesting: dn > >>> # > >>> > >>> # bdteg01662, users, accounts, xxx.gob.ve > >>> dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve > >>> > >>> # search result > >>> search: 4 > >>> result: 0 Success > >>> > >>> # numResponses: 2 > >>> # numEntries: 1 > >>> > >>> real 0m2.344s > >>> user 0m0.007s > >>> sys 0m0.005s > >> Can you post excerpts from your 389 access log showing the sequence of > >> operations for this connection, bind and search? > > Here they are: > > > > [01/Aug/2012:10:39:40 -041800] conn=33 fd=70 slot=70 connection from 172.18.32.246 to 172.18.32.246 > > [01/Aug/2012:10:39:40 -041800] conn=33 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI > > [01/Aug/2012:10:39:42 -041800] conn=33 op=0 RESULT err=14 tag=97 nentries=0 etime=2, SASL bind in progress > > Yep, this is it - this should not be taking 2 seconds. I'd like to see > what internal operations are going on in this time. Try this - follow > the directions at http://port389.org/wiki/FAQ#Troubleshooting but for > the access log, to turn on Heavy trace output debugging: > > dn: cn=config > changetype: modify > replace: nsslapd-accesslog-level > nsslapd-accesslog-level: 4 > > Then turn the access log level back to the default (256) after > reproducing the problem. We should then be able to see the sequence of > internal operations triggered by the BIND dn="" method=sasl version=3 > mech=GSSAPI > Here they are, with the 2 second delay: [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=config,cn=userRoot,cn=ldbm database,cn=plugins,cn=config" scope=1 filter="objectclass=vlvsearch" attrs=ALL [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=config,cn=userRoot,cn=ldbm database,cn=plugins,cn=config" scope=1 filter="objectclass=vlvsearch" attrs=ALL [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="dc=XXXX,dc=XXXX,dc=ve" scope=2 filter="(krbPrincipalName=admin at XXXX.XXXX.VE)" attrs=ALL [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs=ALL [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 Just for kick I tried the search that appears in the log, and it returns immediately: ldapsearch -x -b "cn=anonymous-limits,cn=etc,dc=bt,dc=gob,dc=ve" "(|(objectclass=*)(objectclass=ldapsubentry))" nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout # extended LDIF # # LDAPv3 # base with scope subtree # filter: (|(objectclass=*)(objectclass=ldapsubentry)) # requesting: nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout # # anonymous-limits, etc, bt.gob.ve dn: cn=anonymous-limits,cn=etc,dc=bt,dc=gob,dc=ve nsLookThroughLimit: 5000 nsSizeLimit: 5000 # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 > > [01/Aug/2012:10:39:42 -041800] conn=33 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI > > [01/Aug/2012:10:39:42 -041800] conn=33 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress > > [01/Aug/2012:10:39:42 -041800] conn=33 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI > > [01/Aug/2012:10:39:42 -041800] conn=33 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" > > [01/Aug/2012:10:39:42 -041800] conn=33 op=3 SRCH base="dc=XXXX,dc=XXXX,dc=ve" scope=2 filter="(uid=bdteg01662)" attrs=ALL > > [01/Aug/2012:10:39:42 -041800] conn=33 op=3 RESULT err=0 tag=101 nentries=1 etime=0 > > > > > >>> > >>> > >>> > >>>> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI > >>>> (you need a kerberos ticket) and telling me if you experience any > >>>> delay ? > >>>> If you could run a bunch of searches in a loop and take a wireshark > >>>> trace that may help analyzing the timings and seeing if there is a > >>>> correlation. > >>>> > >>>> Simo. > >>>> > >>> > >>> _______________________________________________ > >>> Freeipa-devel mailing list > >>> Freeipa-devel at redhat.com > >>> https://www.redhat.com/mailman/listinfo/freeipa-devel -- Loris Santamaria linux user #70506 xmpp:loris at lgs.com.ve Links Global Services, C.A. http://www.lgs.com.ve Tel: 0286 952.06.87 Cel: 0414 095.00.10 sip:103 at lgs.com.ve ------------------------------------------------------------ "If I'd asked my customers what they wanted, they'd have said a faster horse" - Henry Ford -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6187 bytes Desc: not available URL: From rmeggins at redhat.com Wed Aug 1 20:54:14 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 01 Aug 2012 14:54:14 -0600 Subject: [Freeipa-devel] slow response In-Reply-To: <1343854153.5050.23.camel@toron.pzo.lgs.com.ve> References: <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343834410.5050.0.camel@toron.pzo.lgs.! com.ve> <50195229.3090505@redhat.com> <1343849667.5050.4.camel@toron.pzo.lgs.com! .ve> <5019888D.1000308@redhat.com> <1343854153.5050.23.camel@toron.pzo.lgs.com.ve> Message-ID: <50199776.2080505@redhat.com> On 08/01/2012 02:49 PM, Loris Santamaria wrote: > El mi?, 01-08-2012 a las 13:50 -0600, Rich Megginson escribi?: >> On 08/01/2012 01:34 PM, Loris Santamaria wrote: >>> El mi?, 01-08-2012 a las 09:58 -0600, Rich Megginson escribi?: >>>> On 08/01/2012 09:20 AM, Loris Santamaria wrote: >>>>> El mi?, 01-08-2012 a las 10:35 -0400, Simo Sorce escribi?: >>>>>> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>>>>>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >>>>>>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>>>>>> What is taking so long with session bookkeeping? I don't know yet. I would >>>>>>>>> need more timing instrumentation. I will say when I looked at the >>>>>>>>> python-krb5 >>>>>>>>> code (which we use to populate the ccache from the session and read back >>>>>>>>> to >>>>>>>>> store in the session) seemed to be remarkably inefficient. We also elected >>>>>>>>> to >>>>>>>>> use file based ccache rather than in-memory ccache (that means there is a >>>>>>>>> bit >>>>>>>>> of file-IO occurring). >>>>>>>> A note regarding python-krbV: >>>>>>>> I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >>>>>>>> can obtain several thousands of TGTs per second (even with ccache in a >>>>>>>> file). AFAIK VFS calls are not done synchronously. But others parts of >>>>>>>> python-krbV were left uncovered, so it can contain some surprises. >>>>>>>> >>>>>>>> === Wild speculation follows === >>>>>>>> 1.5 second is incredibly long time, it sounds like some kind of timeout. >>>>>>>> Krb5 libs have usual timeout = 1 second per request. >>>>>>>> >>>>>>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>>>>>> In this case, as I'm referring to the extreme slowness of the Web UI, >>>>>>> the KDC is on the same system (the ipa server) that is making the >>>>>>> request, correct? >>>>>>> >>>>>>>> Is SSSD running on problematic server? >>>>>>> Yes. Again, I'm guessing the problematic server is the IPA server itself. >>>>>>> >>>>>>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>>>>>> (See /var/lib/sss/pubconf/) >>>>>>> Yes, I checked that file and it is the IP address of the IPA server on >>>>>>> the same server. Perhaps should this be 127.0.0.1 instead? >>>>>>> >>>>>>> I also have checked the resolv.conf file, and indeed the IP points to >>>>>>> the IPA server itself (same machine) as expected. Both forward and >>>>>>> reverse DNS work. I'm not really sure what else could be setup >>>>>>> incorrectly to cause any KDC slowness. >>>>>>> >>>>>>> Due to the extreme UI slowness issue, I have not created any replicas >>>>>>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>>>>>> delay if it weren't compounded by the overall slowness of the Web UI, >>>>>>> however, the KDC seems to perform well for other systems in the realm. >>>>>>> I'm certainly not taxing it with a huge load, but tickets seem to be >>>>>>> issued without delay. >>>>>> Stephen, >>>>>> another user sent me a wireshark trace for a similar performance issue. >>>>>> So far I see a pause when doing the first leg of a SASL authentication. >>>>>> This may well explain also your issue. >>>>> Hi, I experience the same delay in SASL authentication. The number I >>>>> posted on freeipa-users, show a 1-2 second delay with SASL >>>>> authentication: >>>>> >>>>> # time ldapsearch -x uid=bdteg01662 dn >>>>> # extended LDIF >>>>> # >>>>> # LDAPv3 >>>>> # base (default) with scope subtree >>>>> # filter: uid=bdteg01662 >>>>> # requesting: dn >>>>> # >>>>> >>>>> # bdteg01662, users, accounts, xxx.gob.ve >>>>> dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve >>>>> >>>>> # search result >>>>> search: 2 >>>>> result: 0 Success >>>>> >>>>> # numResponses: 2 >>>>> # numEntries: 1 >>>>> >>>>> real 0m0.006s >>>>> user 0m0.001s >>>>> sys 0m0.003s >>>>> >>>>> # time ldapsearch -Y GSSAPI uid=bdteg01662 dn >>>>> SASL/GSSAPI authentication started >>>>> SASL username: admin at XXX.GOB.VE >>>>> SASL SSF: 56 >>>>> SASL data security layer installed. >>>>> # extended LDIF >>>>> # >>>>> # LDAPv3 >>>>> # base (default) with scope subtree >>>>> # filter: uid=bdteg01662 >>>>> # requesting: dn >>>>> # >>>>> >>>>> # bdteg01662, users, accounts, xxx.gob.ve >>>>> dn: uid=bdteg01662,cn=users,cn=accounts,dc=xxx,dc=gob,dc=ve >>>>> >>>>> # search result >>>>> search: 4 >>>>> result: 0 Success >>>>> >>>>> # numResponses: 2 >>>>> # numEntries: 1 >>>>> >>>>> real 0m2.344s >>>>> user 0m0.007s >>>>> sys 0m0.005s >>>> Can you post excerpts from your 389 access log showing the sequence of >>>> operations for this connection, bind and search? >>> Here they are: >>> >>> [01/Aug/2012:10:39:40 -041800] conn=33 fd=70 slot=70 connection from 172.18.32.246 to 172.18.32.246 >>> [01/Aug/2012:10:39:40 -041800] conn=33 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=0 RESULT err=14 tag=97 nentries=0 etime=2, SASL bind in progress >> Yep, this is it - this should not be taking 2 seconds. I'd like to see >> what internal operations are going on in this time. Try this - follow >> the directions at http://port389.org/wiki/FAQ#Troubleshooting but for >> the access log, to turn on Heavy trace output debugging: >> >> dn: cn=config >> changetype: modify >> replace: nsslapd-accesslog-level >> nsslapd-accesslog-level: 4 >> >> Then turn the access log level back to the default (256) after >> reproducing the problem. We should then be able to see the sequence of >> internal operations triggered by the BIND dn="" method=sasl version=3 >> mech=GSSAPI >> > Here they are, with the 2 second delay: > > [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" > [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" > [01/Aug/2012:16:09:48 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=config,cn=userRoot,cn=ldbm database,cn=plugins,cn=config" scope=1 filter="objectclass=vlvsearch" attrs=ALL > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=config,cn=userRoot,cn=ldbm database,cn=plugins,cn=config" scope=1 filter="objectclass=vlvsearch" attrs=ALL > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="cn=anonymous-limits,cn=etc,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="dc=XXXX,dc=XXXX,dc=ve" scope=2 filter="(krbPrincipalName=admin at XXXX.XXXX.VE)" attrs=ALL > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs="nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout" > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 SRCH base="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" scope=0 filter="(|(objectclass=*)(objectclass=ldapsubentry))" attrs=ALL > [01/Aug/2012:16:09:50 -041800] conn=Internal op=-1 RESULT err=0 tag=48 nentries=1 etime=0 > > > Just for kick I tried the search that appears in the log, and it returns > immediately: > > ldapsearch -x -b "cn=anonymous-limits,cn=etc,dc=bt,dc=gob,dc=ve" "(|(objectclass=*)(objectclass=ldapsubentry))" nsLookThroughLimit nsIDListScanLimit nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit nsPagedSizeLimit nsIdleTimeout > > # extended LDIF > # > # LDAPv3 > # base with scope > subtree > # filter: (|(objectclass=*)(objectclass=ldapsubentry)) > # requesting: nsLookThroughLimit nsIDListScanLimit > nsPagedLookThroughLimit nsPagedIDListScanLimit nsSizeLimit nsTimeLimit > nsPagedSizeLimit nsIdleTimeout > # > > # anonymous-limits, etc, bt.gob.ve > dn: cn=anonymous-limits,cn=etc,dc=bt,dc=gob,dc=ve > nsLookThroughLimit: 5000 > nsSizeLimit: 5000 > > # search result > search: 2 > result: 0 Success > > # numResponses: 2 > # numEntries: 1 Thanks - is there a ticket open for this issue? > > >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=admin,cn=users,cn=accounts,dc=XXXX,dc=XXXX,dc=ve" >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=3 SRCH base="dc=XXXX,dc=XXXX,dc=ve" scope=2 filter="(uid=bdteg01662)" attrs=ALL >>> [01/Aug/2012:10:39:42 -041800] conn=33 op=3 RESULT err=0 tag=101 nentries=1 etime=0 >>> >>> >>>>> >>>>> >>>>>> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >>>>>> (you need a kerberos ticket) and telling me if you experience any >>>>>> delay ? >>>>>> If you could run a bunch of searches in a loop and take a wireshark >>>>>> trace that may help analyzing the timings and seeing if there is a >>>>>> correlation. >>>>>> >>>>>> Simo. >>>>>> >>>>> _______________________________________________ >>>>> Freeipa-devel mailing list >>>>> Freeipa-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel From edewata at redhat.com Thu Aug 2 02:42:41 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 01 Aug 2012 21:42:41 -0500 Subject: [Freeipa-devel] [PATCHES] 177-184 Rebase of jquery and jquery-ui libs In-Reply-To: <50193273.6060105@redhat.com> References: <5012C8DE.9090105@redhat.com> <50183338.7060809@redhat.com> <50193273.6060105@redhat.com> Message-ID: <5019E921.60300@redhat.com> On 8/1/2012 8:43 AM, Petr Vobornik wrote: > Silly mistake. I should test builds more often when playing with > files... Updated patch 178 attached. Build and install should work. ACK. -- Endi S. Dewata From edewata at redhat.com Thu Aug 2 02:42:44 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 01 Aug 2012 21:42:44 -0500 Subject: [Freeipa-devel] [PATCH] 185 PAC Type options for services in Web UI In-Reply-To: <501953A7.9080405@redhat.com> References: <501953A7.9080405@redhat.com> Message-ID: <5019E924.3030407@redhat.com> On 8/1/2012 11:04 AM, Petr Vobornik wrote: > Following options were added to Web UI > * PAC Type in service > * PAC Type in configuration > > Testing metadata for objects and commands were regenerated. > > https://fedorahosted.org/freeipa/ticket/2958 ACK. -- Endi S. Dewata From pvoborni at redhat.com Thu Aug 2 08:25:59 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 02 Aug 2012 10:25:59 +0200 Subject: [Freeipa-devel] [PATCH] 185 PAC Type options for services in Web UI In-Reply-To: <5019E924.3030407@redhat.com> References: <501953A7.9080405@redhat.com> <5019E924.3030407@redhat.com> Message-ID: <501A3997.5010202@redhat.com> On 08/02/2012 04:42 AM, Endi Sukma Dewata wrote: > On 8/1/2012 11:04 AM, Petr Vobornik wrote: >> Following options were added to Web UI >> * PAC Type in service >> * PAC Type in configuration >> >> Testing metadata for objects and commands were regenerated. >> >> https://fedorahosted.org/freeipa/ticket/2958 > > ACK. > Pushed to master. -- Petr Vobornik From pvoborni at redhat.com Thu Aug 2 08:40:25 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 02 Aug 2012 10:40:25 +0200 Subject: [Freeipa-devel] [PATCHES] 177-184 Rebase of jquery and jquery-ui libs In-Reply-To: <5019E921.60300@redhat.com> References: <5012C8DE.9090105@redhat.com> <50183338.7060809@redhat.com> <50193273.6060105@redhat.com> <5019E921.60300@redhat.com> Message-ID: <501A3CF9.70608@redhat.com> On 08/02/2012 04:42 AM, Endi Sukma Dewata wrote: > On 8/1/2012 8:43 AM, Petr Vobornik wrote: >> Silly mistake. I should test builds more often when playing with >> files... Updated patch 178 attached. Build and install should work. > > ACK. > Patches 177,178-1,179-184 pushed to master. -- Petr Vobornik From sbose at redhat.com Thu Aug 2 11:25:33 2012 From: sbose at redhat.com (Sumit Bose) Date: Thu, 2 Aug 2012 13:25:33 +0200 Subject: [Freeipa-devel] [PATCHSET] 496 add some PAC verification In-Reply-To: <1342479266.3219.32.camel@willson.li.ssimo.org> References: <1342479266.3219.32.camel@willson.li.ssimo.org> Message-ID: <20120802112533.GD9505@localhost.localdomain> On Mon, Jul 16, 2012 at 06:54:26PM -0400, Simo Sorce wrote: > This patchset is about Ticket #2849 > > The point is to verify that the PAC information we are getting from a > trusted realm is actually consistent with the information we know about > that trust relationship. > > The patchset adds a way to load trust information in the kdb driver > (first 2 patches), reorganizes a bit the code around PAC verification > and adds a filtering function to match realm with AD and SID data. > > Tested on my trust environment and seem to work fine. Works for me as well. ACK. bye, Sumit > > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Thu Aug 2 12:00:57 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 02 Aug 2012 14:00:57 +0200 Subject: [Freeipa-devel] [PATCH] 291 Avoid redundant info message during RPM update In-Reply-To: <5017CE24.4090102@redhat.com> References: <5017CE24.4090102@redhat.com> Message-ID: <501A6BF9.8030200@redhat.com> On 07/31/2012 02:23 PM, Martin Kosek wrote: > This is just s short patch fixing a regression introduced in ticket 2652. I > would like to get this one to beta 2. > > -- > A change to ipa-ldap-updater (and thus an RPM update %post scriptlet) > avoiding redundat "IPA is not configured" message in stderr introdocued > in c20d4c71b87365b3b8d9c53418a79f992e68cd00 was reverted in another > patch (b5c1ce88a4a3b35adb3b22bc68fb10b49322641a). > > Return the change back to avoid this message during every RPM update > when IPA is not configured. admintool framework was also fixed to > avoid print an empty line when an exception without an error message > is raised. > > https://fedorahosted.org/freeipa/ticket/2892 > I just noticed that the error message may not be printed at all in same cases. Sending a fixed version. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-291-2-avoid-redundant-info-message-during-rpm-update.patch Type: text/x-patch Size: 2650 bytes Desc: not available URL: From simo at redhat.com Thu Aug 2 12:56:31 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 02 Aug 2012 08:56:31 -0400 Subject: [Freeipa-devel] Strange issue I keep hitting with invalid tickets In-Reply-To: <50197A62.9080007@redhat.com> References: <5018530E.70105@redhat.com> <1343831843.20530.81.camel@willson.li.ssimo.org> <50197A62.9080007@redhat.com> Message-ID: <1343912191.20530.111.camel@willson.li.ssimo.org> On Wed, 2012-08-01 at 11:50 -0700, Michael Gregg wrote: > On 08/01/2012 07:37 AM, Simo Sorce wrote: > > On Tue, 2012-07-31 at 14:50 -0700, Michael Gregg wrote: > >> I am not sure why, but when I let my ipa machines sit around for a > >> while(overnight-24hours), and then kinit. When I try to run IPA commands > >> I get output like this: > >> > >> [root at zippyvm12 ~]# ipa host-find > >> ipa: ERROR: Local error: SASL(-1): generic failure: GSSAPI Error: > >> Unspecified GSS failure. Minor code may provide more information > >> (Ticket not yet valid) > >> > >> This issue seems to be addressed here: > >> > >> https://access.redhat.com/knowledge/solutions/133433 > >> > >> It's strange, because when I kinit again, I seem to have a valid > >> credentials, like here: > >> > >> [root at zippyvm12 ~]# klist > >> Ticket cache: FILE:/tmp/krb5cc_0 > >> Default principal: admin at TESTRELM.COM > >> Valid starting Expires Service principal > >> 07/31/12 17:31:16 08/01/12 17:31:14 krbtgt/TESTRELM.COM at TESTRELM.COM > >> 07/31/12 17:32:39 08/01/12 17:31:14 > >> HTTP/zippyvm12.testrelm.com at TESTRELM.COM > >> > >> > >> The work around for me seems to be deleting /tmp/krb5* > >> Then, I kinit again, and it all starts to work again. > >> > >> My question is, why is this happening? Any ideas? > > On what distro/krb5 libs version ? > > > > We fixed a bug where krb5 was badly using the timestamp in the cache and > > thus sometimes failing to properly set the clock skew in the request. > > > > You may be falling for the same bug (normally you'd see this with > > krb5-auth-dialog when it tried to renew tickets). > > > > Simo. > > > distro is rhel6.3 > krb5-libs-1.9-33.el6.x86_64 > > Was the fix included later than this version? > > Michael- > This is the bug I am thinking of: https://bugzilla.redhat.com/show_bug.cgi?id=773496 Apparently it is scheduled for 6.4, and not yet fixed in 6.3 Simo. -- Simo Sorce * Red Hat, Inc * New York From jcholast at redhat.com Thu Aug 2 14:00:33 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 02 Aug 2012 16:00:33 +0200 Subject: [Freeipa-devel] [PATCH] 291 Avoid redundant info message during RPM update In-Reply-To: <501A6BF9.8030200@redhat.com> References: <5017CE24.4090102@redhat.com> <501A6BF9.8030200@redhat.com> Message-ID: <501A8801.9060302@redhat.com> Dne 2.8.2012 14:00, Martin Kosek napsal(a): > On 07/31/2012 02:23 PM, Martin Kosek wrote: >> This is just s short patch fixing a regression introduced in ticket 2652. I >> would like to get this one to beta 2. >> >> -- >> A change to ipa-ldap-updater (and thus an RPM update %post scriptlet) >> avoiding redundat "IPA is not configured" message in stderr introdocued >> in c20d4c71b87365b3b8d9c53418a79f992e68cd00 was reverted in another >> patch (b5c1ce88a4a3b35adb3b22bc68fb10b49322641a). >> >> Return the change back to avoid this message during every RPM update >> when IPA is not configured. admintool framework was also fixed to >> avoid print an empty line when an exception without an error message >> is raised. >> >> https://fedorahosted.org/freeipa/ticket/2892 >> > > I just noticed that the error message may not be printed at all in same cases. > Sending a fixed version. > > Martin > ACK. Honza -- Jan Cholasta From mkosek at redhat.com Thu Aug 2 14:37:32 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 02 Aug 2012 16:37:32 +0200 Subject: [Freeipa-devel] [PATCH] 292 Bump bind-dyndb-ldap version for F18 Message-ID: <501A90AC.3080707@redhat.com> bind-dyndb-ldap with SOA serial autoincrement was released. Bump the package version in the spec file. The version is bumped for F18 only as it was released only to rawhide and we don't want to break development on F17. https://fedorahosted.org/freeipa/ticket/2554 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-292-bump-bind-dyndb-ldap-version-for-f18.patch Type: text/x-patch Size: 1564 bytes Desc: not available URL: From simo at redhat.com Thu Aug 2 15:18:36 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 02 Aug 2012 11:18:36 -0400 Subject: [Freeipa-devel] [PATCH] 292 Bump bind-dyndb-ldap version for F18 In-Reply-To: <501A90AC.3080707@redhat.com> References: <501A90AC.3080707@redhat.com> Message-ID: <1343920716.20530.126.camel@willson.li.ssimo.org> On Thu, 2012-08-02 at 16:37 +0200, Martin Kosek wrote: > bind-dyndb-ldap with SOA serial autoincrement was released. Bump > the package version in the spec file. The version is bumped for > F18 only as it was released only to rawhide and we don't want to > break development on F17. > > https://fedorahosted.org/freeipa/ticket/2554 ACK Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Thu Aug 2 15:24:05 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 02 Aug 2012 17:24:05 +0200 Subject: [Freeipa-devel] [PATCH] 292 Bump bind-dyndb-ldap version for F18 In-Reply-To: <1343920716.20530.126.camel@willson.li.ssimo.org> References: <501A90AC.3080707@redhat.com> <1343920716.20530.126.camel@willson.li.ssimo.org> Message-ID: <501A9B95.1090302@redhat.com> On 08/02/2012 05:18 PM, Simo Sorce wrote: > On Thu, 2012-08-02 at 16:37 +0200, Martin Kosek wrote: >> bind-dyndb-ldap with SOA serial autoincrement was released. Bump >> the package version in the spec file. The version is bumped for >> F18 only as it was released only to rawhide and we don't want to >> break development on F17. >> >> https://fedorahosted.org/freeipa/ticket/2554 > > ACK > > Simo. > Pushed to master. Martin From tbabej at redhat.com Thu Aug 2 15:26:47 2012 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 2 Aug 2012 11:26:47 -0400 (EDT) Subject: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py In-Reply-To: <50127A4A.3020200@redhat.com> Message-ID: <983279605.1707957.1343921207471.JavaMail.root@redhat.com> I implemented your suggestions. New version of the patch attached. ----- Original Message ----- From: "Petr Viktorin" To: freeipa-devel at redhat.com Sent: Friday, July 27, 2012 1:23:54 PM Subject: Re: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py On 07/27/2012 01:13 PM, Tomas Babej wrote: > Hi, > > this patch simply checks if ipa-join exists in ipa-client folder, if not, skips tests relying on it. > Uses nose.plugin.skip. > > https://fedorahosted.org/freeipa/ticket/2905 > > Tomas Babej > Hello, This would be better done in class setup, so you don't have to repeat it in every method. Look at XMLRPC_test.setUpClass() in xmlrpc_test.py for isnpiration. While you're at it, it would be good to move the mkstemp() call into setUpClass as well, so that importing the module doesn't have unnecessary side effects. -- Petr? _______________________________________________ Freeipa-devel mailing list Freeipa-devel at redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0001-2-Adds-check-for-existence-of-ipa-client-ipa-join-exec.patch Type: text/x-patch Size: 2561 bytes Desc: not available URL: From simo at redhat.com Thu Aug 2 15:28:59 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 02 Aug 2012 11:28:59 -0400 Subject: [Freeipa-devel] [PATCHSET] 496 add some PAC verification In-Reply-To: <20120802112533.GD9505@localhost.localdomain> References: <1342479266.3219.32.camel@willson.li.ssimo.org> <20120802112533.GD9505@localhost.localdomain> Message-ID: <1343921339.20530.129.camel@willson.li.ssimo.org> On Thu, 2012-08-02 at 13:25 +0200, Sumit Bose wrote: > On Mon, Jul 16, 2012 at 06:54:26PM -0400, Simo Sorce wrote: > > This patchset is about Ticket #2849 > > > > The point is to verify that the PAC information we are getting from a > > trusted realm is actually consistent with the information we know about > > that trust relationship. > > > > The patchset adds a way to load trust information in the kdb driver > > (first 2 patches), reorganizes a bit the code around PAC verification > > and adds a filtering function to match realm with AD and SID data. > > > > Tested on my trust environment and seem to work fine. > > Works for me as well. ACK. Pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From sbingram at gmail.com Thu Aug 2 17:17:14 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Thu, 2 Aug 2012 10:17:14 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <1343831702.20530.79.camel@willson.li.ssimo.org> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> Message-ID: On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: > On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >> > On 07/31/2012 12:27 AM, John Dennis wrote: >> >> >> >> >> >> What is taking so long with session bookkeeping? I don't know yet. I would >> >> need more timing instrumentation. I will say when I looked at the >> >> python-krb5 >> >> code (which we use to populate the ccache from the session and read back >> >> to >> >> store in the session) seemed to be remarkably inefficient. We also elected >> >> to >> >> use file based ccache rather than in-memory ccache (that means there is a >> >> bit >> >> of file-IO occurring). >> > >> > >> > A note regarding python-krbV: >> > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >> > can obtain several thousands of TGTs per second (even with ccache in a >> > file). AFAIK VFS calls are not done synchronously. But others parts of >> > python-krbV were left uncovered, so it can contain some surprises. >> > >> > === Wild speculation follows === >> > 1.5 second is incredibly long time, it sounds like some kind of timeout. >> > Krb5 libs have usual timeout = 1 second per request. >> > >> > Are all KDCs in /etc/krb5.conf alive and reachable? >> >> In this case, as I'm referring to the extreme slowness of the Web UI, >> the KDC is on the same system (the ipa server) that is making the >> request, correct? >> >> > Is SSSD running on problematic server? >> >> Yes. Again, I'm guessing the problematic server is the IPA server itself. >> >> > Is proper KDC selected by SSSD KDC auto-locator plugin? >> > (See /var/lib/sss/pubconf/) >> >> Yes, I checked that file and it is the IP address of the IPA server on >> the same server. Perhaps should this be 127.0.0.1 instead? >> >> I also have checked the resolv.conf file, and indeed the IP points to >> the IPA server itself (same machine) as expected. Both forward and >> reverse DNS work. I'm not really sure what else could be setup >> incorrectly to cause any KDC slowness. >> >> Due to the extreme UI slowness issue, I have not created any replicas >> so this system is it. I'm not so sure I would be able to see the 1.5 s >> delay if it weren't compounded by the overall slowness of the Web UI, >> however, the KDC seems to perform well for other systems in the realm. >> I'm certainly not taxing it with a huge load, but tickets seem to be >> issued without delay. > > Stephen, > another user sent me a wireshark trace for a similar performance issue. > So far I see a pause when doing the first leg of a SASL authentication. > This may well explain also your issue. > > Can you test connecting to the ldap server using ldapsearch -Y GSSAPI > (you need a kerberos ticket) and telling me if you experience any > delay ? > If you could run a bunch of searches in a loop and take a wireshark > trace that may help analyzing the timings and seeing if there is a > correlation. I've done this. It looks like this delay has been uncovered already though? I can still send you the dump privately if you think it would help. Steve From rmeggins at redhat.com Thu Aug 2 17:22:17 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 02 Aug 2012 11:22:17 -0600 Subject: [Freeipa-devel] slow response In-Reply-To: References: <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> Message-ID: <501AB749.3070304@redhat.com> On 08/02/2012 11:17 AM, Stephen Ingram wrote: > On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: >> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>> >>>>> What is taking so long with session bookkeeping? I don't know yet. I would >>>>> need more timing instrumentation. I will say when I looked at the >>>>> python-krb5 >>>>> code (which we use to populate the ccache from the session and read back >>>>> to >>>>> store in the session) seemed to be remarkably inefficient. We also elected >>>>> to >>>>> use file based ccache rather than in-memory ccache (that means there is a >>>>> bit >>>>> of file-IO occurring). >>>> >>>> A note regarding python-krbV: >>>> I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >>>> can obtain several thousands of TGTs per second (even with ccache in a >>>> file). AFAIK VFS calls are not done synchronously. But others parts of >>>> python-krbV were left uncovered, so it can contain some surprises. >>>> >>>> === Wild speculation follows === >>>> 1.5 second is incredibly long time, it sounds like some kind of timeout. >>>> Krb5 libs have usual timeout = 1 second per request. >>>> >>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>> In this case, as I'm referring to the extreme slowness of the Web UI, >>> the KDC is on the same system (the ipa server) that is making the >>> request, correct? >>> >>>> Is SSSD running on problematic server? >>> Yes. Again, I'm guessing the problematic server is the IPA server itself. >>> >>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>> (See /var/lib/sss/pubconf/) >>> Yes, I checked that file and it is the IP address of the IPA server on >>> the same server. Perhaps should this be 127.0.0.1 instead? >>> >>> I also have checked the resolv.conf file, and indeed the IP points to >>> the IPA server itself (same machine) as expected. Both forward and >>> reverse DNS work. I'm not really sure what else could be setup >>> incorrectly to cause any KDC slowness. >>> >>> Due to the extreme UI slowness issue, I have not created any replicas >>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>> delay if it weren't compounded by the overall slowness of the Web UI, >>> however, the KDC seems to perform well for other systems in the realm. >>> I'm certainly not taxing it with a huge load, but tickets seem to be >>> issued without delay. >> Stephen, >> another user sent me a wireshark trace for a similar performance issue. >> So far I see a pause when doing the first leg of a SASL authentication. >> This may well explain also your issue. >> >> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >> (you need a kerberos ticket) and telling me if you experience any >> delay ? >> If you could run a bunch of searches in a loop and take a wireshark >> trace that may help analyzing the timings and seeing if there is a >> correlation. > I've done this. It looks like this delay has been uncovered already > though? I can still send you the dump privately if you think it would > help. We have a "somewhat" reproducible test case - see https://bugzilla.redhat.com/show_bug.cgi?id=845125 - note that the problem only seems to occur on RHEL 6.3 with the latest Z-stream packages. > > Steve From simo at redhat.com Thu Aug 2 17:23:29 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 02 Aug 2012 13:23:29 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> Message-ID: <1343928209.20530.132.camel@willson.li.ssimo.org> On Thu, 2012-08-02 at 10:17 -0700, Stephen Ingram wrote: > On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: > > On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: > >> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: > >> > On 07/31/2012 12:27 AM, John Dennis wrote: > >> >> > >> >> > >> >> What is taking so long with session bookkeeping? I don't know yet. I would > >> >> need more timing instrumentation. I will say when I looked at the > >> >> python-krb5 > >> >> code (which we use to populate the ccache from the session and read back > >> >> to > >> >> store in the session) seemed to be remarkably inefficient. We also elected > >> >> to > >> >> use file based ccache rather than in-memory ccache (that means there is a > >> >> bit > >> >> of file-IO occurring). > >> > > >> > > >> > A note regarding python-krbV: > >> > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV > >> > can obtain several thousands of TGTs per second (even with ccache in a > >> > file). AFAIK VFS calls are not done synchronously. But others parts of > >> > python-krbV were left uncovered, so it can contain some surprises. > >> > > >> > === Wild speculation follows === > >> > 1.5 second is incredibly long time, it sounds like some kind of timeout. > >> > Krb5 libs have usual timeout = 1 second per request. > >> > > >> > Are all KDCs in /etc/krb5.conf alive and reachable? > >> > >> In this case, as I'm referring to the extreme slowness of the Web UI, > >> the KDC is on the same system (the ipa server) that is making the > >> request, correct? > >> > >> > Is SSSD running on problematic server? > >> > >> Yes. Again, I'm guessing the problematic server is the IPA server itself. > >> > >> > Is proper KDC selected by SSSD KDC auto-locator plugin? > >> > (See /var/lib/sss/pubconf/) > >> > >> Yes, I checked that file and it is the IP address of the IPA server on > >> the same server. Perhaps should this be 127.0.0.1 instead? > >> > >> I also have checked the resolv.conf file, and indeed the IP points to > >> the IPA server itself (same machine) as expected. Both forward and > >> reverse DNS work. I'm not really sure what else could be setup > >> incorrectly to cause any KDC slowness. > >> > >> Due to the extreme UI slowness issue, I have not created any replicas > >> so this system is it. I'm not so sure I would be able to see the 1.5 s > >> delay if it weren't compounded by the overall slowness of the Web UI, > >> however, the KDC seems to perform well for other systems in the realm. > >> I'm certainly not taxing it with a huge load, but tickets seem to be > >> issued without delay. > > > > Stephen, > > another user sent me a wireshark trace for a similar performance issue. > > So far I see a pause when doing the first leg of a SASL authentication. > > This may well explain also your issue. > > > > Can you test connecting to the ldap server using ldapsearch -Y GSSAPI > > (you need a kerberos ticket) and telling me if you experience any > > delay ? > > If you could run a bunch of searches in a loop and take a wireshark > > trace that may help analyzing the timings and seeing if there is a > > correlation. > > I've done this. It looks like this delay has been uncovered already > though? I can still send you the dump privately if you think it would > help. I think we reproduced it, can you confirm you are also running on RHEL ? So far it seem the only platfrom we can reproduce is RHEL 6.3 Simo. -- Simo Sorce * Red Hat, Inc * New York From sbingram at gmail.com Thu Aug 2 17:29:31 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Thu, 2 Aug 2012 10:29:31 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <1343928209.20530.132.camel@willson.li.ssimo.org> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> Message-ID: On Thu, Aug 2, 2012 at 10:23 AM, Simo Sorce wrote: > On Thu, 2012-08-02 at 10:17 -0700, Stephen Ingram wrote: >> On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: >> > On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >> >> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >> >> > On 07/31/2012 12:27 AM, John Dennis wrote: >> >> >> >> >> >> >> >> >> What is taking so long with session bookkeeping? I don't know yet. I would >> >> >> need more timing instrumentation. I will say when I looked at the >> >> >> python-krb5 >> >> >> code (which we use to populate the ccache from the session and read back >> >> >> to >> >> >> store in the session) seemed to be remarkably inefficient. We also elected >> >> >> to >> >> >> use file based ccache rather than in-memory ccache (that means there is a >> >> >> bit >> >> >> of file-IO occurring). >> >> > >> >> > >> >> > A note regarding python-krbV: >> >> > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >> >> > can obtain several thousands of TGTs per second (even with ccache in a >> >> > file). AFAIK VFS calls are not done synchronously. But others parts of >> >> > python-krbV were left uncovered, so it can contain some surprises. >> >> > >> >> > === Wild speculation follows === >> >> > 1.5 second is incredibly long time, it sounds like some kind of timeout. >> >> > Krb5 libs have usual timeout = 1 second per request. >> >> > >> >> > Are all KDCs in /etc/krb5.conf alive and reachable? >> >> >> >> In this case, as I'm referring to the extreme slowness of the Web UI, >> >> the KDC is on the same system (the ipa server) that is making the >> >> request, correct? >> >> >> >> > Is SSSD running on problematic server? >> >> >> >> Yes. Again, I'm guessing the problematic server is the IPA server itself. >> >> >> >> > Is proper KDC selected by SSSD KDC auto-locator plugin? >> >> > (See /var/lib/sss/pubconf/) >> >> >> >> Yes, I checked that file and it is the IP address of the IPA server on >> >> the same server. Perhaps should this be 127.0.0.1 instead? >> >> >> >> I also have checked the resolv.conf file, and indeed the IP points to >> >> the IPA server itself (same machine) as expected. Both forward and >> >> reverse DNS work. I'm not really sure what else could be setup >> >> incorrectly to cause any KDC slowness. >> >> >> >> Due to the extreme UI slowness issue, I have not created any replicas >> >> so this system is it. I'm not so sure I would be able to see the 1.5 s >> >> delay if it weren't compounded by the overall slowness of the Web UI, >> >> however, the KDC seems to perform well for other systems in the realm. >> >> I'm certainly not taxing it with a huge load, but tickets seem to be >> >> issued without delay. >> > >> > Stephen, >> > another user sent me a wireshark trace for a similar performance issue. >> > So far I see a pause when doing the first leg of a SASL authentication. >> > This may well explain also your issue. >> > >> > Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >> > (you need a kerberos ticket) and telling me if you experience any >> > delay ? >> > If you could run a bunch of searches in a loop and take a wireshark >> > trace that may help analyzing the timings and seeing if there is a >> > correlation. >> >> I've done this. It looks like this delay has been uncovered already >> though? I can still send you the dump privately if you think it would >> help. > > I think we reproduced it, can you confirm you are also running on RHEL ? > So far it seem the only platfrom we can reproduce is RHEL 6.3 Yes, I'm running RHEL 6.3. I just ran the command in the BZ and it takes 1.542s for me. What are Z-stream packages? Is this new for 389ds? Steve From rmeggins at redhat.com Thu Aug 2 17:33:00 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 02 Aug 2012 11:33:00 -0600 Subject: [Freeipa-devel] slow response In-Reply-To: References: <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> Message-ID: <501AB9CC.1040509@redhat.com> On 08/02/2012 11:29 AM, Stephen Ingram wrote: > On Thu, Aug 2, 2012 at 10:23 AM, Simo Sorce wrote: >> On Thu, 2012-08-02 at 10:17 -0700, Stephen Ingram wrote: >>> On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: >>>> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>>>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: >>>>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>>>> >>>>>>> What is taking so long with session bookkeeping? I don't know yet. I would >>>>>>> need more timing instrumentation. I will say when I looked at the >>>>>>> python-krb5 >>>>>>> code (which we use to populate the ccache from the session and read back >>>>>>> to >>>>>>> store in the session) seemed to be remarkably inefficient. We also elected >>>>>>> to >>>>>>> use file based ccache rather than in-memory ccache (that means there is a >>>>>>> bit >>>>>>> of file-IO occurring). >>>>>> >>>>>> A note regarding python-krbV: >>>>>> I used python-krbV extensively in my thesis for KDC stress test. Python-krbV >>>>>> can obtain several thousands of TGTs per second (even with ccache in a >>>>>> file). AFAIK VFS calls are not done synchronously. But others parts of >>>>>> python-krbV were left uncovered, so it can contain some surprises. >>>>>> >>>>>> === Wild speculation follows === >>>>>> 1.5 second is incredibly long time, it sounds like some kind of timeout. >>>>>> Krb5 libs have usual timeout = 1 second per request. >>>>>> >>>>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>>>> In this case, as I'm referring to the extreme slowness of the Web UI, >>>>> the KDC is on the same system (the ipa server) that is making the >>>>> request, correct? >>>>> >>>>>> Is SSSD running on problematic server? >>>>> Yes. Again, I'm guessing the problematic server is the IPA server itself. >>>>> >>>>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>>>> (See /var/lib/sss/pubconf/) >>>>> Yes, I checked that file and it is the IP address of the IPA server on >>>>> the same server. Perhaps should this be 127.0.0.1 instead? >>>>> >>>>> I also have checked the resolv.conf file, and indeed the IP points to >>>>> the IPA server itself (same machine) as expected. Both forward and >>>>> reverse DNS work. I'm not really sure what else could be setup >>>>> incorrectly to cause any KDC slowness. >>>>> >>>>> Due to the extreme UI slowness issue, I have not created any replicas >>>>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>>>> delay if it weren't compounded by the overall slowness of the Web UI, >>>>> however, the KDC seems to perform well for other systems in the realm. >>>>> I'm certainly not taxing it with a huge load, but tickets seem to be >>>>> issued without delay. >>>> Stephen, >>>> another user sent me a wireshark trace for a similar performance issue. >>>> So far I see a pause when doing the first leg of a SASL authentication. >>>> This may well explain also your issue. >>>> >>>> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >>>> (you need a kerberos ticket) and telling me if you experience any >>>> delay ? >>>> If you could run a bunch of searches in a loop and take a wireshark >>>> trace that may help analyzing the timings and seeing if there is a >>>> correlation. >>> I've done this. It looks like this delay has been uncovered already >>> though? I can still send you the dump privately if you think it would >>> help. >> I think we reproduced it, can you confirm you are also running on RHEL ? >> So far it seem the only platfrom we can reproduce is RHEL 6.3 > > Yes, I'm running RHEL 6.3. I just ran the command in the BZ and it > takes 1.542s for me. What are Z-stream packages? They are packages delivered between minor RHEL releases e.g. between RHEL 6.3 and RHEL 6.4 - the 389-ds-base package released with RHEL 6.3 was 389-ds-base-1.2.10.2-15.el6 - since RHEL 6.3, we released some bugfix packages, and the latest is now 389-ds-base-1.2.10.2-20.el6_3 - note the "_3" at the end - this means it is a bugfix release for RHEL 6.3 - "Z-stream" is just Red Hat terminology for a package released between minor RHEL releases - the "Z" stands for the "Z" in the versioning scheme "X.Y.Z" > Is this new for > 389ds? > > Steve From sbingram at gmail.com Thu Aug 2 17:56:56 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Thu, 2 Aug 2012 10:56:56 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <501AB9CC.1040509@redhat.com> References: <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> <501AB9CC.1040509@redhat.com> Message-ID: On Thu, Aug 2, 2012 at 10:33 AM, Rich Megginson wrote: > On 08/02/2012 11:29 AM, Stephen Ingram wrote: >> >> On Thu, Aug 2, 2012 at 10:23 AM, Simo Sorce wrote: >>> >>> On Thu, 2012-08-02 at 10:17 -0700, Stephen Ingram wrote: >>>> >>>> On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: >>>>> >>>>> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>>>>> >>>>>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek >>>>>> wrote: >>>>>>> >>>>>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>>>>> >>>>>>>> >>>>>>>> What is taking so long with session bookkeeping? I don't know yet. I >>>>>>>> would >>>>>>>> need more timing instrumentation. I will say when I looked at the >>>>>>>> python-krb5 >>>>>>>> code (which we use to populate the ccache from the session and read >>>>>>>> back >>>>>>>> to >>>>>>>> store in the session) seemed to be remarkably inefficient. We also >>>>>>>> elected >>>>>>>> to >>>>>>>> use file based ccache rather than in-memory ccache (that means there >>>>>>>> is a >>>>>>>> bit >>>>>>>> of file-IO occurring). >>>>>>> >>>>>>> >>>>>>> A note regarding python-krbV: >>>>>>> I used python-krbV extensively in my thesis for KDC stress test. >>>>>>> Python-krbV >>>>>>> can obtain several thousands of TGTs per second (even with ccache in >>>>>>> a >>>>>>> file). AFAIK VFS calls are not done synchronously. But others parts >>>>>>> of >>>>>>> python-krbV were left uncovered, so it can contain some surprises. >>>>>>> >>>>>>> === Wild speculation follows === >>>>>>> 1.5 second is incredibly long time, it sounds like some kind of >>>>>>> timeout. >>>>>>> Krb5 libs have usual timeout = 1 second per request. >>>>>>> >>>>>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>>>>> >>>>>> In this case, as I'm referring to the extreme slowness of the Web UI, >>>>>> the KDC is on the same system (the ipa server) that is making the >>>>>> request, correct? >>>>>> >>>>>>> Is SSSD running on problematic server? >>>>>> >>>>>> Yes. Again, I'm guessing the problematic server is the IPA server >>>>>> itself. >>>>>> >>>>>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>>>>> (See /var/lib/sss/pubconf/) >>>>>> >>>>>> Yes, I checked that file and it is the IP address of the IPA server on >>>>>> the same server. Perhaps should this be 127.0.0.1 instead? >>>>>> >>>>>> I also have checked the resolv.conf file, and indeed the IP points to >>>>>> the IPA server itself (same machine) as expected. Both forward and >>>>>> reverse DNS work. I'm not really sure what else could be setup >>>>>> incorrectly to cause any KDC slowness. >>>>>> >>>>>> Due to the extreme UI slowness issue, I have not created any replicas >>>>>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>>>>> delay if it weren't compounded by the overall slowness of the Web UI, >>>>>> however, the KDC seems to perform well for other systems in the realm. >>>>>> I'm certainly not taxing it with a huge load, but tickets seem to be >>>>>> issued without delay. >>>>> >>>>> Stephen, >>>>> another user sent me a wireshark trace for a similar performance issue. >>>>> So far I see a pause when doing the first leg of a SASL authentication. >>>>> This may well explain also your issue. >>>>> >>>>> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >>>>> (you need a kerberos ticket) and telling me if you experience any >>>>> delay ? >>>>> If you could run a bunch of searches in a loop and take a wireshark >>>>> trace that may help analyzing the timings and seeing if there is a >>>>> correlation. >>>> >>>> I've done this. It looks like this delay has been uncovered already >>>> though? I can still send you the dump privately if you think it would >>>> help. >>> >>> I think we reproduced it, can you confirm you are also running on RHEL ? >>> So far it seem the only platfrom we can reproduce is RHEL 6.3 >> >> >> Yes, I'm running RHEL 6.3. I just ran the command in the BZ and it >> takes 1.542s for me. What are Z-stream packages? > > > They are packages delivered between minor RHEL releases e.g. between RHEL > 6.3 and RHEL 6.4 - the 389-ds-base package released with RHEL 6.3 was > 389-ds-base-1.2.10.2-15.el6 - since RHEL 6.3, we released some bugfix > packages, and the latest is now 389-ds-base-1.2.10.2-20.el6_3 - note the > "_3" at the end - this means it is a bugfix release for RHEL 6.3 - > "Z-stream" is just Red Hat terminology for a package released between minor > RHEL releases - the "Z" stands for the "Z" in the versioning scheme "X.Y.Z" Got it. We are using those packages now. Although this might or might not be the cause of the slow Web UI, the Web UI has been slow since the initial 6.3 release. If this 389 slowness was only introduced in the Z-stream 389ds, then it is likely not, or not the only, cause of the Web UI slowness. Steve From rmeggins at redhat.com Thu Aug 2 18:06:30 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 02 Aug 2012 12:06:30 -0600 Subject: [Freeipa-devel] slow response In-Reply-To: References: <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> <501AB9CC.1040509@redhat.com> Message-ID: <501AC1A6.3090600@redhat.com> On 08/02/2012 11:56 AM, Stephen Ingram wrote: > On Thu, Aug 2, 2012 at 10:33 AM, Rich Megginson wrote: >> On 08/02/2012 11:29 AM, Stephen Ingram wrote: >>> On Thu, Aug 2, 2012 at 10:23 AM, Simo Sorce wrote: >>>> On Thu, 2012-08-02 at 10:17 -0700, Stephen Ingram wrote: >>>>> On Wed, Aug 1, 2012 at 7:35 AM, Simo Sorce wrote: >>>>>> On Tue, 2012-07-31 at 08:49 -0700, Stephen Ingram wrote: >>>>>>> On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek >>>>>>> wrote: >>>>>>>> On 07/31/2012 12:27 AM, John Dennis wrote: >>>>>>>>> >>>>>>>>> What is taking so long with session bookkeeping? I don't know yet. I >>>>>>>>> would >>>>>>>>> need more timing instrumentation. I will say when I looked at the >>>>>>>>> python-krb5 >>>>>>>>> code (which we use to populate the ccache from the session and read >>>>>>>>> back >>>>>>>>> to >>>>>>>>> store in the session) seemed to be remarkably inefficient. We also >>>>>>>>> elected >>>>>>>>> to >>>>>>>>> use file based ccache rather than in-memory ccache (that means there >>>>>>>>> is a >>>>>>>>> bit >>>>>>>>> of file-IO occurring). >>>>>>>> >>>>>>>> A note regarding python-krbV: >>>>>>>> I used python-krbV extensively in my thesis for KDC stress test. >>>>>>>> Python-krbV >>>>>>>> can obtain several thousands of TGTs per second (even with ccache in >>>>>>>> a >>>>>>>> file). AFAIK VFS calls are not done synchronously. But others parts >>>>>>>> of >>>>>>>> python-krbV were left uncovered, so it can contain some surprises. >>>>>>>> >>>>>>>> === Wild speculation follows === >>>>>>>> 1.5 second is incredibly long time, it sounds like some kind of >>>>>>>> timeout. >>>>>>>> Krb5 libs have usual timeout = 1 second per request. >>>>>>>> >>>>>>>> Are all KDCs in /etc/krb5.conf alive and reachable? >>>>>>> In this case, as I'm referring to the extreme slowness of the Web UI, >>>>>>> the KDC is on the same system (the ipa server) that is making the >>>>>>> request, correct? >>>>>>> >>>>>>>> Is SSSD running on problematic server? >>>>>>> Yes. Again, I'm guessing the problematic server is the IPA server >>>>>>> itself. >>>>>>> >>>>>>>> Is proper KDC selected by SSSD KDC auto-locator plugin? >>>>>>>> (See /var/lib/sss/pubconf/) >>>>>>> Yes, I checked that file and it is the IP address of the IPA server on >>>>>>> the same server. Perhaps should this be 127.0.0.1 instead? >>>>>>> >>>>>>> I also have checked the resolv.conf file, and indeed the IP points to >>>>>>> the IPA server itself (same machine) as expected. Both forward and >>>>>>> reverse DNS work. I'm not really sure what else could be setup >>>>>>> incorrectly to cause any KDC slowness. >>>>>>> >>>>>>> Due to the extreme UI slowness issue, I have not created any replicas >>>>>>> so this system is it. I'm not so sure I would be able to see the 1.5 s >>>>>>> delay if it weren't compounded by the overall slowness of the Web UI, >>>>>>> however, the KDC seems to perform well for other systems in the realm. >>>>>>> I'm certainly not taxing it with a huge load, but tickets seem to be >>>>>>> issued without delay. >>>>>> Stephen, >>>>>> another user sent me a wireshark trace for a similar performance issue. >>>>>> So far I see a pause when doing the first leg of a SASL authentication. >>>>>> This may well explain also your issue. >>>>>> >>>>>> Can you test connecting to the ldap server using ldapsearch -Y GSSAPI >>>>>> (you need a kerberos ticket) and telling me if you experience any >>>>>> delay ? >>>>>> If you could run a bunch of searches in a loop and take a wireshark >>>>>> trace that may help analyzing the timings and seeing if there is a >>>>>> correlation. >>>>> I've done this. It looks like this delay has been uncovered already >>>>> though? I can still send you the dump privately if you think it would >>>>> help. >>>> I think we reproduced it, can you confirm you are also running on RHEL ? >>>> So far it seem the only platfrom we can reproduce is RHEL 6.3 >>> >>> Yes, I'm running RHEL 6.3. I just ran the command in the BZ and it >>> takes 1.542s for me. What are Z-stream packages? >> >> They are packages delivered between minor RHEL releases e.g. between RHEL >> 6.3 and RHEL 6.4 - the 389-ds-base package released with RHEL 6.3 was >> 389-ds-base-1.2.10.2-15.el6 - since RHEL 6.3, we released some bugfix >> packages, and the latest is now 389-ds-base-1.2.10.2-20.el6_3 - note the >> "_3" at the end - this means it is a bugfix release for RHEL 6.3 - >> "Z-stream" is just Red Hat terminology for a package released between minor >> RHEL releases - the "Z" stands for the "Z" in the versioning scheme "X.Y.Z" > Got it. We are using those packages now. Although this might or might > not be the cause of the slow Web UI, the Web UI has been slow since > the initial 6.3 release. If this 389 slowness was only introduced in > the Z-stream 389ds, then it is likely not, or not the only, cause of > the Web UI slowness. The 389 slowness was not introduced in the Z-stream package (-20.el6_3) - the slowness is present in the RHEL 6.3.0 package (-15.el6) > > Steve From simo at redhat.com Thu Aug 2 20:04:02 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 02 Aug 2012 16:04:02 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: <501AC1A6.3090600@redhat.com> References: <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> <501AB9CC.1040509@redhat.com> <501AC1A6.3090600@redhat.com> Message-ID: <1343937842.20530.150.camel@willson.li.ssimo.org> Quick heads up in this thread, apparently we have isolated the issue to libkrb5 and its selinux integration. I have made the whole delay go away by disabling selinux entirely (which makes libkrb5 stop trying to do some selinux related operations). We will be working on a fix to have libkrb5 not cause this delay (F17 doesn't have it). You can follow progress on this bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=845125 Simo. -- Simo Sorce * Red Hat, Inc * New York From jcholast at redhat.com Fri Aug 3 07:28:31 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 03 Aug 2012 09:28:31 +0200 Subject: [Freeipa-devel] [PATCH] 80 Add --{set, add, del}attr options to commands which are missing them Message-ID: <501B7D9F.7050008@redhat.com> Hi, this patch fixes . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-80-add-missing-set-add-del-attr.patch Type: text/x-patch Size: 8157 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 3 08:21:30 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 03 Aug 2012 10:21:30 +0200 Subject: [Freeipa-devel] [PATCH] 80 Add --{set, add, del}attr options to commands which are missing them In-Reply-To: <501B7D9F.7050008@redhat.com> References: <501B7D9F.7050008@redhat.com> Message-ID: <501B8A0A.2010903@redhat.com> On 08/03/2012 09:28 AM, Jan Cholasta wrote: > Hi, > > this patch fixes . > > Honza > The patch is OK, I did not find any regressions. ACK. Pushed to master. Martin From mkosek at redhat.com Fri Aug 3 09:07:33 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 03 Aug 2012 11:07:33 +0200 Subject: [Freeipa-devel] [PATCH] 0067 Handle exceptions when establishing trusts In-Reply-To: <20120801083350.GA12189@redhat.com> References: <20120801082508.GA11348@redhat.com> <20120801083350.GA12189@redhat.com> Message-ID: <501B94D5.7010606@redhat.com> On 08/01/2012 10:33 AM, Alexander Bokovoy wrote: > On Wed, 01 Aug 2012, Alexander Bokovoy wrote: >> Hi, >> >> I'm almost on vacation ... after this patch. :) >> >> Translate exceptions produced by DCERPC bindings when establishing >> trusts. >> >> There are two types of errors that may be produced by DCERPC bindings: >> - RuntimeError with a text >> (RuntimeError('NT_STATUS_OBJECT_NAME_NOT_FOUND') >> - RuntimeError with a numeric code and 'friendly' message >> >> Error codes could have two prefixes: >> - NT error codes, start with NT_STATUS_ prefix >> - Windows error codes, start with WERR_ prefix >> >> Full list of errors is available in Samba source code: >> libcli/util/ntstatus.h: NT_STATUS error codes >> libcli/util/werror.h: Windows error codes >> >> Majority of errors returned when dealing with trusts are of NT_STATUS >> type, these also include all typical POSIX errors mapped to corresponding NT >> errors. >> >> Unfortunately, in the textual RuntimeError case very little can be done >> to get better clarification of the error. More error paths will need to be >> added as they will be discovered -- DCERPC error messaging is complex. >> >> https://fedorahosted.org/freeipa/ticket/2868 > Too fast :) > > Updated patch, fixed one copy paste place where different type of > RuntimeError is used (all LSA exceptions are of (num, message) type). > Looks OK. I just had to fix few formatting errors (8-space indentation, ...) before I could push this. I also noticed that i18n is missing for many error messages in the new code. I created a ticket to address it later: https://fedorahosted.org/freeipa/ticket/2964 ACK. Pushed to master. Martin From mkosek at redhat.com Fri Aug 3 09:24:03 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 03 Aug 2012 11:24:03 +0200 Subject: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py In-Reply-To: <983279605.1707957.1343921207471.JavaMail.root@redhat.com> References: <983279605.1707957.1343921207471.JavaMail.root@redhat.com> Message-ID: <501B98B3.1060206@redhat.com> Hello, few issues found: 1) There is a change that should not be there: managedby_host=[fqdn1], - ipakrbauthzdata=[u'MS-PAC'], ipauniqueid=[fuzzy_uuid], 2) When raising SkipTest it would be better to pass a reason why we are skipping the text so that it is more clear in test output. I.e. instead of: + raise SkipTest I would do something like that: + raise SkipTest("Command '%s' not found" % cls.command) Otherwise the patch looks ok. Martin On 08/02/2012 05:26 PM, Tomas Babej wrote: > I implemented your suggestions. New version of the patch attached. > > ----- Original Message ----- > From: "Petr Viktorin" > To: freeipa-devel at redhat.com > Sent: Friday, July 27, 2012 1:23:54 PM > Subject: Re: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py > > On 07/27/2012 01:13 PM, Tomas Babej wrote: >> Hi, >> >> this patch simply checks if ipa-join exists in ipa-client folder, if not, skips tests relying on it. >> Uses nose.plugin.skip. >> >> https://fedorahosted.org/freeipa/ticket/2905 >> >> Tomas Babej >> > > > Hello, > This would be better done in class setup, so you don't have to repeat it > in every method. Look at XMLRPC_test.setUpClass() in xmlrpc_test.py for > isnpiration. > > While you're at it, it would be good to move the mkstemp() call into > setUpClass as well, so that importing the module doesn't have > unnecessary side effects. > From mkosek at redhat.com Fri Aug 3 10:12:03 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 03 Aug 2012 12:12:03 +0200 Subject: [Freeipa-devel] [PATCH] 0071 Create /etc/sysconfig/network if it doesn't exist In-Reply-To: <500EAEC8.4010006@redhat.com> References: <500EAEC8.4010006@redhat.com> Message-ID: <501BA3F3.50707@redhat.com> On 07/24/2012 04:18 PM, Petr Viktorin wrote: > When --hostname is given, client installation tries to write to > /etc/sysconfig/network, which is not guaranteed to exist. Create it if it > doesn't exist and we need to write to it. > > https://fedorahosted.org/freeipa/ticket/2840 > ACK. Pushed to master. Martin From jcholast at redhat.com Fri Aug 3 11:35:58 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 03 Aug 2012 13:35:58 +0200 Subject: [Freeipa-devel] [PATCH] 81 Make --{set,add,del}attr more robust Message-ID: <501BB79E.8030001@redhat.com> Hi, this patch fixes --addattr on single value attributes in add commands and --delattr on non-unicode attributes in mod commands. https://fedorahosted.org/freeipa/ticket/2954 Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-81-set-add-del-attr-fixes.patch Type: text/x-patch Size: 11044 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 3 12:18:52 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 03 Aug 2012 14:18:52 +0200 Subject: [Freeipa-devel] [PATCH] 81 Make --{set, add, del}attr more robust In-Reply-To: <501BB79E.8030001@redhat.com> References: <501BB79E.8030001@redhat.com> Message-ID: <501BC1AC.5080600@redhat.com> On 08/03/2012 01:35 PM, Jan Cholasta wrote: > Hi, > > this patch fixes --addattr on single value attributes in add commands and > --delattr on non-unicode attributes in mod commands. > > https://fedorahosted.org/freeipa/ticket/2954 > > Honza > Works fine and fixes the reported ugly *attr internal errors. No regression found. ACK. Pushed to master. Martin From pvoborni at redhat.com Fri Aug 3 12:40:32 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 03 Aug 2012 14:40:32 +0200 Subject: [Freeipa-devel] [PATCH] 186-192 Web UI: Group external member support Message-ID: <501BC6C0.4030504@redhat.com> [PATCH] 186 Add external group Added new option to group adder dialog for defining external group. Posix group and external group are mutually exclusive. https://fedorahosted.org/freeipa/ticket/2895 [PATCH] 187 Make group external New action for creating plain group external. Posix group can't be made external. https://fedorahosted.org/freeipa/ticket/2895 [PATCH] 188 Make group posix New option for creating plain user group posix group. External group can't be made posix. https://fedorahosted.org/freeipa/ticket/2338 [PATCH] 189 Display group type Added two fields to group details facet to display whether a group is external or posix. The decision is based on group's objectclass. https://fedorahosted.org/freeipa/ticket/2895 [PATCH] 190 Attribute facet Created new type of facet: attribute facet. This facet is similar to association facet but it serves for displaying object's multivalued attributes which behaves like association attributes. It will serve as a basis for displaying group's externalmember attribute. https://fedorahosted.org/freeipa/ticket/2895 [PATCH] 191 Group external member facet Added 'external' attribute facet to group entity. It serves for displaying group's externalmember attribute. https://fedorahosted.org/freeipa/ticket/2895 [PATCH] 192 Read-only external facet for non-external groups Added evaluators to decide if attribute facet should be read-only based on attribute level rights. Default values serves well for group's external member. https://fedorahosted.org/freeipa/ticket/2895 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0186-Add-external-group.patch Type: text/x-patch Size: 3405 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0187-Make-group-external.patch Type: text/x-patch Size: 5468 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0188-Make-group-posix.patch Type: text/x-patch Size: 3148 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0189-Display-group-type.patch Type: text/x-patch Size: 4949 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0190-Attribute-facet.patch Type: text/x-patch Size: 12054 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0191-Group-external-member-facet.patch Type: text/x-patch Size: 1316 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0192-Read-only-external-facet-for-non-external-groups.patch Type: text/x-patch Size: 3640 bytes Desc: not available URL: From tbabej at redhat.com Fri Aug 3 13:41:26 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 3 Aug 2012 09:41:26 -0400 (EDT) Subject: [Freeipa-devel] [Patch] 0001 Adds check for ipa-join. In-Reply-To: <501B98B3.1060206@redhat.com> Message-ID: <1465765101.1910694.1344001286038.JavaMail.root@redhat.com> All suggestions implemented. Tomas ----- Original Message ----- From: "Martin Kosek" To: "Tomas Babej" Cc: freeipa-devel at redhat.com Sent: Friday, August 3, 2012 11:24:03 AM Subject: Re: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py Hello, few issues found: 1) There is a change that should not be there: managedby_host=[fqdn1], - ipakrbauthzdata=[u'MS-PAC'], ipauniqueid=[fuzzy_uuid], 2) When raising SkipTest it would be better to pass a reason why we are skipping the text so that it is more clear in test output. I.e. instead of: + raise SkipTest I would do something like that: + raise SkipTest("Command '%s' not found" % cls.command) Otherwise the patch looks ok. Martin On 08/02/2012 05:26 PM, Tomas Babej wrote: > I implemented your suggestions. New version of the patch attached. > > ----- Original Message ----- > From: "Petr Viktorin" > To: freeipa-devel at redhat.com > Sent: Friday, July 27, 2012 1:23:54 PM > Subject: Re: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py > > On 07/27/2012 01:13 PM, Tomas Babej wrote: >> Hi, >> >> this patch simply checks if ipa-join exists in ipa-client folder, if not, skips tests relying on it. >> Uses nose.plugin.skip. >> >> https://fedorahosted.org/freeipa/ticket/2905 >> >> Tomas Babej >> > > > Hello, > This would be better done in class setup, so you don't have to repeat it > in every method. Look at XMLRPC_test.setUpClass() in xmlrpc_test.py for > isnpiration. > > While you're at it, it would be good to move the mkstemp() call into > setUpClass as well, so that importing the module doesn't have > unnecessary side effects. > -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0001-3-Adds-check-for-ipa-join.patch Type: text/x-patch Size: 2162 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 3 14:27:58 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 03 Aug 2012 16:27:58 +0200 Subject: [Freeipa-devel] [Patch] 0001 Adds check for ipa-join. In-Reply-To: <1465765101.1910694.1344001286038.JavaMail.root@redhat.com> References: <1465765101.1910694.1344001286038.JavaMail.root@redhat.com> Message-ID: <501BDFEE.4020903@redhat.com> On 08/03/2012 03:41 PM, Tomas Babej wrote: > All suggestions implemented. > > Tomas > > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Friday, August 3, 2012 11:24:03 AM > Subject: Re: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py > > Hello, > > few issues found: > > 1) There is a change that should not be there: > > managedby_host=[fqdn1], > - ipakrbauthzdata=[u'MS-PAC'], > ipauniqueid=[fuzzy_uuid], > > 2) When raising SkipTest it would be better to pass a reason why we are > skipping the text so that it is more clear in test output. I.e. instead of: > > + raise SkipTest > > I would do something like that: > + raise SkipTest("Command '%s' not found" % cls.command) > > Otherwise the patch looks ok. > > Martin > > On 08/02/2012 05:26 PM, Tomas Babej wrote: >> I implemented your suggestions. New version of the patch attached. >> >> ----- Original Message ----- >> From: "Petr Viktorin" >> To: freeipa-devel at redhat.com >> Sent: Friday, July 27, 2012 1:23:54 PM >> Subject: Re: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py >> >> On 07/27/2012 01:13 PM, Tomas Babej wrote: >>> Hi, >>> >>> this patch simply checks if ipa-join exists in ipa-client folder, if not, skips tests relying on it. >>> Uses nose.plugin.skip. >>> >>> https://fedorahosted.org/freeipa/ticket/2905 >>> >>> Tomas Babej >>> >> >> >> Hello, >> This would be better done in class setup, so you don't have to repeat it >> in every method. Look at XMLRPC_test.setUpClass() in xmlrpc_test.py for >> isnpiration. >> >> While you're at it, it would be good to move the mkstemp() call into >> setUpClass as well, so that importing the module doesn't have >> unnecessary side effects. ACK. Pushed to master. Martin From pvoborni at redhat.com Mon Aug 6 07:08:36 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 06 Aug 2012 09:08:36 +0200 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI Message-ID: <501F6D74.7050509@redhat.com> Range web UI was implemented. It consist of: * new menu item - 'ranges' in 'IPA Server' tab * new search page * new details page https://fedorahosted.org/freeipa/ticket/2894 Some comments/questions: 1) I'm not sure about options in adder dialog. Should they all be there or should I remove some? And also labels seems to long. 2) FreeIPA creates default range. This range doesn't have required attribute 'First RID of the corresponding RID range'. It prevents from editing the range in Web UI until a some value is set. Not sure if even should be edited though. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0193-Range-Web-UI.patch Type: text/x-patch Size: 14757 bytes Desc: not available URL: From jcholast at redhat.com Mon Aug 6 08:55:05 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 06 Aug 2012 10:55:05 +0200 Subject: [Freeipa-devel] Data source-agnostic parameters Message-ID: <501F8669.5020000@redhat.com> Hi, while thinking about , I had an idea how to make loading data from files available for all parameters: I think we can use URI-like strings in parameter values that the CLI would interpret and extract the wanted information from them (similar to what openssl does in the -pass command line option, see PASS PHRASE ARGUMENTS in openssl(1)). So, instead of adding a new parameter as a file-accepting alternative to any existing parameter (i.e. what is suggested in the ticket), the user would be able to specify the file in a URI-like string: (use new parameter --sshpubkeyfile) $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub vs. (use file URI-like string) $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub and the CLI would take care of reading the file and using its contents as the parameter value. This could be extended with additional URI(-like) schemes: - data: - use as the value (useful for escaping values that look like URIs, but you don't want them to be treated as such) - base64: - use the value of base64 decoded (useful for --delattr on ugly raw binary values) - fd: - read value from file descriptor - env: - read value from environment variable - ask: - always prompt interactively for the value - default: - use default value, never prompt interactively Thoughts? Honza -- Jan Cholasta From tbabej at redhat.com Mon Aug 6 13:13:57 2012 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 6 Aug 2012 09:13:57 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] Permissions of replica files changed to 0600. In-Reply-To: <526800277.120662.1344258831426.JavaMail.root@redhat.com> Message-ID: <1975589707.120685.1344258837568.JavaMail.root@redhat.com> Hi, file system permissions on replica files in /var/lib/ipa were changed to 0600. https://fedorahosted.org/freeipa/ticket/2847 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0002-Permissions-of-replica-files-changed-to-0600.patch Type: text/x-patch Size: 1115 bytes Desc: not available URL: From simo at redhat.com Mon Aug 6 13:20:39 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 06 Aug 2012 09:20:39 -0400 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <501F8669.5020000@redhat.com> References: <501F8669.5020000@redhat.com> Message-ID: <1344259239.20530.179.camel@willson.li.ssimo.org> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: > Hi, > > while thinking about , I > had an idea how to make loading data from files available for all > parameters: > > I think we can use URI-like strings in parameter values that the CLI > would interpret and extract the wanted information from them (similar to > what openssl does in the -pass command line option, see PASS PHRASE > ARGUMENTS in openssl(1)). > > So, instead of adding a new parameter as a file-accepting alternative to > any existing parameter (i.e. what is suggested in the ticket), the user > would be able to specify the file in a URI-like string: > > (use new parameter --sshpubkeyfile) > $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." > $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub > > vs. > > (use file URI-like string) > $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." > $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub > > and the CLI would take care of reading the file and using its contents > as the parameter value. > > This could be extended with additional URI(-like) schemes: > > - data: - use as the value (useful for escaping values > that look like URIs, but you don't want them to be treated as such) > - base64: - use the value of base64 decoded (useful for > --delattr on ugly raw binary values) > - fd: - read value from file descriptor > - env: - read value from environment variable > - ask: - always prompt interactively for the value > - default: - use default value, never prompt interactively > > Thoughts? How do you handle values that are actually URI strings, how do you tell the command to not interpret them ? Simo. -- Simo Sorce * Red Hat, Inc * New York From jcholast at redhat.com Mon Aug 6 13:50:14 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 06 Aug 2012 15:50:14 +0200 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <1344259239.20530.179.camel@willson.li.ssimo.org> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> Message-ID: <501FCB96.9060007@redhat.com> Dne 6.8.2012 15:20, Simo Sorce napsal(a): > On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >> Hi, >> >> while thinking about , I >> had an idea how to make loading data from files available for all >> parameters: >> >> I think we can use URI-like strings in parameter values that the CLI >> would interpret and extract the wanted information from them (similar to >> what openssl does in the -pass command line option, see PASS PHRASE >> ARGUMENTS in openssl(1)). >> >> So, instead of adding a new parameter as a file-accepting alternative to >> any existing parameter (i.e. what is suggested in the ticket), the user >> would be able to specify the file in a URI-like string: >> >> (use new parameter --sshpubkeyfile) >> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >> >> vs. >> >> (use file URI-like string) >> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >> >> and the CLI would take care of reading the file and using its contents >> as the parameter value. >> >> This could be extended with additional URI(-like) schemes: >> >> - data: - use as the value (useful for escaping values >> that look like URIs, but you don't want them to be treated as such) >> - base64: - use the value of base64 decoded (useful for >> --delattr on ugly raw binary values) >> - fd: - read value from file descriptor >> - env: - read value from environment variable >> - ask: - always prompt interactively for the value >> - default: - use default value, never prompt interactively >> >> Thoughts? > > How do you handle values that are actually URI strings, how do you tell > the command to not interpret them ? > > Simo. > You can escape them like this: --someparam data:actual://uri/string As for backward compatibility, this change has the potential to break things (user scripts which use the CLI, etc.), but AFAIK there is no parameter in IPA which expects URI string values specifically, so the impact would be miminal IMHO. Honza -- Jan Cholasta From abokovoy at redhat.com Mon Aug 6 14:10:46 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 6 Aug 2012 17:10:46 +0300 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <501FCB96.9060007@redhat.com> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> Message-ID: <20120806141046.GA9501@redhat.com> On Mon, 06 Aug 2012, Jan Cholasta wrote: >Dne 6.8.2012 15:20, Simo Sorce napsal(a): >>On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >>>Hi, >>> >>>while thinking about , I >>>had an idea how to make loading data from files available for all >>>parameters: >>> >>>I think we can use URI-like strings in parameter values that the CLI >>>would interpret and extract the wanted information from them (similar to >>>what openssl does in the -pass command line option, see PASS PHRASE >>>ARGUMENTS in openssl(1)). >>> >>>So, instead of adding a new parameter as a file-accepting alternative to >>>any existing parameter (i.e. what is suggested in the ticket), the user >>>would be able to specify the file in a URI-like string: >>> >>>(use new parameter --sshpubkeyfile) >>>$ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>$ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >>> >>>vs. >>> >>>(use file URI-like string) >>>$ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>$ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >>> >>>and the CLI would take care of reading the file and using its contents >>>as the parameter value. >>> >>>This could be extended with additional URI(-like) schemes: >>> >>> - data: - use as the value (useful for escaping values >>>that look like URIs, but you don't want them to be treated as such) >>> - base64: - use the value of base64 decoded (useful for >>>--delattr on ugly raw binary values) >>> - fd: - read value from file descriptor >>> - env: - read value from environment variable >>> - ask: - always prompt interactively for the value >>> - default: - use default value, never prompt interactively >>> >>>Thoughts? >> >>How do you handle values that are actually URI strings, how do you tell >>the command to not interpret them ? >> >>Simo. >> > >You can escape them like this: --someparam data:actual://uri/string > >As for backward compatibility, this change has the potential to break >things (user scripts which use the CLI, etc.), but AFAIK there is no >parameter in IPA which expects URI string values specifically, so the >impact would be miminal IMHO. I don't think you need to invent anything here. RFC2397 is available for long time and provides already effective way to represent any data in URI. http://tools.ietf.org/html/rfc2397 -- / Alexander Bokovoy From jcholast at redhat.com Mon Aug 6 14:27:22 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 06 Aug 2012 16:27:22 +0200 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <20120806141046.GA9501@redhat.com> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> <20120806141046.GA9501@redhat.com> Message-ID: <501FD44A.8030806@redhat.com> Dne 6.8.2012 16:10, Alexander Bokovoy napsal(a): > On Mon, 06 Aug 2012, Jan Cholasta wrote: >> Dne 6.8.2012 15:20, Simo Sorce napsal(a): >>> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >>>> Hi, >>>> >>>> while thinking about , I >>>> had an idea how to make loading data from files available for all >>>> parameters: >>>> >>>> I think we can use URI-like strings in parameter values that the CLI >>>> would interpret and extract the wanted information from them >>>> (similar to >>>> what openssl does in the -pass command line option, see PASS PHRASE >>>> ARGUMENTS in openssl(1)). >>>> >>>> So, instead of adding a new parameter as a file-accepting >>>> alternative to >>>> any existing parameter (i.e. what is suggested in the ticket), the user >>>> would be able to specify the file in a URI-like string: >>>> >>>> (use new parameter --sshpubkeyfile) >>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >>>> >>>> vs. >>>> >>>> (use file URI-like string) >>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >>>> >>>> and the CLI would take care of reading the file and using its contents >>>> as the parameter value. >>>> >>>> This could be extended with additional URI(-like) schemes: >>>> >>>> - data: - use as the value (useful for escaping values >>>> that look like URIs, but you don't want them to be treated as such) >>>> - base64: - use the value of base64 decoded (useful for >>>> --delattr on ugly raw binary values) >>>> - fd: - read value from file descriptor >>>> - env: - read value from environment variable >>>> - ask: - always prompt interactively for the value >>>> - default: - use default value, never prompt interactively >>>> >>>> Thoughts? >>> >>> How do you handle values that are actually URI strings, how do you tell >>> the command to not interpret them ? >>> >>> Simo. >>> >> >> You can escape them like this: --someparam data:actual://uri/string >> >> As for backward compatibility, this change has the potential to break >> things (user scripts which use the CLI, etc.), but AFAIK there is no >> parameter in IPA which expects URI string values specifically, so the >> impact would be miminal IMHO. > > I don't think you need to invent anything here. RFC2397 is available for > long time and provides already effective way to represent any data in > URI. > > http://tools.ietf.org/html/rfc2397 > I have considered this, but given that these URL-like strings would be mainly used directly by users on the command-line, I think that "base64:" is more user friendly than "data:;base64,". Honza -- Jan Cholasta From jdennis at redhat.com Mon Aug 6 14:27:20 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 06 Aug 2012 10:27:20 -0400 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <20120806141046.GA9501@redhat.com> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> <20120806141046.GA9501@redhat.com> Message-ID: <501FD448.1040408@redhat.com> On 08/06/2012 10:10 AM, Alexander Bokovoy wrote: > On Mon, 06 Aug 2012, Jan Cholasta wrote: >> Dne 6.8.2012 15:20, Simo Sorce napsal(a): >>> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >>>> Hi, >>>> >>>> while thinking about , I >>>> had an idea how to make loading data from files available for all >>>> parameters: >>>> >>>> I think we can use URI-like strings in parameter values that the CLI >>>> would interpret and extract the wanted information from them (similar to >>>> what openssl does in the -pass command line option, see PASS PHRASE >>>> ARGUMENTS in openssl(1)). >>>> >>>> So, instead of adding a new parameter as a file-accepting alternative to >>>> any existing parameter (i.e. what is suggested in the ticket), the user >>>> would be able to specify the file in a URI-like string: >>>> >>>> (use new parameter --sshpubkeyfile) >>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >>>> >>>> vs. >>>> >>>> (use file URI-like string) >>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >>>> >>>> and the CLI would take care of reading the file and using its contents >>>> as the parameter value. >>>> >>>> This could be extended with additional URI(-like) schemes: >>>> >>>> - data: - use as the value (useful for escaping values >>>> that look like URIs, but you don't want them to be treated as such) >>>> - base64: - use the value of base64 decoded (useful for >>>> --delattr on ugly raw binary values) >>>> - fd: - read value from file descriptor >>>> - env: - read value from environment variable >>>> - ask: - always prompt interactively for the value >>>> - default: - use default value, never prompt interactively >>>> >>>> Thoughts? >>> >>> How do you handle values that are actually URI strings, how do you tell >>> the command to not interpret them ? >>> >>> Simo. >>> >> >> You can escape them like this: --someparam data:actual://uri/string >> >> As for backward compatibility, this change has the potential to break >> things (user scripts which use the CLI, etc.), but AFAIK there is no >> parameter in IPA which expects URI string values specifically, so the >> impact would be miminal IMHO. > > I don't think you need to invent anything here. RFC2397 is available for > long time and provides already effective way to represent any data in > URI. > > http://tools.ietf.org/html/rfc2397 I think it would be much better to follow RFC's including the one Alexander cited above. Rather than $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub shouldn't it be: $ ipa user-mod --sshpubkey=file:///.ssh/id_rsa.pub I'm not sure we need or want to pass file descriptors and I wonder about the security implications of reading from environment variables. So I'm not sure about supporting the fd and env input. I don't think we need 'default' either, if the parameter does not begin with URI syntax then it's processes just as before. I can see a value in 'ask', but I'd rather see this expressed as a URI scheme. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Mon Aug 6 14:30:19 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 06 Aug 2012 10:30:19 -0400 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <501FD44A.8030806@redhat.com> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> <20120806141046.GA9501@redhat.com> <501FD44A.8030806@redhat.com> Message-ID: <501FD4FB.9010805@redhat.com> On 08/06/2012 10:27 AM, Jan Cholasta wrote: > Dne 6.8.2012 16:10, Alexander Bokovoy napsal(a): >> On Mon, 06 Aug 2012, Jan Cholasta wrote: >>> Dne 6.8.2012 15:20, Simo Sorce napsal(a): >>>> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> while thinking about , I >>>>> had an idea how to make loading data from files available for all >>>>> parameters: >>>>> >>>>> I think we can use URI-like strings in parameter values that the CLI >>>>> would interpret and extract the wanted information from them >>>>> (similar to >>>>> what openssl does in the -pass command line option, see PASS PHRASE >>>>> ARGUMENTS in openssl(1)). >>>>> >>>>> So, instead of adding a new parameter as a file-accepting >>>>> alternative to >>>>> any existing parameter (i.e. what is suggested in the ticket), the user >>>>> would be able to specify the file in a URI-like string: >>>>> >>>>> (use new parameter --sshpubkeyfile) >>>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>>> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >>>>> >>>>> vs. >>>>> >>>>> (use file URI-like string) >>>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>>> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >>>>> >>>>> and the CLI would take care of reading the file and using its contents >>>>> as the parameter value. >>>>> >>>>> This could be extended with additional URI(-like) schemes: >>>>> >>>>> - data: - use as the value (useful for escaping values >>>>> that look like URIs, but you don't want them to be treated as such) >>>>> - base64: - use the value of base64 decoded (useful for >>>>> --delattr on ugly raw binary values) >>>>> - fd: - read value from file descriptor >>>>> - env: - read value from environment variable >>>>> - ask: - always prompt interactively for the value >>>>> - default: - use default value, never prompt interactively >>>>> >>>>> Thoughts? >>>> >>>> How do you handle values that are actually URI strings, how do you tell >>>> the command to not interpret them ? >>>> >>>> Simo. >>>> >>> >>> You can escape them like this: --someparam data:actual://uri/string >>> >>> As for backward compatibility, this change has the potential to break >>> things (user scripts which use the CLI, etc.), but AFAIK there is no >>> parameter in IPA which expects URI string values specifically, so the >>> impact would be miminal IMHO. >> >> I don't think you need to invent anything here. RFC2397 is available for >> long time and provides already effective way to represent any data in >> URI. >> >> http://tools.ietf.org/html/rfc2397 >> > > I have considered this, but given that these URL-like strings would be > mainly used directly by users on the command-line, I think that > "base64:" is more user friendly than "data:;base64,". Hmm... user friendly to me means not having to remember odd proprietary exceptions. It's easier to remember what's a standard because in theory it will always be the same no matter what piece of software is I'm using. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jcholast at redhat.com Mon Aug 6 14:40:01 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 06 Aug 2012 16:40:01 +0200 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <501FD448.1040408@redhat.com> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> <20120806141046.GA9501@redhat.com> <501FD448.1040408@redhat.com> Message-ID: <501FD741.8030906@redhat.com> Dne 6.8.2012 16:27, John Dennis napsal(a): > On 08/06/2012 10:10 AM, Alexander Bokovoy wrote: >> On Mon, 06 Aug 2012, Jan Cholasta wrote: >>> Dne 6.8.2012 15:20, Simo Sorce napsal(a): >>>> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> while thinking about , I >>>>> had an idea how to make loading data from files available for all >>>>> parameters: >>>>> >>>>> I think we can use URI-like strings in parameter values that the CLI >>>>> would interpret and extract the wanted information from them >>>>> (similar to >>>>> what openssl does in the -pass command line option, see PASS PHRASE >>>>> ARGUMENTS in openssl(1)). >>>>> >>>>> So, instead of adding a new parameter as a file-accepting >>>>> alternative to >>>>> any existing parameter (i.e. what is suggested in the ticket), the >>>>> user >>>>> would be able to specify the file in a URI-like string: >>>>> >>>>> (use new parameter --sshpubkeyfile) >>>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>>> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >>>>> >>>>> vs. >>>>> >>>>> (use file URI-like string) >>>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>>> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >>>>> >>>>> and the CLI would take care of reading the file and using its contents >>>>> as the parameter value. >>>>> >>>>> This could be extended with additional URI(-like) schemes: >>>>> >>>>> - data: - use as the value (useful for escaping >>>>> values >>>>> that look like URIs, but you don't want them to be treated as such) >>>>> - base64: - use the value of base64 decoded >>>>> (useful for >>>>> --delattr on ugly raw binary values) >>>>> - fd: - read value from file descriptor >>>>> - env: - read value from environment variable >>>>> - ask: - always prompt interactively for the value >>>>> - default: - use default value, never prompt interactively >>>>> >>>>> Thoughts? >>>> >>>> How do you handle values that are actually URI strings, how do you tell >>>> the command to not interpret them ? >>>> >>>> Simo. >>>> >>> >>> You can escape them like this: --someparam data:actual://uri/string >>> >>> As for backward compatibility, this change has the potential to break >>> things (user scripts which use the CLI, etc.), but AFAIK there is no >>> parameter in IPA which expects URI string values specifically, so the >>> impact would be miminal IMHO. >> >> I don't think you need to invent anything here. RFC2397 is available for >> long time and provides already effective way to represent any data in >> URI. >> >> http://tools.ietf.org/html/rfc2397 > > I think it would be much better to follow RFC's including the one > Alexander cited above. > > Rather than > > $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub > > shouldn't it be: > > $ ipa user-mod --sshpubkey=file:///.ssh/id_rsa.pub > Not necesarrily, RFC 3986 allows the former format (but AFAIK there is no RFC which allows relative file names in the URIs). > > I'm not sure we need or want to pass file descriptors and I wonder about > the security implications of reading from environment variables. So I'm > not sure about supporting the fd and env input. Sure, these are just ideas, not something absolutely necessary. > > I don't think we need 'default' either, if the parameter does not begin > with URI syntax then it's processes just as before. > > I can see a value in 'ask', but I'd rather see this expressed as a URI > scheme. > Both "default" and "ask" are scheme names in my proposal (the URIs using these schemes don't have anything after the colon, but the colon must be present). The only use of "default:" is when a parameter has a default value and the always_ask flag set, but you don't actually want the CLI to prompt for it. Honza -- Jan Cholasta From simo at redhat.com Mon Aug 6 15:29:40 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 06 Aug 2012 11:29:40 -0400 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <501FD44A.8030806@redhat.com> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> <20120806141046.GA9501@redhat.com> <501FD44A.8030806@redhat.com> Message-ID: <1344266980.20530.207.camel@willson.li.ssimo.org> On Mon, 2012-08-06 at 16:27 +0200, Jan Cholasta wrote: > Dne 6.8.2012 16:10, Alexander Bokovoy napsal(a): > > On Mon, 06 Aug 2012, Jan Cholasta wrote: > >> Dne 6.8.2012 15:20, Simo Sorce napsal(a): > >>> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: > >>>> Hi, > >>>> > >>>> while thinking about , I > >>>> had an idea how to make loading data from files available for all > >>>> parameters: > >>>> > >>>> I think we can use URI-like strings in parameter values that the CLI > >>>> would interpret and extract the wanted information from them > >>>> (similar to > >>>> what openssl does in the -pass command line option, see PASS PHRASE > >>>> ARGUMENTS in openssl(1)). > >>>> > >>>> So, instead of adding a new parameter as a file-accepting > >>>> alternative to > >>>> any existing parameter (i.e. what is suggested in the ticket), the user > >>>> would be able to specify the file in a URI-like string: > >>>> > >>>> (use new parameter --sshpubkeyfile) > >>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." > >>>> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub > >>>> > >>>> vs. > >>>> > >>>> (use file URI-like string) > >>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." > >>>> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub > >>>> > >>>> and the CLI would take care of reading the file and using its contents > >>>> as the parameter value. > >>>> > >>>> This could be extended with additional URI(-like) schemes: > >>>> > >>>> - data: - use as the value (useful for escaping values > >>>> that look like URIs, but you don't want them to be treated as such) > >>>> - base64: - use the value of base64 decoded (useful for > >>>> --delattr on ugly raw binary values) > >>>> - fd: - read value from file descriptor > >>>> - env: - read value from environment variable > >>>> - ask: - always prompt interactively for the value > >>>> - default: - use default value, never prompt interactively > >>>> > >>>> Thoughts? > >>> > >>> How do you handle values that are actually URI strings, how do you tell > >>> the command to not interpret them ? > >>> > >>> Simo. > >>> > >> > >> You can escape them like this: --someparam data:actual://uri/string > >> > >> As for backward compatibility, this change has the potential to break > >> things (user scripts which use the CLI, etc.), but AFAIK there is no > >> parameter in IPA which expects URI string values specifically, so the > >> impact would be miminal IMHO. > > > > I don't think you need to invent anything here. RFC2397 is available for > > long time and provides already effective way to represent any data in > > URI. > > > > http://tools.ietf.org/html/rfc2397 > > > > I have considered this, but given that these URL-like strings would be > mainly used directly by users on the command-line, I think that > "base64:" is more user friendly than "data:;base64,". If you want to propose URIs then use standard URIs please. Simo. -- Simo Sorce * Red Hat, Inc * New York From edewata at redhat.com Mon Aug 6 20:14:31 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 06 Aug 2012 15:14:31 -0500 Subject: [Freeipa-devel] [PATCH] 186-192 Web UI: Group external member support In-Reply-To: <501BC6C0.4030504@redhat.com> References: <501BC6C0.4030504@redhat.com> Message-ID: <502025A7.9010208@redhat.com> ACK. The patches work, but I have some comments: 1. Would it be better to use radio buttons to define mutually exclusive options? Something like this: Group type: (o) Normal ( ) External ( ) POSIX GID: [ ] 2. If you decide to use radio buttons for the adder dialog you might want to use similar layout in the details page. 3. Minor thing, the "posix" in "Change to posix group" probably should be capitalized to be consistent with the "POSIX group" field label. Feel free to fix before push. 4. I don't have the environment to test patch #192 but the code looks fine. 5. I noticed this problem, but I couldn't reproduce it anymore. When I specified a GID in the group adder dialog, the value seemed to be ignored, so the GID would always be generated. Also the GID wasn't validated. Probably just a glitch. -- Endi S. Dewata From jcholast at redhat.com Tue Aug 7 05:24:13 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 07 Aug 2012 07:24:13 +0200 Subject: [Freeipa-devel] Data source-agnostic parameters In-Reply-To: <1344266980.20530.207.camel@willson.li.ssimo.org> References: <501F8669.5020000@redhat.com> <1344259239.20530.179.camel@willson.li.ssimo.org> <501FCB96.9060007@redhat.com> <20120806141046.GA9501@redhat.com> <501FD44A.8030806@redhat.com> <1344266980.20530.207.camel@willson.li.ssimo.org> Message-ID: <5020A67D.1050907@redhat.com> Dne 6.8.2012 17:29, Simo Sorce napsal(a): > On Mon, 2012-08-06 at 16:27 +0200, Jan Cholasta wrote: >> Dne 6.8.2012 16:10, Alexander Bokovoy napsal(a): >>> On Mon, 06 Aug 2012, Jan Cholasta wrote: >>>> Dne 6.8.2012 15:20, Simo Sorce napsal(a): >>>>> On Mon, 2012-08-06 at 10:55 +0200, Jan Cholasta wrote: >>>>>> Hi, >>>>>> >>>>>> while thinking about , I >>>>>> had an idea how to make loading data from files available for all >>>>>> parameters: >>>>>> >>>>>> I think we can use URI-like strings in parameter values that the CLI >>>>>> would interpret and extract the wanted information from them >>>>>> (similar to >>>>>> what openssl does in the -pass command line option, see PASS PHRASE >>>>>> ARGUMENTS in openssl(1)). >>>>>> >>>>>> So, instead of adding a new parameter as a file-accepting >>>>>> alternative to >>>>>> any existing parameter (i.e. what is suggested in the ticket), the user >>>>>> would be able to specify the file in a URI-like string: >>>>>> >>>>>> (use new parameter --sshpubkeyfile) >>>>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>>>> $ ipa user-mod --sshpubkeyfile=.ssh/id_rsa.pub >>>>>> >>>>>> vs. >>>>>> >>>>>> (use file URI-like string) >>>>>> $ ipa user-mod --sshpubkey="ssh-rsa AAAA ..." >>>>>> $ ipa user-mod --sshpubkey=file:.ssh/id_rsa.pub >>>>>> >>>>>> and the CLI would take care of reading the file and using its contents >>>>>> as the parameter value. >>>>>> >>>>>> This could be extended with additional URI(-like) schemes: >>>>>> >>>>>> - data: - use as the value (useful for escaping values >>>>>> that look like URIs, but you don't want them to be treated as such) >>>>>> - base64: - use the value of base64 decoded (useful for >>>>>> --delattr on ugly raw binary values) >>>>>> - fd: - read value from file descriptor >>>>>> - env: - read value from environment variable >>>>>> - ask: - always prompt interactively for the value >>>>>> - default: - use default value, never prompt interactively >>>>>> >>>>>> Thoughts? >>>>> >>>>> How do you handle values that are actually URI strings, how do you tell >>>>> the command to not interpret them ? >>>>> >>>>> Simo. >>>>> >>>> >>>> You can escape them like this: --someparam data:actual://uri/string >>>> >>>> As for backward compatibility, this change has the potential to break >>>> things (user scripts which use the CLI, etc.), but AFAIK there is no >>>> parameter in IPA which expects URI string values specifically, so the >>>> impact would be miminal IMHO. >>> >>> I don't think you need to invent anything here. RFC2397 is available for >>> long time and provides already effective way to represent any data in >>> URI. >>> >>> http://tools.ietf.org/html/rfc2397 >>> >> >> I have considered this, but given that these URL-like strings would be >> mainly used directly by users on the command-line, I think that >> "base64:" is more user friendly than "data:;base64,". > > If you want to propose URIs then use standard URIs please. > > Simo. > I don't - URIs are just one of the options we can use. I think sticking with URIs might be limiting (no relative paths in file URIs, not everything can be represented by standard URI schemes, etc.), that's why I called the strings URI-*like* in the original email. Honza -- Jan Cholasta From jcholast at redhat.com Tue Aug 7 05:50:23 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 07 Aug 2012 07:50:23 +0200 Subject: [Freeipa-devel] [PATCH] 82 Raise Base64DecodeError instead of ConversionError when base64 decoding fails in Bytes parameters Message-ID: <5020AC9F.8070604@redhat.com> Hi, this patch fixes . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-82-bytes-base64-error.patch Type: text/x-patch Size: 1347 bytes Desc: not available URL: From mkosek at redhat.com Tue Aug 7 08:20:40 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 07 Aug 2012 10:20:40 +0200 Subject: [Freeipa-devel] [PATCH] Permissions of replica files changed to 0600. In-Reply-To: <1975589707.120685.1344258837568.JavaMail.root@redhat.com> References: <1975589707.120685.1344258837568.JavaMail.root@redhat.com> Message-ID: <5020CFD8.9090606@redhat.com> On 08/06/2012 03:13 PM, Tomas Babej wrote: > Hi, > > file system permissions on replica files in /var/lib/ipa were > changed to 0600. > > https://fedorahosted.org/freeipa/ticket/2847 > > Tomas > This is OK, thanks. ACK. Pushed to master. Martin From tbabej at redhat.com Tue Aug 7 15:58:32 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 7 Aug 2012 11:58:32 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] Handle SSSD restart crash more gently. Message-ID: <1192828651.345498.1344355112889.JavaMail.root@redhat.com> Hi, In ipa-client-install, failure of restart of sssd service no longer causes the crash of the install process. Adds a warning message to the root logger instead. https://fedorahosted.org/freeipa/ticket/2827 Tomas From tbabej at redhat.com Tue Aug 7 16:14:43 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 7 Aug 2012 12:14:43 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0003 Handle SSSD restart crash more gently. In-Reply-To: <1192828651.345498.1344355112889.JavaMail.root@redhat.com> Message-ID: <356227500.348722.1344356083497.JavaMail.root@redhat.com> Sorry, I forgot to attach the patch. ----- Original Message ----- From: "Tomas Babej" To: freeipa-devel at redhat.com Sent: Tuesday, August 7, 2012 5:58:32 PM Subject: [PATCH] Handle SSSD restart crash more gently. Hi, In ipa-client-install, failure of restart of sssd service no longer causes the crash of the install process. Adds a warning message to the root logger instead. https://fedorahosted.org/freeipa/ticket/2827 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0003-Handle-SSSD-restart-crash-more-gently.patch Type: text/x-patch Size: 1234 bytes Desc: not available URL: From sbingram at gmail.com Tue Aug 7 20:30:11 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Tue, 7 Aug 2012 13:30:11 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <1343937842.20530.150.camel@willson.li.ssimo.org> References: <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> <501AB9CC.1040509@redhat.com> <501AC1A6.3090600@redhat.com> <1343937842.20530.150.camel@willson.li.ssimo.org> Message-ID: On Thu, Aug 2, 2012 at 1:04 PM, Simo Sorce wrote: > Quick heads up in this thread, > > apparently we have isolated the issue to libkrb5 and its selinux > integration. > I have made the whole delay go away by disabling selinux entirely (which > makes libkrb5 stop trying to do some selinux related operations). > > We will be working on a fix to have libkrb5 not cause this delay (F17 > doesn't have it). > > You can follow progress on this bugzilla: > https://bugzilla.redhat.com/show_bug.cgi?id=845125 I could see this BZ for a couple of days, but now it says that I must log in to an account first with the appropriate permissions. Has this been moved or something? Steve From simo at redhat.com Tue Aug 7 20:53:39 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 07 Aug 2012 16:53:39 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> <1343831702.20530.79.camel@willson.li.ssimo.org> <1343928209.20530.132.camel@willson.li.ssimo.org> <501AB9CC.1040509@redhat.com> <501AC1A6.3090600@redhat.com> <1343937842.20530.150.camel@willson.li.ssimo.org> Message-ID: <1344372819.20530.314.camel@willson.li.ssimo.org> On Tue, 2012-08-07 at 13:30 -0700, Stephen Ingram wrote: > On Thu, Aug 2, 2012 at 1:04 PM, Simo Sorce wrote: > > Quick heads up in this thread, > > > > apparently we have isolated the issue to libkrb5 and its selinux > > integration. > > I have made the whole delay go away by disabling selinux entirely (which > > makes libkrb5 stop trying to do some selinux related operations). > > > > We will be working on a fix to have libkrb5 not cause this delay (F17 > > doesn't have it). > > > > You can follow progress on this bugzilla: > > https://bugzilla.redhat.com/show_bug.cgi?id=845125 > > > I could see this BZ for a couple of days, but now it says that I must > log in to an account first with the appropriate permissions. Has this > been moved or something? > > Steve Sorry Stpehen we have a stupid bot that sometimes decides to flag bugs as internal for reasons I do not understand, I removed the flag, you should be able to see it again now. Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Wed Aug 8 13:37:03 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 08 Aug 2012 15:37:03 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5012888A.3090207@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> Message-ID: <50226B7F.9040702@redhat.com> On 07/27/2012 02:24 PM, Petr Viktorin wrote: > On 07/26/2012 11:48 PM, John Dennis wrote: >> I have applied the suggested fixes, rebased against master, run all the >> unit tests successfully, built RPM's, did a full install without errors, >> and brought up the web UI successfully. >> >> The current code can be found here: >> >> git clone git://fedorapeople.org/~jdennis/freeipa.dn.git >> git checkout dn >> >> I did not squash the individual commits (but they should be before we >> apply to master). > > Thank you! > >> Please test (again). >> >> I continue to believe the greatest lurking liability is the installer >> code and the individual command line utilities (e.g. replica-manage, >> etc.) Aside from the server install I have not exercised those components. > > Please test them, most of them just don't work. They're practically the only > ones that use the old Entity & Entry, so related bugs won't show up unless you > run the utilities. > > > > > ipa-ldap-updater still fails: > > 2012-07-27T10:21:05Z DEBUG Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", > line 112, in __upgrade > self.modified = ld.update(self.files) > File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line > 879, in update > updates = api.Backend.updateclient.update(POST_UPDATE, self.dm_password, > self.ldapi, self.live_run) > File > "/usr/lib/python2.7/site-packages/ipaserver/install/plugins/updateclient.py", > line 134, in update > if dn not in rdn_count_list[rdn_count]: > IndexError: list index out of range > > The offending code is: > rdn_count = len(DN(dn)) > rdn_count_list = dn_by_rdn_count.setdefault(rdn_count, []) > if dn not in rdn_count_list[rdn_count]: > rdn_count_list[rdn_count].append(dn) > > rdn_count_list is dn_by_rdn_count[rdn_count]; indexing with rdn_count again is > an error. > > I find the variable names are a bit confusing here. > > > > > ipa-replica-prepare is also unusable: > > $ sudo ipa-replica-prepare vm-125.$DOMAIN --ip-address $IP > Directory Manager (existing master) password: > > Preparing replica for vm-125.idm.lab.bos.redhat.com from > vm-134.idm.lab.bos.redhat.com > preparation of replica failed: '__getitem__' > '__getitem__' > File "/sbin/ipa-replica-prepare", line 461, in > main() > > File "/sbin/ipa-replica-prepare", line 309, in main > dirman_password) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", > line 99, in enable_replication_version_checking > conn.modify_s(entry[0].dn, [(ldap.MOD_REPLACE, 'nsslapd-pluginenabled', > 'on')]) > > File "/usr/lib/python2.7/site-packages/ipaserver/ipaldap.py", line 143, in > __getattr__ > return self.__dict__[name] > > i.e. entry[0] tries to call entry.__getitem__. > > I haven't tested any replica-related tools since I couldn't prepare a replica. > > > > > ipa-compliance still has the same error as before > > > > > ipa-managed-entries still fails: > File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", > line 607, in run_script > return_value = main_function() > > File "install/tools/ipa-managed-entries", line 133, in main > managed_entries = [entry.cn for entry in entries] > > You need entry.data['cn'] instead. > > > > > I also get several errors in the DNS plugin test suite: > > Traceback (most recent call last): > File "/home/pviktori/freeipa/ipaserver/rpcserver.py", line 332, in wsgi_execute > result = self.Command[name](*args, **options) > File "/home/pviktori/freeipa/ipalib/frontend.py", line 435, in __call__ > ret = self.run(*args, **options) > File "/home/pviktori/freeipa/ipalib/frontend.py", line 747, in run > return self.execute(*args, **options) > File "/home/pviktori/freeipa/ipalib/plugins/dns.py", line 2458, in execute > result = super(dnsrecord_mod, self).execute(*keys, **options) > File "/home/pviktori/freeipa/ipalib/plugins/baseldap.py", line 1351, in execute > assert isinstance(dn, DN) > AssertionError > > ipa: INFO: admin at IDM.LAB.BOS.REDHAT.COM: dnsrecord_mod(u'dnszone.test', > u'testcnamerec', arecord=(u'10.0.0.1',), cnamerecord=None, rights=False, > structured=False, all=False, raw=False, version=u'2.41'): AssertionError > > This is a good catch; the dnsrecord_mod post_callback should return the DN, not > None. > I started reviewing the latest state of your DN effort in your git repo. It is in much better shape than before, but I still found some issues in utilities we use. I am sending what I have found so far. 1) ipa-managed-entries is broken # ipa-managed-entries -l Available Managed Entry Definitions: [u'UPG Definition'] [u'NGP Definition'] # ipa-managed-entries -e 'UPG Definition' status Unexpected error AttributeError: 'LDAPEntry' object has no attribute 'originfilter' 2) ipa-replica-prepare is broken when --ip-address is passed # ipa-replica-prepare vm-055.idm.lab.bos.redhat.com --ip-address=10.16.78.55 Directory Manager (existing master) password: Preparing replica for vm-055.idm.lab.bos.redhat.com from vm-086.idm.lab.bos.redhat.com Creating SSL certificate for the Directory Server Creating SSL certificate for the dogtag Directory Server Creating SSL certificate for the Web Server Exporting RA certificate Copying additional files Finalizing configuration Packaging replica information into /var/lib/ipa/replica-info-vm-055.idm.lab.bos.redhat.com.gpg Adding DNS records for vm-055.idm.lab.bos.redhat.com preparation of replica failed: invalid 'ip_address': Gettext('invalid IP address format', domain='ipa', localedir=None) invalid 'ip_address': Gettext('invalid IP address format', domain='ipa', localedir=None) File "/sbin/ipa-replica-prepare", line 464, in main() File "/sbin/ipa-replica-prepare", line 452, in main add_zone(domain) File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", line 302, in add_zone idnsallowtransfer=u'none',) File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 433, in __call__ self.validate(**params) File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 705, in validate param.validate(value, self.env.context, supplied=param.name in kw) File "/usr/lib/python2.7/site-packages/ipalib/parameters.py", line 879, in validate self._validate_scalar(value) File "/usr/lib/python2.7/site-packages/ipalib/parameters.py", line 900, in _validate_scalar rule=rule, 3) ipa-replica-manage list is broken: # ipa-replica-manage list Failed to get data from 'vm-086.idm.lab.bos.redhat.com': base="cn=replicas,cn=ipa,cn=etc,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com", scope=1, filterstr="(objectClass=*)" I think the problem here is that the following code in ipa-replica-manage returns an exception when no entry in cn=replicas is found (which is ok): dn = DN(('cn', 'replicas'), ('cn', 'ipa'), ('cn', 'etc'), ipautil.realm_to_suffix(realm)) entries = conn.getList(dn, ldap.SCOPE_ONELEVEL) 4) IPA compliance is broken # ipa-compliance IPA compliance checking failed: This is the traceback (some DN was left in string format): Traceback (most recent call last): File "/sbin/ipa-compliance", line 198, in main() File "/sbin/ipa-compliance", line 179, in main check_compliance(tmpdir, options.debug) File "/sbin/ipa-compliance", line 121, in check_compliance size_limit = -1) File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 1087, in find_entries assert isinstance(base_dn, DN) AssertionError Btw. Petr Vobornik is testing Web UI, so far so good on this side... Martin From jdennis at redhat.com Wed Aug 8 18:49:14 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 08 Aug 2012 14:49:14 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <50226B7F.9040702@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> Message-ID: <5022B4AA.20304@redhat.com> On 08/08/2012 09:37 AM, Martin Kosek wrote: > I started reviewing the latest state of your DN effort in your git repo. It is > in much better shape than before, but I still found some issues in utilities we > use. I am sending what I have found so far. Thanks! > > 1) ipa-managed-entries is broken > # ipa-managed-entries -l > Available Managed Entry Definitions: > [u'UPG Definition'] > [u'NGP Definition'] > > # ipa-managed-entries -e 'UPG Definition' status > Unexpected error > AttributeError: 'LDAPEntry' object has no attribute 'originfilter' O.K. will investigate > 2) ipa-replica-prepare is broken when --ip-address is passed > # ipa-replica-prepare vm-055.idm.lab.bos.redhat.com --ip-address=10.16.78.55 > Directory Manager (existing master) password: > > Preparing replica for vm-055.idm.lab.bos.redhat.com from > vm-086.idm.lab.bos.redhat.com > Creating SSL certificate for the Directory Server > Creating SSL certificate for the dogtag Directory Server > Creating SSL certificate for the Web Server > Exporting RA certificate > Copying additional files > Finalizing configuration > Packaging replica information into > /var/lib/ipa/replica-info-vm-055.idm.lab.bos.redhat.com.gpg > Adding DNS records for vm-055.idm.lab.bos.redhat.com > preparation of replica failed: invalid 'ip_address': Gettext('invalid IP > address format', domain='ipa', localedir=None) > invalid 'ip_address': Gettext('invalid IP address format', domain='ipa', > localedir=None) > File "/sbin/ipa-replica-prepare", line 464, in > main() > > File "/sbin/ipa-replica-prepare", line 452, in main > add_zone(domain) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/bindinstance.py", > line 302, in add_zone > idnsallowtransfer=u'none',) > > File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 433, in __call__ > self.validate(**params) > > File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 705, in validate > param.validate(value, self.env.context, supplied=param.name in kw) > > File "/usr/lib/python2.7/site-packages/ipalib/parameters.py", line 879, in > validate > self._validate_scalar(value) > > File "/usr/lib/python2.7/site-packages/ipalib/parameters.py", line 900, in > _validate_scalar > rule=rule, Yes, I saw the same thing, but I don't think it's has anything to do with dn's. I even asked about this on IRC yesterday. Are you sure this isn't broken on master as well? When I looked at the code it just looked wrong and I didn't touch anything in this area. Can someone do a quick check on master and see if the problem exists there too? > 3) ipa-replica-manage list is broken: > # ipa-replica-manage list > Failed to get data from 'vm-086.idm.lab.bos.redhat.com': > base="cn=replicas,cn=ipa,cn=etc,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com", > scope=1, filterstr="(objectClass=*)" > > I think the problem here is that the following code in ipa-replica-manage > returns an exception when no entry in cn=replicas is found (which is ok): > > dn = DN(('cn', 'replicas'), ('cn', 'ipa'), ('cn', 'etc'), > ipautil.realm_to_suffix(realm)) > entries = conn.getList(dn, ldap.SCOPE_ONELEVEL) O.K. thanks, will investigate, seems like a simple fix. > > 4) IPA compliance is broken > > # ipa-compliance > IPA compliance checking failed: > > This is the traceback (some DN was left in string format): > Traceback (most recent call last): > File "/sbin/ipa-compliance", line 198, in > main() > File "/sbin/ipa-compliance", line 179, in main > check_compliance(tmpdir, options.debug) > File "/sbin/ipa-compliance", line 121, in check_compliance > size_limit = -1) > File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line > 1087, in find_entries > assert isinstance(base_dn, DN) > AssertionError O.K. will investigate, seems like a simple fix. > > Btw. Petr Vobornik is testing Web UI, so far so good Great. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From alee at redhat.com Wed Aug 8 20:05:26 2012 From: alee at redhat.com (Ade Lee) Date: Wed, 08 Aug 2012 16:05:26 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 Message-ID: <1344456327.21349.37.camel@aleeredhat.laptop> Hi, Dogtag 10 is being released on f18, and has a number of changes that will affect IPA. In particular, the following changes will affect current IPA code. * The directory layout of the dogtag instance has changed. Instead of using separate tomcat instances to host different subsystems, the standard dogtag installation will allow one to install a CA. KRA, OCSP and TKS within the same instance. There have been corresponding changes in the directory layout, as well as the default instance name (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead of pki-cad, pki-krad etc.) * The default instance will use only four ports (HTTPS, HTTP, AJP and tomcat shutdown port) rather than the 6 previously used. The default ports will be changed to the standard tomcat ports. As these ports are local to the ipa server machine, this should not cause too much disruption. * There is a new single step installer written in python. (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. * Dogtag 10 runs on tomcat7 - with a new corresponding version of tomcatjss. The attached patch integrates all the above changes in IPA installation and maintenance code. Once the patch is applied, users will be able to: 1. run ipa-server-install to completion on f18 with dogtag 10. 2. install a new replica on f18 on dogtag 10. 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag 10 - and have that old-style dogtag instance continue to run correctly. This will require the installation of the latest version of tomcatjss as well as the installation of tomcat6. The old-style instance will continue to use tomcat6. 4. in addition, the new cert renewal code has been patched and should continue to work. What is not yet completed / supported: 1. Installation with an external CA is not yet completed in the new installer. We plan to complete this soon. 2. There is some IPA upgrade code that has not yet been touched (install/tools/ipa-upgradeconfig). 3. A script needs to be written to allow admins to convert their old-style dogtag instances to new style instances, as well as code to periodically prompt admins to do this. 4. Installation of old-style instances using pkicreate/pkisilent on dogtag 10 will no longer be supported, and will be disabled soon. 5. The pki-selinux policy has been updated to reflect these changes, but is still in flux. In fact, it is our intention to place the dogtag selinux policy in the base selinux policy for f18. In the meantime, it may be necessary to run installs in permissive mode. The dogtag 10 code will be released shortly into f18. Prior to that though, we have placed the new dogtag 10 and tomcatjss code in a developer repo that is located at http://nkinder.fedorapeople.org/dogtag-devel/ Testing can be done on both f18 and f17 - although the target platform - and the only platform for which official builds will be created is f18. Thanks, Ade -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Modifications-to-install-scripts-for-dogtag-10.patch Type: text/x-patch Size: 43546 bytes Desc: not available URL: From jdennis at redhat.com Wed Aug 8 21:02:16 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 08 Aug 2012 17:02:16 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5022B4AA.20304@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> Message-ID: <5022D3D8.6060607@redhat.com> All the issues Martin discovered (except for the ip-address parameter) are now fixed and pushed to the dn repo. Also now the dn repo is fully rebased against master (except for one commit for ticket 2954 which I had to revert, see ticket for details). Thank you for the continued testing. FYI: to use the dn repo: git clone git://fedorapeople.org/~jdennis/freeipa.dn.git git checkout dn -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Thu Aug 9 12:44:20 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 09 Aug 2012 14:44:20 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5022D3D8.6060607@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> Message-ID: <5023B0A4.3060202@redhat.com> On 08/08/2012 11:02 PM, John Dennis wrote: > All the issues Martin discovered (except for the ip-address parameter) are now > fixed and pushed to the dn repo. Also now the dn repo is fully rebased against > master (except for one commit for ticket 2954 which I had to revert, see ticket > for details). > > Thank you for the continued testing. > > FYI: to use the dn repo: > > git clone git://fedorapeople.org/~jdennis/freeipa.dn.git > git checkout dn > Good. I see that all issues except the ipa-replica-prepare are now fixed. I verified that this is indeed an issue caused by the DN refactoring, I am attaching a patch fixing the issue. With this patch, ipa-replica-prepare issue disappears. Petr Vobornik also noticed an issue with trust-show command. I am attaching a patch with fix as well. We push the patches when we are pushing your work. I have not found any more show-stopping issues, so I will just continue my testing, and also give space to other testers in case they discover something else. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-DN-usage-in-adtrust-show-command.patch Type: text/x-patch Size: 879 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Do-not-crash-in-ipa-replica-prepare-DNS-phase.patch Type: text/x-patch Size: 2998 bytes Desc: not available URL: From rcritten at redhat.com Thu Aug 9 18:08:51 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 09 Aug 2012 14:08:51 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5023B0A4.3060202@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> <5023B0A4.3060202@redhat.com> Message-ID: <5023FCB3.30806@redhat.com> Martin Kosek wrote: > On 08/08/2012 11:02 PM, John Dennis wrote: >> All the issues Martin discovered (except for the ip-address parameter) are now >> fixed and pushed to the dn repo. Also now the dn repo is fully rebased against >> master (except for one commit for ticket 2954 which I had to revert, see ticket >> for details). >> >> Thank you for the continued testing. >> >> FYI: to use the dn repo: >> >> git clone git://fedorapeople.org/~jdennis/freeipa.dn.git >> git checkout dn >> > > Good. I see that all issues except the ipa-replica-prepare are now fixed. I > verified that this is indeed an issue caused by the DN refactoring, I am > attaching a patch fixing the issue. With this patch, ipa-replica-prepare issue > disappears. > > Petr Vobornik also noticed an issue with trust-show command. I am attaching a > patch with fix as well. We push the patches when we are pushing your work. > > I have not found any more show-stopping issues, so I will just continue my > testing, and also give space to other testers in case they discover something else. > > Martin I've been going through the diffs and have some questions in ldap2.py, these are primarily pedantic: What is the significance of L here: '2.16.840.1.113730.3.8.L.8' : DN, # ipaTemplateRef There are quite a few assert isinstance(dn, DN). I assume this is mainly meant for developers, we aren't expecting to handle that gracefully at runtime? It seems to me that allowing a DN passed in as a string would be nice in some cases. Calling bind, for example, looks very awkward and isn't as readable as cn=directory manager, IMHO. What is the downside of having a converter for these? search_ext() has an extra embedded line which makes the function a bit confusing to read. Do we need to keep _debug_log_ldap? In search_ext_s() (and a few others) should we just return self.convert_result() directly? In ldap2::__init__ what would raise an AttributeError and would we want to hide that fact with an empty base_dn? Is this attempting to allow the use of ldap2.ldap2 in cases where the api is not initialized? In ipaserver/ipaldap.py there is a continuance of the assertions, some of which don't seem to be needed. For example, in toDict() it shouldn't be possible to create an Entry object without having self.dn be a DN object, so is this assertion necessary? rob From jdennis at redhat.com Thu Aug 9 19:31:13 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 09 Aug 2012 15:31:13 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5023FCB3.30806@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> <5023B0A4.3060202@redhat.com> <5023FCB3.30806@redhat.com> Message-ID: <50241001.5030809@redhat.com> On 08/09/2012 02:08 PM, Rob Crittenden wrote: > I've been going through the diffs and have some questions in ldap2.py, > these are primarily pedantic: Some of your questions can be answered by: http://jdennis.fedorapeople.org/dn_summary.html > > What is the significance of L here: > > '2.16.840.1.113730.3.8.L.8' : DN, # ipaTemplateRef These came from: install/share/60policyv2.ldif I didn't notice the .L in the OID which isn't legal (correct?). So beats me, I can't explain why the OID is specified that way. > > There are quite a few assert isinstance(dn, DN). I assume this is mainly > meant for developers, we aren't expecting to handle that gracefully at > runtime? Correct. They are there to prevent future use of dn strings by developers whose habits die hard. The goal is 100% DN usage 100% of the time. If we allow strings some of the time we're on a slippery slope. Think of it as type enforcement meant to protect ourselves. The assertions also proved valuable in finding a number of places where functions failed to return values or correct values. This showed up a lot in the pre and post callbacks whose signature specifies a return value but the developer forgot to return a value. Apparently pylint does not pick these things up. In production we should disable assertions, we should open a ticket to disable assertions in production. > > It seems to me that allowing a DN passed in as a string would be nice in > some cases. Calling bind, for example, looks very awkward and isn't as > readable as cn=directory manager, IMHO. What is the downside of having a > converter for these? See the above documentation for the rationale. In particular the section called "Why did I use tuples instead of strings when initializing DN's?" > search_ext() has an extra embedded line which makes the function a bit > confusing to read. O.K. you lost me on that one :-) > Do we need to keep _debug_log_ldap? I don't think it hurts. I found it very useful to see the actual LDAP data when debugging. > In search_ext_s() (and a few others) should we just return > self.convert_result() directly? Petr asked this question too. I don't have strong feelings either way. The reason it's stored in another variable is it's easy to add a print statement or stop in the debugger and examine it when need be. It also makes it clear it's the IPA formatted results. I don't think there is much cost to using the variable. I'm not attached to it, it can be changed. > > In ldap2::__init__ what would raise an AttributeError and would we want > to hide that fact with an empty base_dn? Is this attempting to allow the > use of ldap2.ldap2 in cases where the api is not initialized? Beats me, that's been in the code for a long time. An empty DN is the same as an empty string. Maybe we should set it to None instead so we know base_dn was never initialized properly. > In ipaserver/ipaldap.py there is a continuance of the assertions, some > of which don't seem to be needed. For example, in toDict() it shouldn't > be possible to create an Entry object without having self.dn be a DN > object, so is this assertion necessary? Many objects now enforce DN's for their dn attribute. A dn attribute may only ever be None or an instance of a DN. This is implemented with dn = ipautil.dn_attribute_property('_dn') In objects which define their dn property this way an assert isinstance(self.dn, DN) is not necessary because the dn property enforces it. So you're correct, those particular asserts could be removed. They were added before dn type enforcement via object property was added. I could go through and clean those up, but perhaps we should open a separate ticket for that. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rmeggins at redhat.com Thu Aug 9 19:34:16 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 09 Aug 2012 13:34:16 -0600 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <50241001.5030809@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> <5023B0A4.3060202@redhat.com> <5023FCB3.30806@redhat.com> <50241001.5030809@redhat.com> Message-ID: <502410B8.601@redhat.com> On 08/09/2012 01:31 PM, John Dennis wrote: > On 08/09/2012 02:08 PM, Rob Crittenden wrote: >> I've been going through the diffs and have some questions in ldap2.py, >> these are primarily pedantic: > > Some of your questions can be answered by: > > http://jdennis.fedorapeople.org/dn_summary.html > >> >> What is the significance of L here: >> >> '2.16.840.1.113730.3.8.L.8' : DN, # ipaTemplateRef > > These came from: > > install/share/60policyv2.ldif > > I didn't notice the .L in the OID which isn't legal (correct?). So > beats me, I can't explain why the OID is specified that way. hmm - did you see this comment at the top of the file? # Policy related schema. # This file should not be loaded. # Remove this comment and assign right OIDs when time comes to do something # about this functionality. > >> >> There are quite a few assert isinstance(dn, DN). I assume this is mainly >> meant for developers, we aren't expecting to handle that gracefully at >> runtime? > > Correct. They are there to prevent future use of dn strings by > developers whose habits die hard. The goal is 100% DN usage 100% of > the time. > > If we allow strings some of the time we're on a slippery slope. Think > of it as type enforcement meant to protect ourselves. > > The assertions also proved valuable in finding a number of places > where functions failed to return values or correct values. This showed > up a lot in the pre and post callbacks whose signature specifies a > return value but the developer forgot to return a value. Apparently > pylint does not pick these things up. > > In production we should disable assertions, we should open a ticket to > disable assertions in production. > >> >> It seems to me that allowing a DN passed in as a string would be nice in >> some cases. Calling bind, for example, looks very awkward and isn't as >> readable as cn=directory manager, IMHO. What is the downside of having a >> converter for these? > > See the above documentation for the rationale. In particular the > section called "Why did I use tuples instead of strings when > initializing DN's?" > >> search_ext() has an extra embedded line which makes the function a bit >> confusing to read. > > O.K. you lost me on that one :-) > >> Do we need to keep _debug_log_ldap? > > I don't think it hurts. I found it very useful to see the actual LDAP > data when debugging. > >> In search_ext_s() (and a few others) should we just return >> self.convert_result() directly? > > Petr asked this question too. I don't have strong feelings either way. > The reason it's stored in another variable is it's easy to add a print > statement or stop in the debugger and examine it when need be. It also > makes it clear it's the IPA formatted results. I don't think there is > much cost to using the variable. I'm not attached to it, it can be > changed. >> >> In ldap2::__init__ what would raise an AttributeError and would we want >> to hide that fact with an empty base_dn? Is this attempting to allow the >> use of ldap2.ldap2 in cases where the api is not initialized? > > Beats me, that's been in the code for a long time. An empty DN is the > same as an empty string. Maybe we should set it to None instead so we > know base_dn was never initialized properly. > >> In ipaserver/ipaldap.py there is a continuance of the assertions, some >> of which don't seem to be needed. For example, in toDict() it shouldn't >> be possible to create an Entry object without having self.dn be a DN >> object, so is this assertion necessary? > > Many objects now enforce DN's for their dn attribute. A dn attribute > may only ever be None or an instance of a DN. This is implemented with > > dn = ipautil.dn_attribute_property('_dn') > > In objects which define their dn property this way an assert > isinstance(self.dn, DN) is not necessary because the dn property > enforces it. So you're correct, those particular asserts could be > removed. They were added before dn type enforcement via object > property was added. I could go through and clean those up, but perhaps > we should open a separate ticket for that. > > > From jdennis at redhat.com Thu Aug 9 19:46:47 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 09 Aug 2012 15:46:47 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <502410B8.601@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> <5023B0A4.3060202@redhat.com> <5023FCB3.30806@redhat.com> <50241001.5030809@redhat.com> <502410B8.601@redhat.com> Message-ID: <502413A7.6090902@redhat.com> On 08/09/2012 03:34 PM, Rich Megginson wrote: > On 08/09/2012 01:31 PM, John Dennis wrote: >> On 08/09/2012 02:08 PM, Rob Crittenden wrote: >>> What is the significance of L here: >>> >>> '2.16.840.1.113730.3.8.L.8' : DN, # ipaTemplateRef >> >> These came from: >> >> install/share/60policyv2.ldif >> >> I didn't notice the .L in the OID which isn't legal (correct?). So >> beats me, I can't explain why the OID is specified that way. > > hmm - did you see this comment at the top of the file? > > # Policy related schema. > # This file should not be loaded. > # Remove this comment and assign right OIDs when time comes to do something > # about this functionality. Ah, good catch Rich! No I didn't see that comment. What I did was grep'ed for all dn syntax attributes in all our ldiff files so I didn't see the comment. I'll remove these as they are obviously incorrect. Thank you both Rob and Rich for the sharp eyes. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From tbabej at redhat.com Fri Aug 10 10:44:41 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 10 Aug 2012 06:44:41 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. Message-ID: <943275987.932106.1344595481612.JavaMail.root@redhat.com> Hi, I checked the rest of the description as well, seems alright. https://fedorahosted.org/freeipa/ticket/2959 Tomas From pvoborni at redhat.com Fri Aug 10 10:57:50 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 10 Aug 2012 12:57:50 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5023B0A4.3060202@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> <5023B0A4.3060202@redhat.com> Message-ID: <5024E92E.7080404@redhat.com> On 08/09/2012 02:44 PM, Martin Kosek wrote: > On 08/08/2012 11:02 PM, John Dennis wrote: >> All the issues Martin discovered (except for the ip-address parameter) are now >> fixed and pushed to the dn repo. Also now the dn repo is fully rebased against >> master (except for one commit for ticket 2954 which I had to revert, see ticket >> for details). >> >> Thank you for the continued testing. >> >> FYI: to use the dn repo: >> >> git clone git://fedorapeople.org/~jdennis/freeipa.dn.git >> git checkout dn >> > > Good. I see that all issues except the ipa-replica-prepare are now fixed. I > verified that this is indeed an issue caused by the DN refactoring, I am > attaching a patch fixing the issue. With this patch, ipa-replica-prepare issue > disappears. > > Petr Vobornik also noticed an issue with trust-show command. I am attaching a > patch with fix as well. We push the patches when we are pushing your work. > > I have not found any more show-stopping issues, so I will just continue my > testing, and also give space to other testers in case they discover something else. > > Martin > I found another issue while adding external member to a group: freeipa-server-3.0.0GIT675d841-0.fc17.x86_64 [Fri Aug 10 06:48:45 2012] [error] ipa: ERROR: non-public: AssertionError: [Fri Aug 10 06:48:45 2012] [error] Traceback (most recent call last): [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 332, in wsgi_execute [Fri Aug 10 06:48:45 2012] [error] result = self.Command[name](*args, **options) [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 435, in __call__ [Fri Aug 10 06:48:45 2012] [error] ret = self.run(*args, **options) [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 747, in run [Fri Aug 10 06:48:45 2012] [error] return self.execute(*args, **options) [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipalib/plugins/baseldap.py", line 1568, in execute [Fri Aug 10 06:48:45 2012] [error] **options) [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipalib/plugins/group.py", line 342, in post_callback [Fri Aug 10 06:48:45 2012] [error] if domain_validator.is_trusted_sid_valid(sid): [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 148, in is_trusted_sid_valid [Fri Aug 10 06:48:45 2012] [error] self._domains = self.get_trusted_domains() [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 119, in get_trusted_domains [Fri Aug 10 06:48:45 2012] [error] attrs_list=[self.ATTR_TRUSTED_SID, 'dn']) [Fri Aug 10 06:48:45 2012] [error] File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 1087, in find_entries [Fri Aug 10 06:48:45 2012] [error] assert isinstance(base_dn, DN) [Fri Aug 10 06:48:45 2012] [error] AssertionError [Fri Aug 10 06:48:45 2012] [error] ipa: INFO: admin at IDM.LAB.BOS.REDHAT.COM: group_add_member(u'external', ipaexternalmember=u'S-1-5-21-2085708479-1865276630-1146473440-1'): AssertionError -- Petr Vobornik From pvoborni at redhat.com Fri Aug 10 11:04:19 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 10 Aug 2012 13:04:19 +0200 Subject: [Freeipa-devel] [PATCH] 186-192 Web UI: Group external member support In-Reply-To: <502025A7.9010208@redhat.com> References: <501BC6C0.4030504@redhat.com> <502025A7.9010208@redhat.com> Message-ID: <5024EAB3.9030906@redhat.com> On 08/06/2012 10:14 PM, Endi Sukma Dewata wrote: > ACK. Attaching all patches. Patches 186-189 are changed or rebased. Rest is intact. > > The patches work, but I have some comments: > > 1. Would it be better to use radio buttons to define mutually exclusive > options? Something like this: > > Group type: (o) Normal > ( ) External > ( ) POSIX > GID: [ ] Patch 186 reworked. > > 2. If you decide to use radio buttons for the adder dialog you might > want to use similar layout in the details page. Patch 189 reworked. > > 3. Minor thing, the "posix" in "Change to posix group" probably should > be capitalized to be consistent with the "POSIX group" field label. Feel > free to fix before push. Fixed. > > 4. I don't have the environment to test patch #192 but the code looks fine. > > 5. I noticed this problem, but I couldn't reproduce it anymore. When I > specified a GID in the group adder dialog, the value seemed to be > ignored, so the GID would always be generated. Also the GID wasn't > validated. Probably just a glitch. > I didn't notice it. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0186-1-Add-external-group.patch Type: text/x-patch Size: 5367 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0187-1-Make-group-external.patch Type: text/x-patch Size: 5528 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0188-1-Make-group-posix.patch Type: text/x-patch Size: 3208 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0189-1-Display-group-type.patch Type: text/x-patch Size: 5144 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0190-Attribute-facet.patch Type: text/x-patch Size: 12054 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0191-Group-external-member-facet.patch Type: text/x-patch Size: 1316 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0192-Read-only-external-facet-for-non-external-groups.patch Type: text/x-patch Size: 3640 bytes Desc: not available URL: From tbabej at redhat.com Fri Aug 10 11:09:24 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 10 Aug 2012 07:09:24 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. In-Reply-To: <943275987.932106.1344595481612.JavaMail.root@redhat.com> Message-ID: <869392160.934054.1344596964961.JavaMail.root@redhat.com> I forgot the patch once again. ----- Original Message ----- From: "Tomas Babej" To: freeipa-devel at redhat.com Sent: Friday, August 10, 2012 12:44:41 PM Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. Hi, I checked the rest of the description as well, seems alright. https://fedorahosted.org/freeipa/ticket/2959 Tomas _______________________________________________ Freeipa-devel mailing list Freeipa-devel at redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0004-Corrects-help-description-of-selinuxusermap.patch Type: text/x-patch Size: 1092 bytes Desc: not available URL: From pvoborni at redhat.com Fri Aug 10 11:11:46 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 10 Aug 2012 13:11:46 +0200 Subject: [Freeipa-devel] [PATCH] 194 Handle case when trusted domain user access the Web UI Message-ID: <5024EC72.1000902@redhat.com> WebUI catches the fact that the user can't access LDAP server with a current ticket. It shows form-based auth login dialog. Previoustly an ugly error was returned on an almost empty page, and user had no recourse. https://fedorahosted.org/freeipa/ticket/2897 I don't like the implementation much. Problem is that we don't separate state variables and framework objects in IPA object. It is probably a topic for fixing in 3.2. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0194-Handle-case-when-trusted-domain-user-access-the-Web-.patch Type: text/x-patch Size: 2911 bytes Desc: not available URL: From pvoborni at redhat.com Fri Aug 10 11:13:35 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 10 Aug 2012 13:13:35 +0200 Subject: [Freeipa-devel] [PATCH] 195 Disable caching of Web UI login_kerberos request Message-ID: <5024ECDF.3010607@redhat.com> IE caches login_kerberos request so SSO doesn't work after logout. This patch disables the caching. https://fedorahosted.org/freeipa/ticket/2991 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0195-Disable-caching-of-Web-UI-login_kerberos-request.patch Type: text/x-patch Size: 847 bytes Desc: not available URL: From pvoborni at redhat.com Fri Aug 10 11:48:09 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 10 Aug 2012 13:48:09 +0200 Subject: [Freeipa-devel] [PATCH] 196 Update other facets on delete from search page Message-ID: <5024F4F9.2050306@redhat.com> When an object in search facet was deleted, other facets were not notified that they need to refresh. If one crated object with same pkey as deleted object and then navigated to it's details he could see old object's data. This notification was added. https://fedorahosted.org/freeipa/ticket/2618 Note: I found a refresh issue in search facet while using 'Add and edit' in adder dialog. This will be fixed separately. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0196-Update-other-facets-on-delete-from-search-page.patch Type: text/x-patch Size: 2956 bytes Desc: not available URL: From pspacek at redhat.com Fri Aug 10 12:08:53 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 10 Aug 2012 14:08:53 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <50126A5D.4060805@redhat.com> References: <50126A5D.4060805@redhat.com> Message-ID: <5024F9D5.8090405@redhat.com> On 07/27/2012 12:15 PM, Petr Spacek wrote: > Hello, > > this patch implements "Flush zones and RRs cache when handling persistent > search reconnection" behaviour as requested > in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . > > Petr^2 Spacek Self-NACK :-) This second version has cache flush postponed a bit: Cache is flushed after receiving first result from LDAP. It should prevent unwanted cache flush in case of timeout or similar problems. Simo, are you ok with this approach? Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0042-2-Flush-zones-and-RRs-cache-when-handling-persistent-s.patch Type: text/x-patch Size: 6519 bytes Desc: not available URL: From jdennis at redhat.com Fri Aug 10 12:50:07 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 10 Aug 2012 08:50:07 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5024E92E.7080404@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> <5012888A.3090207@redhat.com> <50226B7F.9040702@redhat.com> <5022B4AA.20304@redhat.com> <5022D3D8.6060607@redhat.com> <5023B0A4.3060202@redhat.com> <5024E92E.7080404@redhat.com> Message-ID: <5025037F.4040201@redhat.com> On 08/10/2012 06:57 AM, Petr Vobornik wrote: > I found another issue while adding external member to a group: Thank you, fix was simple, get_trusted_domains() was converting a dn to unicode when calling find_entries(), not sure why since we've never used unicode for dn's (I had searched the code looking for str conversions). Fix is in the fedorapeople repo and the final patch I'm about to send out. I'll also do another search looking for unicode conversions. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From tbabej at redhat.com Fri Aug 10 13:07:21 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 10 Aug 2012 09:07:21 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0005 Improves exception handling in ipa-replica-prepare. In-Reply-To: <597674153.951167.1344604038760.JavaMail.root@redhat.com> Message-ID: <1016046819.951172.1344604041247.JavaMail.root@redhat.com> Hi, A backtrace is no longer displayed when trying to prepare a replica file with the local LDAP server down. Also adds --debug option and no longer displays info log messages without it. https://fedorahosted.org/freeipa/ticket/2939 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0005-Improves-exception-handling-in-ipa-replica-prepare.patch Type: text/x-patch Size: 2076 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 10 13:43:06 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 10 Aug 2012 15:43:06 +0200 Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. In-Reply-To: <869392160.934054.1344596964961.JavaMail.root@redhat.com> References: <869392160.934054.1344596964961.JavaMail.root@redhat.com> Message-ID: <50250FEA.4010509@redhat.com> On 08/10/2012 01:09 PM, Tomas Babej wrote: > I forgot the patch once again. > > ----- Original Message ----- > From: "Tomas Babej" > To: freeipa-devel at redhat.com > Sent: Friday, August 10, 2012 12:44:41 PM > Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. > > Hi, > > I checked the rest of the description as well, seems alright. > > https://fedorahosted.org/freeipa/ticket/2959 > > Tomas > That's a good start. I would also change the following descriptions a bit: Disable a named rule: ipa selinuxusermap-disable test1 Enable a named rule: ipa selinuxusermap-enable test1 ... Remove a named rule: ipa selinuxusermap-del john_unconfined I don't know why all rule are specified as "named". I think that simple Disable a rule: Enable a rule: Remove a rule: would be enough and less confusing. Martin From mkosek at redhat.com Fri Aug 10 13:49:45 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 10 Aug 2012 15:49:45 +0200 Subject: [Freeipa-devel] [PATCH] 0005 Improves exception handling in ipa-replica-prepare. In-Reply-To: <1016046819.951172.1344604041247.JavaMail.root@redhat.com> References: <1016046819.951172.1344604041247.JavaMail.root@redhat.com> Message-ID: <50251179.7090208@redhat.com> On 08/10/2012 03:07 PM, Tomas Babej wrote: > Hi, > > A backtrace is no longer displayed when trying to prepare a replica > file with the local LDAP server down. Also adds --debug option and > no longer displays info log messages without it. > > https://fedorahosted.org/freeipa/ticket/2939 > > Tomas > Looks OK. We just also need to update the ipa-replica-prepare man page, when changing its options. Martin From tbabej at redhat.com Fri Aug 10 14:32:23 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 10 Aug 2012 10:32:23 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0005-2 Improves exception handling in ipa-replica-prepare. In-Reply-To: <50251179.7090208@redhat.com> Message-ID: <222233737.971584.1344609143323.JavaMail.root@redhat.com> Man page edited. ----- Original Message ----- From: "Martin Kosek" To: "Tomas Babej" Cc: freeipa-devel at redhat.com Sent: Friday, August 10, 2012 3:49:45 PM Subject: Re: [Freeipa-devel] [PATCH] 0005 Improves exception handling in ipa-replica-prepare. On 08/10/2012 03:07 PM, Tomas Babej wrote: > Hi, > > A backtrace is no longer displayed when trying to prepare a replica > file with the local LDAP server down. Also adds --debug option and > no longer displays info log messages without it. > > https://fedorahosted.org/freeipa/ticket/2939 > > Tomas > Looks OK. We just also need to update the ipa-replica-prepare man page, when changing its options. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0005-Improves-exception-handling-in-ipa-replica-prepare.patch Type: text/x-patch Size: 2645 bytes Desc: not available URL: From tbabej at redhat.com Fri Aug 10 14:44:31 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 10 Aug 2012 10:44:31 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0004-2 Corrects help description of selinuxusermap. In-Reply-To: <50250FEA.4010509@redhat.com> Message-ID: <379666122.977071.1344609871858.JavaMail.root@redhat.com> Suggestion incorporated. Tomas ----- Original Message ----- From: "Martin Kosek" To: "Tomas Babej" Cc: freeipa-devel at redhat.com Sent: Friday, August 10, 2012 3:43:06 PM Subject: Re: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. On 08/10/2012 01:09 PM, Tomas Babej wrote: > I forgot the patch once again. > > ----- Original Message ----- > From: "Tomas Babej" > To: freeipa-devel at redhat.com > Sent: Friday, August 10, 2012 12:44:41 PM > Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. > > Hi, > > I checked the rest of the description as well, seems alright. > > https://fedorahosted.org/freeipa/ticket/2959 > > Tomas > That's a good start. I would also change the following descriptions a bit: Disable a named rule: ipa selinuxusermap-disable test1 Enable a named rule: ipa selinuxusermap-enable test1 ... Remove a named rule: ipa selinuxusermap-del john_unconfined I don't know why all rule are specified as "named". I think that simple Disable a rule: Enable a rule: Remove a rule: would be enough and less confusing. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0004-2-Corrects-help-description-of-selinuxusermap.patch Type: text/x-patch Size: 1763 bytes Desc: not available URL: From jdennis at redhat.com Fri Aug 10 17:11:23 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 10 Aug 2012 13:11:23 -0400 Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings Message-ID: <502540BB.4060602@redhat.com> Due to the size and disruptive nature of this patch it was reviewed and tested outside the normal patch review process, see the thread "DN patch and documentation" as well as other threads on this list. This patch is the cumulative result of that review process. John -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0076-Use-DN-objects-instead-of-strings.patch Type: text/x-patch Size: 888522 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 10 21:03:18 2012 From: rcritten at redhat.com (Robert Crittenden) Date: Fri, 10 Aug 2012 17:03:18 -0400 (EDT) Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings In-Reply-To: <502540BB.4060602@redhat.com> Message-ID: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> This looks good, I just want to amend the commit with the tickets that this addresses before pushing. It seems that this covers 1670-1674. Is 1392 included as well? rob ----- Original Message ----- From: "John Dennis" To: "freeipa-devel" Sent: Friday, August 10, 2012 1:11:23 PM Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings Due to the size and disruptive nature of this patch it was reviewed and tested outside the normal patch review process, see the thread "DN patch and documentation" as well as other threads on this list. This patch is the cumulative result of that review process. John -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ _______________________________________________ Freeipa-devel mailing list Freeipa-devel at redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel From rcritten at redhat.com Fri Aug 10 21:45:11 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 10 Aug 2012 17:45:11 -0400 Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings In-Reply-To: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> References: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> Message-ID: <502580E7.4080503@redhat.com> Robert Crittenden wrote: > This looks good, I just want to amend the commit with the tickets that this addresses before pushing. > > It seems that this covers 1670-1674. Is 1392 included as well? > > rob > > ----- Original Message ----- > From: "John Dennis" > To: "freeipa-devel" > Sent: Friday, August 10, 2012 1:11:23 PM > Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings > > Due to the size and disruptive nature of this patch it was reviewed and > tested outside the normal patch review process, see the thread "DN patch > and documentation" as well as other threads on this list. This patch is > the cumulative result of that review process. > > John > Posting this since I haven't pushed the patch yet. I found this installing with --selfsign: diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py index 1bbee35..eebaa48 100644 --- a/ipaserver/install/certs.py +++ b/ipaserver/install/certs.py @@ -1048,7 +1048,7 @@ class CertDB(object): # Prepare a simple cert request req_dict = dict(PASSWORD=self.gen_password(), SUBJBASE=self.subject_base, - CERTNAME=DN('CN', nickname)) + CERTNAME=DN(('CN', nickname))) req_template = ipautil.SHARE_DIR + reqcfg + ".template" conf = ipautil.template_file(req_template, req_dict) fd = open(reqcfg, "w+") From ssorce at redhat.com Sun Aug 12 09:59:36 2012 From: ssorce at redhat.com (Simo Sorce) Date: Sun, 12 Aug 2012 05:59:36 -0400 (EDT) Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <5024F9D5.8090405@redhat.com> Message-ID: <882765854.3533744.1344765576291.JavaMail.root@redhat.com> > On 07/27/2012 12:15 PM, Petr Spacek wrote: > > Hello, > > > > this patch implements "Flush zones and RRs cache when handling > > persistent > > search reconnection" behaviour as requested > > in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . > > > > Petr^2 Spacek > > Self-NACK :-) > > This second version has cache flush postponed a bit: Cache is flushed > after > receiving first result from LDAP. It should prevent unwanted cache > flush in > case of timeout or similar problems. > > Simo, are you ok with this approach? Sounds better. Ideally you do not flush until you get all results and no errors, but if that's difficult, waiting until the first results come in maybe be a decent first step. Simo. From pspacek at redhat.com Mon Aug 13 09:03:33 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 13 Aug 2012 11:03:33 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <882765854.3533744.1344765576291.JavaMail.root@redhat.com> References: <882765854.3533744.1344765576291.JavaMail.root@redhat.com> Message-ID: <5028C2E5.6010002@redhat.com> On 08/12/2012 11:59 AM, Simo Sorce wrote: >> On 07/27/2012 12:15 PM, Petr Spacek wrote: >>> Hello, >>> >>> this patch implements "Flush zones and RRs cache when handling >>> persistent >>> search reconnection" behaviour as requested >>> in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . >>> >>> Petr^2 Spacek >> >> Self-NACK :-) >> >> This second version has cache flush postponed a bit: Cache is flushed >> after >> receiving first result from LDAP. It should prevent unwanted cache >> flush in >> case of timeout or similar problems. >> >> Simo, are you ok with this approach? > > Sounds better. > Ideally you do not flush until you get all results and no errors, but if that's difficult, waiting until the first results come in maybe be a decent first step. > > Simo. > AFAIK there is no SearchResultDone LDAP message, so it is a bit problematic. I created ticket for further improvements: https://fedorahosted.org/bind-dyndb-ldap/ticket/86 Ideas from ticket: We can measure time between ldap_result() calls and say "done" if interval between two received results is greater than x seconds... but it is not very reliable. There is problem with high modification rates (10/sec as mentioned on freeipa-users list), because inter-result interval can be too long for those cases. This "wait interval" can be shortened if result with Entry Change Notification is received ... but it can lead to problem with result interleaving and so on. Further investigation is needed. Petr^2 Spacek From atkac at redhat.com Mon Aug 13 11:01:57 2012 From: atkac at redhat.com (Adam Tkac) Date: Mon, 13 Aug 2012 13:01:57 +0200 Subject: [Freeipa-devel] [PATCH 0045] Fix zone transfers with non-FQDNs In-Reply-To: <5016C899.6050303@redhat.com> References: <5016C899.6050303@redhat.com> Message-ID: <20120813110154.GA14614@redhat.com> On Mon, Jul 30, 2012 at 07:47:05PM +0200, Petr Spacek wrote: > Hello, > > this patch enables (finally!) full-fledged AXFR zone-transfers. > > One-liner was enough, after two days of debugging ... :-) > > I tried to find all consequences for this change, but please test it thoroughly. Ack. > From c73219008ddd8e68ab6bfa7485efc25effc04d1b Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 30 Jul 2012 19:39:14 +0200 > Subject: [PATCH] Fix zone transfers with non-FQDNs. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/47 > > Signed-off-by: Petr Spacek > --- > src/ldap_driver.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/ldap_driver.c b/src/ldap_driver.c > index cae45d4f6cc1f201c40ca3413d1f626e03a0318e..d958d15bdebe5e89dc4948655ffba655073d53e0 100644 > --- a/src/ldap_driver.c > +++ b/src/ldap_driver.c > @@ -689,7 +689,7 @@ createiterator(dns_db_t *db, > > ldapdb_t *ldapdb = (ldapdb_t *) db; > result = ldapdb_nodelist_get(ldapdb->common.mctx, ldapdb->ldap_inst, > - &ldapdb->common.origin, NULL, > + &ldapdb->common.origin, &ldapdb->common.origin, > &ldapiter->nodelist); > > *iteratorp = (dns_dbiterator_t *) ldapiter; > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Mon Aug 13 11:43:25 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 13 Aug 2012 13:43:25 +0200 Subject: [Freeipa-devel] [PATCH 0045] Fix zone transfers with non-FQDNs In-Reply-To: <20120813110154.GA14614@redhat.com> References: <5016C899.6050303@redhat.com> <20120813110154.GA14614@redhat.com> Message-ID: <5028E85D.4010202@redhat.com> On 08/13/2012 01:01 PM, Adam Tkac wrote: > On Mon, Jul 30, 2012 at 07:47:05PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch enables (finally!) full-fledged AXFR zone-transfers. >> >> One-liner was enough, after two days of debugging ... :-) >> >> I tried to find all consequences for this change, but please test it thoroughly. > > Ack. Pushed to the master: http://git.fedorahosted.org/cgit/bind-dyndb-ldap.git/commit/?id=a116dccbbf652f1ace443d3eb3fc7dde793acf13 Petr^2 Spacek From mkosek at redhat.com Mon Aug 13 11:50:46 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 13 Aug 2012 13:50:46 +0200 Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings In-Reply-To: <502580E7.4080503@redhat.com> References: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> <502580E7.4080503@redhat.com> Message-ID: <5028EA16.3040302@redhat.com> On 08/10/2012 11:45 PM, Rob Crittenden wrote: > Robert Crittenden wrote: >> This looks good, I just want to amend the commit with the tickets that this >> addresses before pushing. >> >> It seems that this covers 1670-1674. Is 1392 included as well? >> >> rob >> >> ----- Original Message ----- >> From: "John Dennis" >> To: "freeipa-devel" >> Sent: Friday, August 10, 2012 1:11:23 PM >> Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings >> >> Due to the size and disruptive nature of this patch it was reviewed and >> tested outside the normal patch review process, see the thread "DN patch >> and documentation" as well as other threads on this list. This patch is >> the cumulative result of that review process. >> >> John >> > > Posting this since I haven't pushed the patch yet. I found this installing with > --selfsign: > > diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py > index 1bbee35..eebaa48 100644 > --- a/ipaserver/install/certs.py > +++ b/ipaserver/install/certs.py > @@ -1048,7 +1048,7 @@ class CertDB(object): > # Prepare a simple cert request > req_dict = dict(PASSWORD=self.gen_password(), > SUBJBASE=self.subject_base, > - CERTNAME=DN('CN', nickname)) > + CERTNAME=DN(('CN', nickname))) > req_template = ipautil.SHARE_DIR + reqcfg + ".template" > conf = ipautil.template_file(req_template, req_dict) > fd = open(reqcfg, "w+") > Good job John, I think we are very close to push. I am attaching 2 patch to fix the last major issues I found. One for "ipa-replica-manage list" which crashed when winsync agreements were on and also few blocking issues in migration plugin. Both patches are tested and issues resolved. winsync replica agreements setup will need one more patch. But this issue is not originated in DN - I will send a separate patch based on top of your work. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-DN-usage-in-ipa-replica-manage-list.patch Type: text/x-patch Size: 1007 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Fix-DN-usage-in-migration-plugin.patch Type: text/x-patch Size: 3992 bytes Desc: not available URL: From mkosek at redhat.com Mon Aug 13 12:22:29 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 13 Aug 2012 14:22:29 +0200 Subject: [Freeipa-devel] [PATCH] 292 Fix winsync agreements creation Message-ID: <5028F185.4010601@redhat.com> Due to recent addition of ID range support to DsInstance, the class could no longer be instantiated when realm_name was passed but ID range parameters were not. This condition broke winsync agreements creation in ipa-replica-manage. Make sure that ID range computation in DsInstance does not crash in this cases so that winsync replica can be created. Also convert --binddn option of ipa-replica-manage script to IPA native DN type so that setup_agreement does not crash. https://fedorahosted.org/freeipa/ticket/2987 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-292-fix-winsync-agreements-creation.patch Type: text/x-patch Size: 5678 bytes Desc: not available URL: From pspacek at redhat.com Mon Aug 13 13:15:52 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 13 Aug 2012 15:15:52 +0200 Subject: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query() Message-ID: <5028FE08.4060207@redhat.com> Hello, this patch improves connection management in bind-dyndb-ldap and closes https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . It should prevent all deadlocks on connection pool in future. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0047-Avoid-manual-connection-management-outside-ldap_quer.patch Type: text/x-patch Size: 15009 bytes Desc: not available URL: From jdennis at redhat.com Mon Aug 13 13:30:12 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 13 Aug 2012 09:30:12 -0400 Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings In-Reply-To: <502580E7.4080503@redhat.com> References: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> <502580E7.4080503@redhat.com> Message-ID: <50290164.9060900@redhat.com> On 08/10/2012 05:45 PM, Rob Crittenden wrote: > Robert Crittenden wrote: >> This looks good, I just want to amend the commit with the tickets that this addresses before pushing. >> >> It seems that this covers 1670-1674. Is 1392 included as well? >> >> rob >> >> ----- Original Message ----- >> From: "John Dennis" >> To: "freeipa-devel" >> Sent: Friday, August 10, 2012 1:11:23 PM >> Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings >> >> Due to the size and disruptive nature of this patch it was reviewed and >> tested outside the normal patch review process, see the thread "DN patch >> and documentation" as well as other threads on this list. This patch is >> the cumulative result of that review process. >> >> John >> > > Posting this since I haven't pushed the patch yet. I found this > installing with --selfsign: > > diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py > index 1bbee35..eebaa48 100644 > --- a/ipaserver/install/certs.py > +++ b/ipaserver/install/certs.py > @@ -1048,7 +1048,7 @@ class CertDB(object): > # Prepare a simple cert request > req_dict = dict(PASSWORD=self.gen_password(), > SUBJBASE=self.subject_base, > - CERTNAME=DN('CN', nickname)) > + CERTNAME=DN(('CN', nickname))) > req_template = ipautil.SHARE_DIR + reqcfg + ".template" > conf = ipautil.template_file(req_template, req_dict) > fd = open(reqcfg, "w+") > > ACK, please apply this on top of the original patch before pushing -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Mon Aug 13 13:30:28 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 13 Aug 2012 09:30:28 -0400 Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings In-Reply-To: <5028EA16.3040302@redhat.com> References: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> <502580E7.4080503@redhat.com> <5028EA16.3040302@redhat.com> Message-ID: <50290174.4050101@redhat.com> On 08/13/2012 07:50 AM, Martin Kosek wrote: > On 08/10/2012 11:45 PM, Rob Crittenden wrote: >> Robert Crittenden wrote: >>> This looks good, I just want to amend the commit with the tickets that this >>> addresses before pushing. >>> >>> It seems that this covers 1670-1674. Is 1392 included as well? >>> >>> rob >>> >>> ----- Original Message ----- >>> From: "John Dennis" >>> To: "freeipa-devel" >>> Sent: Friday, August 10, 2012 1:11:23 PM >>> Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings >>> >>> Due to the size and disruptive nature of this patch it was reviewed and >>> tested outside the normal patch review process, see the thread "DN patch >>> and documentation" as well as other threads on this list. This patch is >>> the cumulative result of that review process. >>> >>> John >>> >> >> Posting this since I haven't pushed the patch yet. I found this installing with >> --selfsign: >> >> diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py >> index 1bbee35..eebaa48 100644 >> --- a/ipaserver/install/certs.py >> +++ b/ipaserver/install/certs.py >> @@ -1048,7 +1048,7 @@ class CertDB(object): >> # Prepare a simple cert request >> req_dict = dict(PASSWORD=self.gen_password(), >> SUBJBASE=self.subject_base, >> - CERTNAME=DN('CN', nickname)) >> + CERTNAME=DN(('CN', nickname))) >> req_template = ipautil.SHARE_DIR + reqcfg + ".template" >> conf = ipautil.template_file(req_template, req_dict) >> fd = open(reqcfg, "w+") >> > > > Good job John, I think we are very close to push. I am attaching 2 patch to fix > the last major issues I found. One for "ipa-replica-manage list" which crashed > when winsync agreements were on and also few blocking issues in migration > plugin. Both patches are tested and issues resolved. > > winsync replica agreements setup will need one more patch. But this issue is > not originated in DN - I will send a separate patch based on top of your work. > > Martin > ACK, please apply this on top of the original patch before pushing -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pspacek at redhat.com Mon Aug 13 13:31:44 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 13 Aug 2012 15:31:44 +0200 Subject: [Freeipa-devel] [PATCH 0048] Lower minimal connection count for persistent search. Message-ID: <502901C0.9070905@redhat.com> Hello, As result of better connection management it is possible to run persistent search with smaller connection pool. Attached patch updates built-in configuration checks with new lower bound. It closes patch series for https://fedorahosted.org/bind-dyndb-ldap/ticket/68: Avoid manual connection management outside ldap_query() Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0048-Lower-minimal-connection-count-for-persistent-search.patch Type: text/x-patch Size: 1834 bytes Desc: not available URL: From rcritten at redhat.com Mon Aug 13 13:56:47 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 13 Aug 2012 09:56:47 -0400 Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings In-Reply-To: <5028EA16.3040302@redhat.com> References: <1653975251.6026787.1344632598104.JavaMail.root@redhat.com> <502580E7.4080503@redhat.com> <5028EA16.3040302@redhat.com> Message-ID: <5029079F.5090802@redhat.com> Martin Kosek wrote: > On 08/10/2012 11:45 PM, Rob Crittenden wrote: >> Robert Crittenden wrote: >>> This looks good, I just want to amend the commit with the tickets that this >>> addresses before pushing. >>> >>> It seems that this covers 1670-1674. Is 1392 included as well? >>> >>> rob >>> >>> ----- Original Message ----- >>> From: "John Dennis" >>> To: "freeipa-devel" >>> Sent: Friday, August 10, 2012 1:11:23 PM >>> Subject: [Freeipa-devel] [PATCH 76] Use DN objects instead of strings >>> >>> Due to the size and disruptive nature of this patch it was reviewed and >>> tested outside the normal patch review process, see the thread "DN patch >>> and documentation" as well as other threads on this list. This patch is >>> the cumulative result of that review process. >>> >>> John >>> >> >> Posting this since I haven't pushed the patch yet. I found this installing with >> --selfsign: >> >> diff --git a/ipaserver/install/certs.py b/ipaserver/install/certs.py >> index 1bbee35..eebaa48 100644 >> --- a/ipaserver/install/certs.py >> +++ b/ipaserver/install/certs.py >> @@ -1048,7 +1048,7 @@ class CertDB(object): >> # Prepare a simple cert request >> req_dict = dict(PASSWORD=self.gen_password(), >> SUBJBASE=self.subject_base, >> - CERTNAME=DN('CN', nickname)) >> + CERTNAME=DN(('CN', nickname))) >> req_template = ipautil.SHARE_DIR + reqcfg + ".template" >> conf = ipautil.template_file(req_template, req_dict) >> fd = open(reqcfg, "w+") >> > > > Good job John, I think we are very close to push. I am attaching 2 patch to fix > the last major issues I found. One for "ipa-replica-manage list" which crashed > when winsync agreements were on and also few blocking issues in migration > plugin. Both patches are tested and issues resolved. > > winsync replica agreements setup will need one more patch. But this issue is > not originated in DN - I will send a separate patch based on top of your work. > > Martin > Squashed these in, along with my certs patch, and pushed the whole thing to master. rob From rcritten at redhat.com Mon Aug 13 20:43:23 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 13 Aug 2012 16:43:23 -0400 Subject: [Freeipa-devel] [PATCH] 292 Fix winsync agreements creation In-Reply-To: <5028F185.4010601@redhat.com> References: <5028F185.4010601@redhat.com> Message-ID: <502966EB.9060308@redhat.com> Martin Kosek wrote: > Due to recent addition of ID range support to DsInstance, the class > could no longer be instantiated when realm_name was passed but > ID range parameters were not. This condition broke winsync agreements > creation in ipa-replica-manage. > > Make sure that ID range computation in DsInstance does not crash in > this cases so that winsync replica can be created. Also convert --binddn > option of ipa-replica-manage script to IPA native DN type so that > setup_agreement does not crash. > > https://fedorahosted.org/freeipa/ticket/2987 ACK, pushed to master. rob From edewata at redhat.com Mon Aug 13 22:50:55 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 13 Aug 2012 17:50:55 -0500 Subject: [Freeipa-devel] [PATCH] 186-192 Web UI: Group external member support In-Reply-To: <5024EAB3.9030906@redhat.com> References: <501BC6C0.4030504@redhat.com> <502025A7.9010208@redhat.com> <5024EAB3.9030906@redhat.com> Message-ID: <502984CF.201@redhat.com> On 8/10/2012 6:04 AM, Petr Vobornik wrote: > Attaching all patches. Patches 186-189 are changed or rebased. Rest is > intact. ACK. -- Endi S. Dewata From edewata at redhat.com Tue Aug 14 00:41:01 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 13 Aug 2012 19:41:01 -0500 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI In-Reply-To: <501F6D74.7050509@redhat.com> References: <501F6D74.7050509@redhat.com> Message-ID: <50299E9D.9060309@redhat.com> On 8/6/2012 2:08 AM, Petr Vobornik wrote: > Range web UI was implemented. > > It consist of: > * new menu item - 'ranges' in 'IPA Server' tab > * new search page > * new details page > > https://fedorahosted.org/freeipa/ticket/2894 > > > Some comments/questions: > 1) I'm not sure about options in adder dialog. Should they all be there > or should I remove some? And also labels seems to long. Depending on the range type each of these options might be required, so it's necessary to show them all in the adder dialog, but they can be enabled/disabled (see #4). The labels probably can be shortened to mimic the CLI parameters, and the current labels can be moved into the doc attributes. So it may look something like this (feel free to change): Add ID Range ------------------------------------------------------ Range name: [ ] Range type: (*) Local domain ( ) Active Directory domain Base ID: [ ] Range size: [ ] Primary RID base: [ ] Secondary RID base: [ ] Domain SID: [ ] > 2) FreeIPA creates default range. This range doesn't have required > attribute 'First RID of the corresponding RID range'. It prevents from > editing the range in Web UI until a some value is set. Not sure if even > should be edited though. Someone else more knowledgeable should answer this. One possibility is to introduce another range type (e.g. POSIX) that doesn't use RID. 3. As shown in #1, it might be better to call it "ID Ranges" as in the CLI. "Ranges" by itself doesn't sound very meaningful. 4. In range.py the range type seems to be not user enterable/editable and automatically generated based on the domain SID. However, in the adder dialog the range type options can be used to enable/disable the appropriate fields for the type. For example, selecting "local domain" will make the secondary RID base required. 5. This is a CLI issue. If you call ipa range-add without parameters it will ask for the parameters, but then it will fail before asking the secondary RID base or domain SID: # ipa range-add Range name: test First Posix ID of the range: 10000 Number of IDs in the range: 100 First RID of the corresponding RID range: 50 ipa: ERROR: invalid Gettext('Range setup', domain='ipa', localedir=None): Ranges for local domain must have a secondary RID base -- Endi S. Dewata From edewata at redhat.com Tue Aug 14 01:21:49 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 13 Aug 2012 20:21:49 -0500 Subject: [Freeipa-devel] [PATCH] 194 Handle case when trusted domain user access the Web UI In-Reply-To: <5024EC72.1000902@redhat.com> References: <5024EC72.1000902@redhat.com> Message-ID: <5029A82D.7000105@redhat.com> On 8/10/2012 6:11 AM, Petr Vobornik wrote: > WebUI catches the fact that the user can't access LDAP server with a > current ticket. It shows form-based auth login dialog. Previoustly an > ugly error was returned on an almost empty page, and user had no recourse. > > https://fedorahosted.org/freeipa/ticket/2897 > > > I don't like the implementation much. Problem is that we don't separate > state variables and framework objects in IPA object. It is probably a > topic for fixing in 3.2. I don't have an environment to test this, but the code looks fine, so it's ACKed. Some comments: 1. The logged_kerberos and logged_password cannot be true at the same time, right? Maybe they can be combined into a single variable (e.g. login_status) which different values for unauthenticated, logged in via kerberos, and logged in via password. Maybe the 'initialized' variable can be combined too. 2. I agree about the state variables & framework objects separation. Currently the 'IPA' object is both used as a singleton/global variable and also as a name space for the framework. I think ideally we should use a generic/non-IPA specific name for the framework. Probably something like this: // UI Framework var UI = { ... }; UI.entity = function() { ... }; UI.facet = function() { ... }; // IPA UI var IPA = UI(); IPA.user.entity = function() { ... }; IPA.user.details_facet = function() { ... }; IPA.init(); -- Endi S. Dewata From edewata at redhat.com Tue Aug 14 01:29:33 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 13 Aug 2012 20:29:33 -0500 Subject: [Freeipa-devel] [PATCH] 195 Disable caching of Web UI login_kerberos request In-Reply-To: <5024ECDF.3010607@redhat.com> References: <5024ECDF.3010607@redhat.com> Message-ID: <5029A9FD.6050505@redhat.com> On 8/10/2012 6:13 AM, Petr Vobornik wrote: > IE caches login_kerberos request so SSO doesn't work after logout. This > patch disables the caching. > > https://fedorahosted.org/freeipa/ticket/2991 ACK. -- Endi S. Dewata From edewata at redhat.com Tue Aug 14 01:35:36 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 13 Aug 2012 20:35:36 -0500 Subject: [Freeipa-devel] [PATCH] 196 Update other facets on delete from search page In-Reply-To: <5024F4F9.2050306@redhat.com> References: <5024F4F9.2050306@redhat.com> Message-ID: <5029AB68.1030508@redhat.com> On 8/10/2012 6:48 AM, Petr Vobornik wrote: > When an object in search facet was deleted, other facets were not > notified that they need to refresh. If one crated object with same pkey > as deleted object and then navigated to it's details he could see old > object's data. > > This notification was added. > > https://fedorahosted.org/freeipa/ticket/2618 > > Note: I found a refresh issue in search facet while using 'Add and edit' > in adder dialog. This will be fixed separately. ACK. -- Endi S. Dewata From mkosek at redhat.com Tue Aug 14 06:18:51 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 08:18:51 +0200 Subject: [Freeipa-devel] [PATCH] 186-192 Web UI: Group external member support In-Reply-To: <502984CF.201@redhat.com> References: <501BC6C0.4030504@redhat.com> <502025A7.9010208@redhat.com> <5024EAB3.9030906@redhat.com> <502984CF.201@redhat.com> Message-ID: <5029EDCB.0@redhat.com> On 08/14/2012 12:50 AM, Endi Sukma Dewata wrote: > On 8/10/2012 6:04 AM, Petr Vobornik wrote: >> Attaching all patches. Patches 186-189 are changed or rebased. Rest is >> intact. > > ACK. > Pushed all to master. Martin From mkosek at redhat.com Tue Aug 14 06:22:05 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 08:22:05 +0200 Subject: [Freeipa-devel] [PATCH] 194 Handle case when trusted domain user access the Web UI In-Reply-To: <5029A82D.7000105@redhat.com> References: <5024EC72.1000902@redhat.com> <5029A82D.7000105@redhat.com> Message-ID: <5029EE8D.7070003@redhat.com> On 08/14/2012 03:21 AM, Endi Sukma Dewata wrote: > On 8/10/2012 6:11 AM, Petr Vobornik wrote: >> WebUI catches the fact that the user can't access LDAP server with a >> current ticket. It shows form-based auth login dialog. Previoustly an >> ugly error was returned on an almost empty page, and user had no recourse. >> >> https://fedorahosted.org/freeipa/ticket/2897 >> >> >> I don't like the implementation much. Problem is that we don't separate >> state variables and framework objects in IPA object. It is probably a >> topic for fixing in 3.2. > > I don't have an environment to test this, but the code looks fine, so it's ACKed. > > Some comments: > > 1. The logged_kerberos and logged_password cannot be true at the same time, > right? Maybe they can be combined into a single variable (e.g. login_status) > which different values for unauthenticated, logged in via kerberos, and logged > in via password. Maybe the 'initialized' variable can be combined too. > > 2. I agree about the state variables & framework objects separation. Currently > the 'IPA' object is both used as a singleton/global variable and also as a name > space for the framework. I think ideally we should use a generic/non-IPA > specific name for the framework. Probably something like this: > > // UI Framework > var UI = { ... }; > UI.entity = function() { ... }; > UI.facet = function() { ... }; > > // IPA UI > var IPA = UI(); > IPA.user.entity = function() { ... }; > IPA.user.details_facet = function() { ... }; > > IPA.init(); > Pushed to master. Petr, please follow up with Endi on these comments when you return. Martin From mkosek at redhat.com Tue Aug 14 06:25:40 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 08:25:40 +0200 Subject: [Freeipa-devel] [PATCH] 195 Disable caching of Web UI login_kerberos request In-Reply-To: <5029A9FD.6050505@redhat.com> References: <5024ECDF.3010607@redhat.com> <5029A9FD.6050505@redhat.com> Message-ID: <5029EF64.7060600@redhat.com> On 08/14/2012 03:29 AM, Endi Sukma Dewata wrote: > On 8/10/2012 6:13 AM, Petr Vobornik wrote: >> IE caches login_kerberos request so SSO doesn't work after logout. This >> patch disables the caching. >> >> https://fedorahosted.org/freeipa/ticket/2991 > > ACK. > Pushed to master. Martin From mkosek at redhat.com Tue Aug 14 06:28:34 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 08:28:34 +0200 Subject: [Freeipa-devel] [PATCH] 196 Update other facets on delete from search page In-Reply-To: <5029AB68.1030508@redhat.com> References: <5024F4F9.2050306@redhat.com> <5029AB68.1030508@redhat.com> Message-ID: <5029F012.3050707@redhat.com> On 08/14/2012 03:35 AM, Endi Sukma Dewata wrote: > On 8/10/2012 6:48 AM, Petr Vobornik wrote: >> When an object in search facet was deleted, other facets were not >> notified that they need to refresh. If one crated object with same pkey >> as deleted object and then navigated to it's details he could see old >> object's data. >> >> This notification was added. >> >> https://fedorahosted.org/freeipa/ticket/2618 >> >> Note: I found a refresh issue in search facet while using 'Add and edit' >> in adder dialog. This will be fixed separately. > > ACK. > Pushed to master. Martin From mkosek at redhat.com Tue Aug 14 06:41:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 08:41:44 +0200 Subject: [Freeipa-devel] [PATCH] 0004-2 Corrects help description of selinuxusermap. In-Reply-To: <379666122.977071.1344609871858.JavaMail.root@redhat.com> References: <379666122.977071.1344609871858.JavaMail.root@redhat.com> Message-ID: <5029F328.6030109@redhat.com> Looking at the changes, I think there is still one line that does not make much sense: - Display the properties of a named HBAC rule: + Display the properties of a HBAC rule: ipa selinuxusermap-show test1 This should rather really show a HBAC rule used in previous command (hbacrule-show command) or not mention HBAC rule at all and show test2 rule added in previous step. Martin On 08/10/2012 04:44 PM, Tomas Babej wrote: > Suggestion incorporated. > > Tomas > > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Friday, August 10, 2012 3:43:06 PM > Subject: Re: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. > > On 08/10/2012 01:09 PM, Tomas Babej wrote: >> I forgot the patch once again. >> >> ----- Original Message ----- >> From: "Tomas Babej" >> To: freeipa-devel at redhat.com >> Sent: Friday, August 10, 2012 12:44:41 PM >> Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. >> >> Hi, >> >> I checked the rest of the description as well, seems alright. >> >> https://fedorahosted.org/freeipa/ticket/2959 >> >> Tomas >> > > That's a good start. I would also change the following descriptions a bit: > > Disable a named rule: > ipa selinuxusermap-disable test1 > > Enable a named rule: > ipa selinuxusermap-enable test1 > > ... > > Remove a named rule: > ipa selinuxusermap-del john_unconfined > > I don't know why all rule are specified as "named". I think that simple > > Disable a rule: > Enable a rule: > Remove a rule: > > would be enough and less confusing. > > Martin > > From mkosek at redhat.com Tue Aug 14 06:55:19 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 08:55:19 +0200 Subject: [Freeipa-devel] [PATCH] 0005-2 Improves exception handling in ipa-replica-prepare. In-Reply-To: <222233737.971584.1344609143323.JavaMail.root@redhat.com> References: <222233737.971584.1344609143323.JavaMail.root@redhat.com> Message-ID: <5029F657.9010000@redhat.com> On 08/10/2012 04:32 PM, Tomas Babej wrote: > Man page edited. > > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Friday, August 10, 2012 3:49:45 PM > Subject: Re: [Freeipa-devel] [PATCH] 0005 Improves exception handling in ipa-replica-prepare. > > On 08/10/2012 03:07 PM, Tomas Babej wrote: >> Hi, >> >> A backtrace is no longer displayed when trying to prepare a replica >> file with the local LDAP server down. Also adds --debug option and >> no longer displays info log messages without it. >> >> https://fedorahosted.org/freeipa/ticket/2939 >> >> Tomas >> > > Looks OK. We just also need to update the ipa-replica-prepare man page, when > changing its options. > > Martin > I tested the patch and the ipa-replica-prepare part works fine, except for the man page which is not formatted correctly. Please, always test your changes in man pages, it is easy to do a formatting error here. Martin From mkosek at redhat.com Tue Aug 14 07:26:03 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 09:26:03 +0200 Subject: [Freeipa-devel] [PATCH] 0003 Handle SSSD restart crash more gently. In-Reply-To: <356227500.348722.1344356083497.JavaMail.root@redhat.com> References: <356227500.348722.1344356083497.JavaMail.root@redhat.com> Message-ID: <5029FD8B.9050201@redhat.com> On 08/07/2012 06:14 PM, Tomas Babej wrote: > Sorry, I forgot to attach the patch. > > ----- Original Message ----- > From: "Tomas Babej" > To: freeipa-devel at redhat.com > Sent: Tuesday, August 7, 2012 5:58:32 PM > Subject: [PATCH] Handle SSSD restart crash more gently. > > Hi, > > In ipa-client-install, failure of restart of sssd service no longer > causes the crash of the install process. Adds a warning message to > the root logger instead. > > https://fedorahosted.org/freeipa/ticket/2827 > > Tomas > This works fine, I see just one issue. It is better to catch for more specific exceptions that with the bare "except" clause. Otherwise for example in this case, if the sssd restart would got stuck, one could not CTRL+C it properly as the bare except clause would catch the appropriate exception. You may want to check with PEP8 on that one. I would suggest rather catching CalledProcessError only. Martin From sbose at redhat.com Tue Aug 14 07:34:12 2012 From: sbose at redhat.com (Sumit Bose) Date: Tue, 14 Aug 2012 09:34:12 +0200 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI In-Reply-To: <50299E9D.9060309@redhat.com> References: <501F6D74.7050509@redhat.com> <50299E9D.9060309@redhat.com> Message-ID: <20120814073412.GE11558@localhost.localdomain> On Mon, Aug 13, 2012 at 07:41:01PM -0500, Endi Sukma Dewata wrote: > On 8/6/2012 2:08 AM, Petr Vobornik wrote: > >Range web UI was implemented. > > > >It consist of: > > * new menu item - 'ranges' in 'IPA Server' tab > > * new search page > > * new details page > > > >https://fedorahosted.org/freeipa/ticket/2894 > > > > > >Some comments/questions: > >1) I'm not sure about options in adder dialog. Should they all be there > >or should I remove some? And also labels seems to long. > > Depending on the range type each of these options might be required, > so it's necessary to show them all in the adder dialog, but they can > be enabled/disabled (see #4). The labels probably can be shortened > to mimic the CLI parameters, and the current labels can be moved > into the doc attributes. So it may look something like this (feel > free to change): > > Add ID Range > ------------------------------------------------------ > Range name: [ ] > Range type: (*) Local domain > ( ) Active Directory domain > > Base ID: [ ] > Range size: [ ] > > Primary RID base: [ ] > Secondary RID base: [ ] > > Domain SID: [ ] I agree, all options should be display and depending on the Range type either Secondary RID base or Domain SID should be greyed out. > > >2) FreeIPA creates default range. This range doesn't have required > >attribute 'First RID of the corresponding RID range'. It prevents from > >editing the range in Web UI until a some value is set. Not sure if even > >should be edited though. > > Someone else more knowledgeable should answer this. One possibility > is to introduce another range type (e.g. POSIX) that doesn't use > RID. ah, the attribute is not required by the schema, but the CLI currently requires it. I was focused on trusts while writing it, but in general it makes sense to allow to add an ID range with only base ID and range size. One possible use case would be a stand alone IPA domain which runs out of Posix ID because the range created at installation time is full. In this case it makes no sense to add the RID attributes. I have created https://fedorahosted.org/freeipa/ticket/2999 to track this and also the issue without parameters mentioned below. > > 3. As shown in #1, it might be better to call it "ID Ranges" as in > the CLI. "Ranges" by itself doesn't sound very meaningful. yes > > 4. In range.py the range type seems to be not user > enterable/editable and automatically generated based on the domain > SID. However, in the adder dialog the range type options can be used > to enable/disable the appropriate fields for the type. For example, > selecting "local domain" will make the secondary RID base required. yes, see my comment above > > 5. This is a CLI issue. If you call ipa range-add without parameters > it will ask for the parameters, but then it will fail before asking > the secondary RID base or domain SID: > > # ipa range-add > Range name: test > First Posix ID of the range: 10000 > Number of IDs in the range: 100 > First RID of the corresponding RID range: 50 > ipa: ERROR: invalid Gettext('Range setup', domain='ipa', > localedir=None): Ranges for local domain must have a secondary RID > base https://fedorahosted.org/freeipa/ticket/2999 bye, Sumit From abokovoy at redhat.com Tue Aug 14 08:58:43 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 14 Aug 2012 11:58:43 +0300 Subject: [Freeipa-devel] [PATCH] 0069 Add internationalization to DCE RPC code Message-ID: <20120814085842.GE3783@redhat.com> Hi, fairly simple patch to wrap error messages to look up in translation catalog. https://fedorahosted.org/freeipa/ticket/2964 -- / Alexander Bokovoy -------------- next part -------------- >From 203aa5c5711842eef6c3bcdb8ba6b956855e9217 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 13 Aug 2012 16:35:19 +0300 Subject: [PATCH 3/3] Add internationalization to DCE RPC code https://fedorahosted.org/freeipa/ticket/2964 --- ipaserver/dcerpc.py | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/ipaserver/dcerpc.py b/ipaserver/dcerpc.py index 982e12b31a9525a35c5d921e6559471a7633e025..f754664baeeb38d3da201459103171097de093b2 100644 --- a/ipaserver/dcerpc.py +++ b/ipaserver/dcerpc.py @@ -50,18 +50,22 @@ The code in this module relies heavily on samba4-python package and Samba4 python bindings. """) -access_denied_error = errors.ACIError(info='CIFS server denied your credentials') +access_denied_error = errors.ACIError(info=_('CIFS server denied your credentials')) dcerpc_error_codes = { - -1073741823: errors.RemoteRetrieveError(reason='communication with CIFS server was unsuccessful'), + -1073741823: + errors.RemoteRetrieveError(reason=_('communication with CIFS server was unsuccessful')), -1073741790: access_denied_error, -1073741715: access_denied_error, -1073741614: access_denied_error, - -1073741603: errors.ValidationError(name='AD domain controller', error='unsupported functional level'), + -1073741603: + errors.ValidationError(name=_('AD domain controller'), error=_('unsupported functional level')), } dcerpc_error_messages = { - "NT_STATUS_OBJECT_NAME_NOT_FOUND": errors.NotFound(reason='Cannot find specified domain or server name'), - "NT_STATUS_INVALID_PARAMETER_MIX": errors.RequirementError(name='At least the domain or IP address should be specified'), + "NT_STATUS_OBJECT_NAME_NOT_FOUND": + errors.NotFound(reason=_('Cannot find specified domain or server name')), + "NT_STATUS_INVALID_PARAMETER_MIX": + errors.RequirementError(name=_('At least the domain or IP address should be specified')), } def assess_dcerpc_exception(num=None,message=None): @@ -73,7 +77,9 @@ def assess_dcerpc_exception(num=None,message=None): return dcerpc_error_codes[num] if message and message in dcerpc_error_messages: return dcerpc_error_messages[message] - return errors.RemoteRetrieveError(reason='CIFS server communication error: code "%s", message "%s" (both may be "None")' % (num, message)) + reason = _('''CIFS server communication error: code "%(num)s", + message "%(message)s" (both may be "None")''') % dict(num=num, message=message) + return errors.RemoteRetrieveError(reason=reason) class ExtendedDNControl(_ldap.controls.RequestControl): def __init__(self): @@ -173,7 +179,7 @@ class TrustDomainInstance(object): def __gen_lsa_connection(self, binding): if self.creds is None: - raise errors.RequirementError(name='CIFS credentials object') + raise errors.RequirementError(name=_('CIFS credentials object')) try: result = lsa.lsarpc(binding, self.parm, self.creds) return result @@ -205,10 +211,12 @@ class TrustDomainInstance(object): attempts = attempts + 1 if self._pipe is None and attempts == len(bindings): - raise errors.ACIError(info='CIFS server %s denied your credentials' % (remote_host)) + raise errors.ACIError( + info=_('CIFS server %(host)s denied your credentials') % dict(host=remote_host)) if self._pipe is None: - raise errors.RemoteRetrieveError(reason='Cannot establish LSA connection to %s. Is CIFS server running?' % (remote_host)) + raise errors.RemoteRetrieveError( + reason=_('Cannot establish LSA connection to %(host)s. Is CIFS server running?') % dict(host=remote_host)) def __gen_lsa_bindings(self, remote_host): """ @@ -255,7 +263,9 @@ class TrustDomainInstance(object): result = res['defaultNamingContext'][0] self.info['dns_hostname'] = res['dnsHostName'][0] except _ldap.LDAPError, e: - root_logger.error("LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e))) + root_logger.error( + _("LDAP error when connecting to %(host)s: %(error)s") % + dict(host=unicode(result.pdc_name), error=str(e))) if result: self.info['sid'] = self.parse_naming_context(result) -- 1.7.11.2 From pspacek at redhat.com Tue Aug 14 12:32:55 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 14 Aug 2012 14:32:55 +0200 Subject: [Freeipa-devel] [PATCH 0049] Fix two memory leaks in persistent search Message-ID: <502A4577.80702@redhat.com> Hello, This patch fixes two memory leaks in persistent search. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0049-Fix-two-memory-leaks-in-persistent-search.patch Type: text/x-patch Size: 3824 bytes Desc: not available URL: From tbabej at redhat.com Tue Aug 14 13:12:20 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 14 Aug 2012 09:12:20 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0003-2 Handle SSSD restart crash more gently. In-Reply-To: <5029FD8B.9050201@redhat.com> Message-ID: <1570075244.1386419.1344949940166.JavaMail.root@redhat.com> Issue resolved. Tomas ----- Original Message ----- From: "Martin Kosek" To: "Tomas Babej" Cc: freeipa-devel at redhat.com Sent: Tuesday, August 14, 2012 9:26:03 AM Subject: Re: [Freeipa-devel] [PATCH] 0003 Handle SSSD restart crash more gently. On 08/07/2012 06:14 PM, Tomas Babej wrote: > Sorry, I forgot to attach the patch. > > ----- Original Message ----- > From: "Tomas Babej" > To: freeipa-devel at redhat.com > Sent: Tuesday, August 7, 2012 5:58:32 PM > Subject: [PATCH] Handle SSSD restart crash more gently. > > Hi, > > In ipa-client-install, failure of restart of sssd service no longer > causes the crash of the install process. Adds a warning message to > the root logger instead. > > https://fedorahosted.org/freeipa/ticket/2827 > > Tomas > This works fine, I see just one issue. It is better to catch for more specific exceptions that with the bare "except" clause. Otherwise for example in this case, if the sssd restart would got stuck, one could not CTRL+C it properly as the bare except clause would catch the appropriate exception. You may want to check with PEP8 on that one. I would suggest rather catching CalledProcessError only. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0003-2-Handle-SSSD-restart-crash-more-gently.patch Type: text/x-patch Size: 1253 bytes Desc: not available URL: From tbabej at redhat.com Tue Aug 14 13:17:27 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 14 Aug 2012 09:17:27 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0004-3 Corrects help description of selinuxusermap. In-Reply-To: <5029F328.6030109@redhat.com> Message-ID: <1588279350.1387671.1344950247681.JavaMail.root@redhat.com> ----- Original Message ----- From: "Martin Kosek" To: "Tomas Babej" Cc: freeipa-devel at redhat.com Sent: Tuesday, August 14, 2012 8:41:44 AM Subject: Re: [Freeipa-devel] [PATCH] 0004-2 Corrects help description of selinuxusermap. Looking at the changes, I think there is still one line that does not make much sense: - Display the properties of a named HBAC rule: + Display the properties of a HBAC rule: ipa selinuxusermap-show test1 This should rather really show a HBAC rule used in previous command (hbacrule-show command) or not mention HBAC rule at all and show test2 rule added in previous step. Martin On 08/10/2012 04:44 PM, Tomas Babej wrote: > Suggestion incorporated. > > Tomas > > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Friday, August 10, 2012 3:43:06 PM > Subject: Re: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. > > On 08/10/2012 01:09 PM, Tomas Babej wrote: >> I forgot the patch once again. >> >> ----- Original Message ----- >> From: "Tomas Babej" >> To: freeipa-devel at redhat.com >> Sent: Friday, August 10, 2012 12:44:41 PM >> Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. >> >> Hi, >> >> I checked the rest of the description as well, seems alright. >> >> https://fedorahosted.org/freeipa/ticket/2959 >> >> Tomas >> > > That's a good start. I would also change the following descriptions a bit: > > Disable a named rule: > ipa selinuxusermap-disable test1 > > Enable a named rule: > ipa selinuxusermap-enable test1 > > ... > > Remove a named rule: > ipa selinuxusermap-del john_unconfined > > I don't know why all rule are specified as "named". I think that simple > > Disable a rule: > Enable a rule: > Remove a rule: > > would be enough and less confusing. > > Martin > > I chose the second variant. As these are examples for ipa selinuxusermap, it seemed more appropriate. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0004-3-Corrects-help-description-of-selinuxusermap.patch Type: text/x-patch Size: 1825 bytes Desc: not available URL: From tbabej at redhat.com Tue Aug 14 13:22:16 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 14 Aug 2012 09:22:16 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0005-3 Improves exception handling in ipa-replica-prepare. In-Reply-To: <5029F657.9010000@redhat.com> Message-ID: <2016063526.1389378.1344950536604.JavaMail.root@redhat.com> ----- Original Message ----- From: "Martin Kosek" To: "Tomas Babej" Cc: freeipa-devel at redhat.com Sent: Tuesday, August 14, 2012 8:55:19 AM Subject: Re: [Freeipa-devel] [PATCH] 0005-2 Improves exception handling in ipa-replica-prepare. >On 08/10/2012 04:32 PM, Tomas Babej wrote: >> Man page edited. >> >> ----- Original Message ----- >> From: "Martin Kosek" >> To: "Tomas Babej" >> Cc: freeipa-devel at redhat.com >> Sent: Friday, August 10, 2012 3:49:45 PM >> Subject: Re: [Freeipa-devel] [PATCH] 0005 Improves exception handling in ipa-replica-prepare. >> >> On 08/10/2012 03:07 PM, Tomas Babej wrote: >>> Hi, >>> >>> A backtrace is no longer displayed when trying to prepare a replica >>> file with the local LDAP server down. Also adds --debug option and >>> no longer displays info log messages without it. >>> >>> https://fedorahosted.org/freeipa/ticket/2939 >>> >>> Tomas >>> >> >> Looks OK. We just also need to update the ipa-replica-prepare man page, when >> changing its options. >> >> Martin >> > > I tested the patch and the ipa-replica-prepare part works fine, except for the > man page which is not formatted correctly. > > Please, always test your changes in man pages, it is easy to do a formatting > error here. > > Martin I corrected the hidden glitch in the man page. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0005-3-Improves-exception-handling-in-ipa-replica-prepare.patch Type: text/x-patch Size: 2652 bytes Desc: not available URL: From rcritten at redhat.com Tue Aug 14 13:27:06 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 14 Aug 2012 09:27:06 -0400 Subject: [Freeipa-devel] [PATCH] 1041 pull in cachememsize logging Message-ID: <502A522A.1020507@redhat.com> 389-ds-base added logging if the entry cache is smaller than the database so users will know they need to tune their DS install. Set this as the minimum for IPA. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1041-cachesize.patch Type: text/x-diff Size: 1453 bytes Desc: not available URL: From sbose at redhat.com Tue Aug 14 13:38:26 2012 From: sbose at redhat.com (Sumit Bose) Date: Tue, 14 Aug 2012 15:38:26 +0200 Subject: [Freeipa-devel] [PATCH] trust CLI: add ID range for new trusted domain Message-ID: <20120814133826.GG11558@localhost.localdomain> Hi, currently only a default ID range was used for users from trusted domains. With these two patches an individual range is created during ipa trust-add and it will be used by the extdom plugin to calculate the Poisx UID for the users from the trusted domain. 'ipa trust-add' is getting two new options, --base-id and --range-size to specify the first Posix ID of the range and the size of the range respectively. If --range-size is not given the default will be 200000 and if --base-id is not given it will be calculated with the help of a hash of the domain SID. To be compatible with the AD provider of SSSD murmurhash3 is used here. The python binding for the hash will be provided by SSSD, the patch is currently under review. But since it is not required to have murmurhash3, an error message will be send if it is not installed on the server, I think this patch can be pushed independently of the SSSD patch. bye, Sumit -------------- next part -------------- From f9515cb32526a078a01604c072a7bc6e9b265b19 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 6 Aug 2012 14:30:38 +0200 Subject: [PATCH 1/2] extdom: read ranges from LDAP --- .../ipa-extdom-extop/ipa_extdom_common.c | 72 ++++++++++++++++++++ 1 Datei ge?ndert, 72 Zeilen hinzugef?gt(+) diff --git a/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c b/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c index 294b00d50dd76c6541831b5c53bf70a4d377dcc3..f48bead04cbb18b040557b7b78a6cb27a3368422 100644 --- a/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c +++ b/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c @@ -162,6 +162,72 @@ static void free_domain_info(struct domain_info *domain_info) free(domain_info); } +static int set_domain_range(struct ipa_extdom_ctx *ctx, const char *dom_sid_str, + struct sss_idmap_range *range) +{ + Slapi_PBlock *pb = NULL; + Slapi_Entry **e = NULL; + char *filter = NULL; + int ret; + unsigned long ulong_val; + + pb = slapi_pblock_new(); + if (pb == NULL) { + return ENOMEM; + } + + ret = asprintf(&filter, "(&(ipaNTTrustedDomainSID=%s)" \ + "(objectclass=ipaTrustedADDomainRange))", + dom_sid_str); + if (ret == -1) { + ret = ENOMEM; + goto done; + } + + slapi_search_internal_set_pb(pb, ctx->base_dn, + LDAP_SCOPE_SUBTREE, filter, + NULL, 0, NULL, NULL, ctx->plugin_id, 0); + + slapi_search_internal_pb(pb); + slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &ret); + + if (ret != EOK) { + ret = ENOENT; + goto done; + } + + slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_SEARCH_ENTRIES, &e); + if (!e || !e[0]) { + /* no matches */ + ret = ENOENT; + goto done; + } + + /* TODO: handle more than one range per domain */ + ulong_val = slapi_entry_attr_get_ulong(e[0], "ipaBaseID"); + if (ulong_val >= UINT32_MAX) { + ret = EINVAL; + goto done; + } + range->min = (uint32_t) ulong_val; + + ulong_val = slapi_entry_attr_get_ulong(e[0], "ipaIDRangeSize"); + if ((range->min + ulong_val -1) >= UINT32_MAX) { + ret = EINVAL; + goto done; + } + range->max = (range->min + ulong_val -1); + + ret = 0; + +done: + slapi_free_search_results_internal(pb); + slapi_pblock_destroy(pb); + free(filter); + + return ret; +} + /* TODO: A similar call is used in ipa_cldap_netlogon.c, maybe a candidate for * a common library */ static int get_domain_info(struct ipa_extdom_ctx *ctx, const char *domain_name, @@ -219,8 +285,14 @@ static int get_domain_info(struct ipa_extdom_ctx *ctx, const char *domain_name, "ipaNTFlatName"); /* TODO: read range from LDAP server */ +/* range.min = 200000; range.max = 400000; +*/ + ret = set_domain_range(ctx, domain_info->sid, &range); + if (ret != 0) { + goto done; + } err = sss_idmap_init(NULL, NULL, NULL, &domain_info->idmap_ctx); if (err == IDMAP_SUCCESS) { -- 1.7.10.2 -------------- next part -------------- From 87b0656de9c7faebfdb0eb25cd0d2fd8f236abe4 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 8 Aug 2012 13:45:55 +0200 Subject: [PATCH 2/2] trust CLI: add ID range for new trusted domain --- API.txt | 4 ++- ipalib/plugins/range.py | 1 + ipalib/plugins/trust.py | 63 ++++++++++++++++++++++++++++++++++++++++++++++- 3 Dateien ge?ndert, 66 Zeilen hinzugef?gt(+), 2 Zeilen entfernt(-) diff --git a/API.txt b/API.txt index d32d6393b0fc2ba9f1a9fe23d01d92d552c10302..aef12b7eb6e458d614c84ba20d782ef3154c09f0 100644 --- a/API.txt +++ b/API.txt @@ -3200,7 +3200,7 @@ output: Output('summary', (, ), None) output: Entry('result', , Gettext('A dictionary representing an LDAP entry', domain='ipa', localedir=None)) output: Output('value', , None) command: trust_add -args: 1,10,3 +args: 1,12,3 arg: Str('cn', attribute=True, cli_name='realm', multivalue=False, primary_key=True, required=True) option: Str('setattr*', cli_name='setattr', exclude='webui') option: Str('addattr*', cli_name='addattr', exclude='webui') @@ -3209,6 +3209,8 @@ option: Str('realm_admin?', cli_name='admin') option: Password('realm_passwd?', cli_name='password', confirm=False) option: Str('realm_server?', cli_name='server') option: Password('trust_secret?', cli_name='trust_secret', confirm=False) +option: Int('base_id?', cli_name='base_id') +option: Int('range_size?', autofill=True, cli_name='range_size', default=200000) option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui') option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui') option: Str('version?', exclude='webui') diff --git a/ipalib/plugins/range.py b/ipalib/plugins/range.py index c1d91867996cb0df863ecd82ad4c56f6fbb0cbaf..95b00b39bc64723829a5d17ed22590199d1d516f 100644 --- a/ipalib/plugins/range.py +++ b/ipalib/plugins/range.py @@ -24,6 +24,7 @@ from ipalib import Command from ipalib import errors from ipapython import ipautil from ipalib import util +from ipapython.dn import DN __doc__ = _(""" diff --git a/ipalib/plugins/trust.py b/ipalib/plugins/trust.py index a70293bff4fb0ce087d5e5f157b352aaab8439ff..1064a06783892f27c56f0310046216881db5b42a 100644 --- a/ipalib/plugins/trust.py +++ b/ipalib/plugins/trust.py @@ -24,6 +24,12 @@ from ipalib import Command from ipalib import errors from ipapython import ipautil from ipalib import util +try: + import pysss_murmur + _murmur_installed = True +except Exception, e: + _murmur_installed = False + if api.env.in_server and api.env.context in ['lite', 'server']: try: import ipaserver.dcerpc @@ -142,9 +148,18 @@ class trust_add(LDAPCreate): label=_('Shared secret for the trust'), confirm=False, ), + Int('base_id?', + cli_name='base_id', + label=_('First Posix ID of the range reserved for the trusted domain'), + ), + Int('range_size?', + cli_name='range_size', + label=_('Size of the ID range reserved for the trusted domain'), + default=200000, + autofill=True + ), ) - msg_summary = _('Added Active Directory trust for realm "%(value)s"') def execute(self, *keys, **options): @@ -155,8 +170,54 @@ class trust_add(LDAPCreate): raise errors.ValidationError(name=_('trust type'), error=_('only "ad" is supported')) else: raise errors.RequirementError(name=_('trust type')) + + self.add_range(*keys, **options) + return result + def add_range(self, *keys, **options): + new_obj = api.Command['trust_show'](keys[-1]) + dom_sid = new_obj['result']['ipanttrusteddomainsid'][0]; + + range_name = keys[-1].upper()+'_id_range' + + try: + old_range = api.Command['range_show'](range_name) + except errors.NotFound, e: + old_range = None + + if old_range: + old_dom_sid = old_range['result']['ipanttrusteddomainsid'][0]; + + if old_dom_sid == dom_sid: + return + + raise errors.ValidationError(name=_('range exists'), + error=_('ID range with the same name but different ' \ + 'domain SID already exists. The ID range for ' \ + 'the new trusted domain must be created manually.')) + + if 'base_id' in options: + base_id = options['base_id'] + else: + if not _murmur_installed: + raise errors.ValidationError(name=_('missing base_id'), + error=_('pysss_murmur is not available on the server ' \ + 'and no base_id is given, ' \ + 'ID range must be create manually')) + + base_id = 200000 + (pysss_murmur.murmurhash3(dom_sid, len(dom_sid), 0xdeadbeef) % 10000) * 200000 + + try: + new_range = api.Command['range_add'](range_name, + ipabaseid=base_id, + ipaidrangesize=options['range_size'], + ipabaserid=0, + ipanttrusteddomainsid=dom_sid) + except Exception, e: + raise errors.ValidationError(name=_('ID range exists'), + error = _('ID range already exists, must be added manually')) + def execute_ad(self, *keys, **options): # Join domain using full credentials and with random trustdom # secret (will be generated by the join method) -- 1.7.10.2 From mkosek at redhat.com Tue Aug 14 13:50:37 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 15:50:37 +0200 Subject: [Freeipa-devel] [PATCH] 0003-2 Handle SSSD restart crash more gently. In-Reply-To: <1570075244.1386419.1344949940166.JavaMail.root@redhat.com> References: <1570075244.1386419.1344949940166.JavaMail.root@redhat.com> Message-ID: <502A57AD.1020704@redhat.com> On 08/14/2012 03:12 PM, Tomas Babej wrote: > Issue resolved. > > Tomas > > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Tuesday, August 14, 2012 9:26:03 AM > Subject: Re: [Freeipa-devel] [PATCH] 0003 Handle SSSD restart crash more gently. > > On 08/07/2012 06:14 PM, Tomas Babej wrote: >> Sorry, I forgot to attach the patch. >> >> ----- Original Message ----- >> From: "Tomas Babej" >> To: freeipa-devel at redhat.com >> Sent: Tuesday, August 7, 2012 5:58:32 PM >> Subject: [PATCH] Handle SSSD restart crash more gently. >> >> Hi, >> >> In ipa-client-install, failure of restart of sssd service no longer >> causes the crash of the install process. Adds a warning message to >> the root logger instead. >> >> https://fedorahosted.org/freeipa/ticket/2827 >> >> Tomas >> > > This works fine, I see just one issue. It is better to catch for more specific > exceptions that with the bare "except" clause. Otherwise for example in this > case, if the sssd restart would got stuck, one could not CTRL+C it properly as > the bare except clause would catch the appropriate exception. You may want to > check with PEP8 on that one. > > I would suggest rather catching CalledProcessError only. > > Martin > ACK. Pushed to master. Martin From mkosek at redhat.com Tue Aug 14 13:50:52 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 15:50:52 +0200 Subject: [Freeipa-devel] [PATCH] 0004-3 Corrects help description of selinuxusermap. In-Reply-To: <1588279350.1387671.1344950247681.JavaMail.root@redhat.com> References: <1588279350.1387671.1344950247681.JavaMail.root@redhat.com> Message-ID: <502A57BC.7090705@redhat.com> On 08/14/2012 03:17 PM, Tomas Babej wrote: > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Tuesday, August 14, 2012 8:41:44 AM > Subject: Re: [Freeipa-devel] [PATCH] 0004-2 Corrects help description of selinuxusermap. > > Looking at the changes, I think there is still one line that does not make much > sense: > > - Display the properties of a named HBAC rule: > + Display the properties of a HBAC rule: > ipa selinuxusermap-show test1 > > This should rather really show a HBAC rule used in previous command > (hbacrule-show command) or not mention HBAC rule at all and show test2 rule > added in previous step. > > Martin > > On 08/10/2012 04:44 PM, Tomas Babej wrote: >> Suggestion incorporated. >> >> Tomas >> >> ----- Original Message ----- >> From: "Martin Kosek" >> To: "Tomas Babej" >> Cc: freeipa-devel at redhat.com >> Sent: Friday, August 10, 2012 3:43:06 PM >> Subject: Re: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. >> >> On 08/10/2012 01:09 PM, Tomas Babej wrote: >>> I forgot the patch once again. >>> >>> ----- Original Message ----- >>> From: "Tomas Babej" >>> To: freeipa-devel at redhat.com >>> Sent: Friday, August 10, 2012 12:44:41 PM >>> Subject: [Freeipa-devel] [PATCH] 0004 Corrects help description of selinuxusermap. >>> >>> Hi, >>> >>> I checked the rest of the description as well, seems alright. >>> >>> https://fedorahosted.org/freeipa/ticket/2959 >>> >>> Tomas >>> >> >> That's a good start. I would also change the following descriptions a bit: >> >> Disable a named rule: >> ipa selinuxusermap-disable test1 >> >> Enable a named rule: >> ipa selinuxusermap-enable test1 >> >> ... >> >> Remove a named rule: >> ipa selinuxusermap-del john_unconfined >> >> I don't know why all rule are specified as "named". I think that simple >> >> Disable a rule: >> Enable a rule: >> Remove a rule: >> > > >> would be enough and less confusing. >> >> Martin >> >> > > I chose the second variant. As these are examples for ipa selinuxusermap, > it seemed more appropriate. > > Tomas > ACK. Pushed to master. Martin From mkosek at redhat.com Tue Aug 14 13:50:59 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 15:50:59 +0200 Subject: [Freeipa-devel] [PATCH] 0005-3 Improves exception handling in ipa-replica-prepare. In-Reply-To: <2016063526.1389378.1344950536604.JavaMail.root@redhat.com> References: <2016063526.1389378.1344950536604.JavaMail.root@redhat.com> Message-ID: <502A57C3.3060908@redhat.com> On 08/14/2012 03:22 PM, Tomas Babej wrote: > ----- Original Message ----- > From: "Martin Kosek" > To: "Tomas Babej" > Cc: freeipa-devel at redhat.com > Sent: Tuesday, August 14, 2012 8:55:19 AM > Subject: Re: [Freeipa-devel] [PATCH] 0005-2 Improves exception handling in ipa-replica-prepare. > >> On 08/10/2012 04:32 PM, Tomas Babej wrote: >>> Man page edited. >>> >>> ----- Original Message ----- >>> From: "Martin Kosek" >>> To: "Tomas Babej" >>> Cc: freeipa-devel at redhat.com >>> Sent: Friday, August 10, 2012 3:49:45 PM >>> Subject: Re: [Freeipa-devel] [PATCH] 0005 Improves exception handling in ipa-replica-prepare. >>> >>> On 08/10/2012 03:07 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> A backtrace is no longer displayed when trying to prepare a replica >>>> file with the local LDAP server down. Also adds --debug option and >>>> no longer displays info log messages without it. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2939 >>>> >>>> Tomas >>>> >>> >>> Looks OK. We just also need to update the ipa-replica-prepare man page, when >>> changing its options. >>> >>> Martin >>> >> >> I tested the patch and the ipa-replica-prepare part works fine, except for the >> man page which is not formatted correctly. >> >> Please, always test your changes in man pages, it is easy to do a formatting >> error here. >> >> Martin > > I corrected the hidden glitch in the man page. > > Tomas > ACK. Pushed to master. Martin From mkosek at redhat.com Tue Aug 14 13:56:43 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 15:56:43 +0200 Subject: [Freeipa-devel] [PATCH] 82 Raise Base64DecodeError instead of ConversionError when base64 decoding fails in Bytes parameters In-Reply-To: <5020AC9F.8070604@redhat.com> References: <5020AC9F.8070604@redhat.com> Message-ID: <502A591B.4080907@redhat.com> On 08/07/2012 07:50 AM, Jan Cholasta wrote: > Hi, > > this patch fixes . > > Honza > ACK. Rebased and pushed to master. Martin From pspacek at redhat.com Tue Aug 14 14:00:21 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 14 Aug 2012 16:00:21 +0200 Subject: [Freeipa-devel] [PATCH 0050] Fix memory leak in configuration with multiple LDAP instances Message-ID: <502A59F5.90300@redhat.com> Hello, this patch fixes $SUBJ$. Adam, please double-check correctness of this change. I had two assumptions: - all locking is done inside dns_db_(un)register() functions - LDAP instances are decommissioned before dynamic_driver_destroy() call Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0050-Fix-memory-leak-in-configuration-with-multiple-LDAP-.patch Type: text/x-patch Size: 1375 bytes Desc: not available URL: From mkosek at redhat.com Tue Aug 14 14:13:11 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 16:13:11 +0200 Subject: [Freeipa-devel] [PATCH] 0069 Add internationalization to DCE RPC code In-Reply-To: <20120814085842.GE3783@redhat.com> References: <20120814085842.GE3783@redhat.com> Message-ID: <502A5CF7.9010300@redhat.com> On 08/14/2012 10:58 AM, Alexander Bokovoy wrote: > Hi, > > fairly simple patch to wrap error messages to look up in translation > catalog. > > https://fedorahosted.org/freeipa/ticket/2964 > Looks and works OK. I just don't think its necessary to translate our error logging: - root_logger.error("LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e))) + root_logger.error( + _("LDAP error when connecting to %(host)s: %(error)s") % + dict(host=unicode(result.pdc_name), error=str(e))) Martin From abokovoy at redhat.com Tue Aug 14 14:34:28 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 14 Aug 2012 17:34:28 +0300 Subject: [Freeipa-devel] [PATCH] 0069 Add internationalization to DCE RPC code In-Reply-To: <502A5CF7.9010300@redhat.com> References: <20120814085842.GE3783@redhat.com> <502A5CF7.9010300@redhat.com> Message-ID: <20120814143428.GG3783@redhat.com> On Tue, 14 Aug 2012, Martin Kosek wrote: >On 08/14/2012 10:58 AM, Alexander Bokovoy wrote: >> Hi, >> >> fairly simple patch to wrap error messages to look up in translation >> catalog. >> >> https://fedorahosted.org/freeipa/ticket/2964 >> > >Looks and works OK. I just don't think its necessary to translate our error >logging: > >- root_logger.error("LDAP error when connecting to %s: %s" % >(unicode(result.pdc_name), str(e))) >+ root_logger.error( >+ _("LDAP error when connecting to %(host)s: %(error)s") % >+ dict(host=unicode(result.pdc_name), error=str(e))) > >Martin Yep. I've attached updated patch that still changes this place to use named arguments but drops _(). -- / Alexander Bokovoy -------------- next part -------------- >From a32b5645dde56951ec96873128eb1e65983cb848 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 13 Aug 2012 16:35:19 +0300 Subject: [PATCH] Add internationalization to DCE RPC code https://fedorahosted.org/freeipa/ticket/2964 --- ipaserver/dcerpc.py | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/ipaserver/dcerpc.py b/ipaserver/dcerpc.py index f7abf62e4f1965897817267ead00f73ad524c7f5..b7ccd15d3e9008fddb6dc5419fc05c50ede39d26 100644 --- a/ipaserver/dcerpc.py +++ b/ipaserver/dcerpc.py @@ -50,18 +50,22 @@ The code in this module relies heavily on samba4-python package and Samba4 python bindings. """) -access_denied_error = errors.ACIError(info='CIFS server denied your credentials') +access_denied_error = errors.ACIError(info=_('CIFS server denied your credentials')) dcerpc_error_codes = { - -1073741823: errors.RemoteRetrieveError(reason='communication with CIFS server was unsuccessful'), + -1073741823: + errors.RemoteRetrieveError(reason=_('communication with CIFS server was unsuccessful')), -1073741790: access_denied_error, -1073741715: access_denied_error, -1073741614: access_denied_error, - -1073741603: errors.ValidationError(name='AD domain controller', error='unsupported functional level'), + -1073741603: + errors.ValidationError(name=_('AD domain controller'), error=_('unsupported functional level')), } dcerpc_error_messages = { - "NT_STATUS_OBJECT_NAME_NOT_FOUND": errors.NotFound(reason='Cannot find specified domain or server name'), - "NT_STATUS_INVALID_PARAMETER_MIX": errors.RequirementError(name='At least the domain or IP address should be specified'), + "NT_STATUS_OBJECT_NAME_NOT_FOUND": + errors.NotFound(reason=_('Cannot find specified domain or server name')), + "NT_STATUS_INVALID_PARAMETER_MIX": + errors.RequirementError(name=_('At least the domain or IP address should be specified')), } def assess_dcerpc_exception(num=None,message=None): @@ -73,7 +77,9 @@ def assess_dcerpc_exception(num=None,message=None): return dcerpc_error_codes[num] if message and message in dcerpc_error_messages: return dcerpc_error_messages[message] - return errors.RemoteRetrieveError(reason='CIFS server communication error: code "%s", message "%s" (both may be "None")' % (num, message)) + reason = _('''CIFS server communication error: code "%(num)s", + message "%(message)s" (both may be "None")''') % dict(num=num, message=message) + return errors.RemoteRetrieveError(reason=reason) class ExtendedDNControl(_ldap.controls.RequestControl): def __init__(self): @@ -173,7 +179,7 @@ class TrustDomainInstance(object): def __gen_lsa_connection(self, binding): if self.creds is None: - raise errors.RequirementError(name='CIFS credentials object') + raise errors.RequirementError(name=_('CIFS credentials object')) try: result = lsa.lsarpc(binding, self.parm, self.creds) return result @@ -205,10 +211,12 @@ class TrustDomainInstance(object): attempts = attempts + 1 if self._pipe is None and attempts == len(bindings): - raise errors.ACIError(info='CIFS server %s denied your credentials' % (remote_host)) + raise errors.ACIError( + info=_('CIFS server %(host)s denied your credentials') % dict(host=remote_host)) if self._pipe is None: - raise errors.RemoteRetrieveError(reason='Cannot establish LSA connection to %s. Is CIFS server running?' % (remote_host)) + raise errors.RemoteRetrieveError( + reason=_('Cannot establish LSA connection to %(host)s. Is CIFS server running?') % dict(host=remote_host)) def __gen_lsa_bindings(self, remote_host): """ @@ -255,7 +263,9 @@ class TrustDomainInstance(object): result = res['defaultNamingContext'][0] self.info['dns_hostname'] = res['dnsHostName'][0] except _ldap.LDAPError, e: - root_logger.error("LDAP error when connecting to %s: %s" % (unicode(result.pdc_name), str(e))) + root_logger.error( + "LDAP error when connecting to %(host)s: %(error)s" % + dict(host=unicode(result.pdc_name), error=str(e))) if result: self.info['sid'] = self.parse_naming_context(result) -- 1.7.11.4 From james.hogarth at gmail.com Tue Aug 14 14:38:50 2012 From: james.hogarth at gmail.com (James Hogarth) Date: Tue, 14 Aug 2012 15:38:50 +0100 Subject: [Freeipa-devel] [PATCH] Set TTL during ipa-client-install for DNS records Message-ID: Hi, Please see attached patch to allow the TTL to be specified when an IPA client is configured. The default is to use the 1200 that is currently in place. Regardless of is it is set or not as an argument ipa_dyndns_ttl is set in sssd.conf so that it's transparent for an admin to see what the TTL will be if he enables updates in SSSD. Until SSSD allows for the TTL to be set in sssd.conf (patch sent in and pending review for possible future inclusion) this patch will only affect the initial registration and not any ongoing changes. Comments would be most welcome! Kind regards, James -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Allow-TTL-to-be-configured-dring-ipa-client-install.patch Type: application/octet-stream Size: 5261 bytes Desc: not available URL: From mkosek at redhat.com Tue Aug 14 14:54:09 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 14 Aug 2012 16:54:09 +0200 Subject: [Freeipa-devel] [PATCH] 0069 Add internationalization to DCE RPC code In-Reply-To: <20120814143428.GG3783@redhat.com> References: <20120814085842.GE3783@redhat.com> <502A5CF7.9010300@redhat.com> <20120814143428.GG3783@redhat.com> Message-ID: <502A6691.9070502@redhat.com> On 08/14/2012 04:34 PM, Alexander Bokovoy wrote: > On Tue, 14 Aug 2012, Martin Kosek wrote: >> On 08/14/2012 10:58 AM, Alexander Bokovoy wrote: >>> Hi, >>> >>> fairly simple patch to wrap error messages to look up in translation >>> catalog. >>> >>> https://fedorahosted.org/freeipa/ticket/2964 >>> >> >> Looks and works OK. I just don't think its necessary to translate our error >> logging: >> >> - root_logger.error("LDAP error when connecting to %s: %s" % >> (unicode(result.pdc_name), str(e))) >> + root_logger.error( >> + _("LDAP error when connecting to %(host)s: %(error)s") % >> + dict(host=unicode(result.pdc_name), error=str(e))) >> >> Martin > > Yep. I've attached updated patch that still changes this place to use > named arguments but drops _(). > ACK. Pushed to master. Martin From ssorce at redhat.com Tue Aug 14 18:25:58 2012 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 14 Aug 2012 14:25:58 -0400 (EDT) Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <5028C2E5.6010002@redhat.com> Message-ID: <1253338745.4849605.1344968758236.JavaMail.root@redhat.com> > On 08/12/2012 11:59 AM, Simo Sorce wrote: > >> On 07/27/2012 12:15 PM, Petr Spacek wrote: > >>> Hello, > >>> > >>> this patch implements "Flush zones and RRs cache when handling > >>> persistent > >>> search reconnection" behaviour as requested > >>> in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . > >>> > >>> Petr^2 Spacek > >> > >> Self-NACK :-) > >> > >> This second version has cache flush postponed a bit: Cache is > >> flushed > >> after > >> receiving first result from LDAP. It should prevent unwanted cache > >> flush in > >> case of timeout or similar problems. > >> > >> Simo, are you ok with this approach? > > > > Sounds better. > > Ideally you do not flush until you get all results and no errors, > > but if that's difficult, waiting until the first results come in > > maybe be a decent first step. > > > > Simo. > > > AFAIK there is no SearchResultDone LDAP message, so it is a bit > problematic. See man ldap_result, the entries return with type LDAP_RES_SEARCH_ENTRY, the last message is instead LDAP_RES_SEARCH_RESULT which tells you the searc is complete. This last message is never sent for a persistent search of course (IIRC :-). But in case of a persistent search you should never need to flush as entries are valid until you see a change. > I created ticket for further improvements: > https://fedorahosted.org/bind-dyndb-ldap/ticket/86 > > Ideas from ticket: > We can measure time between ldap_result() calls and say "done" if > interval > between two received results is greater than x seconds... but it is > not very > reliable. > There is problem with high modification rates (10/sec as mentioned on > freeipa-users list), because inter-result interval can be too long > for those > cases. > This "wait interval" can be shortened if result with Entry Change > Notification > is received ... but it can lead to problem with result interleaving > and so on. > > Further investigation is needed. Indeed. Simo. From rcritten at redhat.com Tue Aug 14 18:50:54 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 14 Aug 2012 14:50:54 -0400 Subject: [Freeipa-devel] [PATCH] 1042 fix selfsign subject checking Message-ID: <502A9E0E.8090705@redhat.com> Fix the data type coming out of the pkcs10 module to make the DN module happy. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1042-selfsign.patch Type: text/x-diff Size: 1140 bytes Desc: not available URL: From mkosek at redhat.com Wed Aug 15 06:36:35 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 15 Aug 2012 08:36:35 +0200 Subject: [Freeipa-devel] [PATCH] 1042 fix selfsign subject checking In-Reply-To: <502A9E0E.8090705@redhat.com> References: <502A9E0E.8090705@redhat.com> Message-ID: <502B4373.4010303@redhat.com> On 08/14/2012 08:50 PM, Rob Crittenden wrote: > Fix the data type coming out of the pkcs10 module to make the DN module happy. > > rob > Fixes the Internal Error nicely. ACK. Pushed to master. Martin From pspacek at redhat.com Wed Aug 15 08:06:57 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 10:06:57 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <1253338745.4849605.1344968758236.JavaMail.root@redhat.com> References: <1253338745.4849605.1344968758236.JavaMail.root@redhat.com> Message-ID: <502B58A1.8020100@redhat.com> On 08/14/2012 08:25 PM, Simo Sorce wrote: >> On 08/12/2012 11:59 AM, Simo Sorce wrote: >>>> On 07/27/2012 12:15 PM, Petr Spacek wrote: >>>>> Hello, >>>>> >>>>> this patch implements "Flush zones and RRs cache when handling >>>>> persistent >>>>> search reconnection" behaviour as requested >>>>> in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . >>>>> >>>>> Petr^2 Spacek >>>> >>>> Self-NACK :-) >>>> >>>> This second version has cache flush postponed a bit: Cache is >>>> flushed >>>> after >>>> receiving first result from LDAP. It should prevent unwanted cache >>>> flush in >>>> case of timeout or similar problems. >>>> >>>> Simo, are you ok with this approach? >>> >>> Sounds better. >>> Ideally you do not flush until you get all results and no errors, >>> but if that's difficult, waiting until the first results come in >>> maybe be a decent first step. >>> >>> Simo. >>> >> AFAIK there is no SearchResultDone LDAP message, so it is a bit >> problematic. > > See man ldap_result, the entries return with type LDAP_RES_SEARCH_ENTRY, the last message is instead LDAP_RES_SEARCH_RESULT which tells you the searc is complete. > > This last message is never sent for a persistent search of course (IIRC :-). Right, it is what I tried to say with "there is no SearchResultDone LDAP message". http://tools.ietf.org/html/draft-smith-psearch-ldap-01#page-3 section 4 paragraph b). This patch deals with persistent search only. Non-persistent search scenarios have cache records with limited TTL value. > > But in case of a persistent search you should never need to flush as entries are valid until you see a change. Unfortunately, cache flush is necessary after persistent search reconnection. Records can change in meanwhile... Example: 1. BIND started, cache was populated. 2. Persistent search connection was lost 3. Record was deleted (but Entry Change Notification wasn't delivered) 4. Persistent search connection was re-established - there is no information about deleted record/zone, BIND still sees old data in the cache. For this reason https://fedorahosted.org/bind-dyndb-ldap/ticket/44 was filled. What I missed? Thanks for your time! Petr^2 Spacek > >> I created ticket for further improvements: >> https://fedorahosted.org/bind-dyndb-ldap/ticket/86 >> >> Ideas from ticket: >> We can measure time between ldap_result() calls and say "done" if >> interval >> between two received results is greater than x seconds... but it is >> not very >> reliable. >> There is problem with high modification rates (10/sec as mentioned on >> freeipa-users list), because inter-result interval can be too long >> for those >> cases. >> This "wait interval" can be shortened if result with Entry Change >> Notification >> is received ... but it can lead to problem with result interleaving >> and so on. >> >> Further investigation is needed. > > Indeed. > > Simo. > From ssorce at redhat.com Wed Aug 15 09:18:31 2012 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 15 Aug 2012 05:18:31 -0400 (EDT) Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <502B58A1.8020100@redhat.com> Message-ID: <190240025.5168295.1345022311962.JavaMail.root@redhat.com> ----- Original Message ----- > On 08/14/2012 08:25 PM, Simo Sorce wrote: > > See man ldap_result, the entries return with type > > LDAP_RES_SEARCH_ENTRY, the last message is instead > > LDAP_RES_SEARCH_RESULT which tells you the searc is complete. > > > > This last message is never sent for a persistent search of course > > (IIRC :-). > > Right, it is what I tried to say with "there is no SearchResultDone > LDAP message". I see. > http://tools.ietf.org/html/draft-smith-psearch-ldap-01#page-3 > section 4 paragraph b). > > This patch deals with persistent search only. Non-persistent search > scenarios > have cache records with limited TTL value. > > > > > But in case of a persistent search you should never need to flush > > as entries are valid until you see a change. > Unfortunately, cache flush is necessary after persistent search > reconnection. > Records can change in meanwhile... > > Example: > 1. BIND started, cache was populated. > 2. Persistent search connection was lost > 3. Record was deleted (but Entry Change Notification wasn't > delivered) > 4. Persistent search connection was re-established - there is no > information > about deleted record/zone, BIND still sees old data in the cache. > > For this reason https://fedorahosted.org/bind-dyndb-ldap/ticket/44 > was filled. > > What I missed? You missed nothing, I did. Howeve I have an idea to handle this situation smartly. We can issue 2 searches on 2 separate connections. The first search will be a persistent search, however it will be started with a filter that has an additional element. Before the persistent search we do a rootdse search and find either the higher modifyTimestamp or the higher entryUSN or equivalent, depending on what is available in the server. Worst case we do a ase search of the DNS tree and pick that modifyTimestamp. Then we start the persistent search with (&(entryUSN>%d)) or (&(modifyTimestamp>=%d) (note modifyTimestamp requires >= due to low resolution). On the other connection we start instead a nornmal search. This normal search will terminate with a 'done' result, so you know that once that search is finished you can flush any cache that has not been re-validated. While you get any changes that are occurring live on the persistent search connection. This way we should be able to not loose any update and still know when we got all results and drop unvalidated caches. HTH, Simo. From pspacek at redhat.com Wed Aug 15 11:20:08 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 13:20:08 +0200 Subject: [Freeipa-devel] [PATCH 0051-0052] Log successful reconnection to LDAP server Message-ID: <502B85E8.3070302@redhat.com> Hello, this two patches solves upstream ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/71 "Log successful reconnect" Patch 51: Adds log_info(): logging facility with log level INFO. Patch 52: Logs successful reconnection to LDAP server. LDAP connection error handling was modified: Errors are handled exclusively by handle_connection_error() now. Direct calls to ldap_connect() and ldap_reconnect() should be avoided. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0051-Add-log_info-logging-facility-with-log-level-INFO.patch Type: text/x-patch Size: 820 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0052-Log-successful-reconnection-to-LDAP-server.patch Type: text/x-patch Size: 3948 bytes Desc: not available URL: From pspacek at redhat.com Wed Aug 15 11:23:45 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 13:23:45 +0200 Subject: [Freeipa-devel] [PATCH 0053] Use richer set of return codes for LDAP connection error handling code Message-ID: <502B86C1.1040907@redhat.com> Hello, current code return very generic ISC_R_FAILURE code in nearly all (error) cases. This patch distinguishes between different LDAP errors and returns richer set of return codes from LDAP connection error handling code. It should lead to clearer log messages. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0053-Use-richer-set-of-return-codes-for-LDAP-connection-e.patch Type: text/x-patch Size: 1806 bytes Desc: not available URL: From mkosek at redhat.com Wed Aug 15 11:24:37 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 15 Aug 2012 13:24:37 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1344456327.21349.37.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> Message-ID: <502B86F5.4070008@redhat.com> On 08/08/2012 10:05 PM, Ade Lee wrote: > Hi, > > Dogtag 10 is being released on f18, and has a number of changes that > will affect IPA. In particular, the following changes will affect > current IPA code. > > * The directory layout of the dogtag instance has changed. Instead of > using separate tomcat instances to host different subsystems, the > standard dogtag installation will allow one to install a CA. KRA, OCSP > and TKS within the same instance. There have been corresponding changes > in the directory layout, as well as the default instance name > (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > of pki-cad, pki-krad etc.) > > * The default instance will use only four ports (HTTPS, HTTP, AJP and > tomcat shutdown port) rather than the 6 previously used. The default > ports will be changed to the standard tomcat ports. As these ports are > local to the ipa server machine, this should not cause too much > disruption. > > * There is a new single step installer written in python. > (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > > * Dogtag 10 runs on tomcat7 - with a new corresponding version of > tomcatjss. > > The attached patch integrates all the above changes in IPA installation > and maintenance code. Once the patch is applied, users will be able to: > > 1. run ipa-server-install to completion on f18 with dogtag 10. > 2. install a new replica on f18 on dogtag 10. > 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > 10 - and have that old-style dogtag instance continue to run correctly. > This will require the installation of the latest version of tomcatjss as > well as the installation of tomcat6. The old-style instance will > continue to use tomcat6. > 4. in addition, the new cert renewal code has been patched and should > continue to work. > > What is not yet completed / supported: > > 1. Installation with an external CA is not yet completed in the new > installer. We plan to complete this soon. > > 2. There is some IPA upgrade code that has not yet been touched > (install/tools/ipa-upgradeconfig). > > 3. A script needs to be written to allow admins to convert their > old-style dogtag instances to new style instances, as well as code to > periodically prompt admins to do this. > > 4. Installation of old-style instances using pkicreate/pkisilent on > dogtag 10 will no longer be supported, and will be disabled soon. > > 5. The pki-selinux policy has been updated to reflect these changes, > but is still in flux. In fact, it is our intention to place the dogtag > selinux policy in the base selinux policy for f18. In the meantime, it > may be necessary to run installs in permissive mode. > > The dogtag 10 code will be released shortly into f18. Prior to that > though, we have placed the new dogtag 10 and tomcatjss code in a > developer repo that is located at > http://nkinder.fedorapeople.org/dogtag-devel/ > > Testing can be done on both f18 and f17 - although the target platform - > and the only platform for which official builds will be created is f18. > > Thanks, > Ade > Hi Ade, Thanks for the patch, I started with review and integration tests (currently running on Fedora 17 with Nathan's repo). Installation on single master was smooth, it worked just fine, even with enforced SELinux, without any error - kudos to you and the whole dogtag team. The resulting logs and the structure of your log directory seems improved. I believe that the brand new Python installers will make it easier to debug issues with dogtag installation when they come. When I tried our unit tests or some simple cert operation, it worked fine as well. Now the bad news, or rather few issues and suggestions I found: 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on Fedora 17 when I updated pki-ca, you somewhere miss a Requires. 2) I had installed IPA with dogtag10 on master. However, CA installation on a replica (ipa-ca-install) with dogtag9 failed - is this expectable? 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well and I got the following error: # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg ... Configuring certificate server: Estimated time 3 minutes 30 seconds [1/14]: creating certificate server user [2/14]: configuring certificate server instance Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Unexpected error - see /var/log/ipareplica-ca-install.log for details: IOError: [Errno 2] No such file or directory: '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' Root cause: ... File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 625, in __spawn_instance "/root/cacert.p12") ... 4) What ports does replica need to be open on the master when installing a CA? Currently, ipa-replica-conncheck does a port check and checks only for 7389, i.e. a port of backend LDAP instance for CA. Now to the patch itself. 5) The patch needs a rebase 6) The patch itself looks and works fine (with the issues I wrote above), but there is currently a lot of hard-coded file paths, instance names or ports. I think it would be really great to move most of this to the platform code (ipapython/platform/). I think that ideal solution would allow us to choose if we want to build FreeIPA with dogtag9 or dogtag10 environment in a build time. I am not saying that it is something you would need to do, its just a proposal also for other developers. This step would enable us to: a) Consolidate CA ports, file paths, instance names etc. to one place b) Easily build IPA to non-Fedora platforms where we still need to use dogtag9 Comments welcome. Martin From atkac at redhat.com Wed Aug 15 13:11:03 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 15 Aug 2012 15:11:03 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <50126A67.1090306@redhat.com> References: <50126A67.1090306@redhat.com> Message-ID: <20120815130928.GA3461@redhat.com> On Fri, Jul 27, 2012 at 12:16:07PM +0200, Petr Spacek wrote: > Hello, > > this patch implements "Flush zones and RRs cache when handling > persistent search reconnection" behaviour as requested > in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . > > Petr^2 Spacek > +isc_result_t > +flush_ldap_cache(ldap_cache_t *cache) > +{ > + isc_result_t result; > + > + REQUIRE(cache != NULL); > + > + LOCK(&cache->mutex); > + if (!ldap_cache_enabled(cache)) { > + result = ISC_R_SUCCESS; > + } else { > + dns_rbt_destroy(&cache->rbt); > + CHECK(dns_rbt_create(cache->mctx, cache_node_deleter, NULL, > + &cache->rbt)); In my opinion usage of dns_rbt_deletename(cache->rbt, dns_rootname, ISC_TRUE) is better, isn't it? Otherwise OK. > + } > + > +cleanup: > + if (result != ISC_R_SUCCESS) > + log_error_r("cache flush failed"); > + UNLOCK(&cache->mutex); > + return result; > +} Regards, Adam -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Wed Aug 15 13:11:56 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 15 Aug 2012 15:11:56 +0200 Subject: [Freeipa-devel] [PATCH 0043] Extend API to be compatible with libdns interface >= 90 In-Reply-To: <50128855.8050002@redhat.com> References: <50128855.8050002@redhat.com> Message-ID: <20120815131153.GB3461@redhat.com> On Fri, Jul 27, 2012 at 02:23:49PM +0200, Petr Spacek wrote: > Hello, > > this patch prevents compiler warning on systems with libdns > interface version >= 90. This libdns version comes with BIND 9.0.0. > > Both new methods are not obligatory, see in bind/lib/dns/db.c, > functions dns_db_findext() and dns_db_findnodeext(). > > Petr^2 Spacek Ack > From 9481fc6f6032f236d7e5e48f651906b25fd49b61 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Fri, 27 Jul 2012 14:18:15 +0200 > Subject: [PATCH] Extend API to be compatible with libdns interface >= 90. > > Signed-off-by: Petr Spacek > --- > src/ldap_driver.c | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/src/ldap_driver.c b/src/ldap_driver.c > index cae45d4f6cc1f201c40ca3413d1f626e03a0318e..51d618c5a2395c58b362a047096b1cf1fc40fbfd 100644 > --- a/src/ldap_driver.c > +++ b/src/ldap_driver.c > @@ -1213,8 +1213,12 @@ static dns_dbmethods_t ldapdb_methods = { > #endif /* LIBDNS_VERSION_MAJOR >= 45 */ > #if LIBDNS_VERSION_MAJOR >= 82 > NULL, /* rpz_enabled */ > - NULL /* rpz_findips */ > + NULL, /* rpz_findips */ > #endif /* LIBDNS_VERSION_MAJOR >= 82 */ > +#if LIBDNS_VERSION_MAJOR >= 90 > + NULL, /* findnodeext */ > + NULL /* findext */ > +#endif /* LIBDNS_VERSION_MAJOR >= 90 */ > }; > > static isc_result_t > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Wed Aug 15 13:13:18 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 15 Aug 2012 15:13:18 +0200 Subject: [Freeipa-devel] [PATCH 0044] Fix and comment ispersistent() call in LDAP driver interface In-Reply-To: <5012923D.10605@redhat.com> References: <5012923D.10605@redhat.com> Message-ID: <20120815131316.GC3461@redhat.com> On Fri, Jul 27, 2012 at 03:06:05PM +0200, Petr Spacek wrote: > Hello, > > this patch fixes ispersistent() call in LDAP driver interface. > > We were lucky, because ISC_R_NOTIMPLEMENTED is evaluated as ISC_TRUE > every time, but I want to be sure. > > Petr^2 Spacek Ack > From bfa32f2fa7d880a5c137cf1705202e939f1928e5 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Fri, 27 Jul 2012 14:58:22 +0200 > Subject: [PATCH] Fix and comment ispersistent() call in LDAP driver > interface. > > Signed-off-by: Petr Spacek > --- > src/ldap_driver.c | 15 ++++++++++++++- > 1 file changed, 14 insertions(+), 1 deletion(-) > > diff --git a/src/ldap_driver.c b/src/ldap_driver.c > index 51d618c5a2395c58b362a047096b1cf1fc40fbfd..470b6f315f0f4483eb60703b369891892368548a 100644 > --- a/src/ldap_driver.c > +++ b/src/ldap_driver.c > @@ -309,6 +309,11 @@ free_ldapdb(ldapdb_t *ldapdb) > isc_mem_putanddetach(&ldapdb->common.mctx, ldapdb, sizeof(*ldapdb)); > } > > + > +/** > + * This method should never be called, because LDAP DB is "persistent". > + * See ispersistent() function. > + */ > static isc_result_t > beginload(dns_db_t *db, dns_addrdatasetfunc_t *addp, dns_dbload_t **dbloadp) > { > @@ -323,6 +328,10 @@ beginload(dns_db_t *db, dns_addrdatasetfunc_t *addp, dns_dbload_t **dbloadp) > return ISC_R_SUCCESS; > } > > +/** > + * This method should never be called, because LDAP DB is "persistent". > + * See ispersistent() function. > + */ > static isc_result_t > endload(dns_db_t *db, dns_dbload_t **dbloadp) > { > @@ -1114,12 +1123,16 @@ nodecount(dns_db_t *db) > return ISC_R_NOTIMPLEMENTED; > } > > +/** > + * Return TRUE, because database does not need to be loaded from disk > + * or written to disk. > + */ > static isc_boolean_t > ispersistent(dns_db_t *db) > { > UNUSED(db); > > - return ISC_R_NOTIMPLEMENTED; > + return ISC_TRUE; > } > > static void > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Wed Aug 15 13:31:48 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 15 Aug 2012 15:31:48 +0200 Subject: [Freeipa-devel] [PATCH 0046] Separate RR data parsing from LDAP connections In-Reply-To: <50193ADF.8020609@redhat.com> References: <50193ADF.8020609@redhat.com> Message-ID: <20120815133143.GA11662@redhat.com> On Wed, Aug 01, 2012 at 04:19:11PM +0200, Petr Spacek wrote: > Hello, > > this patch finishes LDAP connection vs. LDAP result separation. > > It is first step necessary for: > https://fedorahosted.org/bind-dyndb-ldap/ticket/68 > Avoid manual connection management outside ldap_query() > > It should prevent deadlocks in future. > > Petr^2 Spacek Thanks for the patch, please check one comment below. Regards, Adam > From 4ba44be9e9bb7ef5abc9e077d6620de496ae7c0d Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Tue, 31 Jul 2012 14:33:53 +0200 > Subject: [PATCH] Separate RR data parsing from LDAP connections. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 76 ++++++++++++++++++++++++++++++------------------------- > 1 file changed, 42 insertions(+), 34 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index cc7003a6cdcd2d27404fec936623ed8a3e8fa7f8..af0ec092d29bce170d5c2dfa8836a728440116a3 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -196,11 +196,6 @@ struct ldap_connection { > LDAPControl *serverctrls[2]; /* psearch/NULL or NULL/NULL */ > int msgid; > > - /* Parsing. */ > - isc_lex_t *lex; > - isc_buffer_t rdata_target; > - unsigned char *rdata_target_mem; > - > /* For reconnection logic. */ > isc_time_t next_reconnect; > unsigned int tries; > @@ -214,6 +209,11 @@ struct ldap_qresult { > ld_string_t *query_string; > LDAPMessage *result; > ldap_entrylist_t ldap_entries; > + > + /* Parsing. */ > + isc_lex_t *lex; > + isc_buffer_t rdata_target; > + unsigned char *rdata_target_mem; > }; > > /* > @@ -256,15 +256,15 @@ static void destroy_ldap_connection(ldap_pool_t *pool, > static isc_result_t findrdatatype_or_create(isc_mem_t *mctx, > ldapdb_rdatalist_t *rdatalist, dns_rdataclass_t rdclass, > dns_rdatatype_t rdtype, dns_ttl_t ttl, dns_rdatalist_t **rdlistp); > -static isc_result_t add_soa_record(isc_mem_t *mctx, ldap_connection_t *ldap_conn, > +static isc_result_t add_soa_record(isc_mem_t *mctx, ldap_qresult_t *qresult, > dns_name_t *origin, ldap_entry_t *entry, > ldapdb_rdatalist_t *rdatalist, const ld_string_t *fake_mname); > -static isc_result_t parse_rdata(isc_mem_t *mctx, ldap_connection_t *ldap_conn, > +static isc_result_t parse_rdata(isc_mem_t *mctx, ldap_qresult_t *qresult, > dns_rdataclass_t rdclass, dns_rdatatype_t rdtype, > dns_name_t *origin, const char *rdata_text, > dns_rdata_t **rdatap); > static isc_result_t ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, > - ldap_connection_t *conn, dns_name_t *origin, > + ldap_qresult_t *qresult, dns_name_t *origin, > const ld_string_t *fake_mname, ld_string_t *buf, > ldapdb_rdatalist_t *rdatalist); > static inline isc_result_t ldap_get_zone_serial(ldap_instance_t *inst, > @@ -637,8 +637,6 @@ new_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) > > isc_mem_attach(pool->mctx, &ldap_conn->mctx); > > - CHECK(isc_lex_create(ldap_conn->mctx, TOKENSIZ, &ldap_conn->lex)); > - CHECKED_MEM_GET(ldap_conn->mctx, ldap_conn->rdata_target_mem, MINTSIZ); > CHECK(ldap_pscontrol_create(ldap_conn->mctx, > &ldap_conn->serverctrls[0])); > > @@ -667,12 +665,6 @@ destroy_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) > if (ldap_conn->handle != NULL) > ldap_unbind_ext_s(ldap_conn->handle, NULL, NULL); > > - if (ldap_conn->lex != NULL) > - isc_lex_destroy(&ldap_conn->lex); > - if (ldap_conn->rdata_target_mem != NULL) { > - isc_mem_put(ldap_conn->mctx, > - ldap_conn->rdata_target_mem, MINTSIZ); > - } > if (ldap_conn->serverctrls[0] != NULL) { > ldap_pscontrol_destroy(ldap_conn->mctx, > &ldap_conn->serverctrls[0]); > @@ -1431,7 +1423,7 @@ free_rdatalist(isc_mem_t *mctx, dns_rdatalist_t *rdlist) > > static isc_result_t > ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, > - ldap_connection_t *conn, dns_name_t *origin, > + ldap_qresult_t *qresult, dns_name_t *origin, > const ld_string_t *fake_mname, ld_string_t *buf, > ldapdb_rdatalist_t *rdatalist) > { > @@ -1443,7 +1435,7 @@ ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, > dns_rdatalist_t *rdlist = NULL; > ldap_attribute_t *attr; > > - result = add_soa_record(mctx, conn, origin, entry, > + result = add_soa_record(mctx, qresult, origin, entry, > rdatalist, fake_mname); > if (result != ISC_R_SUCCESS && result != ISC_R_NOTFOUND) > goto cleanup; > @@ -1458,7 +1450,7 @@ ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, > CHECK(findrdatatype_or_create(mctx, rdatalist, rdclass, > rdtype, ttl, &rdlist)); > while (ldap_attr_nextvalue(attr, buf) != NULL) { > - CHECK(parse_rdata(mctx, conn, rdclass, > + CHECK(parse_rdata(mctx, qresult, rdclass, > rdtype, origin, > str_buf(buf), &rdata)); > APPEND(rdlist->rdata, rdata, link); > @@ -1518,7 +1510,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > result = ldapdbnode_create(mctx, &node_name, &node); > dns_name_free(&node_name, mctx); > if (result == ISC_R_SUCCESS) { > - result = ldap_parse_rrentry(mctx, entry, ldap_conn, > + result = ldap_parse_rrentry(mctx, entry, ldap_qresult, > origin, ldap_inst->fake_mname, > string, &node->rdatalist); > } > @@ -1584,7 +1576,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > for (entry = HEAD(ldap_qresult->ldap_entries); > entry != NULL; > entry = NEXT(entry, link)) { > - if (ldap_parse_rrentry(mctx, entry, ldap_conn, > + if (ldap_parse_rrentry(mctx, entry, ldap_qresult, > origin, ldap_inst->fake_mname, > string, rdatalist) != ISC_R_SUCCESS ) { > log_error("Failed to parse RR entry (%s)", str_buf(string)); > @@ -1610,7 +1602,7 @@ cleanup: > } > > static isc_result_t > -add_soa_record(isc_mem_t *mctx, ldap_connection_t *ldap_conn, dns_name_t *origin, > +add_soa_record(isc_mem_t *mctx, ldap_qresult_t *qresult, dns_name_t *origin, > ldap_entry_t *entry, ldapdb_rdatalist_t *rdatalist, > const ld_string_t *fake_mname) > { > @@ -1624,7 +1616,7 @@ add_soa_record(isc_mem_t *mctx, ldap_connection_t *ldap_conn, dns_name_t *origin > > CHECK(ldap_entry_getfakesoa(entry, fake_mname, string)); > rdclass = ldap_entry_getrdclass(entry); > - CHECK(parse_rdata(mctx, ldap_conn, rdclass, dns_rdatatype_soa, origin, > + CHECK(parse_rdata(mctx, qresult, rdclass, dns_rdatatype_soa, origin, > str_buf(string), &rdata)); > > CHECK(findrdatatype_or_create(mctx, rdatalist, rdclass, dns_rdatatype_soa, > @@ -1641,17 +1633,17 @@ cleanup: > } > > static isc_result_t > -parse_rdata(isc_mem_t *mctx, ldap_connection_t *ldap_conn, > +parse_rdata(isc_mem_t *mctx, ldap_qresult_t *qresult, > dns_rdataclass_t rdclass, dns_rdatatype_t rdtype, > dns_name_t *origin, const char *rdata_text, dns_rdata_t **rdatap) > { > isc_result_t result; > isc_consttextregion_t text; > isc_buffer_t lex_buffer; > isc_region_t rdatamem; > dns_rdata_t *rdata; > > - REQUIRE(ldap_conn != NULL); > + REQUIRE(qresult != NULL); > REQUIRE(rdata_text != NULL); > REQUIRE(rdatap != NULL); > > @@ -1665,30 +1657,30 @@ parse_rdata(isc_mem_t *mctx, ldap_connection_t *ldap_conn, > isc_buffer_add(&lex_buffer, text.length); > isc_buffer_setactive(&lex_buffer, text.length); > > - CHECK(isc_lex_openbuffer(ldap_conn->lex, &lex_buffer)); > + CHECK(isc_lex_openbuffer(qresult->lex, &lex_buffer)); > > - isc_buffer_init(&ldap_conn->rdata_target, ldap_conn->rdata_target_mem, > + isc_buffer_init(&qresult->rdata_target, qresult->rdata_target_mem, > MINTSIZ); > - CHECK(dns_rdata_fromtext(NULL, rdclass, rdtype, ldap_conn->lex, origin, > - 0, mctx, &ldap_conn->rdata_target, NULL)); > + CHECK(dns_rdata_fromtext(NULL, rdclass, rdtype, qresult->lex, origin, > + 0, mctx, &qresult->rdata_target, NULL)); > > CHECKED_MEM_GET_PTR(mctx, rdata); > dns_rdata_init(rdata); > > - rdatamem.length = isc_buffer_usedlength(&ldap_conn->rdata_target); > + rdatamem.length = isc_buffer_usedlength(&qresult->rdata_target); > CHECKED_MEM_GET(mctx, rdatamem.base, rdatamem.length); > > - memcpy(rdatamem.base, isc_buffer_base(&ldap_conn->rdata_target), > + memcpy(rdatamem.base, isc_buffer_base(&qresult->rdata_target), > rdatamem.length); > dns_rdata_fromregion(rdata, rdclass, rdtype, &rdatamem); > > - isc_lex_close(ldap_conn->lex); > + isc_lex_close(qresult->lex); > > *rdatap = rdata; > return ISC_R_SUCCESS; > > cleanup: > - isc_lex_close(ldap_conn->lex); > + isc_lex_close(qresult->lex); > if (rdata != NULL) > isc_mem_put(mctx, rdata, sizeof(*rdata)); > if (rdatamem.base != NULL) > @@ -1793,14 +1785,25 @@ ldap_query_create(isc_mem_t *mctx, ldap_qresult_t **ldap_qresultp) { > ldap_qresult->mctx = mctx; > ldap_qresult->result = NULL; > ldap_qresult->query_string = NULL; > + ldap_qresult->lex = NULL; I recommend to use ZERO_PTR(ldap_qresult) instead of many "= NULL" assignments. It's safer and used in other *_create functions. > INIT_LIST(ldap_qresult->ldap_entries); > CHECK(str_new(mctx, &ldap_qresult->query_string)); > > + CHECKED_MEM_GET(ldap_qresult->mctx, ldap_qresult->rdata_target_mem, MINTSIZ); > + CHECK(isc_lex_create(ldap_qresult->mctx, TOKENSIZ, &ldap_qresult->lex)); > + > *ldap_qresultp = ldap_qresult; > return ISC_R_SUCCESS; > > cleanup: > - SAFE_MEM_PUT_PTR(mctx, ldap_qresult); > + if (ldap_qresult != NULL) { > + str_destroy(&ldap_qresult->query_string); > + SAFE_MEM_PUT(ldap_qresult->mctx, ldap_qresult->rdata_target_mem, MINTSIZ); > + if (ldap_qresult->lex != NULL) > + isc_lex_destroy(&ldap_qresult->lex); > + SAFE_MEM_PUT_PTR(mctx, ldap_qresult); > + } > + > return result; > } > > @@ -1833,8 +1836,13 @@ ldap_query_free(isc_boolean_t prepare_reuse, ldap_qresult_t **ldap_qresultp) > if (prepare_reuse) { > str_clear(qresult->query_string); > INIT_LIST(qresult->ldap_entries); > + isc_lex_close(qresult->lex); > } else { /* free the whole structure */ > str_destroy(&qresult->query_string); > + if (qresult->lex != NULL) > + isc_lex_destroy(&qresult->lex); > + if (qresult->rdata_target_mem != NULL) > + isc_mem_put(qresult->mctx, qresult->rdata_target_mem, MINTSIZ); > SAFE_MEM_PUT_PTR(qresult->mctx, qresult); > *ldap_qresultp = NULL; > } > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Wed Aug 15 13:41:11 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 15:41:11 +0200 Subject: [Freeipa-devel] [PATCH 0043] Extend API to be compatible with libdns interface >= 90 In-Reply-To: <20120815131153.GB3461@redhat.com> References: <50128855.8050002@redhat.com> <20120815131153.GB3461@redhat.com> Message-ID: <502BA6F7.2000409@redhat.com> On 08/15/2012 03:11 PM, Adam Tkac wrote: > On Fri, Jul 27, 2012 at 02:23:49PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch prevents compiler warning on systems with libdns >> interface version >= 90. This libdns version comes with BIND 9.0.0. >> >> Both new methods are not obligatory, see in bind/lib/dns/db.c, >> functions dns_db_findext() and dns_db_findnodeext(). >> >> Petr^2 Spacek > > Ack > Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/815f075d3dd36fa44c59300361e02e5a61caaa51 -- Petr^2 Spacek From pspacek at redhat.com Wed Aug 15 13:44:31 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 15:44:31 +0200 Subject: [Freeipa-devel] [PATCH 0044] Fix and comment ispersistent() call in LDAP driver interface In-Reply-To: <20120815131316.GC3461@redhat.com> References: <5012923D.10605@redhat.com> <20120815131316.GC3461@redhat.com> Message-ID: <502BA7BF.8000809@redhat.com> On 08/15/2012 03:13 PM, Adam Tkac wrote: > On Fri, Jul 27, 2012 at 03:06:05PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch fixes ispersistent() call in LDAP driver interface. >> >> We were lucky, because ISC_R_NOTIMPLEMENTED is evaluated as ISC_TRUE >> every time, but I want to be sure. >> >> Petr^2 Spacek > > Ack > Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/b2b5fc80b0ae472be40d4c5096aa9adcd8222922 -- Petr^2 Spacek From alee at redhat.com Wed Aug 15 13:54:15 2012 From: alee at redhat.com (Ade Lee) Date: Wed, 15 Aug 2012 09:54:15 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <502B86F5.4070008@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> Message-ID: <1345038855.21349.144.camel@aleeredhat.laptop> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > On 08/08/2012 10:05 PM, Ade Lee wrote: > > Hi, > > > > Dogtag 10 is being released on f18, and has a number of changes that > > will affect IPA. In particular, the following changes will affect > > current IPA code. > > > > * The directory layout of the dogtag instance has changed. Instead of > > using separate tomcat instances to host different subsystems, the > > standard dogtag installation will allow one to install a CA. KRA, OCSP > > and TKS within the same instance. There have been corresponding changes > > in the directory layout, as well as the default instance name > > (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > > of pki-cad, pki-krad etc.) > > > > * The default instance will use only four ports (HTTPS, HTTP, AJP and > > tomcat shutdown port) rather than the 6 previously used. The default > > ports will be changed to the standard tomcat ports. As these ports are > > local to the ipa server machine, this should not cause too much > > disruption. > > > > * There is a new single step installer written in python. > > (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > > > > * Dogtag 10 runs on tomcat7 - with a new corresponding version of > > tomcatjss. > > > > The attached patch integrates all the above changes in IPA installation > > and maintenance code. Once the patch is applied, users will be able to: > > > > 1. run ipa-server-install to completion on f18 with dogtag 10. > > 2. install a new replica on f18 on dogtag 10. > > 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > > 10 - and have that old-style dogtag instance continue to run correctly. > > This will require the installation of the latest version of tomcatjss as > > well as the installation of tomcat6. The old-style instance will > > continue to use tomcat6. > > 4. in addition, the new cert renewal code has been patched and should > > continue to work. > > > > What is not yet completed / supported: > > > > 1. Installation with an external CA is not yet completed in the new > > installer. We plan to complete this soon. > > > > 2. There is some IPA upgrade code that has not yet been touched > > (install/tools/ipa-upgradeconfig). > > > > 3. A script needs to be written to allow admins to convert their > > old-style dogtag instances to new style instances, as well as code to > > periodically prompt admins to do this. > > > > 4. Installation of old-style instances using pkicreate/pkisilent on > > dogtag 10 will no longer be supported, and will be disabled soon. > > > > 5. The pki-selinux policy has been updated to reflect these changes, > > but is still in flux. In fact, it is our intention to place the dogtag > > selinux policy in the base selinux policy for f18. In the meantime, it > > may be necessary to run installs in permissive mode. > > > > The dogtag 10 code will be released shortly into f18. Prior to that > > though, we have placed the new dogtag 10 and tomcatjss code in a > > developer repo that is located at > > http://nkinder.fedorapeople.org/dogtag-devel/ > > > > Testing can be done on both f18 and f17 - although the target platform - > > and the only platform for which official builds will be created is f18. > > > > Thanks, > > Ade > > > > Hi Ade, > > Thanks for the patch, I started with review and integration tests (currently > running on Fedora 17 with Nathan's repo). > > Installation on single master was smooth, it worked just fine, even with > enforced SELinux, without any error - kudos to you and the whole dogtag team. > The resulting logs and the structure of your log directory seems improved. I > believe that the brand new Python installers will make it easier to debug > issues with dogtag installation when they come. When I tried our unit tests or > some simple cert operation, it worked fine as well. > > Now the bad news, or rather few issues and suggestions I found: > > 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > We have a dogtag patch that is currently in review that will address this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, rather than f18+ > 2) I had installed IPA with dogtag10 on master. However, CA installation on a > replica (ipa-ca-install) with dogtag9 failed - is this expectable? > Yes. The current IPA patch is designed to work with dogtag 10 only, which will be officially available on f18+. So if you update to dogtag 10, you must have this patch and visa versa. We probably need to add the relevant requires to IPA to enforce this. If you have an existing dogtag 9 instance, and you upgrade to the new dogtag 10 and patched IPA, then that instance will continue to work. But any new instances would be created using dogtag 10. > 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > and I got the following error: > > # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > ... > Configuring certificate server: Estimated time 3 minutes 30 seconds > [1/14]: creating certificate server user > [2/14]: configuring certificate server instance > > Your system may be partly configured. > Run /usr/sbin/ipa-server-install --uninstall to clean up. > > Unexpected error - see /var/log/ipareplica-ca-install.log for details: > IOError: [Errno 2] No such file or directory: > '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > > Root cause: > ... > File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > 625, in __spawn_instance > "/root/cacert.p12") > ... > I need to look into this. I had fixed ipa-replica-install, rather than ipa-ca-install to create replicas. I didn't know ipa-ca-install existed! Should not be too bad to fix though - most likely just need to move files to the right place. > 4) What ports does replica need to be open on the master when installing a CA? > Currently, ipa-replica-conncheck does a port check and checks only for 7389, > i.e. a port of backend LDAP instance for CA. > Well, right now - the ports required are 80,443 (to get to the CA) and 7389 (for the backend LDAP of the CA). Later, when I submit the patch to merge the CA and IPA databases, the ldap port required will be 389. > > Now to the patch itself. > > 5) The patch needs a rebase > Yup - will rebase in my next submission. > 6) The patch itself looks and works fine (with the issues I wrote above), but > there is currently a lot of hard-coded file paths, instance names or ports. I > think it would be really great to move most of this to the platform code > (ipapython/platform/). I think that ideal solution would allow us to choose if > we want to build FreeIPA with dogtag9 or dogtag10 environment in a build time. > > I am not saying that it is something you would need to do, its just a proposal > also for other developers. This step would enable us to: > a) Consolidate CA ports, file paths, instance names etc. to one place > b) Easily build IPA to non-Fedora platforms where we still need to use dogtag9 > This is a great idea. The hard-coded paths etc. are all following the conventions that were already present. I think you should open a ticket for this - and we should address this in a separate patch. > Comments welcome. > > Martin From pspacek at redhat.com Wed Aug 15 13:55:01 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 15:55:01 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <20120815130928.GA3461@redhat.com> References: <50126A67.1090306@redhat.com> <20120815130928.GA3461@redhat.com> Message-ID: <502BAA35.4040402@redhat.com> On 08/15/2012 03:11 PM, Adam Tkac wrote: > On Fri, Jul 27, 2012 at 12:16:07PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch implements "Flush zones and RRs cache when handling >> persistent search reconnection" behaviour as requested >> in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . >> >> Petr^2 Spacek > >> +isc_result_t >> +flush_ldap_cache(ldap_cache_t *cache) >> +{ >> + isc_result_t result; >> + >> + REQUIRE(cache != NULL); >> + >> + LOCK(&cache->mutex); >> + if (!ldap_cache_enabled(cache)) { >> + result = ISC_R_SUCCESS; >> + } else { >> + dns_rbt_destroy(&cache->rbt); >> + CHECK(dns_rbt_create(cache->mctx, cache_node_deleter, NULL, >> + &cache->rbt)); > > In my opinion usage of dns_rbt_deletename(cache->rbt, dns_rootname, ISC_TRUE) is > better, isn't it? I looked into implementation of both functions. dns_rbt_deletenode() does horribly complicated magic for simple task as "delete whole tree" is. For this reason I called dns_rbt_destroy() - it is much simpler and should be faster (it doesn't try to maintain RBT invariants during whole process). Petr^2 Spacek > > Otherwise OK. > >> + } >> + >> +cleanup: >> + if (result != ISC_R_SUCCESS) >> + log_error_r("cache flush failed"); >> + UNLOCK(&cache->mutex); >> + return result; >> +} > > Regards, Adam > From pspacek at redhat.com Wed Aug 15 14:04:26 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 15 Aug 2012 16:04:26 +0200 Subject: [Freeipa-devel] [PATCH 0046] Separate RR data parsing from LDAP connections In-Reply-To: <20120815133143.GA11662@redhat.com> References: <50193ADF.8020609@redhat.com> <20120815133143.GA11662@redhat.com> Message-ID: <502BAC6A.4060606@redhat.com> On 08/15/2012 03:31 PM, Adam Tkac wrote: > On Wed, Aug 01, 2012 at 04:19:11PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch finishes LDAP connection vs. LDAP result separation. >> >> It is first step necessary for: >> https://fedorahosted.org/bind-dyndb-ldap/ticket/68 >> Avoid manual connection management outside ldap_query() >> >> It should prevent deadlocks in future. >> >> Petr^2 Spacek > > Thanks for the patch, please check one comment below. > > Regards, Adam > >> From 4ba44be9e9bb7ef5abc9e077d6620de496ae7c0d Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Tue, 31 Jul 2012 14:33:53 +0200 >> Subject: [PATCH] Separate RR data parsing from LDAP connections. >> >> Signed-off-by: Petr Spacek >> --- >> src/ldap_helper.c | 76 ++++++++++++++++++++++++++++++------------------------- >> 1 file changed, 42 insertions(+), 34 deletions(-) >> >> diff --git a/src/ldap_helper.c b/src/ldap_helper.c >> index cc7003a6cdcd2d27404fec936623ed8a3e8fa7f8..af0ec092d29bce170d5c2dfa8836a728440116a3 100644 >> --- a/src/ldap_helper.c >> +++ b/src/ldap_helper.c >> @@ -196,11 +196,6 @@ struct ldap_connection { >> LDAPControl *serverctrls[2]; /* psearch/NULL or NULL/NULL */ >> int msgid; >> >> - /* Parsing. */ >> - isc_lex_t *lex; >> - isc_buffer_t rdata_target; >> - unsigned char *rdata_target_mem; >> - >> /* For reconnection logic. */ >> isc_time_t next_reconnect; >> unsigned int tries; >> @@ -214,6 +209,11 @@ struct ldap_qresult { >> ld_string_t *query_string; >> LDAPMessage *result; >> ldap_entrylist_t ldap_entries; >> + >> + /* Parsing. */ >> + isc_lex_t *lex; >> + isc_buffer_t rdata_target; >> + unsigned char *rdata_target_mem; >> }; >> >> /* >> @@ -256,15 +256,15 @@ static void destroy_ldap_connection(ldap_pool_t *pool, >> static isc_result_t findrdatatype_or_create(isc_mem_t *mctx, >> ldapdb_rdatalist_t *rdatalist, dns_rdataclass_t rdclass, >> dns_rdatatype_t rdtype, dns_ttl_t ttl, dns_rdatalist_t **rdlistp); >> -static isc_result_t add_soa_record(isc_mem_t *mctx, ldap_connection_t *ldap_conn, >> +static isc_result_t add_soa_record(isc_mem_t *mctx, ldap_qresult_t *qresult, >> dns_name_t *origin, ldap_entry_t *entry, >> ldapdb_rdatalist_t *rdatalist, const ld_string_t *fake_mname); >> -static isc_result_t parse_rdata(isc_mem_t *mctx, ldap_connection_t *ldap_conn, >> +static isc_result_t parse_rdata(isc_mem_t *mctx, ldap_qresult_t *qresult, >> dns_rdataclass_t rdclass, dns_rdatatype_t rdtype, >> dns_name_t *origin, const char *rdata_text, >> dns_rdata_t **rdatap); >> static isc_result_t ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, >> - ldap_connection_t *conn, dns_name_t *origin, >> + ldap_qresult_t *qresult, dns_name_t *origin, >> const ld_string_t *fake_mname, ld_string_t *buf, >> ldapdb_rdatalist_t *rdatalist); >> static inline isc_result_t ldap_get_zone_serial(ldap_instance_t *inst, >> @@ -637,8 +637,6 @@ new_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) >> >> isc_mem_attach(pool->mctx, &ldap_conn->mctx); >> >> - CHECK(isc_lex_create(ldap_conn->mctx, TOKENSIZ, &ldap_conn->lex)); >> - CHECKED_MEM_GET(ldap_conn->mctx, ldap_conn->rdata_target_mem, MINTSIZ); >> CHECK(ldap_pscontrol_create(ldap_conn->mctx, >> &ldap_conn->serverctrls[0])); >> >> @@ -667,12 +665,6 @@ destroy_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) >> if (ldap_conn->handle != NULL) >> ldap_unbind_ext_s(ldap_conn->handle, NULL, NULL); >> >> - if (ldap_conn->lex != NULL) >> - isc_lex_destroy(&ldap_conn->lex); >> - if (ldap_conn->rdata_target_mem != NULL) { >> - isc_mem_put(ldap_conn->mctx, >> - ldap_conn->rdata_target_mem, MINTSIZ); >> - } >> if (ldap_conn->serverctrls[0] != NULL) { >> ldap_pscontrol_destroy(ldap_conn->mctx, >> &ldap_conn->serverctrls[0]); >> @@ -1431,7 +1423,7 @@ free_rdatalist(isc_mem_t *mctx, dns_rdatalist_t *rdlist) >> >> static isc_result_t >> ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, >> - ldap_connection_t *conn, dns_name_t *origin, >> + ldap_qresult_t *qresult, dns_name_t *origin, >> const ld_string_t *fake_mname, ld_string_t *buf, >> ldapdb_rdatalist_t *rdatalist) >> { >> @@ -1443,7 +1435,7 @@ ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, >> dns_rdatalist_t *rdlist = NULL; >> ldap_attribute_t *attr; >> >> - result = add_soa_record(mctx, conn, origin, entry, >> + result = add_soa_record(mctx, qresult, origin, entry, >> rdatalist, fake_mname); >> if (result != ISC_R_SUCCESS && result != ISC_R_NOTFOUND) >> goto cleanup; >> @@ -1458,7 +1450,7 @@ ldap_parse_rrentry(isc_mem_t *mctx, ldap_entry_t *entry, >> CHECK(findrdatatype_or_create(mctx, rdatalist, rdclass, >> rdtype, ttl, &rdlist)); >> while (ldap_attr_nextvalue(attr, buf) != NULL) { >> - CHECK(parse_rdata(mctx, conn, rdclass, >> + CHECK(parse_rdata(mctx, qresult, rdclass, >> rdtype, origin, >> str_buf(buf), &rdata)); >> APPEND(rdlist->rdata, rdata, link); >> @@ -1518,7 +1510,7 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam >> result = ldapdbnode_create(mctx, &node_name, &node); >> dns_name_free(&node_name, mctx); >> if (result == ISC_R_SUCCESS) { >> - result = ldap_parse_rrentry(mctx, entry, ldap_conn, >> + result = ldap_parse_rrentry(mctx, entry, ldap_qresult, >> origin, ldap_inst->fake_mname, >> string, &node->rdatalist); >> } >> @@ -1584,7 +1576,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na >> for (entry = HEAD(ldap_qresult->ldap_entries); >> entry != NULL; >> entry = NEXT(entry, link)) { >> - if (ldap_parse_rrentry(mctx, entry, ldap_conn, >> + if (ldap_parse_rrentry(mctx, entry, ldap_qresult, >> origin, ldap_inst->fake_mname, >> string, rdatalist) != ISC_R_SUCCESS ) { >> log_error("Failed to parse RR entry (%s)", str_buf(string)); >> @@ -1610,7 +1602,7 @@ cleanup: >> } >> >> static isc_result_t >> -add_soa_record(isc_mem_t *mctx, ldap_connection_t *ldap_conn, dns_name_t *origin, >> +add_soa_record(isc_mem_t *mctx, ldap_qresult_t *qresult, dns_name_t *origin, >> ldap_entry_t *entry, ldapdb_rdatalist_t *rdatalist, >> const ld_string_t *fake_mname) >> { >> @@ -1624,7 +1616,7 @@ add_soa_record(isc_mem_t *mctx, ldap_connection_t *ldap_conn, dns_name_t *origin >> >> CHECK(ldap_entry_getfakesoa(entry, fake_mname, string)); >> rdclass = ldap_entry_getrdclass(entry); >> - CHECK(parse_rdata(mctx, ldap_conn, rdclass, dns_rdatatype_soa, origin, >> + CHECK(parse_rdata(mctx, qresult, rdclass, dns_rdatatype_soa, origin, >> str_buf(string), &rdata)); >> >> CHECK(findrdatatype_or_create(mctx, rdatalist, rdclass, dns_rdatatype_soa, >> @@ -1641,17 +1633,17 @@ cleanup: >> } >> >> static isc_result_t >> -parse_rdata(isc_mem_t *mctx, ldap_connection_t *ldap_conn, >> +parse_rdata(isc_mem_t *mctx, ldap_qresult_t *qresult, >> dns_rdataclass_t rdclass, dns_rdatatype_t rdtype, >> dns_name_t *origin, const char *rdata_text, dns_rdata_t **rdatap) >> { >> isc_result_t result; >> isc_consttextregion_t text; >> isc_buffer_t lex_buffer; >> isc_region_t rdatamem; >> dns_rdata_t *rdata; >> >> - REQUIRE(ldap_conn != NULL); >> + REQUIRE(qresult != NULL); >> REQUIRE(rdata_text != NULL); >> REQUIRE(rdatap != NULL); >> >> @@ -1665,30 +1657,30 @@ parse_rdata(isc_mem_t *mctx, ldap_connection_t *ldap_conn, >> isc_buffer_add(&lex_buffer, text.length); >> isc_buffer_setactive(&lex_buffer, text.length); >> >> - CHECK(isc_lex_openbuffer(ldap_conn->lex, &lex_buffer)); >> + CHECK(isc_lex_openbuffer(qresult->lex, &lex_buffer)); >> >> - isc_buffer_init(&ldap_conn->rdata_target, ldap_conn->rdata_target_mem, >> + isc_buffer_init(&qresult->rdata_target, qresult->rdata_target_mem, >> MINTSIZ); >> - CHECK(dns_rdata_fromtext(NULL, rdclass, rdtype, ldap_conn->lex, origin, >> - 0, mctx, &ldap_conn->rdata_target, NULL)); >> + CHECK(dns_rdata_fromtext(NULL, rdclass, rdtype, qresult->lex, origin, >> + 0, mctx, &qresult->rdata_target, NULL)); >> >> CHECKED_MEM_GET_PTR(mctx, rdata); >> dns_rdata_init(rdata); >> >> - rdatamem.length = isc_buffer_usedlength(&ldap_conn->rdata_target); >> + rdatamem.length = isc_buffer_usedlength(&qresult->rdata_target); >> CHECKED_MEM_GET(mctx, rdatamem.base, rdatamem.length); >> >> - memcpy(rdatamem.base, isc_buffer_base(&ldap_conn->rdata_target), >> + memcpy(rdatamem.base, isc_buffer_base(&qresult->rdata_target), >> rdatamem.length); >> dns_rdata_fromregion(rdata, rdclass, rdtype, &rdatamem); >> >> - isc_lex_close(ldap_conn->lex); >> + isc_lex_close(qresult->lex); >> >> *rdatap = rdata; >> return ISC_R_SUCCESS; >> >> cleanup: >> - isc_lex_close(ldap_conn->lex); >> + isc_lex_close(qresult->lex); >> if (rdata != NULL) >> isc_mem_put(mctx, rdata, sizeof(*rdata)); >> if (rdatamem.base != NULL) >> @@ -1793,14 +1785,25 @@ ldap_query_create(isc_mem_t *mctx, ldap_qresult_t **ldap_qresultp) { >> ldap_qresult->mctx = mctx; >> ldap_qresult->result = NULL; >> ldap_qresult->query_string = NULL; >> + ldap_qresult->lex = NULL; > > I recommend to use ZERO_PTR(ldap_qresult) instead of many "= NULL" assignments. > It's safer and used in other *_create functions. Amended patch is attached. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0046-2-Separate-RR-data-parsing-from-LDAP-connections.patch Type: text/x-patch Size: 9569 bytes Desc: not available URL: From mkosek at redhat.com Wed Aug 15 14:34:10 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 15 Aug 2012 16:34:10 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345038855.21349.144.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> Message-ID: <502BB362.4090300@redhat.com> On 08/15/2012 03:54 PM, Ade Lee wrote: > On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >> On 08/08/2012 10:05 PM, Ade Lee wrote: >>> Hi, >>> >>> Dogtag 10 is being released on f18, and has a number of changes that >>> will affect IPA. In particular, the following changes will affect >>> current IPA code. >>> >>> * The directory layout of the dogtag instance has changed. Instead of >>> using separate tomcat instances to host different subsystems, the >>> standard dogtag installation will allow one to install a CA. KRA, OCSP >>> and TKS within the same instance. There have been corresponding changes >>> in the directory layout, as well as the default instance name >>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead >>> of pki-cad, pki-krad etc.) >>> >>> * The default instance will use only four ports (HTTPS, HTTP, AJP and >>> tomcat shutdown port) rather than the 6 previously used. The default >>> ports will be changed to the standard tomcat ports. As these ports are >>> local to the ipa server machine, this should not cause too much >>> disruption. >>> >>> * There is a new single step installer written in python. >>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>> >>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of >>> tomcatjss. >>> >>> The attached patch integrates all the above changes in IPA installation >>> and maintenance code. Once the patch is applied, users will be able to: >>> >>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>> 2. install a new replica on f18 on dogtag 10. >>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag >>> 10 - and have that old-style dogtag instance continue to run correctly. >>> This will require the installation of the latest version of tomcatjss as >>> well as the installation of tomcat6. The old-style instance will >>> continue to use tomcat6. >>> 4. in addition, the new cert renewal code has been patched and should >>> continue to work. >>> >>> What is not yet completed / supported: >>> >>> 1. Installation with an external CA is not yet completed in the new >>> installer. We plan to complete this soon. >>> >>> 2. There is some IPA upgrade code that has not yet been touched >>> (install/tools/ipa-upgradeconfig). >>> >>> 3. A script needs to be written to allow admins to convert their >>> old-style dogtag instances to new style instances, as well as code to >>> periodically prompt admins to do this. >>> >>> 4. Installation of old-style instances using pkicreate/pkisilent on >>> dogtag 10 will no longer be supported, and will be disabled soon. >>> >>> 5. The pki-selinux policy has been updated to reflect these changes, >>> but is still in flux. In fact, it is our intention to place the dogtag >>> selinux policy in the base selinux policy for f18. In the meantime, it >>> may be necessary to run installs in permissive mode. >>> >>> The dogtag 10 code will be released shortly into f18. Prior to that >>> though, we have placed the new dogtag 10 and tomcatjss code in a >>> developer repo that is located at >>> http://nkinder.fedorapeople.org/dogtag-devel/ >>> >>> Testing can be done on both f18 and f17 - although the target platform - >>> and the only platform for which official builds will be created is f18. >>> >>> Thanks, >>> Ade >>> >> >> Hi Ade, >> >> Thanks for the patch, I started with review and integration tests (currently >> running on Fedora 17 with Nathan's repo). >> >> Installation on single master was smooth, it worked just fine, even with >> enforced SELinux, without any error - kudos to you and the whole dogtag team. >> The resulting logs and the structure of your log directory seems improved. I >> believe that the brand new Python installers will make it easier to debug >> issues with dogtag installation when they come. When I tried our unit tests or >> some simple cert operation, it worked fine as well. >> >> Now the bad news, or rather few issues and suggestions I found: >> >> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on >> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >> > We have a dogtag patch that is currently in review that will address > this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > rather than f18+ > >> 2) I had installed IPA with dogtag10 on master. However, CA installation on a >> replica (ipa-ca-install) with dogtag9 failed - is this expectable? >> > Yes. The current IPA patch is designed to work with dogtag 10 only, > which will be officially available on f18+. So if you update to dogtag > 10, you must have this patch and visa versa. We probably need to add > the relevant requires to IPA to enforce this. > > If you have an existing dogtag 9 instance, and you upgrade to the new > dogtag 10 and patched IPA, then that instance will continue to work. > But any new instances would be created using dogtag 10. > >> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well >> and I got the following error: >> >> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >> ... >> Configuring certificate server: Estimated time 3 minutes 30 seconds >> [1/14]: creating certificate server user >> [2/14]: configuring certificate server instance >> >> Your system may be partly configured. >> Run /usr/sbin/ipa-server-install --uninstall to clean up. >> >> Unexpected error - see /var/log/ipareplica-ca-install.log for details: >> IOError: [Errno 2] No such file or directory: >> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >> >> Root cause: >> ... >> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line >> 625, in __spawn_instance >> "/root/cacert.p12") >> ... >> > I need to look into this. I had fixed ipa-replica-install, rather than > ipa-ca-install to create replicas. I didn't know ipa-ca-install > existed! Should not be too bad to fix though - most likely just need to > move files to the right place. Ok, thanks! Btw. CA on replica can be either installed during ipa-replica-install time (when --setup-ca option is passed, you probably used that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > >> 4) What ports does replica need to be open on the master when installing a CA? >> Currently, ipa-replica-conncheck does a port check and checks only for 7389, >> i.e. a port of backend LDAP instance for CA. >> > Well, right now - the ports required are 80,443 (to get to the CA) and > 7389 (for the backend LDAP of the CA). Later, when I submit the patch > to merge the CA and IPA databases, the ldap port required will be 389. Then it is OK, these all ports are already covered in ipa-replica-conncheck. > >> >> Now to the patch itself. >> >> 5) The patch needs a rebase >> > Yup - will rebase in my next submission. > >> 6) The patch itself looks and works fine (with the issues I wrote above), but >> there is currently a lot of hard-coded file paths, instance names or ports. I >> think it would be really great to move most of this to the platform code >> (ipapython/platform/). I think that ideal solution would allow us to choose if >> we want to build FreeIPA with dogtag9 or dogtag10 environment in a build time. >> >> I am not saying that it is something you would need to do, its just a proposal >> also for other developers. This step would enable us to: >> a) Consolidate CA ports, file paths, instance names etc. to one place >> b) Easily build IPA to non-Fedora platforms where we still need to use dogtag9 >> > > This is a great idea. The hard-coded paths etc. are all following the > conventions that were already present. I think you should open a ticket > for this - and we should address this in a separate patch. I will let other developers comment on this proposal, if we get some agreement, I will open a ticket. I have also found 2 more issues: 7) pki-deploy package does not require any other pki-* package, this does not look ok. This way I was able to have pki-ca-9.* and pki-deploy-10.* installed at one time. I doubt it would work that way. 8) Did you test upgrade from installed IPA+dogtag9 to patchedIPA+dogtag10? I did that on Fedora 17 and pki-ca did not start after upgrade. Attaching logs my VM after I tried to (re)start pki-ca. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-ca-log.tgz Type: application/x-compressed-tar Size: 7322 bytes Desc: not available URL: From rcritten at redhat.com Wed Aug 15 17:59:37 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 15 Aug 2012 13:59:37 -0400 Subject: [Freeipa-devel] [PATCH] 1043 fix ipa-replica-manage connect Message-ID: <502BE389.90003@redhat.com> A dn needed to be converted to a DN object. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1043-dn.patch Type: text/x-diff Size: 1116 bytes Desc: not available URL: From rcritten at redhat.com Wed Aug 15 20:31:44 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 15 Aug 2012 16:31:44 -0400 Subject: [Freeipa-devel] [PATCH] 1044 DN ldap syntax exception Message-ID: <502C0730.5030506@redhat.com> Raise the proper IPA exception when a value isn't a valid DN. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1044-syntax.patch Type: text/x-diff Size: 966 bytes Desc: not available URL: From rcritten at redhat.com Wed Aug 15 21:23:40 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 15 Aug 2012 17:23:40 -0400 Subject: [Freeipa-devel] [PATCH] 1045 selinuxusermap fixes Message-ID: <502C135C.3020904@redhat.com> Fix setting the user in a rule using setattr. We weren't verifying that it was in the ordered list. I also noticed that no mls was allowed when it shouldn't be. Made that required. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1045-selinux.patch Type: text/x-diff Size: 4270 bytes Desc: not available URL: From jdennis at redhat.com Thu Aug 16 01:53:07 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 15 Aug 2012 21:53:07 -0400 Subject: [Freeipa-devel] [PATCH 77] Ticket #2584 - Installation fails when CN is set in, certificate subject base Message-ID: <502C5283.2040506@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0077-Ticket-2584-Installation-fails-when-CN-is-set-in-cer.patch Type: text/x-patch Size: 5029 bytes Desc: not available URL: From alee at redhat.com Thu Aug 16 03:41:39 2012 From: alee at redhat.com (Ade Lee) Date: Wed, 15 Aug 2012 23:41:39 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <502BB362.4090300@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> Message-ID: <1345088499.21349.163.camel@aleeredhat.laptop> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > On 08/15/2012 03:54 PM, Ade Lee wrote: > > On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > >> On 08/08/2012 10:05 PM, Ade Lee wrote: > >>> Hi, > >>> > >>> Dogtag 10 is being released on f18, and has a number of changes that > >>> will affect IPA. In particular, the following changes will affect > >>> current IPA code. > >>> > >>> * The directory layout of the dogtag instance has changed. Instead of > >>> using separate tomcat instances to host different subsystems, the > >>> standard dogtag installation will allow one to install a CA. KRA, OCSP > >>> and TKS within the same instance. There have been corresponding changes > >>> in the directory layout, as well as the default instance name > >>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > >>> of pki-cad, pki-krad etc.) > >>> > >>> * The default instance will use only four ports (HTTPS, HTTP, AJP and > >>> tomcat shutdown port) rather than the 6 previously used. The default > >>> ports will be changed to the standard tomcat ports. As these ports are > >>> local to the ipa server machine, this should not cause too much > >>> disruption. > >>> > >>> * There is a new single step installer written in python. > >>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > >>> > >>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of > >>> tomcatjss. > >>> > >>> The attached patch integrates all the above changes in IPA installation > >>> and maintenance code. Once the patch is applied, users will be able to: > >>> > >>> 1. run ipa-server-install to completion on f18 with dogtag 10. > >>> 2. install a new replica on f18 on dogtag 10. > >>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > >>> 10 - and have that old-style dogtag instance continue to run correctly. > >>> This will require the installation of the latest version of tomcatjss as > >>> well as the installation of tomcat6. The old-style instance will > >>> continue to use tomcat6. > >>> 4. in addition, the new cert renewal code has been patched and should > >>> continue to work. > >>> > >>> What is not yet completed / supported: > >>> > >>> 1. Installation with an external CA is not yet completed in the new > >>> installer. We plan to complete this soon. > >>> > >>> 2. There is some IPA upgrade code that has not yet been touched > >>> (install/tools/ipa-upgradeconfig). > >>> > >>> 3. A script needs to be written to allow admins to convert their > >>> old-style dogtag instances to new style instances, as well as code to > >>> periodically prompt admins to do this. > >>> > >>> 4. Installation of old-style instances using pkicreate/pkisilent on > >>> dogtag 10 will no longer be supported, and will be disabled soon. > >>> > >>> 5. The pki-selinux policy has been updated to reflect these changes, > >>> but is still in flux. In fact, it is our intention to place the dogtag > >>> selinux policy in the base selinux policy for f18. In the meantime, it > >>> may be necessary to run installs in permissive mode. > >>> > >>> The dogtag 10 code will be released shortly into f18. Prior to that > >>> though, we have placed the new dogtag 10 and tomcatjss code in a > >>> developer repo that is located at > >>> http://nkinder.fedorapeople.org/dogtag-devel/ > >>> > >>> Testing can be done on both f18 and f17 - although the target platform - > >>> and the only platform for which official builds will be created is f18. > >>> > >>> Thanks, > >>> Ade > >>> > >> > >> Hi Ade, > >> > >> Thanks for the patch, I started with review and integration tests (currently > >> running on Fedora 17 with Nathan's repo). > >> > >> Installation on single master was smooth, it worked just fine, even with > >> enforced SELinux, without any error - kudos to you and the whole dogtag team. > >> The resulting logs and the structure of your log directory seems improved. I > >> believe that the brand new Python installers will make it easier to debug > >> issues with dogtag installation when they come. When I tried our unit tests or > >> some simple cert operation, it worked fine as well. > >> > >> Now the bad news, or rather few issues and suggestions I found: > >> > >> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > >> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > >> > > We have a dogtag patch that is currently in review that will address > > this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > > rather than f18+ > > > >> 2) I had installed IPA with dogtag10 on master. However, CA installation on a > >> replica (ipa-ca-install) with dogtag9 failed - is this expectable? > >> > > Yes. The current IPA patch is designed to work with dogtag 10 only, > > which will be officially available on f18+. So if you update to dogtag > > 10, you must have this patch and visa versa. We probably need to add > > the relevant requires to IPA to enforce this. > > > > If you have an existing dogtag 9 instance, and you upgrade to the new > > dogtag 10 and patched IPA, then that instance will continue to work. > > But any new instances would be created using dogtag 10. > > > >> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > >> and I got the following error: > >> > >> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > >> ... > >> Configuring certificate server: Estimated time 3 minutes 30 seconds > >> [1/14]: creating certificate server user > >> [2/14]: configuring certificate server instance > >> > >> Your system may be partly configured. > >> Run /usr/sbin/ipa-server-install --uninstall to clean up. > >> > >> Unexpected error - see /var/log/ipareplica-ca-install.log for details: > >> IOError: [Errno 2] No such file or directory: > >> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > >> > >> Root cause: > >> ... > >> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > >> 625, in __spawn_instance > >> "/root/cacert.p12") > >> ... > >> > > I need to look into this. I had fixed ipa-replica-install, rather than > > ipa-ca-install to create replicas. I didn't know ipa-ca-install > > existed! Should not be too bad to fix though - most likely just need to > > move files to the right place. > > Ok, thanks! Btw. CA on replica can be either installed during > ipa-replica-install time (when --setup-ca option is passed, you probably used > that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > I will be testing this out again. As ipa-ca-install uses the same code as ipa-replica-install, I would have expected it to work. Did you try it with selinux in permissive mode? > > > >> 4) What ports does replica need to be open on the master when installing a CA? > >> Currently, ipa-replica-conncheck does a port check and checks only for 7389, > >> i.e. a port of backend LDAP instance for CA. > >> > > Well, right now - the ports required are 80,443 (to get to the CA) and > > 7389 (for the backend LDAP of the CA). Later, when I submit the patch > > to merge the CA and IPA databases, the ldap port required will be 389. > > > Then it is OK, these all ports are already covered in ipa-replica-conncheck. > > > > >> > >> Now to the patch itself. > >> > >> 5) The patch needs a rebase > >> > > Yup - will rebase in my next submission. > > > >> 6) The patch itself looks and works fine (with the issues I wrote above), but > >> there is currently a lot of hard-coded file paths, instance names or ports. I > >> think it would be really great to move most of this to the platform code > >> (ipapython/platform/). I think that ideal solution would allow us to choose if > >> we want to build FreeIPA with dogtag9 or dogtag10 environment in a build time. > >> > >> I am not saying that it is something you would need to do, its just a proposal > >> also for other developers. This step would enable us to: > >> a) Consolidate CA ports, file paths, instance names etc. to one place > >> b) Easily build IPA to non-Fedora platforms where we still need to use dogtag9 > >> > > > > This is a great idea. The hard-coded paths etc. are all following the > > conventions that were already present. I think you should open a ticket > > for this - and we should address this in a separate patch. > > I will let other developers comment on this proposal, if we get some agreement, > I will open a ticket. > > I have also found 2 more issues: > > 7) pki-deploy package does not require any other pki-* package, this does not > look ok. This way I was able to have pki-ca-9.* and pki-deploy-10.* installed > at one time. I doubt it would work that way. > We have opened a dogtag ticket to address this. Some kind of dependency will be added so that pki-deploy and pki-common-10 are co-dependencies. > 8) Did you test upgrade from installed IPA+dogtag9 to patchedIPA+dogtag10? I > did that on Fedora 17 and pki-ca did not start after upgrade. Attaching logs my > VM after I tried to (re)start pki-ca. > I did test this. From the logs though, this looks very much like it is not starting because of selinux. Can you try to restart with selinux permissive? I fully expect there to be selinux issues here as some changes are required in the base policy where the default ports are defined. > Martin From alee at redhat.com Thu Aug 16 05:53:25 2012 From: alee at redhat.com (Ade Lee) Date: Thu, 16 Aug 2012 01:53:25 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> Message-ID: <1345096405.21349.176.camel@aleeredhat.laptop> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > > On 08/15/2012 03:54 PM, Ade Lee wrote: > > > On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > > >> On 08/08/2012 10:05 PM, Ade Lee wrote: > > >>> Hi, > > >>> > > >>> Dogtag 10 is being released on f18, and has a number of changes that > > >>> will affect IPA. In particular, the following changes will affect > > >>> current IPA code. > > >>> > > >>> * The directory layout of the dogtag instance has changed. Instead of > > >>> using separate tomcat instances to host different subsystems, the > > >>> standard dogtag installation will allow one to install a CA. KRA, OCSP > > >>> and TKS within the same instance. There have been corresponding changes > > >>> in the directory layout, as well as the default instance name > > >>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > > >>> of pki-cad, pki-krad etc.) > > >>> > > >>> * The default instance will use only four ports (HTTPS, HTTP, AJP and > > >>> tomcat shutdown port) rather than the 6 previously used. The default > > >>> ports will be changed to the standard tomcat ports. As these ports are > > >>> local to the ipa server machine, this should not cause too much > > >>> disruption. > > >>> > > >>> * There is a new single step installer written in python. > > >>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > > >>> > > >>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of > > >>> tomcatjss. > > >>> > > >>> The attached patch integrates all the above changes in IPA installation > > >>> and maintenance code. Once the patch is applied, users will be able to: > > >>> > > >>> 1. run ipa-server-install to completion on f18 with dogtag 10. > > >>> 2. install a new replica on f18 on dogtag 10. > > >>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > > >>> 10 - and have that old-style dogtag instance continue to run correctly. > > >>> This will require the installation of the latest version of tomcatjss as > > >>> well as the installation of tomcat6. The old-style instance will > > >>> continue to use tomcat6. > > >>> 4. in addition, the new cert renewal code has been patched and should > > >>> continue to work. > > >>> > > >>> What is not yet completed / supported: > > >>> > > >>> 1. Installation with an external CA is not yet completed in the new > > >>> installer. We plan to complete this soon. > > >>> > > >>> 2. There is some IPA upgrade code that has not yet been touched > > >>> (install/tools/ipa-upgradeconfig). > > >>> > > >>> 3. A script needs to be written to allow admins to convert their > > >>> old-style dogtag instances to new style instances, as well as code to > > >>> periodically prompt admins to do this. > > >>> > > >>> 4. Installation of old-style instances using pkicreate/pkisilent on > > >>> dogtag 10 will no longer be supported, and will be disabled soon. > > >>> > > >>> 5. The pki-selinux policy has been updated to reflect these changes, > > >>> but is still in flux. In fact, it is our intention to place the dogtag > > >>> selinux policy in the base selinux policy for f18. In the meantime, it > > >>> may be necessary to run installs in permissive mode. > > >>> > > >>> The dogtag 10 code will be released shortly into f18. Prior to that > > >>> though, we have placed the new dogtag 10 and tomcatjss code in a > > >>> developer repo that is located at > > >>> http://nkinder.fedorapeople.org/dogtag-devel/ > > >>> > > >>> Testing can be done on both f18 and f17 - although the target platform - > > >>> and the only platform for which official builds will be created is f18. > > >>> > > >>> Thanks, > > >>> Ade > > >>> > > >> > > >> Hi Ade, > > >> > > >> Thanks for the patch, I started with review and integration tests (currently > > >> running on Fedora 17 with Nathan's repo). > > >> > > >> Installation on single master was smooth, it worked just fine, even with > > >> enforced SELinux, without any error - kudos to you and the whole dogtag team. > > >> The resulting logs and the structure of your log directory seems improved. I > > >> believe that the brand new Python installers will make it easier to debug > > >> issues with dogtag installation when they come. When I tried our unit tests or > > >> some simple cert operation, it worked fine as well. > > >> > > >> Now the bad news, or rather few issues and suggestions I found: > > >> > > >> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > > >> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > > >> > > > We have a dogtag patch that is currently in review that will address > > > this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > > > rather than f18+ > > > > > >> 2) I had installed IPA with dogtag10 on master. However, CA installation on a > > >> replica (ipa-ca-install) with dogtag9 failed - is this expectable? > > >> > > > Yes. The current IPA patch is designed to work with dogtag 10 only, > > > which will be officially available on f18+. So if you update to dogtag > > > 10, you must have this patch and visa versa. We probably need to add > > > the relevant requires to IPA to enforce this. > > > > > > If you have an existing dogtag 9 instance, and you upgrade to the new > > > dogtag 10 and patched IPA, then that instance will continue to work. > > > But any new instances would be created using dogtag 10. > > > > > >> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > > >> and I got the following error: > > >> > > >> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > > >> ... > > >> Configuring certificate server: Estimated time 3 minutes 30 seconds > > >> [1/14]: creating certificate server user > > >> [2/14]: configuring certificate server instance > > >> > > >> Your system may be partly configured. > > >> Run /usr/sbin/ipa-server-install --uninstall to clean up. > > >> > > >> Unexpected error - see /var/log/ipareplica-ca-install.log for details: > > >> IOError: [Errno 2] No such file or directory: > > >> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > > >> > > >> Root cause: > > >> ... > > >> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > > >> 625, in __spawn_instance > > >> "/root/cacert.p12") > > >> ... > > >> > > > I need to look into this. I had fixed ipa-replica-install, rather than > > > ipa-ca-install to create replicas. I didn't know ipa-ca-install > > > existed! Should not be too bad to fix though - most likely just need to > > > move files to the right place. > > > > Ok, thanks! Btw. CA on replica can be either installed during > > ipa-replica-install time (when --setup-ca option is passed, you probably used > > that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > > > I will be testing this out again. As ipa-ca-install uses the same code > as ipa-replica-install, I would have expected it to work. Did you try > it with selinux in permissive mode? > OK -- I tested this out and realized that between the time I initially tested this and your tests, we had changed a URL from /ca/pki/installer/installToken to /ca/rest/installer/installToken, but I forgot to update ipa-pki-proxy.conf. The new patch has been rebased and changes the relevant patch. With this, the CA replica installs correctly. There was one issue that I encountered. I have seen this once before and will need to investigate further. IPA sometimes restarts the dogtag instance by doing systemctl stop pki-tomcatd.target. and then systemctl start pki-tomcatd.target. I have noticed that occasionally the start invocation fails, while the stop and restart invocations succeed just fine. This makes absolutely no sense because restart is essentially just stop and then start. I think this is actually a bug in systemd. If I reboot the machine, it all works perfectly. If we can't figure out whats going on with systemd, we can always replace this start/stop with starting/stopping the service directly. (pki-tomcatd at pki-tomcat.service). That seems to work all the time with no problems. I suggest we leave this fix (if needed) for a separate patch. > > > > > >> 4) What ports does replica need to be open on the master when installing a CA? > > >> Currently, ipa-replica-conncheck does a port check and checks only for 7389, > > >> i.e. a port of backend LDAP instance for CA. > > >> > > > Well, right now - the ports required are 80,443 (to get to the CA) and > > > 7389 (for the backend LDAP of the CA). Later, when I submit the patch > > > to merge the CA and IPA databases, the ldap port required will be 389. > > > > > > Then it is OK, these all ports are already covered in ipa-replica-conncheck. > > > > > > > >> > > >> Now to the patch itself. > > >> > > >> 5) The patch needs a rebase > > >> > > > Yup - will rebase in my next submission. > > > > > >> 6) The patch itself looks and works fine (with the issues I wrote above), but > > >> there is currently a lot of hard-coded file paths, instance names or ports. I > > >> think it would be really great to move most of this to the platform code > > >> (ipapython/platform/). I think that ideal solution would allow us to choose if > > >> we want to build FreeIPA with dogtag9 or dogtag10 environment in a build time. > > >> > > >> I am not saying that it is something you would need to do, its just a proposal > > >> also for other developers. This step would enable us to: > > >> a) Consolidate CA ports, file paths, instance names etc. to one place > > >> b) Easily build IPA to non-Fedora platforms where we still need to use dogtag9 > > >> > > > > > > This is a great idea. The hard-coded paths etc. are all following the > > > conventions that were already present. I think you should open a ticket > > > for this - and we should address this in a separate patch. > > > > I will let other developers comment on this proposal, if we get some agreement, > > I will open a ticket. > > > > I have also found 2 more issues: > > > > 7) pki-deploy package does not require any other pki-* package, this does not > > look ok. This way I was able to have pki-ca-9.* and pki-deploy-10.* installed > > at one time. I doubt it would work that way. > > > We have opened a dogtag ticket to address this. Some kind of dependency > will be added so that pki-deploy and pki-common-10 are co-dependencies. > > > 8) Did you test upgrade from installed IPA+dogtag9 to patchedIPA+dogtag10? I > > did that on Fedora 17 and pki-ca did not start after upgrade. Attaching logs my > > VM after I tried to (re)start pki-ca. > > > I did test this. From the logs though, this looks very much like it is > not starting because of selinux. Can you try to restart with selinux > permissive? I fully expect there to be selinux issues here as some > changes are required in the base policy where the default ports are > defined. > > > > Martin > From mkosek at redhat.com Thu Aug 16 06:03:35 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 08:03:35 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345088499.21349.163.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345088499.21349.163.camel@aleeredhat.laptop> Message-ID: <502C8D37.9050200@redhat.com> On 08/16/2012 05:41 AM, Ade Lee wrote: > On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: .. >>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well >>>> and I got the following error: >>>> >>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>> ... >>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>> [1/14]: creating certificate server user >>>> [2/14]: configuring certificate server instance >>>> >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>> >>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: >>>> IOError: [Errno 2] No such file or directory: >>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>> >>>> Root cause: >>>> ... >>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line >>>> 625, in __spawn_instance >>>> "/root/cacert.p12") >>>> ... >>>> >>> I need to look into this. I had fixed ipa-replica-install, rather than >>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>> existed! Should not be too bad to fix though - most likely just need to >>> move files to the right place. >> >> Ok, thanks! Btw. CA on replica can be either installed during >> ipa-replica-install time (when --setup-ca option is passed, you probably used >> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. >> > I will be testing this out again. As ipa-ca-install uses the same code > as ipa-replica-install, I would have expected it to work. Did you try > it with selinux in permissive mode? I had SELinux en enforcing mode. ipa-server-install with SELinux worked fine, so I thought that replica installation will work fine too. I will re-test with SELinux turned off. ... >> 7) pki-deploy package does not require any other pki-* package, this does not >> look ok. This way I was able to have pki-ca-9.* and pki-deploy-10.* installed >> at one time. I doubt it would work that way. >> > We have opened a dogtag ticket to address this. Some kind of dependency > will be added so that pki-deploy and pki-common-10 are co-dependencies. > >> 8) Did you test upgrade from installed IPA+dogtag9 to patchedIPA+dogtag10? I >> did that on Fedora 17 and pki-ca did not start after upgrade. Attaching logs my >> VM after I tried to (re)start pki-ca. >> > I did test this. From the logs though, this looks very much like it is > not starting because of selinux. Can you try to restart with selinux > permissive? I fully expect there to be selinux issues here as some > changes are required in the base policy where the default ports are > defined. > Ok then. I will test it with permissive SELinux as well. Thanks, Martin From mkosek at redhat.com Thu Aug 16 07:12:23 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 09:12:23 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345096405.21349.176.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> Message-ID: <502C9D57.2060105@redhat.com> On 08/16/2012 07:53 AM, Ade Lee wrote: > On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>> Hi, >>>>>> >>>>>> Dogtag 10 is being released on f18, and has a number of changes that >>>>>> will affect IPA. In particular, the following changes will affect >>>>>> current IPA code. >>>>>> >>>>>> * The directory layout of the dogtag instance has changed. Instead of >>>>>> using separate tomcat instances to host different subsystems, the >>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP >>>>>> and TKS within the same instance. There have been corresponding changes >>>>>> in the directory layout, as well as the default instance name >>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead >>>>>> of pki-cad, pki-krad etc.) >>>>>> >>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and >>>>>> tomcat shutdown port) rather than the 6 previously used. The default >>>>>> ports will be changed to the standard tomcat ports. As these ports are >>>>>> local to the ipa server machine, this should not cause too much >>>>>> disruption. >>>>>> >>>>>> * There is a new single step installer written in python. >>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>> >>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of >>>>>> tomcatjss. >>>>>> >>>>>> The attached patch integrates all the above changes in IPA installation >>>>>> and maintenance code. Once the patch is applied, users will be able to: >>>>>> >>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag >>>>>> 10 - and have that old-style dogtag instance continue to run correctly. >>>>>> This will require the installation of the latest version of tomcatjss as >>>>>> well as the installation of tomcat6. The old-style instance will >>>>>> continue to use tomcat6. >>>>>> 4. in addition, the new cert renewal code has been patched and should >>>>>> continue to work. >>>>>> >>>>>> What is not yet completed / supported: >>>>>> >>>>>> 1. Installation with an external CA is not yet completed in the new >>>>>> installer. We plan to complete this soon. >>>>>> >>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>> (install/tools/ipa-upgradeconfig). >>>>>> >>>>>> 3. A script needs to be written to allow admins to convert their >>>>>> old-style dogtag instances to new style instances, as well as code to >>>>>> periodically prompt admins to do this. >>>>>> >>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on >>>>>> dogtag 10 will no longer be supported, and will be disabled soon. >>>>>> >>>>>> 5. The pki-selinux policy has been updated to reflect these changes, >>>>>> but is still in flux. In fact, it is our intention to place the dogtag >>>>>> selinux policy in the base selinux policy for f18. In the meantime, it >>>>>> may be necessary to run installs in permissive mode. >>>>>> >>>>>> The dogtag 10 code will be released shortly into f18. Prior to that >>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a >>>>>> developer repo that is located at >>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>> >>>>>> Testing can be done on both f18 and f17 - although the target platform - >>>>>> and the only platform for which official builds will be created is f18. >>>>>> >>>>>> Thanks, >>>>>> Ade >>>>>> >>>>> >>>>> Hi Ade, >>>>> >>>>> Thanks for the patch, I started with review and integration tests (currently >>>>> running on Fedora 17 with Nathan's repo). >>>>> >>>>> Installation on single master was smooth, it worked just fine, even with >>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. >>>>> The resulting logs and the structure of your log directory seems improved. I >>>>> believe that the brand new Python installers will make it easier to debug >>>>> issues with dogtag installation when they come. When I tried our unit tests or >>>>> some simple cert operation, it worked fine as well. >>>>> >>>>> Now the bad news, or rather few issues and suggestions I found: >>>>> >>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on >>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>> >>>> We have a dogtag patch that is currently in review that will address >>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, >>>> rather than f18+ >>>> >>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a >>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? >>>>> >>>> Yes. The current IPA patch is designed to work with dogtag 10 only, >>>> which will be officially available on f18+. So if you update to dogtag >>>> 10, you must have this patch and visa versa. We probably need to add >>>> the relevant requires to IPA to enforce this. >>>> >>>> If you have an existing dogtag 9 instance, and you upgrade to the new >>>> dogtag 10 and patched IPA, then that instance will continue to work. >>>> But any new instances would be created using dogtag 10. >>>> >>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well >>>>> and I got the following error: >>>>> >>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>> ... >>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>>> [1/14]: creating certificate server user >>>>> [2/14]: configuring certificate server instance >>>>> >>>>> Your system may be partly configured. >>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>> >>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: >>>>> IOError: [Errno 2] No such file or directory: >>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>> >>>>> Root cause: >>>>> ... >>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line >>>>> 625, in __spawn_instance >>>>> "/root/cacert.p12") >>>>> ... >>>>> >>>> I need to look into this. I had fixed ipa-replica-install, rather than >>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>> existed! Should not be too bad to fix though - most likely just need to >>>> move files to the right place. >>> >>> Ok, thanks! Btw. CA on replica can be either installed during >>> ipa-replica-install time (when --setup-ca option is passed, you probably used >>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. >>> >> I will be testing this out again. As ipa-ca-install uses the same code >> as ipa-replica-install, I would have expected it to work. Did you try >> it with selinux in permissive mode? >> > OK -- I tested this out and realized that between the time I initially > tested this and your tests, we had changed a URL > from /ca/pki/installer/installToken to /ca/rest/installer/installToken, > but I forgot to update ipa-pki-proxy.conf. > > The new patch has been rebased and changes the relevant patch. With > this, the CA replica installs correctly. Umh, where can I find the new rebased patch? :-) > > There was one issue that I encountered. I have seen this once before > and will need to investigate further. IPA sometimes restarts the dogtag > instance by doing systemctl stop pki-tomcatd.target. and then systemctl > start pki-tomcatd.target. I have noticed that occasionally the start > invocation fails, while the stop and restart invocations succeed just > fine. This makes absolutely no sense because restart is essentially > just stop and then start. > > I think this is actually a bug in systemd. If I reboot the machine, it > all works perfectly. If we can't figure out whats going on with > systemd, we can always replace this start/stop with starting/stopping > the service directly. (pki-tomcatd at pki-tomcat.service). That seems to > work all the time with no problems. > > I suggest we leave this fix (if needed) for a separate patch. Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to use target units this way? I thought that only service files are meant to be used for manipulation with a service. As you said yourself, using service unit file for restart worked every time. Martin From atkac at redhat.com Thu Aug 16 09:01:21 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 16 Aug 2012 11:01:21 +0200 Subject: [Freeipa-devel] [PATCH 0046] Separate RR data parsing from LDAP connections In-Reply-To: <502BAC6A.4060606@redhat.com> References: <50193ADF.8020609@redhat.com> <20120815133143.GA11662@redhat.com> <502BAC6A.4060606@redhat.com> Message-ID: <20120816090119.GA1576@redhat.com> On Wed, Aug 15, 2012 at 04:04:26PM +0200, Petr Spacek wrote: > On 08/15/2012 03:31 PM, Adam Tkac wrote: > >On Wed, Aug 01, 2012 at 04:19:11PM +0200, Petr Spacek wrote: > >>Hello, > >> > >>this patch finishes LDAP connection vs. LDAP result separation. > >> > >>It is first step necessary for: > >>https://fedorahosted.org/bind-dyndb-ldap/ticket/68 > >>Avoid manual connection management outside ldap_query() > >> > >>It should prevent deadlocks in future. > >> > >>Petr^2 Spacek > > > > ... > >I recommend to use ZERO_PTR(ldap_qresult) instead of many "= NULL" assignments. > >It's safer and used in other *_create functions. > > Amended patch is attached. Ack -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Thu Aug 16 09:08:57 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 16 Aug 2012 11:08:57 +0200 Subject: [Freeipa-devel] [PATCH 0046] Separate RR data parsing from LDAP connections In-Reply-To: <20120816090119.GA1576@redhat.com> References: <50193ADF.8020609@redhat.com> <20120815133143.GA11662@redhat.com> <502BAC6A.4060606@redhat.com> <20120816090119.GA1576@redhat.com> Message-ID: <502CB8A9.7090100@redhat.com> On 08/16/2012 11:01 AM, Adam Tkac wrote: > On Wed, Aug 15, 2012 at 04:04:26PM +0200, Petr Spacek wrote: >> On 08/15/2012 03:31 PM, Adam Tkac wrote: >>> On Wed, Aug 01, 2012 at 04:19:11PM +0200, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch finishes LDAP connection vs. LDAP result separation. >>>> >>>> It is first step necessary for: >>>> https://fedorahosted.org/bind-dyndb-ldap/ticket/68 >>>> Avoid manual connection management outside ldap_query() >>>> >>>> It should prevent deadlocks in future. >>>> >>>> Petr^2 Spacek >>> >>> > > ... > >>> I recommend to use ZERO_PTR(ldap_qresult) instead of many "= NULL" assignments. >>> It's safer and used in other *_create functions. >> >> Amended patch is attached. > > Ack > Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/ce547f03a86fbbcdb2db0629da615f04a35579b8 -- Petr^2 Spacek From mkosek at redhat.com Thu Aug 16 10:55:17 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 12:55:17 +0200 Subject: [Freeipa-devel] [PATCH] 1043 fix ipa-replica-manage connect In-Reply-To: <502BE389.90003@redhat.com> References: <502BE389.90003@redhat.com> Message-ID: <502CD195.9050208@redhat.com> On 08/15/2012 07:59 PM, Rob Crittenden wrote: > A dn needed to be converted to a DN object. > > rob > Good catch, I re-tested all ipa-replica-manage actions and they worked fine. ACK. Pushed to master. Martin From mkosek at redhat.com Thu Aug 16 10:55:46 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 12:55:46 +0200 Subject: [Freeipa-devel] [PATCH] 1044 DN ldap syntax exception In-Reply-To: <502C0730.5030506@redhat.com> References: <502C0730.5030506@redhat.com> Message-ID: <502CD1B2.1000107@redhat.com> On 08/15/2012 10:31 PM, Rob Crittenden wrote: > Raise the proper IPA exception when a value isn't a valid DN. > > rob > Works fine. ACK. Pushed to master. Martin From mkosek at redhat.com Thu Aug 16 10:56:28 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 12:56:28 +0200 Subject: [Freeipa-devel] [PATCH] 1045 selinuxusermap fixes In-Reply-To: <502C135C.3020904@redhat.com> References: <502C135C.3020904@redhat.com> Message-ID: <502CD1DC.2000701@redhat.com> On 08/15/2012 11:23 PM, Rob Crittenden wrote: > Fix setting the user in a rule using setattr. We weren't verifying that it was > in the ordered list. > > I also noticed that no mls was allowed when it shouldn't be. Made that required. > > rob ACK. Pushed to master. Martin From mkosek at redhat.com Thu Aug 16 10:57:18 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 12:57:18 +0200 Subject: [Freeipa-devel] [PATCH 77] Ticket #2584 - Installation fails when CN is set in, certificate subject base In-Reply-To: <502C5283.2040506@redhat.com> References: <502C5283.2040506@redhat.com> Message-ID: <502CD20E.4020608@redhat.com> On 08/16/2012 03:53 AM, John Dennis wrote: > >>From 32cf59ac8963982d2de59562f3f1570e67e92a3e Mon Sep 17 00:00:00 2001 > From: John Dennis > Date: Wed, 15 Aug 2012 21:33:15 -0400 > Subject: [PATCH 77] Ticket #2584 - Installation fails when CN is set in > certificate subject base ACK. Pushed to master. Martin From alee at redhat.com Thu Aug 16 11:28:47 2012 From: alee at redhat.com (Ade Lee) Date: Thu, 16 Aug 2012 07:28:47 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <502C9D57.2060105@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> Message-ID: <1345116527.21349.178.camel@aleeredhat.laptop> Patch attached this time. I should know better than to do this in the middle of the night .. On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > On 08/16/2012 07:53 AM, Ade Lee wrote: > > On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > >> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > >>> On 08/15/2012 03:54 PM, Ade Lee wrote: > >>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > >>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: > >>>>>> Hi, > >>>>>> > >>>>>> Dogtag 10 is being released on f18, and has a number of changes that > >>>>>> will affect IPA. In particular, the following changes will affect > >>>>>> current IPA code. > >>>>>> > >>>>>> * The directory layout of the dogtag instance has changed. Instead of > >>>>>> using separate tomcat instances to host different subsystems, the > >>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP > >>>>>> and TKS within the same instance. There have been corresponding changes > >>>>>> in the directory layout, as well as the default instance name > >>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > >>>>>> of pki-cad, pki-krad etc.) > >>>>>> > >>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and > >>>>>> tomcat shutdown port) rather than the 6 previously used. The default > >>>>>> ports will be changed to the standard tomcat ports. As these ports are > >>>>>> local to the ipa server machine, this should not cause too much > >>>>>> disruption. > >>>>>> > >>>>>> * There is a new single step installer written in python. > >>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > >>>>>> > >>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of > >>>>>> tomcatjss. > >>>>>> > >>>>>> The attached patch integrates all the above changes in IPA installation > >>>>>> and maintenance code. Once the patch is applied, users will be able to: > >>>>>> > >>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. > >>>>>> 2. install a new replica on f18 on dogtag 10. > >>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > >>>>>> 10 - and have that old-style dogtag instance continue to run correctly. > >>>>>> This will require the installation of the latest version of tomcatjss as > >>>>>> well as the installation of tomcat6. The old-style instance will > >>>>>> continue to use tomcat6. > >>>>>> 4. in addition, the new cert renewal code has been patched and should > >>>>>> continue to work. > >>>>>> > >>>>>> What is not yet completed / supported: > >>>>>> > >>>>>> 1. Installation with an external CA is not yet completed in the new > >>>>>> installer. We plan to complete this soon. > >>>>>> > >>>>>> 2. There is some IPA upgrade code that has not yet been touched > >>>>>> (install/tools/ipa-upgradeconfig). > >>>>>> > >>>>>> 3. A script needs to be written to allow admins to convert their > >>>>>> old-style dogtag instances to new style instances, as well as code to > >>>>>> periodically prompt admins to do this. > >>>>>> > >>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on > >>>>>> dogtag 10 will no longer be supported, and will be disabled soon. > >>>>>> > >>>>>> 5. The pki-selinux policy has been updated to reflect these changes, > >>>>>> but is still in flux. In fact, it is our intention to place the dogtag > >>>>>> selinux policy in the base selinux policy for f18. In the meantime, it > >>>>>> may be necessary to run installs in permissive mode. > >>>>>> > >>>>>> The dogtag 10 code will be released shortly into f18. Prior to that > >>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a > >>>>>> developer repo that is located at > >>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ > >>>>>> > >>>>>> Testing can be done on both f18 and f17 - although the target platform - > >>>>>> and the only platform for which official builds will be created is f18. > >>>>>> > >>>>>> Thanks, > >>>>>> Ade > >>>>>> > >>>>> > >>>>> Hi Ade, > >>>>> > >>>>> Thanks for the patch, I started with review and integration tests (currently > >>>>> running on Fedora 17 with Nathan's repo). > >>>>> > >>>>> Installation on single master was smooth, it worked just fine, even with > >>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. > >>>>> The resulting logs and the structure of your log directory seems improved. I > >>>>> believe that the brand new Python installers will make it easier to debug > >>>>> issues with dogtag installation when they come. When I tried our unit tests or > >>>>> some simple cert operation, it worked fine as well. > >>>>> > >>>>> Now the bad news, or rather few issues and suggestions I found: > >>>>> > >>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > >>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > >>>>> > >>>> We have a dogtag patch that is currently in review that will address > >>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > >>>> rather than f18+ > >>>> > >>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a > >>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? > >>>>> > >>>> Yes. The current IPA patch is designed to work with dogtag 10 only, > >>>> which will be officially available on f18+. So if you update to dogtag > >>>> 10, you must have this patch and visa versa. We probably need to add > >>>> the relevant requires to IPA to enforce this. > >>>> > >>>> If you have an existing dogtag 9 instance, and you upgrade to the new > >>>> dogtag 10 and patched IPA, then that instance will continue to work. > >>>> But any new instances would be created using dogtag 10. > >>>> > >>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > >>>>> and I got the following error: > >>>>> > >>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > >>>>> ... > >>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds > >>>>> [1/14]: creating certificate server user > >>>>> [2/14]: configuring certificate server instance > >>>>> > >>>>> Your system may be partly configured. > >>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > >>>>> > >>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: > >>>>> IOError: [Errno 2] No such file or directory: > >>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > >>>>> > >>>>> Root cause: > >>>>> ... > >>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > >>>>> 625, in __spawn_instance > >>>>> "/root/cacert.p12") > >>>>> ... > >>>>> > >>>> I need to look into this. I had fixed ipa-replica-install, rather than > >>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install > >>>> existed! Should not be too bad to fix though - most likely just need to > >>>> move files to the right place. > >>> > >>> Ok, thanks! Btw. CA on replica can be either installed during > >>> ipa-replica-install time (when --setup-ca option is passed, you probably used > >>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > >>> > >> I will be testing this out again. As ipa-ca-install uses the same code > >> as ipa-replica-install, I would have expected it to work. Did you try > >> it with selinux in permissive mode? > >> > > OK -- I tested this out and realized that between the time I initially > > tested this and your tests, we had changed a URL > > from /ca/pki/installer/installToken to /ca/rest/installer/installToken, > > but I forgot to update ipa-pki-proxy.conf. > > > > The new patch has been rebased and changes the relevant patch. With > > this, the CA replica installs correctly. > > Umh, where can I find the new rebased patch? :-) > > > > > There was one issue that I encountered. I have seen this once before > > and will need to investigate further. IPA sometimes restarts the dogtag > > instance by doing systemctl stop pki-tomcatd.target. and then systemctl > > start pki-tomcatd.target. I have noticed that occasionally the start > > invocation fails, while the stop and restart invocations succeed just > > fine. This makes absolutely no sense because restart is essentially > > just stop and then start. > > > > I think this is actually a bug in systemd. If I reboot the machine, it > > all works perfectly. If we can't figure out whats going on with > > systemd, we can always replace this start/stop with starting/stopping > > the service directly. (pki-tomcatd at pki-tomcat.service). That seems to > > work all the time with no problems. > > > > I suggest we leave this fix (if needed) for a separate patch. > > Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to > use target units this way? I thought that only service files are meant to be > used for manipulation with a service. As you said yourself, using service unit > file for restart worked every time. > > Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Modifications-to-install-scripts-for-dogtag-10.patch Type: text/x-patch Size: 43691 bytes Desc: not available URL: From atkac at redhat.com Thu Aug 16 12:51:39 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 16 Aug 2012 14:51:39 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <502BAA35.4040402@redhat.com> References: <50126A67.1090306@redhat.com> <20120815130928.GA3461@redhat.com> <502BAA35.4040402@redhat.com> Message-ID: <20120816125122.GA3692@redhat.com> On Wed, Aug 15, 2012 at 03:55:01PM +0200, Petr Spacek wrote: > On 08/15/2012 03:11 PM, Adam Tkac wrote: > >On Fri, Jul 27, 2012 at 12:16:07PM +0200, Petr Spacek wrote: > >>Hello, > >> > >>this patch implements "Flush zones and RRs cache when handling > >>persistent search reconnection" behaviour as requested > >>in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . > >> > >>Petr^2 Spacek > > > >>+isc_result_t > >>+flush_ldap_cache(ldap_cache_t *cache) > >>+{ > >>+ isc_result_t result; > >>+ > >>+ REQUIRE(cache != NULL); > >>+ > >>+ LOCK(&cache->mutex); > >>+ if (!ldap_cache_enabled(cache)) { > >>+ result = ISC_R_SUCCESS; > >>+ } else { > >>+ dns_rbt_destroy(&cache->rbt); > >>+ CHECK(dns_rbt_create(cache->mctx, cache_node_deleter, NULL, > >>+ &cache->rbt)); > > > >In my opinion usage of dns_rbt_deletename(cache->rbt, dns_rootname, ISC_TRUE) is > >better, isn't it? > > I looked into implementation of both functions. dns_rbt_deletenode() > does horribly complicated magic for simple task as "delete whole > tree" is. > > For this reason I called dns_rbt_destroy() - it is much simpler and > should be faster (it doesn't try to maintain RBT invariants during > whole process). Sounds fine, ack for the patch as is. A > > > >Otherwise OK. > > > >>+ } > >>+ > >>+cleanup: > >>+ if (result != ISC_R_SUCCESS) > >>+ log_error_r("cache flush failed"); > >>+ UNLOCK(&cache->mutex); > >>+ return result; > >>+} > > > >Regards, Adam > > > -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Thu Aug 16 12:57:44 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 16 Aug 2012 14:57:44 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <20120816125122.GA3692@redhat.com> References: <50126A67.1090306@redhat.com> <20120815130928.GA3461@redhat.com> <502BAA35.4040402@redhat.com> <20120816125122.GA3692@redhat.com> Message-ID: <502CEE48.3000002@redhat.com> On 08/16/2012 02:51 PM, Adam Tkac wrote: > On Wed, Aug 15, 2012 at 03:55:01PM +0200, Petr Spacek wrote: >> On 08/15/2012 03:11 PM, Adam Tkac wrote: >>> On Fri, Jul 27, 2012 at 12:16:07PM +0200, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch implements "Flush zones and RRs cache when handling >>>> persistent search reconnection" behaviour as requested >>>> in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . >>>> >>>> Petr^2 Spacek >>> >>>> +isc_result_t >>>> +flush_ldap_cache(ldap_cache_t *cache) >>>> +{ >>>> + isc_result_t result; >>>> + >>>> + REQUIRE(cache != NULL); >>>> + >>>> + LOCK(&cache->mutex); >>>> + if (!ldap_cache_enabled(cache)) { >>>> + result = ISC_R_SUCCESS; >>>> + } else { >>>> + dns_rbt_destroy(&cache->rbt); >>>> + CHECK(dns_rbt_create(cache->mctx, cache_node_deleter, NULL, >>>> + &cache->rbt)); >>> >>> In my opinion usage of dns_rbt_deletename(cache->rbt, dns_rootname, ISC_TRUE) is >>> better, isn't it? >> >> I looked into implementation of both functions. dns_rbt_deletenode() >> does horribly complicated magic for simple task as "delete whole >> tree" is. >> >> For this reason I called dns_rbt_destroy() - it is much simpler and >> should be faster (it doesn't try to maintain RBT invariants during >> whole process). > > Sounds fine, ack for the patch as is. > > A Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/468329216825f1694e0163f12c9f6d7c50dcc075 It closes the ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/44 Further improvement will be handled as ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/86 Petr^2 Spacek > >>> >>> Otherwise OK. >>> >>>> + } >>>> + >>>> +cleanup: >>>> + if (result != ISC_R_SUCCESS) >>>> + log_error_r("cache flush failed"); >>>> + UNLOCK(&cache->mutex); >>>> + return result; >>>> +} >>> >>> Regards, Adam >>> >> > From mkosek at redhat.com Thu Aug 16 16:45:08 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 16 Aug 2012 18:45:08 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345116527.21349.178.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> Message-ID: <1345135508.2452.4.camel@priserak> On 08/16/2012 01:28 PM, Ade Lee wrote: > Patch attached this time. I should know better than to do this in the > middle of the night .. > > On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >> On 08/16/2012 07:53 AM, Ade Lee wrote: >>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Dogtag 10 is being released on f18, and has a number of changes that >>>>>>>> will affect IPA. In particular, the following changes will affect >>>>>>>> current IPA code. >>>>>>>> >>>>>>>> * The directory layout of the dogtag instance has changed. Instead of >>>>>>>> using separate tomcat instances to host different subsystems, the >>>>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP >>>>>>>> and TKS within the same instance. There have been corresponding changes >>>>>>>> in the directory layout, as well as the default instance name >>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead >>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>> >>>>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and >>>>>>>> tomcat shutdown port) rather than the 6 previously used. The default >>>>>>>> ports will be changed to the standard tomcat ports. As these ports are >>>>>>>> local to the ipa server machine, this should not cause too much >>>>>>>> disruption. >>>>>>>> >>>>>>>> * There is a new single step installer written in python. >>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>> >>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of >>>>>>>> tomcatjss. >>>>>>>> >>>>>>>> The attached patch integrates all the above changes in IPA installation >>>>>>>> and maintenance code. Once the patch is applied, users will be able to: >>>>>>>> >>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag >>>>>>>> 10 - and have that old-style dogtag instance continue to run correctly. >>>>>>>> This will require the installation of the latest version of tomcatjss as >>>>>>>> well as the installation of tomcat6. The old-style instance will >>>>>>>> continue to use tomcat6. >>>>>>>> 4. in addition, the new cert renewal code has been patched and should >>>>>>>> continue to work. >>>>>>>> >>>>>>>> What is not yet completed / supported: >>>>>>>> >>>>>>>> 1. Installation with an external CA is not yet completed in the new >>>>>>>> installer. We plan to complete this soon. >>>>>>>> >>>>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>> >>>>>>>> 3. A script needs to be written to allow admins to convert their >>>>>>>> old-style dogtag instances to new style instances, as well as code to >>>>>>>> periodically prompt admins to do this. >>>>>>>> >>>>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on >>>>>>>> dogtag 10 will no longer be supported, and will be disabled soon. >>>>>>>> >>>>>>>> 5. The pki-selinux policy has been updated to reflect these changes, >>>>>>>> but is still in flux. In fact, it is our intention to place the dogtag >>>>>>>> selinux policy in the base selinux policy for f18. In the meantime, it >>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>> >>>>>>>> The dogtag 10 code will be released shortly into f18. Prior to that >>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a >>>>>>>> developer repo that is located at >>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>> >>>>>>>> Testing can be done on both f18 and f17 - although the target platform - >>>>>>>> and the only platform for which official builds will be created is f18. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Ade >>>>>>>> >>>>>>> >>>>>>> Hi Ade, >>>>>>> >>>>>>> Thanks for the patch, I started with review and integration tests (currently >>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>> >>>>>>> Installation on single master was smooth, it worked just fine, even with >>>>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. >>>>>>> The resulting logs and the structure of your log directory seems improved. I >>>>>>> believe that the brand new Python installers will make it easier to debug >>>>>>> issues with dogtag installation when they come. When I tried our unit tests or >>>>>>> some simple cert operation, it worked fine as well. >>>>>>> >>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>> >>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on >>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>> >>>>>> We have a dogtag patch that is currently in review that will address >>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, >>>>>> rather than f18+ >>>>>> >>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a >>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? >>>>>>> >>>>>> Yes. The current IPA patch is designed to work with dogtag 10 only, >>>>>> which will be officially available on f18+. So if you update to dogtag >>>>>> 10, you must have this patch and visa versa. We probably need to add >>>>>> the relevant requires to IPA to enforce this. >>>>>> >>>>>> If you have an existing dogtag 9 instance, and you upgrade to the new >>>>>> dogtag 10 and patched IPA, then that instance will continue to work. >>>>>> But any new instances would be created using dogtag 10. >>>>>> >>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well >>>>>>> and I got the following error: >>>>>>> >>>>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>> ... >>>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>>>>> [1/14]: creating certificate server user >>>>>>> [2/14]: configuring certificate server instance >>>>>>> >>>>>>> Your system may be partly configured. >>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>> >>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: >>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>> >>>>>>> Root cause: >>>>>>> ... >>>>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line >>>>>>> 625, in __spawn_instance >>>>>>> "/root/cacert.p12") >>>>>>> ... >>>>>>> >>>>>> I need to look into this. I had fixed ipa-replica-install, rather than >>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>> existed! Should not be too bad to fix though - most likely just need to >>>>>> move files to the right place. >>>>> >>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>> ipa-replica-install time (when --setup-ca option is passed, you probably used >>>>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. >>>>> >>>> I will be testing this out again. As ipa-ca-install uses the same code >>>> as ipa-replica-install, I would have expected it to work. Did you try >>>> it with selinux in permissive mode? >>>> >>> OK -- I tested this out and realized that between the time I initially >>> tested this and your tests, we had changed a URL >>> from /ca/pki/installer/installToken to /ca/rest/installer/installToken, >>> but I forgot to update ipa-pki-proxy.conf. >>> >>> The new patch has been rebased and changes the relevant patch. With >>> this, the CA replica installs correctly. >> >> Umh, where can I find the new rebased patch? :-) >> >>> >>> There was one issue that I encountered. I have seen this once before >>> and will need to investigate further. IPA sometimes restarts the dogtag >>> instance by doing systemctl stop pki-tomcatd.target. and then systemctl >>> start pki-tomcatd.target. I have noticed that occasionally the start >>> invocation fails, while the stop and restart invocations succeed just >>> fine. This makes absolutely no sense because restart is essentially >>> just stop and then start. >>> >>> I think this is actually a bug in systemd. If I reboot the machine, it >>> all works perfectly. If we can't figure out whats going on with >>> systemd, we can always replace this start/stop with starting/stopping >>> the service directly. (pki-tomcatd at pki-tomcat.service). That seems to >>> work all the time with no problems. >>> >>> I suggest we leave this fix (if needed) for a separate patch. >> >> Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to >> use target units this way? I thought that only service files are meant to be >> used for manipulation with a service. As you said yourself, using service unit >> file for restart worked every time. >> >> Martin > Now, I tested the rebased patch with VMs with permissive SELinux. IPA server install was OK, as always, but replica installation ended up with error. Although it got much further than before: # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg ... [3/3]: restarting directory server done configuring pkids. Configuring certificate server: Estimated time 3 minutes 30 seconds [1/14]: creating certificate server user [2/14]: configuring certificate server instance [3/14]: disabling nonces [4/14]: importing CA chain to RA certificate database [5/14]: fixing RA database permissions [6/14]: setting up signing cert profile [7/14]: set up CRL publishing [8/14]: set certificate subject base [9/14]: enabling Subject Key Identifier [10/14]: configuring certificate server to start on boot [11/14]: configure certmonger for renewals [12/14]: configure clone certificate renewals [13/14]: configure Server-Cert certificate renewal [14/14]: Configure HTTP to proxy connections done configuring pki-tomcatd. Restarting the directory and certificate servers Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. It seems like that CA on a replica did not start and so a check for port 8080 failed. Attached logs (pki-ca-logs.tgz) may provide more information to this issue. Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 (permissive SELinux). Now, the service was upgraded smoothly, CA service could be restarted without failure. However, certificate operations now did not work: # ipactl restart Restarting Directory Service Restarting KDC Service Restarting KPASSWD Service Restarting MEMCACHE Service Restarting HTTP Service Restarting CA Service # ipactl status Directory Service: RUNNING KDC Service: RUNNING KPASSWD Service: RUNNING MEMCACHE Service: RUNNING HTTP Service: RUNNING CA Service: RUNNING # ipa cert-show 1 ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (Internal Server Error) Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). HTH, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-ca-logs.tgz Type: application/x-compressed-tar Size: 16601 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ipa-ca-logs-upgrade.tgz Type: application/x-compressed-tar Size: 92630 bytes Desc: not available URL: From alee at redhat.com Thu Aug 16 18:39:20 2012 From: alee at redhat.com (Ade Lee) Date: Thu, 16 Aug 2012 14:39:20 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345135508.2452.4.camel@priserak> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> Message-ID: <1345142361.21349.182.camel@aleeredhat.laptop> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: > On 08/16/2012 01:28 PM, Ade Lee wrote: > > Patch attached this time. I should know better than to do this in the > > middle of the night .. > > > > On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > >> On 08/16/2012 07:53 AM, Ade Lee wrote: > >>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > >>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > >>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: > >>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > >>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> Dogtag 10 is being released on f18, and has a number of changes that > >>>>>>>> will affect IPA. In particular, the following changes will affect > >>>>>>>> current IPA code. > >>>>>>>> > >>>>>>>> * The directory layout of the dogtag instance has changed. Instead of > >>>>>>>> using separate tomcat instances to host different subsystems, the > >>>>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP > >>>>>>>> and TKS within the same instance. There have been corresponding changes > >>>>>>>> in the directory layout, as well as the default instance name > >>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > >>>>>>>> of pki-cad, pki-krad etc.) > >>>>>>>> > >>>>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and > >>>>>>>> tomcat shutdown port) rather than the 6 previously used. The default > >>>>>>>> ports will be changed to the standard tomcat ports. As these ports are > >>>>>>>> local to the ipa server machine, this should not cause too much > >>>>>>>> disruption. > >>>>>>>> > >>>>>>>> * There is a new single step installer written in python. > >>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > >>>>>>>> > >>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of > >>>>>>>> tomcatjss. > >>>>>>>> > >>>>>>>> The attached patch integrates all the above changes in IPA installation > >>>>>>>> and maintenance code. Once the patch is applied, users will be able to: > >>>>>>>> > >>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. > >>>>>>>> 2. install a new replica on f18 on dogtag 10. > >>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > >>>>>>>> 10 - and have that old-style dogtag instance continue to run correctly. > >>>>>>>> This will require the installation of the latest version of tomcatjss as > >>>>>>>> well as the installation of tomcat6. The old-style instance will > >>>>>>>> continue to use tomcat6. > >>>>>>>> 4. in addition, the new cert renewal code has been patched and should > >>>>>>>> continue to work. > >>>>>>>> > >>>>>>>> What is not yet completed / supported: > >>>>>>>> > >>>>>>>> 1. Installation with an external CA is not yet completed in the new > >>>>>>>> installer. We plan to complete this soon. > >>>>>>>> > >>>>>>>> 2. There is some IPA upgrade code that has not yet been touched > >>>>>>>> (install/tools/ipa-upgradeconfig). > >>>>>>>> > >>>>>>>> 3. A script needs to be written to allow admins to convert their > >>>>>>>> old-style dogtag instances to new style instances, as well as code to > >>>>>>>> periodically prompt admins to do this. > >>>>>>>> > >>>>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on > >>>>>>>> dogtag 10 will no longer be supported, and will be disabled soon. > >>>>>>>> > >>>>>>>> 5. The pki-selinux policy has been updated to reflect these changes, > >>>>>>>> but is still in flux. In fact, it is our intention to place the dogtag > >>>>>>>> selinux policy in the base selinux policy for f18. In the meantime, it > >>>>>>>> may be necessary to run installs in permissive mode. > >>>>>>>> > >>>>>>>> The dogtag 10 code will be released shortly into f18. Prior to that > >>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a > >>>>>>>> developer repo that is located at > >>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ > >>>>>>>> > >>>>>>>> Testing can be done on both f18 and f17 - although the target platform - > >>>>>>>> and the only platform for which official builds will be created is f18. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Ade > >>>>>>>> > >>>>>>> > >>>>>>> Hi Ade, > >>>>>>> > >>>>>>> Thanks for the patch, I started with review and integration tests (currently > >>>>>>> running on Fedora 17 with Nathan's repo). > >>>>>>> > >>>>>>> Installation on single master was smooth, it worked just fine, even with > >>>>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. > >>>>>>> The resulting logs and the structure of your log directory seems improved. I > >>>>>>> believe that the brand new Python installers will make it easier to debug > >>>>>>> issues with dogtag installation when they come. When I tried our unit tests or > >>>>>>> some simple cert operation, it worked fine as well. > >>>>>>> > >>>>>>> Now the bad news, or rather few issues and suggestions I found: > >>>>>>> > >>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > >>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > >>>>>>> > >>>>>> We have a dogtag patch that is currently in review that will address > >>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > >>>>>> rather than f18+ > >>>>>> > >>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a > >>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? > >>>>>>> > >>>>>> Yes. The current IPA patch is designed to work with dogtag 10 only, > >>>>>> which will be officially available on f18+. So if you update to dogtag > >>>>>> 10, you must have this patch and visa versa. We probably need to add > >>>>>> the relevant requires to IPA to enforce this. > >>>>>> > >>>>>> If you have an existing dogtag 9 instance, and you upgrade to the new > >>>>>> dogtag 10 and patched IPA, then that instance will continue to work. > >>>>>> But any new instances would be created using dogtag 10. > >>>>>> > >>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > >>>>>>> and I got the following error: > >>>>>>> > >>>>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > >>>>>>> ... > >>>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds > >>>>>>> [1/14]: creating certificate server user > >>>>>>> [2/14]: configuring certificate server instance > >>>>>>> > >>>>>>> Your system may be partly configured. > >>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > >>>>>>> > >>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: > >>>>>>> IOError: [Errno 2] No such file or directory: > >>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > >>>>>>> > >>>>>>> Root cause: > >>>>>>> ... > >>>>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > >>>>>>> 625, in __spawn_instance > >>>>>>> "/root/cacert.p12") > >>>>>>> ... > >>>>>>> > >>>>>> I need to look into this. I had fixed ipa-replica-install, rather than > >>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install > >>>>>> existed! Should not be too bad to fix though - most likely just need to > >>>>>> move files to the right place. > >>>>> > >>>>> Ok, thanks! Btw. CA on replica can be either installed during > >>>>> ipa-replica-install time (when --setup-ca option is passed, you probably used > >>>>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > >>>>> > >>>> I will be testing this out again. As ipa-ca-install uses the same code > >>>> as ipa-replica-install, I would have expected it to work. Did you try > >>>> it with selinux in permissive mode? > >>>> > >>> OK -- I tested this out and realized that between the time I initially > >>> tested this and your tests, we had changed a URL > >>> from /ca/pki/installer/installToken to /ca/rest/installer/installToken, > >>> but I forgot to update ipa-pki-proxy.conf. > >>> > >>> The new patch has been rebased and changes the relevant patch. With > >>> this, the CA replica installs correctly. > >> > >> Umh, where can I find the new rebased patch? :-) > >> > >>> > >>> There was one issue that I encountered. I have seen this once before > >>> and will need to investigate further. IPA sometimes restarts the dogtag > >>> instance by doing systemctl stop pki-tomcatd.target. and then systemctl > >>> start pki-tomcatd.target. I have noticed that occasionally the start > >>> invocation fails, while the stop and restart invocations succeed just > >>> fine. This makes absolutely no sense because restart is essentially > >>> just stop and then start. > >>> > >>> I think this is actually a bug in systemd. If I reboot the machine, it > >>> all works perfectly. If we can't figure out whats going on with > >>> systemd, we can always replace this start/stop with starting/stopping > >>> the service directly. (pki-tomcatd at pki-tomcat.service). That seems to > >>> work all the time with no problems. > >>> > >>> I suggest we leave this fix (if needed) for a separate patch. > >> > >> Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to > >> use target units this way? I thought that only service files are meant to be > >> used for manipulation with a service. As you said yourself, using service unit > >> file for restart worked every time. > >> > >> Martin > > > > Now, I tested the rebased patch with VMs with permissive SELinux. IPA server > install was OK, as always, but replica installation ended up with error. > Although it got much further than before: > > # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > ... > [3/3]: restarting directory server > done configuring pkids. > Configuring certificate server: Estimated time 3 minutes 30 seconds > [1/14]: creating certificate server user > [2/14]: configuring certificate server instance > [3/14]: disabling nonces > [4/14]: importing CA chain to RA certificate database > [5/14]: fixing RA database permissions > [6/14]: setting up signing cert profile > [7/14]: set up CRL publishing > [8/14]: set certificate subject base > [9/14]: enabling Subject Key Identifier > [10/14]: configuring certificate server to start on boot > [11/14]: configure certmonger for renewals > [12/14]: configure clone certificate renewals > [13/14]: configure Server-Cert certificate renewal > [14/14]: Configure HTTP to proxy connections > done configuring pki-tomcatd. > Restarting the directory and certificate servers > > Your system may be partly configured. > Run /usr/sbin/ipa-server-install --uninstall to clean up. > > It seems like that CA on a replica did not start and so a check for port 8080 > failed. Attached logs (pki-ca-logs.tgz) may provide more information to this issue. This is the issue I mentioned before about systemctl not starting the target. If you try a systemctl restart pki-tomcatd.target, it works. I will change the code to start the service rather than the target to avoid this whole problem. > > Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 > (permissive SELinux). Now, the service was upgraded smoothly, CA service > could be restarted without failure. However, certificate operations now > did not work: > > # ipactl restart > Restarting Directory Service > Restarting KDC Service > Restarting KPASSWD Service > Restarting MEMCACHE Service > Restarting HTTP Service > Restarting CA Service > # ipactl status > Directory Service: RUNNING > KDC Service: RUNNING > KPASSWD Service: RUNNING > MEMCACHE Service: RUNNING > HTTP Service: RUNNING > CA Service: RUNNING > # ipa cert-show 1 > ipa: ERROR: Certificate operation cannot be completed: Unable to > communicate with CMS (Internal Server Error) > > Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). > Will look into this. > HTH, > Martin From abokovoy at redhat.com Thu Aug 16 20:10:05 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 16 Aug 2012 23:10:05 +0300 Subject: [Freeipa-devel] [PATCH] trust CLI: add ID range for new trusted domain In-Reply-To: <20120814133826.GG11558@localhost.localdomain> References: <20120814133826.GG11558@localhost.localdomain> Message-ID: <20120816201005.GD2549@redhat.com> On Tue, 14 Aug 2012, Sumit Bose wrote: >Hi, > >currently only a default ID range was used for users from trusted >domains. With these two patches an individual range is created during >ipa trust-add and it will be used by the extdom plugin to calculate the >Poisx UID for the users from the trusted domain. > >'ipa trust-add' is getting two new options, --base-id and --range-size >to specify the first Posix ID of the range and the size of the range >respectively. If --range-size is not given the default will be 200000 >and if --base-id is not given it will be calculated with the help of a >hash of the domain SID. To be compatible with the AD provider of SSSD >murmurhash3 is used here. The python binding for the hash will be >provided by SSSD, the patch is currently under review. But since it is >not required to have murmurhash3, an error message will be send if it is >not installed on the server, I think this patch can be pushed >independently of the SSSD patch. ACK with one fix (attached): ignore missing murmurhash python module and samba4/server code. This will be required if building client-side only since ipalib/plugins/trust.py will be included into the client tarball. -- / Alexander Bokovoy -------------- next part -------------- >From 86a336823c8de7ccba4edfbe7bd5ccf88f0f3d7c Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Thu, 16 Aug 2012 19:57:30 +0300 Subject: [PATCH 4/5] Ignore lint errors if pysssd_murmur and samba4 support not installed when building client code. Since ipalib.plugins.trust has both client-side and server-side code, this is the only way to properly handle linting errors. --- ipalib/plugins/trust.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ipalib/plugins/trust.py b/ipalib/plugins/trust.py index 1064a06783892f27c56f0310046216881db5b42a..f19a0a874057860d0ae32f1cc2336bdc3accf6e5 100644 --- a/ipalib/plugins/trust.py +++ b/ipalib/plugins/trust.py @@ -25,14 +25,14 @@ from ipalib import errors from ipapython import ipautil from ipalib import util try: - import pysss_murmur + import pysss_murmur #pylint: disable=F0401 _murmur_installed = True except Exception, e: _murmur_installed = False if api.env.in_server and api.env.context in ['lite', 'server']: try: - import ipaserver.dcerpc + import ipaserver.dcerpc #pylint: disable=F0401 _bindings_installed = True except Exception, e: _bindings_installed = False -- 1.7.11.4 From rcritten at redhat.com Thu Aug 16 20:58:19 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 16 Aug 2012 16:58:19 -0400 Subject: [Freeipa-devel] [PATCH] trust CLI: add ID range for new trusted domain In-Reply-To: <20120816201005.GD2549@redhat.com> References: <20120814133826.GG11558@localhost.localdomain> <20120816201005.GD2549@redhat.com> Message-ID: <502D5EEB.6060409@redhat.com> Alexander Bokovoy wrote: > On Tue, 14 Aug 2012, Sumit Bose wrote: >> Hi, >> >> currently only a default ID range was used for users from trusted >> domains. With these two patches an individual range is created during >> ipa trust-add and it will be used by the extdom plugin to calculate the >> Poisx UID for the users from the trusted domain. >> >> 'ipa trust-add' is getting two new options, --base-id and --range-size >> to specify the first Posix ID of the range and the size of the range >> respectively. If --range-size is not given the default will be 200000 >> and if --base-id is not given it will be calculated with the help of a >> hash of the domain SID. To be compatible with the AD provider of SSSD >> murmurhash3 is used here. The python binding for the hash will be >> provided by SSSD, the patch is currently under review. But since it is >> not required to have murmurhash3, an error message will be send if it is >> not installed on the server, I think this patch can be pushed >> independently of the SSSD patch. > > ACK with one fix (attached): ignore missing murmurhash python module and > samba4/server code. This will be required if building client-side > only since ipalib/plugins/trust.py will be included into the client > tarball. Pushed all three to master rob From jdennis at redhat.com Fri Aug 17 00:38:41 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 16 Aug 2012 20:38:41 -0400 Subject: [Freeipa-devel] [PATCH 78] Ticket #2979 - prevent last admin from being disabled Message-ID: <502D9291.2090102@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0078-Ticket-2979-prevent-last-admin-from-being-disabled.patch Type: text/x-patch Size: 2374 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 17 05:36:29 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 07:36:29 +0200 Subject: [Freeipa-devel] [PATCH] 293 Bump bind-dyndb-ldap version in spec file Message-ID: <1345181789.3796.1.camel@priserak> The updated version of the BIND LDAP plugin includes completed support of DNS zone transfers. With the new version, users will be able to configure slave DNS servers for IPA master DNS server. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-293-bump-bind-dyndb-ldap-version-in-spec-file.patch Type: text/x-patch Size: 1416 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 17 07:55:07 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 09:55:07 +0200 Subject: [Freeipa-devel] [PATCH] 294 Read DM password from option in external CA install Message-ID: <1345190107.3796.2.camel@priserak> ipa-server-install with external CA could not be run in an unattended mode as DM password was required to decipher answer cache. https://fedorahosted.org/freeipa/ticket/2793 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-294-read-dm-password-from-option-in-external-ca-install.patch Type: text/x-patch Size: 1269 bytes Desc: not available URL: From mkosek at redhat.com Fri Aug 17 09:07:41 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 11:07:41 +0200 Subject: [Freeipa-devel] [PATCH] 295 Fix managedBy label for DNS zone Message-ID: <1345194461.3796.7.camel@priserak> Even though managedBy output parameter was only used for failed host managedBy memberships, it was defined in global baseldap.py classes. Incorrect label was then being displayed also for DNS zone per-zone permission attribute with the same name. Move managedBy output parameter to host plugin. Define proper managedBy output parameter in DNS plugin to improve clarity of this attribute. https://fedorahosted.org/freeipa/ticket/2946 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-295-fix-managedby-label-for-dns-zone.patch Type: text/x-patch Size: 3549 bytes Desc: not available URL: From ssorce at redhat.com Fri Aug 17 09:55:17 2012 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 17 Aug 2012 05:55:17 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 293 Bump bind-dyndb-ldap version in spec file In-Reply-To: <1345181789.3796.1.camel@priserak> Message-ID: <666526073.6570712.1345197317673.JavaMail.root@redhat.com> ----- Original Message ----- > The updated version of the BIND LDAP plugin includes completed > support of DNS zone transfers. With the new version, users will be > able to configure slave DNS servers for IPA master DNS server. ACK. Simo. From ssorce at redhat.com Fri Aug 17 09:57:10 2012 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 17 Aug 2012 05:57:10 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 294 Read DM password from option in external CA install In-Reply-To: <1345190107.3796.2.camel@priserak> Message-ID: <992829518.6571075.1345197430875.JavaMail.root@redhat.com> ----- Original Message ----- > ipa-server-install with external CA could not be run in > an unattended mode as DM password was required to decipher answer > cache. > > https://fedorahosted.org/freeipa/ticket/2793 ACK Simo. From mkosek at redhat.com Fri Aug 17 10:08:13 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 12:08:13 +0200 Subject: [Freeipa-devel] [PATCH] 293 Bump bind-dyndb-ldap version in spec file In-Reply-To: <666526073.6570712.1345197317673.JavaMail.root@redhat.com> References: <666526073.6570712.1345197317673.JavaMail.root@redhat.com> Message-ID: <1345198093.3796.8.camel@priserak> On Fri, 2012-08-17 at 05:55 -0400, Simo Sorce wrote: > ----- Original Message ----- > > The updated version of the BIND LDAP plugin includes completed > > support of DNS zone transfers. With the new version, users will be > > able to configure slave DNS servers for IPA master DNS server. > > ACK. > > Simo. Pushed to master. Martin From mkosek at redhat.com Fri Aug 17 10:09:17 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 12:09:17 +0200 Subject: [Freeipa-devel] [PATCH] 294 Read DM password from option in external CA install In-Reply-To: <992829518.6571075.1345197430875.JavaMail.root@redhat.com> References: <992829518.6571075.1345197430875.JavaMail.root@redhat.com> Message-ID: <1345198157.3796.9.camel@priserak> On Fri, 2012-08-17 at 05:57 -0400, Simo Sorce wrote: > ----- Original Message ----- > > ipa-server-install with external CA could not be run in > > an unattended mode as DM password was required to decipher answer > > cache. > > > > https://fedorahosted.org/freeipa/ticket/2793 > > ACK > > Simo. Pushed to master. Martin From jhrozek at redhat.com Fri Aug 17 11:44:30 2012 From: jhrozek at redhat.com (Jakub Hrozek) Date: Fri, 17 Aug 2012 13:44:30 +0200 Subject: [Freeipa-devel] [PATCH] client: include the directory with domain-realm mappings in krb5.conf Message-ID: <20120817114430.GC24190@zeppelin.brq.redhat.com> Hi, the attached patches add the directory the SSSD writes domain-realm mappings as includedir to krb5.conf when installing the client. [PATCH 1/3] ipachangeconf: allow specifying non-default delimeter for options ipachangeconf only allows one delimeter between keys and values. This patch adds the possibility of also specifying "delim" in the option dictionary to override the default delimeter. On a slightly-unrelated note, we really should think about adopting Augeas. Changing configuration with home-grown scripts is getting tricky. [PATCH 2/3] Specify includedir in krb5.conf on new installs This patch utilizes the new functionality from the previous patch to add the includedir on top of the krb5.conf file [PATCH 3/3] Add the includedir to krb5.conf on upgrades This patch is completely untested and I'm only posting it to get opinions. At first I was going to use an upgrade script in %post but then I thought it would be overengineering when all we want to do is prepend one line.. Would a simple munging like this be acceptable or shall I write a full script? -------------- next part -------------- >From 507dad241486258348153bedb06011e0f884c88f Mon Sep 17 00:00:00 2001 From: Jakub Hrozek Date: Fri, 17 Aug 2012 11:19:03 +0200 Subject: [PATCH 1/3] ipachangeconf: allow specifying non-default delimeter for options --- ipa-client/ipaclient/ipachangeconf.py | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/ipa-client/ipaclient/ipachangeconf.py b/ipa-client/ipaclient/ipachangeconf.py index f6288062be5c5d1b29341ed90814e1fa1431019c..f5ec227fee6a46eb666a1bfe9a3349e40f80b7e9 100644 --- a/ipa-client/ipaclient/ipachangeconf.py +++ b/ipa-client/ipaclient/ipachangeconf.py @@ -161,7 +161,10 @@ class IPAChangeConf: output += self.indent[level]+self.subsectdel[1]+self.deol continue if o['type'] == "option": - output += self.indent[level]+o['name']+self.dassign+o['value']+self.deol + delim = o.get('delim', self.dassign) + if delim not in self.assign: + raise ValueError('Unknown delim "%s" must be one of "%s"' % (delim, " ".join([d for d in self.assign]))) + output += self.indent[level]+o['name']+delim+o['value']+self.deol continue if o['type'] == "comment": output += self.dcomment+o['value']+self.deol @@ -182,11 +185,21 @@ class IPAChangeConf: if value: return {'name':'comment', 'type':'comment', 'value':value.rstrip()} #pylint: disable=E1103 + o = dict() parts = line.split(self.dassign, 1) if len(parts) < 2: - raise SyntaxError, 'Syntax Error: Unknown line format' + # The default assign didn't match, try the non-default + for d in self.assign[1:]: + parts = line.split(d, 1) + if len(parts) >= 2: + o['delim'] = d + break - return {'name':parts[0].strip(), 'type':'option', 'value':parts[1].rstrip()} + if 'delim' not in o: + raise SyntaxError, 'Syntax Error: Unknown line format' + + o.update({'name':parts[0].strip(), 'type':'option', 'value':parts[1].rstrip()}) + return o def findOpts(self, opts, type, name, exclude_sections=False): @@ -224,7 +237,9 @@ class IPAChangeConf: opts.append({'name':'comment', 'type':'comment', 'value':val}) continue if o['type'] == 'option': - val = self.indent[level]+o['name']+self.dassign+o['value'] + delim = o.get('delim', self.dassign) + if delim not in self.assign: + val = self.indent[level]+o['name']+delim+o['value'] opts.append({'name':'comment', 'type':'comment', 'value':val}) continue if o['type'] == 'comment': -- 1.7.11.2 -------------- next part -------------- >From 74091875ec511b1b271f723eb17918544258a957 Mon Sep 17 00:00:00 2001 From: Jakub Hrozek Date: Sun, 5 Aug 2012 20:47:12 +0200 Subject: [PATCH 2/3] Specify includedir in krb5.conf on new installs --- ipa-client/ipa-install/ipa-client-install | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/ipa-client/ipa-install/ipa-client-install b/ipa-client/ipa-install/ipa-client-install index 2e65921e8de2dfe68443f5b5875954d71dd48ed2..a4c1b6fbdfefc26f9be8297b6e60ac4d0da42876 100755 --- a/ipa-client/ipa-install/ipa-client-install +++ b/ipa-client/ipa-install/ipa-client-install @@ -642,7 +642,7 @@ def hardcode_ldap_server(cli_server): def configure_krb5_conf(fstore, cli_basedn, cli_realm, cli_domain, cli_server, cli_kdc, dnsok, options, filename, client_domain): krbconf = ipaclient.ipachangeconf.IPAChangeConf("IPA Installer") - krbconf.setOptionAssignment(" = ") + krbconf.setOptionAssignment((" = ", " ")) krbconf.setSectionNameDelimiters(("[","]")) krbconf.setSubSectionDelimiters(("{","}")) krbconf.setIndent((""," "," ")) @@ -650,6 +650,10 @@ def configure_krb5_conf(fstore, cli_basedn, cli_realm, cli_domain, cli_server, c opts = [{'name':'comment', 'type':'comment', 'value':'File modified by ipa-client-install'}, {'name':'empty', 'type':'empty'}] + # SSSD include dir + opts.append({'name':'includedir', 'type':'option', 'value':'/var/lib/sss/pubconf/krb5.include.d/', 'delim':' '}) + opts.append({'name':'empty', 'type':'empty'}) + #[libdefaults] libopts = [{'name':'default_realm', 'type':'option', 'value':cli_realm}] if not dnsok or not cli_kdc or options.force: -- 1.7.11.2 -------------- next part -------------- >From c163d4dac69243195e2d95f3a71cb31c61a1f43a Mon Sep 17 00:00:00 2001 From: Jakub Hrozek Date: Fri, 17 Aug 2012 13:34:06 +0200 Subject: [PATCH 3/3] Add the includedir to krb5.conf on upgrades --- ipa-client/ipa-client.spec.in | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/ipa-client/ipa-client.spec.in b/ipa-client/ipa-client.spec.in index 686259ad24b241c232dce83b695a05f6fd6c3849..4cd6ee66aa384c1181bcaca1c2c755cf235e79b9 100644 --- a/ipa-client/ipa-client.spec.in +++ b/ipa-client/ipa-client.spec.in @@ -36,6 +36,15 @@ mkdir -p %{buildroot}/%{_localstatedir}/lib/ipa-client/sysrestore %clean rm -rf %{buildroot} +%post +if egrep -q 'File modified by ipa-client-install' /etc/krb5.conf 2>/dev/null ; then + if ! egrep -q '/var/lib/sss/pubconf/krb5.include.d/' /etc/krb5.conf 2>/dev/null ; then + cp /etc/krb5.conf{,.ipaclientupgrade} + echo "includedir /var/lib/sss/pubconf/krb5.include.d/" > /etc/krb5.conf + cat /etc/krb5.conf.ipaclientupgrade >> /etc/krb5.conf + fi +fi + %files %defattr(-,root,root,-) -- 1.7.11.2 From mkosek at redhat.com Fri Aug 17 12:24:42 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 14:24:42 +0200 Subject: [Freeipa-devel] [PATCH] 296 Fix client-only build Message-ID: <1345206282.3796.11.camel@priserak> Client-only build unconditionally touched some files from freeipa-server package and thus the installation crashed. Fix spec file to enable client-only builds like "make client-rpms". -- Pushed to master as a one-liner. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-296-fix-client-only-build.patch Type: text/x-patch Size: 1237 bytes Desc: not available URL: From tbabej at redhat.com Fri Aug 17 13:04:23 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 17 Aug 2012 09:04:23 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0006 Removes sssd.conf after uninstall. In-Reply-To: <650187313.1969070.1345208656080.JavaMail.root@redhat.com> Message-ID: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> Hi, The sssd.conf file is no longer left behind in case sssd was not configured before the installation. https://fedorahosted.org/freeipa/ticket/2740 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0006-Removes-sssd.conf-after-uninstall.patch Type: text/x-patch Size: 1260 bytes Desc: not available URL: From alee at redhat.com Fri Aug 17 13:34:53 2012 From: alee at redhat.com (Ade Lee) Date: Fri, 17 Aug 2012 09:34:53 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345135508.2452.4.camel@priserak> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> Message-ID: <1345210494.21349.190.camel@aleeredhat.laptop> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: > On 08/16/2012 01:28 PM, Ade Lee wrote: > > Patch attached this time. I should know better than to do this in the > > middle of the night .. > > > > On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > >> On 08/16/2012 07:53 AM, Ade Lee wrote: > >>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > >>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > >>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: > >>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > >>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> Dogtag 10 is being released on f18, and has a number of changes that > >>>>>>>> will affect IPA. In particular, the following changes will affect > >>>>>>>> current IPA code. > >>>>>>>> > >>>>>>>> * The directory layout of the dogtag instance has changed. Instead of > >>>>>>>> using separate tomcat instances to host different subsystems, the > >>>>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP > >>>>>>>> and TKS within the same instance. There have been corresponding changes > >>>>>>>> in the directory layout, as well as the default instance name > >>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > >>>>>>>> of pki-cad, pki-krad etc.) > >>>>>>>> > >>>>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and > >>>>>>>> tomcat shutdown port) rather than the 6 previously used. The default > >>>>>>>> ports will be changed to the standard tomcat ports. As these ports are > >>>>>>>> local to the ipa server machine, this should not cause too much > >>>>>>>> disruption. > >>>>>>>> > >>>>>>>> * There is a new single step installer written in python. > >>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > >>>>>>>> > >>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of > >>>>>>>> tomcatjss. > >>>>>>>> > >>>>>>>> The attached patch integrates all the above changes in IPA installation > >>>>>>>> and maintenance code. Once the patch is applied, users will be able to: > >>>>>>>> > >>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. > >>>>>>>> 2. install a new replica on f18 on dogtag 10. > >>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > >>>>>>>> 10 - and have that old-style dogtag instance continue to run correctly. > >>>>>>>> This will require the installation of the latest version of tomcatjss as > >>>>>>>> well as the installation of tomcat6. The old-style instance will > >>>>>>>> continue to use tomcat6. > >>>>>>>> 4. in addition, the new cert renewal code has been patched and should > >>>>>>>> continue to work. > >>>>>>>> > >>>>>>>> What is not yet completed / supported: > >>>>>>>> > >>>>>>>> 1. Installation with an external CA is not yet completed in the new > >>>>>>>> installer. We plan to complete this soon. > >>>>>>>> > >>>>>>>> 2. There is some IPA upgrade code that has not yet been touched > >>>>>>>> (install/tools/ipa-upgradeconfig). > >>>>>>>> > >>>>>>>> 3. A script needs to be written to allow admins to convert their > >>>>>>>> old-style dogtag instances to new style instances, as well as code to > >>>>>>>> periodically prompt admins to do this. > >>>>>>>> > >>>>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on > >>>>>>>> dogtag 10 will no longer be supported, and will be disabled soon. > >>>>>>>> > >>>>>>>> 5. The pki-selinux policy has been updated to reflect these changes, > >>>>>>>> but is still in flux. In fact, it is our intention to place the dogtag > >>>>>>>> selinux policy in the base selinux policy for f18. In the meantime, it > >>>>>>>> may be necessary to run installs in permissive mode. > >>>>>>>> > >>>>>>>> The dogtag 10 code will be released shortly into f18. Prior to that > >>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a > >>>>>>>> developer repo that is located at > >>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ > >>>>>>>> > >>>>>>>> Testing can be done on both f18 and f17 - although the target platform - > >>>>>>>> and the only platform for which official builds will be created is f18. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Ade > >>>>>>>> > >>>>>>> > >>>>>>> Hi Ade, > >>>>>>> > >>>>>>> Thanks for the patch, I started with review and integration tests (currently > >>>>>>> running on Fedora 17 with Nathan's repo). > >>>>>>> > >>>>>>> Installation on single master was smooth, it worked just fine, even with > >>>>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. > >>>>>>> The resulting logs and the structure of your log directory seems improved. I > >>>>>>> believe that the brand new Python installers will make it easier to debug > >>>>>>> issues with dogtag installation when they come. When I tried our unit tests or > >>>>>>> some simple cert operation, it worked fine as well. > >>>>>>> > >>>>>>> Now the bad news, or rather few issues and suggestions I found: > >>>>>>> > >>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > >>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > >>>>>>> > >>>>>> We have a dogtag patch that is currently in review that will address > >>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > >>>>>> rather than f18+ > >>>>>> > >>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a > >>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? > >>>>>>> > >>>>>> Yes. The current IPA patch is designed to work with dogtag 10 only, > >>>>>> which will be officially available on f18+. So if you update to dogtag > >>>>>> 10, you must have this patch and visa versa. We probably need to add > >>>>>> the relevant requires to IPA to enforce this. > >>>>>> > >>>>>> If you have an existing dogtag 9 instance, and you upgrade to the new > >>>>>> dogtag 10 and patched IPA, then that instance will continue to work. > >>>>>> But any new instances would be created using dogtag 10. > >>>>>> > >>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > >>>>>>> and I got the following error: > >>>>>>> > >>>>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > >>>>>>> ... > >>>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds > >>>>>>> [1/14]: creating certificate server user > >>>>>>> [2/14]: configuring certificate server instance > >>>>>>> > >>>>>>> Your system may be partly configured. > >>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > >>>>>>> > >>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: > >>>>>>> IOError: [Errno 2] No such file or directory: > >>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > >>>>>>> > >>>>>>> Root cause: > >>>>>>> ... > >>>>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > >>>>>>> 625, in __spawn_instance > >>>>>>> "/root/cacert.p12") > >>>>>>> ... > >>>>>>> > >>>>>> I need to look into this. I had fixed ipa-replica-install, rather than > >>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install > >>>>>> existed! Should not be too bad to fix though - most likely just need to > >>>>>> move files to the right place. > >>>>> > >>>>> Ok, thanks! Btw. CA on replica can be either installed during > >>>>> ipa-replica-install time (when --setup-ca option is passed, you probably used > >>>>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > >>>>> > >>>> I will be testing this out again. As ipa-ca-install uses the same code > >>>> as ipa-replica-install, I would have expected it to work. Did you try > >>>> it with selinux in permissive mode? > >>>> > >>> OK -- I tested this out and realized that between the time I initially > >>> tested this and your tests, we had changed a URL > >>> from /ca/pki/installer/installToken to /ca/rest/installer/installToken, > >>> but I forgot to update ipa-pki-proxy.conf. > >>> > >>> The new patch has been rebased and changes the relevant patch. With > >>> this, the CA replica installs correctly. > >> > >> Umh, where can I find the new rebased patch? :-) > >> > >>> > >>> There was one issue that I encountered. I have seen this once before > >>> and will need to investigate further. IPA sometimes restarts the dogtag > >>> instance by doing systemctl stop pki-tomcatd.target. and then systemctl > >>> start pki-tomcatd.target. I have noticed that occasionally the start > >>> invocation fails, while the stop and restart invocations succeed just > >>> fine. This makes absolutely no sense because restart is essentially > >>> just stop and then start. > >>> > >>> I think this is actually a bug in systemd. If I reboot the machine, it > >>> all works perfectly. If we can't figure out whats going on with > >>> systemd, we can always replace this start/stop with starting/stopping > >>> the service directly. (pki-tomcatd at pki-tomcat.service). That seems to > >>> work all the time with no problems. > >>> > >>> I suggest we leave this fix (if needed) for a separate patch. > >> > >> Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to > >> use target units this way? I thought that only service files are meant to be > >> used for manipulation with a service. As you said yourself, using service unit > >> file for restart worked every time. > >> > >> Martin > > > > Now, I tested the rebased patch with VMs with permissive SELinux. IPA server > install was OK, as always, but replica installation ended up with error. > Although it got much further than before: > > # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > ... > [3/3]: restarting directory server > done configuring pkids. > Configuring certificate server: Estimated time 3 minutes 30 seconds > [1/14]: creating certificate server user > [2/14]: configuring certificate server instance > [3/14]: disabling nonces > [4/14]: importing CA chain to RA certificate database > [5/14]: fixing RA database permissions > [6/14]: setting up signing cert profile > [7/14]: set up CRL publishing > [8/14]: set certificate subject base > [9/14]: enabling Subject Key Identifier > [10/14]: configuring certificate server to start on boot > [11/14]: configure certmonger for renewals > [12/14]: configure clone certificate renewals > [13/14]: configure Server-Cert certificate renewal > [14/14]: Configure HTTP to proxy connections > done configuring pki-tomcatd. > Restarting the directory and certificate servers > > Your system may be partly configured. > Run /usr/sbin/ipa-server-install --uninstall to clean up. > > It seems like that CA on a replica did not start and so a check for port 8080 > failed. Attached logs (pki-ca-logs.tgz) may provide more information to this issue. > This is the systemd issue I mentioned before. I will submit a patch which will change the restart mechanism to restart the service and not the target. > > Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 > (permissive SELinux). Now, the service was upgraded smoothly, CA service > could be restarted without failure. However, certificate operations now > did not work: > > # ipactl restart > Restarting Directory Service > Restarting KDC Service > Restarting KPASSWD Service > Restarting MEMCACHE Service > Restarting HTTP Service > Restarting CA Service > # ipactl status > Directory Service: RUNNING > KDC Service: RUNNING > KPASSWD Service: RUNNING > MEMCACHE Service: RUNNING > HTTP Service: RUNNING > CA Service: RUNNING > # ipa cert-show 1 > ipa: ERROR: Certificate operation cannot be completed: Unable to > communicate with CMS (Internal Server Error) > > Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). > The reason for this is that the old dogtag instances are missing: 1. a new parameter in CS.cfg 2. a couple of symlinks I plan to fix this in the dogtag code. Specifically, 1. I will modify the dogtag code to provide a default value in the case where the parameter does not exist. 2. I plan to add a function to the dogtag startup scripts to check and remake any required symlinks. Both of these changes are in the dogtag code, and do not affect this patch. As a workaround, to confirm that the upgrade actually works, do the following manual steps on your upgraded instance: 1. Add the following parameter to CS.cfg pidDir=/var/run 2. Add the following links; cd /var/lib/pki-ca/common/lib ln -s /usr/share/java/apache-commons-codec.jar apache-commons-codec.jar ln -s /usr/share/java/log4j.jar log4j.jar 3 Restart the dogtag instance > HTH, > Martin From mkosek at redhat.com Fri Aug 17 13:44:49 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 17 Aug 2012 15:44:49 +0200 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA Message-ID: <1345211089.3796.34.camel@priserak> Hi guys, I am now investigating ticket #2866: https://fedorahosted.org/freeipa/ticket/2866 And I am thinking about possible solutions for this problem. In a nutshell, we do not properly check referential integrity in some IPA objects where we keep one-way DN references to other objects, e.g. in - managedBy attribute for a host object - memberhost attribute for HBAC rule object - memberuser attribute for user object - memberallowcmd or memberdenycmd for SUDO command object (reported in #2866) ... Currently, I see 2 approaches to solve this: 1) Add relevant checks to our ipalib plugins where problematic operations with these operations are being executed (like we do for selinuxusermap's seealso attribute in HBAC plugin) This of course would not prevent direct LDAP deletes. 2) Implement a preop DS plugin that would hook to MODRDN and DELETE callbacks and check that this object's DN is not referenced in other objects. And if it does, it would reject such modification. Second option would be to delete the attribute value with now invalid reference. This would be probably more suitable for example for references to user objects. Any comments to these possible approaches are welcome. Rich, do you think that as an alternative to these 2 approaches, memberOf plugin could be eventually modified to do this task? Thank you, Martin From rmeggins at redhat.com Fri Aug 17 14:00:24 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Fri, 17 Aug 2012 08:00:24 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <1345211089.3796.34.camel@priserak> References: <1345211089.3796.34.camel@priserak> Message-ID: <502E4E78.1090407@redhat.com> On 08/17/2012 07:44 AM, Martin Kosek wrote: > Hi guys, > > I am now investigating ticket #2866: > https://fedorahosted.org/freeipa/ticket/2866 > > And I am thinking about possible solutions for this problem. In a > nutshell, we do not properly check referential integrity in some IPA > objects where we keep one-way DN references to other objects, e.g. in > - managedBy attribute for a host object > - memberhost attribute for HBAC rule object > - memberuser attribute for user object > - memberallowcmd or memberdenycmd for SUDO command object (reported in > #2866) > ... > > Currently, I see 2 approaches to solve this: > 1) Add relevant checks to our ipalib plugins where problematic > operations with these operations are being executed (like we do for > selinuxusermap's seealso attribute in HBAC plugin) > This of course would not prevent direct LDAP deletes. > > 2) Implement a preop DS plugin that would hook to MODRDN and DELETE > callbacks and check that this object's DN is not referenced in other > objects. And if it does, it would reject such modification. Second > option would be to delete the attribute value with now invalid > reference. This would be probably more suitable for example for > references to user objects. > > Any comments to these possible approaches are welcome. > > Rich, do you think that as an alternative to these 2 approaches, > memberOf plugin could be eventually modified to do this task? This is very similar to the referential integrity plugin already in 389, except instead of cleaning up references to moved and deleted entries, you want it to prevent moving or deleting an entry if that entry is referenced by the managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other entry. Note that the managed entry plugin (mep) already handles this for the managedby attribute. Are you already using the memberof plugin for memberhost/memberuser/memberallowcmd/memberdenycmd? This doesn't seem like a job for memberof, this seems like more of a new check for the referential integrity plugin. > > Thank you, > Martin > From abokovoy at redhat.com Fri Aug 17 15:04:51 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 17 Aug 2012 18:04:51 +0300 Subject: [Freeipa-devel] [PATCH] 0070 Ask for admin password in ipa-adtrust-install Message-ID: <20120817150451.GC513@redhat.com> Hi, The credentials of the admin user will be used to obtain Kerberos ticket before configuring cross-realm trusts support and afterwards, to ensure that the ticket contains MS-PAC information required to actually add a trust with Active Directory domain via 'ipa trust-add --type=ad' command. We discussed few other approaches with Simo and decided to go for this one as the simplest. By default Kerberos tickets issued in IPA install are not renewable so it is not possible to use 'kinit -R' to renew existing ticket. Another approach was to modify our KDB driver to attach MS-PAC to selected service tickets rather than to TGT but this means we are losing advantage of 'caching' MS-PAC creation (which may be costly due to LDAP lookups for gathering group membership) as part of TGT ticket. In the end, adding two options to ipa-adtrust-install which is run only once is simpler. -A (--admin-name, defaults to 'admin') allows to specify admin user -a (--admin-password) allows to specify admin user's password If admin password is not specified, existing default ccache credentials are used and warning message about need to re-kinit is shown at the end. Unattended install is treated as if admin password was not specified. http://fedorahosted.org/freeipa/ticket/2852 -- / Alexander Bokovoy -------------- next part -------------- >From 6012df490ba4de0c69d048bef590f184b4194154 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 17 Aug 2012 15:26:58 +0300 Subject: [PATCH] Ask for admin password in ipa-adtrust-install The credentials of the admin user will be used to obtain Kerberos ticket before configuring cross-realm trusts support and afterwards, to ensure that the ticket contains MS-PAC information required to actually add a trust with Active Directory domain via 'ipa trust-add --type=ad' command. https://fedorahosted.org/freeipa/ticket/2852 --- install/tools/ipa-adtrust-install | 37 +++++++++++++++++++++++++++++++++ install/tools/man/ipa-adtrust-install.1 | 9 ++++++++ 2 files changed, 46 insertions(+) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 8cea077cca20cc6c35cff43630c422b0c869cf08..d6a35371f39b00dc750c1dbbff5c95c8e8e9f689 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -55,6 +55,12 @@ def parse_options(): "UIDs and GIDs to RIDs") parser.add_option("-U", "--unattended", dest="unattended", action="store_true", default=False, help="unattended installation never prompts the user") + parser.add_option("-a", "--admin-password", + sensitive=True, dest="admin_password", + help="admin user kerberos password") + parser.add_option("-A", "--admin-name", + sensitive=True, dest="admin_name", default='admin', + help="admin user principal") options, args = parser.parse_args() safe_options = parser.get_safe_opts(options) @@ -85,6 +91,21 @@ def read_netbios_name(netbios_default): return netbios_name +def read_admin_password(admin_name): + print "Configuring cross-realm trusts for IPA server requires password for user '%s'." % (admin_name) + print "This user is a regular system account used for IPA server administration." + print "" + admin_password = read_password(admin_name,confirm=False,validate=None) + return admin_password + +def ensure_admin_kinit(admin_name, admin_password): + try: + ipautil.run(['kinit', admin_name],stdin=admin_password+'\n') + except ipautil.CalledProcessError, e: + print "There was error to automatically re-kinit your admin user ticket." + return False + return True + def main(): safe_options, options = parse_options() @@ -193,6 +214,16 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) + admin_password = options.admin_password + if not (options.unattended or admin_password): + admin_password = read_admin_password(options.admin_name) + + admin_kinited = None + if admin_password: + admin_kinited = ensure_admin_kinit(options.admin_name, admin_password) + if not admin_kinited: + print "Proceeding with credentials that existed before" + try: ctx = krbV.default_context() ccache = ctx.default_ccache() @@ -252,6 +283,12 @@ You may want to choose to REJECT the network packets instead of DROPing them to avoid timeouts on the AD domain controllers. ============================================================================= +""" + if admin_password: + admin_kinited = ensure_admin_kinit(options.admin_name, admin_password) + + if not admin_kinited: + print """ WARNING: you MUST re-kinit admin user before using 'ipa trust-*' commands family in order to re-generate Kerberos tickets to include AD-specific information""" diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index 22337c3df8827670657bf405b6c49ba2f8624d6d..936e04c2472c272180124aa8b84749722dde0bc2 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -41,6 +41,15 @@ Do not create DNS service records for Windows in managed DNS server .TP \fB\-U\fR, \fB\-\-unattended\fR An unattended installation that will never prompt for user input +.TP +\fB\-A\fR, \fB\-\-admin\-name\fR=\fIADMIN_NAME\fR +The name of the user with administrative privileges for this IPA server. Defaults to 'admin'. +.TP +\fB\-a\fR, \fB\-\-admin\-password\fR=\fIpassword\fR +The password of the user with administrative privileges for this IPA server. Will be asked interactively if \fB\-U\fR is not specified. +.TP +The credentials of the admin user will be used to obtain Kerberos ticket before configuring cross-realm trusts support and afterwards, to ensure that the ticket contains MS-PAC information required to actually add a trust with Active Directory domain via 'ipa trust-add --type=ad' command. +.TP .SH "EXIT STATUS" 0 if the installation was successful -- 1.7.11.4 From alee at redhat.com Fri Aug 17 16:04:53 2012 From: alee at redhat.com (Ade Lee) Date: Fri, 17 Aug 2012 12:04:53 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345210494.21349.190.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> Message-ID: <1345219494.21349.193.camel@aleeredhat.laptop> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: > On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: > > On 08/16/2012 01:28 PM, Ade Lee wrote: > > > Patch attached this time. I should know better than to do this in the > > > middle of the night .. > > > > > > On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > > >> On 08/16/2012 07:53 AM, Ade Lee wrote: > > >>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > > >>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > > >>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: > > >>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > > >>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: > > >>>>>>>> Hi, > > >>>>>>>> > > >>>>>>>> Dogtag 10 is being released on f18, and has a number of changes that > > >>>>>>>> will affect IPA. In particular, the following changes will affect > > >>>>>>>> current IPA code. > > >>>>>>>> > > >>>>>>>> * The directory layout of the dogtag instance has changed. Instead of > > >>>>>>>> using separate tomcat instances to host different subsystems, the > > >>>>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP > > >>>>>>>> and TKS within the same instance. There have been corresponding changes > > >>>>>>>> in the directory layout, as well as the default instance name > > >>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead > > >>>>>>>> of pki-cad, pki-krad etc.) > > >>>>>>>> > > >>>>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and > > >>>>>>>> tomcat shutdown port) rather than the 6 previously used. The default > > >>>>>>>> ports will be changed to the standard tomcat ports. As these ports are > > >>>>>>>> local to the ipa server machine, this should not cause too much > > >>>>>>>> disruption. > > >>>>>>>> > > >>>>>>>> * There is a new single step installer written in python. > > >>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > > >>>>>>>> > > >>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of > > >>>>>>>> tomcatjss. > > >>>>>>>> > > >>>>>>>> The attached patch integrates all the above changes in IPA installation > > >>>>>>>> and maintenance code. Once the patch is applied, users will be able to: > > >>>>>>>> > > >>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. > > >>>>>>>> 2. install a new replica on f18 on dogtag 10. > > >>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag > > >>>>>>>> 10 - and have that old-style dogtag instance continue to run correctly. > > >>>>>>>> This will require the installation of the latest version of tomcatjss as > > >>>>>>>> well as the installation of tomcat6. The old-style instance will > > >>>>>>>> continue to use tomcat6. > > >>>>>>>> 4. in addition, the new cert renewal code has been patched and should > > >>>>>>>> continue to work. > > >>>>>>>> > > >>>>>>>> What is not yet completed / supported: > > >>>>>>>> > > >>>>>>>> 1. Installation with an external CA is not yet completed in the new > > >>>>>>>> installer. We plan to complete this soon. > > >>>>>>>> > > >>>>>>>> 2. There is some IPA upgrade code that has not yet been touched > > >>>>>>>> (install/tools/ipa-upgradeconfig). > > >>>>>>>> > > >>>>>>>> 3. A script needs to be written to allow admins to convert their > > >>>>>>>> old-style dogtag instances to new style instances, as well as code to > > >>>>>>>> periodically prompt admins to do this. > > >>>>>>>> > > >>>>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on > > >>>>>>>> dogtag 10 will no longer be supported, and will be disabled soon. > > >>>>>>>> > > >>>>>>>> 5. The pki-selinux policy has been updated to reflect these changes, > > >>>>>>>> but is still in flux. In fact, it is our intention to place the dogtag > > >>>>>>>> selinux policy in the base selinux policy for f18. In the meantime, it > > >>>>>>>> may be necessary to run installs in permissive mode. > > >>>>>>>> > > >>>>>>>> The dogtag 10 code will be released shortly into f18. Prior to that > > >>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a > > >>>>>>>> developer repo that is located at > > >>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ > > >>>>>>>> > > >>>>>>>> Testing can be done on both f18 and f17 - although the target platform - > > >>>>>>>> and the only platform for which official builds will be created is f18. > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> Ade > > >>>>>>>> > > >>>>>>> > > >>>>>>> Hi Ade, > > >>>>>>> > > >>>>>>> Thanks for the patch, I started with review and integration tests (currently > > >>>>>>> running on Fedora 17 with Nathan's repo). > > >>>>>>> > > >>>>>>> Installation on single master was smooth, it worked just fine, even with > > >>>>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. > > >>>>>>> The resulting logs and the structure of your log directory seems improved. I > > >>>>>>> believe that the brand new Python installers will make it easier to debug > > >>>>>>> issues with dogtag installation when they come. When I tried our unit tests or > > >>>>>>> some simple cert operation, it worked fine as well. > > >>>>>>> > > >>>>>>> Now the bad news, or rather few issues and suggestions I found: > > >>>>>>> > > >>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on > > >>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. > > >>>>>>> > > >>>>>> We have a dogtag patch that is currently in review that will address > > >>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, > > >>>>>> rather than f18+ > > >>>>>> > > >>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a > > >>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? > > >>>>>>> > > >>>>>> Yes. The current IPA patch is designed to work with dogtag 10 only, > > >>>>>> which will be officially available on f18+. So if you update to dogtag > > >>>>>> 10, you must have this patch and visa versa. We probably need to add > > >>>>>> the relevant requires to IPA to enforce this. > > >>>>>> > > >>>>>> If you have an existing dogtag 9 instance, and you upgrade to the new > > >>>>>> dogtag 10 and patched IPA, then that instance will continue to work. > > >>>>>> But any new instances would be created using dogtag 10. > > >>>>>> > > >>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well > > >>>>>>> and I got the following error: > > >>>>>>> > > >>>>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > > >>>>>>> ... > > >>>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds > > >>>>>>> [1/14]: creating certificate server user > > >>>>>>> [2/14]: configuring certificate server instance > > >>>>>>> > > >>>>>>> Your system may be partly configured. > > >>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > > >>>>>>> > > >>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: > > >>>>>>> IOError: [Errno 2] No such file or directory: > > >>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > > >>>>>>> > > >>>>>>> Root cause: > > >>>>>>> ... > > >>>>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line > > >>>>>>> 625, in __spawn_instance > > >>>>>>> "/root/cacert.p12") > > >>>>>>> ... > > >>>>>>> > > >>>>>> I need to look into this. I had fixed ipa-replica-install, rather than > > >>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install > > >>>>>> existed! Should not be too bad to fix though - most likely just need to > > >>>>>> move files to the right place. > > >>>>> > > >>>>> Ok, thanks! Btw. CA on replica can be either installed during > > >>>>> ipa-replica-install time (when --setup-ca option is passed, you probably used > > >>>>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. > > >>>>> > > >>>> I will be testing this out again. As ipa-ca-install uses the same code > > >>>> as ipa-replica-install, I would have expected it to work. Did you try > > >>>> it with selinux in permissive mode? > > >>>> > > >>> OK -- I tested this out and realized that between the time I initially > > >>> tested this and your tests, we had changed a URL > > >>> from /ca/pki/installer/installToken to /ca/rest/installer/installToken, > > >>> but I forgot to update ipa-pki-proxy.conf. > > >>> > > >>> The new patch has been rebased and changes the relevant patch. With > > >>> this, the CA replica installs correctly. > > >> > > >> Umh, where can I find the new rebased patch? :-) > > >> > > >>> > > >>> There was one issue that I encountered. I have seen this once before > > >>> and will need to investigate further. IPA sometimes restarts the dogtag > > >>> instance by doing systemctl stop pki-tomcatd.target. and then systemctl > > >>> start pki-tomcatd.target. I have noticed that occasionally the start > > >>> invocation fails, while the stop and restart invocations succeed just > > >>> fine. This makes absolutely no sense because restart is essentially > > >>> just stop and then start. > > >>> > > >>> I think this is actually a bug in systemd. If I reboot the machine, it > > >>> all works perfectly. If we can't figure out whats going on with > > >>> systemd, we can always replace this start/stop with starting/stopping > > >>> the service directly. (pki-tomcatd at pki-tomcat.service). That seems to > > >>> work all the time with no problems. > > >>> > > >>> I suggest we leave this fix (if needed) for a separate patch. > > >> > > >> Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to > > >> use target units this way? I thought that only service files are meant to be > > >> used for manipulation with a service. As you said yourself, using service unit > > >> file for restart worked every time. > > >> > > >> Martin > > > > > > > Now, I tested the rebased patch with VMs with permissive SELinux. IPA server > > install was OK, as always, but replica installation ended up with error. > > Although it got much further than before: > > > > # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > > ... > > [3/3]: restarting directory server > > done configuring pkids. > > Configuring certificate server: Estimated time 3 minutes 30 seconds > > [1/14]: creating certificate server user > > [2/14]: configuring certificate server instance > > [3/14]: disabling nonces > > [4/14]: importing CA chain to RA certificate database > > [5/14]: fixing RA database permissions > > [6/14]: setting up signing cert profile > > [7/14]: set up CRL publishing > > [8/14]: set certificate subject base > > [9/14]: enabling Subject Key Identifier > > [10/14]: configuring certificate server to start on boot > > [11/14]: configure certmonger for renewals > > [12/14]: configure clone certificate renewals > > [13/14]: configure Server-Cert certificate renewal > > [14/14]: Configure HTTP to proxy connections > > done configuring pki-tomcatd. > > Restarting the directory and certificate servers > > > > Your system may be partly configured. > > Run /usr/sbin/ipa-server-install --uninstall to clean up. > > > > It seems like that CA on a replica did not start and so a check for port 8080 > > failed. Attached logs (pki-ca-logs.tgz) may provide more information to this issue. > > > This is the systemd issue I mentioned before. I will submit a patch > which will change the restart mechanism to restart the service and not > the target. > > Patch attached. This patch should be applied on top of the large patch (being reviewed). > > Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 > > (permissive SELinux). Now, the service was upgraded smoothly, CA service > > could be restarted without failure. However, certificate operations now > > did not work: > > > > # ipactl restart > > Restarting Directory Service > > Restarting KDC Service > > Restarting KPASSWD Service > > Restarting MEMCACHE Service > > Restarting HTTP Service > > Restarting CA Service > > # ipactl status > > Directory Service: RUNNING > > KDC Service: RUNNING > > KPASSWD Service: RUNNING > > MEMCACHE Service: RUNNING > > HTTP Service: RUNNING > > CA Service: RUNNING > > # ipa cert-show 1 > > ipa: ERROR: Certificate operation cannot be completed: Unable to > > communicate with CMS (Internal Server Error) > > > > Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). > > > The reason for this is that the old dogtag instances are missing: > 1. a new parameter in CS.cfg > 2. a couple of symlinks > > I plan to fix this in the dogtag code. Specifically, > 1. I will modify the dogtag code to provide a default value in the case > where the parameter does not exist. > 2. I plan to add a function to the dogtag startup scripts to check and > remake any required symlinks. > > Both of these changes are in the dogtag code, and do not affect this > patch. As a workaround, to confirm that the upgrade actually works, do > the following manual steps on your upgraded instance: > > 1. Add the following parameter to CS.cfg > pidDir=/var/run > > 2. Add the following links; > cd /var/lib/pki-ca/common/lib > ln -s /usr/share/java/apache-commons-codec.jar apache-commons-codec.jar > ln -s /usr/share/java/log4j.jar log4j.jar > > 3 Restart the dogtag instance > > > HTH, > > Martin > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Modified-replication-code-to-restart-pki-tomcat-serv.patch Type: text/x-patch Size: 1669 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 17 16:06:21 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 17 Aug 2012 12:06:21 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345219494.21349.193.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> Message-ID: <502E6BFD.90400@redhat.com> Ade Lee wrote: > On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>> Patch attached this time. I should know better than to do this in the >>>> middle of the night .. >>>> >>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of changes that >>>>>>>>>>> will affect IPA. In particular, the following changes will affect >>>>>>>>>>> current IPA code. >>>>>>>>>>> >>>>>>>>>>> * The directory layout of the dogtag instance has changed. Instead of >>>>>>>>>>> using separate tomcat instances to host different subsystems, the >>>>>>>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP >>>>>>>>>>> and TKS within the same instance. There have been corresponding changes >>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead >>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>> >>>>>>>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and >>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. The default >>>>>>>>>>> ports will be changed to the standard tomcat ports. As these ports are >>>>>>>>>>> local to the ipa server machine, this should not cause too much >>>>>>>>>>> disruption. >>>>>>>>>>> >>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>> >>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of >>>>>>>>>>> tomcatjss. >>>>>>>>>>> >>>>>>>>>>> The attached patch integrates all the above changes in IPA installation >>>>>>>>>>> and maintenance code. Once the patch is applied, users will be able to: >>>>>>>>>>> >>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag >>>>>>>>>>> 10 - and have that old-style dogtag instance continue to run correctly. >>>>>>>>>>> This will require the installation of the latest version of tomcatjss as >>>>>>>>>>> well as the installation of tomcat6. The old-style instance will >>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>> 4. in addition, the new cert renewal code has been patched and should >>>>>>>>>>> continue to work. >>>>>>>>>>> >>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>> >>>>>>>>>>> 1. Installation with an external CA is not yet completed in the new >>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>> >>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>> >>>>>>>>>>> 3. A script needs to be written to allow admins to convert their >>>>>>>>>>> old-style dogtag instances to new style instances, as well as code to >>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>> >>>>>>>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on >>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled soon. >>>>>>>>>>> >>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect these changes, >>>>>>>>>>> but is still in flux. In fact, it is our intention to place the dogtag >>>>>>>>>>> selinux policy in the base selinux policy for f18. In the meantime, it >>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>> >>>>>>>>>>> The dogtag 10 code will be released shortly into f18. Prior to that >>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a >>>>>>>>>>> developer repo that is located at >>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>> >>>>>>>>>>> Testing can be done on both f18 and f17 - although the target platform - >>>>>>>>>>> and the only platform for which official builds will be created is f18. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Ade >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Ade, >>>>>>>>>> >>>>>>>>>> Thanks for the patch, I started with review and integration tests (currently >>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>> >>>>>>>>>> Installation on single master was smooth, it worked just fine, even with >>>>>>>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. >>>>>>>>>> The resulting logs and the structure of your log directory seems improved. I >>>>>>>>>> believe that the brand new Python installers will make it easier to debug >>>>>>>>>> issues with dogtag installation when they come. When I tried our unit tests or >>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>> >>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>> >>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on >>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>> >>>>>>>>> We have a dogtag patch that is currently in review that will address >>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, >>>>>>>>> rather than f18+ >>>>>>>>> >>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a >>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? >>>>>>>>>> >>>>>>>>> Yes. The current IPA patch is designed to work with dogtag 10 only, >>>>>>>>> which will be officially available on f18+. So if you update to dogtag >>>>>>>>> 10, you must have this patch and visa versa. We probably need to add >>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>> >>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to the new >>>>>>>>> dogtag 10 and patched IPA, then that instance will continue to work. >>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>> >>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well >>>>>>>>>> and I got the following error: >>>>>>>>>> >>>>>>>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>> ... >>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>> >>>>>>>>>> Your system may be partly configured. >>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>> >>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: >>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>> >>>>>>>>>> Root cause: >>>>>>>>>> ... >>>>>>>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line >>>>>>>>>> 625, in __spawn_instance >>>>>>>>>> "/root/cacert.p12") >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>> I need to look into this. I had fixed ipa-replica-install, rather than >>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>> existed! Should not be too bad to fix though - most likely just need to >>>>>>>>> move files to the right place. >>>>>>>> >>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>> ipa-replica-install time (when --setup-ca option is passed, you probably used >>>>>>>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. >>>>>>>> >>>>>>> I will be testing this out again. As ipa-ca-install uses the same code >>>>>>> as ipa-replica-install, I would have expected it to work. Did you try >>>>>>> it with selinux in permissive mode? >>>>>>> >>>>>> OK -- I tested this out and realized that between the time I initially >>>>>> tested this and your tests, we had changed a URL >>>>>> from /ca/pki/installer/installToken to /ca/rest/installer/installToken, >>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>> >>>>>> The new patch has been rebased and changes the relevant patch. With >>>>>> this, the CA replica installs correctly. >>>>> >>>>> Umh, where can I find the new rebased patch? :-) >>>>> >>>>>> >>>>>> There was one issue that I encountered. I have seen this once before >>>>>> and will need to investigate further. IPA sometimes restarts the dogtag >>>>>> instance by doing systemctl stop pki-tomcatd.target. and then systemctl >>>>>> start pki-tomcatd.target. I have noticed that occasionally the start >>>>>> invocation fails, while the stop and restart invocations succeed just >>>>>> fine. This makes absolutely no sense because restart is essentially >>>>>> just stop and then start. >>>>>> >>>>>> I think this is actually a bug in systemd. If I reboot the machine, it >>>>>> all works perfectly. If we can't figure out whats going on with >>>>>> systemd, we can always replace this start/stop with starting/stopping >>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That seems to >>>>>> work all the time with no problems. >>>>>> >>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>> >>>>> Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to >>>>> use target units this way? I thought that only service files are meant to be >>>>> used for manipulation with a service. As you said yourself, using service unit >>>>> file for restart worked every time. >>>>> >>>>> Martin >>>> >>> >>> Now, I tested the rebased patch with VMs with permissive SELinux. IPA server >>> install was OK, as always, but replica installation ended up with error. >>> Although it got much further than before: >>> >>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>> ... >>> [3/3]: restarting directory server >>> done configuring pkids. >>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>> [1/14]: creating certificate server user >>> [2/14]: configuring certificate server instance >>> [3/14]: disabling nonces >>> [4/14]: importing CA chain to RA certificate database >>> [5/14]: fixing RA database permissions >>> [6/14]: setting up signing cert profile >>> [7/14]: set up CRL publishing >>> [8/14]: set certificate subject base >>> [9/14]: enabling Subject Key Identifier >>> [10/14]: configuring certificate server to start on boot >>> [11/14]: configure certmonger for renewals >>> [12/14]: configure clone certificate renewals >>> [13/14]: configure Server-Cert certificate renewal >>> [14/14]: Configure HTTP to proxy connections >>> done configuring pki-tomcatd. >>> Restarting the directory and certificate servers >>> >>> Your system may be partly configured. >>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>> >>> It seems like that CA on a replica did not start and so a check for port 8080 >>> failed. Attached logs (pki-ca-logs.tgz) may provide more information to this issue. >>> >> This is the systemd issue I mentioned before. I will submit a patch >> which will change the restart mechanism to restart the service and not >> the target. >>> > > Patch attached. This patch should be applied on top of the large patch > (being reviewed). > >>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>> (permissive SELinux). Now, the service was upgraded smoothly, CA service >>> could be restarted without failure. However, certificate operations now >>> did not work: >>> >>> # ipactl restart >>> Restarting Directory Service >>> Restarting KDC Service >>> Restarting KPASSWD Service >>> Restarting MEMCACHE Service >>> Restarting HTTP Service >>> Restarting CA Service >>> # ipactl status >>> Directory Service: RUNNING >>> KDC Service: RUNNING >>> KPASSWD Service: RUNNING >>> MEMCACHE Service: RUNNING >>> HTTP Service: RUNNING >>> CA Service: RUNNING >>> # ipa cert-show 1 >>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>> communicate with CMS (Internal Server Error) >>> >>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>> >> The reason for this is that the old dogtag instances are missing: >> 1. a new parameter in CS.cfg >> 2. a couple of symlinks >> >> I plan to fix this in the dogtag code. Specifically, >> 1. I will modify the dogtag code to provide a default value in the case >> where the parameter does not exist. >> 2. I plan to add a function to the dogtag startup scripts to check and >> remake any required symlinks. >> >> Both of these changes are in the dogtag code, and do not affect this >> patch. As a workaround, to confirm that the upgrade actually works, do >> the following manual steps on your upgraded instance: >> >> 1. Add the following parameter to CS.cfg >> pidDir=/var/run >> >> 2. Add the following links; >> cd /var/lib/pki-ca/common/lib >> ln -s /usr/share/java/apache-commons-codec.jar apache-commons-codec.jar >> ln -s /usr/share/java/log4j.jar log4j.jar >> >> 3 Restart the dogtag instance > To throw a little twist in here, we should make sure that the certmonger restart scripts still work as well. rob From rcritten at redhat.com Fri Aug 17 16:13:29 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 17 Aug 2012 12:13:29 -0400 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 2 Release Message-ID: <502E6DA9.30901@redhat.com> The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 2. It can be downloaded from http://www.freeipa.org/page/Downloads. A build is available in the Fedora 18 and rawhide repositories or for Fedora 17 via the freeipa-devel repo on www.freeipa.org: http://freeipa.org/downloads/freeipa-devel.repo . To install in Fedora 17 and 18 the updates-testing repository needs to be enabled as well. NOTE: The Fedora 18 build was submitted this morning (8/17) and has yet to hit updates-testing. The packages are also at http://koji.fedoraproject.org/koji/buildinfo?buildID=348836 For additional information see the AD Trust design page http://freeipa.org/page/IPAv3_AD_trust and the AD Trust testing page http://freeipa.org/page/IPAv3_testing_AD_trust. == Highlights since 3.0.0 beta 1 == * NTLM password hash is generated for existing users on first use of IPA cross-realm environment based on their Kerberos keys without requiring a password change. * Secure identifiers compatible with Active Directory are generated automatically for existing users upon set up of IPA cross-realm environment. * Use certmonger to renew CA subsystem certificates * Support for DNS zone transfers to non-IPA slaves * Internal change to LDAP Distinguished Name handling to be more robust * Better support for Internet Explorer 9 in the UI * Allow multiple servers on client install command-line and configuring without DNS discovery. * Translation updates == Upgrading == An IPA server can be upgraded simply by installing updated rpms. The server does not need to be shut down in advance. If you have multiple servers you may upgrade them one at a time. It is expected that all servers will be upgraded in a relatively short period (days or weeks not months). They should be able to co-exist peacefully but new features will not be available on old servers and enrolling a new client against an old server will result in the SSH keys not being uploaded. Downgrading a server once upgraded is not supported. Upgrading from 2.2.0 should work but has not been fully tested. Proceed with caution. An enrolled client does not need the new packages installed unless you want to re-enroll it. SSH keys for already installed clients are not uploaded, you will have to re-enroll the client or manually upload the keys (using host-mod). == Feedback == Please provide comments, bugs and other feedback via the freeipa-devel mailing list: http://www.redhat.com/mailman/listinfo/freeipa-devel == Detailed changelog == Alexander Bokovoy (11): * ipasam: improve SASL bind callback * Use smb.conf 'dedicated keytab file' parameter instead of hard-coded value * reduce redundant checks in ldapsam_search_users() to a single statement * ipalib/plugins/trust.py: ValidationError takes 'error' named argument, not 'reason' * Handle various forms of admin accounts when establishing trusts * Follow change in samba4 beta4 for sid_check_is_domain to sid_check_is_our_sam * Rework task naming in LDAP updates to avoid conflicting names in certain cases * When ipaNTHash is missing, ask IPA to generate it from kerberos keys * Ensure ipa-adtrust-install is run with Kerberos ticket for admin user * Handle exceptions when establishing trusts * Add internationalization to DCE RPC code David Sp?ngberg (1): * Indirect roles in WebUI Gowrishankar Rajaiyan (1): * Adding exit status 3 & 4 to ipa-client-install man page Jan Cholasta (2): * Add --{set,add,del}attr options to commands which are missing them. * Raise Base64DecodeError instead of ConversionError when base64 decoding fails in Bytes parameters. John Dennis (2): * Use DN objects instead of strings * Installation fails when CN is set in certificate subject base Martin Kosek (12): * Do not change LDAPObject objectclass list * Add automount map/key update permissions * Fix ipa-managed-entries man page typo * Improve address family handling in sockets * Enable SOA serial autoincrement * Add range-mod command * Warn user if an ID range with incorrect size was created * Print ipa-ldap-updater errors during RPM upgrade * Enforce CNAME constrains for DNS commands * Avoid redundant info message during RPM update * Bump bind-dyndb-ldap version for F18 * Fix winsync agreements creation Petr Viktorin (7): * Fix batch command error reporting * Fix wrong option name in ipa-managed-entries man page * Fix updating minimum_connections in ipa-upgradeconfig * Framework for admin/install tools, with ipa-ldap-updater * Arrange stripping .po files * Update translations * Create /etc/sysconfig/network if it doesn't exist Petr Vobornik (31): * Moved configuration to last position in navigation * Display loginas information only after login * Password policy measurement units. * Web UI: kerberos ticket policy measurement units * Add and remove dns per-domain permission in Web UI * Differentiation of widget type and text_widget input type * Fixed display of attributes_widget in IE9 * Bigger textarea for permission type=subtree * IDs and names for dialogs * Fix autoscroll to top in tables in IE * Fixed: Unable to select option in combobox in IE and Chrome * Fixed: Unable to select option in combobox in IE and Chrome * Fixed: combobox stacking in service adder dialog * PAC Type options for services in Web UI * Update to jquery.1.7.2.min * Update to jquery-ui-1.8.21.custom * Fix for incorrect event handler definition * Removal of unnecessary overrides of jquery-ui styles * Unified buttons * Web UI tests fix * Fixed incorrect use of jQuery.attr for setting disabled attribute * Replace use of attr with prop for booleans * Add external group * Make group external * Make group posix * Display group type * Attribute facet * Group external member facet * Read-only external facet for non-external groups * Handle case when trusted domain user access the Web UI * Disable caching of Web UI login_kerberos request * Update other facets on delete from search page Rob Crittenden (12): * Centralize timeout for waiting for servers to start. * Make client server option multi-valued, allow disabling DNS discovery * Don't hardcode serial_autoincrement to True. * Support per-principal sessions and handle session update failures * Default to no when trying trying to install a replica on wrong server. * Fix validator for SELinux user map settings in config plugin. * Use certmonger to renew CA subsystem certificates * Add per-service option to store the types of PAC it supports * Convert PKCS#11 subject to string before passing to ipapython.DN * Use DN object for Directory Manager in ipa-replica-manage connect command * Raise proper exception when given a bad DN attribute. * Validate default user in ordered list when using setattr, require MLS Simo Sorce (14): * Fix wrong check after allocation. * Fix safety checks to prevent orphaning replicas * Fix detection of deleted masters * Add libtalloc-devel as spec file BuildRequire * Add all external samba libraries to BuildRequires * Do not check for DNA magic values * Move code into common krb5 utils * Improve loops around slapi mods * Add special modify op to regen ipaNTHash * Move mspac structure to be a private pointer * Load list of trusted domain on connecting to ldap * Properly name function to add ipa external groups * Split out manipulation of logon_info blob * Add PAC filtering Sumit Bose (4): * Allow silent build if available * ipasam: fixes for clang warnings * ipasam: replace testing code * Fix typo Tomas Babej (5): * Adds check for ipa-join. * Permissions of replica files changed to 0600. * Handle SSSD restart crash more gently. * Corrects help description of selinuxusermap. * Improves exception handling in ipa-replica-prepare. From ssorce at redhat.com Fri Aug 17 16:20:27 2012 From: ssorce at redhat.com (Simo Sorce) Date: Fri, 17 Aug 2012 12:20:27 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] client: include the directory with domain-realm mappings in krb5.conf In-Reply-To: <20120817114430.GC24190@zeppelin.brq.redhat.com> Message-ID: <223750653.6801841.1345220427417.JavaMail.root@redhat.com> ----- Original Message ----- > Hi, > > the attached patches add the directory the SSSD writes domain-realm > mappings as includedir to krb5.conf when installing the client. > > [PATCH 1/3] ipachangeconf: allow specifying non-default delimeter for > options > ipachangeconf only allows one delimeter between keys and values. This > patch adds the possibility of also specifying "delim" in the option > dictionary to override the default delimeter. > > On a slightly-unrelated note, we really should think about adopting > Augeas. Changing configuration with home-grown scripts is getting > tricky. > > [PATCH 2/3] Specify includedir in krb5.conf on new installs > This patch utilizes the new functionality from the previous patch to > add > the includedir on top of the krb5.conf file > > [PATCH 3/3] Add the includedir to krb5.conf on upgrades > This patch is completely untested and I'm only posting it to get > opinions. At first I was going to use an upgrade script in %post but > then I thought it would be overengineering when all we want to do is > prepend one line.. Would a simple munging like this be acceptable or > shall I write a full script? NACK, using a scriptlet is fine, but not the way you did, as it has a huge race condition where krb5.conf exists and has only one line in it (the include line). You should first create the new file: echo "include ..." > /etc/krb.conf.ipanew Then cat the contents of the existing file in i:t cat /etc/krb.conf >> /etc/krb.conf.ipanew And finally atomically rename it: mv /etc/krb.conf.ipanew /etc/krb.conf This method is also safe wrt something killing the yum process ... Simo. From jdennis at redhat.com Fri Aug 17 20:25:19 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 17 Aug 2012 16:25:19 -0400 Subject: [Freeipa-devel] [PATCH 79] Ticket #3008: DN objects hash differently depending on case Message-ID: <502EA8AF.5000305@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0079-Ticket-3008-DN-objects-hash-differently-depending-on.patch Type: text/x-patch Size: 10075 bytes Desc: not available URL: From tbabej at redhat.com Mon Aug 20 08:08:41 2012 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 20 Aug 2012 04:08:41 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <1373440535.2144904.1345450118691.JavaMail.root@redhat.com> Message-ID: <1903220732.2144911.1345450121103.JavaMail.root@redhat.com> Hi, Dependency on samba4-winbind has been added to the package freeipa-server-trust-ad. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0007-Adds-dependency-on-samba4-windbind.patch Type: text/x-patch Size: 1209 bytes Desc: not available URL: From tbabej at redhat.com Mon Aug 20 08:16:14 2012 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 20 Aug 2012 04:16:14 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <1903220732.2144911.1345450121103.JavaMail.root@redhat.com> Message-ID: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> Fixed typo in the commit message. Tomas ----- Original Message ----- From: "Tomas Babej" To: freeipa-devel at redhat.com Sent: Monday, August 20, 2012 10:08:41 AM Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. Hi, Dependency on samba4-winbind has been added to the package freeipa-server-trust-ad. Tomas _______________________________________________ Freeipa-devel mailing list Freeipa-devel at redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0007-Adds-dependency-on-samba4-winbind.patch Type: text/x-patch Size: 1208 bytes Desc: not available URL: From abokovoy at redhat.com Mon Aug 20 11:30:20 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 20 Aug 2012 14:30:20 +0300 Subject: [Freeipa-devel] [PATCH 79] Ticket #3008: DN objects hash differently depending on case In-Reply-To: <502EA8AF.5000305@redhat.com> References: <502EA8AF.5000305@redhat.com> Message-ID: <20120820113020.GA4312@redhat.com> On Fri, 17 Aug 2012, John Dennis wrote: >Because the attrs & values in DN's, RDN's and AVA's are comparison case- >insensitive the hash value between two objects which compare as equal but >differ in case must also yield the same hash value. This is critical when >these objects are used as a dict key or in a set because dicts and sets >use the object's __hash__ value in conjunction with the objects __eq__ >method to lookup the object. > >The defect is the DN, RDN & AVA objects computed their hash from the case- >preserving string representation thus two otherwise equal objects >incorrectly yielded different hash values. > >The problem manifests itself when one of these objects is used as a key in >a dict, for example a dn. > >dn1 = DN(('cn', 'Bob')) >dn2 = DN(('cn', 'bob')) > >dn1 == dn2 --> True > >hash(dn1) == hash(dn2) --> False > >d = {} > >d[dn1] = x >d[dn2] = y > >len(d) --> 2 > >The patch fixes the above by lower casing the string representation of >the object prior to computing it's hash. > >The patch also corrects a spelling mistake and a bogus return value in >ldapupdate.py which happened to be discovered while researching this >bug. ACK. With this patch I am no longer getting constraint violation errors. -- / Alexander Bokovoy From abokovoy at redhat.com Mon Aug 20 11:38:40 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 20 Aug 2012 14:38:40 +0300 Subject: [Freeipa-devel] [PATCH] 0071 Recover from invalid cached credentials in ipasam Message-ID: <20120820113840.GB4312@redhat.com> Hi, https://fedorahosted.org/freeipa/ticket/3009 -- / Alexander Bokovoy -------------- next part -------------- >From c5904ea253dae18db383d2efed8a85fbbe4d5c8b Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 20 Aug 2012 13:26:20 +0300 Subject: [PATCH 3/3] Recover from invalid cached kerberos credentials in ipasam When developing and testing in the same environment, multiple re-installs may be needed. This means previously issued and cached Kerberos credentials will become invalid upon new install. ipasam passdb module for Samba uses Kerberos authentication when talking to IPA LDAP server. Obtained Kerberos credentials are cached during their lifetime. However, the ccache is not removed automatically and if IPA setup is made again, cached credentials are used, only to discover that they are invalid. With this change invalid correctly obtained cached credentials are recognized and, if LDAP SASL bind fails, new credentials are requested from the KDC. https://fedorahosted.org/freeipa/ticket/3009 --- daemons/ipa-sam/ipa_sam.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index aa54429b5bec4b26906b2a34e59ff95299a67f80..2ecbc055df60bc40d4032c17cbdf5fb9b3d1842f 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -3311,6 +3311,7 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo krb5_free_principal(data.context, in_creds.client); if (rc) { +notdone: rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); if (rc) { bind_callback_cleanup(&data, rc); @@ -3337,6 +3338,19 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo LDAP_SASL_QUIET, ldap_sasl_interact, &data); if (ret != LDAP_SUCCESS) { + if (LDAP_SECURITY_ERROR(ret)) { + /* LDAP server rejected our ccache. There may be several issues: + * 1. Credentials are invalid due to outdated ccache leftover from previous install + * Wipe out old ccache and start again + * 2. Key in the keytab is not enough to obtain ticket for cifs/FQDN at REALM service + * Cannot continue without proper keytab + */ + if ((ret == LDAP_INVALID_CREDENTIALS) && (rc == 0)) { + /* LDAP reported credentials are invalid but Kerberos result was 0 + * thus we were using invalid cached creds. Force reinitialize. */ + goto notdone; + } + } DEBUG(0, ("bind_callback: cannot perform interactive SASL bind with GSSAPI\n")); } -- 1.7.11.4 From abokovoy at redhat.com Mon Aug 20 11:40:53 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 20 Aug 2012 14:40:53 +0300 Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> References: <1903220732.2144911.1345450121103.JavaMail.root@redhat.com> <648385417.2145989.1345450574787.JavaMail.root@redhat.com> Message-ID: <20120820114053.GC4312@redhat.com> On Mon, 20 Aug 2012, Tomas Babej wrote: >Fixed typo in the commit message. > >Tomas > >----- Original Message ----- >From: "Tomas Babej" >To: freeipa-devel at redhat.com >Sent: Monday, August 20, 2012 10:08:41 AM >Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. > >Hi, > >Dependency on samba4-winbind has been added to the package >freeipa-server-trust-ad. ACK. -- / Alexander Bokovoy From pvoborni at redhat.com Mon Aug 20 11:50:03 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 20 Aug 2012 13:50:03 +0200 Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> References: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> Message-ID: <5032246B.1080800@redhat.com> On 08/20/2012 10:16 AM, Tomas Babej wrote: > Fixed typo in the commit message. > > Tomas > > ----- Original Message ----- > From: "Tomas Babej" > To: freeipa-devel at redhat.com > Sent: Monday, August 20, 2012 10:08:41 AM > Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. > > Hi, > > Dependency on samba4-winbind has been added to the package > freeipa-server-trust-ad. > > Tomas > +* Mon Aug 20 2012 Tomas Babej - 3.0.0-1 > +- Add samba4-winbind to build dependencies for AD server-side code > + Maybe a nitpick: Shouldn't the version be 2.99.0-42? -- Petr Vobornik From abokovoy at redhat.com Mon Aug 20 12:05:50 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 20 Aug 2012 15:05:50 +0300 Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <5032246B.1080800@redhat.com> References: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> <5032246B.1080800@redhat.com> Message-ID: <20120820120550.GD4312@redhat.com> On Mon, 20 Aug 2012, Petr Vobornik wrote: >On 08/20/2012 10:16 AM, Tomas Babej wrote: >>Fixed typo in the commit message. >> >>Tomas >> >>----- Original Message ----- >>From: "Tomas Babej" >>To: freeipa-devel at redhat.com >>Sent: Monday, August 20, 2012 10:08:41 AM >>Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. >> >>Hi, >> >>Dependency on samba4-winbind has been added to the package >>freeipa-server-trust-ad. >> >>Tomas > >>+* Mon Aug 20 2012 Tomas Babej - 3.0.0-1 >>+- Add samba4-winbind to build dependencies for AD server-side code >>+ > >Maybe a nitpick: > >Shouldn't the version be 2.99.0-42? Yes, good catch. We haven't released 3.0 yet, so 2.99.0 is our current aim. -- / Alexander Bokovoy From pvoborni at redhat.com Mon Aug 20 15:49:41 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 20 Aug 2012 17:49:41 +0200 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI In-Reply-To: <20120814073412.GE11558@localhost.localdomain> References: <501F6D74.7050509@redhat.com> <50299E9D.9060309@redhat.com> <20120814073412.GE11558@localhost.localdomain> Message-ID: <50325C95.8060703@redhat.com> Updated patch attached. Preview can be seen at: http://pvoborni.fedorapeople.org/ranges/#ipaserver=range&navigation=ipaserver&range-facet=search Some comments below. On 08/14/2012 09:34 AM, Sumit Bose wrote: > On Mon, Aug 13, 2012 at 07:41:01PM -0500, Endi Sukma Dewata wrote: >> On 8/6/2012 2:08 AM, Petr Vobornik wrote: >>> Range web UI was implemented. >>> >>> It consist of: >>> * new menu item - 'ranges' in 'IPA Server' tab >>> * new search page >>> * new details page >>> >>> https://fedorahosted.org/freeipa/ticket/2894 >>> >>> >>> Some comments/questions: >>> 1) I'm not sure about options in adder dialog. Should they all be there >>> or should I remove some? And also labels seems to long. >> >> Depending on the range type each of these options might be required, >> so it's necessary to show them all in the adder dialog, but they can >> be enabled/disabled (see #4). The labels probably can be shortened >> to mimic the CLI parameters, and the current labels can be moved >> into the doc attributes. So it may look something like this (feel >> free to change): >> >> Add ID Range >> ------------------------------------------------------ >> Range name: [ ] >> Range type: (*) Local domain >> ( ) Active Directory domain >> >> Base ID: [ ] >> Range size: [ ] >> >> Primary RID base: [ ] >> Secondary RID base: [ ] >> >> Domain SID: [ ] > > I agree, all options should be display and depending on the Range type > either Secondary RID base or Domain SID should be greyed out. Implemented this way with minor modification, see preview mentioned above. Also used in details page to be consistent. > >> >>> 2) FreeIPA creates default range. This range doesn't have required >>> attribute 'First RID of the corresponding RID range'. It prevents from >>> editing the range in Web UI until a some value is set. Not sure if even >>> should be edited though. >> >> Someone else more knowledgeable should answer this. One possibility >> is to introduce another range type (e.g. POSIX) that doesn't use >> RID. > > ah, the attribute is not required by the schema, but the CLI currently > requires it. I was focused on trusts while writing it, but in general it > makes sense to allow to add an ID range with only base ID and range > size. One possible use case would be a stand alone IPA domain which runs > out of Posix ID because the range created at installation time is full. > In this case it makes no sense to add the RID attributes. > > I have created https://fedorahosted.org/freeipa/ticket/2999 to track > this and also the issue without parameters mentioned below. > >> >> 3. As shown in #1, it might be better to call it "ID Ranges" as in >> the CLI. "Ranges" by itself doesn't sound very meaningful. > > yes > I can hard-code it in Web UI but I think it would be wrong. IMO it should be change in ranges.py: label = _('Ranges') label_singular = _('Range') >> >> 4. In range.py the range type seems to be not user >> enterable/editable and automatically generated based on the domain >> SID. However, in the adder dialog the range type options can be used >> to enable/disable the appropriate fields for the type. For example, >> selecting "local domain" will make the secondary RID base required. > > yes, see my comment above iparangetype is no longer send in range-add request. > >> >> 5. This is a CLI issue. If you call ipa range-add without parameters >> it will ask for the parameters, but then it will fail before asking >> the secondary RID base or domain SID: >> >> # ipa range-add >> Range name: test >> First Posix ID of the range: 10000 >> Number of IDs in the range: 100 >> First RID of the corresponding RID range: 50 >> ipa: ERROR: invalid Gettext('Range setup', domain='ipa', >> localedir=None): Ranges for local domain must have a secondary RID >> base > > https://fedorahosted.org/freeipa/ticket/2999 > > bye, > Sumit > -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0193-1-Range-Web-UI.patch Type: text/x-patch Size: 20449 bytes Desc: not available URL: From pviktori at redhat.com Mon Aug 20 17:37:29 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 20 Aug 2012 13:37:29 -0400 Subject: [Freeipa-devel] [PATCH 78] Ticket #2979 - prevent last admin from being disabled In-Reply-To: <502D9291.2090102@redhat.com> References: <502D9291.2090102@redhat.com> Message-ID: <503275D9.2020907@redhat.com> (Sorry if you're getting this twice; I didn't send it to the list) On 08/16/2012 08:38 PM, John Dennis wrote: > > -- > John Dennis > > Looking to carve out IT costs? > www.redhat.com/carveoutcosts/ > > freeipa-jdennis-0078-Ticket-2979-prevent-last-admin-from-being-disabled.patch > > >>From c47109c63530e188db76986fdda48c76bf681d10 Mon Sep 17 00:00:00 2001 > From: John Dennis > Date: Thu, 16 Aug 2012 20:28:44 -0400 > Subject: [PATCH 78] Ticket #2979 - prevent last admin from being disabled > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: 8bit > > We prevent the last member of the admin group from being deleted. The > same check needs to be performed when disabling a user. > > Moved the code in del_user to a common subroutine and call it from > both user_del and user_disable. Note, unlike user_del user_disable > does not have a 'pre' callback therefore the check function is called > in user_disable's execute routine. This should also prevent disabling all admins if there's more than one: # ipa user-add admin2 --first=a --last=b ------------------- Added user "admin2" ------------------- ... # ipa group-add-member admins --user=admin2 ------------------------- Number of members added 1 ------------------------- # ipa user-disable admin2 ------------------------------ Disabled user account "admin2" ------------------------------ # ipa user-disable admin ------------------------------ Disabled user account "admin" ------------------------------ # ipa ping ipa: ERROR: Server is unwilling to perform: Account inactivated. Contact system administrator. Also with one enabled and one disabled admin, it shouldn't be possible to delete the enabled one. Please add some tests; you can extend the ones added in commit f8e7b51. -- Petr? From jfenal at gmail.com Mon Aug 20 20:11:40 2012 From: jfenal at gmail.com (=?UTF-8?B?SsOpcsO0bWUgRmVuYWw=?=) Date: Mon, 20 Aug 2012 22:11:40 +0200 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 2 Release In-Reply-To: <502E6DA9.30901@redhat.com> References: <502E6DA9.30901@redhat.com> Message-ID: 2012/8/17 Rob Crittenden > The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 2. > > Hi Rob, Regarding translations, I don't see yet a 3.0 branch on Transifex. Is it in the pipeline, so we could have time to work on translations for 3.0 GA? Regards, J. -- J?r?me Fenal - jfenal AT gmail.com - http://fenal.org/ Paris.pm - http://paris.mongueurs.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdennis at redhat.com Mon Aug 20 21:10:07 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 20 Aug 2012 17:10:07 -0400 Subject: [Freeipa-devel] [PATCH 80] Ticket #2850 - Ipactl exception not handled well Message-ID: <5032A7AF.104@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0080-Ticket-2850-Ipactl-exception-not-handled-well.patch Type: text/x-patch Size: 3466 bytes Desc: not available URL: From jdennis at redhat.com Mon Aug 20 21:11:13 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 20 Aug 2012 17:11:13 -0400 Subject: [Freeipa-devel] [PATCH 80] Ticket #2850 - Ipactl exception not handled well Message-ID: <5032A7F1.1060505@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0080-Ticket-2850-Ipactl-exception-not-handled-well.patch Type: text/x-patch Size: 3466 bytes Desc: not available URL: From rcritten at redhat.com Mon Aug 20 21:27:58 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 20 Aug 2012 17:27:58 -0400 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 2 Release In-Reply-To: References: <502E6DA9.30901@redhat.com> Message-ID: <5032ABDE.8000907@redhat.com> J?r?me Fenal wrote: > > 2012/8/17 Rob Crittenden > > > The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 2. > > > Hi Rob, > > Regarding translations, I don't see yet a 3.0 branch on Transifex. Is it > in the pipeline, so we could have time to work on translations for 3.0 GA? The master branch in Transifex equivalent to the master branch in IPA, and has been updated recently. We plan to branch for 3.0 in the IPA repo, I suppose we can look into branching in Transifex at the same time, but for now it should be safe to work from master. I don't believe Transifex really handles branches very well, but John knows more about it than I. rob From jdennis at redhat.com Mon Aug 20 21:36:00 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 20 Aug 2012 17:36:00 -0400 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 2 Release In-Reply-To: <5032ABDE.8000907@redhat.com> References: <502E6DA9.30901@redhat.com> <5032ABDE.8000907@redhat.com> Message-ID: <5032ADC0.4030208@redhat.com> On 08/20/2012 05:27 PM, Rob Crittenden wrote: > J?r?me Fenal wrote: >> >> 2012/8/17 Rob Crittenden > >> >> The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 2. >> >> >> Hi Rob, >> >> Regarding translations, I don't see yet a 3.0 branch on Transifex. Is it >> in the pipeline, so we could have time to work on translations for 3.0 GA? > > The master branch in Transifex equivalent to the master branch in IPA, > and has been updated recently. > > We plan to branch for 3.0 in the IPA repo, I suppose we can look into > branching in Transifex at the same time, but for now it should be safe > to work from master. I don't believe Transifex really handles branches > very well, but John knows more about it than I. Transifex has no concept of branches much to many developers dismay. The suggested solution in Transifex to create a new resource for each branch (which is what we did with 2.2). Currently in TX we have an ipa resource which maps to git's master and a 2.2 resource which maps to git's 2.2 branch.. It's kind of sucky because there is no connection in TX between resources and hence branches, each resource stands on it's own. That means even though 90% of the strings in one resource are the same as another resource (because they're just different branch versions) translators won't have the advantage of that information (unless they know). I brought the question of software branches up in the TX forums and was met with deafening silence. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From edewata at redhat.com Mon Aug 20 21:53:50 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 20 Aug 2012 16:53:50 -0500 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI In-Reply-To: <50325C95.8060703@redhat.com> References: <501F6D74.7050509@redhat.com> <50299E9D.9060309@redhat.com> <20120814073412.GE11558@localhost.localdomain> <50325C95.8060703@redhat.com> Message-ID: <5032B1EE.7010908@redhat.com> On 8/20/2012 10:49 AM, Petr Vobornik wrote: > Updated patch attached. > Preview can be seen at: > http://pvoborni.fedorapeople.org/ranges/#ipaserver=range&navigation=ipaserver&range-facet=search ACK. >> I agree, all options should be display and depending on the Range type >> either Secondary RID base or Domain SID should be greyed out. > > Implemented this way with minor modification, see preview mentioned > above. Also used in details page to be consistent. Possible improvement, in the details page the fields that are not relevant to the range type could be hidden because they will be empty and not editable anyway. For instance, it's not necessary to show the Domain SID field for local domain range. >>> 3. As shown in #1, it might be better to call it "ID Ranges" as in >>> the CLI. "Ranges" by itself doesn't sound very meaningful. > > I can hard-code it in Web UI but I think it would be wrong. IMO it > should be change in ranges.py: > > label = _('Ranges') > label_singular = _('Range') Yeah. How about renaming the CLI command from range-* to idrange-*? -- Endi S. Dewata From ssorce at redhat.com Tue Aug 21 06:43:08 2012 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 21 Aug 2012 02:43:08 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0071 Recover from invalid cached credentials in ipasam In-Reply-To: <20120820113840.GB4312@redhat.com> Message-ID: <611978218.8097353.1345531388957.JavaMail.root@redhat.com> ----- Original Message ----- > Hi, > > https://fedorahosted.org/freeipa/ticket/3009 What prevents this patch from causing an infinite loop if we keep getting the same error back at each interaction ? Simo. From sbose at redhat.com Tue Aug 21 07:00:13 2012 From: sbose at redhat.com (Sumit Bose) Date: Tue, 21 Aug 2012 09:00:13 +0200 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI In-Reply-To: <5032B1EE.7010908@redhat.com> References: <501F6D74.7050509@redhat.com> <50299E9D.9060309@redhat.com> <20120814073412.GE11558@localhost.localdomain> <50325C95.8060703@redhat.com> <5032B1EE.7010908@redhat.com> Message-ID: <20120821070013.GB2021@localhost.localdomain> On Mon, Aug 20, 2012 at 04:53:50PM -0500, Endi Sukma Dewata wrote: > On 8/20/2012 10:49 AM, Petr Vobornik wrote: > >Updated patch attached. > >Preview can be seen at: > >http://pvoborni.fedorapeople.org/ranges/#ipaserver=range&navigation=ipaserver&range-facet=search > > ACK. > > >>I agree, all options should be display and depending on the Range type > >>either Secondary RID base or Domain SID should be greyed out. > > > >Implemented this way with minor modification, see preview mentioned > >above. Also used in details page to be consistent. > > Possible improvement, in the details page the fields that are not > relevant to the range type could be hidden because they will be > empty and not editable anyway. For instance, it's not necessary to > show the Domain SID field for local domain range. > > >>>3. As shown in #1, it might be better to call it "ID Ranges" as in > >>>the CLI. "Ranges" by itself doesn't sound very meaningful. > > > >I can hard-code it in Web UI but I think it would be wrong. IMO it > >should be change in ranges.py: > > > > label = _('Ranges') > > label_singular = _('Range') > > Yeah. How about renaming the CLI command from range-* to idrange-*? I have opened https://fedorahosted.org/freeipa/ticket/3010 so that it can be discussed during the IPA call. I'm fine with the change, but I would like to hear if there are concerns changing the name. bye, Sumit > > -- > Endi S. Dewata > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From abokovoy at redhat.com Tue Aug 21 07:11:57 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 21 Aug 2012 10:11:57 +0300 Subject: [Freeipa-devel] [PATCH] 0071 Recover from invalid cached credentials in ipasam In-Reply-To: <611978218.8097353.1345531388957.JavaMail.root@redhat.com> References: <20120820113840.GB4312@redhat.com> <611978218.8097353.1345531388957.JavaMail.root@redhat.com> Message-ID: <20120821071157.GE4312@redhat.com> On Tue, 21 Aug 2012, Simo Sorce wrote: >----- Original Message ----- >> Hi, >> >> https://fedorahosted.org/freeipa/ticket/3009 > >What prevents this patch from causing an infinite loop if we keep getting the same error back at each interaction ? The loop is triggered when kerberos credentials were obtained successfully based on cached credentials in the ccache but SASL operation denied them. At this point a code after notdone label will wipe out content of the ccache and attempt to acquire credentials online based on the content of samba's keytab. Obtained credentials will be put into the ccache for further cached use. If any step in acquiring credentials fails, the callback returns with LDAP_LOCAL_ERROR, effectively ending current SASL auth attempt. On higher level smbldap API user retries several times (up to two dozen times) to authenticate and on complete failure calls smb_panic(). If credentials were acquired at previous step correctly SASL step cannot fail with LDAP_INVALID_CREDENTIALS, there will be another error message, either LDAP_INAPPROPRIATE_AUTH or LDAP_INSUFFICIENT_ACCESS. In case of FreeIPA setup we shouldn't remaining security error, LDAP_X_PROXY_AUTHZ_FAILURE. Any of those will get us out of the loop. Thus, this loop is run at most twice. -- / Alexander Bokovoy From ssorce at redhat.com Tue Aug 21 09:59:16 2012 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 21 Aug 2012 05:59:16 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0071 Recover from invalid cached credentials in ipasam In-Reply-To: <20120821071157.GE4312@redhat.com> Message-ID: <2110262191.8248794.1345543156644.JavaMail.root@redhat.com> ----- Original Message ----- > On Tue, 21 Aug 2012, Simo Sorce wrote: > >----- Original Message ----- > >> Hi, > >> > >> https://fedorahosted.org/freeipa/ticket/3009 > > > >What prevents this patch from causing an infinite loop if we keep > >getting the same error back at each interaction ? > > The loop is triggered when kerberos credentials were obtained > successfully based on cached credentials in the ccache but SASL > operation denied them. At this point a code after notdone label will > wipe out content of the ccache and attempt to acquire credentials > online based on the content of samba's keytab. > > Obtained credentials will be put into the ccache for further cached > use. > > If any step in acquiring credentials fails, the callback returns with > LDAP_LOCAL_ERROR, effectively ending current SASL auth attempt. On > higher level smbldap API user retries several times (up to two dozen > times) to authenticate and on complete failure calls smb_panic(). > > If credentials were acquired at previous step correctly SASL step > cannot fail > with LDAP_INVALID_CREDENTIALS, there will be another error message, > either LDAP_INAPPROPRIATE_AUTH or LDAP_INSUFFICIENT_ACCESS. In case > of > FreeIPA setup we shouldn't remaining security error, > LDAP_X_PROXY_AUTHZ_FAILURE. Any of those will get us out of the loop. > > Thus, this loop is run at most twice. Ok, then ACK. Simo. From sbose at redhat.com Tue Aug 21 10:57:37 2012 From: sbose at redhat.com (Sumit Bose) Date: Tue, 21 Aug 2012 12:57:37 +0200 Subject: [Freeipa-devel] [PATCH] ipadb_iterate(): handle match_entry == NULL Message-ID: <20120821105737.GD2021@localhost.localdomain> Hi, there was an issue reported yesterday on #freeipa (https://fedorahosted.org/freeipa/ticket/3011). It is easy to reproduce 'kdb5_util dump' just core dumps. The attached patch adds a parameter check to the call where the core dump occured and fixes the reason why the parameter was invalid. Please note that 'kdb5_util dump' will return 'kdb5_util: error performing Kerberos version 5 release 1.8 dump (Plugin does not support the operation)' with the patch applied, because ipadb_iterate_pwd_policy() is not implemented. -------------- next part -------------- From 30dca9cf940487bd6907308ef8f4e2394556e08d Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 21 Aug 2012 12:48:29 +0200 Subject: [PATCH] ipadb_iterate(): handle match_entry == NULL If match_entry == NULL all principals should be iterated. Additionally this patch adds a check in ipadb_filter_escape() to make sure that the input is not NULL. Fixes: https://fedorahosted.org/freeipa/ticket/3011 --- daemons/ipa-kdb/ipa_kdb_common.c | 4 ++++ daemons/ipa-kdb/ipa_kdb_principals.c | 6 ++++++ 2 Dateien ge?ndert, 10 Zeilen hinzugef?gt(+) diff --git a/daemons/ipa-kdb/ipa_kdb_common.c b/daemons/ipa-kdb/ipa_kdb_common.c index 6f5ac1d74f04c03bccdb19187a34d07b9784fa59..71df9634c4e25378494b165db9a9381f2b8fc206 100644 --- a/daemons/ipa-kdb/ipa_kdb_common.c +++ b/daemons/ipa-kdb/ipa_kdb_common.c @@ -30,6 +30,10 @@ char *ipadb_filter_escape(const char *input, bool star) size_t i = 0; size_t j = 0; + if (!input) { + return NULL; + } + /* Assume the worst-case. */ output = malloc(strlen(input) * 3 + 1); if (!output) { diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c index 6f8b296fa4cb19cbfe5c37536316d6f0e7f83b9c..62155816201f705b7828c861915bf63c6b00177b 100644 --- a/daemons/ipa-kdb/ipa_kdb_principals.c +++ b/daemons/ipa-kdb/ipa_kdb_principals.c @@ -1879,6 +1879,12 @@ krb5_error_code ipadb_iterate(krb5_context kcontext, return KRB5_KDB_DBNOTINITED; } + /* If no match_entry is given iterate through all krb princs like the db2 + * or ldap plugin */ + if (match_entry == NULL) { + match_entry = "*"; + } + /* fetch list of principal matching filter */ kerr = ipadb_fetch_principals(ipactx, 0, match_entry, &res); if (kerr != 0) { -- 1.7.11.4 From tbabej at redhat.com Tue Aug 21 12:24:11 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 21 Aug 2012 14:24:11 +0200 Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <20120820120550.GD4312@redhat.com> References: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> <5032246B.1080800@redhat.com> <20120820120550.GD4312@redhat.com> Message-ID: <50337DEB.8060706@redhat.com> On 08/20/2012 02:05 PM, Alexander Bokovoy wrote: > On Mon, 20 Aug 2012, Petr Vobornik wrote: >> On 08/20/2012 10:16 AM, Tomas Babej wrote: >>> Fixed typo in the commit message. >>> >>> Tomas >>> >>> ----- Original Message ----- >>> From: "Tomas Babej" >>> To: freeipa-devel at redhat.com >>> Sent: Monday, August 20, 2012 10:08:41 AM >>> Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on >>> samba4-windbind. >>> >>> Hi, >>> >>> Dependency on samba4-winbind has been added to the package >>> freeipa-server-trust-ad. >>> >>> Tomas >> >>> +* Mon Aug 20 2012 Tomas Babej - 3.0.0-1 >>> +- Add samba4-winbind to build dependencies for AD server-side code >>> + >> >> Maybe a nitpick: >> >> Shouldn't the version be 2.99.0-42? > Yes, good catch. We haven't released 3.0 yet, so 2.99.0 is our current > aim. > > Sorry, I got confused by the release of the 3.0 beta 2 version. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0007-2-Adds-dependency-on-samba4-winbind.patch Type: text/x-patch Size: 1211 bytes Desc: not available URL: From mmercier at gmail.com Tue Aug 21 12:45:00 2012 From: mmercier at gmail.com (Michael Mercier) Date: Tue, 21 Aug 2012 08:45:00 -0400 Subject: [Freeipa-devel] freeipa / tacacs+ integration Message-ID: Hello, Back in Aug 2010, someone posted a message to the freeipa-users mail list asking about tacacs+ / freeipa integration. It was mentioned in the thread that this integration was not on the timeline for freeipa. I was wondering the following: 1. Is there any plans on RedHat's part to integrate tacacs+ with freeipa? 2. Does anyone know if someone has done any development towards this (RedHat or otherwise)? 3. If I was interested in attempting this integration using the tac_plus (www.shrubbery.net/tac_plus/) daemon, where is the best place to find find information regarding: a) designing the ldap schema (and get OID's?)? b) what would be the best way to authenticate users (ldap calls, sssd?) c) anything else I have not considered 4. Which version should I use for integration? 5. Where is the best place to track down documentation for extending IPA (I don't have much luck using freeipa.org) Thanks, Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From ssorce at redhat.com Tue Aug 21 12:53:50 2012 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 21 Aug 2012 08:53:50 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] ipadb_iterate(): handle match_entry == NULL In-Reply-To: <20120821105737.GD2021@localhost.localdomain> Message-ID: <1954125606.8363935.1345553630617.JavaMail.root@redhat.com> ----- Original Message ----- > Hi, > > there was an issue reported yesterday on #freeipa > (https://fedorahosted.org/freeipa/ticket/3011). It is easy to > reproduce > 'kdb5_util dump' just core dumps. The attached patch adds a parameter > check to the call where the core dump occured and fixes the reason > why > the parameter was invalid. > > Please note that 'kdb5_util dump' will return 'kdb5_util: error > performing Kerberos version 5 release 1.8 dump (Plugin does not > support > the operation)' with the patch applied, because > ipadb_iterate_pwd_policy() is not implemented. Can you open a ticket on this ? We used the dump utility in the past to perform bulk master key changes in the past and we do want to have the option open bot for backup purposes and other reasons. Simo. From ssorce at redhat.com Tue Aug 21 13:06:37 2012 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 21 Aug 2012 09:06:37 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] ipadb_iterate(): handle match_entry == NULL In-Reply-To: <1954125606.8363935.1345553630617.JavaMail.root@redhat.com> Message-ID: <1292598576.8371089.1345554397961.JavaMail.root@redhat.com> ----- Original Message ----- > ----- Original Message ----- > > Hi, > > > > there was an issue reported yesterday on #freeipa > > (https://fedorahosted.org/freeipa/ticket/3011). It is easy to > > reproduce > > 'kdb5_util dump' just core dumps. The attached patch adds a > > parameter > > check to the call where the core dump occured and fixes the reason > > why > > the parameter was invalid. > > > > Please note that 'kdb5_util dump' will return 'kdb5_util: error > > performing Kerberos version 5 release 1.8 dump (Plugin does not > > support > > the operation)' with the patch applied, because > > ipadb_iterate_pwd_policy() is not implemented. > > Can you open a ticket on this ? We used the dump utility in the past > to perform bulk master key changes in the past and we do want to > have the option open bot for backup purposes and other reasons. Also obvious ack to the patch. Simo. From pviktori at redhat.com Tue Aug 21 13:31:37 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 21 Aug 2012 09:31:37 -0400 Subject: [Freeipa-devel] [PATCH] 0072 Internationalization for public errors In-Reply-To: <500FFB88.6080902@redhat.com> References: <500FFB88.6080902@redhat.com> Message-ID: <50338DB9.3050606@redhat.com> On 07/25/2012 09:58 AM, Petr Viktorin wrote: > I spent some time while IPA was installing tracking down error mesages > that weren't marked for translation :) > I also changed the error docstrings, as people are likely to look there > for inspiration. > > > https://fedorahosted.org/freeipa/ticket/1953 Attaching rebased patch. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0072-02-Internationalization-for-public-errors.patch Type: text/x-patch Size: 50998 bytes Desc: not available URL: From pvoborni at redhat.com Tue Aug 21 13:42:43 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 21 Aug 2012 15:42:43 +0200 Subject: [Freeipa-devel] [PATCH] 193 Range Web UI In-Reply-To: <20120821070013.GB2021@localhost.localdomain> References: <501F6D74.7050509@redhat.com> <50299E9D.9060309@redhat.com> <20120814073412.GE11558@localhost.localdomain> <50325C95.8060703@redhat.com> <5032B1EE.7010908@redhat.com> <20120821070013.GB2021@localhost.localdomain> Message-ID: <50339053.2000107@redhat.com> On 08/21/2012 09:00 AM, Sumit Bose wrote: > On Mon, Aug 20, 2012 at 04:53:50PM -0500, Endi Sukma Dewata wrote: >> On 8/20/2012 10:49 AM, Petr Vobornik wrote: >>> Updated patch attached. >>> Preview can be seen at: >>> http://pvoborni.fedorapeople.org/ranges/#ipaserver=range&navigation=ipaserver&range-facet=search >> >> ACK. Pushed to master. >> >>>> I agree, all options should be display and depending on the Range type >>>> either Secondary RID base or Domain SID should be greyed out. >>> >>> Implemented this way with minor modification, see preview mentioned >>> above. Also used in details page to be consistent. >> >> Possible improvement, in the details page the fields that are not >> relevant to the range type could be hidden because they will be >> empty and not editable anyway. For instance, it's not necessary to >> show the Domain SID field for local domain range. >> Yeah. Such change could be good for other parts of the UI too, ie in groups a posix field. Will be done separately. >>>>> 3. As shown in #1, it might be better to call it "ID Ranges" as in >>>>> the CLI. "Ranges" by itself doesn't sound very meaningful. >>> >>> I can hard-code it in Web UI but I think it would be wrong. IMO it >>> should be change in ranges.py: >>> >>> label = _('Ranges') >>> label_singular = _('Range') >> >> Yeah. How about renaming the CLI command from range-* to idrange-*? > > I have opened https://fedorahosted.org/freeipa/ticket/3010 so that it > can be discussed during the IPA call. I'm fine with the change, but I > would like to hear if there are concerns changing the name. > > bye, > Sumit > >> >> -- >> Endi S. Dewata -- Petr Vobornik From sbose at redhat.com Tue Aug 21 14:00:02 2012 From: sbose at redhat.com (Sumit Bose) Date: Tue, 21 Aug 2012 16:00:02 +0200 Subject: [Freeipa-devel] [PATCH] ipadb_iterate(): handle match_entry == NULL In-Reply-To: <1954125606.8363935.1345553630617.JavaMail.root@redhat.com> References: <20120821105737.GD2021@localhost.localdomain> <1954125606.8363935.1345553630617.JavaMail.root@redhat.com> Message-ID: <20120821140002.GE2021@localhost.localdomain> On Tue, Aug 21, 2012 at 08:53:50AM -0400, Simo Sorce wrote: > ----- Original Message ----- > > Hi, > > > > there was an issue reported yesterday on #freeipa > > (https://fedorahosted.org/freeipa/ticket/3011). It is easy to > > reproduce > > 'kdb5_util dump' just core dumps. The attached patch adds a parameter > > check to the call where the core dump occured and fixes the reason > > why > > the parameter was invalid. > > > > Please note that 'kdb5_util dump' will return 'kdb5_util: error > > performing Kerberos version 5 release 1.8 dump (Plugin does not > > support > > the operation)' with the patch applied, because > > ipadb_iterate_pwd_policy() is not implemented. > > Can you open a ticket on this ? We used the dump utility in the past to perform bulk master key changes in the past and we do want to have the option open bot for backup purposes and other reasons. > sure, https://fedorahosted.org/freeipa/ticket/3015 . bye, Sumit > Simo. From pvoborni at redhat.com Tue Aug 21 14:03:15 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 21 Aug 2012 16:03:15 +0200 Subject: [Freeipa-devel] [PATCH] 197 Fixed search in HBAC test Message-ID: <50339523.1070909@redhat.com> Search in HBAC test wasn't working because expired flag wasn't set. https://fedorahosted.org/freeipa/ticket/2931 Notes: HBAC facets don't have refresh button. They can be refreshed by changing filter and searching. If one search with same filter, it sets expired flag but it doesn't refresh (search) because page state isn't changed. It refreshes when one go to different facet and returns back. Is this behavior acceptable? Or should we a) don't set expired flag when searching with unchanged filter b) force refresh when searching with unchanged filter c) add refresh button along with a) I prefer leave it as is or b) -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0197-Fixed-search-in-HBAC-test.patch Type: text/x-patch Size: 887 bytes Desc: not available URL: From abokovoy at redhat.com Tue Aug 21 14:15:11 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 21 Aug 2012 17:15:11 +0300 Subject: [Freeipa-devel] [PATCH] 0071 Recover from invalid cached credentials in ipasam In-Reply-To: <2110262191.8248794.1345543156644.JavaMail.root@redhat.com> References: <20120821071157.GE4312@redhat.com> <2110262191.8248794.1345543156644.JavaMail.root@redhat.com> Message-ID: <20120821141511.GG4312@redhat.com> On Tue, 21 Aug 2012, Simo Sorce wrote: >----- Original Message ----- >> On Tue, 21 Aug 2012, Simo Sorce wrote: >> >----- Original Message ----- >> >> Hi, >> >> >> >> https://fedorahosted.org/freeipa/ticket/3009 >> > >> >What prevents this patch from causing an infinite loop if we keep >> >getting the same error back at each interaction ? >> >> The loop is triggered when kerberos credentials were obtained >> successfully based on cached credentials in the ccache but SASL >> operation denied them. At this point a code after notdone label will >> wipe out content of the ccache and attempt to acquire credentials >> online based on the content of samba's keytab. >> >> Obtained credentials will be put into the ccache for further cached >> use. >> >> If any step in acquiring credentials fails, the callback returns with >> LDAP_LOCAL_ERROR, effectively ending current SASL auth attempt. On >> higher level smbldap API user retries several times (up to two dozen >> times) to authenticate and on complete failure calls smb_panic(). >> >> If credentials were acquired at previous step correctly SASL step >> cannot fail >> with LDAP_INVALID_CREDENTIALS, there will be another error message, >> either LDAP_INAPPROPRIATE_AUTH or LDAP_INSUFFICIENT_ACCESS. In case >> of >> FreeIPA setup we shouldn't remaining security error, >> LDAP_X_PROXY_AUTHZ_FAILURE. Any of those will get us out of the loop. >> >> Thus, this loop is run at most twice. > >Ok, then ACK. I've rewrote the patch to use helper functions instead of looping. -- / Alexander Bokovoy -------------- next part -------------- >From efa143dbd798951659a370543dcb218e93c05667 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 20 Aug 2012 13:26:20 +0300 Subject: [PATCH 3/4] Recover from invalid cached kerberos credentials in ipasam When developing and testing in the same environment, multiple re-installs may be needed. This means previously issued and cached Kerberos credentials will become invalid upon new install. ipasam passdb module for Samba uses Kerberos authentication when talking to IPA LDAP server. Obtained Kerberos credentials are cached during their lifetime. However, the ccache is not removed automatically and if IPA setup is made again, cached credentials are used, only to discover that they are invalid. With this change invalid correctly obtained cached credentials are recognized and, if LDAP SASL bind fails, new credentials are requested from the KDC. https://fedorahosted.org/freeipa/ticket/3009 --- daemons/ipa-sam/ipa_sam.c | 116 +++++++++++++++++++++++++++++++--------------- 1 file changed, 78 insertions(+), 38 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index aa54429b5bec4b26906b2a34e59ff95299a67f80..059109374bd0e1aa1de118b4767b5692d0e483a2 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -3204,49 +3204,70 @@ static int ldap_sasl_interact(LDAP *ld, unsigned flags, void *priv_data, void *s return ret; } -static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data, krb5_error_code rc) -{ + +static void bind_callback_cleanup_creds(struct ipasam_sasl_interact_priv *datap) { + krb5_free_cred_contents(datap->context, &datap->creds); + + if (datap->options) { + krb5_get_init_creds_opt_free(datap->context, datap->options); + datap->options = NULL; + } +} + +static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *datap, krb5_error_code rc) { const char *errstring = NULL; - if (!data->context) { + if (!datap->context) { return; } if (rc) { - errstring = krb5_get_error_message(data->context, rc); + errstring = krb5_get_error_message(datap->context, rc); DEBUG(0,("kerberos error: code=%d, message=%s\n", rc, errstring)); - krb5_free_error_message(data->context, errstring); + krb5_free_error_message(datap->context, errstring); } - krb5_free_cred_contents(data->context, &data->creds); + bind_callback_cleanup_creds(datap); - if (data->options) { - krb5_get_init_creds_opt_free(data->context, data->options); - data->options = NULL; + if (datap->keytab) { + krb5_kt_close(datap->context, datap->keytab); + datap->keytab = NULL; } - if (data->keytab) { - krb5_kt_close(data->context, data->keytab); - data->keytab = NULL; + if (datap->ccache) { + krb5_cc_close(datap->context, datap->ccache); + datap->ccache = NULL; } - if (data->ccache) { - krb5_cc_close(data->context, data->ccache); - data->ccache = NULL; + if (datap->principal) { + krb5_free_principal(datap->context, datap->principal); + datap->principal = NULL; } - if (data->principal) { - krb5_free_principal(data->context, data->principal); - data->principal = NULL; + krb5_free_context(datap->context); + datap->context = NULL; +} + +static krb5_error_code bind_callback_obtain_creds(struct ipasam_sasl_interact_priv *datap) { + krb5_error_code rc; + + rc = krb5_get_init_creds_opt_alloc(datap->context, &datap->options); + if (rc) { + return rc; + } + + rc = krb5_get_init_creds_opt_set_out_ccache(datap->context, datap->options, datap->ccache); + if (rc) { + return rc; } - krb5_free_context(data->context); - data->context = NULL; + rc = krb5_get_init_creds_keytab(datap->context, &datap->creds, datap->principal, datap->keytab, + 0, NULL, datap->options); + return rc; } extern const char * lp_dedicated_keytab_file(void); -static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_priv) -{ +static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_priv) { krb5_error_code rc; krb5_creds *out_creds = NULL; krb5_creds in_creds; @@ -3311,20 +3332,7 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo krb5_free_principal(data.context, in_creds.client); if (rc) { - rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); - if (rc) { - bind_callback_cleanup(&data, rc); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_get_init_creds_opt_set_out_ccache(data.context, data.options, data.ccache); - if (rc) { - bind_callback_cleanup(&data, rc); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_get_init_creds_keytab(data.context, &data.creds, data.principal, data.keytab, - 0, NULL, data.options); + rc = bind_callback_obtain_creds(&data); if (rc) { bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; @@ -3336,8 +3344,40 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo NULL, NULL, LDAP_SASL_QUIET, ldap_sasl_interact, &data); - if (ret != LDAP_SUCCESS) { - DEBUG(0, ("bind_callback: cannot perform interactive SASL bind with GSSAPI\n")); + + /* By now we have 'ret' for LDAP result and 'rc' for Kerberos result + * if ret is LDAP_INVALID_CREDENTIALS, LDAP server rejected our ccache. There may be several issues: + * + * 1. Credentials are invalid due to outdated ccache leftover from previous install + * Wipe out old ccache and start again + * + * 2. Key in the keytab is not enough to obtain ticket for cifs/FQDN at REALM service + * Cannot continue without proper keytab + * + * Only process (1) because (2) and other errors will be taken care of by smbd after multiple retries. + * + * Since both smbd and winbindd will use this passdb module, on startup both will try to access the same + * ccache. It may happen that if ccache was missing or contained invalid cached credentials, that one of + * them will complain loudly about missing ccache file at the time when the other one will be creating + * a new ccache file by the above call of bind_callback_obtain_creds(). This is expected and correct behavior. + * + */ + if ((ret == LDAP_INVALID_CREDENTIALS) && (rc == 0)) { + bind_callback_cleanup_creds(&data); + rc = bind_callback_obtain_creds(&data); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } + ret = ldap_sasl_interactive_bind_s(ldap_struct, + NULL, "GSSAPI", + NULL, NULL, + LDAP_SASL_QUIET, + ldap_sasl_interact, &data); + } + + if (LDAP_SECURITY_ERROR(ret)) { + DEBUG(0, ("bind_callback: cannot perform interactive SASL bind with GSSAPI. LDAP security error is %d\n", ret)); } if (out_creds) { -- 1.7.11.4 From abokovoy at redhat.com Tue Aug 21 14:27:33 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 21 Aug 2012 17:27:33 +0300 Subject: [Freeipa-devel] [PATCH] 0072 Add ACI to allow magic regen of ipaNThash Message-ID: <20120821142733.GH4312@redhat.com> Hi, I finally managed to get all ends together for magic regen of ipaNTHash based on availability of RC4 key in Kerberos keys. The patch should be applied after 0071 and can be tested by following: 0. run ipa-adtrust-install 1. ipa user-add foo 2. ipa passwd foo 3. Remember current ipaNTHash value: # ldapsearch -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket 'uid=foo' ipaNTHash > foo.current.ldif 4. Remove generated ipaNThash with ldapmodify: removal.ldif: ---8<---8<---- dn: uid=foo,cn=users,cn=accounts,dc=ipa,dc=local delete:ipaNtHash --->8--->8---- # ldapmodify -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket -f removal.ldif 5. Use 'wbinfo -i foo' (from samba4-winbind-clients) to trigger regeneration 6. Retrieve new ipaNTHash value: # ldapsearch -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket 'uid=foo' ipaNTHash > foo.regen.ldif 7. Check foo.current.ldif and foo.regen.ldif, there should be no difference. https://fedorahosted.org/freeipa/ticket/3016 -- / Alexander Bokovoy -------------- next part -------------- From db693373270ab2129406c90d49efb62ffa112d1b Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Tue, 21 Aug 2012 12:05:28 +0300 Subject: [PATCH 4/4] Add ACI to allow regenerating ipaNTHash from ipasam and fix ipaNTHash retrieval ACI was lacking to allow actually writing MagicRegen into ipaNTHash attribute, and empty filter wasn't picked up as libldap library default for (objectclass=*). With this change ipasam is able to ask for ipaNTHash generation and if corresponding Kerberos key is available, will be able to retrieve generated ipaNTHash. https://fedorahosted.org/freeipa/ticket/3016 --- daemons/ipa-sam/ipa_sam.c | 22 +++++++++------------- install/updates/60-trusts.update | 1 + 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 059109374bd0e1aa1de118b4767b5692d0e483a2..8a4a08bc7a5951553a463805a8aedb82ee887936 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -2417,7 +2417,7 @@ static bool ipasam_nthash_retrieve(struct ldapsam_privates *ldap_state, }; ret = smbldap_search(smbldap_state, entry_dn, - LDAP_SCOPE_BASE, "", attr_list, 0, + LDAP_SCOPE_BASE, "(objectclass=*)", attr_list, 0, &result); if (ret != LDAP_SUCCESS) { DEBUG(1, ("Failed to get NT hash: %s\n", @@ -2453,15 +2453,13 @@ static bool ipasam_nthash_regen(struct ldapsam_privates *ldap_state, TALLOC_CTX *mem_ctx, char * entry_dn) { - LDAPMod **mods; + LDAPMod **mods = NULL; int ret; - mods = NULL; - smbldap_make_mod(ldap_state->smbldap_state->ldap_struct, - NULL, &mods, LDAP_ATTRIBUTE_NTHASH, "MagicRegen"); - + smbldap_set_mod(&mods, LDAP_MOD_ADD, LDAP_ATTRIBUTE_NTHASH, "MagicRegen"); talloc_autofree_ldapmod(mem_ctx, mods); - ret = smbldap_add(ldap_state->smbldap_state, entry_dn, mods); + + ret = smbldap_modify(ldap_state->smbldap_state, entry_dn, mods); if (ret != LDAP_SUCCESS) { DEBUG(5, ("ipasam: attempt to regen ipaNTHash failed\n")); } @@ -2585,13 +2583,11 @@ static bool init_sam_from_ldap(struct ldapsam_privates *ldap_state, * */ temp = smbldap_talloc_dn(tmp_ctx, ldap_state->smbldap_state->ldap_struct, entry); if (temp) { - retval = ipasam_nthash_regen(tmp_ctx, - ldap_state->smbldap_state->ldap_struct, - temp); + retval = ipasam_nthash_regen(ldap_state, + tmp_ctx, temp); if (retval) { - retval = ipasam_nthash_retrieve(tmp_ctx, - ldap_state->smbldap_state->ldap_struct, - temp, &nthash); + retval = ipasam_nthash_retrieve(ldap_state, + tmp_ctx, temp, &nthash); } } } diff --git a/install/updates/60-trusts.update b/install/updates/60-trusts.update index 0e40ca4d16133f0c1e93300fc13a08dd5ba4ddf7..61013287d3e96079e041f1cb109274b4ab409b27 100644 --- a/install/updates/60-trusts.update +++ b/install/updates/60-trusts.update @@ -61,6 +61,7 @@ add:aci: '(target = "ldap:///cn=trusts,$SUFFIX")(targetattr = "ipaNTTrustType || # Add ipaNTHash to global ACIs, leave DNS tree out of global allow access rule dn: $SUFFIX add:aci: '(targetattr = "ipaNTHash")(version 3.0; acl "Samba system principals can read NT passwords"; allow (read) groupdn="ldap:///cn=adtrust agents,cn=sysaccounts,cn=etc,$SUFFIX";)' +add:aci: '(targetattr = "ipaNTHash")(version 3.0; acl "Samba system principals can write NT passwords"; allow (write) groupdn="ldap:///cn=adtrust agents,cn=sysaccounts,cn=etc,$SUFFIX";)' replace:aci:'(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || userPKCS12")(version 3.0; acl "Enable Anonymous access"; allow (read, search, compare) userdn = "ldap:///anyone";)::(target != "ldap:///idnsname=*,cn=dns,$SUFFIX")(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || userPKCS12 || ipaNTHash")(version 3.0; acl "Enable Anonymous access"; allow (read, search, compare) userdn = "ldap:///anyone";)' replace:aci:'(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || krbPrincipalName || krbCanonicalName || krbUPEnabled || krbTicketPolicyReference || krbPrincipalExpiration || krbPasswordExpiration || krbPwdPolicyReference || krbPrincipalType || krbPwdHistory || krbLastPwdChange || krbPrincipalAliases || krbExtraData || krbLastSuccessfulAuth || krbLastFailedAuth || krbLoginFailedCount || krbTicketFlags || ipaUniqueId || memberOf || serverHostName || enrolledBy")(version 3.0; acl "Admin can manage any entry"; allow (all) groupdn = "ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)::(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || krbPrincipalName || krbCanonicalName || krbUPEnabled || krbTicketPolicyReference || krbPrincipalExpiration || krbPasswordExpiration || krbPwdPolicyReference || krbPrincipalType || krbPwdHistory || krbLastPwdChange || krbPrincipalAliases || krbExtraData || krbLastSuccessfulAuth || krbLastFailedAuth || krbLoginFailedCount || krbTicketFlags || ipaUniqueId || memberOf || serverHostName || enrolledBy || ipaNTHash")(version 3.0; acl "Admin can manage any entry"; allow (all) groupdn = "ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)' replace:aci:'(targetattr = "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory")(version 3.0; acl "Admins can write passwords"; allow (add,delete,write) groupdn="ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)::(targetattr = "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || ipaNTHash")(version 3.0; acl "Admins can write passwords"; allow (add,delete,write) groupdn="ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)' -- 1.7.11.4 From ssorce at redhat.com Tue Aug 21 15:29:25 2012 From: ssorce at redhat.com (Simo Sorce) Date: Tue, 21 Aug 2012 11:29:25 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0071 Recover from invalid cached credentials in ipasam In-Reply-To: <20120821141511.GG4312@redhat.com> Message-ID: <106250197.8547420.1345562965695.JavaMail.root@redhat.com> ----- Original Message ----- > On Tue, 21 Aug 2012, Simo Sorce wrote: > >----- Original Message ----- > >> On Tue, 21 Aug 2012, Simo Sorce wrote: > >> >----- Original Message ----- > >> >> Hi, > >> >> > >> >> https://fedorahosted.org/freeipa/ticket/3009 > >> > > >> >What prevents this patch from causing an infinite loop if we keep > >> >getting the same error back at each interaction ? > >> > >> The loop is triggered when kerberos credentials were obtained > >> successfully based on cached credentials in the ccache but SASL > >> operation denied them. At this point a code after notdone label > >> will > >> wipe out content of the ccache and attempt to acquire credentials > >> online based on the content of samba's keytab. > >> > >> Obtained credentials will be put into the ccache for further > >> cached > >> use. > >> > >> If any step in acquiring credentials fails, the callback returns > >> with > >> LDAP_LOCAL_ERROR, effectively ending current SASL auth attempt. On > >> higher level smbldap API user retries several times (up to two > >> dozen > >> times) to authenticate and on complete failure calls smb_panic(). > >> > >> If credentials were acquired at previous step correctly SASL step > >> cannot fail > >> with LDAP_INVALID_CREDENTIALS, there will be another error > >> message, > >> either LDAP_INAPPROPRIATE_AUTH or LDAP_INSUFFICIENT_ACCESS. In > >> case > >> of > >> FreeIPA setup we shouldn't remaining security error, > >> LDAP_X_PROXY_AUTHZ_FAILURE. Any of those will get us out of the > >> loop. > >> > >> Thus, this loop is run at most twice. > > > >Ok, then ACK. > > I've rewrote the patch to use helper functions instead of looping. Indeed it looks better this way. Simo. From ssorce at redhat.com Wed Aug 22 06:38:32 2012 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 22 Aug 2012 02:38:32 -0400 (EDT) Subject: [Freeipa-devel] [BRANCH 3.0] Heads up new branch created Message-ID: <1679582802.8799836.1345617512021.JavaMail.root@redhat.com> Hello fellow developers, Rob asked me to go ahead and create a 3.0 branch. It's up, and the name follows the usual scheme: ipa-3.0 Please remind from now on to specify whether your patches are supposed to be pushed only in 3.0 or master or both. We do not expect the branches to diverge wildly immediatey, but 3.0 is now in stabilization mode only, so any patch that disrupts APIs or implements major new features should go in master only. The new branch has been forked off the beta2 release but I already applied on top all the following patches available in master as they all apply to the 3.0 goal so at this moment the 2 branches are still aligned. I want to also take a moment to thank all contirbutors and users from our wonderful community, please keep engaging with us as you did so far, the success of our project has been in no small part a success of our community and I hope version 3.0 will be a great tool for all of us. Happy hacking! Simo. -- Simo Sorce * Red Hat, Inc. * New York From pvoborni at redhat.com Wed Aug 22 07:50:18 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 22 Aug 2012 09:50:18 +0200 Subject: [Freeipa-devel] [PATCH] 198 Revert change causing failure in test automation Message-ID: <50348F3A.3060200@redhat.com> Move of click handler in patch for #2834 causes failure of automation tests. This patch reverts the problematic part. It should not affect function of fix for #2834. https://fedorahosted.org/freeipa/ticket/3014 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0198-Revert-change-causing-failure-in-test-automation.patch Type: text/x-patch Size: 1394 bytes Desc: not available URL: From jgalipea at redhat.com Wed Aug 22 11:44:46 2012 From: jgalipea at redhat.com (Jenny Galipeau) Date: Wed, 22 Aug 2012 07:44:46 -0400 Subject: [Freeipa-devel] [BRANCH 3.0] Heads up new branch created In-Reply-To: <1679582802.8799836.1345617512021.JavaMail.root@redhat.com> References: <1679582802.8799836.1345617512021.JavaMail.root@redhat.com> Message-ID: <5034C62E.8060303@redhat.com> On 08/22/2012 02:38 AM, Simo Sorce wrote: > Hello fellow developers, > > Rob asked me to go ahead and create a 3.0 branch. > > It's up, and the name follows the usual scheme: ipa-3.0 > > Please remind from now on to specify whether your patches are supposed to be pushed only in 3.0 or master or both. > > We do not expect the branches to diverge wildly immediatey, but 3.0 is now in stabilization mode only, so any patch that disrupts APIs or implements major new features should go in master only. > > The new branch has been forked off the beta2 release but I already applied on top all the following patches available in master as they all apply to the 3.0 goal so at this moment the 2 branches are still aligned. > > I want to also take a moment to thank all contirbutors and users from our wonderful community, please keep engaging with us as you did so far, the success of our project has been in no small part a success of our community and I hope version 3.0 will be a great tool for all of us. +1 Thanks!! > > Happy hacking! > > Simo. > -- Jenny Galipeau Principal Software QA Engineer Red Hat, Inc. Security Engineering Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ From pvoborni at redhat.com Wed Aug 22 12:17:29 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 22 Aug 2012 14:17:29 +0200 Subject: [Freeipa-devel] [PATCH] 199 Permissions: select only applicable options on type change Message-ID: <5034CDD9.2050900@redhat.com> Problem: When a permission is edited, and Type switched, the attributes selected for previous Type are still selected, and update fails, if they are invalid for the new Type. But it should get deselected or not even listed if Type changes. Fix: When Type is changed, attribute list is refreshed and still applicable attributes are chosen. If Type is reverted back, previously chosen attributes are back as chosen. If attributes are extended outside Web UI by not listed attr, this attr is listed at the list end. Note: If user makes change in attribute list before type change, this change is forgotten. https://fedorahosted.org/freeipa/ticket/2617 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0199-Permissions-select-only-applicable-options-on-type-c.patch Type: text/x-patch Size: 6022 bytes Desc: not available URL: From abokovoy at redhat.com Wed Aug 22 12:31:25 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 15:31:25 +0300 Subject: [Freeipa-devel] [PATCH] 0072 Add ACI to allow magic regen of ipaNThash In-Reply-To: <20120821142733.GH4312@redhat.com> References: <20120821142733.GH4312@redhat.com> Message-ID: <20120822123125.GI4312@redhat.com> On Tue, 21 Aug 2012, Alexander Bokovoy wrote: > Hi, > > I finally managed to get all ends together for magic regen of ipaNTHash > based on availability of RC4 key in Kerberos keys. > > The patch should be applied after 0071 and can be tested by following: > > 0. run ipa-adtrust-install > > 1. ipa user-add foo > > 2. ipa passwd foo > > 3. Remember current ipaNTHash value: > # ldapsearch -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket 'uid=foo' ipaNTHash > foo.current.ldif > > 4. Remove generated ipaNThash with ldapmodify: > > removal.ldif: > ---8<---8<---- > dn: uid=foo,cn=users,cn=accounts,dc=ipa,dc=local > delete:ipaNtHash > --->8--->8---- > # ldapmodify -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket -f removal.ldif > > 5. Use 'wbinfo -i foo' (from samba4-winbind-clients) to trigger regeneration > > 6. Retrieve new ipaNTHash value: > # ldapsearch -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket 'uid=foo' ipaNTHash > foo.regen.ldif > > 7. Check foo.current.ldif and foo.regen.ldif, there should be no difference. > > https://fedorahosted.org/freeipa/ticket/3016 Patch split into two and ACI change is merged into a single ACI for read and write. Originally Simo wanted me to have them separate but later he decided to follow my original plan. :) Since we have 3.0 beta versions in the wild which already have 'read' ACI, I'm explicitly removing the old ACI and adding a new one to help with cases of 2.x -> 3.x upgrades. -- / Alexander Bokovoy -------------- next part -------------- From 22176f6382b2a16b5d10f2a5e605246964e02a96 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 22 Aug 2012 14:24:33 +0300 Subject: [PATCH 5/5] Add ACI to allow regenerating ipaNTHash from ipasam ACI was lacking to allow actually writing MagicRegen into ipaNTHash attribute, Part 2 of https://fedorahosted.org/freeipa/ticket/3016 --- install/updates/60-trusts.update | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/install/updates/60-trusts.update b/install/updates/60-trusts.update index 0e40ca4d16133f0c1e93300fc13a08dd5ba4ddf7..cc9a771df901a90b457357c570dc06d34c0db4c8 100644 --- a/install/updates/60-trusts.update +++ b/install/updates/60-trusts.update @@ -60,7 +60,8 @@ add:aci: '(target = "ldap:///cn=trusts,$SUFFIX")(targetattr = "ipaNTTrustType || # Samba user should be able to read NT passwords to authenticate # Add ipaNTHash to global ACIs, leave DNS tree out of global allow access rule dn: $SUFFIX -add:aci: '(targetattr = "ipaNTHash")(version 3.0; acl "Samba system principals can read NT passwords"; allow (read) groupdn="ldap:///cn=adtrust agents,cn=sysaccounts,cn=etc,$SUFFIX";)' +add:aci: '(targetattr = "ipaNTHash")(version 3.0; acl "Samba system principals can read and write NT passwords"; allow (read,write) groupdn="ldap:///cn=adtrust agents,cn=sysaccounts,cn=etc,$SUFFIX";)' +remove:aci: '(targetattr = "ipaNTHash")(version 3.0; acl "Samba system principals can read NT passwords"; allow (read) groupdn="ldap:///cn=adtrust agents,cn=sysaccounts,cn=etc,$SUFFIX";)' replace:aci:'(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || userPKCS12")(version 3.0; acl "Enable Anonymous access"; allow (read, search, compare) userdn = "ldap:///anyone";)::(target != "ldap:///idnsname=*,cn=dns,$SUFFIX")(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || userPKCS12 || ipaNTHash")(version 3.0; acl "Enable Anonymous access"; allow (read, search, compare) userdn = "ldap:///anyone";)' replace:aci:'(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || krbPrincipalName || krbCanonicalName || krbUPEnabled || krbTicketPolicyReference || krbPrincipalExpiration || krbPasswordExpiration || krbPwdPolicyReference || krbPrincipalType || krbPwdHistory || krbLastPwdChange || krbPrincipalAliases || krbExtraData || krbLastSuccessfulAuth || krbLastFailedAuth || krbLoginFailedCount || krbTicketFlags || ipaUniqueId || memberOf || serverHostName || enrolledBy")(version 3.0; acl "Admin can manage any entry"; allow (all) groupdn = "ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)::(targetattr != "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || krbMKey || krbPrincipalName || krbCanonicalName || krbUPEnabled || krbTicketPolicyReference || krbPrincipalExpiration || krbPasswordExpiration || krbPwdPolicyReference || krbPrincipalType || krbPwdHistory || krbLastPwdChange || krbPrincipalAliases || krbExtraData || krbLastSuccessfulAuth || krbLastFailedAuth || krbLoginFailedCount || krbTicketFlags || ipaUniqueId || memberOf || serverHostName || enrolledBy || ipaNTHash")(version 3.0; acl "Admin can manage any entry"; allow (all) groupdn = "ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)' replace:aci:'(targetattr = "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory")(version 3.0; acl "Admins can write passwords"; allow (add,delete,write) groupdn="ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)::(targetattr = "userPassword || krbPrincipalKey || sambaLMPassword || sambaNTPassword || passwordHistory || ipaNTHash")(version 3.0; acl "Admins can write passwords"; allow (add,delete,write) groupdn="ldap:///cn=admins,cn=groups,cn=accounts,$SUFFIX";)' -- 1.7.11.4 -------------- next part -------------- >From c9f743c986e2af749d51152c0678ca77392e36b2 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 22 Aug 2012 14:19:54 +0300 Subject: [PATCH 4/5] Fix ipasam ipaNThash magic regen to actually fetch updated password With this change ipasam is able to ask for ipaNTHash generation and if corresponding Kerberos key is available, will be able to retrieve generated ipaNTHash. Part 1 of https://fedorahosted.org/freeipa/ticket/3016 --- daemons/ipa-sam/ipa_sam.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 059109374bd0e1aa1de118b4767b5692d0e483a2..8a4a08bc7a5951553a463805a8aedb82ee887936 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -2417,7 +2417,7 @@ static bool ipasam_nthash_retrieve(struct ldapsam_privates *ldap_state, }; ret = smbldap_search(smbldap_state, entry_dn, - LDAP_SCOPE_BASE, "", attr_list, 0, + LDAP_SCOPE_BASE, "(objectclass=*)", attr_list, 0, &result); if (ret != LDAP_SUCCESS) { DEBUG(1, ("Failed to get NT hash: %s\n", @@ -2453,15 +2453,13 @@ static bool ipasam_nthash_regen(struct ldapsam_privates *ldap_state, TALLOC_CTX *mem_ctx, char * entry_dn) { - LDAPMod **mods; + LDAPMod **mods = NULL; int ret; - mods = NULL; - smbldap_make_mod(ldap_state->smbldap_state->ldap_struct, - NULL, &mods, LDAP_ATTRIBUTE_NTHASH, "MagicRegen"); - + smbldap_set_mod(&mods, LDAP_MOD_ADD, LDAP_ATTRIBUTE_NTHASH, "MagicRegen"); talloc_autofree_ldapmod(mem_ctx, mods); - ret = smbldap_add(ldap_state->smbldap_state, entry_dn, mods); + + ret = smbldap_modify(ldap_state->smbldap_state, entry_dn, mods); if (ret != LDAP_SUCCESS) { DEBUG(5, ("ipasam: attempt to regen ipaNTHash failed\n")); } @@ -2585,13 +2583,11 @@ static bool init_sam_from_ldap(struct ldapsam_privates *ldap_state, * */ temp = smbldap_talloc_dn(tmp_ctx, ldap_state->smbldap_state->ldap_struct, entry); if (temp) { - retval = ipasam_nthash_regen(tmp_ctx, - ldap_state->smbldap_state->ldap_struct, - temp); + retval = ipasam_nthash_regen(ldap_state, + tmp_ctx, temp); if (retval) { - retval = ipasam_nthash_retrieve(tmp_ctx, - ldap_state->smbldap_state->ldap_struct, - temp, &nthash); + retval = ipasam_nthash_retrieve(ldap_state, + tmp_ctx, temp, &nthash); } } } -- 1.7.11.4 From sbose at redhat.com Wed Aug 22 13:15:38 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 22 Aug 2012 15:15:38 +0200 Subject: [Freeipa-devel] [PATCH] Use libsamba-security instead of libsecurity Message-ID: <20120822131538.GH2021@localhost.localdomain> Hi, this patch fixes a build issue caused by a name change of a library in samba4 beta6. Since we currently always build with the latest samba4 version I think we do not need an extra configure check here. I you think otherwise let me known, then I can add a check. bye, Sumit -------------- next part -------------- From 76e2fe12ea73832b2c121c6421d6f04b29dc7b5d Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 22 Aug 2012 15:10:31 +0200 Subject: [PATCH] Use libsamba-security instead of libsecurity In samba4-beta6 the name of a library was changed from libsecurity to libsamba-security. --- daemons/ipa-sam/Makefile.am | 2 +- 1 Datei ge?ndert, 1 Zeile hinzugef?gt(+), 1 Zeile entfernt(-) diff --git a/daemons/ipa-sam/Makefile.am b/daemons/ipa-sam/Makefile.am index 275cce629385b1719544a7832a00e9ee6664b739..ad7e516f0c94f82cc209ee55ff0b67c6a6bd54f9 100644 --- a/daemons/ipa-sam/Makefile.am +++ b/daemons/ipa-sam/Makefile.am @@ -3,7 +3,7 @@ SAMBA40EXTRA_LIBS = $(SAMBA40EXTRA_LIBPATH) \ -lsmbldap \ -lcliauth \ -lpdb \ - -lsecurity \ + -lsamba-security \ -lsmbconf \ $(NULL) -- 1.7.11.4 From atkac at redhat.com Wed Aug 22 13:35:04 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 22 Aug 2012 15:35:04 +0200 Subject: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query() In-Reply-To: <5028FE08.4060207@redhat.com> References: <5028FE08.4060207@redhat.com> Message-ID: <20120822133503.GA7769@redhat.com> On Mon, Aug 13, 2012 at 03:15:52PM +0200, Petr Spacek wrote: > Hello, > > this patch improves connection management in bind-dyndb-ldap and closes > https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . > > It should prevent all deadlocks on connection pool in future. Ack, just check my pedantic comments below, please. > From 0cd91def54ea9ac92e25ee50e54c5e55034e2c47 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 13 Aug 2012 15:06:50 +0200 > Subject: [PATCH] Avoid manual connection management outside ldap_query() > > https://fedorahosted.org/bind-dyndb-ldap/ticket/68 > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 153 +++++++++++++++++++++++++++++++----------------------- > 1 file changed, 87 insertions(+), 66 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index c34b881a8430980f41eb02d2bb0f0229421d7fa1..fdc629e1c0cd1fabde27d887e391ef81823453c1 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -289,8 +289,9 @@ static isc_result_t ldap_query_create(isc_mem_t *mctx, ldap_qresult_t **ldap_qre > static void ldap_query_free(isc_boolean_t prepare_reuse, ldap_qresult_t **ldap_qresultp); > > /* Functions for writing to LDAP. */ > -static isc_result_t ldap_modify_do(ldap_connection_t *ldap_conn, const char *dn, > - LDAPMod **mods, isc_boolean_t delete_node); > +static isc_result_t ldap_modify_do(ldap_instance_t *ldap_inst, > + ldap_connection_t *ldap_conn, const char *dn, LDAPMod **mods, > + isc_boolean_t delete_node); > static isc_result_t ldap_rdttl_to_ldapmod(isc_mem_t *mctx, > dns_rdatalist_t *rdlist, LDAPMod **changep); > static isc_result_t ldap_rdatalist_to_ldapmod(isc_mem_t *mctx, > @@ -1476,7 +1477,6 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > dns_name_t *origin, ldapdb_nodelist_t *nodelist) > { > isc_result_t result; > - ldap_connection_t *ldap_conn = NULL; > ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > @@ -1488,13 +1488,11 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > REQUIRE(nodelist != NULL); > > /* RRs aren't in the cache, perform ordinary LDAP query */ > - CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - > INIT_LIST(*nodelist); > CHECK(str_new(mctx, &string)); > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > - CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, str_buf(string), > + CHECK(ldap_query(ldap_inst, NULL, &ldap_qresult, str_buf(string), > LDAP_SCOPE_SUBTREE, NULL, 0, "(objectClass=idnsRecord)")); > > if (EMPTY(ldap_qresult->ldap_entries)) { > @@ -1536,7 +1534,6 @@ ldapdb_nodelist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *nam > > cleanup: > ldap_query_free(ISC_FALSE, &ldap_qresult); > - ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > return result; > @@ -1547,7 +1544,6 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > dns_name_t *origin, ldapdb_rdatalist_t *rdatalist) > { > isc_result_t result; > - ldap_connection_t *ldap_conn = NULL; > ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > ld_string_t *string = NULL; > @@ -1570,8 +1566,7 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > CHECK(str_new(mctx, &string)); > CHECK(dnsname_to_dn(ldap_inst->zone_register, name, string)); > > - CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > - CHECK(ldap_query(ldap_inst, ldap_conn, &ldap_qresult, str_buf(string), > + CHECK(ldap_query(ldap_inst, NULL, &ldap_qresult, str_buf(string), > LDAP_SCOPE_BASE, NULL, 0, "(objectClass=idnsRecord)")); > > if (EMPTY(ldap_qresult->ldap_entries)) { > @@ -1598,7 +1593,6 @@ ldapdb_rdatalist_get(isc_mem_t *mctx, ldap_instance_t *ldap_inst, dns_name_t *na > > cleanup: > ldap_query_free(ISC_FALSE, &ldap_qresult); > - ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > str_destroy(&string); > > if (result != ISC_R_SUCCESS) > @@ -1713,11 +1707,15 @@ ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > int ret; > int ldap_err_code; > int once = 0; > + isc_boolean_t autoconn = (ldap_conn == NULL); > > - REQUIRE(ldap_conn != NULL); > + REQUIRE(ldap_inst != NULL); > + REQUIRE(base != NULL); > REQUIRE(ldap_qresultp != NULL && *ldap_qresultp == NULL); > > CHECK(ldap_query_create(ldap_inst->mctx, &ldap_qresult)); > + if (autoconn) > + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > > va_start(ap, filter); > str_vsprintf(ldap_qresult->query_string, filter, ap); > @@ -1751,29 +1749,34 @@ retry: > &ldap_qresult->ldap_entries); > if (result != ISC_R_SUCCESS) { > log_error("failed to save LDAP query results"); > - goto cleanup; I would recommend to leave this goto here. In future, someone might add code before "cleanup:" label and can forget to add "goto cleanup". > } > + /* LDAP call suceeded, errors from ldap_entrylist_create() will be > + * handled in cleanup section */ > > + } else { /* LDAP error - continue with error handler */ > + result = ISC_R_FAILURE; > + ret = ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > + (void *)&ldap_err_code); > + if (ret == LDAP_OPT_SUCCESS && ldap_err_code == LDAP_NO_SUCH_OBJECT) { > + result = ISC_R_NOTFOUND; > + } else if (!once) { > + /* some error happened during ldap_search, try to recover */ > + once++; > + result = handle_connection_error(ldap_inst, ldap_conn, > + ISC_FALSE); > + if (result == ISC_R_SUCCESS) > + goto retry; > + } > + } > + > +cleanup: > + if (autoconn) > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > + if (result != ISC_R_SUCCESS) { > + ldap_query_free(ISC_FALSE, &ldap_qresult); > + } else { > *ldap_qresultp = ldap_qresult; > - return ISC_R_SUCCESS; > - } else { > - result = ISC_R_FAILURE; > } > - > - ret = ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > - (void *)&ldap_err_code); > - if (ret == LDAP_OPT_SUCCESS && ldap_err_code == LDAP_NO_SUCH_OBJECT) { > - result = ISC_R_NOTFOUND; > - } else if (!once) { > - /* some error happened during ldap_search, try to recover */ > - once++; > - result = handle_connection_error(ldap_inst, ldap_conn, > - ISC_FALSE); > - if (result == ISC_R_SUCCESS) > - goto retry; > - } > -cleanup: > - ldap_query_free(ISC_FALSE, &ldap_qresult); > return result; > } > > @@ -2111,35 +2114,56 @@ reconnect: > return ISC_R_FAILURE; > } > > -/* FIXME: Handle the case where the LDAP handle is NULL -> try to reconnect. */ > static isc_result_t > -ldap_modify_do(ldap_connection_t *ldap_conn, const char *dn, LDAPMod **mods, > - isc_boolean_t delete_node) > +ldap_modify_do(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > + const char *dn, LDAPMod **mods, isc_boolean_t delete_node) > { > int ret; > int err_code; > const char *operation_str; > + isc_result_t result; > + isc_boolean_t autoconn = (ldap_conn == NULL); > > - REQUIRE(ldap_conn != NULL); > REQUIRE(dn != NULL); > REQUIRE(mods != NULL); > + REQUIRE(ldap_inst != NULL); > + > + if (autoconn) > + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); > + > + if (ldap_conn->handle == NULL) { > + /* > + * handle can be NULL when the first connection to LDAP wasn't > + * successful > + * TODO: handle this case inside ldap_pool_getconnection()? > + */ > + CHECK(ldap_connect(ldap_inst, ldap_conn, ISC_FALSE)); > + } > > if (delete_node) { > log_debug(2, "deleting whole node: '%s'", dn); > ret = ldap_delete_ext_s(ldap_conn->handle, dn, NULL, NULL); > } else { > log_debug(2, "writing to '%s'", dn); > ret = ldap_modify_ext_s(ldap_conn->handle, dn, mods, NULL, NULL); > } > > + result = (ret == LDAP_SUCCESS) ? ISC_R_SUCCESS : ISC_R_FAILURE; > if (ret == LDAP_SUCCESS) > - return ISC_R_SUCCESS; > + goto cleanup; > > - ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, &err_code); > if (mods[0]->mod_op == LDAP_MOD_ADD) > operation_str = "modifying(add)"; > - else > + else if (mods[0]->mod_op == LDAP_MOD_DELETE) > operation_str = "modifying(del)"; > + else { > + operation_str = "modifying(unknown operation)"; > + CHECK(ISC_R_NOTIMPLEMENTED); > + } > + > + LDAP_OPT_CHECK(ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > + &err_code), "ldap_modify_do(%s) failed to obtain ldap error code", > + operation_str); > > /* If there is no object yet, create it with an ldap add operation. */ > if (mods[0]->mod_op == LDAP_MOD_ADD && err_code == LDAP_NO_SUCH_OBJECT) { > @@ -2164,10 +2188,12 @@ ldap_modify_do(ldap_connection_t *ldap_conn, const char *dn, LDAPMod **mods, > new_mods[i + 1] = NULL; > > ret = ldap_add_ext_s(ldap_conn->handle, dn, new_mods, NULL, NULL); > + result = (ret == LDAP_SUCCESS) ? ISC_R_SUCCESS : ISC_R_FAILURE; > if (ret == LDAP_SUCCESS) > - return ISC_R_SUCCESS; > - ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > - &err_code); > + goto cleanup; > + LDAP_OPT_CHECK(ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > + &err_code), > + "ldap_modify_do(add) failed to obtain ldap error code"); > operation_str = "adding"; > } > > @@ -2178,11 +2204,13 @@ ldap_modify_do(ldap_connection_t *ldap_conn, const char *dn, LDAPMod **mods, > * unexisting attribute */ > if (mods[0]->mod_op != LDAP_MOD_DELETE || > err_code != LDAP_NO_SUCH_ATTRIBUTE) { > - > - return ISC_R_FAILURE; > + result = ISC_R_FAILURE; > } > +cleanup: > + if (autoconn) > + ldap_pool_putconnection(ldap_inst->pool, &ldap_conn); > > - return ISC_R_SUCCESS; > + return result; > } > > static isc_result_t > @@ -2348,18 +2376,21 @@ cleanup: > * refresh, retry, expire and minimum attributes for each SOA record. > */ > static isc_result_t > -modify_soa_record(ldap_connection_t *ldap_conn, const char *zone_dn, > - dns_rdata_t *rdata) > +modify_soa_record(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > + const char *zone_dn, dns_rdata_t *rdata) > { > isc_result_t result; > - isc_mem_t *mctx = ldap_conn->mctx; > dns_rdata_soa_t soa; > LDAPMod change[5]; > LDAPMod *changep[6] = { > &change[0], &change[1], &change[2], &change[3], &change[4], > NULL > }; > > + REQUIRE(zone_dn != NULL); > + REQUIRE(rdata != NULL); > + REQUIRE(ldap_inst != NULL); > + All of those REQUIREs are redundant. Functions which use those paramaters check for their validity. > /* all values in SOA record are isc_uint32_t, i.e. max. 2^32-1 */ > #define MAX_SOANUM_LENGTH (10 + 1) > #define SET_LDAP_MOD(index, name) \ > @@ -2371,17 +2402,17 @@ modify_soa_record(ldap_connection_t *ldap_conn, const char *zone_dn, > CHECK(isc_string_printf(change[index].mod_values[0], \ > MAX_SOANUM_LENGTH, "%u", soa.name)); > > - dns_rdata_tostruct(rdata, (void *)&soa, mctx); > + dns_rdata_tostruct(rdata, (void *)&soa, ldap_inst->mctx); > > SET_LDAP_MOD(0, serial); > SET_LDAP_MOD(1, refresh); > SET_LDAP_MOD(2, retry); > SET_LDAP_MOD(3, expire); > SET_LDAP_MOD(4, minimum); > > dns_rdata_freestruct((void *)&soa); > > - result = ldap_modify_do(ldap_conn, zone_dn, changep, ISC_FALSE); > + result = ldap_modify_do(ldap_inst, ldap_conn, zone_dn, changep, ISC_FALSE); > > cleanup: > return result; > @@ -2457,7 +2488,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > CHECK(discard_from_cache(cache, owner)); > > if (rdlist->type == dns_rdatatype_soa) { > - result = modify_soa_record(ldap_conn, str_buf(owner_dn), > + result = modify_soa_record(ldap_inst, ldap_conn, str_buf(owner_dn), > HEAD(rdlist->rdata)); > goto cleanup; > } > @@ -2468,7 +2499,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > CHECK(ldap_rdttl_to_ldapmod(mctx, rdlist, &change[1])); > } > > - CHECK(ldap_modify_do(ldap_conn, str_buf(owner_dn), change, delete_node)); > + CHECK(ldap_modify_do(ldap_inst, ldap_conn, str_buf(owner_dn), change, delete_node)); > > /* Keep the PTR of corresponding A/AAAA record synchronized. */ > if (rdlist->type == dns_rdatatype_a || rdlist->type == dns_rdatatype_aaaa) { > @@ -2648,7 +2679,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst, > change_ptr = NULL; > > /* Modify PTR record. */ > - CHECK(ldap_modify_do(ldap_conn, str_buf(owner_dn_ptr), change, delete_node)); > + CHECK(ldap_modify_do(ldap_inst, ldap_conn, str_buf(owner_dn_ptr), change, delete_node)); > (void) discard_from_cache(ldap_instance_getcache(ldap_inst), > dns_fixedname_name(&name)); > } > @@ -2947,7 +2978,6 @@ static isc_result_t > soa_serial_increment(isc_mem_t *mctx, ldap_instance_t *inst, > dns_name_t *zone_name) { > isc_result_t result = ISC_R_FAILURE; > - ldap_connection_t * conn = NULL; > ld_string_t *zone_dn = NULL; > ldapdb_rdatalist_t rdatalist; > dns_rdatalist_t *rdlist = NULL; > @@ -2986,8 +3016,7 @@ soa_serial_increment(isc_mem_t *mctx, ldap_instance_t *inst, > dns_soa_setserial(new_serial, soa_rdata); > > /* write the new serial back to DB */ > - CHECK(ldap_pool_getconnection(inst->pool, &conn)); > - CHECK(modify_soa_record(conn, str_buf(zone_dn), soa_rdata)); > + CHECK(modify_soa_record(inst, NULL, str_buf(zone_dn), soa_rdata)); > CHECK(discard_from_cache(ldap_instance_getcache(inst), zone_name)); > > /* put the new SOA to inst->cache and compare old and new serials */ > @@ -2999,7 +3028,6 @@ cleanup: > log_error("SOA serial number incrementation failed in zone '%s'", > str_buf(zone_dn)); > > - ldap_pool_putconnection(inst->pool, &conn); > str_destroy(&zone_dn); > ldapdb_rdatalist_destroy(mctx, &rdatalist); > return result; > @@ -3019,7 +3047,6 @@ update_zone(isc_task_t *task, isc_event_t *event) > ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; > isc_result_t result ; > ldap_instance_t *inst = NULL; > - ldap_connection_t *conn = NULL; > ldap_qresult_t *ldap_qresult_zone = NULL; > ldap_qresult_t *ldap_qresult_record = NULL; > ldap_entry_t *entry_zone = NULL; > @@ -3041,8 +3068,7 @@ update_zone(isc_task_t *task, isc_event_t *event) > dns_name_init(&prevname, NULL); > > CHECK(manager_get_ldap_instance(pevent->dbname, &inst)); > - CHECK(ldap_pool_getconnection(inst->pool, &conn)); > - CHECK(ldap_query(inst, conn, &ldap_qresult_zone, pevent->dn, > + CHECK(ldap_query(inst, NULL, &ldap_qresult_zone, pevent->dn, > LDAP_SCOPE_BASE, attrs_zone, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > @@ -3062,7 +3088,7 @@ update_zone(isc_task_t *task, isc_event_t *event) > } > > /* fill the cache with records from renamed zone */ > - CHECK(ldap_query(inst, conn, &ldap_qresult_record, pevent->dn, > + CHECK(ldap_query(inst, NULL, &ldap_qresult_record, pevent->dn, > LDAP_SCOPE_ONELEVEL, attrs_record, 0, > "(objectClass=idnsRecord)")); > > @@ -3090,7 +3116,6 @@ cleanup: > > ldap_query_free(ISC_FALSE, &ldap_qresult_zone); > ldap_query_free(ISC_FALSE, &ldap_qresult_record); > - ldap_pool_putconnection(inst->pool, &conn); > if (dns_name_dynamic(&prevname)) > dns_name_free(&prevname, inst->mctx); > isc_mem_free(mctx, pevent->dbname); > @@ -3107,7 +3132,6 @@ update_config(isc_task_t *task, isc_event_t *event) > ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; > isc_result_t result ; > ldap_instance_t *inst = NULL; > - ldap_connection_t *conn = NULL; > ldap_qresult_t *ldap_qresult = NULL; > ldap_entry_t *entry; > isc_mem_t *mctx; > @@ -3124,9 +3148,7 @@ update_config(isc_task_t *task, isc_event_t *event) > if (result != ISC_R_SUCCESS) > goto cleanup; > > - CHECK(ldap_pool_getconnection(inst->pool, &conn)); > - > - CHECK(ldap_query(inst, conn, &ldap_qresult, pevent->dn, > + CHECK(ldap_query(inst, NULL, &ldap_qresult, pevent->dn, > LDAP_SCOPE_BASE, attrs, 0, > "(objectClass=idnsConfigObject)")); > > @@ -3149,7 +3171,6 @@ cleanup: > pevent->dn); > > ldap_query_free(ISC_FALSE, &ldap_qresult); > - ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Wed Aug 22 13:37:43 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 22 Aug 2012 15:37:43 +0200 Subject: [Freeipa-devel] [PATCH 0048] Lower minimal connection count for persistent search. In-Reply-To: <502901C0.9070905@redhat.com> References: <502901C0.9070905@redhat.com> Message-ID: <20120822133742.GA8074@redhat.com> On Mon, Aug 13, 2012 at 03:31:44PM +0200, Petr Spacek wrote: > Hello, > > As result of better connection management it is possible to run > persistent search with smaller connection pool. > > Attached patch updates built-in configuration checks with new lower > bound. It closes patch series for > https://fedorahosted.org/bind-dyndb-ldap/ticket/68: > Avoid manual connection management outside ldap_query() Ack > From 786531c5806331dc0486ed1d877563b00d9219da Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 13 Aug 2012 15:24:48 +0200 > Subject: [PATCH] Lower minimal connection count for persistent search. > > As result of better connection management is possible > to run persistent search with smaller connection pool. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 14 ++++---------- > 1 file changed, 4 insertions(+), 10 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index fdc629e1c0cd1fabde27d887e391ef81823453c1..01e0a165905163fc955362548b99c0f3501fb37e 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -479,25 +479,19 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > > ldap_inst->task = task; > > - if (ldap_inst->psearch && ldap_inst->connections < 3) { > - /* watcher needs one and update_action() can acquire two */ > - log_debug(1, "psearch needs at least 3 connections, " > + if (ldap_inst->psearch && ldap_inst->connections < 2) { > + /* watcher needs one and update_*() will request second */ > + log_error("psearch needs at least 2 connections, " > "increasing limit"); > - ldap_inst->connections = 3; > + ldap_inst->connections = 2; > } > if (ldap_inst->serial_autoincrement == ISC_TRUE > && ldap_inst->psearch != ISC_TRUE) { > log_error("SOA serial number auto-increment feature requires " > "persistent search"); > result = ISC_R_FAILURE; > goto cleanup; > } > - if (ldap_inst->serial_autoincrement == ISC_TRUE > - && ldap_inst->connections < 4) { > - log_error("serial_autoincrement needs at least 4 connections, " > - "increasing limit"); > - ldap_inst->connections = 4; > - } > > CHECK(new_ldap_cache(mctx, argv, &ldap_inst->cache, ldap_inst->psearch)); > CHECK(ldap_pool_create(mctx, ldap_inst->connections, &ldap_inst->pool)); > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Wed Aug 22 13:43:31 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 22 Aug 2012 15:43:31 +0200 Subject: [Freeipa-devel] [PATCH 0049] Fix two memory leaks in persistent search In-Reply-To: <502A4577.80702@redhat.com> References: <502A4577.80702@redhat.com> Message-ID: <20120822134330.GA8117@redhat.com> On Tue, Aug 14, 2012 at 02:32:55PM +0200, Petr Spacek wrote: > Hello, > > This patch fixes two memory leaks in persistent search. Ack. > From 892f1d5c59a97cdad7a2807ecd172488605ab181 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Tue, 14 Aug 2012 12:38:43 +0200 > Subject: [PATCH] Fix two memory leaks in persistent search. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 45 ++++++++++++--------------------------------- > 1 file changed, 12 insertions(+), 33 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index cc7003a6cdcd2d27404fec936623ed8a3e8fa7f8..6f9711c55abc008a1abfd4b36382bdf697cd2bd8 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -313,8 +313,7 @@ static isc_result_t ldap_pool_connect(ldap_pool_t *pool, > ldap_instance_t *ldap_inst); > > /* Functions for manipulating LDAP persistent search control */ > -static isc_result_t ldap_pscontrol_create(isc_mem_t *mctx, LDAPControl **ctrlp); > -static void ldap_pscontrol_destroy(isc_mem_t *mctx, LDAPControl **ctrlp); > +static isc_result_t ldap_pscontrol_create(LDAPControl **ctrlp); > > static isc_threadresult_t ldap_psearch_watcher(isc_threadarg_t arg); > static void psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, > @@ -639,8 +638,7 @@ new_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) > > CHECK(isc_lex_create(ldap_conn->mctx, TOKENSIZ, &ldap_conn->lex)); > CHECKED_MEM_GET(ldap_conn->mctx, ldap_conn->rdata_target_mem, MINTSIZ); > - CHECK(ldap_pscontrol_create(ldap_conn->mctx, > - &ldap_conn->serverctrls[0])); > + CHECK(ldap_pscontrol_create(&ldap_conn->serverctrls[0])); > > *ldap_connp = ldap_conn; > > @@ -674,8 +672,7 @@ destroy_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) > ldap_conn->rdata_target_mem, MINTSIZ); > } > if (ldap_conn->serverctrls[0] != NULL) { > - ldap_pscontrol_destroy(ldap_conn->mctx, > - &ldap_conn->serverctrls[0]); > + ldap_control_free(ldap_conn->serverctrls[0]); > } > > isc_mem_detach(&ldap_conn->mctx); > @@ -2819,10 +2816,10 @@ cleanup: > * http://tools.ietf.org/id/draft-ietf-ldapext-psearch-03.txt) control. > */ > static isc_result_t > -ldap_pscontrol_create(isc_mem_t *mctx, LDAPControl **ctrlp) > +ldap_pscontrol_create(LDAPControl **ctrlp) > { > BerElement *ber; > - LDAPControl *ctrl = NULL; > + struct berval *berval; > isc_result_t result = ISC_R_FAILURE; > > REQUIRE(ctrlp != NULL && *ctrlp == NULL); > @@ -2841,45 +2838,25 @@ ldap_pscontrol_create(isc_mem_t *mctx, LDAPControl **ctrlp) > if (ber_printf(ber, "{ibb}", LDAP_ENTRYCHANGE_ALL, 0, 1) == -1) > goto cleanup; > > - CHECKED_MEM_GET(mctx, ctrl, sizeof(*ctrl)); > - ZERO_PTR(ctrl); > - ctrl->ldctl_iscritical = 1; > - ctrl->ldctl_oid = strdup(LDAP_CONTROL_PERSISTENTSEARCH); > - if (ctrl->ldctl_oid == NULL) > + if (ber_flatten(ber, &berval) < 0) > goto cleanup; > > - if (ber_flatten2(ber, &ctrl->ldctl_value, 1) < 0) > + if (ldap_control_create(LDAP_CONTROL_PERSISTENTSEARCH, 1, berval, 1, ctrlp) > + != LDAP_SUCCESS) > goto cleanup; > > ber_free(ber, 1); > - *ctrlp = ctrl; > + ber_bvfree(berval); > > return ISC_R_SUCCESS; > > cleanup: > ber_free(ber, 1); > - ldap_pscontrol_destroy(mctx, &ctrl); > + ber_bvfree(berval); > > return result; > } > > -static void > -ldap_pscontrol_destroy(isc_mem_t *mctx, LDAPControl **ctrlp) > -{ > - LDAPControl *ctrl; > - > - REQUIRE(ctrlp != NULL); > - > - if (*ctrlp == NULL) > - return; > - > - ctrl = *ctrlp; > - if (ctrl->ldctl_oid != NULL) > - free(ctrl->ldctl_oid); > - SAFE_MEM_PUT(mctx, ctrl, sizeof(*ctrl)); > - *ctrlp = NULL; > -} > - > static inline isc_result_t > ldap_get_zone_serial(ldap_instance_t *inst, dns_name_t *zone_name, > isc_uint32_t *serial) { > @@ -3357,6 +3334,8 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > isc_task_send(inst->task, (isc_event_t **)&pevent); > > cleanup: > + if (ctrls != NULL) > + ldap_controls_free(ctrls); > if (result != ISC_R_SUCCESS) { > if (dbname != NULL) > isc_mem_free(mctx, dbname); > -- > 1.7.11.2 > -- Adam Tkac, Red Hat, Inc. From abokovoy at redhat.com Wed Aug 22 14:04:12 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 17:04:12 +0300 Subject: [Freeipa-devel] [PATCH] Use libsamba-security instead of libsecurity In-Reply-To: <20120822131538.GH2021@localhost.localdomain> References: <20120822131538.GH2021@localhost.localdomain> Message-ID: <20120822140412.GK4312@redhat.com> On Wed, 22 Aug 2012, Sumit Bose wrote: >Hi, > >this patch fixes a build issue caused by a name change of a library in >samba4 beta6. Since we currently always build with the latest samba4 >version I think we do not need an extra configure check here. I you >think otherwise let me known, then I can add a check. I think you'd need to modify freeipa.spec.in as we did with the previous 'upgrade'. BuildRequires: samba4-devel >= 4.0.0-134 -- / Alexander Bokovoy From sbose at redhat.com Wed Aug 22 14:10:38 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 22 Aug 2012 16:10:38 +0200 Subject: [Freeipa-devel] [PATCH] Use libsamba-security instead of libsecurity In-Reply-To: <20120822140412.GK4312@redhat.com> References: <20120822131538.GH2021@localhost.localdomain> <20120822140412.GK4312@redhat.com> Message-ID: <20120822141038.GJ2021@localhost.localdomain> On Wed, Aug 22, 2012 at 05:04:12PM +0300, Alexander Bokovoy wrote: > On Wed, 22 Aug 2012, Sumit Bose wrote: > >Hi, > > > >this patch fixes a build issue caused by a name change of a library in > >samba4 beta6. Since we currently always build with the latest samba4 > >version I think we do not need an extra configure check here. I you > >think otherwise let me known, then I can add a check. > I think you'd need to modify freeipa.spec.in as we did with the previous > 'upgrade'. > > BuildRequires: samba4-devel >= 4.0.0-134 ok, new version attached. bye, Sumit > > -- > / Alexander Bokovoy -------------- next part -------------- From cd711dd5319c0be37aa542d5fbfb0da3482267c0 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 22 Aug 2012 15:10:31 +0200 Subject: [PATCH] Use libsamba-security instead of libsecurity In samba4-beta6 the name of a library was changed from libsecurity to libsamba-security. --- daemons/ipa-sam/Makefile.am | 2 +- freeipa.spec.in | 2 +- 2 Dateien ge?ndert, 2 Zeilen hinzugef?gt(+), 2 Zeilen entfernt(-) diff --git a/daemons/ipa-sam/Makefile.am b/daemons/ipa-sam/Makefile.am index 275cce629385b1719544a7832a00e9ee6664b739..ad7e516f0c94f82cc209ee55ff0b67c6a6bd54f9 100644 --- a/daemons/ipa-sam/Makefile.am +++ b/daemons/ipa-sam/Makefile.am @@ -3,7 +3,7 @@ SAMBA40EXTRA_LIBS = $(SAMBA40EXTRA_LIBPATH) \ -lsmbldap \ -lcliauth \ -lpdb \ - -lsecurity \ + -lsamba-security \ -lsmbconf \ $(NULL) diff --git a/freeipa.spec.in b/freeipa.spec.in index f582f22488a195fd29a1900d6df156d8b56a259f..9e25d5ace0de7bf21e39ce534abd97d5a60d94b4 100644 --- a/freeipa.spec.in +++ b/freeipa.spec.in @@ -31,7 +31,7 @@ BuildRequires: policycoreutils >= %{POLICYCOREUTILSVER} %if 0%{?fedora} >= 16 BuildRequires: systemd-units %endif -BuildRequires: samba4-devel >= 4.0.0-128 +BuildRequires: samba4-devel >= 4.0.0-134 BuildRequires: samba4-python BuildRequires: libtalloc-devel BuildRequires: libtevent-devel -- 1.7.11.4 From ssorce at redhat.com Wed Aug 22 14:16:15 2012 From: ssorce at redhat.com (Simo Sorce) Date: Wed, 22 Aug 2012 10:16:15 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] 0072 Add ACI to allow magic regen of ipaNThash In-Reply-To: <20120822123125.GI4312@redhat.com> Message-ID: <813884947.9122678.1345644975573.JavaMail.root@redhat.com> ----- Original Message ----- > On Tue, 21 Aug 2012, Alexander Bokovoy wrote: > Patch split into two and ACI change is merged into a single ACI for > read > and write. Originally Simo wanted me to have them separate but later > he > decided to follow my original plan. :) > > Since we have 3.0 beta versions in the wild which already have 'read' > ACI, I'm explicitly removing the old ACI and adding a new one to help > with cases of 2.x -> 3.x upgrades. ACK Simo. From abokovoy at redhat.com Wed Aug 22 14:16:47 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 17:16:47 +0300 Subject: [Freeipa-devel] [PATCH] Use libsamba-security instead of libsecurity In-Reply-To: <20120822141038.GJ2021@localhost.localdomain> References: <20120822131538.GH2021@localhost.localdomain> <20120822140412.GK4312@redhat.com> <20120822141038.GJ2021@localhost.localdomain> Message-ID: <20120822141647.GL4312@redhat.com> On Wed, 22 Aug 2012, Sumit Bose wrote: >On Wed, Aug 22, 2012 at 05:04:12PM +0300, Alexander Bokovoy wrote: >> On Wed, 22 Aug 2012, Sumit Bose wrote: >> >Hi, >> > >> >this patch fixes a build issue caused by a name change of a library in >> >samba4 beta6. Since we currently always build with the latest samba4 >> >version I think we do not need an extra configure check here. I you >> >think otherwise let me known, then I can add a check. >> I think you'd need to modify freeipa.spec.in as we did with the previous >> 'upgrade'. >> >> BuildRequires: samba4-devel >= 4.0.0-134 > >ok, new version attached. ACK now. -- / Alexander Bokovoy From abokovoy at redhat.com Wed Aug 22 14:23:59 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 17:23:59 +0300 Subject: [Freeipa-devel] [PATCH 79] Ticket #3008: DN objects hash differently depending on case In-Reply-To: <20120820113020.GA4312@redhat.com> References: <502EA8AF.5000305@redhat.com> <20120820113020.GA4312@redhat.com> Message-ID: <20120822142359.GM4312@redhat.com> On Mon, 20 Aug 2012, Alexander Bokovoy wrote: >On Fri, 17 Aug 2012, John Dennis wrote: >>Because the attrs & values in DN's, RDN's and AVA's are comparison case- >>insensitive the hash value between two objects which compare as equal but >>differ in case must also yield the same hash value. This is critical when >>these objects are used as a dict key or in a set because dicts and sets >>use the object's __hash__ value in conjunction with the objects __eq__ >>method to lookup the object. >> >>The defect is the DN, RDN & AVA objects computed their hash from the case- >>preserving string representation thus two otherwise equal objects >>incorrectly yielded different hash values. >> >>The problem manifests itself when one of these objects is used as a key in >>a dict, for example a dn. >> >>dn1 = DN(('cn', 'Bob')) >>dn2 = DN(('cn', 'bob')) >> >>dn1 == dn2 --> True >> >>hash(dn1) == hash(dn2) --> False >> >>d = {} >> >>d[dn1] = x >>d[dn2] = y >> >>len(d) --> 2 >> >>The patch fixes the above by lower casing the string representation of >>the object prior to computing it's hash. >> >>The patch also corrects a spelling mistake and a bogus return value in >>ldapupdate.py which happened to be discovered while researching this >>bug. >ACK. > >With this patch I am no longer getting constraint violation errors. Pushed to master and 3.0 -- / Alexander Bokovoy From abokovoy at redhat.com Wed Aug 22 14:24:28 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 17:24:28 +0300 Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on samba4-windbind. In-Reply-To: <50337DEB.8060706@redhat.com> References: <648385417.2145989.1345450574787.JavaMail.root@redhat.com> <5032246B.1080800@redhat.com> <20120820120550.GD4312@redhat.com> <50337DEB.8060706@redhat.com> Message-ID: <20120822142428.GN4312@redhat.com> On Tue, 21 Aug 2012, Tomas Babej wrote: >On 08/20/2012 02:05 PM, Alexander Bokovoy wrote: >>On Mon, 20 Aug 2012, Petr Vobornik wrote: >>>On 08/20/2012 10:16 AM, Tomas Babej wrote: >>>>Fixed typo in the commit message. >>>> >>>>Tomas >>>> >>>>----- Original Message ----- >>>>From: "Tomas Babej" >>>>To: freeipa-devel at redhat.com >>>>Sent: Monday, August 20, 2012 10:08:41 AM >>>>Subject: [Freeipa-devel] [PATCH] 0007 Adds dependency on >>>>samba4-windbind. >>>> >>>>Hi, >>>> >>>>Dependency on samba4-winbind has been added to the package >>>>freeipa-server-trust-ad. >>>> >>>>Tomas >>> >>>>+* Mon Aug 20 2012 Tomas Babej - 3.0.0-1 >>>>+- Add samba4-winbind to build dependencies for AD server-side code >>>>+ >>> >>>Maybe a nitpick: >>> >>>Shouldn't the version be 2.99.0-42? >>Yes, good catch. We haven't released 3.0 yet, so 2.99.0 is our current >>aim. >> >> >Sorry, I got confused by the release of the 3.0 beta 2 version. Pushed to master and 3.0 -- / Alexander Bokovoy From abokovoy at redhat.com Wed Aug 22 14:25:11 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 17:25:11 +0300 Subject: [Freeipa-devel] [PATCH] 0072 Add ACI to allow magic regen of ipaNThash In-Reply-To: <813884947.9122678.1345644975573.JavaMail.root@redhat.com> References: <20120822123125.GI4312@redhat.com> <813884947.9122678.1345644975573.JavaMail.root@redhat.com> Message-ID: <20120822142511.GO4312@redhat.com> On Wed, 22 Aug 2012, Simo Sorce wrote: >----- Original Message ----- >> On Tue, 21 Aug 2012, Alexander Bokovoy wrote: >> Patch split into two and ACI change is merged into a single ACI for >> read >> and write. Originally Simo wanted me to have them separate but later >> he >> decided to follow my original plan. :) >> >> Since we have 3.0 beta versions in the wild which already have 'read' >> ACI, I'm explicitly removing the old ACI and adding a new one to help >> with cases of 2.x -> 3.x upgrades. > >ACK Pushed to master and 3.0 -- / Alexander Bokovoy From abokovoy at redhat.com Wed Aug 22 14:25:39 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 22 Aug 2012 17:25:39 +0300 Subject: [Freeipa-devel] [PATCH] Use libsamba-security instead of libsecurity In-Reply-To: <20120822141647.GL4312@redhat.com> References: <20120822131538.GH2021@localhost.localdomain> <20120822140412.GK4312@redhat.com> <20120822141038.GJ2021@localhost.localdomain> <20120822141647.GL4312@redhat.com> Message-ID: <20120822142539.GP4312@redhat.com> On Wed, 22 Aug 2012, Alexander Bokovoy wrote: >On Wed, 22 Aug 2012, Sumit Bose wrote: >>On Wed, Aug 22, 2012 at 05:04:12PM +0300, Alexander Bokovoy wrote: >>>On Wed, 22 Aug 2012, Sumit Bose wrote: >>>>Hi, >>>> >>>>this patch fixes a build issue caused by a name change of a library in >>>>samba4 beta6. Since we currently always build with the latest samba4 >>>>version I think we do not need an extra configure check here. I you >>>>think otherwise let me known, then I can add a check. >>>I think you'd need to modify freeipa.spec.in as we did with the previous >>>'upgrade'. >>> >>>BuildRequires: samba4-devel >= 4.0.0-134 >> >>ok, new version attached. > >ACK now. Pushed to master and 3.0 -- / Alexander Bokovoy From pviktori at redhat.com Wed Aug 22 14:45:01 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 22 Aug 2012 10:45:01 -0400 Subject: [Freeipa-devel] [PATCH 80] Ticket #2850 - Ipactl exception not handled well In-Reply-To: <5032A7F1.1060505@redhat.com> References: <5032A7F1.1060505@redhat.com> Message-ID: <5034F06D.3050708@redhat.com> > From fda504233ee46a494b7ed6b85593e7e586739425 Mon Sep 17 00:00:00 2001 > From: John Dennis > Date: Mon, 20 Aug 2012 16:47:52 -0400 > Subject: [PATCH 80] Ticket #2850 - Ipactl exception not handled well > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: 8bit > > Ticket #2850 - Ipactl exception not handled well > > There were various places in ipactl which intialized IpactlError with > None as the msg. If you called str() on that exception all was well > because ScriptError.__str__() converted a msg with None to the empty > string (IpactlError is subclassed from ScriptError). But a few places > directly access e.msg which will be None if initialized that way. It's > hard to tell from the stack traces but I'm pretty sure it's those > places which use e.msg directly which will cause the problems seen in > the bug report. > > I do not believe it is ever correct to initialize an exception message > to None, I don't even understand what that means. On the other hand > initializing to the empty string is sensible and for that matter is > the default for the class. > > This patch makes two fixes: > > 1) The ScriptError initializer will now convert a msg parameter of > None to the empty string. > > 2) All places that initialized IpactlError's msg parameter to None > removed the None initializer allowing the msg parameter to default > to the empty string. > > I don't know how to test the fix for Ticket #2850 because it's not > clear how it got into that state in the first place, but I do believe > initialing the msg value to None is clearly wrong and should fix the > problem. > --- ACK -- Petr? From tbabej at redhat.com Wed Aug 22 14:47:41 2012 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 22 Aug 2012 16:47:41 +0200 Subject: [Freeipa-devel] [PATCH] 0008 Fixes different behaviour of permission-mod and show. Message-ID: <5034F10D.7050901@redhat.com> Hi, Both commands now produce the same output regarding the attributelevelrights. https://fedorahosted.org/freeipa/ticket/2875 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0008-Fixes-different-behaviour-of-permission-mod-and-show.patch Type: text/x-patch Size: 1228 bytes Desc: not available URL: From rcritten at redhat.com Wed Aug 22 15:15:20 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 22 Aug 2012 11:15:20 -0400 Subject: [Freeipa-devel] [PATCH] 0008 Fixes different behaviour of permission-mod and show. In-Reply-To: <5034F10D.7050901@redhat.com> References: <5034F10D.7050901@redhat.com> Message-ID: <5034F788.30507@redhat.com> Tomas Babej wrote: > Hi, > > Both commands now produce the same output regarding > the attributelevelrights. > > https://fedorahosted.org/freeipa/ticket/2875 I think some unit tests would be helpful so we don't regress and we know which other commands this fixes. rob From tbabej at redhat.com Wed Aug 22 17:39:32 2012 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 22 Aug 2012 19:39:32 +0200 Subject: [Freeipa-devel] [PATCH] 0008 Fixes different behaviour of permission-mod and show. In-Reply-To: <5034F788.30507@redhat.com> References: <5034F10D.7050901@redhat.com> <5034F788.30507@redhat.com> Message-ID: <50351954.5070102@redhat.com> On 08/22/2012 05:15 PM, Rob Crittenden wrote: > Tomas Babej wrote: >> Hi, >> >> Both commands now produce the same output regarding >> the attributelevelrights. >> >> https://fedorahosted.org/freeipa/ticket/2875 > > I think some unit tests would be helpful so we don't regress and we > know which other commands this fixes. > > rob I ran the tests for the permission plugin (test_permission_plugin.py) and all of them passed. Tested on clean VM with newly built IPA from the master, so there should be no regression. Results themselves attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: test-result.log Type: text/x-log Size: 5353 bytes Desc: not available URL: From sbose at redhat.com Wed Aug 22 19:15:47 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 22 Aug 2012 21:15:47 +0200 Subject: [Freeipa-devel] [PATCH] 0070 Ask for admin password in ipa-adtrust-install In-Reply-To: <20120817150451.GC513@redhat.com> References: <20120817150451.GC513@redhat.com> Message-ID: <20120822191547.GM2021@localhost.localdomain> On Fri, Aug 17, 2012 at 06:04:51PM +0300, Alexander Bokovoy wrote: > Hi, > > The credentials of the admin user will be used to obtain Kerberos ticket > before configuring cross-realm trusts support and afterwards, to > ensure that the ticket contains MS-PAC information required to actually > add a trust with Active Directory domain via 'ipa trust-add --type=ad' > command. > > We discussed few other approaches with Simo and decided to go for this > one as the simplest. By default Kerberos tickets issued in IPA install > are not renewable so it is not possible to use 'kinit -R' to renew > existing ticket. Another approach was to modify our KDB driver to attach > MS-PAC to selected service tickets rather than to TGT but this means we > are losing advantage of 'caching' MS-PAC creation (which may be costly > due to LDAP lookups for gathering group membership) as part of TGT > ticket. > > In the end, adding two options to ipa-adtrust-install which is run only > once is simpler. > > -A (--admin-name, defaults to 'admin') allows to specify admin user > -a (--admin-password) allows to specify admin user's password > > If admin password is not specified, existing default ccache credentials > are used and warning message about need to re-kinit is shown at the end. > > Unattended install is treated as if admin password was not specified. > > http://fedorahosted.org/freeipa/ticket/2852 > > -- > / Alexander Bokovoy Working as described and expected, ACK. bye, Sumit From rcritten at redhat.com Thu Aug 23 12:46:12 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 23 Aug 2012 08:46:12 -0400 Subject: [Freeipa-devel] [PATCH] 0008 Fixes different behaviour of permission-mod and show. In-Reply-To: <50351954.5070102@redhat.com> References: <5034F10D.7050901@redhat.com> <5034F788.30507@redhat.com> <50351954.5070102@redhat.com> Message-ID: <50362614.7050100@redhat.com> Tomas Babej wrote: > On 08/22/2012 05:15 PM, Rob Crittenden wrote: >> Tomas Babej wrote: >>> Hi, >>> >>> Both commands now produce the same output regarding >>> the attributelevelrights. >>> >>> https://fedorahosted.org/freeipa/ticket/2875 >> >> I think some unit tests would be helpful so we don't regress and we >> know which other commands this fixes. >> >> rob > > I ran the tests for the permission plugin (test_permission_plugin.py) > and all of them passed. Tested on clean VM with newly built IPA from the > master, so there should be no regression. Results themselves attached. > > Tomas Right, but those tests all passed prior to your fix as well. We need a test that does a permission-mod and confirms that the rights contains the full list of attributes (and perhaps testing any other commands that were similarly fixed). rob From pviktori at redhat.com Thu Aug 23 14:26:01 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 23 Aug 2012 10:26:01 -0400 Subject: [Freeipa-devel] [PATCH] 0070 Ask for admin password in ipa-adtrust-install In-Reply-To: <20120817150451.GC513@redhat.com> References: <20120817150451.GC513@redhat.com> Message-ID: <50363D79.9000004@redhat.com> On 08/17/2012 11:04 AM, Alexander Bokovoy wrote: > Hi, > > The credentials of the admin user will be used to obtain Kerberos ticket > before configuring cross-realm trusts support and afterwards, to > ensure that the ticket contains MS-PAC information required to actually > add a trust with Active Directory domain via 'ipa trust-add --type=ad' > command. > > We discussed few other approaches with Simo and decided to go for this > one as the simplest. By default Kerberos tickets issued in IPA install > are not renewable so it is not possible to use 'kinit -R' to renew > existing ticket. Another approach was to modify our KDB driver to attach > MS-PAC to selected service tickets rather than to TGT but this means we > are losing advantage of 'caching' MS-PAC creation (which may be costly > due to LDAP lookups for gathering group membership) as part of TGT > ticket. > > In the end, adding two options to ipa-adtrust-install which is run only > once is simpler. > > -A (--admin-name, defaults to 'admin') allows to specify admin user > -a (--admin-password) allows to specify admin user's password > > If admin password is not specified, existing default ccache credentials > are used and warning message about need to re-kinit is shown at the end. > > Unattended install is treated as if admin password was not specified. > > http://fedorahosted.org/freeipa/ticket/2852 > Looks good, ACK. Just put in spaces after the commas before you push: + admin_password = read_password(admin_name,confirm=False,validate=None) -- Petr? From abokovoy at redhat.com Fri Aug 24 13:20:37 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 24 Aug 2012 16:20:37 +0300 Subject: [Freeipa-devel] [PATCH] 0070 Ask for admin password in ipa-adtrust-install In-Reply-To: <50363D79.9000004@redhat.com> References: <20120817150451.GC513@redhat.com> <50363D79.9000004@redhat.com> Message-ID: <20120824132037.GF5558@redhat.com> On Thu, 23 Aug 2012, Petr Viktorin wrote: >On 08/17/2012 11:04 AM, Alexander Bokovoy wrote: >>Hi, >> >>The credentials of the admin user will be used to obtain Kerberos ticket >>before configuring cross-realm trusts support and afterwards, to >>ensure that the ticket contains MS-PAC information required to actually >>add a trust with Active Directory domain via 'ipa trust-add --type=ad' >>command. >> >>We discussed few other approaches with Simo and decided to go for this >>one as the simplest. By default Kerberos tickets issued in IPA install >>are not renewable so it is not possible to use 'kinit -R' to renew >>existing ticket. Another approach was to modify our KDB driver to attach >>MS-PAC to selected service tickets rather than to TGT but this means we >>are losing advantage of 'caching' MS-PAC creation (which may be costly >>due to LDAP lookups for gathering group membership) as part of TGT >>ticket. >> >>In the end, adding two options to ipa-adtrust-install which is run only >>once is simpler. >> >>-A (--admin-name, defaults to 'admin') allows to specify admin user >>-a (--admin-password) allows to specify admin user's password >> >>If admin password is not specified, existing default ccache credentials >>are used and warning message about need to re-kinit is shown at the end. >> >>Unattended install is treated as if admin password was not specified. >> >>http://fedorahosted.org/freeipa/ticket/2852 >> > >Looks good, ACK. Just put in spaces after the commas before you push: >+ admin_password = read_password(admin_name,confirm=False,validate=None) Thanks. Fixed this and another place and pushed to master + 3.0. -- / Alexander Bokovoy From rcritten at redhat.com Fri Aug 24 17:19:26 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 24 Aug 2012 13:19:26 -0400 Subject: [Freeipa-devel] [PATCH] 1031 run cleanallruv task In-Reply-To: <4FFBCC38.7080105@redhat.com> References: <4FF3047C.9020202@redhat.com> <4FF41C8F.2020003@redhat.com> <4FF5DF79.6090806@redhat.com> <4FFBCC38.7080105@redhat.com> Message-ID: <5037B79E.3070603@redhat.com> Martin Kosek wrote: > On 07/05/2012 08:39 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 07/03/2012 04:41 PM, Rob Crittenden wrote: >>>> Deleting a replica can leave a replication vector (RUV) on the other servers. >>>> This can confuse things if the replica is re-added, and it also causes the >>>> server to calculate changes against a server that may no longer exist. >>>> >>>> 389-ds-base provides a new task that self-propogates itself to all available >>>> replicas to clean this RUV data. >>>> >>>> This patch will create this task at deletion time to hopefully clean things up. >>>> >>>> It isn't perfect. If any replica is down or unavailable at the time the >>>> cleanruv task fires, and then comes back up, the old RUV data may be >>>> re-propogated around. >>>> >>>> To make things easier in this case I've added two new commands to >>>> ipa-replica-manage. The first lists the replication ids of all the servers we >>>> have a RUV for. Using this you can call clean_ruv with the replication id of a >>>> server that no longer exists to try the cleanallruv step again. >>>> >>>> This is quite dangerous though. If you run cleanruv against a replica id that >>>> does exist it can cause a loss of data. I believe I've put in enough scary >>>> warnings about this. >>>> >>>> rob >>>> >>> >>> Good work there, this should make cleaning RUVs much easier than with the >>> previous version. >>> >>> This is what I found during review: >>> >>> 1) list_ruv and clean_ruv command help in man is quite lost. I think it would >>> help if we for example have all info for commands indented. This way user could >>> simply over-look the new commands in the man page. >>> >>> >>> 2) I would rename new commands to clean-ruv and list-ruv to make them >>> consistent with the rest of the commands (re-initialize, force-sync). >>> >>> >>> 3) It would be nice to be able to run clean_ruv command in an unattended way >>> (for better testing), i.e. respect --force option as we already do for >>> ipa-replica-manage del. This fix would aid test automation in the future. >>> >>> >>> 4) (minor) The new question (and the del too) does not react too well for >>> CTRL+D: >>> >>> # ipa-replica-manage clean_ruv 3 --force >>> Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 >>> >>> Cleaning the wrong replica ID will cause that server to no >>> longer replicate so it may miss updates while the process >>> is running. It would need to be re-initialized to maintain >>> consistency. Be very careful. >>> Continue to clean? [no]: unexpected error: >>> >>> >>> 5) Help for clean_ruv command without a required parameter is quite confusing >>> as it reports that command is wrong and not the parameter: >>> >>> # ipa-replica-manage clean_ruv >>> Usage: ipa-replica-manage [options] >>> >>> ipa-replica-manage: error: must provide a command [clean_ruv | force-sync | >>> disconnect | connect | del | re-initialize | list | list_ruv] >>> >>> It seems you just forgot to specify the error message in the command definition >>> >>> >>> 6) When the remote replica is down, the clean_ruv command fails with an >>> unexpected error: >>> >>> [root at vm-086 ~]# ipa-replica-manage clean_ruv 5 >>> Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 >>> >>> Cleaning the wrong replica ID will cause that server to no >>> longer replicate so it may miss updates while the process >>> is running. It would need to be re-initialized to maintain >>> consistency. Be very careful. >>> Continue to clean? [no]: y >>> unexpected error: {'desc': 'Operations error'} >>> >>> >>> /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors: >>> [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: failed >>> to connect to repl agreement connection >>> (cn=meTovm-055.idm.lab.bos.redhat.com,cn=replica, >>> cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping >>> tree,cn=config), error 105 >>> [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: replica >>> (cn=meTovm-055.idm.lab. >>> bos.redhat.com,cn=replica,cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping >>> >>> tree, cn=config) has not been cleaned. You will need to rerun the >>> CLEANALLRUV task on this replica. >>> [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: Task >>> failed (1) >>> >>> In this case I think we should inform user that the command failed, possibly >>> because of disconnected replicas and that they could enable the replicas and >>> try again. >>> >>> >>> 7) (minor) "pass" is now redundant in replication.py: >>> + except ldap.INSUFFICIENT_ACCESS: >>> + # We can't make the server we're removing read-only but >>> + # this isn't a show-stopper >>> + root_logger.debug("No permission to switch replica to read-only, >>> continuing anyway") >>> + pass >>> >> >> I think this addresses everything. >> >> rob > > Thanks, almost there! I just found one more issue which needs to be fixed > before we push: > > # ipa-replica-manage del vm-055.idm.lab.bos.redhat.com --force > Directory Manager password: > > Unable to connect to replica vm-055.idm.lab.bos.redhat.com, forcing removal > Failed to get data from 'vm-055.idm.lab.bos.redhat.com': {'desc': "Can't > contact LDAP server"} > Forcing removal on 'vm-086.idm.lab.bos.redhat.com' > > There were issues removing a connection: %d format: a number is required, not str > > Failed to get data from 'vm-055.idm.lab.bos.redhat.com': {'desc': "Can't > contact LDAP server"} > > This is a traceback I retrieved: > Traceback (most recent call last): > File "/sbin/ipa-replica-manage", line 425, in del_master > del_link(realm, r, hostname, options.dirman_passwd, force=True) > File "/sbin/ipa-replica-manage", line 271, in del_link > repl1.cleanallruv(replica_id) > File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", > line 1094, in cleanallruv > root_logger.debug("Creating CLEANALLRUV task for replica id %d" % replicaId) > > > The problem here is that you don't convert replica_id to int in this part: > + replica_id = None > + if repl2: > + replica_id = repl2._get_replica_id(repl2.conn, None) > + else: > + servers = get_ruv(realm, replica1, dirman_passwd) > + for (netloc, rid) in servers: > + if netloc.startswith(replica2): > + replica_id = rid > + break > > Martin > Updated patch using new mechanism in 389-ds-base. This should more thoroughly clean out RUV data when a replica is being deleted, and provide for a way to delete RUV data afterwards too if necessary. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1031-3-cleanruv.patch Type: text/x-diff Size: 14509 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 24 17:54:51 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 24 Aug 2012 13:54:51 -0400 Subject: [Freeipa-devel] [PATCH] 1046 add e-mail by default Message-ID: <5037BFEB.8050602@redhat.com> We weren't automatically creating the mail attribute despite having the default e-mail domain. This patch will add it to all new users. To disable creating this set the default e-mail domain to empty in ipa config. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1046-mail.patch Type: text/x-diff Size: 20485 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 24 17:58:56 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 24 Aug 2012 13:58:56 -0400 Subject: [Freeipa-devel] [PATCH] 1047 remove bogus complaint on no password history Message-ID: <5037C0E0.7080103@redhat.com> If the password policy is set to store no history (hostory=0) then whenever a password is set a bogus log entry is created on 389-ds reporting "failed to generate new password history!?" It fails to generate history because policy tells it not to. This patch will suppress the message. See the ticket for more reproduction details, c#4. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1047-history.patch Type: text/x-diff Size: 1139 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 24 21:52:15 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 24 Aug 2012 17:52:15 -0400 Subject: [Freeipa-devel] [PATCH] 1048 update certificate renewal scripts Message-ID: <5037F78F.5010509@redhat.com> A couple of issues were found in the CA renewal scripts. The api wasn't being initialized so restart_dirsrv() didn't have access to api.env.startup_timeout() A cert was missing from our list of certs to translate into CS.cfg directives. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1048-renewal.patch Type: text/x-diff Size: 4031 bytes Desc: not available URL: From jdennis at redhat.com Sun Aug 26 17:19:07 2012 From: jdennis at redhat.com (John Dennis) Date: Sun, 26 Aug 2012 13:19:07 -0400 Subject: [Freeipa-devel] [PATCH 78] Ticket #2979 - prevent last admin from being disabled In-Reply-To: <503275D9.2020907@redhat.com> References: <502D9291.2090102@redhat.com> <503275D9.2020907@redhat.com> Message-ID: <503A5A8B.1050602@redhat.com> On 08/20/2012 01:37 PM, Petr Viktorin wrote: > (Sorry if you're getting this twice; I didn't send it to the list) > > On 08/16/2012 08:38 PM, John Dennis wrote: >> >> -- >> John Dennis >> >> Looking to carve out IT costs? >> www.redhat.com/carveoutcosts/ >> >> freeipa-jdennis-0078-Ticket-2979-prevent-last-admin-from-being-disabled.patch >> >> >> >From c47109c63530e188db76986fdda48c76bf681d10 Mon Sep 17 00:00:00 2001 >> From: John Dennis >> Date: Thu, 16 Aug 2012 20:28:44 -0400 >> Subject: [PATCH 78] Ticket #2979 - prevent last admin from being disabled >> Content-Type: text/plain; charset="utf-8" >> Content-Transfer-Encoding: 8bit >> >> We prevent the last member of the admin group from being deleted. The >> same check needs to be performed when disabling a user. >> >> Moved the code in del_user to a common subroutine and call it from >> both user_del and user_disable. Note, unlike user_del user_disable >> does not have a 'pre' callback therefore the check function is called >> in user_disable's execute routine. > > This should also prevent disabling all admins if there's more than one: > > # ipa user-add admin2 --first=a --last=b > ------------------- > Added user "admin2" > ------------------- > ... > # ipa group-add-member admins --user=admin2 > ------------------------- > Number of members added 1 > ------------------------- > # ipa user-disable admin2 > ------------------------------ > Disabled user account "admin2" > ------------------------------ > # ipa user-disable admin > ------------------------------ > Disabled user account "admin" > ------------------------------ > # ipa ping > ipa: ERROR: Server is unwilling to perform: Account inactivated. Contact > system administrator. > > Also with one enabled and one disabled admin, it shouldn't be possible > to delete the enabled one. > > > Please add some tests; you can extend the ones added in commit f8e7b51. Good catch with respect to disabled users, thank you. Reworked patch attached, see patch comments. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0078-1-Ticket-2979-prevent-last-admin-from-being-disabled.patch Type: text/x-patch Size: 49861 bytes Desc: not available URL: From pvoborni at redhat.com Mon Aug 27 10:52:01 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 27 Aug 2012 12:52:01 +0200 Subject: [Freeipa-devel] [PATCH] 200 Fix issue which broke setup of Web UI unit tests Message-ID: <503B5151.1050606@redhat.com> Fix issue which broke setup of Web UI unit tests Web UI itself wasn't negatively affected. Issue introduced in be144da672e0634f7aaeff69d662cbc4d11aff0f (#2897). https://fedorahosted.org/freeipa/ticket/2897 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0200-Fix-issue-which-broke-setup-of-Web-UI-unit-tests.patch Type: text/x-patch Size: 1037 bytes Desc: not available URL: From pvoborni at redhat.com Mon Aug 27 10:57:21 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 27 Aug 2012 12:57:21 +0200 Subject: [Freeipa-devel] [PATCH] 201 Successful action notification Message-ID: <503B5291.3090502@redhat.com> User was not notified about success of actions executed from action list, action panel or facet control bar. This patch adds IPA.notify_success(message) call. It creates a yellow notification area with supplied message in Web UI header in the middle of the green area (empty space of first level navigation). This area is displayed for 3s and then it fades out (800ms). It also fades out when it is clicked. This call is used(directly or indirectly) in: * search facets: delete, disable, enable actions * details facets: delete action * user details facet: reset password action * host details facet: unprovision, set OTP actions * service details facet: unprovision action * host and service details facet: request, revoke, restore certificates actions * group details facet: change to POSIX/external actions * dns zone details facet: add/remove permission actions https://fedorahosted.org/freeipa/ticket/2977 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0201-Successful-action-notification.patch Type: text/x-patch Size: 24902 bytes Desc: not available URL: From mkosek at redhat.com Mon Aug 27 11:42:01 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 13:42:01 +0200 Subject: [Freeipa-devel] [PATCH] 0006 Removes sssd.conf after uninstall. In-Reply-To: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> References: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> Message-ID: <503B5D09.7040401@redhat.com> On 08/17/2012 03:04 PM, Tomas Babej wrote: > Hi, > > The sssd.conf file is no longer left behind in case sssd was not > configured before the installation. > > https://fedorahosted.org/freeipa/ticket/2740 > > Tomas > I found few issues with this approach: 1) (major) We do not want to delete sssd.conf when there were more domain's that just the IPA one configured (was_sssd_configured variable). I would consider changing it like that: ... if was_sssd_installed and was_sssd_configured: # SSSD was installed before our installation, config now is restored, restart it ... elif not was_sssd_configured: - remove sssd.conf file (or move it to sssd.conf.deleted as Stephen suggested if we want to be more defensive) ... If we choose the rename operation over delete operation, we should inform a user about the rename as well. 2) (minor) As I mentioned earlier, as a rule of thumb it is better to catch more specific exceptions that just bare "except" clause, PEP8 should contain more info to this cause. In this case, I would catch just for OSError exception. Martin From tbabej at redhat.com Mon Aug 27 12:07:46 2012 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 27 Aug 2012 14:07:46 +0200 Subject: [Freeipa-devel] [PATCH] 0008 Fixes different behaviour of permission-mod and show. In-Reply-To: <50362614.7050100@redhat.com> References: <5034F10D.7050901@redhat.com> <5034F788.30507@redhat.com> <50351954.5070102@redhat.com> <50362614.7050100@redhat.com> Message-ID: <503B6312.7070309@redhat.com> On 08/23/2012 02:46 PM, Rob Crittenden wrote: > Tomas Babej wrote: >> On 08/22/2012 05:15 PM, Rob Crittenden wrote: >>> Tomas Babej wrote: >>>> Hi, >>>> >>>> Both commands now produce the same output regarding >>>> the attributelevelrights. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2875 >>> >>> I think some unit tests would be helpful so we don't regress and we >>> know which other commands this fixes. >>> >>> rob >> >> I ran the tests for the permission plugin (test_permission_plugin.py) >> and all of them passed. Tested on clean VM with newly built IPA from the >> master, so there should be no regression. Results themselves attached. >> >> Tomas > > Right, but those tests all passed prior to your fix as well. We need a > test that does a permission-mod and confirms that the rights contains > the full list of attributes (and perhaps testing any other commands > that were similarly fixed). > > rob > I added unit tests for permission-mod and permission-show. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0008-2-Fixes-different-behaviour-of-permission-mod-and-show.patch Type: text/x-patch Size: 5874 bytes Desc: not available URL: From dpal at redhat.com Mon Aug 27 12:39:52 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:39:52 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <502E6BFD.90400@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <502E6BFD.90400@redhat.com> Message-ID: <503B6A98.2040902@redhat.com> On 08/17/2012 12:06 PM, Rob Crittenden wrote: > Ade Lee wrote: >> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>>> Patch attached this time. I should know better than to do this in >>>>> the >>>>> middle of the night .. >>>>> >>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of >>>>>>>>>>>> changes that >>>>>>>>>>>> will affect IPA. In particular, the following changes will >>>>>>>>>>>> affect >>>>>>>>>>>> current IPA code. >>>>>>>>>>>> >>>>>>>>>>>> * The directory layout of the dogtag instance has changed. >>>>>>>>>>>> Instead of >>>>>>>>>>>> using separate tomcat instances to host different >>>>>>>>>>>> subsystems, the >>>>>>>>>>>> standard dogtag installation will allow one to install a >>>>>>>>>>>> CA. KRA, OCSP >>>>>>>>>>>> and TKS within the same instance. There have been >>>>>>>>>>>> corresponding changes >>>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon >>>>>>>>>>>> (pki-tomcatd, instead >>>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>>> >>>>>>>>>>>> * The default instance will use only four ports (HTTPS, >>>>>>>>>>>> HTTP, AJP and >>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. >>>>>>>>>>>> The default >>>>>>>>>>>> ports will be changed to the standard tomcat ports. As >>>>>>>>>>>> these ports are >>>>>>>>>>>> local to the ipa server machine, this should not cause too >>>>>>>>>>>> much >>>>>>>>>>>> disruption. >>>>>>>>>>>> >>>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>>> >>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding >>>>>>>>>>>> version of >>>>>>>>>>>> tomcatjss. >>>>>>>>>>>> >>>>>>>>>>>> The attached patch integrates all the above changes in IPA >>>>>>>>>>>> installation >>>>>>>>>>>> and maintenance code. Once the patch is applied, users >>>>>>>>>>>> will be able to: >>>>>>>>>>>> >>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to >>>>>>>>>>>> f18/ dogtag >>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to >>>>>>>>>>>> run correctly. >>>>>>>>>>>> This will require the installation of the latest version of >>>>>>>>>>>> tomcatjss as >>>>>>>>>>>> well as the installation of tomcat6. The old-style >>>>>>>>>>>> instance will >>>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched >>>>>>>>>>>> and should >>>>>>>>>>>> continue to work. >>>>>>>>>>>> >>>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>>> >>>>>>>>>>>> 1. Installation with an external CA is not yet completed in >>>>>>>>>>>> the new >>>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>>> >>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been >>>>>>>>>>>> touched >>>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>>> >>>>>>>>>>>> 3. A script needs to be written to allow admins to convert >>>>>>>>>>>> their >>>>>>>>>>>> old-style dogtag instances to new style instances, as well >>>>>>>>>>>> as code to >>>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>>> >>>>>>>>>>>> 4. Installation of old-style instances using >>>>>>>>>>>> pkicreate/pkisilent on >>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled >>>>>>>>>>>> soon. >>>>>>>>>>>> >>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect >>>>>>>>>>>> these changes, >>>>>>>>>>>> but is still in flux. In fact, it is our intention to >>>>>>>>>>>> place the dogtag >>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the >>>>>>>>>>>> meantime, it >>>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>>> >>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. >>>>>>>>>>>> Prior to that >>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code >>>>>>>>>>>> in a >>>>>>>>>>>> developer repo that is located at >>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>>> >>>>>>>>>>>> Testing can be done on both f18 and f17 - although the >>>>>>>>>>>> target platform - >>>>>>>>>>>> and the only platform for which official builds will be >>>>>>>>>>>> created is f18. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ade >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Ade, >>>>>>>>>>> >>>>>>>>>>> Thanks for the patch, I started with review and integration >>>>>>>>>>> tests (currently >>>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>>> >>>>>>>>>>> Installation on single master was smooth, it worked just >>>>>>>>>>> fine, even with >>>>>>>>>>> enforced SELinux, without any error - kudos to you and the >>>>>>>>>>> whole dogtag team. >>>>>>>>>>> The resulting logs and the structure of your log directory >>>>>>>>>>> seems improved. I >>>>>>>>>>> believe that the brand new Python installers will make it >>>>>>>>>>> easier to debug >>>>>>>>>>> issues with dogtag installation when they come. When I >>>>>>>>>>> tried our unit tests or >>>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>>> >>>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>>> >>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled >>>>>>>>>>> automatically on >>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>>> >>>>>>>>>> We have a dogtag patch that is currently in review that will >>>>>>>>>> address >>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required >>>>>>>>>> for f17+, >>>>>>>>>> rather than f18+ >>>>>>>>>> >>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA >>>>>>>>>>> installation on a >>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this >>>>>>>>>>> expectable? >>>>>>>>>>> >>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag >>>>>>>>>> 10 only, >>>>>>>>>> which will be officially available on f18+. So if you update >>>>>>>>>> to dogtag >>>>>>>>>> 10, you must have this patch and visa versa. We probably >>>>>>>>>> need to add >>>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>>> >>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to >>>>>>>>>> the new >>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue >>>>>>>>>> to work. >>>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>>> >>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had >>>>>>>>>>> dogtag10 as well >>>>>>>>>>> and I got the following error: >>>>>>>>>>> >>>>>>>>>>> # ipa-ca-install >>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>>> ... >>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 >>>>>>>>>>> seconds >>>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>>> >>>>>>>>>>> Your system may be partly configured. >>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>>> >>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log >>>>>>>>>>> for details: >>>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>>> >>>>>>>>>>> Root cause: >>>>>>>>>>> ... >>>>>>>>>>> File >>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>>>>>> line >>>>>>>>>>> 625, in __spawn_instance >>>>>>>>>>> "/root/cacert.p12") >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, >>>>>>>>>> rather than >>>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>>> existed! Should not be too bad to fix though - most likely >>>>>>>>>> just need to >>>>>>>>>> move files to the right place. >>>>>>>>> >>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, >>>>>>>>> you probably used >>>>>>>>> that one) and the aforementioned ipa-ca-install run after >>>>>>>>> ipa-replica-install. >>>>>>>>> >>>>>>>> I will be testing this out again. As ipa-ca-install uses the >>>>>>>> same code >>>>>>>> as ipa-replica-install, I would have expected it to work. Did >>>>>>>> you try >>>>>>>> it with selinux in permissive mode? >>>>>>>> >>>>>>> OK -- I tested this out and realized that between the time I >>>>>>> initially >>>>>>> tested this and your tests, we had changed a URL >>>>>>> from /ca/pki/installer/installToken to >>>>>>> /ca/rest/installer/installToken, >>>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>>> >>>>>>> The new patch has been rebased and changes the relevant patch. >>>>>>> With >>>>>>> this, the CA replica installs correctly. >>>>>> >>>>>> Umh, where can I find the new rebased patch? :-) >>>>>> >>>>>>> >>>>>>> There was one issue that I encountered. I have seen this once >>>>>>> before >>>>>>> and will need to investigate further. IPA sometimes restarts >>>>>>> the dogtag >>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then >>>>>>> systemctl >>>>>>> start pki-tomcatd.target. I have noticed that occasionally the >>>>>>> start >>>>>>> invocation fails, while the stop and restart invocations succeed >>>>>>> just >>>>>>> fine. This makes absolutely no sense because restart is >>>>>>> essentially >>>>>>> just stop and then start. >>>>>>> >>>>>>> I think this is actually a bug in systemd. If I reboot the >>>>>>> machine, it >>>>>>> all works perfectly. If we can't figure out whats going on with >>>>>>> systemd, we can always replace this start/stop with >>>>>>> starting/stopping >>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That >>>>>>> seems to >>>>>>> work all the time with no problems. >>>>>>> >>>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>>> >>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended >>>>>> in systemd to >>>>>> use target units this way? I thought that only service files are >>>>>> meant to be >>>>>> used for manipulation with a service. As you said yourself, using >>>>>> service unit >>>>>> file for restart worked every time. >>>>>> >>>>>> Martin >>>>> >>>> >>>> Now, I tested the rebased patch with VMs with permissive SELinux. >>>> IPA server >>>> install was OK, as always, but replica installation ended up with >>>> error. >>>> Although it got much further than before: >>>> >>>> # ipa-ca-install >>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>> ... >>>> [3/3]: restarting directory server >>>> done configuring pkids. >>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>> [1/14]: creating certificate server user >>>> [2/14]: configuring certificate server instance >>>> [3/14]: disabling nonces >>>> [4/14]: importing CA chain to RA certificate database >>>> [5/14]: fixing RA database permissions >>>> [6/14]: setting up signing cert profile >>>> [7/14]: set up CRL publishing >>>> [8/14]: set certificate subject base >>>> [9/14]: enabling Subject Key Identifier >>>> [10/14]: configuring certificate server to start on boot >>>> [11/14]: configure certmonger for renewals >>>> [12/14]: configure clone certificate renewals >>>> [13/14]: configure Server-Cert certificate renewal >>>> [14/14]: Configure HTTP to proxy connections >>>> done configuring pki-tomcatd. >>>> Restarting the directory and certificate servers >>>> >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>> >>>> It seems like that CA on a replica did not start and so a check for >>>> port 8080 >>>> failed. Attached logs (pki-ca-logs.tgz) may provide more >>>> information to this issue. >>>> >>> This is the systemd issue I mentioned before. I will submit a patch >>> which will change the restart mechanism to restart the service and not >>> the target. >>>> >> >> Patch attached. This patch should be applied on top of the large patch >> (being reviewed). >> >>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>>> (permissive SELinux). Now, the service was upgraded smoothly, CA >>>> service >>>> could be restarted without failure. However, certificate operations >>>> now >>>> did not work: >>>> >>>> # ipactl restart >>>> Restarting Directory Service >>>> Restarting KDC Service >>>> Restarting KPASSWD Service >>>> Restarting MEMCACHE Service >>>> Restarting HTTP Service >>>> Restarting CA Service >>>> # ipactl status >>>> Directory Service: RUNNING >>>> KDC Service: RUNNING >>>> KPASSWD Service: RUNNING >>>> MEMCACHE Service: RUNNING >>>> HTTP Service: RUNNING >>>> CA Service: RUNNING >>>> # ipa cert-show 1 >>>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>>> communicate with CMS (Internal Server Error) >>>> >>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>>> >>> The reason for this is that the old dogtag instances are missing: >>> 1. a new parameter in CS.cfg >>> 2. a couple of symlinks >>> >>> I plan to fix this in the dogtag code. Specifically, >>> 1. I will modify the dogtag code to provide a default value in the case >>> where the parameter does not exist. >>> 2. I plan to add a function to the dogtag startup scripts to check and >>> remake any required symlinks. >>> >>> Both of these changes are in the dogtag code, and do not affect this >>> patch. As a workaround, to confirm that the upgrade actually works, do >>> the following manual steps on your upgraded instance: >>> >>> 1. Add the following parameter to CS.cfg >>> pidDir=/var/run >>> >>> 2. Add the following links; >>> cd /var/lib/pki-ca/common/lib >>> ln -s /usr/share/java/apache-commons-codec.jar >>> apache-commons-codec.jar >>> ln -s /usr/share/java/log4j.jar log4j.jar >>> >>> 3 Restart the dogtag instance >> > > To throw a little twist in here, we should make sure that the > certmonger restart scripts still work as well. > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel What is the situation with this patch? Was it commited? Or is it in works/review? Was is superseded by another patch (that I have not got to yet and thus asking)? -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Mon Aug 27 12:40:04 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:40:04 -0400 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <502E4E78.1090407@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> Message-ID: <503B6AA4.30201@redhat.com> On 08/17/2012 10:00 AM, Rich Megginson wrote: > On 08/17/2012 07:44 AM, Martin Kosek wrote: >> Hi guys, >> >> I am now investigating ticket #2866: >> https://fedorahosted.org/freeipa/ticket/2866 >> >> And I am thinking about possible solutions for this problem. In a >> nutshell, we do not properly check referential integrity in some IPA >> objects where we keep one-way DN references to other objects, e.g. in >> - managedBy attribute for a host object >> - memberhost attribute for HBAC rule object >> - memberuser attribute for user object >> - memberallowcmd or memberdenycmd for SUDO command object (reported in >> #2866) >> ... >> >> Currently, I see 2 approaches to solve this: >> 1) Add relevant checks to our ipalib plugins where problematic >> operations with these operations are being executed (like we do for >> selinuxusermap's seealso attribute in HBAC plugin) >> This of course would not prevent direct LDAP deletes. >> >> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >> callbacks and check that this object's DN is not referenced in other >> objects. And if it does, it would reject such modification. Second >> option would be to delete the attribute value with now invalid >> reference. This would be probably more suitable for example for >> references to user objects. >> >> Any comments to these possible approaches are welcome. >> >> Rich, do you think that as an alternative to these 2 approaches, >> memberOf plugin could be eventually modified to do this task? > > This is very similar to the referential integrity plugin already in > 389, except instead of cleaning up references to moved and deleted > entries, you want it to prevent moving or deleting an entry if that > entry is referenced by the > managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some > other entry. > > Note that the managed entry plugin (mep) already handles this for the > managedby attribute. > > Are you already using the memberof plugin for > memberhost/memberuser/memberallowcmd/memberdenycmd? > > This doesn't seem like a job for memberof, this seems like more of a > new check for the referential integrity plugin. Did it translate into a DS ticket? I suspect it is not a big change and would solve a bunch of ugly referential integrity problems. > >> >> Thank you, >> Martin >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Mon Aug 27 12:40:15 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:40:15 -0400 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection In-Reply-To: <190240025.5168295.1345022311962.JavaMail.root@redhat.com> References: <190240025.5168295.1345022311962.JavaMail.root@redhat.com> Message-ID: <503B6AAF.7030003@redhat.com> On 08/15/2012 05:18 AM, Simo Sorce wrote: > ----- Original Message ----- >> On 08/14/2012 08:25 PM, Simo Sorce wrote: >>> See man ldap_result, the entries return with type >>> LDAP_RES_SEARCH_ENTRY, the last message is instead >>> LDAP_RES_SEARCH_RESULT which tells you the searc is complete. >>> >>> This last message is never sent for a persistent search of course >>> (IIRC :-). >> Right, it is what I tried to say with "there is no SearchResultDone >> LDAP message". > I see. > >> http://tools.ietf.org/html/draft-smith-psearch-ldap-01#page-3 >> section 4 paragraph b). >> >> This patch deals with persistent search only. Non-persistent search >> scenarios >> have cache records with limited TTL value. >> >>> But in case of a persistent search you should never need to flush >>> as entries are valid until you see a change. >> Unfortunately, cache flush is necessary after persistent search >> reconnection. >> Records can change in meanwhile... >> >> Example: >> 1. BIND started, cache was populated. >> 2. Persistent search connection was lost >> 3. Record was deleted (but Entry Change Notification wasn't >> delivered) >> 4. Persistent search connection was re-established - there is no >> information >> about deleted record/zone, BIND still sees old data in the cache. >> >> For this reason https://fedorahosted.org/bind-dyndb-ldap/ticket/44 >> was filled. >> >> What I missed? > You missed nothing, I did. > > Howeve I have an idea to handle this situation smartly. > > We can issue 2 searches on 2 separate connections. > > The first search will be a persistent search, however it will be started with a filter that has an additional element. > Before the persistent search we do a rootdse search and find either the higher modifyTimestamp or the higher entryUSN or equivalent, depending on what is available in the server. Worst case we do a ase search of the DNS tree and pick that modifyTimestamp. > Then we start the persistent search with (&(entryUSN>%d)) or (&(modifyTimestamp>=%d) (note modifyTimestamp requires >= due to low resolution). > > On the other connection we start instead a nornmal search. This normal search will terminate with a 'done' result, so you know that once that search is finished you can flush any cache that has not been re-validated. While you get any changes that are occurring live on the persistent search connection. > > This way we should be able to not loose any update and still know when we got all results and drop unvalidated caches. > What is the status of this proposal? > HTH, > Simo. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Mon Aug 27 12:40:28 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:40:28 -0400 Subject: [Freeipa-devel] Freeipa wiki editing In-Reply-To: <1144308720.645889.1343282483657.JavaMail.root@redhat.com> References: <1144308720.645889.1343282483657.JavaMail.root@redhat.com> Message-ID: <503B6ABC.9090308@redhat.com> On 07/26/2012 02:01 AM, Javier Ramirez wrote: > Hi, > > As per the instructions found at http://freeipa.com/page/Contribute , I send this email to request for a freeipa wiki account . I have some amends to make to http://freeipa.com/page/ConfiguringAixClients . > Javier, sorry for the delay. Was access granted? If not please reply to me directly. > Thanks. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Mon Aug 27 12:40:41 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:40:41 -0400 Subject: [Freeipa-devel] [PATCH] Set TTL during ipa-client-install for DNS records In-Reply-To: References: Message-ID: <503B6AC9.9030808@redhat.com> On 08/14/2012 10:38 AM, James Hogarth wrote: > Hi, > > Please see attached patch to allow the TTL to be specified when an IPA > client is configured. > > The default is to use the 1200 that is currently in place. > > Regardless of is it is set or not as an argument ipa_dyndns_ttl is set > in sssd.conf so that it's transparent for an admin to see what the TTL > will be if he enables updates in SSSD. > > Until SSSD allows for the TTL to be set in sssd.conf (patch sent in > and pending review for possible future inclusion) this patch will only > affect the initial registration and not any ongoing changes. > > Comments would be most welcome! James, thank you for the contribution. I am not sure we would be able to take this patch in 3.0 however SSSD guys might be able to address the issue faster. What is the status of review of this patch? Is there a corresponding ticket? If we think this is something we should take in we need to have a ticket in the right milestone with this patch attached. Thanks Dmitri > Kind regards, > > James > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dpal at redhat.com Mon Aug 27 12:40:49 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:40:49 -0400 Subject: [Freeipa-devel] [PATCH] 194 Handle case when trusted domain user access the Web UI In-Reply-To: <5024EC72.1000902@redhat.com> References: <5024EC72.1000902@redhat.com> Message-ID: <503B6AD1.8020903@redhat.com> On 08/10/2012 07:11 AM, Petr Vobornik wrote: > WebUI catches the fact that the user can't access LDAP server with a > current ticket. It shows form-based auth login dialog. Previoustly an > ugly error was returned on an almost empty page, and user had no > recourse. > > https://fedorahosted.org/freeipa/ticket/2897 > > > I don't like the implementation much. Problem is that we don't > separate state variables and framework objects in IPA object. It is > probably a topic for fixing in 3.2. Was a ticket filed? > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dpal at redhat.com Mon Aug 27 12:41:53 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 08:41:53 -0400 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <502E4E78.1090407@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> Message-ID: <503B6B11.8070406@redhat.com> On 08/17/2012 10:00 AM, Rich Megginson wrote: > On 08/17/2012 07:44 AM, Martin Kosek wrote: >> Hi guys, >> >> I am now investigating ticket #2866: >> https://fedorahosted.org/freeipa/ticket/2866 >> >> And I am thinking about possible solutions for this problem. In a >> nutshell, we do not properly check referential integrity in some IPA >> objects where we keep one-way DN references to other objects, e.g. in >> - managedBy attribute for a host object >> - memberhost attribute for HBAC rule object >> - memberuser attribute for user object >> - memberallowcmd or memberdenycmd for SUDO command object (reported in >> #2866) >> ... >> >> Currently, I see 2 approaches to solve this: >> 1) Add relevant checks to our ipalib plugins where problematic >> operations with these operations are being executed (like we do for >> selinuxusermap's seealso attribute in HBAC plugin) >> This of course would not prevent direct LDAP deletes. >> >> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >> callbacks and check that this object's DN is not referenced in other >> objects. And if it does, it would reject such modification. Second >> option would be to delete the attribute value with now invalid >> reference. This would be probably more suitable for example for >> references to user objects. >> >> Any comments to these possible approaches are welcome. >> >> Rich, do you think that as an alternative to these 2 approaches, >> memberOf plugin could be eventually modified to do this task? > > This is very similar to the referential integrity plugin already in > 389, except instead of cleaning up references to moved and deleted > entries, you want it to prevent moving or deleting an entry if that > entry is referenced by the > managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some > other entry. > > Note that the managed entry plugin (mep) already handles this for the > managedby attribute. > > Are you already using the memberof plugin for > memberhost/memberuser/memberallowcmd/memberdenycmd? > > This doesn't seem like a job for memberof, this seems like more of a > new check for the referential integrity plugin. Did it translate into a DS ticket? I suspect it is not a big change and would solve a bunch of ugly referential integrity problems. > >> >> Thank you, >> Martin >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pviktori at redhat.com Mon Aug 27 12:45:44 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 27 Aug 2012 14:45:44 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <503B6A98.2040902@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <502E6BFD.90400@redhat.com> <503B6A98.2040902@redhat.com> Message-ID: <503B6BF8.8090909@redhat.com> On 08/27/2012 02:39 PM, Dmitri Pal wrote: > On 08/17/2012 12:06 PM, Rob Crittenden wrote: >> Ade Lee wrote: >>> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >>>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>>>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>>>> Patch attached this time. I should know better than to do this in >>>>>> the >>>>>> middle of the night .. >>>>>> >>>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of >>>>>>>>>>>>> changes that >>>>>>>>>>>>> will affect IPA. In particular, the following changes will >>>>>>>>>>>>> affect >>>>>>>>>>>>> current IPA code. >>>>>>>>>>>>> >>>>>>>>>>>>> * The directory layout of the dogtag instance has changed. >>>>>>>>>>>>> Instead of >>>>>>>>>>>>> using separate tomcat instances to host different >>>>>>>>>>>>> subsystems, the >>>>>>>>>>>>> standard dogtag installation will allow one to install a >>>>>>>>>>>>> CA. KRA, OCSP >>>>>>>>>>>>> and TKS within the same instance. There have been >>>>>>>>>>>>> corresponding changes >>>>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon >>>>>>>>>>>>> (pki-tomcatd, instead >>>>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>>>> >>>>>>>>>>>>> * The default instance will use only four ports (HTTPS, >>>>>>>>>>>>> HTTP, AJP and >>>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. >>>>>>>>>>>>> The default >>>>>>>>>>>>> ports will be changed to the standard tomcat ports. As >>>>>>>>>>>>> these ports are >>>>>>>>>>>>> local to the ipa server machine, this should not cause too >>>>>>>>>>>>> much >>>>>>>>>>>>> disruption. >>>>>>>>>>>>> >>>>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>>>> >>>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding >>>>>>>>>>>>> version of >>>>>>>>>>>>> tomcatjss. >>>>>>>>>>>>> >>>>>>>>>>>>> The attached patch integrates all the above changes in IPA >>>>>>>>>>>>> installation >>>>>>>>>>>>> and maintenance code. Once the patch is applied, users >>>>>>>>>>>>> will be able to: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to >>>>>>>>>>>>> f18/ dogtag >>>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to >>>>>>>>>>>>> run correctly. >>>>>>>>>>>>> This will require the installation of the latest version of >>>>>>>>>>>>> tomcatjss as >>>>>>>>>>>>> well as the installation of tomcat6. The old-style >>>>>>>>>>>>> instance will >>>>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched >>>>>>>>>>>>> and should >>>>>>>>>>>>> continue to work. >>>>>>>>>>>>> >>>>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Installation with an external CA is not yet completed in >>>>>>>>>>>>> the new >>>>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>>>> >>>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been >>>>>>>>>>>>> touched >>>>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>>>> >>>>>>>>>>>>> 3. A script needs to be written to allow admins to convert >>>>>>>>>>>>> their >>>>>>>>>>>>> old-style dogtag instances to new style instances, as well >>>>>>>>>>>>> as code to >>>>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>>>> >>>>>>>>>>>>> 4. Installation of old-style instances using >>>>>>>>>>>>> pkicreate/pkisilent on >>>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled >>>>>>>>>>>>> soon. >>>>>>>>>>>>> >>>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect >>>>>>>>>>>>> these changes, >>>>>>>>>>>>> but is still in flux. In fact, it is our intention to >>>>>>>>>>>>> place the dogtag >>>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the >>>>>>>>>>>>> meantime, it >>>>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>>>> >>>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. >>>>>>>>>>>>> Prior to that >>>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code >>>>>>>>>>>>> in a >>>>>>>>>>>>> developer repo that is located at >>>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>>>> >>>>>>>>>>>>> Testing can be done on both f18 and f17 - although the >>>>>>>>>>>>> target platform - >>>>>>>>>>>>> and the only platform for which official builds will be >>>>>>>>>>>>> created is f18. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Ade >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Ade, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for the patch, I started with review and integration >>>>>>>>>>>> tests (currently >>>>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>>>> >>>>>>>>>>>> Installation on single master was smooth, it worked just >>>>>>>>>>>> fine, even with >>>>>>>>>>>> enforced SELinux, without any error - kudos to you and the >>>>>>>>>>>> whole dogtag team. >>>>>>>>>>>> The resulting logs and the structure of your log directory >>>>>>>>>>>> seems improved. I >>>>>>>>>>>> believe that the brand new Python installers will make it >>>>>>>>>>>> easier to debug >>>>>>>>>>>> issues with dogtag installation when they come. When I >>>>>>>>>>>> tried our unit tests or >>>>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>>>> >>>>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>>>> >>>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled >>>>>>>>>>>> automatically on >>>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>>>> >>>>>>>>>>> We have a dogtag patch that is currently in review that will >>>>>>>>>>> address >>>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required >>>>>>>>>>> for f17+, >>>>>>>>>>> rather than f18+ >>>>>>>>>>> >>>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA >>>>>>>>>>>> installation on a >>>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this >>>>>>>>>>>> expectable? >>>>>>>>>>>> >>>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag >>>>>>>>>>> 10 only, >>>>>>>>>>> which will be officially available on f18+. So if you update >>>>>>>>>>> to dogtag >>>>>>>>>>> 10, you must have this patch and visa versa. We probably >>>>>>>>>>> need to add >>>>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>>>> >>>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to >>>>>>>>>>> the new >>>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue >>>>>>>>>>> to work. >>>>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>>>> >>>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had >>>>>>>>>>>> dogtag10 as well >>>>>>>>>>>> and I got the following error: >>>>>>>>>>>> >>>>>>>>>>>> # ipa-ca-install >>>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>>>> ... >>>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 >>>>>>>>>>>> seconds >>>>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>>>> >>>>>>>>>>>> Your system may be partly configured. >>>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>>>> >>>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log >>>>>>>>>>>> for details: >>>>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>>>> >>>>>>>>>>>> Root cause: >>>>>>>>>>>> ... >>>>>>>>>>>> File >>>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>>>>>>> line >>>>>>>>>>>> 625, in __spawn_instance >>>>>>>>>>>> "/root/cacert.p12") >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, >>>>>>>>>>> rather than >>>>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>>>> existed! Should not be too bad to fix though - most likely >>>>>>>>>>> just need to >>>>>>>>>>> move files to the right place. >>>>>>>>>> >>>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, >>>>>>>>>> you probably used >>>>>>>>>> that one) and the aforementioned ipa-ca-install run after >>>>>>>>>> ipa-replica-install. >>>>>>>>>> >>>>>>>>> I will be testing this out again. As ipa-ca-install uses the >>>>>>>>> same code >>>>>>>>> as ipa-replica-install, I would have expected it to work. Did >>>>>>>>> you try >>>>>>>>> it with selinux in permissive mode? >>>>>>>>> >>>>>>>> OK -- I tested this out and realized that between the time I >>>>>>>> initially >>>>>>>> tested this and your tests, we had changed a URL >>>>>>>> from /ca/pki/installer/installToken to >>>>>>>> /ca/rest/installer/installToken, >>>>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>>>> >>>>>>>> The new patch has been rebased and changes the relevant patch. >>>>>>>> With >>>>>>>> this, the CA replica installs correctly. >>>>>>> >>>>>>> Umh, where can I find the new rebased patch? :-) >>>>>>> >>>>>>>> >>>>>>>> There was one issue that I encountered. I have seen this once >>>>>>>> before >>>>>>>> and will need to investigate further. IPA sometimes restarts >>>>>>>> the dogtag >>>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then >>>>>>>> systemctl >>>>>>>> start pki-tomcatd.target. I have noticed that occasionally the >>>>>>>> start >>>>>>>> invocation fails, while the stop and restart invocations succeed >>>>>>>> just >>>>>>>> fine. This makes absolutely no sense because restart is >>>>>>>> essentially >>>>>>>> just stop and then start. >>>>>>>> >>>>>>>> I think this is actually a bug in systemd. If I reboot the >>>>>>>> machine, it >>>>>>>> all works perfectly. If we can't figure out whats going on with >>>>>>>> systemd, we can always replace this start/stop with >>>>>>>> starting/stopping >>>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That >>>>>>>> seems to >>>>>>>> work all the time with no problems. >>>>>>>> >>>>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>>>> >>>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended >>>>>>> in systemd to >>>>>>> use target units this way? I thought that only service files are >>>>>>> meant to be >>>>>>> used for manipulation with a service. As you said yourself, using >>>>>>> service unit >>>>>>> file for restart worked every time. >>>>>>> >>>>>>> Martin >>>>>> >>>>> >>>>> Now, I tested the rebased patch with VMs with permissive SELinux. >>>>> IPA server >>>>> install was OK, as always, but replica installation ended up with >>>>> error. >>>>> Although it got much further than before: >>>>> >>>>> # ipa-ca-install >>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>> ... >>>>> [3/3]: restarting directory server >>>>> done configuring pkids. >>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>>> [1/14]: creating certificate server user >>>>> [2/14]: configuring certificate server instance >>>>> [3/14]: disabling nonces >>>>> [4/14]: importing CA chain to RA certificate database >>>>> [5/14]: fixing RA database permissions >>>>> [6/14]: setting up signing cert profile >>>>> [7/14]: set up CRL publishing >>>>> [8/14]: set certificate subject base >>>>> [9/14]: enabling Subject Key Identifier >>>>> [10/14]: configuring certificate server to start on boot >>>>> [11/14]: configure certmonger for renewals >>>>> [12/14]: configure clone certificate renewals >>>>> [13/14]: configure Server-Cert certificate renewal >>>>> [14/14]: Configure HTTP to proxy connections >>>>> done configuring pki-tomcatd. >>>>> Restarting the directory and certificate servers >>>>> >>>>> Your system may be partly configured. >>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>> >>>>> It seems like that CA on a replica did not start and so a check for >>>>> port 8080 >>>>> failed. Attached logs (pki-ca-logs.tgz) may provide more >>>>> information to this issue. >>>>> >>>> This is the systemd issue I mentioned before. I will submit a patch >>>> which will change the restart mechanism to restart the service and not >>>> the target. >>>>> >>> >>> Patch attached. This patch should be applied on top of the large patch >>> (being reviewed). >>> >>>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>>>> (permissive SELinux). Now, the service was upgraded smoothly, CA >>>>> service >>>>> could be restarted without failure. However, certificate operations >>>>> now >>>>> did not work: >>>>> >>>>> # ipactl restart >>>>> Restarting Directory Service >>>>> Restarting KDC Service >>>>> Restarting KPASSWD Service >>>>> Restarting MEMCACHE Service >>>>> Restarting HTTP Service >>>>> Restarting CA Service >>>>> # ipactl status >>>>> Directory Service: RUNNING >>>>> KDC Service: RUNNING >>>>> KPASSWD Service: RUNNING >>>>> MEMCACHE Service: RUNNING >>>>> HTTP Service: RUNNING >>>>> CA Service: RUNNING >>>>> # ipa cert-show 1 >>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>>>> communicate with CMS (Internal Server Error) >>>>> >>>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>>>> >>>> The reason for this is that the old dogtag instances are missing: >>>> 1. a new parameter in CS.cfg >>>> 2. a couple of symlinks >>>> >>>> I plan to fix this in the dogtag code. Specifically, >>>> 1. I will modify the dogtag code to provide a default value in the case >>>> where the parameter does not exist. >>>> 2. I plan to add a function to the dogtag startup scripts to check and >>>> remake any required symlinks. >>>> >>>> Both of these changes are in the dogtag code, and do not affect this >>>> patch. As a workaround, to confirm that the upgrade actually works, do >>>> the following manual steps on your upgraded instance: >>>> >>>> 1. Add the following parameter to CS.cfg >>>> pidDir=/var/run >>>> >>>> 2. Add the following links; >>>> cd /var/lib/pki-ca/common/lib >>>> ln -s /usr/share/java/apache-commons-codec.jar >>>> apache-commons-codec.jar >>>> ln -s /usr/share/java/log4j.jar log4j.jar >>>> >>>> 3 Restart the dogtag instance >>> >> >> To throw a little twist in here, we should make sure that the >> certmonger restart scripts still work as well. >> >> rob >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > What is the situation with this patch? Was it commited? Or is it in > works/review? Was is superseded by another patch (that I have not got to > yet and thus asking)? > I am finishing and testing a patch to apply on top, which will make it possible to use either Dogtag 9 or 10 depending on what's installed. -- Petr? From mkosek at redhat.com Mon Aug 27 12:57:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 14:57:44 +0200 Subject: [Freeipa-devel] [PATCH] 0006 Removes sssd.conf after uninstall. In-Reply-To: <503B5D09.7040401@redhat.com> References: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> <503B5D09.7040401@redhat.com> Message-ID: <503B6EC8.3060508@redhat.com> On 08/27/2012 01:42 PM, Martin Kosek wrote: > On 08/17/2012 03:04 PM, Tomas Babej wrote: >> Hi, >> >> The sssd.conf file is no longer left behind in case sssd was not >> configured before the installation. >> >> https://fedorahosted.org/freeipa/ticket/2740 >> >> Tomas >> > > I found few issues with this approach: > > 1) (major) We do not want to delete sssd.conf when there were more domain's > that just the IPA one configured (was_sssd_configured variable). > > I would consider changing it like that: > > ... > if was_sssd_installed and was_sssd_configured: > # SSSD was installed before our installation, config now is restored, > restart it > ... > elif not was_sssd_configured: > - remove sssd.conf file (or move it to sssd.conf.deleted as Stephen > suggested if we want to be more defensive) > ... > > If we choose the rename operation over delete operation, we should inform a > user about the rename as well. > > 2) (minor) As I mentioned earlier, as a rule of thumb it is better to catch > more specific exceptions that just bare "except" clause, PEP8 should contain > more info to this cause. In this case, I would catch just for OSError exception. > > Martin I had a short discussion with Tomas and current state of handling of sssd.conf during uninstall looks weird. Uninstall code do check if there is some non-IPA domain configured in current sssd.conf, but if it is, we still rather restore old sssd.conf which was backed up during ipa-client-install instead of modifying current one. I think this may cause user authentication issues if he configured non-IPA domains after ipa-client-install. I think that the right behavior of SSSD conf uninstall should be the following: * sssd.conf existed before IPA install + non-IPA domains in sssd.conf found: - move backed conf up sssd.conf.bkp (and inform the user) - use SSSDConfig delete_domain function to remove ipa domain from sssd.conf - restart sssd afterwards * sssd.conf did not exist before IPA install + non-IPA domains in sssd.conf found: - use SSSDConfig delete_domain function to remove ipa domain from sssd.conf - restart sssd afterwards * sssd.conf did not exist before IPA install + no other domains in sssd.conf: - remove sssd.conf or rename it to sssd.conf.deleted Jakub, any recommendations? Thanks, Martin From pvoborni at redhat.com Mon Aug 27 13:21:53 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 27 Aug 2012 15:21:53 +0200 Subject: [Freeipa-devel] [PATCH] 194 Handle case when trusted domain user access the Web UI In-Reply-To: <5029A82D.7000105@redhat.com> References: <5024EC72.1000902@redhat.com> <5029A82D.7000105@redhat.com> Message-ID: <503B7471.6070108@redhat.com> On 08/14/2012 03:21 AM, Endi Sukma Dewata wrote: > On 8/10/2012 6:11 AM, Petr Vobornik wrote: >> WebUI catches the fact that the user can't access LDAP server with a >> current ticket. It shows form-based auth login dialog. Previoustly an >> ugly error was returned on an almost empty page, and user had no >> recourse. >> >> https://fedorahosted.org/freeipa/ticket/2897 >> >> >> I don't like the implementation much. Problem is that we don't separate >> state variables and framework objects in IPA object. It is probably a >> topic for fixing in 3.2. > > I don't have an environment to test this, but the code looks fine, so > it's ACKed. > > Some comments: > > 1. The logged_kerberos and logged_password cannot be true at the same > time, right? Right. > Maybe they can be combined into a single variable (e.g. > login_status) which different values for unauthenticated, logged in via > kerberos, and logged in via password. Maybe the 'initialized' variable > can be combined too. Yes logged_x can be combined that way. I would not merge it with initialized though. Login and initialization are two separate steps. The patch is pushed and I don't think the merge is important to do, so I will leave it be. We might change it later when needed. > > 2. I agree about the state variables & framework objects separation. > Currently the 'IPA' object is both used as a singleton/global variable > and also as a name space for the framework. I think ideally we should > use a generic/non-IPA specific name for the framework. Probably > something like this: > > // UI Framework > var UI = { ... }; > UI.entity = function() { ... }; > UI.facet = function() { ... }; > > // IPA UI > var IPA = UI(); > IPA.user.entity = function() { ... }; > IPA.user.details_facet = function() { ... }; > > IPA.init(); > Copied to https://fedorahosted.org/freeipa/ticket/3030 -- Petr Vobornik From mkosek at redhat.com Mon Aug 27 13:26:11 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 15:26:11 +0200 Subject: [Freeipa-devel] [PATCH] 1047 remove bogus complaint on no password history In-Reply-To: <5037C0E0.7080103@redhat.com> References: <5037C0E0.7080103@redhat.com> Message-ID: <503B7573.6000002@redhat.com> On 08/24/2012 07:58 PM, Rob Crittenden wrote: > If the password policy is set to store no history (hostory=0) then whenever a > password is set a bogus log entry is created on 389-ds reporting "failed to > generate new password history!?" It fails to generate history because policy > tells it not to. This patch will suppress the message. > > See the ticket for more reproduction details, c#4. > > rob Works like a charm. ACK. Pushed to master, ipa-3-0. Martin From jhrozek at redhat.com Mon Aug 27 13:37:27 2012 From: jhrozek at redhat.com (Jakub Hrozek) Date: Mon, 27 Aug 2012 15:37:27 +0200 Subject: [Freeipa-devel] [PATCH] 0006 Removes sssd.conf after uninstall. In-Reply-To: <503B6EC8.3060508@redhat.com> References: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> <503B5D09.7040401@redhat.com> <503B6EC8.3060508@redhat.com> Message-ID: <20120827133727.GG8612@zeppelin.brq.redhat.com> On Mon, Aug 27, 2012 at 02:57:44PM +0200, Martin Kosek wrote: > I think that the right behavior of SSSD conf uninstall should be the following: > > * sssd.conf existed before IPA install + non-IPA domains in sssd.conf found: > - move backed conf up sssd.conf.bkp (and inform the user) > - use SSSDConfig delete_domain function to remove ipa domain from sssd.conf > - restart sssd afterwards I'm confused here, which of the files is the original pre-ipa-client-install file? How does the non-ipa domain end up in the sssd.conf file? Does it have to be configured manually or does ipa-client-install merge the list of domains on installation? Maybe it would be enough to just remove the IPA domain and inform the user about the original file. I suspect that in most environments the correct file would be restored with something like puppet anyway. > * sssd.conf did not exist before IPA install + non-IPA domains in sssd.conf found: > - use SSSDConfig delete_domain function to remove ipa domain from sssd.conf > - restart sssd afterwards Sounds good to me. > * sssd.conf did not exist before IPA install + no other domains in sssd.conf: > - remove sssd.conf or rename it to sssd.conf.deleted > Sounds good to me. From rcritten at redhat.com Mon Aug 27 13:38:14 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 27 Aug 2012 09:38:14 -0400 Subject: [Freeipa-devel] [PATCH] Set TTL during ipa-client-install for DNS records In-Reply-To: <503B6AC9.9030808@redhat.com> References: <503B6AC9.9030808@redhat.com> Message-ID: <503B7846.1070308@redhat.com> Dmitri Pal wrote: > On 08/14/2012 10:38 AM, James Hogarth wrote: >> Hi, >> >> Please see attached patch to allow the TTL to be specified when an IPA >> client is configured. >> >> The default is to use the 1200 that is currently in place. >> >> Regardless of is it is set or not as an argument ipa_dyndns_ttl is set >> in sssd.conf so that it's transparent for an admin to see what the TTL >> will be if he enables updates in SSSD. >> >> Until SSSD allows for the TTL to be set in sssd.conf (patch sent in >> and pending review for possible future inclusion) this patch will only >> affect the initial registration and not any ongoing changes. >> >> Comments would be most welcome! > > James, thank you for the contribution. > I am not sure we would be able to take this patch in 3.0 however SSSD > guys might be able to address the issue faster. > > What is the status of review of this patch? > Is there a corresponding ticket? > If we think this is something we should take in we need to have a ticket > in the right milestone with this patch attached. IPA ticket https://fedorahosted.org/freeipa/ticket/3031 rob From mkosek at redhat.com Mon Aug 27 13:38:36 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 15:38:36 +0200 Subject: [Freeipa-devel] [PATCH 80] Ticket #2850 - Ipactl exception not handled well In-Reply-To: <5034F06D.3050708@redhat.com> References: <5032A7F1.1060505@redhat.com> <5034F06D.3050708@redhat.com> Message-ID: <503B785C.7010900@redhat.com> On 08/22/2012 04:45 PM, Petr Viktorin wrote: >> From fda504233ee46a494b7ed6b85593e7e586739425 Mon Sep 17 00:00:00 2001 >> From: John Dennis >> Date: Mon, 20 Aug 2012 16:47:52 -0400 >> Subject: [PATCH 80] Ticket #2850 - Ipactl exception not handled well >> Content-Type: text/plain; charset="utf-8" >> Content-Transfer-Encoding: 8bit >> >> Ticket #2850 - Ipactl exception not handled well >> >> There were various places in ipactl which intialized IpactlError with >> None as the msg. If you called str() on that exception all was well >> because ScriptError.__str__() converted a msg with None to the empty >> string (IpactlError is subclassed from ScriptError). But a few places >> directly access e.msg which will be None if initialized that way. It's >> hard to tell from the stack traces but I'm pretty sure it's those >> places which use e.msg directly which will cause the problems seen in >> the bug report. >> >> I do not believe it is ever correct to initialize an exception message >> to None, I don't even understand what that means. On the other hand >> initializing to the empty string is sensible and for that matter is >> the default for the class. >> >> This patch makes two fixes: >> >> 1) The ScriptError initializer will now convert a msg parameter of >> None to the empty string. >> >> 2) All places that initialized IpactlError's msg parameter to None >> removed the None initializer allowing the msg parameter to default >> to the empty string. >> >> I don't know how to test the fix for Ticket #2850 because it's not >> clear how it got into that state in the first place, but I do believe >> initialing the msg value to None is clearly wrong and should fix the >> problem. >> --- > > ACK > Pushed to master, ipa-3-0. Martin From rcritten at redhat.com Mon Aug 27 13:40:53 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 27 Aug 2012 09:40:53 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <503B6BF8.8090909@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <502E6BFD.90400@redhat.com> <503B6A98.2040902@redhat.com> <503B6BF8.8090909@redhat.com> Message-ID: <503B78E5.10008@redhat.com> Petr Viktorin wrote: > On 08/27/2012 02:39 PM, Dmitri Pal wrote: >> On 08/17/2012 12:06 PM, Rob Crittenden wrote: >>> Ade Lee wrote: >>>> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >>>>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>>>>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>>>>> Patch attached this time. I should know better than to do this in >>>>>>> the >>>>>>> middle of the night .. >>>>>>> >>>>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of >>>>>>>>>>>>>> changes that >>>>>>>>>>>>>> will affect IPA. In particular, the following changes will >>>>>>>>>>>>>> affect >>>>>>>>>>>>>> current IPA code. >>>>>>>>>>>>>> >>>>>>>>>>>>>> * The directory layout of the dogtag instance has changed. >>>>>>>>>>>>>> Instead of >>>>>>>>>>>>>> using separate tomcat instances to host different >>>>>>>>>>>>>> subsystems, the >>>>>>>>>>>>>> standard dogtag installation will allow one to install a >>>>>>>>>>>>>> CA. KRA, OCSP >>>>>>>>>>>>>> and TKS within the same instance. There have been >>>>>>>>>>>>>> corresponding changes >>>>>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon >>>>>>>>>>>>>> (pki-tomcatd, instead >>>>>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>>>>> >>>>>>>>>>>>>> * The default instance will use only four ports (HTTPS, >>>>>>>>>>>>>> HTTP, AJP and >>>>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. >>>>>>>>>>>>>> The default >>>>>>>>>>>>>> ports will be changed to the standard tomcat ports. As >>>>>>>>>>>>>> these ports are >>>>>>>>>>>>>> local to the ipa server machine, this should not cause too >>>>>>>>>>>>>> much >>>>>>>>>>>>>> disruption. >>>>>>>>>>>>>> >>>>>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding >>>>>>>>>>>>>> version of >>>>>>>>>>>>>> tomcatjss. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The attached patch integrates all the above changes in IPA >>>>>>>>>>>>>> installation >>>>>>>>>>>>>> and maintenance code. Once the patch is applied, users >>>>>>>>>>>>>> will be able to: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag >>>>>>>>>>>>>> 10. >>>>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to >>>>>>>>>>>>>> f18/ dogtag >>>>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to >>>>>>>>>>>>>> run correctly. >>>>>>>>>>>>>> This will require the installation of the latest version of >>>>>>>>>>>>>> tomcatjss as >>>>>>>>>>>>>> well as the installation of tomcat6. The old-style >>>>>>>>>>>>>> instance will >>>>>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched >>>>>>>>>>>>>> and should >>>>>>>>>>>>>> continue to work. >>>>>>>>>>>>>> >>>>>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Installation with an external CA is not yet completed in >>>>>>>>>>>>>> the new >>>>>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been >>>>>>>>>>>>>> touched >>>>>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. A script needs to be written to allow admins to convert >>>>>>>>>>>>>> their >>>>>>>>>>>>>> old-style dogtag instances to new style instances, as well >>>>>>>>>>>>>> as code to >>>>>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 4. Installation of old-style instances using >>>>>>>>>>>>>> pkicreate/pkisilent on >>>>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled >>>>>>>>>>>>>> soon. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect >>>>>>>>>>>>>> these changes, >>>>>>>>>>>>>> but is still in flux. In fact, it is our intention to >>>>>>>>>>>>>> place the dogtag >>>>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the >>>>>>>>>>>>>> meantime, it >>>>>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. >>>>>>>>>>>>>> Prior to that >>>>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code >>>>>>>>>>>>>> in a >>>>>>>>>>>>>> developer repo that is located at >>>>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Testing can be done on both f18 and f17 - although the >>>>>>>>>>>>>> target platform - >>>>>>>>>>>>>> and the only platform for which official builds will be >>>>>>>>>>>>>> created is f18. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Ade >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Ade, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the patch, I started with review and integration >>>>>>>>>>>>> tests (currently >>>>>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>>>>> >>>>>>>>>>>>> Installation on single master was smooth, it worked just >>>>>>>>>>>>> fine, even with >>>>>>>>>>>>> enforced SELinux, without any error - kudos to you and the >>>>>>>>>>>>> whole dogtag team. >>>>>>>>>>>>> The resulting logs and the structure of your log directory >>>>>>>>>>>>> seems improved. I >>>>>>>>>>>>> believe that the brand new Python installers will make it >>>>>>>>>>>>> easier to debug >>>>>>>>>>>>> issues with dogtag installation when they come. When I >>>>>>>>>>>>> tried our unit tests or >>>>>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>>>>> >>>>>>>>>>>>> Now the bad news, or rather few issues and suggestions I >>>>>>>>>>>>> found: >>>>>>>>>>>>> >>>>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled >>>>>>>>>>>>> automatically on >>>>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a >>>>>>>>>>>>> Requires. >>>>>>>>>>>>> >>>>>>>>>>>> We have a dogtag patch that is currently in review that will >>>>>>>>>>>> address >>>>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required >>>>>>>>>>>> for f17+, >>>>>>>>>>>> rather than f18+ >>>>>>>>>>>> >>>>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA >>>>>>>>>>>>> installation on a >>>>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this >>>>>>>>>>>>> expectable? >>>>>>>>>>>>> >>>>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag >>>>>>>>>>>> 10 only, >>>>>>>>>>>> which will be officially available on f18+. So if you update >>>>>>>>>>>> to dogtag >>>>>>>>>>>> 10, you must have this patch and visa versa. We probably >>>>>>>>>>>> need to add >>>>>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>>>>> >>>>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to >>>>>>>>>>>> the new >>>>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue >>>>>>>>>>>> to work. >>>>>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>>>>> >>>>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had >>>>>>>>>>>>> dogtag10 as well >>>>>>>>>>>>> and I got the following error: >>>>>>>>>>>>> >>>>>>>>>>>>> # ipa-ca-install >>>>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>>>>> ... >>>>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 >>>>>>>>>>>>> seconds >>>>>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>>>>> >>>>>>>>>>>>> Your system may be partly configured. >>>>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>>>>> >>>>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log >>>>>>>>>>>>> for details: >>>>>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>>>>> >>>>>>>>>>>>> Root cause: >>>>>>>>>>>>> ... >>>>>>>>>>>>> File >>>>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>>>>>>>> >>>>>>>>>>>>> line >>>>>>>>>>>>> 625, in __spawn_instance >>>>>>>>>>>>> "/root/cacert.p12") >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, >>>>>>>>>>>> rather than >>>>>>>>>>>> ipa-ca-install to create replicas. I didn't know >>>>>>>>>>>> ipa-ca-install >>>>>>>>>>>> existed! Should not be too bad to fix though - most likely >>>>>>>>>>>> just need to >>>>>>>>>>>> move files to the right place. >>>>>>>>>>> >>>>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, >>>>>>>>>>> you probably used >>>>>>>>>>> that one) and the aforementioned ipa-ca-install run after >>>>>>>>>>> ipa-replica-install. >>>>>>>>>>> >>>>>>>>>> I will be testing this out again. As ipa-ca-install uses the >>>>>>>>>> same code >>>>>>>>>> as ipa-replica-install, I would have expected it to work. Did >>>>>>>>>> you try >>>>>>>>>> it with selinux in permissive mode? >>>>>>>>>> >>>>>>>>> OK -- I tested this out and realized that between the time I >>>>>>>>> initially >>>>>>>>> tested this and your tests, we had changed a URL >>>>>>>>> from /ca/pki/installer/installToken to >>>>>>>>> /ca/rest/installer/installToken, >>>>>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>>>>> >>>>>>>>> The new patch has been rebased and changes the relevant patch. >>>>>>>>> With >>>>>>>>> this, the CA replica installs correctly. >>>>>>>> >>>>>>>> Umh, where can I find the new rebased patch? :-) >>>>>>>> >>>>>>>>> >>>>>>>>> There was one issue that I encountered. I have seen this once >>>>>>>>> before >>>>>>>>> and will need to investigate further. IPA sometimes restarts >>>>>>>>> the dogtag >>>>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then >>>>>>>>> systemctl >>>>>>>>> start pki-tomcatd.target. I have noticed that occasionally the >>>>>>>>> start >>>>>>>>> invocation fails, while the stop and restart invocations succeed >>>>>>>>> just >>>>>>>>> fine. This makes absolutely no sense because restart is >>>>>>>>> essentially >>>>>>>>> just stop and then start. >>>>>>>>> >>>>>>>>> I think this is actually a bug in systemd. If I reboot the >>>>>>>>> machine, it >>>>>>>>> all works perfectly. If we can't figure out whats going on with >>>>>>>>> systemd, we can always replace this start/stop with >>>>>>>>> starting/stopping >>>>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That >>>>>>>>> seems to >>>>>>>>> work all the time with no problems. >>>>>>>>> >>>>>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>>>>> >>>>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended >>>>>>>> in systemd to >>>>>>>> use target units this way? I thought that only service files are >>>>>>>> meant to be >>>>>>>> used for manipulation with a service. As you said yourself, using >>>>>>>> service unit >>>>>>>> file for restart worked every time. >>>>>>>> >>>>>>>> Martin >>>>>>> >>>>>> >>>>>> Now, I tested the rebased patch with VMs with permissive SELinux. >>>>>> IPA server >>>>>> install was OK, as always, but replica installation ended up with >>>>>> error. >>>>>> Although it got much further than before: >>>>>> >>>>>> # ipa-ca-install >>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>> ... >>>>>> [3/3]: restarting directory server >>>>>> done configuring pkids. >>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>>>> [1/14]: creating certificate server user >>>>>> [2/14]: configuring certificate server instance >>>>>> [3/14]: disabling nonces >>>>>> [4/14]: importing CA chain to RA certificate database >>>>>> [5/14]: fixing RA database permissions >>>>>> [6/14]: setting up signing cert profile >>>>>> [7/14]: set up CRL publishing >>>>>> [8/14]: set certificate subject base >>>>>> [9/14]: enabling Subject Key Identifier >>>>>> [10/14]: configuring certificate server to start on boot >>>>>> [11/14]: configure certmonger for renewals >>>>>> [12/14]: configure clone certificate renewals >>>>>> [13/14]: configure Server-Cert certificate renewal >>>>>> [14/14]: Configure HTTP to proxy connections >>>>>> done configuring pki-tomcatd. >>>>>> Restarting the directory and certificate servers >>>>>> >>>>>> Your system may be partly configured. >>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>> >>>>>> It seems like that CA on a replica did not start and so a check for >>>>>> port 8080 >>>>>> failed. Attached logs (pki-ca-logs.tgz) may provide more >>>>>> information to this issue. >>>>>> >>>>> This is the systemd issue I mentioned before. I will submit a patch >>>>> which will change the restart mechanism to restart the service and not >>>>> the target. >>>>>> >>>> >>>> Patch attached. This patch should be applied on top of the large patch >>>> (being reviewed). >>>> >>>>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>>>>> (permissive SELinux). Now, the service was upgraded smoothly, CA >>>>>> service >>>>>> could be restarted without failure. However, certificate operations >>>>>> now >>>>>> did not work: >>>>>> >>>>>> # ipactl restart >>>>>> Restarting Directory Service >>>>>> Restarting KDC Service >>>>>> Restarting KPASSWD Service >>>>>> Restarting MEMCACHE Service >>>>>> Restarting HTTP Service >>>>>> Restarting CA Service >>>>>> # ipactl status >>>>>> Directory Service: RUNNING >>>>>> KDC Service: RUNNING >>>>>> KPASSWD Service: RUNNING >>>>>> MEMCACHE Service: RUNNING >>>>>> HTTP Service: RUNNING >>>>>> CA Service: RUNNING >>>>>> # ipa cert-show 1 >>>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>>>>> communicate with CMS (Internal Server Error) >>>>>> >>>>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>>>>> >>>>> The reason for this is that the old dogtag instances are missing: >>>>> 1. a new parameter in CS.cfg >>>>> 2. a couple of symlinks >>>>> >>>>> I plan to fix this in the dogtag code. Specifically, >>>>> 1. I will modify the dogtag code to provide a default value in the >>>>> case >>>>> where the parameter does not exist. >>>>> 2. I plan to add a function to the dogtag startup scripts to check and >>>>> remake any required symlinks. >>>>> >>>>> Both of these changes are in the dogtag code, and do not affect this >>>>> patch. As a workaround, to confirm that the upgrade actually >>>>> works, do >>>>> the following manual steps on your upgraded instance: >>>>> >>>>> 1. Add the following parameter to CS.cfg >>>>> pidDir=/var/run >>>>> >>>>> 2. Add the following links; >>>>> cd /var/lib/pki-ca/common/lib >>>>> ln -s /usr/share/java/apache-commons-codec.jar >>>>> apache-commons-codec.jar >>>>> ln -s /usr/share/java/log4j.jar log4j.jar >>>>> >>>>> 3 Restart the dogtag instance >>>> >>> >>> To throw a little twist in here, we should make sure that the >>> certmonger restart scripts still work as well. >>> >>> rob >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> >> What is the situation with this patch? Was it commited? Or is it in >> works/review? Was is superseded by another patch (that I have not got to >> yet and thus asking)? >> > > I am finishing and testing a patch to apply on top, which will make it > possible to use either Dogtag 9 or 10 depending on what's installed. > I don't know if the CA renewal issue has been addressed yet. It may impact certmonger because the ports are hardcoded in the certmonger source. I was under the impression Ade was looking into this and would open a certmonger bug as needed. rob From mkosek at redhat.com Mon Aug 27 14:55:33 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 16:55:33 +0200 Subject: [Freeipa-devel] [PATCH] 0006 Removes sssd.conf after uninstall. In-Reply-To: <20120827133727.GG8612@zeppelin.brq.redhat.com> References: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> <503B5D09.7040401@redhat.com> <503B6EC8.3060508@redhat.com> <20120827133727.GG8612@zeppelin.brq.redhat.com> Message-ID: <503B8A65.30100@redhat.com> On 08/27/2012 03:37 PM, Jakub Hrozek wrote: > On Mon, Aug 27, 2012 at 02:57:44PM +0200, Martin Kosek wrote: >> I think that the right behavior of SSSD conf uninstall should be the following: >> >> * sssd.conf existed before IPA install + non-IPA domains in sssd.conf found: >> - move backed conf up sssd.conf.bkp (and inform the user) >> - use SSSDConfig delete_domain function to remove ipa domain from sssd.conf >> - restart sssd afterwards > > I'm confused here, which of the files is the original > pre-ipa-client-install file? This is the "backed up sssd.conf". I thought that it may be useful for user to still have an access to it after uninstall. > > How does the non-ipa domain end up in the sssd.conf file? Does it have > to be configured manually or does ipa-client-install merge the list of > domains on installation? ipa-client-install merge the list of the domains. It overrides the old sssd.conf only when we cannot parse the sssd.conf and --preserve-sssd option was not set. Martin From rmeggins at redhat.com Mon Aug 27 15:25:07 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 27 Aug 2012 09:25:07 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503B6B11.8070406@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B6B11.8070406@redhat.com> Message-ID: <503B9153.7030709@redhat.com> On 08/27/2012 06:41 AM, Dmitri Pal wrote: > On 08/17/2012 10:00 AM, Rich Megginson wrote: >> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>> Hi guys, >>> >>> I am now investigating ticket #2866: >>> https://fedorahosted.org/freeipa/ticket/2866 >>> >>> And I am thinking about possible solutions for this problem. In a >>> nutshell, we do not properly check referential integrity in some IPA >>> objects where we keep one-way DN references to other objects, e.g. in >>> - managedBy attribute for a host object >>> - memberhost attribute for HBAC rule object >>> - memberuser attribute for user object >>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>> #2866) >>> ... >>> >>> Currently, I see 2 approaches to solve this: >>> 1) Add relevant checks to our ipalib plugins where problematic >>> operations with these operations are being executed (like we do for >>> selinuxusermap's seealso attribute in HBAC plugin) >>> This of course would not prevent direct LDAP deletes. >>> >>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>> callbacks and check that this object's DN is not referenced in other >>> objects. And if it does, it would reject such modification. Second >>> option would be to delete the attribute value with now invalid >>> reference. This would be probably more suitable for example for >>> references to user objects. >>> >>> Any comments to these possible approaches are welcome. >>> >>> Rich, do you think that as an alternative to these 2 approaches, >>> memberOf plugin could be eventually modified to do this task? >> This is very similar to the referential integrity plugin already in >> 389, except instead of cleaning up references to moved and deleted >> entries, you want it to prevent moving or deleting an entry if that >> entry is referenced by the >> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some >> other entry. >> >> Note that the managed entry plugin (mep) already handles this for the >> managedby attribute. >> >> Are you already using the memberof plugin for >> memberhost/memberuser/memberallowcmd/memberdenycmd? >> >> This doesn't seem like a job for memberof, this seems like more of a >> new check for the referential integrity plugin. > Did it translate into a DS ticket? No. Is there an IPA ticket to link to? > I suspect it is not a big change and would solve a bunch of ugly > referential integrity problems. > >>> Thank you, >>> Martin >>> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > From pvoborni at redhat.com Mon Aug 27 15:51:15 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 27 Aug 2012 17:51:15 +0200 Subject: [Freeipa-devel] [PATCH] 202 Password policy paging with proper sorting Message-ID: <503B9773.2080904@redhat.com> This patch adds option to disable sorting when paging. It allowed to enable paging in password policy with order of items untouched (they are sorted on server side by priority). Also fixing issue when paging is disabled and command summary = null. It displayed 'null' in facet footer. https://fedorahosted.org/freeipa/ticket/2677 Note: personally I would left paging disabled in password policy page. I don't think it is beneficial. It slows things down and there would be hardly more than 100 policies. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0202-Password-policy-paging-with-proper-sorting.patch Type: text/x-patch Size: 2599 bytes Desc: not available URL: From rmeggins at redhat.com Mon Aug 27 15:53:47 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 27 Aug 2012 09:53:47 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503B9153.7030709@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B6B11.8070406@redhat.com> <503B9153.7030709@redhat.com> Message-ID: <503B980B.5080307@redhat.com> On 08/27/2012 09:25 AM, Rich Megginson wrote: > On 08/27/2012 06:41 AM, Dmitri Pal wrote: >> On 08/17/2012 10:00 AM, Rich Megginson wrote: >>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>> Hi guys, >>>> >>>> I am now investigating ticket #2866: >>>> https://fedorahosted.org/freeipa/ticket/2866 >>>> >>>> And I am thinking about possible solutions for this problem. In a >>>> nutshell, we do not properly check referential integrity in some IPA >>>> objects where we keep one-way DN references to other objects, e.g. in >>>> - managedBy attribute for a host object >>>> - memberhost attribute for HBAC rule object >>>> - memberuser attribute for user object >>>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>>> #2866) >>>> ... >>>> >>>> Currently, I see 2 approaches to solve this: >>>> 1) Add relevant checks to our ipalib plugins where problematic >>>> operations with these operations are being executed (like we do for >>>> selinuxusermap's seealso attribute in HBAC plugin) >>>> This of course would not prevent direct LDAP deletes. >>>> >>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>> callbacks and check that this object's DN is not referenced in other >>>> objects. And if it does, it would reject such modification. Second >>>> option would be to delete the attribute value with now invalid >>>> reference. This would be probably more suitable for example for >>>> references to user objects. >>>> >>>> Any comments to these possible approaches are welcome. >>>> >>>> Rich, do you think that as an alternative to these 2 approaches, >>>> memberOf plugin could be eventually modified to do this task? >>> This is very similar to the referential integrity plugin already in >>> 389, except instead of cleaning up references to moved and deleted >>> entries, you want it to prevent moving or deleting an entry if that >>> entry is referenced by the >>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some >>> other entry. >>> >>> Note that the managed entry plugin (mep) already handles this for the >>> managedby attribute. >>> >>> Are you already using the memberof plugin for >>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>> >>> This doesn't seem like a job for memberof, this seems like more of a >>> new check for the referential integrity plugin. >> Did it translate into a DS ticket? > No. Is there an IPA ticket to link to? https://fedorahosted.org/389/ticket/438 >> I suspect it is not a big change and would solve a bunch of ugly >> referential integrity problems. >> >>>> Thank you, >>>> Martin >>>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From mkosek at redhat.com Mon Aug 27 16:24:46 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 18:24:46 +0200 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <502E4E78.1090407@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> Message-ID: <503B9F4E.4010005@redhat.com> On 08/17/2012 04:00 PM, Rich Megginson wrote: > On 08/17/2012 07:44 AM, Martin Kosek wrote: >> Hi guys, >> >> I am now investigating ticket #2866: >> https://fedorahosted.org/freeipa/ticket/2866 >> >> And I am thinking about possible solutions for this problem. In a >> nutshell, we do not properly check referential integrity in some IPA >> objects where we keep one-way DN references to other objects, e.g. in >> - managedBy attribute for a host object >> - memberhost attribute for HBAC rule object >> - memberuser attribute for user object >> - memberallowcmd or memberdenycmd for SUDO command object (reported in >> #2866) >> ... >> >> Currently, I see 2 approaches to solve this: >> 1) Add relevant checks to our ipalib plugins where problematic >> operations with these operations are being executed (like we do for >> selinuxusermap's seealso attribute in HBAC plugin) >> This of course would not prevent direct LDAP deletes. >> >> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >> callbacks and check that this object's DN is not referenced in other >> objects. And if it does, it would reject such modification. Second >> option would be to delete the attribute value with now invalid >> reference. This would be probably more suitable for example for >> references to user objects. >> >> Any comments to these possible approaches are welcome. >> >> Rich, do you think that as an alternative to these 2 approaches, >> memberOf plugin could be eventually modified to do this task? > > This is very similar to the referential integrity plugin already in 389, except > instead of cleaning up references to moved and deleted entries, you want it to > prevent moving or deleting an entry if that entry is referenced by the > managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other entry. I think that using or enhancing current DS referential integrity plugin will be the best and the most consistent way to go. We already use that plugin for some user attributes like "manager" or "secretary". seeAlso is already covered by default, so for example seeAlso attribute in SELinux usermap object referencing an HBAC rule will get removed when relevant HBAC rule is removed (I just checked that). > > Note that the managed entry plugin (mep) already handles this for the managedby > attribute. I assume you are referencing "mepmanagedby" and "mepmanagedentry" attributes which then produce errors like this one: # ipa netgroup-del foo ipa: ERROR: Server is unwilling to perform: Deleting a managed entry is not allowed. It needs to be manually unlinked first. managedBy attribute used by hosts objects I had in mind seems to not be covered. But you are right, this is pretty much what I wanted. Though in case of MEP, there is a link in both referenced objects, but in our case, we have just the one-way link. > > Are you already using the memberof plugin for > memberhost/memberuser/memberallowcmd/memberdenycmd? > > This doesn't seem like a job for memberof, this seems like more of a new check > for the referential integrity plugin. > I am now considering if current move/delete clean up already present in Referential Integrity plugin would not be sufficient for us. Rich, please correct me if I am wrong, but in that case, we would just need to add relevant attribute names (memberhost/memberuser/memberallowcmd/memberdenycmd...) to Referential Integrity plugin configuration as nsslapd-pluginarg7, nsslapd-pluginarg8, ... I wonder if there would be some performance issues if we add attributes to the list this way. Rob, do you think that cleaning up the broken references during a DS postop instead of raising an preop error is OK for IPA references? I went through the referential attributes we use ("git grep LDAPAddMember") and I think it should be sufficient. We could cover some special cases with a query in our framework like you did in hbacrule-del. Martin From rmeggins at redhat.com Mon Aug 27 16:29:25 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 27 Aug 2012 10:29:25 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503B9F4E.4010005@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> Message-ID: <503BA065.1060207@redhat.com> On 08/27/2012 10:24 AM, Martin Kosek wrote: > On 08/17/2012 04:00 PM, Rich Megginson wrote: >> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>> Hi guys, >>> >>> I am now investigating ticket #2866: >>> https://fedorahosted.org/freeipa/ticket/2866 >>> >>> And I am thinking about possible solutions for this problem. In a >>> nutshell, we do not properly check referential integrity in some IPA >>> objects where we keep one-way DN references to other objects, e.g. in >>> - managedBy attribute for a host object >>> - memberhost attribute for HBAC rule object >>> - memberuser attribute for user object >>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>> #2866) >>> ... >>> >>> Currently, I see 2 approaches to solve this: >>> 1) Add relevant checks to our ipalib plugins where problematic >>> operations with these operations are being executed (like we do for >>> selinuxusermap's seealso attribute in HBAC plugin) >>> This of course would not prevent direct LDAP deletes. >>> >>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>> callbacks and check that this object's DN is not referenced in other >>> objects. And if it does, it would reject such modification. Second >>> option would be to delete the attribute value with now invalid >>> reference. This would be probably more suitable for example for >>> references to user objects. >>> >>> Any comments to these possible approaches are welcome. >>> >>> Rich, do you think that as an alternative to these 2 approaches, >>> memberOf plugin could be eventually modified to do this task? >> This is very similar to the referential integrity plugin already in 389, except >> instead of cleaning up references to moved and deleted entries, you want it to >> prevent moving or deleting an entry if that entry is referenced by the >> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other entry. > I think that using or enhancing current DS referential integrity plugin will be > the best and the most consistent way to go. > > We already use that plugin for some user attributes like "manager" or > "secretary". seeAlso is already covered by default, so for example seeAlso > attribute in SELinux usermap object referencing an HBAC rule will get removed > when relevant HBAC rule is removed (I just checked that). > >> Note that the managed entry plugin (mep) already handles this for the managedby >> attribute. > I assume you are referencing "mepmanagedby" and "mepmanagedentry" attributes > which then produce errors like this one: > > # ipa netgroup-del foo > ipa: ERROR: Server is unwilling to perform: Deleting a managed entry is not > allowed. It needs to be manually unlinked first. > > managedBy attribute used by hosts objects I had in mind seems to not be covered. > > But you are right, this is pretty much what I wanted. Though in case of MEP, > there is a link in both referenced objects, but in our case, we have just the > one-way link. > >> Are you already using the memberof plugin for >> memberhost/memberuser/memberallowcmd/memberdenycmd? >> >> This doesn't seem like a job for memberof, this seems like more of a new check >> for the referential integrity plugin. >> > I am now considering if current move/delete clean up already present in > Referential Integrity plugin would not be sufficient for us. > > Rich, please correct me if I am wrong, but in that case, we would just need to > add relevant attribute names > (memberhost/memberuser/memberallowcmd/memberdenycmd...) to Referential > Integrity plugin configuration as nsslapd-pluginarg7, nsslapd-pluginarg8, ... > I wonder if there would be some performance issues if we add attributes to the > list this way. No, not if they are indexed for presence and equality. But referential integrity will not prevent deletion or moving entries - it will delete/move references. > > Rob, do you think that cleaning up the broken references during a DS postop > instead of raising an preop error is OK for IPA references? I went through the > referential attributes we use ("git grep LDAPAddMember") and I think it should > be sufficient. We could cover some special cases with a query in our framework > like you did in hbacrule-del. > > Martin From jdennis at redhat.com Mon Aug 27 16:39:12 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 27 Aug 2012 12:39:12 -0400 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503B9F4E.4010005@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> Message-ID: <503BA2B0.6030905@redhat.com> Just out of curiosity, I saw something this weekend while testing and I'm wondering if it's expected behavior or if referential integrity would address it. I was able to add a non-existent user to a group. Shouldn't that have been an error? Do we check for that in the ldap pre callback? Do we intend for referential integrity to catch those sorts of things? Or do we allow non-existent users to be members of group for some reason? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Mon Aug 27 17:12:06 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 27 Aug 2012 13:12:06 -0400 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503B9153.7030709@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B6B11.8070406@redhat.com> <503B9153.7030709@redhat.com> Message-ID: <503BAA66.2030804@redhat.com> Rich Megginson wrote: > On 08/27/2012 06:41 AM, Dmitri Pal wrote: >> On 08/17/2012 10:00 AM, Rich Megginson wrote: >>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>> Hi guys, >>>> >>>> I am now investigating ticket #2866: >>>> https://fedorahosted.org/freeipa/ticket/2866 >>>> >>>> And I am thinking about possible solutions for this problem. In a >>>> nutshell, we do not properly check referential integrity in some IPA >>>> objects where we keep one-way DN references to other objects, e.g. in >>>> - managedBy attribute for a host object >>>> - memberhost attribute for HBAC rule object >>>> - memberuser attribute for user object >>>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>>> #2866) >>>> ... >>>> >>>> Currently, I see 2 approaches to solve this: >>>> 1) Add relevant checks to our ipalib plugins where problematic >>>> operations with these operations are being executed (like we do for >>>> selinuxusermap's seealso attribute in HBAC plugin) >>>> This of course would not prevent direct LDAP deletes. >>>> >>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>> callbacks and check that this object's DN is not referenced in other >>>> objects. And if it does, it would reject such modification. Second >>>> option would be to delete the attribute value with now invalid >>>> reference. This would be probably more suitable for example for >>>> references to user objects. >>>> >>>> Any comments to these possible approaches are welcome. >>>> >>>> Rich, do you think that as an alternative to these 2 approaches, >>>> memberOf plugin could be eventually modified to do this task? >>> This is very similar to the referential integrity plugin already in >>> 389, except instead of cleaning up references to moved and deleted >>> entries, you want it to prevent moving or deleting an entry if that >>> entry is referenced by the >>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some >>> other entry. >>> >>> Note that the managed entry plugin (mep) already handles this for the >>> managedby attribute. >>> >>> Are you already using the memberof plugin for >>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>> >>> This doesn't seem like a job for memberof, this seems like more of a >>> new check for the referential integrity plugin. >> Did it translate into a DS ticket? > No. Is there an IPA ticket to link to? Not yet. I wonder if we need to flesh out what this means (and may mean) going forward. Are there any downsides to linking entries in this way. rob From rmeggins at redhat.com Mon Aug 27 17:41:08 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 27 Aug 2012 11:41:08 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503BAA66.2030804@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B6B11.8070406@redhat.com> <503B9153.7030709@redhat.com> <503BAA66.2030804@redhat.com> Message-ID: <503BB134.9060001@redhat.com> On 08/27/2012 11:12 AM, Rob Crittenden wrote: > Rich Megginson wrote: >> On 08/27/2012 06:41 AM, Dmitri Pal wrote: >>> On 08/17/2012 10:00 AM, Rich Megginson wrote: >>>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>>> Hi guys, >>>>> >>>>> I am now investigating ticket #2866: >>>>> https://fedorahosted.org/freeipa/ticket/2866 >>>>> >>>>> And I am thinking about possible solutions for this problem. In a >>>>> nutshell, we do not properly check referential integrity in some IPA >>>>> objects where we keep one-way DN references to other objects, e.g. in >>>>> - managedBy attribute for a host object >>>>> - memberhost attribute for HBAC rule object >>>>> - memberuser attribute for user object >>>>> - memberallowcmd or memberdenycmd for SUDO command object >>>>> (reported in >>>>> #2866) >>>>> ... >>>>> >>>>> Currently, I see 2 approaches to solve this: >>>>> 1) Add relevant checks to our ipalib plugins where problematic >>>>> operations with these operations are being executed (like we do for >>>>> selinuxusermap's seealso attribute in HBAC plugin) >>>>> This of course would not prevent direct LDAP deletes. >>>>> >>>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>>> callbacks and check that this object's DN is not referenced in other >>>>> objects. And if it does, it would reject such modification. Second >>>>> option would be to delete the attribute value with now invalid >>>>> reference. This would be probably more suitable for example for >>>>> references to user objects. >>>>> >>>>> Any comments to these possible approaches are welcome. >>>>> >>>>> Rich, do you think that as an alternative to these 2 approaches, >>>>> memberOf plugin could be eventually modified to do this task? >>>> This is very similar to the referential integrity plugin already in >>>> 389, except instead of cleaning up references to moved and deleted >>>> entries, you want it to prevent moving or deleting an entry if that >>>> entry is referenced by the >>>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some >>>> other entry. >>>> >>>> Note that the managed entry plugin (mep) already handles this for the >>>> managedby attribute. >>>> >>>> Are you already using the memberof plugin for >>>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>>> >>>> This doesn't seem like a job for memberof, this seems like more of a >>>> new check for the referential integrity plugin. >>> Did it translate into a DS ticket? >> No. Is there an IPA ticket to link to? > > Not yet. I wonder if we need to flesh out what this means (and may > mean) going forward. Are there any downsides to linking entries in > this way. Performance is one downside. Although with the work that Noriko is doing on transactions, this may not be as big an issue. SQL databases have long supported this sort of referential integrity checking, so it makes sense to put this sort of "business logic" in the database. > > rob From rmeggins at redhat.com Mon Aug 27 17:41:53 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 27 Aug 2012 11:41:53 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503BA2B0.6030905@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> <503BA2B0.6030905@redhat.com> Message-ID: <503BB161.1020901@redhat.com> On 08/27/2012 10:39 AM, John Dennis wrote: > Just out of curiosity, I saw something this weekend while testing and > I'm wondering if it's expected behavior or if referential integrity > would address it. > > I was able to add a non-existent user to a group. Shouldn't that have > been an error? Do we check for that in the ldap pre callback? Do we > intend for referential integrity to catch those sorts of things? No, no, and no. > > Or do we allow non-existent users to be members of group for some reason? > Yes, but not for some reason, but because it is allowed by LDAP. From dpal at redhat.com Mon Aug 27 18:34:10 2012 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 27 Aug 2012 14:34:10 -0400 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503B9153.7030709@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B6B11.8070406@redhat.com> <503B9153.7030709@redhat.com> Message-ID: <503BBDA2.8000400@redhat.com> On 08/27/2012 11:25 AM, Rich Megginson wrote: > On 08/27/2012 06:41 AM, Dmitri Pal wrote: >> On 08/17/2012 10:00 AM, Rich Megginson wrote: >>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>> Hi guys, >>>> >>>> I am now investigating ticket #2866: >>>> https://fedorahosted.org/freeipa/ticket/2866 >>>> >>>> And I am thinking about possible solutions for this problem. In a >>>> nutshell, we do not properly check referential integrity in some IPA >>>> objects where we keep one-way DN references to other objects, e.g. in >>>> - managedBy attribute for a host object >>>> - memberhost attribute for HBAC rule object >>>> - memberuser attribute for user object >>>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>>> #2866) >>>> ... >>>> >>>> Currently, I see 2 approaches to solve this: >>>> 1) Add relevant checks to our ipalib plugins where problematic >>>> operations with these operations are being executed (like we do for >>>> selinuxusermap's seealso attribute in HBAC plugin) >>>> This of course would not prevent direct LDAP deletes. >>>> >>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>> callbacks and check that this object's DN is not referenced in other >>>> objects. And if it does, it would reject such modification. Second >>>> option would be to delete the attribute value with now invalid >>>> reference. This would be probably more suitable for example for >>>> references to user objects. >>>> >>>> Any comments to these possible approaches are welcome. >>>> >>>> Rich, do you think that as an alternative to these 2 approaches, >>>> memberOf plugin could be eventually modified to do this task? >>> This is very similar to the referential integrity plugin already in >>> 389, except instead of cleaning up references to moved and deleted >>> entries, you want it to prevent moving or deleting an entry if that >>> entry is referenced by the >>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some >>> other entry. >>> >>> Note that the managed entry plugin (mep) already handles this for the >>> managedby attribute. >>> >>> Are you already using the memberof plugin for >>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>> >>> This doesn't seem like a job for memberof, this seems like more of a >>> new check for the referential integrity plugin. >> Did it translate into a DS ticket? > No. Is there an IPA ticket to link to? Yes, the one at the top of this email. >> I suspect it is not a big change and would solve a bunch of ugly >> referential integrity problems. >> >>>> Thank you, >>>> Martin >>>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Mon Aug 27 20:27:16 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 27 Aug 2012 22:27:16 +0200 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503BA065.1060207@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> <503BA065.1060207@redhat.com> Message-ID: <1346099236.3617.6.camel@priserak> On Mon, 2012-08-27 at 10:29 -0600, Rich Megginson wrote: > On 08/27/2012 10:24 AM, Martin Kosek wrote: > > On 08/17/2012 04:00 PM, Rich Megginson wrote: > >> On 08/17/2012 07:44 AM, Martin Kosek wrote: > >>> Hi guys, > >>> > >>> I am now investigating ticket #2866: > >>> https://fedorahosted.org/freeipa/ticket/2866 > >>> > >>> And I am thinking about possible solutions for this problem. In a > >>> nutshell, we do not properly check referential integrity in some IPA > >>> objects where we keep one-way DN references to other objects, e.g. in > >>> - managedBy attribute for a host object > >>> - memberhost attribute for HBAC rule object > >>> - memberuser attribute for user object > >>> - memberallowcmd or memberdenycmd for SUDO command object (reported in > >>> #2866) > >>> ... > >>> > >>> Currently, I see 2 approaches to solve this: > >>> 1) Add relevant checks to our ipalib plugins where problematic > >>> operations with these operations are being executed (like we do for > >>> selinuxusermap's seealso attribute in HBAC plugin) > >>> This of course would not prevent direct LDAP deletes. > >>> > >>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE > >>> callbacks and check that this object's DN is not referenced in other > >>> objects. And if it does, it would reject such modification. Second > >>> option would be to delete the attribute value with now invalid > >>> reference. This would be probably more suitable for example for > >>> references to user objects. > >>> > >>> Any comments to these possible approaches are welcome. > >>> > >>> Rich, do you think that as an alternative to these 2 approaches, > >>> memberOf plugin could be eventually modified to do this task? > >> This is very similar to the referential integrity plugin already in 389, except > >> instead of cleaning up references to moved and deleted entries, you want it to > >> prevent moving or deleting an entry if that entry is referenced by the > >> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other entry. > > I think that using or enhancing current DS referential integrity plugin will be > > the best and the most consistent way to go. > > > > We already use that plugin for some user attributes like "manager" or > > "secretary". seeAlso is already covered by default, so for example seeAlso > > attribute in SELinux usermap object referencing an HBAC rule will get removed > > when relevant HBAC rule is removed (I just checked that). > > > >> Note that the managed entry plugin (mep) already handles this for the managedby > >> attribute. > > I assume you are referencing "mepmanagedby" and "mepmanagedentry" attributes > > which then produce errors like this one: > > > > # ipa netgroup-del foo > > ipa: ERROR: Server is unwilling to perform: Deleting a managed entry is not > > allowed. It needs to be manually unlinked first. > > > > managedBy attribute used by hosts objects I had in mind seems to not be covered. > > > > But you are right, this is pretty much what I wanted. Though in case of MEP, > > there is a link in both referenced objects, but in our case, we have just the > > one-way link. > > > >> Are you already using the memberof plugin for > >> memberhost/memberuser/memberallowcmd/memberdenycmd? > >> > >> This doesn't seem like a job for memberof, this seems like more of a new check > >> for the referential integrity plugin. > >> > > I am now considering if current move/delete clean up already present in > > Referential Integrity plugin would not be sufficient for us. > > > > Rich, please correct me if I am wrong, but in that case, we would just need to > > add relevant attribute names > > (memberhost/memberuser/memberallowcmd/memberdenycmd...) to Referential > > Integrity plugin configuration as nsslapd-pluginarg7, nsslapd-pluginarg8, ... > > I wonder if there would be some performance issues if we add attributes to the > > list this way. > No, not if they are indexed for presence and equality. > > But referential integrity will not prevent deletion or moving entries - > it will delete/move references. I understand that. After some reconsideration, I think that cleaning up dangling references as postop should be OK for most of the referential attributes we use. But I would like a second opinion on that. So do I understand it correctly, that in case we want to go this way in IPA, the recommended approach DS-wise would be to: - add presence and equality indexes to IPA for the attributes we want to have checked for referential integrity - configure DS Referential Integrity plugin to check these attributes Or would it be better to wait on relevant DS changes you mentioned that Noriko is working on? Thanks, Martin > > > > Rob, do you think that cleaning up the broken references during a DS postop > > instead of raising an preop error is OK for IPA references? I went through the > > referential attributes we use ("git grep LDAPAddMember") and I think it should > > be sufficient. We could cover some special cases with a query in our framework > > like you did in hbacrule-del. > > > > Martin > From rcritten at redhat.com Mon Aug 27 20:27:57 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 27 Aug 2012 16:27:57 -0400 Subject: [Freeipa-devel] [PATCH] 295 Fix managedBy label for DNS zone In-Reply-To: <1345194461.3796.7.camel@priserak> References: <1345194461.3796.7.camel@priserak> Message-ID: <503BD84D.8050908@redhat.com> Martin Kosek wrote: > Even though managedBy output parameter was only used for failed host > managedBy memberships, it was defined in global baseldap.py > classes. Incorrect label was then being displayed also for DNS zone > per-zone permission attribute with the same name. > > Move managedBy output parameter to host plugin. Define proper managedBy > output parameter in DNS plugin to improve clarity of this attribute. > > https://fedorahosted.org/freeipa/ticket/2946 ACK pushed to master and ipa-3-0 From rcritten at redhat.com Mon Aug 27 20:32:12 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 27 Aug 2012 16:32:12 -0400 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503BB161.1020901@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> <503BA2B0.6030905@redhat.com> <503BB161.1020901@redhat.com> Message-ID: <503BD94C.8030404@redhat.com> Rich Megginson wrote: > On 08/27/2012 10:39 AM, John Dennis wrote: >> Just out of curiosity, I saw something this weekend while testing and >> I'm wondering if it's expected behavior or if referential integrity >> would address it. >> >> I was able to add a non-existent user to a group. Shouldn't that have >> been an error? Do we check for that in the ldap pre callback? Do we >> intend for referential integrity to catch those sorts of things? > > No, no, and no. > >> >> Or do we allow non-existent users to be members of group for some reason? >> > Yes, but not for some reason, but because it is allowed by LDAP. IPA is supposed to prevent this, we don't allow non-users to be members of groups. I'd recommend looking for the LDAP logs of when this add occurred. rob From rmeggins at redhat.com Mon Aug 27 21:32:39 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 27 Aug 2012 15:32:39 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <1346099236.3617.6.camel@priserak> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> <503BA065.1060207@redhat.com> <1346099236.3617.6.camel@priserak> Message-ID: <503BE777.9040204@redhat.com> On 08/27/2012 02:27 PM, Martin Kosek wrote: > On Mon, 2012-08-27 at 10:29 -0600, Rich Megginson wrote: >> On 08/27/2012 10:24 AM, Martin Kosek wrote: >>> On 08/17/2012 04:00 PM, Rich Megginson wrote: >>>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>>> Hi guys, >>>>> >>>>> I am now investigating ticket #2866: >>>>> https://fedorahosted.org/freeipa/ticket/2866 >>>>> >>>>> And I am thinking about possible solutions for this problem. In a >>>>> nutshell, we do not properly check referential integrity in some IPA >>>>> objects where we keep one-way DN references to other objects, e.g. in >>>>> - managedBy attribute for a host object >>>>> - memberhost attribute for HBAC rule object >>>>> - memberuser attribute for user object >>>>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>>>> #2866) >>>>> ... >>>>> >>>>> Currently, I see 2 approaches to solve this: >>>>> 1) Add relevant checks to our ipalib plugins where problematic >>>>> operations with these operations are being executed (like we do for >>>>> selinuxusermap's seealso attribute in HBAC plugin) >>>>> This of course would not prevent direct LDAP deletes. >>>>> >>>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>>> callbacks and check that this object's DN is not referenced in other >>>>> objects. And if it does, it would reject such modification. Second >>>>> option would be to delete the attribute value with now invalid >>>>> reference. This would be probably more suitable for example for >>>>> references to user objects. >>>>> >>>>> Any comments to these possible approaches are welcome. >>>>> >>>>> Rich, do you think that as an alternative to these 2 approaches, >>>>> memberOf plugin could be eventually modified to do this task? >>>> This is very similar to the referential integrity plugin already in 389, except >>>> instead of cleaning up references to moved and deleted entries, you want it to >>>> prevent moving or deleting an entry if that entry is referenced by the >>>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other entry. >>> I think that using or enhancing current DS referential integrity plugin will be >>> the best and the most consistent way to go. >>> >>> We already use that plugin for some user attributes like "manager" or >>> "secretary". seeAlso is already covered by default, so for example seeAlso >>> attribute in SELinux usermap object referencing an HBAC rule will get removed >>> when relevant HBAC rule is removed (I just checked that). >>> >>>> Note that the managed entry plugin (mep) already handles this for the managedby >>>> attribute. >>> I assume you are referencing "mepmanagedby" and "mepmanagedentry" attributes >>> which then produce errors like this one: >>> >>> # ipa netgroup-del foo >>> ipa: ERROR: Server is unwilling to perform: Deleting a managed entry is not >>> allowed. It needs to be manually unlinked first. >>> >>> managedBy attribute used by hosts objects I had in mind seems to not be covered. >>> >>> But you are right, this is pretty much what I wanted. Though in case of MEP, >>> there is a link in both referenced objects, but in our case, we have just the >>> one-way link. >>> >>>> Are you already using the memberof plugin for >>>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>>> >>>> This doesn't seem like a job for memberof, this seems like more of a new check >>>> for the referential integrity plugin. >>>> >>> I am now considering if current move/delete clean up already present in >>> Referential Integrity plugin would not be sufficient for us. >>> >>> Rich, please correct me if I am wrong, but in that case, we would just need to >>> add relevant attribute names >>> (memberhost/memberuser/memberallowcmd/memberdenycmd...) to Referential >>> Integrity plugin configuration as nsslapd-pluginarg7, nsslapd-pluginarg8, ... >>> I wonder if there would be some performance issues if we add attributes to the >>> list this way. >> No, not if they are indexed for presence and equality. >> >> But referential integrity will not prevent deletion or moving entries - >> it will delete/move references. > I understand that. After some reconsideration, I think that cleaning up > dangling references as postop should be OK for most of the referential > attributes we use. But I would like a second opinion on that. > > So do I understand it correctly, that in case we want to go this way in > IPA, the recommended approach DS-wise would be to: > - add presence and equality indexes to IPA for the attributes we want to > have checked for referential integrity > - configure DS Referential Integrity plugin to check these attributes > > Or would it be better to wait on relevant DS changes you mentioned that > Noriko is working on? Also look at the Linked Attribute plugin - it may be able to do what you want right now - http://port389.org/wiki/Linked_Attributes_Design > > Thanks, > Martin > >>> Rob, do you think that cleaning up the broken references during a DS postop >>> instead of raising an preop error is OK for IPA references? I went through the >>> referential attributes we use ("git grep LDAPAddMember") and I think it should >>> be sufficient. We could cover some special cases with a query in our framework >>> like you did in hbacrule-del. >>> >>> Martin > From pspacek at redhat.com Tue Aug 28 06:51:31 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 28 Aug 2012 08:51:31 +0200 Subject: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query() In-Reply-To: <20120822133503.GA7769@redhat.com> References: <5028FE08.4060207@redhat.com> <20120822133503.GA7769@redhat.com> Message-ID: <503C6A73.4060101@redhat.com> On 08/22/2012 03:35 PM, Adam Tkac wrote: > On Mon, Aug 13, 2012 at 03:15:52PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch improves connection management in bind-dyndb-ldap and closes >> https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . >> >> It should prevent all deadlocks on connection pool in future. > > Ack, just check my pedantic comments below, please. I partially disagree with one comment below. Amended patch is attached. > >> From 0cd91def54ea9ac92e25ee50e54c5e55034e2c47 Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Mon, 13 Aug 2012 15:06:50 +0200 >> Subject: [PATCH] Avoid manual connection management outside ldap_query() >> >> https://fedorahosted.org/bind-dyndb-ldap/ticket/68 >> >> Signed-off-by: Petr Spacek >> --- >> src/ldap_helper.c | 153 +++++++++++++++++++++++++++++++----------------------- >> 1 file changed, 87 insertions(+), 66 deletions(-) >> >> diff --git a/src/ldap_helper.c b/src/ldap_helper.c >> index c34b881a8430980f41eb02d2bb0f0229421d7fa1..fdc629e1c0cd1fabde27d887e391ef81823453c1 100644 >> --- a/src/ldap_helper.c >> +++ b/src/ldap_helper.c >> @@ -1713,11 +1707,15 @@ ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, >> int ret; >> int ldap_err_code; >> int once = 0; >> + isc_boolean_t autoconn = (ldap_conn == NULL); >> >> - REQUIRE(ldap_conn != NULL); >> + REQUIRE(ldap_inst != NULL); >> + REQUIRE(base != NULL); >> REQUIRE(ldap_qresultp != NULL && *ldap_qresultp == NULL); >> >> CHECK(ldap_query_create(ldap_inst->mctx, &ldap_qresult)); >> + if (autoconn) >> + CHECK(ldap_pool_getconnection(ldap_inst->pool, &ldap_conn)); >> >> va_start(ap, filter); >> str_vsprintf(ldap_qresult->query_string, filter, ap); >> @@ -1751,29 +1749,34 @@ retry: >> &ldap_qresult->ldap_entries); >> if (result != ISC_R_SUCCESS) { >> log_error("failed to save LDAP query results"); >> - goto cleanup; > > I would recommend to leave this goto here. In future, someone might add > code before "cleanup:" label and can forget to add "goto cleanup". Ok. > >> } >> + /* LDAP call suceeded, errors from ldap_entrylist_create() will be >> + * handled in cleanup section */ >> * refresh, retry, expire and minimum attributes for each SOA record. >> */ >> static isc_result_t >> -modify_soa_record(ldap_connection_t *ldap_conn, const char *zone_dn, >> - dns_rdata_t *rdata) >> +modify_soa_record(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, >> + const char *zone_dn, dns_rdata_t *rdata) >> { >> isc_result_t result; >> - isc_mem_t *mctx = ldap_conn->mctx; >> dns_rdata_soa_t soa; >> LDAPMod change[5]; >> LDAPMod *changep[6] = { >> &change[0], &change[1], &change[2], &change[3], &change[4], >> NULL >> }; >> >> + REQUIRE(zone_dn != NULL); >> + REQUIRE(rdata != NULL); >> + REQUIRE(ldap_inst != NULL); >> + > > All of those REQUIREs are redundant. Functions which use those paramaters check > for their validity. Well, zone_dn and rdata checks are really redundant. First use of ldap_inst is in ldap_inst->mctx, so check is AFAIK necessary. I left REQUIRE(ldap_inst != NULL); in attached patch, other REQUIREs were deleted. > >> /* all values in SOA record are isc_uint32_t, i.e. max. 2^32-1 */ >> #define MAX_SOANUM_LENGTH (10 + 1) >> #define SET_LDAP_MOD(index, name) \ >> @@ -2371,17 +2402,17 @@ modify_soa_record(ldap_connection_t *ldap_conn, const char *zone_dn, >> CHECK(isc_string_printf(change[index].mod_values[0], \ >> MAX_SOANUM_LENGTH, "%u", soa.name)); >> >> - dns_rdata_tostruct(rdata, (void *)&soa, mctx); >> + dns_rdata_tostruct(rdata, (void *)&soa, ldap_inst->mctx); >> -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0047-2-Avoid-manual-connection-management-outside-ldap_quer.patch Type: text/x-patch Size: 14878 bytes Desc: not available URL: From atkac at redhat.com Tue Aug 28 07:57:15 2012 From: atkac at redhat.com (Adam Tkac) Date: Tue, 28 Aug 2012 09:57:15 +0200 Subject: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query() In-Reply-To: <503C6A73.4060101@redhat.com> References: <5028FE08.4060207@redhat.com> <20120822133503.GA7769@redhat.com> <503C6A73.4060101@redhat.com> Message-ID: <20120828075715.GA1236@redhat.com> On Tue, Aug 28, 2012 at 08:51:31AM +0200, Petr Spacek wrote: > On 08/22/2012 03:35 PM, Adam Tkac wrote: > >On Mon, Aug 13, 2012 at 03:15:52PM +0200, Petr Spacek wrote: > >>Hello, > >> > >>this patch improves connection management in bind-dyndb-ldap and closes > >>https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . > >> > >>It should prevent all deadlocks on connection pool in future. > > > >Ack, just check my pedantic comments below, please. > > I partially disagree with one comment below. Amended patch is attached. > ... > Well, zone_dn and rdata checks are really redundant. First use of > ldap_inst is in ldap_inst->mctx, so check is AFAIK necessary. > > I left REQUIRE(ldap_inst != NULL); in attached patch, other REQUIREs were deleted. You are right, sorry for false alarm. Ack. Regards, Adam -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Tue Aug 28 08:47:33 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 28 Aug 2012 10:47:33 +0200 Subject: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query() In-Reply-To: <20120828075715.GA1236@redhat.com> References: <5028FE08.4060207@redhat.com> <20120822133503.GA7769@redhat.com> <503C6A73.4060101@redhat.com> <20120828075715.GA1236@redhat.com> Message-ID: <503C85A5.2040304@redhat.com> On 08/28/2012 09:57 AM, Adam Tkac wrote: > On Tue, Aug 28, 2012 at 08:51:31AM +0200, Petr Spacek wrote: >> On 08/22/2012 03:35 PM, Adam Tkac wrote: >>> On Mon, Aug 13, 2012 at 03:15:52PM +0200, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch improves connection management in bind-dyndb-ldap and closes >>>> https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . >>>> >>>> It should prevent all deadlocks on connection pool in future. >>> >>> Ack, just check my pedantic comments below, please. >> >> I partially disagree with one comment below. Amended patch is attached. >> > > ... > >> Well, zone_dn and rdata checks are really redundant. First use of >> ldap_inst is in ldap_inst->mctx, so check is AFAIK necessary. >> >> I left REQUIRE(ldap_inst != NULL); in attached patch, other REQUIREs were deleted. > > You are right, sorry for false alarm. > > Ack. > > Regards, Adam > Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/8d77eac3ac3d601c043566c83ebaca581e7591fe -- Petr^2 Spacek From pspacek at redhat.com Tue Aug 28 08:48:24 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 28 Aug 2012 10:48:24 +0200 Subject: [Freeipa-devel] [PATCH 0048] Lower minimal connection count for persistent search. In-Reply-To: <20120822133742.GA8074@redhat.com> References: <502901C0.9070905@redhat.com> <20120822133742.GA8074@redhat.com> Message-ID: <503C85D8.2080204@redhat.com> On 08/22/2012 03:37 PM, Adam Tkac wrote: > On Mon, Aug 13, 2012 at 03:31:44PM +0200, Petr Spacek wrote: >> Hello, >> >> As result of better connection management it is possible to run >> persistent search with smaller connection pool. >> >> Attached patch updates built-in configuration checks with new lower >> bound. It closes patch series for >> https://fedorahosted.org/bind-dyndb-ldap/ticket/68: >> Avoid manual connection management outside ldap_query() > > Ack > >> From 786531c5806331dc0486ed1d877563b00d9219da Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Mon, 13 Aug 2012 15:24:48 +0200 >> Subject: [PATCH] Lower minimal connection count for persistent search. >> >> As result of better connection management is possible >> to run persistent search with smaller connection pool. >> >> Signed-off-by: Petr Spacek >> --- >> src/ldap_helper.c | 14 ++++---------- >> 1 file changed, 4 insertions(+), 10 deletions(-) >> Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/ea66d8daaa0feee14db3e5c8eefa6d9b6a45984c -- Petr^2 Spacek From pspacek at redhat.com Tue Aug 28 11:16:50 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 28 Aug 2012 13:16:50 +0200 Subject: [Freeipa-devel] [PATCH 0049] Fix two memory leaks in persistent search In-Reply-To: <20120822134330.GA8117@redhat.com> References: <502A4577.80702@redhat.com> <20120822134330.GA8117@redhat.com> Message-ID: <503CA8A2.7000800@redhat.com> On 08/22/2012 03:43 PM, Adam Tkac wrote: > On Tue, Aug 14, 2012 at 02:32:55PM +0200, Petr Spacek wrote: >> Hello, >> >> This patch fixes two memory leaks in persistent search. > > Ack. > >> From 892f1d5c59a97cdad7a2807ecd172488605ab181 Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Tue, 14 Aug 2012 12:38:43 +0200 >> Subject: [PATCH] Fix two memory leaks in persistent search. >> >> Signed-off-by: Petr Spacek >> --- >> src/ldap_helper.c | 45 ++++++++++++--------------------------------- >> 1 file changed, 12 insertions(+), 33 deletions(-) Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/6114ae200d8698dff421cf0936f8dd8262247710 -- Petr^2 Spacek From pspacek at redhat.com Tue Aug 28 12:00:57 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 28 Aug 2012 14:00:57 +0200 Subject: [Freeipa-devel] [PATCH 0054] Allow BIND to start if LDAP connection times out Message-ID: <503CB2F9.7080603@redhat.com> Hello, this patch allows BIND to start if LDAP connection times out. BIND will reconnect in same way as after "connection refused" errors. The patch closes https://fedorahosted.org/bind-dyndb-ldap/ticket/84 . -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0054-Allow-BIND-to-start-if-LDAP-connection-times-out.patch Type: text/x-patch Size: 1051 bytes Desc: not available URL: From tbabej at redhat.com Tue Aug 28 12:11:05 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 28 Aug 2012 14:11:05 +0200 Subject: [Freeipa-devel] [PATCH 0009] Improves deletion of PTR records in ipa host-del. Message-ID: <503CB559.4030208@redhat.com> Hi, Command ipa host-del with --updatedns now can deal both with hosts which zones are in FQDN form with or without a trailing dot. https://fedorahosted.org/freeipa/ticket/2809 Tomas From tbabej at redhat.com Tue Aug 28 12:13:30 2012 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 28 Aug 2012 14:13:30 +0200 Subject: [Freeipa-devel] [PATCH 0009] Improves deletion of PTR records in ipa host-del. In-Reply-To: <503CB559.4030208@redhat.com> References: <503CB559.4030208@redhat.com> Message-ID: <503CB5EA.1050107@redhat.com> On 08/28/2012 02:11 PM, Tomas Babej wrote: > Hi, > > Command ipa host-del with --updatedns now can deal both with hosts > which zones are in FQDN form with or without a trailing dot. > > https://fedorahosted.org/freeipa/ticket/2809 > > Tomas > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel And the patch itself, once again, cries forgotten on my hard drive. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0009-Improves-deletion-of-PTR-records-in-ipa-host-del.patch Type: text/x-patch Size: 1313 bytes Desc: not available URL: From pvoborni at redhat.com Tue Aug 28 13:30:09 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 28 Aug 2012 15:30:09 +0200 Subject: [Freeipa-devel] Paging in Web UI Message-ID: <503CC7E1.4040004@redhat.com> I would like to point out a problem in Web UI related to paging and suggest a possible solution: Current implementation of paging in Web UI is not serving it purpose which should be to do operations faster and be able to find all users. How current implementation should work: 1) get all primary keys (pkeys) of certain object type (users, ...) 2) load first 20 of them 3) from 1) we know record count so we can choose other page (different subset of 20) How it actually works: * get up to 2000 keys, with more items a LDAP limit is reached and result is truncated * we don't know how many are there, paging doesn't work as it should Problem: Let's say that paging is working (no LDAP limit) and consider this: Our page size is 20 records. With 400 users it is 20 pages. With 2000 users (current search limit) it is 200 pages and so on. Personally I wouldn't like to go through 20 pages. 200 or more pages sounds like a punishment. There is also a question if we even need to do it. I don't see a benefit of paging here. For smaller setup an increase of page size may help admin to see more users on a page more quickly (rolling mouse wheel is faster than click on next page button). For large setups, size of the pkey list must also be taken on mind. For each record there is a pkey and dn (dn is not required but it would need some hacking in python code to remove it). At the moment the list size raises drastically because JSON response includes indenting by spaces and is not compressed (easy to fix, but still there). With 10 000 or more users this pkey and dn list is pretty big (slow load on slower networks -vpns). This list is also loaded on each page change (record-wise, can be improved - cached). IMO with hundreds or thousands of users the most common use case is search by login or name or something. Paging is not required this case. It may be required if the result count is greater than size limit. In such case an option to temporary increase size limit or enable paging would be beneficial. Apart from this case, paging should be off. It will speed up page load because it executes only one request. Possible solution: 1) I suggest to introduce configuration options for paging. They can be global (default for all search pages) or individual for pages or Web UI's users. Per-user configuration can be stored in browser (to not pollute LDAP with application stuff). Global configuration on server in config plugin (minor polluting). Those options should be (per page and global): * enable paging * page size Note: when paging is disabled page size is basically --sizelimit option. 2) On search pages we should have controls to enable/disable paging and to change sizelimit for this particular moment. 3) Compress JSON responses (on httpd level) This way admin can configure default's for his deployment and user's can adjust for certain situations. Btw always including member_xx attributes in find or show commands is not good for search pages either but that's another topic. Comments are welcome. Regards -- Petr Vobornik From pviktori at redhat.com Tue Aug 28 13:40:21 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 28 Aug 2012 15:40:21 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1345219494.21349.193.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> Message-ID: <503CCA45.1010807@redhat.com> On 08/17/2012 06:04 PM, Ade Lee wrote: > On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>> Patch attached this time. I should know better than to do this in the >>>> middle of the night .. >>>> >>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of changes that >>>>>>>>>>> will affect IPA. In particular, the following changes will affect >>>>>>>>>>> current IPA code. >>>>>>>>>>> >>>>>>>>>>> * The directory layout of the dogtag instance has changed. Instead of >>>>>>>>>>> using separate tomcat instances to host different subsystems, the >>>>>>>>>>> standard dogtag installation will allow one to install a CA. KRA, OCSP >>>>>>>>>>> and TKS within the same instance. There have been corresponding changes >>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead >>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>> >>>>>>>>>>> * The default instance will use only four ports (HTTPS, HTTP, AJP and >>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. The default >>>>>>>>>>> ports will be changed to the standard tomcat ports. As these ports are >>>>>>>>>>> local to the ipa server machine, this should not cause too much >>>>>>>>>>> disruption. >>>>>>>>>>> >>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>> >>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of >>>>>>>>>>> tomcatjss. >>>>>>>>>>> >>>>>>>>>>> The attached patch integrates all the above changes in IPA installation >>>>>>>>>>> and maintenance code. Once the patch is applied, users will be able to: >>>>>>>>>>> >>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag >>>>>>>>>>> 10 - and have that old-style dogtag instance continue to run correctly. >>>>>>>>>>> This will require the installation of the latest version of tomcatjss as >>>>>>>>>>> well as the installation of tomcat6. The old-style instance will >>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>> 4. in addition, the new cert renewal code has been patched and should >>>>>>>>>>> continue to work. >>>>>>>>>>> >>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>> >>>>>>>>>>> 1. Installation with an external CA is not yet completed in the new >>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>> >>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>> >>>>>>>>>>> 3. A script needs to be written to allow admins to convert their >>>>>>>>>>> old-style dogtag instances to new style instances, as well as code to >>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>> >>>>>>>>>>> 4. Installation of old-style instances using pkicreate/pkisilent on >>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled soon. >>>>>>>>>>> >>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect these changes, >>>>>>>>>>> but is still in flux. In fact, it is our intention to place the dogtag >>>>>>>>>>> selinux policy in the base selinux policy for f18. In the meantime, it >>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>> >>>>>>>>>>> The dogtag 10 code will be released shortly into f18. Prior to that >>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code in a >>>>>>>>>>> developer repo that is located at >>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>> >>>>>>>>>>> Testing can be done on both f18 and f17 - although the target platform - >>>>>>>>>>> and the only platform for which official builds will be created is f18. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Ade >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Ade, >>>>>>>>>> >>>>>>>>>> Thanks for the patch, I started with review and integration tests (currently >>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>> >>>>>>>>>> Installation on single master was smooth, it worked just fine, even with >>>>>>>>>> enforced SELinux, without any error - kudos to you and the whole dogtag team. >>>>>>>>>> The resulting logs and the structure of your log directory seems improved. I >>>>>>>>>> believe that the brand new Python installers will make it easier to debug >>>>>>>>>> issues with dogtag installation when they come. When I tried our unit tests or >>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>> >>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>> >>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on >>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>> >>>>>>>>> We have a dogtag patch that is currently in review that will address >>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for f17+, >>>>>>>>> rather than f18+ >>>>>>>>> >>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA installation on a >>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this expectable? >>>>>>>>>> >>>>>>>>> Yes. The current IPA patch is designed to work with dogtag 10 only, >>>>>>>>> which will be officially available on f18+. So if you update to dogtag >>>>>>>>> 10, you must have this patch and visa versa. We probably need to add >>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>> >>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to the new >>>>>>>>> dogtag 10 and patched IPA, then that instance will continue to work. >>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>> >>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well >>>>>>>>>> and I got the following error: >>>>>>>>>> >>>>>>>>>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>> ... >>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>> >>>>>>>>>> Your system may be partly configured. >>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>> >>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for details: >>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>> >>>>>>>>>> Root cause: >>>>>>>>>> ... >>>>>>>>>> File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line >>>>>>>>>> 625, in __spawn_instance >>>>>>>>>> "/root/cacert.p12") >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>> I need to look into this. I had fixed ipa-replica-install, rather than >>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>> existed! Should not be too bad to fix though - most likely just need to >>>>>>>>> move files to the right place. >>>>>>>> >>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>> ipa-replica-install time (when --setup-ca option is passed, you probably used >>>>>>>> that one) and the aforementioned ipa-ca-install run after ipa-replica-install. >>>>>>>> >>>>>>> I will be testing this out again. As ipa-ca-install uses the same code >>>>>>> as ipa-replica-install, I would have expected it to work. Did you try >>>>>>> it with selinux in permissive mode? >>>>>>> >>>>>> OK -- I tested this out and realized that between the time I initially >>>>>> tested this and your tests, we had changed a URL >>>>>> from /ca/pki/installer/installToken to /ca/rest/installer/installToken, >>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>> >>>>>> The new patch has been rebased and changes the relevant patch. With >>>>>> this, the CA replica installs correctly. >>>>> >>>>> Umh, where can I find the new rebased patch? :-) >>>>> >>>>>> >>>>>> There was one issue that I encountered. I have seen this once before >>>>>> and will need to investigate further. IPA sometimes restarts the dogtag >>>>>> instance by doing systemctl stop pki-tomcatd.target. and then systemctl >>>>>> start pki-tomcatd.target. I have noticed that occasionally the start >>>>>> invocation fails, while the stop and restart invocations succeed just >>>>>> fine. This makes absolutely no sense because restart is essentially >>>>>> just stop and then start. >>>>>> >>>>>> I think this is actually a bug in systemd. If I reboot the machine, it >>>>>> all works perfectly. If we can't figure out whats going on with >>>>>> systemd, we can always replace this start/stop with starting/stopping >>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That seems to >>>>>> work all the time with no problems. >>>>>> >>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>> >>>>> Ok, I will see if I hit this issue again. Btw. is it recommended in systemd to >>>>> use target units this way? I thought that only service files are meant to be >>>>> used for manipulation with a service. As you said yourself, using service unit >>>>> file for restart worked every time. >>>>> >>>>> Martin >>>> >>> >>> Now, I tested the rebased patch with VMs with permissive SELinux. IPA server >>> install was OK, as always, but replica installation ended up with error. >>> Although it got much further than before: >>> >>> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>> ... >>> [3/3]: restarting directory server >>> done configuring pkids. >>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>> [1/14]: creating certificate server user >>> [2/14]: configuring certificate server instance >>> [3/14]: disabling nonces >>> [4/14]: importing CA chain to RA certificate database >>> [5/14]: fixing RA database permissions >>> [6/14]: setting up signing cert profile >>> [7/14]: set up CRL publishing >>> [8/14]: set certificate subject base >>> [9/14]: enabling Subject Key Identifier >>> [10/14]: configuring certificate server to start on boot >>> [11/14]: configure certmonger for renewals >>> [12/14]: configure clone certificate renewals >>> [13/14]: configure Server-Cert certificate renewal >>> [14/14]: Configure HTTP to proxy connections >>> done configuring pki-tomcatd. >>> Restarting the directory and certificate servers >>> >>> Your system may be partly configured. >>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>> >>> It seems like that CA on a replica did not start and so a check for port 8080 >>> failed. Attached logs (pki-ca-logs.tgz) may provide more information to this issue. >>> >> This is the systemd issue I mentioned before. I will submit a patch >> which will change the restart mechanism to restart the service and not >> the target. >>> > > Patch attached. This patch should be applied on top of the large patch > (being reviewed). > >>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>> (permissive SELinux). Now, the service was upgraded smoothly, CA service >>> could be restarted without failure. However, certificate operations now >>> did not work: >>> >>> # ipactl restart >>> Restarting Directory Service >>> Restarting KDC Service >>> Restarting KPASSWD Service >>> Restarting MEMCACHE Service >>> Restarting HTTP Service >>> Restarting CA Service >>> # ipactl status >>> Directory Service: RUNNING >>> KDC Service: RUNNING >>> KPASSWD Service: RUNNING >>> MEMCACHE Service: RUNNING >>> HTTP Service: RUNNING >>> CA Service: RUNNING >>> # ipa cert-show 1 >>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>> communicate with CMS (Internal Server Error) >>> >>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>> >> The reason for this is that the old dogtag instances are missing: >> 1. a new parameter in CS.cfg >> 2. a couple of symlinks >> >> I plan to fix this in the dogtag code. Specifically, >> 1. I will modify the dogtag code to provide a default value in the case >> where the parameter does not exist. >> 2. I plan to add a function to the dogtag startup scripts to check and >> remake any required symlinks. >> >> Both of these changes are in the dogtag code, and do not affect this >> patch. As a workaround, to confirm that the upgrade actually works, do >> the following manual steps on your upgraded instance: >> >> 1. Add the following parameter to CS.cfg >> pidDir=/var/run >> >> 2. Add the following links; >> cd /var/lib/pki-ca/common/lib >> ln -s /usr/share/java/apache-commons-codec.jar apache-commons-codec.jar >> ln -s /usr/share/java/log4j.jar log4j.jar >> >> 3 Restart the dogtag instance >> >>> HTH, >>> Martin >> >> The attached patch conditionalizes the changes, so that IPA supports both Dogtag versions. I didn't put it in platform code for two reasons - One platform (f17) can have either Dogtag version installed - I didn't want to conflict with Timo Aaltonen's "platformizing" work. According to his WIP patches, he plans to move the whole dogtag module into platform code. I used the existence of /usr/bin/pkispawn as a test for Dogtag 10. If you know a better way, please comment. The dogtag_version option is now always added to I've reverted the changes to install/ui/test/data/ipa_init.json, so you'll have to change the ports manually when testing the UI with Dogtag 10. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0074-Use-Dogtag-10-only-when-it-is-available.patch Type: text/x-patch Size: 49897 bytes Desc: not available URL: From jdennis at redhat.com Tue Aug 28 14:17:06 2012 From: jdennis at redhat.com (John Dennis) Date: Tue, 28 Aug 2012 10:17:06 -0400 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503CC7E1.4040004@redhat.com> References: <503CC7E1.4040004@redhat.com> Message-ID: <503CD2E2.6010005@redhat.com> On 08/28/2012 09:30 AM, Petr Vobornik wrote: > I would like to point out a problem in Web UI related to paging and > suggest a possible solution: > > Current implementation of paging in Web UI is not serving it purpose > which should be to do operations faster and be able to find all users. > > How current implementation should work: > 1) get all primary keys (pkeys) of certain object type (users, ...) > 2) load first 20 of them > 3) from 1) we know record count so we can choose other page (different > subset of 20) > > How it actually works: > * get up to 2000 keys, with more items a LDAP limit is reached and > result is truncated > * we don't know how many are there, paging doesn't work as it should > > Problem: > Let's say that paging is working (no LDAP limit) and consider this: Our > page size is 20 records. With 400 users it is 20 pages. With 2000 users > (current search limit) it is 200 pages and so on. Personally I wouldn't > like to go through 20 pages. 200 or more pages sounds like a punishment. > There is also a question if we even need to do it. I don't see a benefit > of paging here. For smaller setup an increase of page size may help > admin to see more users on a page more quickly (rolling mouse wheel is > faster than click on next page button). > > For large setups, size of the pkey list must also be taken on mind. For > each record there is a pkey and dn (dn is not required but it would need > some hacking in python code to remove it). At the moment the list size > raises drastically because JSON response includes indenting by spaces > and is not compressed (easy to fix, but still there). With 10 000 or > more users this pkey and dn list is pretty big (slow load on slower > networks -vpns). This list is also loaded on each page change > (record-wise, can be improved - cached). > > IMO with hundreds or thousands of users the most common use case is > search by login or name or something. Paging is not required this case. > It may be required if the result count is greater than size limit. In > such case an option to temporary increase size limit or enable paging > would be beneficial. Apart from this case, paging should be off. It will > speed up page load because it executes only one request. > > Possible solution: > 1) I suggest to introduce configuration options for paging. They can be > global (default for all search pages) or individual for pages or Web > UI's users. Per-user configuration can be stored in browser (to not > pollute LDAP with application stuff). Global configuration on server in > config plugin (minor polluting). > > Those options should be (per page and global): > * enable paging > * page size > > Note: when paging is disabled page size is basically --sizelimit option. > > 2) On search pages we should have controls to enable/disable paging and > to change sizelimit for this particular moment. > 3) Compress JSON responses (on httpd level) > > This way admin can configure default's for his deployment and user's can > adjust for certain situations. > > Btw always including member_xx attributes in find or show commands is > not good for search pages either but that's another topic. > > Comments are welcome. Your possible solution does not address how many results are fetched (unless I misunderstood). I'm not sure how per-user preferences are handled in browsers, but don't forget we now have session support in the server. Server session data is available for use. If you don't want to fetch every record to implement paging smartly in conjunction with it's performance issues why not do the query on the server, cache the results in the session, and have the RPC return the total number of results plus a subset of the result. Another RPC could retrieve the next/previous subset of results from the cached result in the session. I don't think there any need in JSON formatted data for pretty printing with indentation. Is it an accident or oversight we're pretty printing the JSON data in an RPC. For large data sets compression would be a win, but most of our RPC communication is not large. Compression has an overhead. With small data you'll use more cycles compressing and uncompressing than you would sending verbose but small data blobs. Compression should be an RPC option specified by either side of the connection and the receiver should be prepared to conditionally uncompress based on a flag value in the RPC. If you use server session data to support paging you may not need to introduce compression since you would only be passing one page of data at a time. User specified page size (without limitation) is an absolute necessity. I am frequently annoyed by web sites which either do not allow me to specify page size or constrain it to ridiculous hard coded limits such as 10, 20 or 30. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Tue Aug 28 14:39:42 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 28 Aug 2012 16:39:42 +0200 Subject: [Freeipa-devel] [PATCH 0009] Improves deletion of PTR records in ipa host-del. In-Reply-To: <503CB5EA.1050107@redhat.com> References: <503CB559.4030208@redhat.com> <503CB5EA.1050107@redhat.com> Message-ID: <503CD82E.7070705@redhat.com> On 08/28/2012 02:13 PM, Tomas Babej wrote: > On 08/28/2012 02:11 PM, Tomas Babej wrote: >> Hi, >> >> Command ipa host-del with --updatedns now can deal both with hosts >> which zones are in FQDN form with or without a trailing dot. >> >> https://fedorahosted.org/freeipa/ticket/2809 >> >> Tomas >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > And the patch itself, once again, cries forgotten on my hard drive. > > Tomas > ACK. Pushed to master, ipa-3-0. Martin From edewata at redhat.com Tue Aug 28 14:42:52 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 09:42:52 -0500 Subject: [Freeipa-devel] [PATCH] 197 Fixed search in HBAC test In-Reply-To: <50339523.1070909@redhat.com> References: <50339523.1070909@redhat.com> Message-ID: <503CD8EC.6050503@redhat.com> On 8/21/2012 9:03 AM, Petr Vobornik wrote: > Search in HBAC test wasn't working because expired flag wasn't set. > > https://fedorahosted.org/freeipa/ticket/2931 > > Notes: HBAC facets don't have refresh button. They can be refreshed by > changing filter and searching. If one search with same filter, it sets > expired flag but it doesn't refresh (search) because page state isn't > changed. It refreshes when one go to different facet and returns back. > Is this behavior acceptable? Or should we > a) don't set expired flag when searching with unchanged filter > b) force refresh when searching with unchanged filter > c) add refresh button along with a) > I prefer leave it as is or b) Hmm.. the behavior is consistent with the rest of the UI, but I think when the user hits Enter or clicks the Search icon in the filter box he'd expect the UI to run a new search and return the latest result even if it's the same filter. I agree with option (b). A Refresh button can be added too, but it's optional. So this patch is ACKed, but feel free to make a future improvement. Maybe instead of having an 'expired flag' we could store an 'expiration date'. If the user returns to the page before it expires, the UI can show the old data. Otherwise the UI will rerun the search. -- Endi S. Dewata From edewata at redhat.com Tue Aug 28 14:43:00 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 09:43:00 -0500 Subject: [Freeipa-devel] [PATCH] 198 Revert change causing failure in test automation In-Reply-To: <50348F3A.3060200@redhat.com> References: <50348F3A.3060200@redhat.com> Message-ID: <503CD8F4.9080605@redhat.com> On 8/22/2012 2:50 AM, Petr Vobornik wrote: > Move of click handler in patch for #2834 causes failure of automation > tests. > > This patch reverts the problematic part. It should not affect function > of fix for #2834. > > https://fedorahosted.org/freeipa/ticket/3014 ACK. -- Endi S. Dewata From edewata at redhat.com Tue Aug 28 14:46:02 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 09:46:02 -0500 Subject: [Freeipa-devel] [PATCH] 199 Permissions: select only applicable options on type change In-Reply-To: <5034CDD9.2050900@redhat.com> References: <5034CDD9.2050900@redhat.com> Message-ID: <503CD9AA.8090207@redhat.com> Found a couple of issues with Undo: 1. Using the scenario described in the ticket, if I undo the Type back to User Group the Attributes aren't updated, it still shows the Service attributes. 2. After that, if I undo the Attributes it will show the originally selected attribute (description) but the attribute will appear at the end of Service attributes (not User Group attributes) and the attributes are not sorted. I also have some comments below. On 8/22/2012 7:17 AM, Petr Vobornik wrote: > Problem: > When a permission is edited, and Type switched, the attributes > selected for previous Type are still selected, and update fails, if they > are invalid for the new Type. But it should get deselected or not even > listed if Type changes. > > Fix: > When Type is changed, attribute list is refreshed and still applicable > attributes are chosen. If Type is reverted back, previously chosen > attributes are back as chosen. > > If attributes are extended outside Web UI by not listed attr, this > attr is listed at the list end. To my understanding the list of ACI attributes are obtained from the LDAP schema, so if a new attribute is added to the object class the UI will know about it and show it in the attribute list. However, if the attribute is added using the extensibleObject the UI may not know about it because there's no schema change, is this what you meant? In that case the UI won't show a checkbox for the attribute, so we'd probably have to use the Filter or Subtree permission target that accepts arbitrary attributes. Ideally the server should support a generic LDAP ACI target which would accept any combination of LDAP filter, subtree, and attributes, but that probably depends on the actual needs. > Note: > If user makes change in attribute list before type change, this change > is forgotten. > > https://fedorahosted.org/freeipa/ticket/2617 -- Endi S. Dewata From edewata at redhat.com Tue Aug 28 14:46:15 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 09:46:15 -0500 Subject: [Freeipa-devel] [PATCH] 200 Fix issue which broke setup of Web UI unit tests In-Reply-To: <503B5151.1050606@redhat.com> References: <503B5151.1050606@redhat.com> Message-ID: <503CD9B7.9050705@redhat.com> On 8/27/2012 5:52 AM, Petr Vobornik wrote: > Fix issue which broke setup of Web UI unit tests > > Web UI itself wasn't negatively affected. > > Issue introduced in be144da672e0634f7aaeff69d662cbc4d11aff0f (#2897). > > https://fedorahosted.org/freeipa/ticket/2897 ACK. -- Endi S. Dewata From edewata at redhat.com Tue Aug 28 14:49:13 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 09:49:13 -0500 Subject: [Freeipa-devel] [PATCH] 201 Successful action notification In-Reply-To: <503B5291.3090502@redhat.com> References: <503B5291.3090502@redhat.com> Message-ID: <503CDA69.9090003@redhat.com> On 8/27/2012 5:57 AM, Petr Vobornik wrote: > User was not notified about success of actions executed from action > list, action panel or facet control bar. > > This patch adds IPA.notify_success(message) call. It creates a yellow > notification area with supplied message in Web UI header in the middle > of the green area (empty space of first level navigation). > This area is displayed for 3s and then it fades out (800ms). It also > fades out when it is clicked. > > This call is used(directly or indirectly) in: > * search facets: delete, disable, enable actions > * details facets: delete action > * user details facet: reset password action > * host details facet: unprovision, set OTP actions > * service details facet: unprovision action > * host and service details facet: request, revoke, restore > certificates actions > * group details facet: change to POSIX/external actions > * dns zone details facet: add/remove permission actions > > https://fedorahosted.org/freeipa/ticket/2977 ACK. What do you think about creating a console/log page to show the action history? This way if the user misses the notification he still can check the logs. The logs will be stored in memory only and have size limit. The page can also be used to show the CLI commands of those actions. -- Endi S. Dewata From edewata at redhat.com Tue Aug 28 14:50:20 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 09:50:20 -0500 Subject: [Freeipa-devel] [PATCH] 202 Password policy paging with proper sorting In-Reply-To: <503B9773.2080904@redhat.com> References: <503B9773.2080904@redhat.com> Message-ID: <503CDAAC.6090103@redhat.com> ACK. Some comments below. On 8/27/2012 10:51 AM, Petr Vobornik wrote: > This patch adds option to disable sorting when paging. It allowed to > enable paging in password policy with order of items untouched (they are > sorted on server side by priority). Is the sorting we see in the UI and CLI identical to the actual sorting used to resolve the policy? I notice that they are sorted lexicographically instead of numerically. > Also fixing issue when paging is disabled and command summary = null. It > displayed 'null' in facet footer. > > https://fedorahosted.org/freeipa/ticket/2677 > > Note: personally I would left paging disabled in password policy page. I > don't think it is beneficial. It slows things down and there would be > hardly more than 100 policies. This should be confirmed by someone more familiar with actual deployments. I'd have no objections. -- Endi S. Dewata From mkosek at redhat.com Tue Aug 28 15:01:42 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 28 Aug 2012 17:01:42 +0200 Subject: [Freeipa-devel] [PATCH] 297 Update Contributors.txt file Message-ID: <503CDD56.3030900@redhat.com> Update list of active developers working on IPA. --- I also tried to add people from community contributing on our wiki pages on freeipa.org. ACKed by Rob off-list. Pushed to master, ipa-3-0. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-297-update-contributors.txt-file.patch Type: text/x-patch Size: 1821 bytes Desc: not available URL: From pvoborni at redhat.com Tue Aug 28 16:23:54 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 28 Aug 2012 18:23:54 +0200 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503CD2E2.6010005@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> Message-ID: <503CF09A.5030902@redhat.com> On 08/28/2012 04:17 PM, John Dennis wrote: > On 08/28/2012 09:30 AM, Petr Vobornik wrote: >> I would like to point out a problem in Web UI related to paging and >> suggest a possible solution: >> >> Current implementation of paging in Web UI is not serving it purpose >> which should be to do operations faster and be able to find all users. >> >> How current implementation should work: >> 1) get all primary keys (pkeys) of certain object type (users, ...) >> 2) load first 20 of them >> 3) from 1) we know record count so we can choose other page (different >> subset of 20) >> >> How it actually works: >> * get up to 2000 keys, with more items a LDAP limit is reached and >> result is truncated >> * we don't know how many are there, paging doesn't work as it should >> >> Problem: >> Let's say that paging is working (no LDAP limit) and consider this: Our >> page size is 20 records. With 400 users it is 20 pages. With 2000 users >> (current search limit) it is 200 pages and so on. Personally I wouldn't >> like to go through 20 pages. 200 or more pages sounds like a punishment. >> There is also a question if we even need to do it. I don't see a benefit >> of paging here. For smaller setup an increase of page size may help >> admin to see more users on a page more quickly (rolling mouse wheel is >> faster than click on next page button). >> >> For large setups, size of the pkey list must also be taken on mind. For >> each record there is a pkey and dn (dn is not required but it would need >> some hacking in python code to remove it). At the moment the list size >> raises drastically because JSON response includes indenting by spaces >> and is not compressed (easy to fix, but still there). With 10 000 or >> more users this pkey and dn list is pretty big (slow load on slower >> networks -vpns). This list is also loaded on each page change >> (record-wise, can be improved - cached). >> >> IMO with hundreds or thousands of users the most common use case is >> search by login or name or something. Paging is not required this case. >> It may be required if the result count is greater than size limit. In >> such case an option to temporary increase size limit or enable paging >> would be beneficial. Apart from this case, paging should be off. It will >> speed up page load because it executes only one request. >> >> Possible solution: >> 1) I suggest to introduce configuration options for paging. They can be >> global (default for all search pages) or individual for pages or Web >> UI's users. Per-user configuration can be stored in browser (to not >> pollute LDAP with application stuff). Global configuration on server in >> config plugin (minor polluting). >> >> Those options should be (per page and global): >> * enable paging >> * page size >> >> Note: when paging is disabled page size is basically --sizelimit option. >> >> 2) On search pages we should have controls to enable/disable paging and >> to change sizelimit for this particular moment. >> 3) Compress JSON responses (on httpd level) >> >> This way admin can configure default's for his deployment and user's can >> adjust for certain situations. >> >> Btw always including member_xx attributes in find or show commands is >> not good for search pages either but that's another topic. >> >> Comments are welcome. > > Your possible solution does not address how many results are fetched > (unless I misunderstood). If paging is enabled it doesn't, but it expects, that admin will disable it for larger setups. For smaller setups it isn't of much an issue. If paging is disabled, the limit is server 'search size limit' or --sizelimit option supplied by Web UI. > > I'm not sure how per-user preferences are handled in browsers, but don't > forget we now have session support in the server. Server session data is > available for use. I was thinking about using browser local storage (basically key-value DB). It has a benefit over session, that it survives a browser restart but it should contain only non-sensitive data (other users may see it). > > If you don't want to fetch every record to implement paging smartly in > conjunction with it's performance issues why not do the query on the > server, cache the results in the session, and have the RPC return the > total number of results plus a subset of the result. Another RPC could > retrieve the next/previous subset of results from the cached result in > the session. I think most software do paging like this. I don't know the reasons for not doing it that way first time. My solution was counting with that we still don't want to do it. Endi do you know the reasons? Earlier sessions didn't exists, but it is doable without them too. > > I don't think there any need in JSON formatted data for pretty printing > with indentation. Is it an accident or oversight we're pretty printing > the JSON data in an RPC. For large data sets compression would be a win, > but most of our RPC communication is not large. Compression has an > overhead. With small data you'll use more cycles compressing and > uncompressing than you would sending verbose but small data blobs. > Compression should be an RPC option specified by either side of the > connection and the receiver should be prepared to conditionally > uncompress based on a flag value in the RPC. If you use server session > data to support paging you may not need to introduce compression since > you would only be passing one page of data at a time. > > User specified page size (without limitation) is an absolute necessity. > I am frequently annoyed by web sites which either do not allow me to > specify page size or constrain it to ridiculous hard coded limits such > as 10, 20 or 30. > +1 -- Petr Vobornik From edewata at redhat.com Tue Aug 28 17:26:56 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 12:26:56 -0500 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503CC7E1.4040004@redhat.com> References: <503CC7E1.4040004@redhat.com> Message-ID: <503CFF60.6020704@redhat.com> On 8/28/2012 8:30 AM, Petr Vobornik wrote: > I would like to point out a problem in Web UI related to paging and > suggest a possible solution: > > Current implementation of paging in Web UI is not serving it purpose > which should be to do operations faster and be able to find all users. > > How current implementation should work: > 1) get all primary keys (pkeys) of certain object type (users, ...) > 2) load first 20 of them > 3) from 1) we know record count so we can choose other page (different > subset of 20) > > How it actually works: > * get up to 2000 keys, with more items a LDAP limit is reached and > result is truncated > * we don't know how many are there, paging doesn't work as it should We have a 'search limit' which can be specified by the user as a search parameter. It has a default value of 100 which can configured in IPA Server -> Configuration. There's also something like 'max result size' (I don't remember the actual name) of 2000 entries which is a global limit imposed by the LDAP backend. I'm not sure if the 'max result size' is configurable, but if I remember correctly if we want to get the entire entries in LDAP we'd need to use Simple Paged Results or VLV. > Problem: > Let's say that paging is working (no LDAP limit) and consider this: Our > page size is 20 records. With 400 users it is 20 pages. With 2000 users > (current search limit) it is 200 pages and so on. Personally I wouldn't > like to go through 20 pages. 200 or more pages sounds like a punishment. In the current UI we could jump to a particular page directly, so it's not necessary to go through all pages. > There is also a question if we even need to do it. I don't see a benefit > of paging here. For smaller setup an increase of page size may help > admin to see more users on a page more quickly (rolling mouse wheel is > faster than click on next page button). We probably can make it look like a single continuous page, but we would still use paging to retrieve the entries. So initially it will show the entries of the first page, when you scroll down it will get the next page, but it will show the entries one-by-one by scrolling up the page. If none of the entries in the previous page are visible anymore the page can be removed from cache, but we might want to keep several pages in the cache. If we implement this, the interface will look the same regardless of the page size or whether we enable/disable paging. > For large setups, size of the pkey list must also be taken on mind. For > each record there is a pkey and dn (dn is not required but it would need > some hacking in python code to remove it). At the moment the list size > raises drastically because JSON response includes indenting by spaces > and is not compressed (easy to fix, but still there). With 10 000 or > more users this pkey and dn list is pretty big (slow load on slower > networks -vpns). This list is also loaded on each page change > (record-wise, can be improved - cached). Yes, ideally the pkey list should be cached and reloaded on new search or cache expiration. Or the server can do the query as John described, so the UI/CLI will just need to specify the page number. > IMO with hundreds or thousands of users the most common use case is > search by login or name or something. Paging is not required this case. > It may be required if the result count is greater than size limit. In > such case an option to temporary increase size limit or enable paging > would be beneficial. For large setup we might need to use simple paged results or VLV, but they have limitations and require server support. Here's a page I wrote sometime ago: http://freeipa.org/page/IPAv3_Directory_Browsing > Apart from this case, paging should be off. It will > speed up page load because it executes only one request. > > Possible solution: > 1) I suggest to introduce configuration options for paging. They can be > global (default for all search pages) or individual for pages or Web > UI's users. Per-user configuration can be stored in browser (to not > pollute LDAP with application stuff). Global configuration on server in > config plugin (minor polluting). > > Those options should be (per page and global): > * enable paging > * page size > > Note: when paging is disabled page size is basically --sizelimit option. Agreed. The actual number of entries would be different for each deployment and for each entity too, so it makes sense to use different settings to optimize the performance. If we implement the single continuous page above this setting will not affect the visual, but it will affect the performance. > 2) On search pages we should have controls to enable/disable paging and > to change sizelimit for this particular moment. If we keep the current UI users might want to save the setting (either on the server or on the client as you described) so they don't have to reconfigure it every time they log in. But if we implement the single continuous page above this might not be necessary anymore because users won't see the visual difference. > 3) Compress JSON responses (on httpd level) Yes. > This way admin can configure default's for his deployment and user's can > adjust for certain situations. > > Btw always including member_xx attributes in find or show commands is > not good for search pages either but that's another topic. > > Comments are welcome. > > Regards I'm also writing a response for the other email. -- Endi S. Dewata From edewata at redhat.com Tue Aug 28 18:31:58 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 28 Aug 2012 13:31:58 -0500 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503CF09A.5030902@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> Message-ID: <503D0E9E.6000703@redhat.com> On 8/28/2012 11:23 AM, Petr Vobornik wrote: >> Your possible solution does not address how many results are fetched >> (unless I misunderstood). > > If paging is enabled it doesn't, but it expects, that admin will disable > it for larger setups.For smaller setups it isn't of much an issue. If > paging is disabled, the limit is server 'search size limit' or > --sizelimit option supplied by Web UI. > >> I'm not sure how per-user preferences are handled in browsers, but don't >> forget we now have session support in the server. Server session data is >> available for use. > > I was thinking about using browser local storage (basically key-value > DB). It has a benefit over session, that it survives a browser restart > but it should contain only non-sensitive data (other users may see it). You mean browser cookie? >> If you don't want to fetch every record to implement paging smartly in >> conjunction with it's performance issues why not do the query on the >> server, cache the results in the session, and have the RPC return the >> total number of results plus a subset of the result. Another RPC could >> retrieve the next/previous subset of results from the cached result in >> the session. > > I think most software do paging like this. I don't know the reasons for > not doing it that way first time. My solution was counting with that we > still don't want to do it. Endi do you know the reasons? Earlier > sessions didn't exists, but it is doable without them too. Yes, at the time the sessions didn't exist and we needed a quick/temporary solution. I agree ideally this should be done on the server side, but how would the server maintain the paging information? The server can keep the search result (either just the pkey list or the entire entries) in memcached, but the result might be large and there might be multiple users accessing multiple search pages, so the total memory requirement could be large. If we don't want the server to keep the search result, the server could rerun the search and return only the requested page and discard most of the results each time the user change page. This may affect server performance. We can also use Simple Paged Results, but if I understood correctly it requires the httpd to maintain an open connection to the LDAP server for each user and for each page. I'm not sure memcached can be used to move the connection object among forked httpd processes. Also Simple Paged Results can only go forward, so no Prev button unless somebody keeps the results. Another possibility is to use VLV, but it seems to require open connection too and only works with a predefined filter. Here's the page I wrote about this: http://freeipa.org/page/IPAv3_Directory_Browsing Currently we're using Option #2 but I'm not sure if we use SPR between httpd and the LDAP server. But even with SPR the result is still bound by some limits. It looks you have to connect as Directory Manager to get the complete list of pkey list/entries. See the table in this page: http://directory.fedoraproject.org/wiki/Simple_Paged_Results_Design The current implementation (keeping the pkey list in the UI) is conceptually similar to the front-end approach described in the above page. >> I don't think there any need in JSON formatted data for pretty printing >> with indentation. Is it an accident or oversight we're pretty printing >> the JSON data in an RPC. For large data sets compression would be a win, >> but most of our RPC communication is not large. Compression has an >> overhead. With small data you'll use more cycles compressing and >> uncompressing than you would sending verbose but small data blobs. >> Compression should be an RPC option specified by either side of the >> connection and the receiver should be prepared to conditionally >> uncompress based on a flag value in the RPC. If you use server session >> data to support paging you may not need to introduce compression since >> you would only be passing one page of data at a time. >> >> User specified page size (without limitation) is an absolute necessity. >> I am frequently annoyed by web sites which either do not allow me to >> specify page size or constrain it to ridiculous hard coded limits such >> as 10, 20 or 30. > > +1 Agreed, but if we implement the single continuous page I described in the earlier email the page size won't be much of an issue anymore. -- Endi S. Dewata From rcritten at redhat.com Tue Aug 28 21:16:55 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 28 Aug 2012 17:16:55 -0400 Subject: [Freeipa-devel] [PATCH] 1049 validate MLS value Message-ID: <503D3547.8090809@redhat.com> Validate that the MLS value in a SELinux user map user is in the range c0..c1023. Existing tests validate correct values, empty values, I'm just adding a high value test. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1049-selinux.patch Type: text/x-diff Size: 2229 bytes Desc: not available URL: From alee at redhat.com Wed Aug 29 03:53:37 2012 From: alee at redhat.com (Ade Lee) Date: Tue, 28 Aug 2012 23:53:37 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <503B78E5.10008@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <502E6BFD.90400@redhat.com> <503B6A98.2040902@redhat.com> <503B6BF8.8090909@redhat.com> <503B78E5.10008@redhat.com> Message-ID: <1346212418.2539.169.camel@aleeredhat.laptop> I have not opened a certmonger bug, but here is a patch to fix the relevant code in certmonger. Nalin, please review and commit. I tested by renewing one of the dogtag system certs (the ocsp signing cert) Ade On Mon, 2012-08-27 at 09:40 -0400, Rob Crittenden wrote: > Petr Viktorin wrote: > > On 08/27/2012 02:39 PM, Dmitri Pal wrote: > >> On 08/17/2012 12:06 PM, Rob Crittenden wrote: > >>> Ade Lee wrote: > >>>> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: > >>>>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: > >>>>>> On 08/16/2012 01:28 PM, Ade Lee wrote: > >>>>>>> Patch attached this time. I should know better than to do this in > >>>>>>> the > >>>>>>> middle of the night .. > >>>>>>> > >>>>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > >>>>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: > >>>>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > >>>>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > >>>>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: > >>>>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > >>>>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: > >>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of > >>>>>>>>>>>>>> changes that > >>>>>>>>>>>>>> will affect IPA. In particular, the following changes will > >>>>>>>>>>>>>> affect > >>>>>>>>>>>>>> current IPA code. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> * The directory layout of the dogtag instance has changed. > >>>>>>>>>>>>>> Instead of > >>>>>>>>>>>>>> using separate tomcat instances to host different > >>>>>>>>>>>>>> subsystems, the > >>>>>>>>>>>>>> standard dogtag installation will allow one to install a > >>>>>>>>>>>>>> CA. KRA, OCSP > >>>>>>>>>>>>>> and TKS within the same instance. There have been > >>>>>>>>>>>>>> corresponding changes > >>>>>>>>>>>>>> in the directory layout, as well as the default instance name > >>>>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon > >>>>>>>>>>>>>> (pki-tomcatd, instead > >>>>>>>>>>>>>> of pki-cad, pki-krad etc.) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> * The default instance will use only four ports (HTTPS, > >>>>>>>>>>>>>> HTTP, AJP and > >>>>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. > >>>>>>>>>>>>>> The default > >>>>>>>>>>>>>> ports will be changed to the standard tomcat ports. As > >>>>>>>>>>>>>> these ports are > >>>>>>>>>>>>>> local to the ipa server machine, this should not cause too > >>>>>>>>>>>>>> much > >>>>>>>>>>>>>> disruption. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> * There is a new single step installer written in python. > >>>>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding > >>>>>>>>>>>>>> version of > >>>>>>>>>>>>>> tomcatjss. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> The attached patch integrates all the above changes in IPA > >>>>>>>>>>>>>> installation > >>>>>>>>>>>>>> and maintenance code. Once the patch is applied, users > >>>>>>>>>>>>>> will be able to: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag > >>>>>>>>>>>>>> 10. > >>>>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. > >>>>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to > >>>>>>>>>>>>>> f18/ dogtag > >>>>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to > >>>>>>>>>>>>>> run correctly. > >>>>>>>>>>>>>> This will require the installation of the latest version of > >>>>>>>>>>>>>> tomcatjss as > >>>>>>>>>>>>>> well as the installation of tomcat6. The old-style > >>>>>>>>>>>>>> instance will > >>>>>>>>>>>>>> continue to use tomcat6. > >>>>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched > >>>>>>>>>>>>>> and should > >>>>>>>>>>>>>> continue to work. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> What is not yet completed / supported: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 1. Installation with an external CA is not yet completed in > >>>>>>>>>>>>>> the new > >>>>>>>>>>>>>> installer. We plan to complete this soon. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been > >>>>>>>>>>>>>> touched > >>>>>>>>>>>>>> (install/tools/ipa-upgradeconfig). > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 3. A script needs to be written to allow admins to convert > >>>>>>>>>>>>>> their > >>>>>>>>>>>>>> old-style dogtag instances to new style instances, as well > >>>>>>>>>>>>>> as code to > >>>>>>>>>>>>>> periodically prompt admins to do this. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 4. Installation of old-style instances using > >>>>>>>>>>>>>> pkicreate/pkisilent on > >>>>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled > >>>>>>>>>>>>>> soon. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect > >>>>>>>>>>>>>> these changes, > >>>>>>>>>>>>>> but is still in flux. In fact, it is our intention to > >>>>>>>>>>>>>> place the dogtag > >>>>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the > >>>>>>>>>>>>>> meantime, it > >>>>>>>>>>>>>> may be necessary to run installs in permissive mode. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. > >>>>>>>>>>>>>> Prior to that > >>>>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code > >>>>>>>>>>>>>> in a > >>>>>>>>>>>>>> developer repo that is located at > >>>>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Testing can be done on both f18 and f17 - although the > >>>>>>>>>>>>>> target platform - > >>>>>>>>>>>>>> and the only platform for which official builds will be > >>>>>>>>>>>>>> created is f18. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Ade > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi Ade, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks for the patch, I started with review and integration > >>>>>>>>>>>>> tests (currently > >>>>>>>>>>>>> running on Fedora 17 with Nathan's repo). > >>>>>>>>>>>>> > >>>>>>>>>>>>> Installation on single master was smooth, it worked just > >>>>>>>>>>>>> fine, even with > >>>>>>>>>>>>> enforced SELinux, without any error - kudos to you and the > >>>>>>>>>>>>> whole dogtag team. > >>>>>>>>>>>>> The resulting logs and the structure of your log directory > >>>>>>>>>>>>> seems improved. I > >>>>>>>>>>>>> believe that the brand new Python installers will make it > >>>>>>>>>>>>> easier to debug > >>>>>>>>>>>>> issues with dogtag installation when they come. When I > >>>>>>>>>>>>> tried our unit tests or > >>>>>>>>>>>>> some simple cert operation, it worked fine as well. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Now the bad news, or rather few issues and suggestions I > >>>>>>>>>>>>> found: > >>>>>>>>>>>>> > >>>>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled > >>>>>>>>>>>>> automatically on > >>>>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a > >>>>>>>>>>>>> Requires. > >>>>>>>>>>>>> > >>>>>>>>>>>> We have a dogtag patch that is currently in review that will > >>>>>>>>>>>> address > >>>>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required > >>>>>>>>>>>> for f17+, > >>>>>>>>>>>> rather than f18+ > >>>>>>>>>>>> > >>>>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA > >>>>>>>>>>>>> installation on a > >>>>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this > >>>>>>>>>>>>> expectable? > >>>>>>>>>>>>> > >>>>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag > >>>>>>>>>>>> 10 only, > >>>>>>>>>>>> which will be officially available on f18+. So if you update > >>>>>>>>>>>> to dogtag > >>>>>>>>>>>> 10, you must have this patch and visa versa. We probably > >>>>>>>>>>>> need to add > >>>>>>>>>>>> the relevant requires to IPA to enforce this. > >>>>>>>>>>>> > >>>>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to > >>>>>>>>>>>> the new > >>>>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue > >>>>>>>>>>>> to work. > >>>>>>>>>>>> But any new instances would be created using dogtag 10. > >>>>>>>>>>>> > >>>>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had > >>>>>>>>>>>>> dogtag10 as well > >>>>>>>>>>>>> and I got the following error: > >>>>>>>>>>>>> > >>>>>>>>>>>>> # ipa-ca-install > >>>>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > >>>>>>>>>>>>> ... > >>>>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 > >>>>>>>>>>>>> seconds > >>>>>>>>>>>>> [1/14]: creating certificate server user > >>>>>>>>>>>>> [2/14]: configuring certificate server instance > >>>>>>>>>>>>> > >>>>>>>>>>>>> Your system may be partly configured. > >>>>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log > >>>>>>>>>>>>> for details: > >>>>>>>>>>>>> IOError: [Errno 2] No such file or directory: > >>>>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > >>>>>>>>>>>>> > >>>>>>>>>>>>> Root cause: > >>>>>>>>>>>>> ... > >>>>>>>>>>>>> File > >>>>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", > >>>>>>>>>>>>> > >>>>>>>>>>>>> line > >>>>>>>>>>>>> 625, in __spawn_instance > >>>>>>>>>>>>> "/root/cacert.p12") > >>>>>>>>>>>>> ... > >>>>>>>>>>>>> > >>>>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, > >>>>>>>>>>>> rather than > >>>>>>>>>>>> ipa-ca-install to create replicas. I didn't know > >>>>>>>>>>>> ipa-ca-install > >>>>>>>>>>>> existed! Should not be too bad to fix though - most likely > >>>>>>>>>>>> just need to > >>>>>>>>>>>> move files to the right place. > >>>>>>>>>>> > >>>>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during > >>>>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, > >>>>>>>>>>> you probably used > >>>>>>>>>>> that one) and the aforementioned ipa-ca-install run after > >>>>>>>>>>> ipa-replica-install. > >>>>>>>>>>> > >>>>>>>>>> I will be testing this out again. As ipa-ca-install uses the > >>>>>>>>>> same code > >>>>>>>>>> as ipa-replica-install, I would have expected it to work. Did > >>>>>>>>>> you try > >>>>>>>>>> it with selinux in permissive mode? > >>>>>>>>>> > >>>>>>>>> OK -- I tested this out and realized that between the time I > >>>>>>>>> initially > >>>>>>>>> tested this and your tests, we had changed a URL > >>>>>>>>> from /ca/pki/installer/installToken to > >>>>>>>>> /ca/rest/installer/installToken, > >>>>>>>>> but I forgot to update ipa-pki-proxy.conf. > >>>>>>>>> > >>>>>>>>> The new patch has been rebased and changes the relevant patch. > >>>>>>>>> With > >>>>>>>>> this, the CA replica installs correctly. > >>>>>>>> > >>>>>>>> Umh, where can I find the new rebased patch? :-) > >>>>>>>> > >>>>>>>>> > >>>>>>>>> There was one issue that I encountered. I have seen this once > >>>>>>>>> before > >>>>>>>>> and will need to investigate further. IPA sometimes restarts > >>>>>>>>> the dogtag > >>>>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then > >>>>>>>>> systemctl > >>>>>>>>> start pki-tomcatd.target. I have noticed that occasionally the > >>>>>>>>> start > >>>>>>>>> invocation fails, while the stop and restart invocations succeed > >>>>>>>>> just > >>>>>>>>> fine. This makes absolutely no sense because restart is > >>>>>>>>> essentially > >>>>>>>>> just stop and then start. > >>>>>>>>> > >>>>>>>>> I think this is actually a bug in systemd. If I reboot the > >>>>>>>>> machine, it > >>>>>>>>> all works perfectly. If we can't figure out whats going on with > >>>>>>>>> systemd, we can always replace this start/stop with > >>>>>>>>> starting/stopping > >>>>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That > >>>>>>>>> seems to > >>>>>>>>> work all the time with no problems. > >>>>>>>>> > >>>>>>>>> I suggest we leave this fix (if needed) for a separate patch. > >>>>>>>> > >>>>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended > >>>>>>>> in systemd to > >>>>>>>> use target units this way? I thought that only service files are > >>>>>>>> meant to be > >>>>>>>> used for manipulation with a service. As you said yourself, using > >>>>>>>> service unit > >>>>>>>> file for restart worked every time. > >>>>>>>> > >>>>>>>> Martin > >>>>>>> > >>>>>> > >>>>>> Now, I tested the rebased patch with VMs with permissive SELinux. > >>>>>> IPA server > >>>>>> install was OK, as always, but replica installation ended up with > >>>>>> error. > >>>>>> Although it got much further than before: > >>>>>> > >>>>>> # ipa-ca-install > >>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > >>>>>> ... > >>>>>> [3/3]: restarting directory server > >>>>>> done configuring pkids. > >>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds > >>>>>> [1/14]: creating certificate server user > >>>>>> [2/14]: configuring certificate server instance > >>>>>> [3/14]: disabling nonces > >>>>>> [4/14]: importing CA chain to RA certificate database > >>>>>> [5/14]: fixing RA database permissions > >>>>>> [6/14]: setting up signing cert profile > >>>>>> [7/14]: set up CRL publishing > >>>>>> [8/14]: set certificate subject base > >>>>>> [9/14]: enabling Subject Key Identifier > >>>>>> [10/14]: configuring certificate server to start on boot > >>>>>> [11/14]: configure certmonger for renewals > >>>>>> [12/14]: configure clone certificate renewals > >>>>>> [13/14]: configure Server-Cert certificate renewal > >>>>>> [14/14]: Configure HTTP to proxy connections > >>>>>> done configuring pki-tomcatd. > >>>>>> Restarting the directory and certificate servers > >>>>>> > >>>>>> Your system may be partly configured. > >>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > >>>>>> > >>>>>> It seems like that CA on a replica did not start and so a check for > >>>>>> port 8080 > >>>>>> failed. Attached logs (pki-ca-logs.tgz) may provide more > >>>>>> information to this issue. > >>>>>> > >>>>> This is the systemd issue I mentioned before. I will submit a patch > >>>>> which will change the restart mechanism to restart the service and not > >>>>> the target. > >>>>>> > >>>> > >>>> Patch attached. This patch should be applied on top of the large patch > >>>> (being reviewed). > >>>> > >>>>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 > >>>>>> (permissive SELinux). Now, the service was upgraded smoothly, CA > >>>>>> service > >>>>>> could be restarted without failure. However, certificate operations > >>>>>> now > >>>>>> did not work: > >>>>>> > >>>>>> # ipactl restart > >>>>>> Restarting Directory Service > >>>>>> Restarting KDC Service > >>>>>> Restarting KPASSWD Service > >>>>>> Restarting MEMCACHE Service > >>>>>> Restarting HTTP Service > >>>>>> Restarting CA Service > >>>>>> # ipactl status > >>>>>> Directory Service: RUNNING > >>>>>> KDC Service: RUNNING > >>>>>> KPASSWD Service: RUNNING > >>>>>> MEMCACHE Service: RUNNING > >>>>>> HTTP Service: RUNNING > >>>>>> CA Service: RUNNING > >>>>>> # ipa cert-show 1 > >>>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to > >>>>>> communicate with CMS (Internal Server Error) > >>>>>> > >>>>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). > >>>>>> > >>>>> The reason for this is that the old dogtag instances are missing: > >>>>> 1. a new parameter in CS.cfg > >>>>> 2. a couple of symlinks > >>>>> > >>>>> I plan to fix this in the dogtag code. Specifically, > >>>>> 1. I will modify the dogtag code to provide a default value in the > >>>>> case > >>>>> where the parameter does not exist. > >>>>> 2. I plan to add a function to the dogtag startup scripts to check and > >>>>> remake any required symlinks. > >>>>> > >>>>> Both of these changes are in the dogtag code, and do not affect this > >>>>> patch. As a workaround, to confirm that the upgrade actually > >>>>> works, do > >>>>> the following manual steps on your upgraded instance: > >>>>> > >>>>> 1. Add the following parameter to CS.cfg > >>>>> pidDir=/var/run > >>>>> > >>>>> 2. Add the following links; > >>>>> cd /var/lib/pki-ca/common/lib > >>>>> ln -s /usr/share/java/apache-commons-codec.jar > >>>>> apache-commons-codec.jar > >>>>> ln -s /usr/share/java/log4j.jar log4j.jar > >>>>> > >>>>> 3 Restart the dogtag instance > >>>> > >>> > >>> To throw a little twist in here, we should make sure that the > >>> certmonger restart scripts still work as well. > >>> > >>> rob > >>> > >>> _______________________________________________ > >>> Freeipa-devel mailing list > >>> Freeipa-devel at redhat.com > >>> https://www.redhat.com/mailman/listinfo/freeipa-devel > >> > >> What is the situation with this patch? Was it commited? Or is it in > >> works/review? Was is superseded by another patch (that I have not got to > >> yet and thus asking)? > >> > > > > I am finishing and testing a patch to apply on top, which will make it > > possible to use either Dogtag 9 or 10 depending on what's installed. > > > > I don't know if the CA renewal issue has been addressed yet. It may > impact certmonger because the ports are hardcoded in the certmonger > source. I was under the impression Ade was looking into this and would > open a certmonger bug as needed. > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Support-dogtag-10-ports.patch Type: text/x-patch Size: 2328 bytes Desc: not available URL: From mkosek at redhat.com Wed Aug 29 07:32:02 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 29 Aug 2012 09:32:02 +0200 Subject: [Freeipa-devel] [PATCH] 1049 validate MLS value In-Reply-To: <503D3547.8090809@redhat.com> References: <503D3547.8090809@redhat.com> Message-ID: <503DC572.3070702@redhat.com> On 08/28/2012 11:16 PM, Rob Crittenden wrote: > Validate that the MLS value in a SELinux user map user is in the range c0..c1023. > > Existing tests validate correct values, empty values, I'm just adding a high > value test. > > rob > ACK. Pushed to master, ipa-3-0. Martin From pvoborni at redhat.com Wed Aug 29 10:09:14 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 29 Aug 2012 12:09:14 +0200 Subject: [Freeipa-devel] [PATCH] 202 Password policy paging with proper sorting In-Reply-To: <503CDAAC.6090103@redhat.com> References: <503B9773.2080904@redhat.com> <503CDAAC.6090103@redhat.com> Message-ID: <503DEA4A.4070704@redhat.com> On 08/28/2012 04:50 PM, Endi Sukma Dewata wrote: > ACK. Pushed to master, ipa-3-0. > > Some comments below. > > On 8/27/2012 10:51 AM, Petr Vobornik wrote: >> This patch adds option to disable sorting when paging. It allowed to >> enable paging in password policy with order of items untouched (they are >> sorted on server side by priority). > > Is the sorting we see in the UI and CLI identical to the actual sorting > used to resolve the policy? I notice that they are sorted > lexicographically instead of numerically. I hope not. New ticket about the sorting: https://fedorahosted.org/freeipa/ticket/3039 > >> Also fixing issue when paging is disabled and command summary = null. It >> displayed 'null' in facet footer. >> >> https://fedorahosted.org/freeipa/ticket/2677 >> >> Note: personally I would left paging disabled in password policy page. I >> don't think it is beneficial. It slows things down and there would be >> hardly more than 100 policies. > > This should be confirmed by someone more familiar with actual > deployments. I'd have no objections. > -- Petr Vobornik From pvoborni at redhat.com Wed Aug 29 10:17:24 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 29 Aug 2012 12:17:24 +0200 Subject: [Freeipa-devel] [PATCH] 201 Successful action notification In-Reply-To: <503CDA69.9090003@redhat.com> References: <503B5291.3090502@redhat.com> <503CDA69.9090003@redhat.com> Message-ID: <503DEC34.2030000@redhat.com> On 08/28/2012 04:49 PM, Endi Sukma Dewata wrote: > On 8/27/2012 5:57 AM, Petr Vobornik wrote: >> User was not notified about success of actions executed from action >> list, action panel or facet control bar. >> >> This patch adds IPA.notify_success(message) call. It creates a yellow >> notification area with supplied message in Web UI header in the middle >> of the green area (empty space of first level navigation). >> This area is displayed for 3s and then it fades out (800ms). It also >> fades out when it is clicked. >> >> This call is used(directly or indirectly) in: >> * search facets: delete, disable, enable actions >> * details facets: delete action >> * user details facet: reset password action >> * host details facet: unprovision, set OTP actions >> * service details facet: unprovision action >> * host and service details facet: request, revoke, restore >> certificates actions >> * group details facet: change to POSIX/external actions >> * dns zone details facet: add/remove permission actions >> >> https://fedorahosted.org/freeipa/ticket/2977 > > ACK. Pushed to master, ipa-3-0. > > What do you think about creating a console/log page to show the action > history? This way if the user misses the notification he still can check > the logs. The logs will be stored in memory only and have size limit. > The page can also be used to show the CLI commands of those actions. > I was thinking about it as well. New ticket: https://fedorahosted.org/freeipa/ticket/3040 I guess it will have really low priority. -- Petr Vobornik From pvoborni at redhat.com Wed Aug 29 10:17:44 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 29 Aug 2012 12:17:44 +0200 Subject: [Freeipa-devel] [PATCH] 200 Fix issue which broke setup of Web UI unit tests In-Reply-To: <503CD9B7.9050705@redhat.com> References: <503B5151.1050606@redhat.com> <503CD9B7.9050705@redhat.com> Message-ID: <503DEC48.1070409@redhat.com> On 08/28/2012 04:46 PM, Endi Sukma Dewata wrote: > On 8/27/2012 5:52 AM, Petr Vobornik wrote: >> Fix issue which broke setup of Web UI unit tests >> >> Web UI itself wasn't negatively affected. >> >> Issue introduced in be144da672e0634f7aaeff69d662cbc4d11aff0f (#2897). >> >> https://fedorahosted.org/freeipa/ticket/2897 > > ACK. > Pushed to master, ipa-3-0. -- Petr Vobornik From pvoborni at redhat.com Wed Aug 29 10:18:48 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 29 Aug 2012 12:18:48 +0200 Subject: [Freeipa-devel] [PATCH] 198 Revert change causing failure in test automation In-Reply-To: <503CD8F4.9080605@redhat.com> References: <50348F3A.3060200@redhat.com> <503CD8F4.9080605@redhat.com> Message-ID: <503DEC88.4020601@redhat.com> On 08/28/2012 04:43 PM, Endi Sukma Dewata wrote: > On 8/22/2012 2:50 AM, Petr Vobornik wrote: >> Move of click handler in patch for #2834 causes failure of automation >> tests. >> >> This patch reverts the problematic part. It should not affect function >> of fix for #2834. >> >> https://fedorahosted.org/freeipa/ticket/3014 > > ACK. > Pushed to master, ipa-3-0. -- Petr Vobornik From mkosek at redhat.com Wed Aug 29 12:40:57 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 29 Aug 2012 14:40:57 +0200 Subject: [Freeipa-devel] [PATCH] 298 Add safe updates for objectClasses Message-ID: <503E0DD9.2020905@redhat.com> Current objectclass updates in a form of "replace" update instruction dependent on exact match of the old object class specification in the update instruction and the real value in LDAP. However, this approach is very error prone as object class definition can easily differ as for example because of unexpected X-ORIGIN value. Such objectclass update failures may lead to serious malfunctions later. Add new update instruction type "replaceoc" with the following format: replaceoc:OID:new This update instruction will always replace an objectclass with specified OID with the new definition. https://fedorahosted.org/freeipa/ticket/2440 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-298-add-safe-updates-for-objectclasses.patch Type: text/x-patch Size: 16859 bytes Desc: not available URL: From alee at redhat.com Wed Aug 29 12:48:32 2012 From: alee at redhat.com (Ade Lee) Date: Wed, 29 Aug 2012 08:48:32 -0400 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <1346212418.2539.169.camel@aleeredhat.laptop> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <502E6BFD.90400@redhat.com> <503B6A98.2040902@redhat.com> <503B6BF8.8090909@redhat.com> <503B78E5.10008@redhat.com> <1346212418.2539.169.camel@aleeredhat.laptop> Message-ID: <1346244513.2539.172.camel@aleeredhat.laptop> Incidentally, I ran this in permmissive selinux mode. The following rules are required to be added: #============= certmonger_t ============== corenet_tcp_connect_http_cache_port(certmonger_t) files_read_var_lib_symlinks(certmonger_t) On Tue, 2012-08-28 at 23:53 -0400, Ade Lee wrote: > I have not opened a certmonger bug, but here is a patch to fix the > relevant code in certmonger. > > Nalin, please review and commit. > > I tested by renewing one of the dogtag system certs (the ocsp signing > cert) > > Ade > > On Mon, 2012-08-27 at 09:40 -0400, Rob Crittenden wrote: > > Petr Viktorin wrote: > > > On 08/27/2012 02:39 PM, Dmitri Pal wrote: > > >> On 08/17/2012 12:06 PM, Rob Crittenden wrote: > > >>> Ade Lee wrote: > > >>>> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: > > >>>>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: > > >>>>>> On 08/16/2012 01:28 PM, Ade Lee wrote: > > >>>>>>> Patch attached this time. I should know better than to do this in > > >>>>>>> the > > >>>>>>> middle of the night .. > > >>>>>>> > > >>>>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > > >>>>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: > > >>>>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > > >>>>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > > >>>>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: > > >>>>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > > >>>>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: > > >>>>>>>>>>>>>> Hi, > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of > > >>>>>>>>>>>>>> changes that > > >>>>>>>>>>>>>> will affect IPA. In particular, the following changes will > > >>>>>>>>>>>>>> affect > > >>>>>>>>>>>>>> current IPA code. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> * The directory layout of the dogtag instance has changed. > > >>>>>>>>>>>>>> Instead of > > >>>>>>>>>>>>>> using separate tomcat instances to host different > > >>>>>>>>>>>>>> subsystems, the > > >>>>>>>>>>>>>> standard dogtag installation will allow one to install a > > >>>>>>>>>>>>>> CA. KRA, OCSP > > >>>>>>>>>>>>>> and TKS within the same instance. There have been > > >>>>>>>>>>>>>> corresponding changes > > >>>>>>>>>>>>>> in the directory layout, as well as the default instance name > > >>>>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon > > >>>>>>>>>>>>>> (pki-tomcatd, instead > > >>>>>>>>>>>>>> of pki-cad, pki-krad etc.) > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> * The default instance will use only four ports (HTTPS, > > >>>>>>>>>>>>>> HTTP, AJP and > > >>>>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. > > >>>>>>>>>>>>>> The default > > >>>>>>>>>>>>>> ports will be changed to the standard tomcat ports. As > > >>>>>>>>>>>>>> these ports are > > >>>>>>>>>>>>>> local to the ipa server machine, this should not cause too > > >>>>>>>>>>>>>> much > > >>>>>>>>>>>>>> disruption. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> * There is a new single step installer written in python. > > >>>>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding > > >>>>>>>>>>>>>> version of > > >>>>>>>>>>>>>> tomcatjss. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> The attached patch integrates all the above changes in IPA > > >>>>>>>>>>>>>> installation > > >>>>>>>>>>>>>> and maintenance code. Once the patch is applied, users > > >>>>>>>>>>>>>> will be able to: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag > > >>>>>>>>>>>>>> 10. > > >>>>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. > > >>>>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to > > >>>>>>>>>>>>>> f18/ dogtag > > >>>>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to > > >>>>>>>>>>>>>> run correctly. > > >>>>>>>>>>>>>> This will require the installation of the latest version of > > >>>>>>>>>>>>>> tomcatjss as > > >>>>>>>>>>>>>> well as the installation of tomcat6. The old-style > > >>>>>>>>>>>>>> instance will > > >>>>>>>>>>>>>> continue to use tomcat6. > > >>>>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched > > >>>>>>>>>>>>>> and should > > >>>>>>>>>>>>>> continue to work. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> What is not yet completed / supported: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> 1. Installation with an external CA is not yet completed in > > >>>>>>>>>>>>>> the new > > >>>>>>>>>>>>>> installer. We plan to complete this soon. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been > > >>>>>>>>>>>>>> touched > > >>>>>>>>>>>>>> (install/tools/ipa-upgradeconfig). > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> 3. A script needs to be written to allow admins to convert > > >>>>>>>>>>>>>> their > > >>>>>>>>>>>>>> old-style dogtag instances to new style instances, as well > > >>>>>>>>>>>>>> as code to > > >>>>>>>>>>>>>> periodically prompt admins to do this. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> 4. Installation of old-style instances using > > >>>>>>>>>>>>>> pkicreate/pkisilent on > > >>>>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled > > >>>>>>>>>>>>>> soon. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect > > >>>>>>>>>>>>>> these changes, > > >>>>>>>>>>>>>> but is still in flux. In fact, it is our intention to > > >>>>>>>>>>>>>> place the dogtag > > >>>>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the > > >>>>>>>>>>>>>> meantime, it > > >>>>>>>>>>>>>> may be necessary to run installs in permissive mode. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. > > >>>>>>>>>>>>>> Prior to that > > >>>>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code > > >>>>>>>>>>>>>> in a > > >>>>>>>>>>>>>> developer repo that is located at > > >>>>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Testing can be done on both f18 and f17 - although the > > >>>>>>>>>>>>>> target platform - > > >>>>>>>>>>>>>> and the only platform for which official builds will be > > >>>>>>>>>>>>>> created is f18. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Thanks, > > >>>>>>>>>>>>>> Ade > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Hi Ade, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Thanks for the patch, I started with review and integration > > >>>>>>>>>>>>> tests (currently > > >>>>>>>>>>>>> running on Fedora 17 with Nathan's repo). > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Installation on single master was smooth, it worked just > > >>>>>>>>>>>>> fine, even with > > >>>>>>>>>>>>> enforced SELinux, without any error - kudos to you and the > > >>>>>>>>>>>>> whole dogtag team. > > >>>>>>>>>>>>> The resulting logs and the structure of your log directory > > >>>>>>>>>>>>> seems improved. I > > >>>>>>>>>>>>> believe that the brand new Python installers will make it > > >>>>>>>>>>>>> easier to debug > > >>>>>>>>>>>>> issues with dogtag installation when they come. When I > > >>>>>>>>>>>>> tried our unit tests or > > >>>>>>>>>>>>> some simple cert operation, it worked fine as well. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Now the bad news, or rather few issues and suggestions I > > >>>>>>>>>>>>> found: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled > > >>>>>>>>>>>>> automatically on > > >>>>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a > > >>>>>>>>>>>>> Requires. > > >>>>>>>>>>>>> > > >>>>>>>>>>>> We have a dogtag patch that is currently in review that will > > >>>>>>>>>>>> address > > >>>>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required > > >>>>>>>>>>>> for f17+, > > >>>>>>>>>>>> rather than f18+ > > >>>>>>>>>>>> > > >>>>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA > > >>>>>>>>>>>>> installation on a > > >>>>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this > > >>>>>>>>>>>>> expectable? > > >>>>>>>>>>>>> > > >>>>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag > > >>>>>>>>>>>> 10 only, > > >>>>>>>>>>>> which will be officially available on f18+. So if you update > > >>>>>>>>>>>> to dogtag > > >>>>>>>>>>>> 10, you must have this patch and visa versa. We probably > > >>>>>>>>>>>> need to add > > >>>>>>>>>>>> the relevant requires to IPA to enforce this. > > >>>>>>>>>>>> > > >>>>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to > > >>>>>>>>>>>> the new > > >>>>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue > > >>>>>>>>>>>> to work. > > >>>>>>>>>>>> But any new instances would be created using dogtag 10. > > >>>>>>>>>>>> > > >>>>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had > > >>>>>>>>>>>>> dogtag10 as well > > >>>>>>>>>>>>> and I got the following error: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> # ipa-ca-install > > >>>>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > > >>>>>>>>>>>>> ... > > >>>>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 > > >>>>>>>>>>>>> seconds > > >>>>>>>>>>>>> [1/14]: creating certificate server user > > >>>>>>>>>>>>> [2/14]: configuring certificate server instance > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Your system may be partly configured. > > >>>>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log > > >>>>>>>>>>>>> for details: > > >>>>>>>>>>>>> IOError: [Errno 2] No such file or directory: > > >>>>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Root cause: > > >>>>>>>>>>>>> ... > > >>>>>>>>>>>>> File > > >>>>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> line > > >>>>>>>>>>>>> 625, in __spawn_instance > > >>>>>>>>>>>>> "/root/cacert.p12") > > >>>>>>>>>>>>> ... > > >>>>>>>>>>>>> > > >>>>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, > > >>>>>>>>>>>> rather than > > >>>>>>>>>>>> ipa-ca-install to create replicas. I didn't know > > >>>>>>>>>>>> ipa-ca-install > > >>>>>>>>>>>> existed! Should not be too bad to fix though - most likely > > >>>>>>>>>>>> just need to > > >>>>>>>>>>>> move files to the right place. > > >>>>>>>>>>> > > >>>>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during > > >>>>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, > > >>>>>>>>>>> you probably used > > >>>>>>>>>>> that one) and the aforementioned ipa-ca-install run after > > >>>>>>>>>>> ipa-replica-install. > > >>>>>>>>>>> > > >>>>>>>>>> I will be testing this out again. As ipa-ca-install uses the > > >>>>>>>>>> same code > > >>>>>>>>>> as ipa-replica-install, I would have expected it to work. Did > > >>>>>>>>>> you try > > >>>>>>>>>> it with selinux in permissive mode? > > >>>>>>>>>> > > >>>>>>>>> OK -- I tested this out and realized that between the time I > > >>>>>>>>> initially > > >>>>>>>>> tested this and your tests, we had changed a URL > > >>>>>>>>> from /ca/pki/installer/installToken to > > >>>>>>>>> /ca/rest/installer/installToken, > > >>>>>>>>> but I forgot to update ipa-pki-proxy.conf. > > >>>>>>>>> > > >>>>>>>>> The new patch has been rebased and changes the relevant patch. > > >>>>>>>>> With > > >>>>>>>>> this, the CA replica installs correctly. > > >>>>>>>> > > >>>>>>>> Umh, where can I find the new rebased patch? :-) > > >>>>>>>> > > >>>>>>>>> > > >>>>>>>>> There was one issue that I encountered. I have seen this once > > >>>>>>>>> before > > >>>>>>>>> and will need to investigate further. IPA sometimes restarts > > >>>>>>>>> the dogtag > > >>>>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then > > >>>>>>>>> systemctl > > >>>>>>>>> start pki-tomcatd.target. I have noticed that occasionally the > > >>>>>>>>> start > > >>>>>>>>> invocation fails, while the stop and restart invocations succeed > > >>>>>>>>> just > > >>>>>>>>> fine. This makes absolutely no sense because restart is > > >>>>>>>>> essentially > > >>>>>>>>> just stop and then start. > > >>>>>>>>> > > >>>>>>>>> I think this is actually a bug in systemd. If I reboot the > > >>>>>>>>> machine, it > > >>>>>>>>> all works perfectly. If we can't figure out whats going on with > > >>>>>>>>> systemd, we can always replace this start/stop with > > >>>>>>>>> starting/stopping > > >>>>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That > > >>>>>>>>> seems to > > >>>>>>>>> work all the time with no problems. > > >>>>>>>>> > > >>>>>>>>> I suggest we leave this fix (if needed) for a separate patch. > > >>>>>>>> > > >>>>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended > > >>>>>>>> in systemd to > > >>>>>>>> use target units this way? I thought that only service files are > > >>>>>>>> meant to be > > >>>>>>>> used for manipulation with a service. As you said yourself, using > > >>>>>>>> service unit > > >>>>>>>> file for restart worked every time. > > >>>>>>>> > > >>>>>>>> Martin > > >>>>>>> > > >>>>>> > > >>>>>> Now, I tested the rebased patch with VMs with permissive SELinux. > > >>>>>> IPA server > > >>>>>> install was OK, as always, but replica installation ended up with > > >>>>>> error. > > >>>>>> Although it got much further than before: > > >>>>>> > > >>>>>> # ipa-ca-install > > >>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg > > >>>>>> ... > > >>>>>> [3/3]: restarting directory server > > >>>>>> done configuring pkids. > > >>>>>> Configuring certificate server: Estimated time 3 minutes 30 seconds > > >>>>>> [1/14]: creating certificate server user > > >>>>>> [2/14]: configuring certificate server instance > > >>>>>> [3/14]: disabling nonces > > >>>>>> [4/14]: importing CA chain to RA certificate database > > >>>>>> [5/14]: fixing RA database permissions > > >>>>>> [6/14]: setting up signing cert profile > > >>>>>> [7/14]: set up CRL publishing > > >>>>>> [8/14]: set certificate subject base > > >>>>>> [9/14]: enabling Subject Key Identifier > > >>>>>> [10/14]: configuring certificate server to start on boot > > >>>>>> [11/14]: configure certmonger for renewals > > >>>>>> [12/14]: configure clone certificate renewals > > >>>>>> [13/14]: configure Server-Cert certificate renewal > > >>>>>> [14/14]: Configure HTTP to proxy connections > > >>>>>> done configuring pki-tomcatd. > > >>>>>> Restarting the directory and certificate servers > > >>>>>> > > >>>>>> Your system may be partly configured. > > >>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. > > >>>>>> > > >>>>>> It seems like that CA on a replica did not start and so a check for > > >>>>>> port 8080 > > >>>>>> failed. Attached logs (pki-ca-logs.tgz) may provide more > > >>>>>> information to this issue. > > >>>>>> > > >>>>> This is the systemd issue I mentioned before. I will submit a patch > > >>>>> which will change the restart mechanism to restart the service and not > > >>>>> the target. > > >>>>>> > > >>>> > > >>>> Patch attached. This patch should be applied on top of the large patch > > >>>> (being reviewed). > > >>>> > > >>>>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 > > >>>>>> (permissive SELinux). Now, the service was upgraded smoothly, CA > > >>>>>> service > > >>>>>> could be restarted without failure. However, certificate operations > > >>>>>> now > > >>>>>> did not work: > > >>>>>> > > >>>>>> # ipactl restart > > >>>>>> Restarting Directory Service > > >>>>>> Restarting KDC Service > > >>>>>> Restarting KPASSWD Service > > >>>>>> Restarting MEMCACHE Service > > >>>>>> Restarting HTTP Service > > >>>>>> Restarting CA Service > > >>>>>> # ipactl status > > >>>>>> Directory Service: RUNNING > > >>>>>> KDC Service: RUNNING > > >>>>>> KPASSWD Service: RUNNING > > >>>>>> MEMCACHE Service: RUNNING > > >>>>>> HTTP Service: RUNNING > > >>>>>> CA Service: RUNNING > > >>>>>> # ipa cert-show 1 > > >>>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to > > >>>>>> communicate with CMS (Internal Server Error) > > >>>>>> > > >>>>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). > > >>>>>> > > >>>>> The reason for this is that the old dogtag instances are missing: > > >>>>> 1. a new parameter in CS.cfg > > >>>>> 2. a couple of symlinks > > >>>>> > > >>>>> I plan to fix this in the dogtag code. Specifically, > > >>>>> 1. I will modify the dogtag code to provide a default value in the > > >>>>> case > > >>>>> where the parameter does not exist. > > >>>>> 2. I plan to add a function to the dogtag startup scripts to check and > > >>>>> remake any required symlinks. > > >>>>> > > >>>>> Both of these changes are in the dogtag code, and do not affect this > > >>>>> patch. As a workaround, to confirm that the upgrade actually > > >>>>> works, do > > >>>>> the following manual steps on your upgraded instance: > > >>>>> > > >>>>> 1. Add the following parameter to CS.cfg > > >>>>> pidDir=/var/run > > >>>>> > > >>>>> 2. Add the following links; > > >>>>> cd /var/lib/pki-ca/common/lib > > >>>>> ln -s /usr/share/java/apache-commons-codec.jar > > >>>>> apache-commons-codec.jar > > >>>>> ln -s /usr/share/java/log4j.jar log4j.jar > > >>>>> > > >>>>> 3 Restart the dogtag instance > > >>>> > > >>> > > >>> To throw a little twist in here, we should make sure that the > > >>> certmonger restart scripts still work as well. > > >>> > > >>> rob > > >>> > > >>> _______________________________________________ > > >>> Freeipa-devel mailing list > > >>> Freeipa-devel at redhat.com > > >>> https://www.redhat.com/mailman/listinfo/freeipa-devel > > >> > > >> What is the situation with this patch? Was it commited? Or is it in > > >> works/review? Was is superseded by another patch (that I have not got to > > >> yet and thus asking)? > > >> > > > > > > I am finishing and testing a patch to apply on top, which will make it > > > possible to use either Dogtag 9 or 10 depending on what's installed. > > > > > > > I don't know if the CA renewal issue has been addressed yet. It may > > impact certmonger because the ports are hardcoded in the certmonger > > source. I was under the impression Ade was looking into this and would > > open a certmonger bug as needed. > > > > rob > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From tbabej at redhat.com Wed Aug 29 12:54:44 2012 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 29 Aug 2012 14:54:44 +0200 Subject: [Freeipa-devel] [PATCH 0006] Improves sssd.conf handling during ipa-client uninstall In-Reply-To: <503B8A65.30100@redhat.com> References: <717623750.1969105.1345208663118.JavaMail.root@redhat.com> <503B5D09.7040401@redhat.com> <503B6EC8.3060508@redhat.com> <20120827133727.GG8612@zeppelin.brq.redhat.com> <503B8A65.30100@redhat.com> Message-ID: <503E1114.3080309@redhat.com> On 08/27/2012 04:55 PM, Martin Kosek wrote: > On 08/27/2012 03:37 PM, Jakub Hrozek wrote: >> On Mon, Aug 27, 2012 at 02:57:44PM +0200, Martin Kosek wrote: >>> I think that the right behavior of SSSD conf uninstall should be the following: >>> >>> * sssd.conf existed before IPA install + non-IPA domains in sssd.conf found: >>> - move backed conf up sssd.conf.bkp (and inform the user) >>> - use SSSDConfig delete_domain function to remove ipa domain from sssd.conf >>> - restart sssd afterwards >> I'm confused here, which of the files is the original >> pre-ipa-client-install file? > This is the "backed up sssd.conf". I thought that it may be useful for user to > still have an access to it after uninstall. > >> How does the non-ipa domain end up in the sssd.conf file? Does it have >> to be configured manually or does ipa-client-install merge the list of >> domains on installation? > ipa-client-install merge the list of the domains. It overrides the old > sssd.conf only when we cannot parse the sssd.conf and --preserve-sssd option > was not set. > > Martin Hi, The sssd.conf file is no longer left behind in case sssd was not configured before the installation. However, the patch goes behind the scope of this ticked and improves the handling of sssd.conf during the ipa-client-install --uninstall in general. The current behaviour (well documented in source code) is as follows: - In general, the IPA domain is simply removed from the sssd.conf file, instead of sssd.conf being rewritten from the backup. This preserves any domains added after installation. - If sssd.conf existed before the installation, it is restored to sssd.conf.bkp. However, any IPA domains from pre-installation sssd.conf should have been merged during the installation. - If sssd.conf did not exist before the installation, and no other domains than IPA domain exist in it, the patch makes sure that sssd.conf is moved to sssd.conf.deleted so user experiences no crash during any next installation due to its existence. https://fedorahosted.org/freeipa/ticket/2740 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0006-2-Improves-sssd.conf-handling-during-ipa-client-uninst.patch Type: text/x-patch Size: 7311 bytes Desc: not available URL: From jdennis at redhat.com Wed Aug 29 13:34:50 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 29 Aug 2012 09:34:50 -0400 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503D0E9E.6000703@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> Message-ID: <503E1A7A.7040303@redhat.com> On 08/28/2012 02:31 PM, Endi Sukma Dewata wrote: > The server can keep the search result (either just the pkey list or the > entire entries) in memcached, but the result might be large and there > might be multiple users accessing multiple search pages, so the total > memory requirement could be large. The default max size of an entry in memcached is 1MB. It can be increased to an upper limit of 128MB (but the memcached implementors do not recommend this due to degraded performance and the impact on the system). The session data is stored in a dict. You would be sharing the session data with other parts of the system. Currently that only includes the authentication data which is relatively small. I believe there is also some minor bookkeeping overhead that detracts from the per item total. If we need to exceed the upper bound for paged data I suppose we could implement caching within the cache. Almost 1MB of data is a lot of paging (and that limit can be increased), it would take a fair amount of paging to consume all that data. But the cached query could be broken up into "cached chunks" to limit the impact on memcached and to accommodate truly unlimited paging. In most instance you would fetch the next/prev page from the cache but if you walked off either end of the cached query you could query again and cache that result. In fact two levels of caching might be an actual implementation requirement to handle all cases. > We can also use Simple Paged Results, but if I understood correctly it > requires the httpd to maintain an open connection to the LDAP server for > each user and for each page. I'm not sure memcached can be used to move > the connection object among forked httpd processes. Also Simple Paged > Results can only go forward, so no Prev button unless somebody keeps the > results. No, the connection object cannot be moved between processes via memcached because sockets are a property of the process that created it. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Wed Aug 29 13:45:40 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 29 Aug 2012 15:45:40 +0200 Subject: [Freeipa-devel] [PATCH] 0072 Internationalization for public errors In-Reply-To: <50338DB9.3050606@redhat.com> References: <500FFB88.6080902@redhat.com> <50338DB9.3050606@redhat.com> Message-ID: <503E1D04.8070303@redhat.com> On 08/21/2012 03:31 PM, Petr Viktorin wrote: > On 07/25/2012 09:58 AM, Petr Viktorin wrote: >> I spent some time while IPA was installing tracking down error mesages >> that weren't marked for translation :) >> I also changed the error docstrings, as people are likely to look there >> for inspiration. >> >> >> https://fedorahosted.org/freeipa/ticket/1953 > > Attaching rebased patch. > The changes looks OK, good job there. I just found 2 issues: 1) 2 unit tests got broken: ====================================================================== FAIL: test_service[1]: service_mod: Try to update non-existent u'HTTP/testhost1.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM' ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 249, in func = lambda: self.check(nice, **test) File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 262, in check self.check_exception(nice, cmd, args, options, expected) File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 288, in check_exception assert_deepequal(expected.strerror, e.strerror) File "/root/freeipa-master/tests/util.py", line 343, in assert_deepequal VALUE % (doc, expected, got, stack) AssertionError: assert_deepequal: expected != got. expected = u'HTTP/testhost1.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM: service not found' got = u'no such entry' path = () ====================================================================== FAIL: test_service[23]: service_mod: Try to update non-existent u'HTTP/testhost1.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM' ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 249, in func = lambda: self.check(nice, **test) File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 262, in check self.check_exception(nice, cmd, args, options, expected) File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 288, in check_exception assert_deepequal(expected.strerror, e.strerror) File "/root/freeipa-master/tests/util.py", line 343, in assert_deepequal VALUE % (doc, expected, got, stack) AssertionError: assert_deepequal: expected != got. expected = u'HTTP/testhost1.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM: service not found' got = u'no such entry' path = () ---------------------------------------------------------------------- You would have the properly catch and process exception in service_mod plugin in the following line: (dn, entry_attrs_old) = ldap.get_entry(dn, ['usercertificate']) to be able to update tests like that. 2) While at internationalizing public errors, we could fix the following exceptions in bindinstance.py. They are not supposed to be i18n-ed, nor they are correct and will crash horribly. @@ -277,7 +278,7 @@ def add_zone(name, zonemgr=None, dns_backup=None, ns_hostname=None, ns_ip_addres # automatically retrieve list of DNS masters dns_masters = api.Object.dnsrecord.get_dns_masters() if not dns_masters: - raise errors.NotFound("No IPA server with DNS support found!") + raise errors.NotFound(_("No IPA server with DNS support found!")) ns_main = dns_masters.pop(0) ns_replicas = dns_masters addresses = resolve_host(ns_main) @@ -321,7 +322,7 @@ def add_reverse_zone(zone, ns_hostname=None, ns_ip_address=None, # automatically retrieve list of DNS masters dns_masters = api.Object.dnsrecord.get_dns_masters() if not dns_masters: - raise errors.NotFound("No IPA server with DNS support found!") + raise errors.NotFound(_("No IPA server with DNS support found!")) ns_main = dns_masters.pop(0) ns_replicas = dns_masters addresses = resolve_host(ns_main) Martin From pvoborni at redhat.com Wed Aug 29 13:49:10 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 29 Aug 2012 15:49:10 +0200 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503D0E9E.6000703@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> Message-ID: <503E1DD6.9050504@redhat.com> On 08/28/2012 08:31 PM, Endi Sukma Dewata wrote: > On 8/28/2012 11:23 AM, Petr Vobornik wrote: >>> Your possible solution does not address how many results are fetched >>> (unless I misunderstood). > > >> If paging is enabled it doesn't, but it expects, that admin will disable >> it for larger setups.For smaller setups it isn't of much an issue. If >> paging is disabled, the limit is server 'search size limit' or >> --sizelimit option supplied by Web UI. >> >>> I'm not sure how per-user preferences are handled in browsers, but don't >>> forget we now have session support in the server. Server session data is >>> available for use. >> >> I was thinking about using browser local storage (basically key-value >> DB). It has a benefit over session, that it survives a browser restart >> but it should contain only non-sensitive data (other users may see it). > > You mean browser cookie? No. https://developer.mozilla.org/en-US/docs/DOM/Storage Petr Spacek also suggested to have an attribute in user object to store IPA specific data as JSON. Therefore changing browser wouldn't be an issue. > >>> If you don't want to fetch every record to implement paging smartly in >>> conjunction with it's performance issues why not do the query on the >>> server, cache the results in the session, and have the RPC return the >>> total number of results plus a subset of the result. Another RPC could >>> retrieve the next/previous subset of results from the cached result in >>> the session. >> >> I think most software do paging like this. I don't know the reasons for >> not doing it that way first time. My solution was counting with that we >> still don't want to do it. Endi do you know the reasons? Earlier >> sessions didn't exists, but it is doable without them too. > > Yes, at the time the sessions didn't exist and we needed a > quick/temporary solution. I agree ideally this should be done on the > server side, but how would the server maintain the paging information? > > The server can keep the search result (either just the pkey list or the > entire entries) in memcached, but the result might be large and there > might be multiple users accessing multiple search pages, so the total > memory requirement could be large. I can imagine that if used only by admins it may be OK but if such functionality would be used by other applications it may become problem. > > If we don't want the server to keep the search result, the server could > rerun the search and return only the requested page and discard most of > the results each time the user change page. This may affect server > performance. > > We can also use Simple Paged Results, but if I understood correctly it > requires the httpd to maintain an open connection to the LDAP server for > each user and for each page. I'm not sure memcached can be used to move > the connection object among forked httpd processes. Also Simple Paged > Results can only go forward, so no Prev button unless somebody keeps the > results. Seems unusable. > > Another possibility is to use VLV, but it seems to require open > connection too and only works with a predefined filter. Apart from the possible connection problem I think it is very usable for default views of search pages. User can freely page. I don't understand from the wiki page if it can reasonably obtain total record count. > > Here's the page I wrote about this: > http://freeipa.org/page/IPAv3_Directory_Browsing > > Currently we're using Option #2 but I'm not sure if we use SPR between > httpd and the LDAP server. But even with SPR the result is still bound > by some limits. It looks you have to connect as Directory Manager to get > the complete list of pkey list/entries. See the table in this page: This is one of the main reasons why I say it is broken. > > http://directory.fedoraproject.org/wiki/Simple_Paged_Results_Design > > The current implementation (keeping the pkey list in the UI) is > conceptually similar to the front-end approach described in the above page. > IMO a modification of a hybrid solution may be best: When user opens page a VLV result would be shown. User can page and see all the data. Currently we don't support custom sorting so disability of custom sorting isn't an issue. With predefined sorts we can even improved these possibilities. If there is many records and user don't want to go through pages he will use search. If the search string is good he should fit to search limit so I wouldn't use paging at this moment. If not the case and he really don't know how to improve the filter he should have option to enable paging (current implementation) and go through the records by hand. Such record set should fit to limit of 2000 records. If not the filter is really bad. If we don't want to implement VLV or until that time I would set not-paging as default setting. >>> I don't think there any need in JSON formatted data for pretty printing >>> with indentation. Is it an accident or oversight we're pretty printing >>> the JSON data in an RPC. For large data sets compression would be a win, >>> but most of our RPC communication is not large. Compression has an >>> overhead. With small data you'll use more cycles compressing and >>> uncompressing than you would sending verbose but small data blobs. >>> Compression should be an RPC option specified by either side of the >>> connection and the receiver should be prepared to conditionally >>> uncompress based on a flag value in the RPC. If you use server session >>> data to support paging you may not need to introduce compression since >>> you would only be passing one page of data at a time. >>> >>> User specified page size (without limitation) is an absolute necessity. >>> I am frequently annoyed by web sites which either do not allow me to >>> specify page size or constrain it to ridiculous hard coded limits such >>> as 10, 20 or 30. >> >> +1 > > Agreed, but if we implement the single continuous page I described in > the earlier email the page size won't be much of an issue anymore. > It may matter. For example when testing Web UI permissions. It often requires to navigate to the last page. I can enter the number directly and press enter or extend it for times (hypothetically) but I would rather see those 90 permissions right away because I can be at the list end in 0.5s. We can also improve pager to offer some subset of pages. For example: First Prev 22 23 *24* 25 26 Next Last Page [ ] where 24 is current page. Regarding the continuous paging: I would extend the page by clicking at a bar at the end of the list and/or by mouse-scrolling at the end of the list? -- Petr Vobornik From rmeggins at redhat.com Wed Aug 29 14:35:45 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 29 Aug 2012 08:35:45 -0600 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E1A7A.7040303@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1A7A.7040303@redhat.com> Message-ID: <503E28C1.3000206@redhat.com> On 08/29/2012 07:34 AM, John Dennis wrote: > On 08/28/2012 02:31 PM, Endi Sukma Dewata wrote: >> The server can keep the search result (either just the pkey list or the >> entire entries) in memcached, but the result might be large and there >> might be multiple users accessing multiple search pages, so the total >> memory requirement could be large. > > The default max size of an entry in memcached is 1MB. It can be > increased to an upper limit of 128MB (but the memcached implementors > do not recommend this due to degraded performance and the impact on > the system). > > The session data is stored in a dict. You would be sharing the session > data with other parts of the system. Currently that only includes the > authentication data which is relatively small. I believe there is also > some minor bookkeeping overhead that detracts from the per item total. > > If we need to exceed the upper bound for paged data I suppose we could > implement caching within the cache. Almost 1MB of data is a lot of > paging (and that limit can be increased), it would take a fair amount > of paging to consume all that data. But the cached query could be > broken up into "cached chunks" to limit the impact on memcached and to > accommodate truly unlimited paging. In most instance you would fetch > the next/prev page from the cache but if you walked off either end of > the cached query you could query again and cache that result. In fact > two levels of caching might be an actual implementation requirement to > handle all cases. > > >> We can also use Simple Paged Results, but if I understood correctly it >> requires the httpd to maintain an open connection to the LDAP server for >> each user and for each page. Not for each user. In 389-ds-base-1.2.11 you can have multiple simple paged result searches on a single connection - see https://fedorahosted.org/389/ticket/260 This is the problem that VLV and Simple Paged Results are trying to solve - how to allow users to scroll/page through very large result sets. >> I'm not sure memcached can be used to move >> the connection object among forked httpd processes. Also Simple Paged >> Results can only go forward, so no Prev button unless somebody keeps the >> results. > > No, the connection object cannot be moved between processes via > memcached because sockets are a property of the process that created it. > From jdennis at redhat.com Wed Aug 29 14:42:35 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 29 Aug 2012 10:42:35 -0400 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E28C1.3000206@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1A7A.7040303@redhat.com> <503E28C1.3000206@redhat.com> Message-ID: <503E2A5B.1070402@redhat.com> On 08/29/2012 10:35 AM, Rich Megginson wrote: > On 08/29/2012 07:34 AM, John Dennis wrote: >> On 08/28/2012 02:31 PM, Endi Sukma Dewata wrote: >>> We can also use Simple Paged Results, but if I understood correctly it >>> requires the httpd to maintain an open connection to the LDAP server for >>> each user and for each page. > > Not for each user. In 389-ds-base-1.2.11 you can have multiple simple > paged result searches on a single connection - see > https://fedorahosted.org/389/ticket/260 Well this is the crux of the problem. We do not maintain a connection per user. LDAP connections exist for the duration of a single IPA RPC call. Those RPC calls may be multiplexed across multiple IPA server instances, each of which is it's own process. Our LDAP connections are very short lived and are scattered across processes. > This is the problem that VLV and Simple Paged Results are trying to > solve - how to allow users to scroll/page through very large result sets. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rmeggins at redhat.com Wed Aug 29 14:49:27 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 29 Aug 2012 08:49:27 -0600 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E2A5B.1070402@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1A7A.7040303@redhat.com> <503E28C1.3000206@redhat.com> <503E2A5B.1070402@redhat.com> Message-ID: <503E2BF7.20106@redhat.com> On 08/29/2012 08:42 AM, John Dennis wrote: > On 08/29/2012 10:35 AM, Rich Megginson wrote: >> On 08/29/2012 07:34 AM, John Dennis wrote: >>> On 08/28/2012 02:31 PM, Endi Sukma Dewata wrote: > >>>> We can also use Simple Paged Results, but if I understood correctly it >>>> requires the httpd to maintain an open connection to the LDAP >>>> server for >>>> each user and for each page. >> >> Not for each user. In 389-ds-base-1.2.11 you can have multiple simple >> paged result searches on a single connection - see >> https://fedorahosted.org/389/ticket/260 > > Well this is the crux of the problem. We do not maintain a connection > per user. LDAP connections exist for the duration of a single IPA RPC > call. Those RPC calls may be multiplexed across multiple IPA server > instances, each of which is it's own process. > > Our LDAP connections are very short lived and are scattered across > processes. So it sounds like, in order to be useful to IPA, we need to extend simple paged results: 1) ability to have the "cookie" (i.e. the results list and current position in that list) live outside of a connection 2) ability to go backwards in a list Is this correct? If so, please file 389 RFE's for these. > >> This is the problem that VLV and Simple Paged Results are trying to >> solve - how to allow users to scroll/page through very large result >> sets. > > From edewata at redhat.com Wed Aug 29 15:16:37 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 29 Aug 2012 10:16:37 -0500 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E2BF7.20106@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1A7A.7040303@redhat.com> <503E28C1.3000206@redhat.com> <503E2A5B.1070402@redhat.com> <503E2BF7.20106@redhat.com> Message-ID: <503E3255.3070406@redhat.com> On 8/29/2012 9:49 AM, Rich Megginson wrote: >>>>> We can also use Simple Paged Results, but if I understood correctly it >>>>> requires the httpd to maintain an open connection to the LDAP >>>>> server foreach user and for each page. >>> >>> Not for each user. In 389-ds-base-1.2.11 you can have multiple simple >>> paged result searches on a single connection - see >>> https://fedorahosted.org/389/ticket/260 >> >> Well this is the crux of the problem. We do not maintain a connection >> per user. LDAP connections exist for the duration of a single IPA RPC >> call. Those RPC calls may be multiplexed across multiple IPA server >> instances, each of which is it's own process. >> >> Our LDAP connections are very short lived and are scattered across >> processes. > > So it sounds like, in order to be useful to IPA, we need to extend > simple paged results: > 1) ability to have the "cookie" (i.e. the results list and current > position in that list) live outside of a connection > 2) ability to go backwards in a list > > Is this correct? If so, please file 389 RFE's for these. For (1) how does the httpd send the information that it wants to use the result list from a previous connection? Is it going to use a new LDAP control? Or would there be a session ID? If we implement (2) does it mean the pages still need to be accessed sequentially, or can we jump to any random page? Also if I understood correctly the LDAP connections are made using user credentials, not Directory Manager, so things like nsslapd-sizelimit will apply. Does it mean a non-admin cannot browse the entire directory? -- Endi S. Dewata From rmeggins at redhat.com Wed Aug 29 15:24:32 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 29 Aug 2012 09:24:32 -0600 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E3255.3070406@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1A7A.7040303@redhat.com> <503E28C1.3000206@redhat.com> <503E2A5B.1070402@redhat.com> <503E2BF7.20106@redhat.com> <503E3255.3070406@redhat.com> Message-ID: <503E3430.7090908@redhat.com> On 08/29/2012 09:16 AM, Endi Sukma Dewata wrote: > On 8/29/2012 9:49 AM, Rich Megginson wrote: >>>>>> We can also use Simple Paged Results, but if I understood >>>>>> correctly it >>>>>> requires the httpd to maintain an open connection to the LDAP >>>>>> server foreach user and for each page. >>>> >>>> Not for each user. In 389-ds-base-1.2.11 you can have multiple simple >>>> paged result searches on a single connection - see >>>> https://fedorahosted.org/389/ticket/260 >>> >>> Well this is the crux of the problem. We do not maintain a connection >>> per user. LDAP connections exist for the duration of a single IPA RPC >>> call. Those RPC calls may be multiplexed across multiple IPA server >>> instances, each of which is it's own process. >>> >>> Our LDAP connections are very short lived and are scattered across >>> processes. >> >> So it sounds like, in order to be useful to IPA, we need to extend >> simple paged results: >> 1) ability to have the "cookie" (i.e. the results list and current >> position in that list) live outside of a connection >> 2) ability to go backwards in a list >> >> Is this correct? If so, please file 389 RFE's for these. > > For (1) how does the httpd send the information that it wants to use > the result list from a previous connection? Is it going to use a new > LDAP control? Not sure. Might be able to use the existing simple paged result control. > Or would there be a session ID? > > If we implement (2) does it mean the pages still need to be accessed > sequentially, or can we jump to any random page? We should be able support random page access. But I think we could support the ability to go backwards from the current page without random access support. > > Also if I understood correctly the LDAP connections are made using > user credentials, not Directory Manager, so things like > nsslapd-sizelimit will apply. Does it mean a non-admin cannot browse > the entire directory? in 1.2.10 we have different limits for paged result searches: https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/User_Account_Management-Setting_Resource_Limits_Based_on_the_Bind_DN.html From mkosek at redhat.com Wed Aug 29 15:32:57 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 29 Aug 2012 17:32:57 +0200 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503BE777.9040204@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> <503BA065.1060207@redhat.com> <1346099236.3617.6.camel@priserak> <503BE777.9040204@redhat.com> Message-ID: <503E3629.3030908@redhat.com> On 08/27/2012 11:32 PM, Rich Megginson wrote: > On 08/27/2012 02:27 PM, Martin Kosek wrote: >> On Mon, 2012-08-27 at 10:29 -0600, Rich Megginson wrote: >>> On 08/27/2012 10:24 AM, Martin Kosek wrote: >>>> On 08/17/2012 04:00 PM, Rich Megginson wrote: >>>>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>>>> Hi guys, >>>>>> >>>>>> I am now investigating ticket #2866: >>>>>> https://fedorahosted.org/freeipa/ticket/2866 >>>>>> >>>>>> And I am thinking about possible solutions for this problem. In a >>>>>> nutshell, we do not properly check referential integrity in some IPA >>>>>> objects where we keep one-way DN references to other objects, e.g. in >>>>>> - managedBy attribute for a host object >>>>>> - memberhost attribute for HBAC rule object >>>>>> - memberuser attribute for user object >>>>>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>>>>> #2866) >>>>>> ... >>>>>> >>>>>> Currently, I see 2 approaches to solve this: >>>>>> 1) Add relevant checks to our ipalib plugins where problematic >>>>>> operations with these operations are being executed (like we do for >>>>>> selinuxusermap's seealso attribute in HBAC plugin) >>>>>> This of course would not prevent direct LDAP deletes. >>>>>> >>>>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>>>> callbacks and check that this object's DN is not referenced in other >>>>>> objects. And if it does, it would reject such modification. Second >>>>>> option would be to delete the attribute value with now invalid >>>>>> reference. This would be probably more suitable for example for >>>>>> references to user objects. >>>>>> >>>>>> Any comments to these possible approaches are welcome. >>>>>> >>>>>> Rich, do you think that as an alternative to these 2 approaches, >>>>>> memberOf plugin could be eventually modified to do this task? >>>>> This is very similar to the referential integrity plugin already in 389, >>>>> except >>>>> instead of cleaning up references to moved and deleted entries, you want >>>>> it to >>>>> prevent moving or deleting an entry if that entry is referenced by the >>>>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other >>>>> entry. >>>> I think that using or enhancing current DS referential integrity plugin >>>> will be >>>> the best and the most consistent way to go. >>>> >>>> We already use that plugin for some user attributes like "manager" or >>>> "secretary". seeAlso is already covered by default, so for example seeAlso >>>> attribute in SELinux usermap object referencing an HBAC rule will get removed >>>> when relevant HBAC rule is removed (I just checked that). >>>> >>>>> Note that the managed entry plugin (mep) already handles this for the >>>>> managedby >>>>> attribute. >>>> I assume you are referencing "mepmanagedby" and "mepmanagedentry" attributes >>>> which then produce errors like this one: >>>> >>>> # ipa netgroup-del foo >>>> ipa: ERROR: Server is unwilling to perform: Deleting a managed entry is not >>>> allowed. It needs to be manually unlinked first. >>>> >>>> managedBy attribute used by hosts objects I had in mind seems to not be >>>> covered. >>>> >>>> But you are right, this is pretty much what I wanted. Though in case of MEP, >>>> there is a link in both referenced objects, but in our case, we have just the >>>> one-way link. >>>> >>>>> Are you already using the memberof plugin for >>>>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>>>> >>>>> This doesn't seem like a job for memberof, this seems like more of a new >>>>> check >>>>> for the referential integrity plugin. >>>>> >>>> I am now considering if current move/delete clean up already present in >>>> Referential Integrity plugin would not be sufficient for us. >>>> >>>> Rich, please correct me if I am wrong, but in that case, we would just need to >>>> add relevant attribute names >>>> (memberhost/memberuser/memberallowcmd/memberdenycmd...) to Referential >>>> Integrity plugin configuration as nsslapd-pluginarg7, nsslapd-pluginarg8, ... >>>> I wonder if there would be some performance issues if we add attributes to the >>>> list this way. >>> No, not if they are indexed for presence and equality. >>> >>> But referential integrity will not prevent deletion or moving entries - >>> it will delete/move references. >> I understand that. After some reconsideration, I think that cleaning up >> dangling references as postop should be OK for most of the referential >> attributes we use. But I would like a second opinion on that. >> >> So do I understand it correctly, that in case we want to go this way in >> IPA, the recommended approach DS-wise would be to: >> - add presence and equality indexes to IPA for the attributes we want to >> have checked for referential integrity >> - configure DS Referential Integrity plugin to check these attributes >> >> Or would it be better to wait on relevant DS changes you mentioned that >> Noriko is working on? > > Also look at the Linked Attribute plugin - it may be able to do what you want > right now - http://port389.org/wiki/Linked_Attributes_Design Yes, but this plugin is only relevant for bi-directional links and not for uni-directional links we have in IPA (memberhost/memberuser/memberallowcmd/memberdenycmd...), isn't it? I.e. there is nothing to be filled for "managedType" attribute of the config. Maybe if the link plugin is enhanced for the uni-directional links too it would help us. It would help if I would be able to specify a links like that: dn: cn=memberhost Link, cn=Linked Attribute Plugin, cn=config objectclass: extensibleObject linkType: memberhost linkScope: cn=sudorules,cn=sudo,$SUFFIX Without having to specify a managed link. The plugin would then just hda pretty much do what we discussed before - keep referential integrity of the uni-directional plugin. Martin > >> >> Thanks, >> Martin >> >>>> Rob, do you think that cleaning up the broken references during a DS postop >>>> instead of raising an preop error is OK for IPA references? I went through the >>>> referential attributes we use ("git grep LDAPAddMember") and I think it should >>>> be sufficient. We could cover some special cases with a query in our framework >>>> like you did in hbacrule-del. >>>> >>>> Martin >> > From rmeggins at redhat.com Wed Aug 29 16:35:55 2012 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 29 Aug 2012 10:35:55 -0600 Subject: [Freeipa-devel] Ticket #2866 - referential integrity in IPA In-Reply-To: <503E3629.3030908@redhat.com> References: <1345211089.3796.34.camel@priserak> <502E4E78.1090407@redhat.com> <503B9F4E.4010005@redhat.com> <503BA065.1060207@redhat.com> <1346099236.3617.6.camel@priserak> <503BE777.9040204@redhat.com> <503E3629.3030908@redhat.com> Message-ID: <503E44EB.5080204@redhat.com> On 08/29/2012 09:32 AM, Martin Kosek wrote: > On 08/27/2012 11:32 PM, Rich Megginson wrote: >> On 08/27/2012 02:27 PM, Martin Kosek wrote: >>> On Mon, 2012-08-27 at 10:29 -0600, Rich Megginson wrote: >>>> On 08/27/2012 10:24 AM, Martin Kosek wrote: >>>>> On 08/17/2012 04:00 PM, Rich Megginson wrote: >>>>>> On 08/17/2012 07:44 AM, Martin Kosek wrote: >>>>>>> Hi guys, >>>>>>> >>>>>>> I am now investigating ticket #2866: >>>>>>> https://fedorahosted.org/freeipa/ticket/2866 >>>>>>> >>>>>>> And I am thinking about possible solutions for this problem. In a >>>>>>> nutshell, we do not properly check referential integrity in some IPA >>>>>>> objects where we keep one-way DN references to other objects, e.g. in >>>>>>> - managedBy attribute for a host object >>>>>>> - memberhost attribute for HBAC rule object >>>>>>> - memberuser attribute for user object >>>>>>> - memberallowcmd or memberdenycmd for SUDO command object (reported in >>>>>>> #2866) >>>>>>> ... >>>>>>> >>>>>>> Currently, I see 2 approaches to solve this: >>>>>>> 1) Add relevant checks to our ipalib plugins where problematic >>>>>>> operations with these operations are being executed (like we do for >>>>>>> selinuxusermap's seealso attribute in HBAC plugin) >>>>>>> This of course would not prevent direct LDAP deletes. >>>>>>> >>>>>>> 2) Implement a preop DS plugin that would hook to MODRDN and DELETE >>>>>>> callbacks and check that this object's DN is not referenced in other >>>>>>> objects. And if it does, it would reject such modification. Second >>>>>>> option would be to delete the attribute value with now invalid >>>>>>> reference. This would be probably more suitable for example for >>>>>>> references to user objects. >>>>>>> >>>>>>> Any comments to these possible approaches are welcome. >>>>>>> >>>>>>> Rich, do you think that as an alternative to these 2 approaches, >>>>>>> memberOf plugin could be eventually modified to do this task? >>>>>> This is very similar to the referential integrity plugin already in 389, >>>>>> except >>>>>> instead of cleaning up references to moved and deleted entries, you want >>>>>> it to >>>>>> prevent moving or deleting an entry if that entry is referenced by the >>>>>> managedby/memberhost/memberuser/memberallowcmd/memberdenycmd of some other >>>>>> entry. >>>>> I think that using or enhancing current DS referential integrity plugin >>>>> will be >>>>> the best and the most consistent way to go. >>>>> >>>>> We already use that plugin for some user attributes like "manager" or >>>>> "secretary". seeAlso is already covered by default, so for example seeAlso >>>>> attribute in SELinux usermap object referencing an HBAC rule will get removed >>>>> when relevant HBAC rule is removed (I just checked that). >>>>> >>>>>> Note that the managed entry plugin (mep) already handles this for the >>>>>> managedby >>>>>> attribute. >>>>> I assume you are referencing "mepmanagedby" and "mepmanagedentry" attributes >>>>> which then produce errors like this one: >>>>> >>>>> # ipa netgroup-del foo >>>>> ipa: ERROR: Server is unwilling to perform: Deleting a managed entry is not >>>>> allowed. It needs to be manually unlinked first. >>>>> >>>>> managedBy attribute used by hosts objects I had in mind seems to not be >>>>> covered. >>>>> >>>>> But you are right, this is pretty much what I wanted. Though in case of MEP, >>>>> there is a link in both referenced objects, but in our case, we have just the >>>>> one-way link. >>>>> >>>>>> Are you already using the memberof plugin for >>>>>> memberhost/memberuser/memberallowcmd/memberdenycmd? >>>>>> >>>>>> This doesn't seem like a job for memberof, this seems like more of a new >>>>>> check >>>>>> for the referential integrity plugin. >>>>>> >>>>> I am now considering if current move/delete clean up already present in >>>>> Referential Integrity plugin would not be sufficient for us. >>>>> >>>>> Rich, please correct me if I am wrong, but in that case, we would just need to >>>>> add relevant attribute names >>>>> (memberhost/memberuser/memberallowcmd/memberdenycmd...) to Referential >>>>> Integrity plugin configuration as nsslapd-pluginarg7, nsslapd-pluginarg8, ... >>>>> I wonder if there would be some performance issues if we add attributes to the >>>>> list this way. >>>> No, not if they are indexed for presence and equality. >>>> >>>> But referential integrity will not prevent deletion or moving entries - >>>> it will delete/move references. >>> I understand that. After some reconsideration, I think that cleaning up >>> dangling references as postop should be OK for most of the referential >>> attributes we use. But I would like a second opinion on that. >>> >>> So do I understand it correctly, that in case we want to go this way in >>> IPA, the recommended approach DS-wise would be to: >>> - add presence and equality indexes to IPA for the attributes we want to >>> have checked for referential integrity >>> - configure DS Referential Integrity plugin to check these attributes >>> >>> Or would it be better to wait on relevant DS changes you mentioned that >>> Noriko is working on? >> Also look at the Linked Attribute plugin - it may be able to do what you want >> right now - http://port389.org/wiki/Linked_Attributes_Design > Yes, but this plugin is only relevant for bi-directional links and not for > uni-directional links we have in IPA > (memberhost/memberuser/memberallowcmd/memberdenycmd...), isn't it? I.e. there > is nothing to be filled for "managedType" attribute of the config. > > Maybe if the link plugin is enhanced for the uni-directional links too it would > help us. It would help if I would be able to specify a links like that: > > dn: cn=memberhost Link, cn=Linked Attribute Plugin, cn=config > objectclass: extensibleObject > linkType: memberhost > linkScope: cn=sudorules,cn=sudo,$SUFFIX > > Without having to specify a managed link. The plugin would then just hda pretty > much do what we discussed before - keep referential integrity of the > uni-directional plugin. Ok. Please file an RFE with 389 and indicate the time frame and IPA release this is targeted for. > > Martin > >>> Thanks, >>> Martin >>> >>>>> Rob, do you think that cleaning up the broken references during a DS postop >>>>> instead of raising an preop error is OK for IPA references? I went through the >>>>> referential attributes we use ("git grep LDAPAddMember") and I think it should >>>>> be sufficient. We could cover some special cases with a query in our framework >>>>> like you did in hbacrule-del. >>>>> >>>>> Martin From edewata at redhat.com Wed Aug 29 16:50:52 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 29 Aug 2012 11:50:52 -0500 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E1DD6.9050504@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1DD6.9050504@redhat.com> Message-ID: <503E486C.1030006@redhat.com> On 8/29/2012 8:49 AM, Petr Vobornik wrote: >> Another possibility is to use VLV, but it seems to require open >> connection too and only works with a predefined filter. > > Apart from the possible connection problem I think it is very usable for > default views of search pages. User can freely page. I don't understand > from the wiki page if it can reasonably obtain total record count. Correction, VLV doesn't seem to require a continuous connection. The server will return a 'contentCount' in the VLV response: http://www.ietf.org/proceedings/55/I-D/draft-ietf-ldapext-ldapv3-vlv-09.txt > IMO a modification of a hybrid solution may be best: > > When user opens page a VLV result would be shown. User can page and see > all the data. Currently we don't support custom sorting so disability of > custom sorting isn't an issue. With predefined sorts we can even > improved these possibilities. > > If there is many records and user don't want to go through pages he will > use search. If the search string is good he should fit to search limit > so I wouldn't use paging at this moment. If not the case and he really > don't know how to improve the filter he should have option to enable > paging (current implementation) and go through the records by hand. Such > record set should fit to limit of 2000 records. If not the filter is > really bad. The thing is the filter is only used on certain attributes only. For example I have a user with full name "Test User". Searching the "Test User" in the filter doesn't return anything because the server doesn't search the cn or displayName attributes. In that case I'd have to search with first name or last name only, which may return too many results. > If we don't want to implement VLV or until that time I would set > not-paging as default setting. How about this, when you open the search page for the first time it will do a regular search, no paging. Instead of showing the paging controls (Prev & Next buttons) we show "See more results..." If you click that it will rerun the search with paging (which will give us the total number of entries) and show the paging control. >> Agreed, but if we implement the single continuous page I described in >> the earlier email the page size won't be much of an issue anymore. >> > It may matter. For example when testing Web UI permissions. It often > requires to navigate to the last page. I can enter the number directly > and press enter or extend it for times (hypothetically) but I would > rather see those 90 permissions right away because I can be at the list > end in 0.5s. If we use a continuous page we can use a bigger default page size, say 100 entries. We probably could optimize it such that if you scroll directly to the bottom it will load just the last page. We probably could also combine it with initial non-paging search like above, so you'll need to click "See more results...", then the page will extend and you can scroll to the last entry. > We can also improve pager to offer some subset of pages. For example: > > First Prev 22 23 *24* 25 26 Next Last Page [ ] > where 24 is current page. Yes, that will work too. > Regarding the continuous paging: I would extend the page by clicking at > a bar at the end of the list and/or by mouse-scrolling at the end of the > list? Ideally the scroll bar size should correspond to the visible part of the entire page. So you should be able to go anywhere in the entire page by moving the scroll bar around (either by keyboard, mouse click, mouse drag, or mouse scroll). In this case no need to extend the page. The thing is that we might have to create a table with many empty rows, which could be big, and populate them if they become visible. Maybe there's a way to avoid creating empty rows, we'll need to investigate. Alternatively we could do like image search on bing.com. If you bring the scroll bar say within 10% from the bottom it will load the next page. The thing is it will load the page sequentially, and if you continue to the bottom you'll end up with a very big table. Another possibility, we can use the top 10% of the scroll bar as a Prev button, and the bottom %10 as the Next button, and we remove the pages that aren't visible anymore. But this is not much different than the current implementation, only different presentation. -- Endi S. Dewata From edewata at redhat.com Wed Aug 29 17:08:14 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 29 Aug 2012 12:08:14 -0500 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E3430.7090908@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1A7A.7040303@redhat.com> <503E28C1.3000206@redhat.com> <503E2A5B.1070402@redhat.com> <503E2BF7.20106@redhat.com> <503E3255.3070406@redhat.com> <503E3430.7090908@redhat.com> Message-ID: <503E4C7E.9060303@redhat.com> On 8/29/2012 10:24 AM, Rich Megginson wrote: > On 08/29/2012 09:16 AM, Endi Sukma Dewata wrote: >> On 8/29/2012 9:49 AM, Rich Megginson wrote: >>>>>>> We can also use Simple Paged Results, but if I understood >>>>>>> correctly it >>>>>>> requires the httpd to maintain an open connection to the LDAP >>>>>>> server foreach user and for each page. >>>>> >>>>> Not for each user. In 389-ds-base-1.2.11 you can have multiple simple >>>>> paged result searches on a single connection - see >>>>> https://fedorahosted.org/389/ticket/260 >>>> >>>> Well this is the crux of the problem. We do not maintain a connection >>>> per user. LDAP connections exist for the duration of a single IPA RPC >>>> call. Those RPC calls may be multiplexed across multiple IPA server >>>> instances, each of which is it's own process. >>>> >>>> Our LDAP connections are very short lived and are scattered across >>>> processes. >>> >>> So it sounds like, in order to be useful to IPA, we need to extend >>> simple paged results: >>> 1) ability to have the "cookie" (i.e. the results list and current >>> position in that list) live outside of a connection >>> 2) ability to go backwards in a list >>> >>> Is this correct? If so, please file 389 RFE's for these. >> >> For (1) how does the httpd send the information that it wants to use >> the result list from a previous connection? Is it going to use a new >> LDAP control? > > Not sure. Might be able to use the existing simple paged result control. > >> Or would there be a session ID? >> >> If we implement (2) does it mean the pages still need to be accessed >> sequentially, or can we jump to any random page? > > We should be able support random page access. But I think we could > support the ability to go backwards from the current page without random > access support. > >> >> Also if I understood correctly the LDAP connections are made using >> user credentials, not Directory Manager, so things like >> nsslapd-sizelimit will apply. Does it mean a non-admin cannot browse >> the entire directory? > in 1.2.10 we have different limits for paged result searches: > > https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/User_Account_Management-Setting_Resource_Limits_Based_on_the_Bind_DN.html OK. I opened this ticket: https://fedorahosted.org/389/ticket/441 Thanks! -- Endi S. Dewata From rcritten at redhat.com Thu Aug 30 12:53:29 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 30 Aug 2012 08:53:29 -0400 Subject: [Freeipa-devel] [PATCH] 298 Add safe updates for objectClasses In-Reply-To: <503E0DD9.2020905@redhat.com> References: <503E0DD9.2020905@redhat.com> Message-ID: <503F6249.3060209@redhat.com> Martin Kosek wrote: > Current objectclass updates in a form of "replace" update instruction > dependent on exact match of the old object class specification in the > update instruction and the real value in LDAP. However, this approach is > very error prone as object class definition can easily differ as for > example because of unexpected X-ORIGIN value. Such objectclass update > failures may lead to serious malfunctions later. > > Add new update instruction type "replaceoc" with the following format: > replaceoc:OID:new > This update instruction will always replace an objectclass with > specified OID with the new definition. > > https://fedorahosted.org/freeipa/ticket/2440 This works ok. Martin and I had a conversation in IRC about it. This moves from replacing a specific bit of schema with a new one, in all cases. I wonder if we should be more conservative and know what we're replacing in advance. rob From rcritten at redhat.com Thu Aug 30 13:20:57 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 30 Aug 2012 09:20:57 -0400 Subject: [Freeipa-devel] [PATCH] 0008 Fixes different behaviour of permission-mod and show. In-Reply-To: <503B6312.7070309@redhat.com> References: <5034F10D.7050901@redhat.com> <5034F788.30507@redhat.com> <50351954.5070102@redhat.com> <50362614.7050100@redhat.com> <503B6312.7070309@redhat.com> Message-ID: <503F68B9.4060601@redhat.com> Tomas Babej wrote: > On 08/23/2012 02:46 PM, Rob Crittenden wrote: >> Tomas Babej wrote: >>> On 08/22/2012 05:15 PM, Rob Crittenden wrote: >>>> Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> Both commands now produce the same output regarding >>>>> the attributelevelrights. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2875 >>>> >>>> I think some unit tests would be helpful so we don't regress and we >>>> know which other commands this fixes. >>>> >>>> rob >>> >>> I ran the tests for the permission plugin (test_permission_plugin.py) >>> and all of them passed. Tested on clean VM with newly built IPA from the >>> master, so there should be no regression. Results themselves attached. >>> >>> Tomas >> >> Right, but those tests all passed prior to your fix as well. We need a >> test that does a permission-mod and confirms that the rights contains >> the full list of attributes (and perhaps testing any other commands >> that were similarly fixed). >> >> rob >> > > I added unit tests for permission-mod and permission-show. > > Tomas Exactly what I was looking for, thanks. ACK, pushed to master and ipa-3-0. rob From tjaalton at ubuntu.com Fri Aug 31 04:21:52 2012 From: tjaalton at ubuntu.com (Timo Aaltonen) Date: Fri, 31 Aug 2012 07:21:52 +0300 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <503CCA45.1010807@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <503CCA45.1010807@redhat.com> Message-ID: <50403BE0.3080402@ubuntu.com> On 28.08.2012 16:40, Petr Viktorin wrote: > On 08/17/2012 06:04 PM, Ade Lee wrote: >> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>>> Patch attached this time. I should know better than to do this in the >>>>> middle of the night .. >>>>> >>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of >>>>>>>>>>>> changes that >>>>>>>>>>>> will affect IPA. In particular, the following changes will >>>>>>>>>>>> affect >>>>>>>>>>>> current IPA code. >>>>>>>>>>>> >>>>>>>>>>>> * The directory layout of the dogtag instance has changed. >>>>>>>>>>>> Instead of >>>>>>>>>>>> using separate tomcat instances to host different >>>>>>>>>>>> subsystems, the >>>>>>>>>>>> standard dogtag installation will allow one to install a CA. >>>>>>>>>>>> KRA, OCSP >>>>>>>>>>>> and TKS within the same instance. There have been >>>>>>>>>>>> corresponding changes >>>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon >>>>>>>>>>>> (pki-tomcatd, instead >>>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>>> >>>>>>>>>>>> * The default instance will use only four ports (HTTPS, >>>>>>>>>>>> HTTP, AJP and >>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. >>>>>>>>>>>> The default >>>>>>>>>>>> ports will be changed to the standard tomcat ports. As >>>>>>>>>>>> these ports are >>>>>>>>>>>> local to the ipa server machine, this should not cause too much >>>>>>>>>>>> disruption. >>>>>>>>>>>> >>>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>>> >>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding >>>>>>>>>>>> version of >>>>>>>>>>>> tomcatjss. >>>>>>>>>>>> >>>>>>>>>>>> The attached patch integrates all the above changes in IPA >>>>>>>>>>>> installation >>>>>>>>>>>> and maintenance code. Once the patch is applied, users will >>>>>>>>>>>> be able to: >>>>>>>>>>>> >>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to >>>>>>>>>>>> f18/ dogtag >>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to run >>>>>>>>>>>> correctly. >>>>>>>>>>>> This will require the installation of the latest version of >>>>>>>>>>>> tomcatjss as >>>>>>>>>>>> well as the installation of tomcat6. The old-style instance >>>>>>>>>>>> will >>>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched >>>>>>>>>>>> and should >>>>>>>>>>>> continue to work. >>>>>>>>>>>> >>>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>>> >>>>>>>>>>>> 1. Installation with an external CA is not yet completed in >>>>>>>>>>>> the new >>>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>>> >>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>>> >>>>>>>>>>>> 3. A script needs to be written to allow admins to convert >>>>>>>>>>>> their >>>>>>>>>>>> old-style dogtag instances to new style instances, as well >>>>>>>>>>>> as code to >>>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>>> >>>>>>>>>>>> 4. Installation of old-style instances using >>>>>>>>>>>> pkicreate/pkisilent on >>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled >>>>>>>>>>>> soon. >>>>>>>>>>>> >>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect these >>>>>>>>>>>> changes, >>>>>>>>>>>> but is still in flux. In fact, it is our intention to place >>>>>>>>>>>> the dogtag >>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the >>>>>>>>>>>> meantime, it >>>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>>> >>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. Prior >>>>>>>>>>>> to that >>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code >>>>>>>>>>>> in a >>>>>>>>>>>> developer repo that is located at >>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>>> >>>>>>>>>>>> Testing can be done on both f18 and f17 - although the >>>>>>>>>>>> target platform - >>>>>>>>>>>> and the only platform for which official builds will be >>>>>>>>>>>> created is f18. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ade >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Ade, >>>>>>>>>>> >>>>>>>>>>> Thanks for the patch, I started with review and integration >>>>>>>>>>> tests (currently >>>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>>> >>>>>>>>>>> Installation on single master was smooth, it worked just >>>>>>>>>>> fine, even with >>>>>>>>>>> enforced SELinux, without any error - kudos to you and the >>>>>>>>>>> whole dogtag team. >>>>>>>>>>> The resulting logs and the structure of your log directory >>>>>>>>>>> seems improved. I >>>>>>>>>>> believe that the brand new Python installers will make it >>>>>>>>>>> easier to debug >>>>>>>>>>> issues with dogtag installation when they come. When I tried >>>>>>>>>>> our unit tests or >>>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>>> >>>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>>> >>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled >>>>>>>>>>> automatically on >>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>>> >>>>>>>>>> We have a dogtag patch that is currently in review that will >>>>>>>>>> address >>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for >>>>>>>>>> f17+, >>>>>>>>>> rather than f18+ >>>>>>>>>> >>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA >>>>>>>>>>> installation on a >>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this >>>>>>>>>>> expectable? >>>>>>>>>>> >>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag 10 >>>>>>>>>> only, >>>>>>>>>> which will be officially available on f18+. So if you update >>>>>>>>>> to dogtag >>>>>>>>>> 10, you must have this patch and visa versa. We probably need >>>>>>>>>> to add >>>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>>> >>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to >>>>>>>>>> the new >>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue to >>>>>>>>>> work. >>>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>>> >>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had >>>>>>>>>>> dogtag10 as well >>>>>>>>>>> and I got the following error: >>>>>>>>>>> >>>>>>>>>>> # ipa-ca-install >>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>>> ... >>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 >>>>>>>>>>> seconds >>>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>>> >>>>>>>>>>> Your system may be partly configured. >>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>>> >>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for >>>>>>>>>>> details: >>>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>>> >>>>>>>>>>> Root cause: >>>>>>>>>>> ... >>>>>>>>>>> File >>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>>>>>> line >>>>>>>>>>> 625, in __spawn_instance >>>>>>>>>>> "/root/cacert.p12") >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, >>>>>>>>>> rather than >>>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>>> existed! Should not be too bad to fix though - most likely >>>>>>>>>> just need to >>>>>>>>>> move files to the right place. >>>>>>>>> >>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, you >>>>>>>>> probably used >>>>>>>>> that one) and the aforementioned ipa-ca-install run after >>>>>>>>> ipa-replica-install. >>>>>>>>> >>>>>>>> I will be testing this out again. As ipa-ca-install uses the >>>>>>>> same code >>>>>>>> as ipa-replica-install, I would have expected it to work. Did >>>>>>>> you try >>>>>>>> it with selinux in permissive mode? >>>>>>>> >>>>>>> OK -- I tested this out and realized that between the time I >>>>>>> initially >>>>>>> tested this and your tests, we had changed a URL >>>>>>> from /ca/pki/installer/installToken to >>>>>>> /ca/rest/installer/installToken, >>>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>>> >>>>>>> The new patch has been rebased and changes the relevant patch. With >>>>>>> this, the CA replica installs correctly. >>>>>> >>>>>> Umh, where can I find the new rebased patch? :-) >>>>>> >>>>>>> >>>>>>> There was one issue that I encountered. I have seen this once >>>>>>> before >>>>>>> and will need to investigate further. IPA sometimes restarts the >>>>>>> dogtag >>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then >>>>>>> systemctl >>>>>>> start pki-tomcatd.target. I have noticed that occasionally the >>>>>>> start >>>>>>> invocation fails, while the stop and restart invocations succeed >>>>>>> just >>>>>>> fine. This makes absolutely no sense because restart is essentially >>>>>>> just stop and then start. >>>>>>> >>>>>>> I think this is actually a bug in systemd. If I reboot the >>>>>>> machine, it >>>>>>> all works perfectly. If we can't figure out whats going on with >>>>>>> systemd, we can always replace this start/stop with >>>>>>> starting/stopping >>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That >>>>>>> seems to >>>>>>> work all the time with no problems. >>>>>>> >>>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>>> >>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended >>>>>> in systemd to >>>>>> use target units this way? I thought that only service files are >>>>>> meant to be >>>>>> used for manipulation with a service. As you said yourself, using >>>>>> service unit >>>>>> file for restart worked every time. >>>>>> >>>>>> Martin >>>>> >>>> >>>> Now, I tested the rebased patch with VMs with permissive SELinux. >>>> IPA server >>>> install was OK, as always, but replica installation ended up with >>>> error. >>>> Although it got much further than before: >>>> >>>> # ipa-ca-install >>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>> ... >>>> [3/3]: restarting directory server >>>> done configuring pkids. >>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>> [1/14]: creating certificate server user >>>> [2/14]: configuring certificate server instance >>>> [3/14]: disabling nonces >>>> [4/14]: importing CA chain to RA certificate database >>>> [5/14]: fixing RA database permissions >>>> [6/14]: setting up signing cert profile >>>> [7/14]: set up CRL publishing >>>> [8/14]: set certificate subject base >>>> [9/14]: enabling Subject Key Identifier >>>> [10/14]: configuring certificate server to start on boot >>>> [11/14]: configure certmonger for renewals >>>> [12/14]: configure clone certificate renewals >>>> [13/14]: configure Server-Cert certificate renewal >>>> [14/14]: Configure HTTP to proxy connections >>>> done configuring pki-tomcatd. >>>> Restarting the directory and certificate servers >>>> >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>> >>>> It seems like that CA on a replica did not start and so a check for >>>> port 8080 >>>> failed. Attached logs (pki-ca-logs.tgz) may provide more information >>>> to this issue. >>>> >>> This is the systemd issue I mentioned before. I will submit a patch >>> which will change the restart mechanism to restart the service and not >>> the target. >>>> >> >> Patch attached. This patch should be applied on top of the large patch >> (being reviewed). >> >>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>>> (permissive SELinux). Now, the service was upgraded smoothly, CA >>>> service >>>> could be restarted without failure. However, certificate operations now >>>> did not work: >>>> >>>> # ipactl restart >>>> Restarting Directory Service >>>> Restarting KDC Service >>>> Restarting KPASSWD Service >>>> Restarting MEMCACHE Service >>>> Restarting HTTP Service >>>> Restarting CA Service >>>> # ipactl status >>>> Directory Service: RUNNING >>>> KDC Service: RUNNING >>>> KPASSWD Service: RUNNING >>>> MEMCACHE Service: RUNNING >>>> HTTP Service: RUNNING >>>> CA Service: RUNNING >>>> # ipa cert-show 1 >>>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>>> communicate with CMS (Internal Server Error) >>>> >>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>>> >>> The reason for this is that the old dogtag instances are missing: >>> 1. a new parameter in CS.cfg >>> 2. a couple of symlinks >>> >>> I plan to fix this in the dogtag code. Specifically, >>> 1. I will modify the dogtag code to provide a default value in the case >>> where the parameter does not exist. >>> 2. I plan to add a function to the dogtag startup scripts to check and >>> remake any required symlinks. >>> >>> Both of these changes are in the dogtag code, and do not affect this >>> patch. As a workaround, to confirm that the upgrade actually works, do >>> the following manual steps on your upgraded instance: >>> >>> 1. Add the following parameter to CS.cfg >>> pidDir=/var/run >>> >>> 2. Add the following links; >>> cd /var/lib/pki-ca/common/lib >>> ln -s /usr/share/java/apache-commons-codec.jar >>> apache-commons-codec.jar >>> ln -s /usr/share/java/log4j.jar log4j.jar >>> >>> 3 Restart the dogtag instance >>> >>>> HTH, >>>> Martin >>> >>> > > The attached patch conditionalizes the changes, so that IPA supports > both Dogtag versions. > > I didn't put it in platform code for two reasons > - One platform (f17) can have either Dogtag version installed > - I didn't want to conflict with Timo Aaltonen's "platformizing" work. > According to his WIP patches, he plans to move the whole dogtag module > into platform code. Nothing is set in stone yet, I was merely experimenting there :) That said, I'd like to see the dogtag patches in master soon in order to rebase what I have so it could be sent out, and ask for directions where to go from there.. -- t From jdennis at redhat.com Fri Aug 31 13:01:01 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 31 Aug 2012 09:01:01 -0400 Subject: [Freeipa-devel] [PATCH 81] ipa user-find --manager does not find matches Message-ID: <5040B58D.7030102@redhat.com> -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0081-ipa-user-find-manager-does-not-find-matches.patch Type: text/x-patch Size: 2486 bytes Desc: not available URL: From jdennis at redhat.com Fri Aug 31 13:55:24 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 31 Aug 2012 09:55:24 -0400 Subject: [Freeipa-devel] [PATCH 81] ipa user-find --manager does not find matches In-Reply-To: <5040B58D.7030102@redhat.com> References: <5040B58D.7030102@redhat.com> Message-ID: <5040C24C.7030000@redhat.com> The original patch was missing a unit test to verify the fix. This version of the patch now includes a unit test. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jdennis-0081-1-ipa-user-find-manager-does-not-find-matches.patch Type: text/x-patch Size: 4008 bytes Desc: not available URL: From pviktori at redhat.com Fri Aug 31 14:53:35 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 31 Aug 2012 16:53:35 +0200 Subject: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18 In-Reply-To: <503CCA45.1010807@redhat.com> References: <1344456327.21349.37.camel@aleeredhat.laptop> <502B86F5.4070008@redhat.com> <1345038855.21349.144.camel@aleeredhat.laptop> <502BB362.4090300@redhat.com> <1345096405.21349.176.camel@aleeredhat.laptop> <502C9D57.2060105@redhat.com> <1345116527.21349.178.camel@aleeredhat.laptop> <1345135508.2452.4.camel@priserak> <1345210494.21349.190.camel@aleeredhat.laptop> <1345219494.21349.193.camel@aleeredhat.laptop> <503CCA45.1010807@redhat.com> Message-ID: <5040CFEF.8040100@redhat.com> On 08/28/2012 03:40 PM, Petr Viktorin wrote: > On 08/17/2012 06:04 PM, Ade Lee wrote: >> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>>> Patch attached this time. I should know better than to do this in the >>>>> middle of the night .. >>>>> >>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of >>>>>>>>>>>> changes that >>>>>>>>>>>> will affect IPA. In particular, the following changes will >>>>>>>>>>>> affect >>>>>>>>>>>> current IPA code. >>>>>>>>>>>> >>>>>>>>>>>> * The directory layout of the dogtag instance has changed. >>>>>>>>>>>> Instead of >>>>>>>>>>>> using separate tomcat instances to host different >>>>>>>>>>>> subsystems, the >>>>>>>>>>>> standard dogtag installation will allow one to install a CA. >>>>>>>>>>>> KRA, OCSP >>>>>>>>>>>> and TKS within the same instance. There have been >>>>>>>>>>>> corresponding changes >>>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon >>>>>>>>>>>> (pki-tomcatd, instead >>>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>>> >>>>>>>>>>>> * The default instance will use only four ports (HTTPS, >>>>>>>>>>>> HTTP, AJP and >>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. >>>>>>>>>>>> The default >>>>>>>>>>>> ports will be changed to the standard tomcat ports. As >>>>>>>>>>>> these ports are >>>>>>>>>>>> local to the ipa server machine, this should not cause too much >>>>>>>>>>>> disruption. >>>>>>>>>>>> >>>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>>> >>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding >>>>>>>>>>>> version of >>>>>>>>>>>> tomcatjss. >>>>>>>>>>>> >>>>>>>>>>>> The attached patch integrates all the above changes in IPA >>>>>>>>>>>> installation >>>>>>>>>>>> and maintenance code. Once the patch is applied, users will >>>>>>>>>>>> be able to: >>>>>>>>>>>> >>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to >>>>>>>>>>>> f18/ dogtag >>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to run >>>>>>>>>>>> correctly. >>>>>>>>>>>> This will require the installation of the latest version of >>>>>>>>>>>> tomcatjss as >>>>>>>>>>>> well as the installation of tomcat6. The old-style instance >>>>>>>>>>>> will >>>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched >>>>>>>>>>>> and should >>>>>>>>>>>> continue to work. >>>>>>>>>>>> >>>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>>> >>>>>>>>>>>> 1. Installation with an external CA is not yet completed in >>>>>>>>>>>> the new >>>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>>> >>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>>> >>>>>>>>>>>> 3. A script needs to be written to allow admins to convert >>>>>>>>>>>> their >>>>>>>>>>>> old-style dogtag instances to new style instances, as well >>>>>>>>>>>> as code to >>>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>>> >>>>>>>>>>>> 4. Installation of old-style instances using >>>>>>>>>>>> pkicreate/pkisilent on >>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled >>>>>>>>>>>> soon. >>>>>>>>>>>> >>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect these >>>>>>>>>>>> changes, >>>>>>>>>>>> but is still in flux. In fact, it is our intention to place >>>>>>>>>>>> the dogtag >>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the >>>>>>>>>>>> meantime, it >>>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>>> >>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. Prior >>>>>>>>>>>> to that >>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code >>>>>>>>>>>> in a >>>>>>>>>>>> developer repo that is located at >>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>>> >>>>>>>>>>>> Testing can be done on both f18 and f17 - although the >>>>>>>>>>>> target platform - >>>>>>>>>>>> and the only platform for which official builds will be >>>>>>>>>>>> created is f18. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ade >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Ade, >>>>>>>>>>> >>>>>>>>>>> Thanks for the patch, I started with review and integration >>>>>>>>>>> tests (currently >>>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>>> >>>>>>>>>>> Installation on single master was smooth, it worked just >>>>>>>>>>> fine, even with >>>>>>>>>>> enforced SELinux, without any error - kudos to you and the >>>>>>>>>>> whole dogtag team. >>>>>>>>>>> The resulting logs and the structure of your log directory >>>>>>>>>>> seems improved. I >>>>>>>>>>> believe that the brand new Python installers will make it >>>>>>>>>>> easier to debug >>>>>>>>>>> issues with dogtag installation when they come. When I tried >>>>>>>>>>> our unit tests or >>>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>>> >>>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>>> >>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled >>>>>>>>>>> automatically on >>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>>> >>>>>>>>>> We have a dogtag patch that is currently in review that will >>>>>>>>>> address >>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for >>>>>>>>>> f17+, >>>>>>>>>> rather than f18+ >>>>>>>>>> >>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA >>>>>>>>>>> installation on a >>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this >>>>>>>>>>> expectable? >>>>>>>>>>> >>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag 10 >>>>>>>>>> only, >>>>>>>>>> which will be officially available on f18+. So if you update >>>>>>>>>> to dogtag >>>>>>>>>> 10, you must have this patch and visa versa. We probably need >>>>>>>>>> to add >>>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>>> >>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to >>>>>>>>>> the new >>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue to >>>>>>>>>> work. >>>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>>> >>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had >>>>>>>>>>> dogtag10 as well >>>>>>>>>>> and I got the following error: >>>>>>>>>>> >>>>>>>>>>> # ipa-ca-install >>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>>> ... >>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 >>>>>>>>>>> seconds >>>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>>> >>>>>>>>>>> Your system may be partly configured. >>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>>> >>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for >>>>>>>>>>> details: >>>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>>> >>>>>>>>>>> Root cause: >>>>>>>>>>> ... >>>>>>>>>>> File >>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>>>>>> line >>>>>>>>>>> 625, in __spawn_instance >>>>>>>>>>> "/root/cacert.p12") >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, >>>>>>>>>> rather than >>>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>>> existed! Should not be too bad to fix though - most likely >>>>>>>>>> just need to >>>>>>>>>> move files to the right place. >>>>>>>>> >>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, you >>>>>>>>> probably used >>>>>>>>> that one) and the aforementioned ipa-ca-install run after >>>>>>>>> ipa-replica-install. >>>>>>>>> >>>>>>>> I will be testing this out again. As ipa-ca-install uses the >>>>>>>> same code >>>>>>>> as ipa-replica-install, I would have expected it to work. Did >>>>>>>> you try >>>>>>>> it with selinux in permissive mode? >>>>>>>> >>>>>>> OK -- I tested this out and realized that between the time I >>>>>>> initially >>>>>>> tested this and your tests, we had changed a URL >>>>>>> from /ca/pki/installer/installToken to >>>>>>> /ca/rest/installer/installToken, >>>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>>> >>>>>>> The new patch has been rebased and changes the relevant patch. With >>>>>>> this, the CA replica installs correctly. >>>>>> >>>>>> Umh, where can I find the new rebased patch? :-) >>>>>> >>>>>>> >>>>>>> There was one issue that I encountered. I have seen this once >>>>>>> before >>>>>>> and will need to investigate further. IPA sometimes restarts the >>>>>>> dogtag >>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then >>>>>>> systemctl >>>>>>> start pki-tomcatd.target. I have noticed that occasionally the >>>>>>> start >>>>>>> invocation fails, while the stop and restart invocations succeed >>>>>>> just >>>>>>> fine. This makes absolutely no sense because restart is essentially >>>>>>> just stop and then start. >>>>>>> >>>>>>> I think this is actually a bug in systemd. If I reboot the >>>>>>> machine, it >>>>>>> all works perfectly. If we can't figure out whats going on with >>>>>>> systemd, we can always replace this start/stop with >>>>>>> starting/stopping >>>>>>> the service directly. (pki-tomcatd at pki-tomcat.service). That >>>>>>> seems to >>>>>>> work all the time with no problems. >>>>>>> >>>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>>> >>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended >>>>>> in systemd to >>>>>> use target units this way? I thought that only service files are >>>>>> meant to be >>>>>> used for manipulation with a service. As you said yourself, using >>>>>> service unit >>>>>> file for restart worked every time. >>>>>> >>>>>> Martin >>>>> >>>> >>>> Now, I tested the rebased patch with VMs with permissive SELinux. >>>> IPA server >>>> install was OK, as always, but replica installation ended up with >>>> error. >>>> Although it got much further than before: >>>> >>>> # ipa-ca-install >>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>> ... >>>> [3/3]: restarting directory server >>>> done configuring pkids. >>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>> [1/14]: creating certificate server user >>>> [2/14]: configuring certificate server instance >>>> [3/14]: disabling nonces >>>> [4/14]: importing CA chain to RA certificate database >>>> [5/14]: fixing RA database permissions >>>> [6/14]: setting up signing cert profile >>>> [7/14]: set up CRL publishing >>>> [8/14]: set certificate subject base >>>> [9/14]: enabling Subject Key Identifier >>>> [10/14]: configuring certificate server to start on boot >>>> [11/14]: configure certmonger for renewals >>>> [12/14]: configure clone certificate renewals >>>> [13/14]: configure Server-Cert certificate renewal >>>> [14/14]: Configure HTTP to proxy connections >>>> done configuring pki-tomcatd. >>>> Restarting the directory and certificate servers >>>> >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>> >>>> It seems like that CA on a replica did not start and so a check for >>>> port 8080 >>>> failed. Attached logs (pki-ca-logs.tgz) may provide more information >>>> to this issue. >>>> >>> This is the systemd issue I mentioned before. I will submit a patch >>> which will change the restart mechanism to restart the service and not >>> the target. >>>> >> >> Patch attached. This patch should be applied on top of the large patch >> (being reviewed). >> >>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>>> (permissive SELinux). Now, the service was upgraded smoothly, CA >>>> service >>>> could be restarted without failure. However, certificate operations now >>>> did not work: >>>> >>>> # ipactl restart >>>> Restarting Directory Service >>>> Restarting KDC Service >>>> Restarting KPASSWD Service >>>> Restarting MEMCACHE Service >>>> Restarting HTTP Service >>>> Restarting CA Service >>>> # ipactl status >>>> Directory Service: RUNNING >>>> KDC Service: RUNNING >>>> KPASSWD Service: RUNNING >>>> MEMCACHE Service: RUNNING >>>> HTTP Service: RUNNING >>>> CA Service: RUNNING >>>> # ipa cert-show 1 >>>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>>> communicate with CMS (Internal Server Error) >>>> >>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>>> >>> The reason for this is that the old dogtag instances are missing: >>> 1. a new parameter in CS.cfg >>> 2. a couple of symlinks >>> >>> I plan to fix this in the dogtag code. Specifically, >>> 1. I will modify the dogtag code to provide a default value in the case >>> where the parameter does not exist. >>> 2. I plan to add a function to the dogtag startup scripts to check and >>> remake any required symlinks. >>> >>> Both of these changes are in the dogtag code, and do not affect this >>> patch. As a workaround, to confirm that the upgrade actually works, do >>> the following manual steps on your upgraded instance: >>> >>> 1. Add the following parameter to CS.cfg >>> pidDir=/var/run >>> >>> 2. Add the following links; >>> cd /var/lib/pki-ca/common/lib >>> ln -s /usr/share/java/apache-commons-codec.jar >>> apache-commons-codec.jar >>> ln -s /usr/share/java/log4j.jar log4j.jar >>> >>> 3 Restart the dogtag instance >>> >>>> HTH, >>>> Martin >>> >>> > > The attached patch conditionalizes the changes, so that IPA supports > both Dogtag versions. > > I didn't put it in platform code for two reasons > - One platform (f17) can have either Dogtag version installed > - I didn't want to conflict with Timo Aaltonen's "platformizing" work. > According to his WIP patches, he plans to move the whole dogtag module > into platform code. > > I used the existence of /usr/bin/pkispawn as a test for Dogtag 10. If > you know a better way, please comment. > > The dogtag_version option is now always added to > > I've reverted the changes to install/ui/test/data/ipa_init.json, so > you'll have to change the ports manually when testing the UI with Dogtag > 10. > > > Attaching a patch with fixed ipa-pki-proxy.conf, as discussed on IRC. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0074-02-Use-Dogtag-10-only-when-it-is-available.patch Type: text/x-patch Size: 49891 bytes Desc: not available URL: From pvoborni at redhat.com Fri Aug 31 14:54:16 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 31 Aug 2012 16:54:16 +0200 Subject: [Freeipa-devel] Paging in Web UI In-Reply-To: <503E486C.1030006@redhat.com> References: <503CC7E1.4040004@redhat.com> <503CD2E2.6010005@redhat.com> <503CF09A.5030902@redhat.com> <503D0E9E.6000703@redhat.com> <503E1DD6.9050504@redhat.com> <503E486C.1030006@redhat.com> Message-ID: <5040D018.2090902@redhat.com> On 08/29/2012 06:50 PM, Endi Sukma Dewata wrote: > On 8/29/2012 8:49 AM, Petr Vobornik wrote: >>> Another possibility is to use VLV, but it seems to require open >>> connection too and only works with a predefined filter. >> >> Apart from the possible connection problem I think it is very usable for >> default views of search pages. User can freely page. I don't understand >> from the wiki page if it can reasonably obtain total record count. > > Correction, VLV doesn't seem to require a continuous connection. > > The server will return a 'contentCount' in the VLV response: > http://www.ietf.org/proceedings/55/I-D/draft-ietf-ldapext-ldapv3-vlv-09.txt > >> IMO a modification of a hybrid solution may be best: >> >> When user opens page a VLV result would be shown. User can page and see >> all the data. Currently we don't support custom sorting so disability of >> custom sorting isn't an issue. With predefined sorts we can even >> improved these possibilities. >> >> If there is many records and user don't want to go through pages he will >> use search. If the search string is good he should fit to search limit >> so I wouldn't use paging at this moment. If not the case and he really >> don't know how to improve the filter he should have option to enable >> paging (current implementation) and go through the records by hand. Such >> record set should fit to limit of 2000 records. If not the filter is >> really bad. > > The thing is the filter is only used on certain attributes only. For > example I have a user with full name "Test User". Searching the "Test > User" in the filter doesn't return anything because the server doesn't > search the cn or displayName attributes. In that case I'd have to search > with first name or last name only, which may return too many results. > >> If we don't want to implement VLV or until that time I would set >> not-paging as default setting. > > How about this, when you open the search page for the first time it will > do a regular search, no paging. Instead of showing the paging controls > (Prev & Next buttons) we show "See more results..." If you click that it > will rerun the search with paging (which will give us the total number > of entries) and show the paging control. I think we are in an agreement. It's basically what I wrote in different words. I just mentioned current size limit. User workflow when searching for record (the same thing reworded): 1) open search page: VLV or normal search with empty filter 1a) when VLV: user would see paging ui 1b) when normal search: not sure if to show 'see more results' link for switching to paging. For smaller setups (<2000) it might be there, for larger rather not - pkey list would be truncated. 2) search for record: enter filter to textbox, normal search would be used 3) record not listed: switch to paging by clicking on 'see more results' Paging should be OK because most likely we would get less than 2000 records. Otherwise filter should be improved. I guess we can't go around this unless we increase the limit or use other mean of paging. 4) record still not listed: page to find the record using current paging UI or some improved > >>> Agreed, but if we implement the single continuous page I described in >>> the earlier email the page size won't be much of an issue anymore. >>> >> It may matter. For example when testing Web UI permissions. It often >> requires to navigate to the last page. I can enter the number directly >> and press enter or extend it for times (hypothetically) but I would >> rather see those 90 permissions right away because I can be at the list >> end in 0.5s. > > If we use a continuous page we can use a bigger default page size, say > 100 entries. We probably could optimize it such that if you scroll > directly to the bottom it will load just the last page. Comments below. > We probably could also combine it with initial non-paging search like above, so > you'll need to click "See more results...", then the page will extend > and you can scroll to the last entry. Yes as I wrote above. > >> We can also improve pager to offer some subset of pages. For example: >> >> First Prev 22 23 *24* 25 26 Next Last Page [ ] >> where 24 is current page. > > Yes, that will work too. > >> Regarding the continuous paging: I would extend the page by clicking at >> a bar at the end of the list and/or by mouse-scrolling at the end of the >> list? > > Ideally the scroll bar size should correspond to the visible part of the > entire page. So you should be able to go anywhere in the entire page by > moving the scroll bar around (either by keyboard, mouse click, mouse > drag, or mouse scroll). In this case no need to extend the page. > The thing is that we might have to create a table with many empty rows, > which could be big, and populate them if they become visible. Maybe > there's a way to avoid creating empty rows, we'll need to investigate. Did you meant by 'continuous paging' this solution or the one when you click on 'see more results' (or scroll to last 10% of a scrollbar) and it would show you next page? I originally understood it as the latter but now I'm not sure. This solution seems challenging to implement right. I'm also not sure how usable it is because I don't remember if I ever used it somewhere. Anyway empty rows can be also replaced by rows with huge height. > > Alternatively we could do like image search on bing.com. If you bring > the scroll bar say within 10% from the bottom it will load the next > page. The thing is it will load the page sequentially, and if you > continue to the bottom you'll end up with a very big table. > I like this with the Prev extension mentioned below. > Another possibility, we can use the top 10% of the scroll bar as a Prev > button, and the bottom %10 as the Next button, and we remove the pages > that aren't visible anymore. But this is not much different than the > current implementation, only different presentation. > I can image to use each of these 3 solutions for enabled paging. Also switching between second and third solution might be configurable. Anyway. First I would like to make paging/not paging configurable and switchable when needed. Then we can proceed to improving paging ui (better pager, continuous paging and such). -- Petr Vobornik From pvoborni at redhat.com Fri Aug 31 15:00:09 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 31 Aug 2012 17:00:09 +0200 Subject: [Freeipa-devel] Reevaluation of confirmation of actions in Web UI. Message-ID: <5040D179.2030206@redhat.com> Hi Endi, I opened https://fedorahosted.org/freeipa/ticket/3035 can you please comment? If everything seems good, I will try to implement some stuff to RC1 (next week). Thanks -- Petr Vobornik From pvoborni at redhat.com Fri Aug 31 15:01:43 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 31 Aug 2012 17:01:43 +0200 Subject: [Freeipa-devel] Reevaluation of confirmation of actions in Web UI. In-Reply-To: <5040D179.2030206@redhat.com> References: <5040D179.2030206@redhat.com> Message-ID: <5040D1D7.1050908@redhat.com> On 08/31/2012 05:00 PM, Petr Vobornik wrote: > Hi Endi, > > I opened https://fedorahosted.org/freeipa/ticket/3035 can you please > comment? If everything seems good, I will try to implement some stuff to > RC1 (next week). > > Thanks Wrong address... -- Petr Vobornik From tbabej at redhat.com Fri Aug 31 15:35:27 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 31 Aug 2012 17:35:27 +0200 Subject: [Freeipa-devel] [PATCH 0010] Sort policies numerically in pwpolicy-find Message-ID: <5040D9BF.6050404@redhat.com> Hi, this is a fairly simple one-liner. https://fedorahosted.org/freeipa/ticket/3039 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0010-Sort-policies-numerically-in-pwpolicy-find.patch Type: text/x-patch Size: 1029 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 31 17:08:42 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 31 Aug 2012 13:08:42 -0400 Subject: [Freeipa-devel] [PATCH 0010] Sort policies numerically in pwpolicy-find In-Reply-To: <5040D9BF.6050404@redhat.com> References: <5040D9BF.6050404@redhat.com> Message-ID: <5040EF9A.7050904@redhat.com> Tomas Babej wrote: > Hi, > > this is a fairly simple one-liner. > > https://fedorahosted.org/freeipa/ticket/3039 > > Tomas Looks good. Can you add a unit test so we don't have a regression on this? thanks rob From rcritten at redhat.com Fri Aug 31 17:39:46 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 31 Aug 2012 13:39:46 -0400 Subject: [Freeipa-devel] [PATCH] 1050 prevent replica orphans Message-ID: <5040F6E2.7060109@redhat.com> It was possible use ipa-replica-manage connect/disconnect/del to end up orphaning or or more IPA masters. This is an attempt to catch and prevent that case. I tested with this topology, trying to delete B. A <-> B <-> C I got here by creating B and C from A, connecting B to C then deleting the link from A to B, so it went from A -> B and A -> C to the above. What I do is look up the servers that the delete candidate host has connections to and see if we're the last link. I added an escape clause if there are only two masters. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1050-replicaorphan.patch Type: text/x-diff Size: 5845 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 31 17:40:50 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 31 Aug 2012 13:40:50 -0400 Subject: [Freeipa-devel] [PATCH] 1050 prevent replica orphans In-Reply-To: <5040F6E2.7060109@redhat.com> References: <5040F6E2.7060109@redhat.com> Message-ID: <5040F722.4060208@redhat.com> Rob Crittenden wrote: > It was possible use ipa-replica-manage connect/disconnect/del to end up > orphaning or or more IPA masters. This is an attempt to catch and > prevent that case. > > I tested with this topology, trying to delete B. > > A <-> B <-> C > > I got here by creating B and C from A, connecting B to C then deleting > the link from A to B, so it went from A -> B and A -> C to the above. > > What I do is look up the servers that the delete candidate host has > connections to and see if we're the last link. > > I added an escape clause if there are only two masters. > > rob Oh, this relies on my cleanruv patch 1031. rob From rcritten at redhat.com Fri Aug 31 17:43:54 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 31 Aug 2012 13:43:54 -0400 Subject: [Freeipa-devel] [PATCH] 1051 Fix CS replica management Message-ID: <5040F7DA.5020606@redhat.com> The naming in CS replication agreements is different from IPA agreements, we have to live with what the create. The master side should be on the local side, replica1, not the remote. This required reversing a few master variables. Pass in the force flag to del_link. Do a better job of finding the agreements on each side. This should be ipa-csreplica-manage more in line with ipa-replica-manage. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1051-csmanage.patch Type: text/x-diff Size: 9598 bytes Desc: not available URL: From rcritten at redhat.com Fri Aug 31 17:49:15 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 31 Aug 2012 13:49:15 -0400 Subject: [Freeipa-devel] [PATCH] 1052 add version to prepared replica files Message-ID: <5040F91B.6000606@redhat.com> When installing a replica in an upgrade situation we want to be sure we install the same version or higher. This will have to bake a bit until the next full version of IPA but the idea is to prevent installing a newer replica file on an older server. To test this you need to rip apart a prepared file and tweak the version forward or backward. To do this, do something like: # gpg -d replica-info-pitbull.example.com.gpg | tar xf - # edit realm_info/realm_info # tar cf replica-info-pitbull.example.com realm_info # gpg --batch --homedir `pwd`/.gnupg --passphrase-fd 0 --yes --no-tty -o replica-info-pitbull.example.com.gpg -c replica-info-pitbull.example.com rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1052-version.patch Type: text/x-diff Size: 6343 bytes Desc: not available URL: From tbabej at redhat.com Fri Aug 31 23:38:10 2012 From: tbabej at redhat.com (Tomas Babej) Date: Sat, 01 Sep 2012 01:38:10 +0200 Subject: [Freeipa-devel] [PATCH 0010] Sort policies numerically in pwpolicy-find In-Reply-To: <5040EF9A.7050904@redhat.com> References: <5040D9BF.6050404@redhat.com> <5040EF9A.7050904@redhat.com> Message-ID: <50414AE2.6000305@redhat.com> On 08/31/2012 07:08 PM, Rob Crittenden wrote: > Tomas Babej wrote: >> Hi, >> >> this is a fairly simple one-liner. >> >> https://fedorahosted.org/freeipa/ticket/3039 >> >> Tomas > > Looks good. Can you add a unit test so we don't have a regression on > this? > > thanks > > rob > I tweaked one of the existing unit tests to test the expected behaviour, so that we are not cluttered with tests checking the same thing. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0010-2-Sort-policies-numerically-in-pwpolicy-find.patch Type: text/x-patch Size: 2942 bytes Desc: not available URL: