From mkosek at redhat.com Fri Feb 1 07:11:47 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 01 Feb 2013 08:11:47 +0100 Subject: [Freeipa-devel] [PATCH] 358-359 Fix openldap migration errors In-Reply-To: <510AD580.6060001@redhat.com> References: <51092A36.8000103@redhat.com> <510AC7BF.5020404@redhat.com> <510AD457.6070104@redhat.com> <510AD580.6060001@redhat.com> Message-ID: <510B6AB3.3050905@redhat.com> On 01/31/2013 09:35 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 01/31/2013 08:36 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> These 2 attached patches were generated based on my debugging session >>>> with >>>> "tsunamie" and helping him dealing with migration from his openldap >>>> DS. With >>>> these applied, migrate-ds command no longer crashes with an error. >>>> >>>> I can lend my openldap instance I used when developing these patches. >>>> >>>> Martin >>> >>> Doesn't the second patch break the rule where the same enforcement is >>> done on >>> entering the data via a named option and setattr? If I understand this >>> correctly the implication is that you couldn't do: >>> >>> ipa user-mod --description=' foo ' >>> >>> But you could do >>> >>> ipa user-mod --setattr description=' foo ' >>> >>> rob >>> >> >> I don't think so. This patch just removes this restriction from *attr >> parameters themselves, the underlying parameter validators (i.e. >> description parameter) should be still applied. Though in case of the >> leading and trailing spaces, they somehow get trimmed: >> >> # ipa group-mod foo --setattr "description= some spaces " >> -------------------- >> Modified group "foo" >> -------------------- >> Group name: foo >> Description: some spaces >> GID: 1416400004 >> >> But as I wanted to have this patch only because of the failing user_mod >> operation in the migration.py plugin and since you plan to replace it in >> your WIP migration performance patch with direct LDAP mod operation, I >> do not insist on pushing patch 359 and patch 358 would be sufficient. >> >> Martin > > Ok, and patch 358 works fine, ACK. > > rob Patch 358 pushed to master, ipa-3-1, ipa-3-0. Martin From mkosek at redhat.com Fri Feb 1 07:17:41 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 01 Feb 2013 08:17:41 +0100 Subject: [Freeipa-devel] [PATCH] 357 Use fully qualified CCACHE names In-Reply-To: <20130131180750.GJ4506@redhat.com> References: <5109152A.9030205@redhat.com> <20130131160122.GH4506@redhat.com> <510A9B40.50900@redhat.com> <20130131180750.GJ4506@redhat.com> Message-ID: <510B6C15.4060305@redhat.com> On 01/31/2013 07:07 PM, Alexander Bokovoy wrote: > On Thu, 31 Jan 2013, Martin Kosek wrote: >> On 01/31/2013 05:01 PM, Alexander Bokovoy wrote: >>> On Wed, 30 Jan 2013, Martin Kosek wrote: >>>> Some parts of install scripts used only ccache name as returned by >>>> krbV.CCache.name attribute. However, when this name is used again >>>> to initialize krbV.CCache object or when it is used in KRB5CCNAME >>>> environmental variable, it fails for new DIR type of CCACHE. >>>> >>>> We should always use both CCACHE type and name when referring to >>>> them to avoid these crashes. ldap2 backend was also updated to >>>> accept directly krbV.CCache object which contains everything we need >>>> to authenticate with ccache. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3381 >>> Minor comment: there are few cleanups of 'import krbV' in places where >>> Kerberos functions are not used. Maybe it would be better to separate >>> them into their own patch to avoid rebasing issues in future? >> >> Sure, good idea. Attaching both patches. > ACK to both now. Thanks! > Pushed to master, ipa-3-1. Martin From mkosek at redhat.com Fri Feb 1 07:43:10 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 01 Feb 2013 08:43:10 +0100 Subject: [Freeipa-devel] [PATCH] 361 ipa-adtrust-install should ask for SID generation In-Reply-To: <20130131180611.GI4506@redhat.com> References: <510A7C14.4080701@redhat.com> <20130131152910.GG4506@redhat.com> <510A94A9.8040708@redhat.com> <20130131180611.GI4506@redhat.com> Message-ID: <510B720E.7070404@redhat.com> On 01/31/2013 07:06 PM, Alexander Bokovoy wrote: > On Thu, 31 Jan 2013, Martin Kosek wrote: >> On 01/31/2013 04:29 PM, Alexander Bokovoy wrote: >>> On Thu, 31 Jan 2013, Martin Kosek wrote: >>>> When ipa-adtrust-install is run, check if there are any objects >>>> that need to have SID generated. If yes, interactively ask the user >>>> if the sidgen task should be run. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3195 >>> >> ... >>> I would still run this check in options.unattended mode and reported >>> warning, for accounting purposes. >>> >>> Could you please make so? >>> >> >> Sure! Updated patch attached. > Thanks! I have only small addition: > >> + object_count = len(entries) >> + if object_count > 0: >> + print "" >> + print "WARNING: %d existing users or groups do not have a >> SID identifier assigned." \ >> + % len(entries) >> + print "Installer can run a task to have ipa-sidgen Directory >> Server plugin generate" >> + print "the SID identifier for all these users. Please note, >> the in case of a high" >> + print "number of users and groups, the operation might lead >> to high replication" >> + print "traffic and performance degradation. Refer to >> ipa-adtrust-install(1) man page" >> + print "for details." >> + print "" >> + if not options.unattended: >> + if ipautil.user_input("Do you want to run the ipa-sidgen >> task?", default=False, >> + allow_empty=False): >> + options.add_sids = True > ... to make the text of warning consistent it would be good to add > + else: > + print "Unattended mode was selected, installer will *not* > run ipa-sidgen task!" > And here is the updated patch. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-361-3-ipa-adtrust-install-should-ask-for-sid-generation.patch Type: text/x-patch Size: 3843 bytes Desc: not available URL: From jcholast at redhat.com Fri Feb 1 08:33:28 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 01 Feb 2013 09:33:28 +0100 Subject: [Freeipa-devel] [PATCHES] 91-92 Add support for RFC 6594 SSHFP DNS records In-Reply-To: <510ABEFC.807@redhat.com> References: <50EE49EC.7000500@redhat.com> <50EEA00B.8090505@redhat.com> <51006821.9000901@redhat.com> <5106453A.60104@redhat.com> <510ABEFC.807@redhat.com> Message-ID: <510B7DD8.8070203@redhat.com> On 31.1.2013 19:59, Rob Crittenden wrote: > Jan Cholasta wrote: >> On 23.1.2013 23:45, Rob Crittenden wrote: >>> Jan Cholasta wrote: >>>> On 10.1.2013 05:56, Jan Cholasta wrote: >>>>> Hi, >>>>> >>>>> Patch 91 removes module ipapython.compat. The code that uses it >>>>> doesn't >>>>> work with ancient Python versions anyway, so there's no need to >>>>> keep it >>>>> around. >>>>> >>>>> Patch 92 adds support for automatic generation of RFC 6594 SSHFP DNS >>>>> records to ipa-client-install and host plugin, as described in >>>>> . Note that >>>>> still >>>>> applies. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2642 >>>>> >>>>> Honza >>>>> >>>> >>>> Self-NACK, forgot to actually remove ipapython/compat.py in the first >>>> patch. Also removed an unnecessary try block from the second patch. >>>> >>>> Honza >>> >>> These look good. I'm a little concerned about the magic numbers in the >>> SSHFP code. I know these come from the RFCs. Can you add a comment there >>> so future developers know where the values for key type and fingerprint >>> type come from? >>> >>> rob >> >> Comment added. >> > > Sorry, I just noticed that this is an RFE and there is no design page. > Can you write one up real quick, then I'll push both. Umm.. yes there is, it is linked in the first message of this thread: . > > I went back and forth a few times on whether we should have a ticket on > the dropping of compat, if only to codify that we're giving up an python > 2.6, but since this has been a given for a while I think we're ok. It's Python 2.5 that we are giving up on, not Python 2.6. In fact, we already gave up on it, our code does not work with it even if we keep compat in (we use some Python features which are not available in 2.5). > > rob Honza -- Jan Cholasta From pviktori at redhat.com Fri Feb 1 08:47:18 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 01 Feb 2013 09:47:18 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <510AB17B.80909@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> Message-ID: <510B8116.9060409@redhat.com> On 01/31/2013 07:01 PM, Jan Cholasta wrote: > On 31.1.2013 11:00, Petr Viktorin wrote: >> On 01/30/2013 10:53 AM, Petr Viktorin wrote: >>> On 01/29/2013 04:39 PM, Petr Viktorin wrote: >>>> On 01/28/2013 04:09 PM, Petr Viktorin wrote: >>>>> On 01/28/2013 09:34 AM, Jan Cholasta wrote: >>>>>> On 25.1.2013 14:54, Petr Viktorin wrote: >>>>>>> On 01/24/2013 03:06 PM, Petr Viktorin wrote: >>>>>>>> On 01/24/2013 10:43 AM, Petr Viktorin wrote: >>>>>>>>> On 01/22/2013 04:04 PM, Petr Viktorin wrote: >>>>>>>>>> On 01/21/2013 06:38 PM, Petr Viktorin wrote: >>>>>>>>>>> On 01/17/2013 06:27 PM, Petr Viktorin wrote: >>>>>>>>>>>> Hello, >>>>>>>>>>>> This is the first batch of changes aimed to consolidate our >>>>>>>>>>>> LDAP >>>>>>>>>>>> code. >>>>>>>>>>>> Each should be a self-contained change that doesn't break >>>>>>>>>>>> anything. >>>>>>>>>>>> >> [...] >>>>>>> Since this patchset is becoming unwieldy, I've put it in a public >>>>>>> repo >>>>>>> that I'll keep updated. The following command will fetch it into >>>>>>> your >>>>>>> "pviktori-ldap-refactor" branch: >>>>>>> >>>>>>> git fetch git://github.com/encukou/freeipa >>>>>>> ldap-refactor:pviktori-ldap-refactor >>>>>>> >>>>>>> >> [...] >> >> I found a bug in patch 143, here is a fixed version. >> > > I would prefer if you used the semantics of .get() for .get_single() as > well (i.e. when no default value is provided, None is assumed) in patch > 152. Or is there a reason not to? I think you should always have to write extra code to supress exceptions (?Errors should never pass silently?). In Python, the easiest/shortest getter you can write will typically raise an exception when the value is missing (e.g. `d[k]` for dicts, `getattr(a, b)` for objects). -- Petr? From jcholast at redhat.com Fri Feb 1 09:24:10 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 01 Feb 2013 10:24:10 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <510B8116.9060409@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> Message-ID: <510B89BA.2040903@redhat.com> On 1.2.2013 09:47, Petr Viktorin wrote: > On 01/31/2013 07:01 PM, Jan Cholasta wrote: >> On 31.1.2013 11:00, Petr Viktorin wrote: >>> On 01/30/2013 10:53 AM, Petr Viktorin wrote: >>>> On 01/29/2013 04:39 PM, Petr Viktorin wrote: >>>>> On 01/28/2013 04:09 PM, Petr Viktorin wrote: >>>>>> On 01/28/2013 09:34 AM, Jan Cholasta wrote: >>>>>>> On 25.1.2013 14:54, Petr Viktorin wrote: >>>>>>>> On 01/24/2013 03:06 PM, Petr Viktorin wrote: >>>>>>>>> On 01/24/2013 10:43 AM, Petr Viktorin wrote: >>>>>>>>>> On 01/22/2013 04:04 PM, Petr Viktorin wrote: >>>>>>>>>>> On 01/21/2013 06:38 PM, Petr Viktorin wrote: >>>>>>>>>>>> On 01/17/2013 06:27 PM, Petr Viktorin wrote: >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> This is the first batch of changes aimed to consolidate our >>>>>>>>>>>>> LDAP >>>>>>>>>>>>> code. >>>>>>>>>>>>> Each should be a self-contained change that doesn't break >>>>>>>>>>>>> anything. >>>>>>>>>>>>> >>> [...] >>>>>>>> Since this patchset is becoming unwieldy, I've put it in a public >>>>>>>> repo >>>>>>>> that I'll keep updated. The following command will fetch it into >>>>>>>> your >>>>>>>> "pviktori-ldap-refactor" branch: >>>>>>>> >>>>>>>> git fetch git://github.com/encukou/freeipa >>>>>>>> ldap-refactor:pviktori-ldap-refactor >>>>>>>> >>>>>>>> >>> [...] >>> >>> I found a bug in patch 143, here is a fixed version. >>> >> >> I would prefer if you used the semantics of .get() for .get_single() as >> well (i.e. when no default value is provided, None is assumed) in patch >> 152. Or is there a reason not to? > > I think you should always have to write extra code to supress exceptions > (?Errors should never pass silently?). In Python, the easiest/shortest > getter you can write will typically raise an exception when the value is > missing (e.g. `d[k]` for dicts, `getattr(a, b)` for objects). > That is true, but I think consistency is more important here - the name suggests the method behaves the same way .get() does. If you insist on keeping the current behavior, would you at least consider renaming the method (perhaps to just "single" or "single_value")? (This is just a nitpick, so don't worry too much about it.) Honza -- Jan Cholasta From pviktori at redhat.com Fri Feb 1 11:12:10 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 01 Feb 2013 12:12:10 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <510A8B55.70109@redhat.com> References: <510A8B55.70109@redhat.com> Message-ID: <510BA30A.4040601@redhat.com> On 01/31/2013 04:18 PM, Jan Cholasta wrote: > Hi, > > these patches implement attribute name case preservation in LDAPEntry. > Apply on top of Petr Viktorin's LDAP code refactoring patchset (up to > part 5). > > Honza Patches 99 & 101 need some tests to make sure the _names work correctly. Since you can call LDAPEntry.__init__ in different ways which don't always correspond to the argument names, it would be nice to explain them in a docstring. A few nitpicks below. > freeipa-jcholast-99-Preserve-case-of-attribute-names-in-LDAPEntry.patch > > diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py > index 20c11b4..6268ac0 100644 > @@ -650,14 +637,48 @@ class LDAPEntry(dict): > self._orig = self > self._orig = deepcopy(self) > > + def _attr_name(self, name): > + if not isinstance(name, (unicode, str)): Use basestring instead of (unicode, str). > freeipa-jcholast-100-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch > > diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py > index 6268ac0..01fa0c1 100644 > @@ -595,8 +600,10 @@ class LDAPEntry(dict): > _obj = {} > orig = None > > + assert isinstance(_conn, IPASimpleLDAPObject) > assert isinstance(_dn, DN) > > + self._conn = lambda: _conn # do not deepcopy me! This would be better done by overriding __deepcopy__, or just using a custom method for the copying. -- Petr? From mkosek at redhat.com Fri Feb 1 12:35:03 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 01 Feb 2013 13:35:03 +0100 Subject: [Freeipa-devel] [PATCH] 352-354 Add support for AD users to hbactest command In-Reply-To: <1359036276.20683.387.camel@willson.li.ssimo.org> References: <50F98561.1040503@redhat.com> <1358620528.20683.98.camel@willson.li.ssimo.org> <50FF9B05.3020207@redhat.com> <1358947395.20683.365.camel@willson.li.ssimo.org> <5100DF88.8010502@redhat.com> <1359036276.20683.387.camel@willson.li.ssimo.org> Message-ID: <510BB677.1060201@redhat.com> On 01/24/2013 03:04 PM, Simo Sorce wrote: > On Thu, 2013-01-24 at 08:15 +0100, Martin Kosek wrote: >> On 01/23/2013 02:23 PM, Simo Sorce wrote: >>> On Wed, 2013-01-23 at 09:10 +0100, Martin Kosek wrote: >>>> On 01/19/2013 07:35 PM, Simo Sorce wrote: >>>>> On Fri, 2013-01-18 at 18:24 +0100, Martin Kosek wrote: >>>>>> How this works: >>>>>> 1. When a trusted domain user is tested, AD GC is searched >>>>>> for the user entry Distinguished Name >>>>> >>>>> My head is not clear today but it looks to me you are doing 2 searches. >>>>> One to go from samAccountName -> DNa dn then a second for DN -> SID. >>>>> >>>>> Why are you doing 2 searches ? The first one can return you the >>>>> ObjectSid already. >>>>> >>>>> Simo. >>>> >>>> I had to do 2 searches because GC refuses to give me tokenGroups attribute >>>> content when I do not search with exact DN and LDAP SCOPE_BASE. So I have to do >>>> the first search to find out the DN of the searched user and then a second >>>> query to get the tokenGroups (and ObjectSid). >>> >>> I see, yes that makes sense, would you mind adding a comment to this >>> effect so we do not try to 'optimize' at some point ? >>> I have no additional concerns then. >>> >>> Simo. >>> >> >> Hello Simo, >> >> Thanks for review. Anyway, there is already a relevant comment in dcerpc.py, >> where the double search is performed: >> >> ... >> def get_trusted_domain_user_and_groups(self, object_name): >> ... >> entries = self.get_trusted_domain_objects(components.get('domain'), >> components.get('flatname'), filter, attrs, _ldap.SCOPE_SUBTREE) >> >> # Get SIDs of user object and it's groups >> # tokenGroups attribute must be read with scope BASE to avoid search error >> attrs = ['objectSID', 'tokenGroups'] >> ... >> >> I think it's enough to avoid "optimizing" this process - we would find out the >> "optimization" soon anyway, as the tokenGroups search would return error :-) > > Perfect! > > /me just had an eye vision exam, will complain to his doctor :-) > I enhanced the hbactest to also support user SID, not only trusted domain user name. Updated set of patches attached. How it works: # ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 --host `hostname` --service sshd -------------------- Access granted: True -------------------- Matched rules: can_login Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-352-2-generalize-ad-gc-search.patch Type: text/x-patch Size: 12566 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-353-2-do-not-hide-sid-resolver-error-in-group-add-member.patch Type: text/x-patch Size: 1378 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-354-2-add-support-for-ad-users-to-hbactest-command.patch Type: text/x-patch Size: 9256 bytes Desc: not available URL: From sbose at redhat.com Fri Feb 1 13:17:51 2013 From: sbose at redhat.com (Sumit Bose) Date: Fri, 1 Feb 2013 14:17:51 +0100 Subject: [Freeipa-devel] krb5.conf on IPA server and SSSD setup In-Reply-To: <20130129204918.GB4506@redhat.com> References: <20130129204918.GB4506@redhat.com> Message-ID: <20130201131751.GC11711@localhost.localdomain> On Tue, Jan 29, 2013 at 10:50:02PM +0200, Alexander Bokovoy wrote: > Hi! > > I've been chasing few bugs in FreeIPA's trusted domains support and > found out some grave bugs in both SSSD and FreeIPA. > > On FreeIPA server side we configure krb5.conf using following settings: > > ------------------------------------------------- > includedir /var/lib/sss/pubconf/krb5.include.d > [libdefaults] > ... > default_realm = EXAMPLE.COM > dns_lookup_realm = false > dns_lookup_kdc = true > ... > > [domain_realm] > .example.com = EXAMPLE.COM > example.com = EXAMPLE.com > > -------------------------------------------------- > > Then SSSD generates files which contain domain_realm mapping for trusted > domains in /var/lib/sss/pubconf/krb5.include.d and libkrb5 will read them > as part of the krb5.conf sourcing. > > Few problems here: > > 1. KDC needs to know this mapping information in order to issue > referrals to the clients. There is heuristic in libkrb5 that uses > domain_realm mapping first and default_realm value if mapping didn't > catch the principal which was not found in the database. > > 2. krb5.conf is parsed by applications usually only on startup. KDC is > not an exception, so any changes to krb5.conf would require to restart > KDC if we want them to be noticed. > > 3. Adding new trust implies therefore KDC restart. It also implies that > SSSD should have updated the mapping which is not neccessary true > time-wise. > > As result, operations like mapping trusted domain users via external > groups in IPA might fail as IPA code running on IPA server needs to > contact LDAP service at trusted domain's Global Catalog using SASL > GSSAPI authentication. When ticket is obtained, we don't specify > explicitly the realm of the service principal, it is constructed by > underlying libldap/libsasl code. > > If explicit domain_realm mapping is in place on client side (and here > client is the server as request is issued from IPA httpd code), trusted > domain's Global Catalog host will be automatically mapped to trusted > domain realm. Otherwise KDC will hint the client with referral to proper > KDC for trusted domain realm. > > This is the step that might fail if trusted domain is sub-domain of IPA > domain, for example, ad.example.com. In this case our explicit mapping > for example.com will prevail and requests will always be sent for > principal in EXAMPLE.COM realm. > > More to that, since client and KDC are the same host, KDC will use > domain_realm mapping as well and hint client with referral to itself > (since .example.com = EXAMPLE.COM). Obtaining ticket will fail again. > > So, I was trying to solve this issue and I've got to following setup > with Nalin's help: > > 1. Define following settings in [libdefaults] of krb5.conf > > default_realm = EXAMPLE.COM > dns_lookup_realm = false > dns_lookup_kdc = true > realm_try_domains = 0 > > realm_try_domains = 0 forces libkrb5 to fallback discovery of realm to > domain of the host via DNS if there is no other explicit mapping. > > 2. Remove any explicit domain_realm mapping for our default realm since > it will be implicitly generated from default_realm value by the fallback > code anyway. > > With these changes both KDC and libkrb5 will be able to properly serve > out both own domain and trusted domain requests. At some point SSSD will > kick in with its explicit mapping for trusted domain realm. Still, KDC > will not be able to see this mapping until restart but in Krb5 1.12 we > are getting new pluggable interface that will allow to refresh KDC > configuration. If set up an environment like discussed above, and FreeIPA server and an AD server where the AD DNS domain is a sub-domain of the the IPA DNS domain. Then tried to run ldapsearch, smbclient, nsupdate and kvno accessing the AD server. Here are my findings: ldapsearch: - does not work in the default configuration - works even if no domain_realm mapping is available, but in this case the ipa client utility does not work anymore - works with full domain_realm, i.e. IPA and AD DNS domains are listed - setting realm_try_domains or not does not make any difference smbclient: - does not work in the default configuration - does not work with missing domain_realm mapping - works with full domain_realm - setting realm_try_domains or not does not make any difference nsupdate: - does not work in any configuration if the realm option is missing in the input file - works in all configurations if the realm option is given - setting realm_try_domains or not does not make any difference kvno: - I used 'kvno -S server ad-server.ad.domain' - does not work in the default configuration - works even if no domain_realm mapping is available - works with full domain_realm - setting realm_try_domains or not does not make any difference In the case without a domain_realm mapping ldapsearch and kvno first try to get a ticket with the default_realm and the KDC returns UNKNOWN_SERVER. As a second step they try to get a cross realm ticket where the realm is the uppercase version of the destinations DNS domain. With full domain_realm mapping all clients except nsupdate directly ask for the cross realm ticket. For me it looks like realm_try_domains is not needed but domain_realm mappings are. I there anything else which I should test? bye, Sumit > > And here I'm coming to grave error in the SSSD code: the name of > explicit mapping file contains non-filtered domain name, which contains > dot. krb5.conf manual page states that includedir allows to source all > files which names are constructed from alpha-numeric chars, dashes and > underscores. > > Files with other characters are ignored. So dots as in > domain_realm_example.com are ignored and our mapping is never sourced. > > For IDN domains we also will need to transform the name into its > Punycode (RFC3492) to avoid breaking out of alpha-numeric space. > > I'd suggest replacing dots with underscores. > > File name is irrelevant to libkrb5 after it was read as part of > includedir processing, and files are only written by the SSSD. > > > > > -- > / Alexander Bokovoy > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From abokovoy at redhat.com Fri Feb 1 14:07:11 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 1 Feb 2013 16:07:11 +0200 Subject: [Freeipa-devel] krb5.conf on IPA server and SSSD setup In-Reply-To: <20130201131751.GC11711@localhost.localdomain> References: <20130129204918.GB4506@redhat.com> <20130201131751.GC11711@localhost.localdomain> Message-ID: <20130201140710.GK4506@redhat.com> On Fri, 01 Feb 2013, Sumit Bose wrote: >If set up an environment like discussed above, and FreeIPA server and >an AD server where the AD DNS domain is a sub-domain of the the IPA DNS >domain. Then tried to run ldapsearch, smbclient, nsupdate and kvno >accessing the AD server. Here are my findings: > >ldapsearch: > - does not work in the default configuration > - works even if no domain_realm mapping is available, but in this case > the ipa client utility does not work anymore > - works with full domain_realm, i.e. IPA and AD DNS domains are listed > - setting realm_try_domains or not does not make any difference > >smbclient: > - does not work in the default configuration > - does not work with missing domain_realm mapping > - works with full domain_realm > - setting realm_try_domains or not does not make any difference > >nsupdate: > - does not work in any configuration if the realm option is missing in > the input file > - works in all configurations if the realm option is given > - setting realm_try_domains or not does not make any difference > >kvno: > - I used 'kvno -S server ad-server.ad.domain' > - does not work in the default configuration > - works even if no domain_realm mapping is available > - works with full domain_realm > - setting realm_try_domains or not does not make any difference I'm finding hard to parse notes above (what is 'default configuration'?). >In the case without a domain_realm mapping ldapsearch and kvno first try >to get a ticket with the default_realm and the KDC returns >UNKNOWN_SERVER. As a second step they try to get a cross realm ticket >where the realm is the uppercase version of the destinations DNS domain. Yes, this is expected behavior and this is what we want to see. Please note that if you put own realm to domain_realm mapping, KDC will use it to build referral and force you to connect to itself rather than the destination KDC. It happens because .example.com takes precedence over .ad.example.com if the latter is not specified. So KDC sees that host does not exist in its own realm but finds mapping in domain_realm section which covers the host foo.ad.example.com (.example.com) and returns that as a referral which then fails because the same KDC is queried on second attempt. >With full domain_realm mapping all clients except nsupdate directly ask >for the cross realm ticket. > >For me it looks like realm_try_domains is not needed but domain_realm >mappings are. Please note that once you start adding trusted domains, includedir entry in krb5.conf will bring the mappings to them automatically. Since all applications you tested are short-lived, they will read krb5.conf on their startup and those mappings will always be actual for them. For KDC, however, problem is in actualizing domain_realm mapping, as KDC is a long-living process and does not re-read krb5.conf periodically or on any changes. In our case krb5.conf is not changed but some files in includdir are so it is even more complex. >I there anything else which I should test? I think we need to find solution that does not force KDC to issue referral to its own domain. Ideally, if we could use separate krb5.conf for KDC where domain_realm mapping for own domain does not exist, we could have solved referral issue. -- / Alexander Bokovoy From rcritten at redhat.com Fri Feb 1 14:17:44 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 01 Feb 2013 09:17:44 -0500 Subject: [Freeipa-devel] [PATCHES] 91-92 Add support for RFC 6594 SSHFP DNS records In-Reply-To: <510B7DD8.8070203@redhat.com> References: <50EE49EC.7000500@redhat.com> <50EEA00B.8090505@redhat.com> <51006821.9000901@redhat.com> <5106453A.60104@redhat.com> <510ABEFC.807@redhat.com> <510B7DD8.8070203@redhat.com> Message-ID: <510BCE88.1020105@redhat.com> Jan Cholasta wrote: > On 31.1.2013 19:59, Rob Crittenden wrote: >> Jan Cholasta wrote: >>> On 23.1.2013 23:45, Rob Crittenden wrote: >>>> Jan Cholasta wrote: >>>>> On 10.1.2013 05:56, Jan Cholasta wrote: >>>>>> Hi, >>>>>> >>>>>> Patch 91 removes module ipapython.compat. The code that uses it >>>>>> doesn't >>>>>> work with ancient Python versions anyway, so there's no need to >>>>>> keep it >>>>>> around. >>>>>> >>>>>> Patch 92 adds support for automatic generation of RFC 6594 SSHFP DNS >>>>>> records to ipa-client-install and host plugin, as described in >>>>>> . Note that >>>>>> still >>>>>> applies. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/2642 >>>>>> >>>>>> Honza >>>>>> >>>>> >>>>> Self-NACK, forgot to actually remove ipapython/compat.py in the first >>>>> patch. Also removed an unnecessary try block from the second patch. >>>>> >>>>> Honza >>>> >>>> These look good. I'm a little concerned about the magic numbers in the >>>> SSHFP code. I know these come from the RFCs. Can you add a comment >>>> there >>>> so future developers know where the values for key type and fingerprint >>>> type come from? >>>> >>>> rob >>> >>> Comment added. >>> >> >> Sorry, I just noticed that this is an RFE and there is no design page. >> Can you write one up real quick, then I'll push both. > > Umm.. yes there is, it is linked in the first message of this thread: > . I looked in the ticket. Can you add the link there? >> >> I went back and forth a few times on whether we should have a ticket on >> the dropping of compat, if only to codify that we're giving up an python >> 2.6, but since this has been a given for a while I think we're ok. > > It's Python 2.5 that we are giving up on, not Python 2.6. In fact, we > already gave up on it, our code does not work with it even if we keep > compat in (we use some Python features which are not available in 2.5). Yes, off-by-one. Though in fact the client install, which is all we really care about regarding older systems, still does almost work even in python 2.4 with just a few minor changes. But like I said, its fine. pushed both to master rob From pviktori at redhat.com Fri Feb 1 14:38:28 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 01 Feb 2013 15:38:28 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <510B89BA.2040903@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> Message-ID: <510BD364.7050809@redhat.com> On 02/01/2013 10:24 AM, Jan Cholasta wrote: > On 1.2.2013 09:47, Petr Viktorin wrote: >> On 01/31/2013 07:01 PM, Jan Cholasta wrote: >>> On 31.1.2013 11:00, Petr Viktorin wrote: >>>> On 01/30/2013 10:53 AM, Petr Viktorin wrote: >>>>> On 01/29/2013 04:39 PM, Petr Viktorin wrote: >>>>>> On 01/28/2013 04:09 PM, Petr Viktorin wrote: >>>>>>> On 01/28/2013 09:34 AM, Jan Cholasta wrote: >>>>>>>> On 25.1.2013 14:54, Petr Viktorin wrote: >>>>>>>>> On 01/24/2013 03:06 PM, Petr Viktorin wrote: >>>>>>>>>> On 01/24/2013 10:43 AM, Petr Viktorin wrote: >>>>>>>>>>> On 01/22/2013 04:04 PM, Petr Viktorin wrote: >>>>>>>>>>>> On 01/21/2013 06:38 PM, Petr Viktorin wrote: >>>>>>>>>>>>> On 01/17/2013 06:27 PM, Petr Viktorin wrote: >>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>> This is the first batch of changes aimed to consolidate our >>>>>>>>>>>>>> LDAP >>>>>>>>>>>>>> code. >>>>>>>>>>>>>> Each should be a self-contained change that doesn't break >>>>>>>>>>>>>> anything. >>>>>>>>>>>>>> [...] >>>>>>>>> Since this patchset is becoming unwieldy, I've put it in a public >>>>>>>>> repo >>>>>>>>> that I'll keep updated. The following command will fetch it into >>>>>>>>> your >>>>>>>>> "pviktori-ldap-refactor" branch: >>>>>>>>> >>>>>>>>> git fetch git://github.com/encukou/freeipa >>>>>>>>> ldap-refactor:pviktori-ldap-refactor >>>>>>>>> >>>>>>>>> [...] >>> >>> I would prefer if you used the semantics of .get() for .get_single() as >>> well (i.e. when no default value is provided, None is assumed) in patch >>> 152. Or is there a reason not to? >> >> I think you should always have to write extra code to supress exceptions >> (?Errors should never pass silently?). In Python, the easiest/shortest >> getter you can write will typically raise an exception when the value is >> missing (e.g. `d[k]` for dicts, `getattr(a, b)` for objects). >> > > That is true, but I think consistency is more important here - the name > suggests the method behaves the same way .get() does. If you insist on > keeping the current behavior, would you at least consider renaming the > method (perhaps to just "single" or "single_value")? > > (This is just a nitpick, so don't worry too much about it.) Alright, I renamed get_single to single_value(). I also rebased to current master. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0123-03-Use-explicit-loggers-in-ldap2-code.patch Type: text/x-patch Size: 10138 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0125-02-Remove-connection-creating-code-from-ShemaCache.patch Type: text/x-patch Size: 4451 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0127-02-Move-SchemaCache-and-IPASimpleLDAPObject-to-ipaserve.patch Type: text/x-patch Size: 46116 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0130-02-Move-schema-related-methods-to-LDAPConnection.patch Type: text/x-patch Size: 6690 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0152-02-Introduce-LDAPEntry.single_value-for-getting-single-.patch Type: text/x-patch Size: 1644 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0154-02-Replace-entry.getValue-by-entry.single_value.patch Type: text/x-patch Size: 29993 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0156-02-Remove-toTupleList-and-attrList-from-LDAPEntry.patch Type: text/x-patch Size: 2727 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0161-02-replace-getEntry-with-get_entry-or-get_entries-if-sc.patch Type: text/x-patch Size: 21257 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0170-02-Use-ldap-instead-of-_ldap-in-ipaldap.patch Type: text/x-patch Size: 12162 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0173-02-Use-IPAdmin-rather-than-raw-python-ldap-in-ipactl.patch Type: text/x-patch Size: 5421 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0174-02-Remove-some-uses-of-raw-python-ldap.patch Type: text/x-patch Size: 37127 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0144-03-Remove-unused-imports-from-ipaserver-install.patch Type: text/x-patch Size: 11095 bytes Desc: not available URL: From abokovoy at redhat.com Fri Feb 1 14:55:54 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 1 Feb 2013 16:55:54 +0200 Subject: [Freeipa-devel] [PATCH] 356 Add trusconfig-show and trustconfig-mod commands In-Reply-To: <5107BC0D.6070200@redhat.com> References: <5107BC0D.6070200@redhat.com> Message-ID: <20130201145554.GL4506@redhat.com> On Tue, 29 Jan 2013, Martin Kosek wrote: > trust_output_params = ( >@@ -482,3 +499,158 @@ api.register(trust_mod) > api.register(trust_del) > api.register(trust_find) > api.register(trust_show) >+ >+ >+_trust_type_option = ( >+ StrEnum('trust_type', >+ cli_name='type', >+ label=_('Trust type (ad for Active Directory, default)'), >+ values=(u'ad',), >+ default=u'ad', >+ autofill=True, >+ ), >+) We already have various trust type definitions in the same file. Maybe it makes sense to unify those somehow? >+ def get_dn(self, *keys, **kwargs): >+ trust_type = kwargs.get('trust_type') >+ if trust_type is None: >+ raise errors.RequirementError(name='trust_type') >+ if kwargs['trust_type'] == u'ad': Perhaps better to define constants for the trust type values... >+ except ValueError: >+ # The search is performed for groups with "posixgroup" objectclass >+ # and not "ipausergroup" so that it can also match groups like >+ # "Default SMG Group" which does not have this objectclass. 'Default SM_B_ Group' Thanks for the unit tests too! -- / Alexander Bokovoy From mkosek at redhat.com Fri Feb 1 15:20:28 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 01 Feb 2013 16:20:28 +0100 Subject: [Freeipa-devel] [PATCH] 356 Add trusconfig-show and trustconfig-mod commands In-Reply-To: <20130201145554.GL4506@redhat.com> References: <5107BC0D.6070200@redhat.com> <20130201145554.GL4506@redhat.com> Message-ID: <510BDD3C.5030900@redhat.com> On 02/01/2013 03:55 PM, Alexander Bokovoy wrote: > On Tue, 29 Jan 2013, Martin Kosek wrote: >> trust_output_params = ( >> @@ -482,3 +499,158 @@ api.register(trust_mod) >> api.register(trust_del) >> api.register(trust_find) >> api.register(trust_show) >> + >> + >> +_trust_type_option = ( >> + StrEnum('trust_type', >> + cli_name='type', >> + label=_('Trust type (ad for Active Directory, default)'), >> + values=(u'ad',), >> + default=u'ad', >> + autofill=True, >> + ), >> +) > We already have various trust type definitions in the same file. Maybe > it makes sense to unify those somehow? Right, I unified those 2 separate trust_type option definitions. > >> + def get_dn(self, *keys, **kwargs): >> + trust_type = kwargs.get('trust_type') >> + if trust_type is None: >> + raise errors.RequirementError(name='trust_type') >> + if kwargs['trust_type'] == u'ad': > Perhaps better to define constants for the trust type values... I changed it a bit and now it uses a dict instead. I think its now more general and extensible. > >> + except ValueError: >> + # The search is performed for groups with "posixgroup" objectclass >> + # and not "ipausergroup" so that it can also match groups like >> + # "Default SMG Group" which does not have this objectclass. > 'Default SM_B_ Group' Fixed. > > Thanks for the unit tests too! > You are welcome! I also generated API.txt which I forgot to do last time. Updated patch attached. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-356-2-add-trusconfig-show-and-trustconfig-mod-commands.patch Type: text/x-patch Size: 18542 bytes Desc: not available URL: From rcritten at redhat.com Fri Feb 1 15:21:19 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 01 Feb 2013 10:21:19 -0500 Subject: [Freeipa-devel] [PATCH] 1083 improve migration performance Message-ID: <510BDD6F.1020405@redhat.com> I did some analysis on migration and found several areas impacting performance: 1. We were calling user_mod to reset the magic value in description to not create a UPG. This caused a lot of unnecessary queries to display the user. 2. We check the remote LDAP server to make sure that the GID is valid and added a cache. We lacked a negative cache. 3. The biggest drag on performance was managing the default users group. After about 1000 users it would take about half a second to calculate the modlist and another half second for 389-ds to apply the change. This patch addresses all of these. For the last what I do is only do the group addition every 100 records. A query is run to find all users who aren't in the default users group and those are added. I also added a bit of logging so one can better track the progress of migration. I migrated 12.5k users with compat enabled in 3 1/2 hours. I migrated the same 12.5k users and 2k groups with compat disabled in 30 minutes. By contrast when I started, with compat enabled, I migrated: 1000 users in 7 minutes 2000 users in 27 minutes 3000 users in 1 hour rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1083-migrate.patch Type: text/x-diff Size: 8754 bytes Desc: not available URL: From rcritten at redhat.com Fri Feb 1 15:34:55 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 01 Feb 2013 10:34:55 -0500 Subject: [Freeipa-devel] [PATCHES] 94-96 Remove Entry and Entity classes In-Reply-To: <510A4788.2080304@redhat.com> References: <50FEA307.7080502@redhat.com> <510A4788.2080304@redhat.com> Message-ID: <510BE09F.2010103@redhat.com> Jan Cholasta wrote: > On 22.1.2013 15:32, Jan Cholasta wrote: >> Hi, >> >> these patches remove the Entry and Entity classes and move instantiation >> of LDAPEntry objects to LDAPConnection.make_entry factory method. >> >> Apply on top of Petr Viktorin's LDAP code refactoring (part 1 & 2) >> patches. >> >> Honza >> > > Slightly changed patch 95 and rebased all the patches on top of current > master and LDAP code refactoring part 1 & 2. > > Honza I'm curious why you chose to use __slots__ in the LDAPEntry() class. I'm not too familiar with this directive but I always thought it was memory management thing, or are you trying to purposely limit the capabilities of the class (to prevent us rogue programmers from doing bad things)? rob From pviktori at redhat.com Fri Feb 1 15:52:59 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 01 Feb 2013 16:52:59 +0100 Subject: [Freeipa-devel] [PATCHES] 0117-0118 Port ipa-replica-prepare to the admintool framework In-Reply-To: <510A93C6.7050105@redhat.com> References: <50E580E2.8090502@redhat.com> <50E58E01.3060402@redhat.com> <50E6DC9E.5050702@redhat.com> <51069AF1.2020705@redhat.com> <5108F115.9080902@redhat.com> <510A93C6.7050105@redhat.com> Message-ID: <510BE4DB.9030406@redhat.com> On 01/31/2013 04:54 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 01/28/2013 04:36 PM, Petr Viktorin wrote: >>> On 01/04/2013 02:43 PM, Petr Viktorin wrote: >>>> On 01/03/2013 02:56 PM, John Dennis wrote: >>>>> On 01/03/2013 08:00 AM, Petr Viktorin wrote: >>>>>> Hello, >>>>>> >>>>>> The first patch implements logging-related changes to the admintool >>>>>> framework and ipa-ldap-updater (the only thing ported to it so far). >>>>>> The design document is at >>>>>> http://freeipa.org/page/V3/Logging_and_output >>>>>> >>>>>> John, I decided to go ahead and put an explicit "logger" attribute on >>>>>> the tool class rather than adding debug, info, warn. etc methods >>>>>> dynamically using log_mgr.get_logger. I believe it's the cleanest >>>>>> solution. >>>>>> We had a discussion about this in this thread: >>>>>> https://www.redhat.com/archives/freeipa-devel/2012-July/msg00223.html; >>>>>> >>>>>> I >>>>>> didn't get a reaction to my conclusion so I'm letting you know in >>>>>> case >>>>>> you have more to say. >>>>> >>>>> I'm fine with not directly adding the debug, info, warn etc. methods, >>>>> that practice was historical dating back to the days of Jason. >>>>> However I >>>>> do think it's useful to use a named logger and not the global >>>>> root_logger. I'd prefer we got away from using the root_logger, it's >>>>> continued existence is historical as well and the idea was over >>>>> time we >>>>> would slowly eliminate it's usage. FWIW the log_mgr.get_logger() is >>>>> still useful for what you want to do. >>>>> >>>>> def get_logger(self, who, bind_logger_names=False) >>>>> >>>>> If you don't set bind_logger_names to True (and pass the class >>>>> instance >>>>> as who) you won't get the offensive debug, info, etc. methods added to >>>>> the class instance. But it still does all the other bookeeping. >>>>> >>>>> The 'who' in this instance could be either the name of the admin >>>>> tool or >>>>> the class instance. >>>>> >>>>> Also I'd prefer using the attribute 'log' rather than 'logger'. That >>>>> would make it consistent with code which does already use get_logger() >>>>> passing a class instance because it's adds a 'log' attribute which is >>>>> the logger. Also 'log' is twice as succinct than 'logger' (shorter >>>>> line >>>>> lengths). >>>>> >>>>> Thus if you do: >>>>> >>>>> log_mgr.get_logger(self) >>>>> >>>>> I think you'll get exactly what you want. A logger named for the class >>>>> and being able to say >>>>> >>>>> self.log.debug() >>>>> self.log.error() >>>>> >>>>> inside the class. >>>>> >>>>> In summary, just drop the True from the get_logger() call. >>>>> >>>> >>>> Thanks! Yes, this works better. Updated patches attached. >>>> >>> >>> >>> Here is patch 117 rebased to current master. >>> >> >> Rebased again. > > Just a few minor points. > > Patch 117: > > The n-v-r should be -14. Fixed, thanks for the catch. > > ipa-ldap-updater is no longer runable as non-root. Was this intentional? `ipa-ldap-updater /some/file` works for me. You just can't --upgrade or run the plugins as non-root (and as the help says, --plugins is implied when no files are given). See http://www.redhat.com/archives/freeipa-devel/2012-June/msg00109.html > Patch 118: > > Seems to work as it did though as a side effect of the new logging some > things are displayed that we may want to suppress, specifically: > > request 'https://dart.example.com:8443/ca/ee/ca/profileSubmitSSLClient' > > I think changing the log level to DEBUG is probably the way to go. I traced that to the dogtag module. Removing it in separate patch since it will affect other code (though I don't think it will cause trouble). > While you're at it you might consider replacing the ipa_replica_prepare > remove_file() with the one in installutils. They differ slightly in > implementation but basically do the same thing. Done, thanks. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0117-05-Better-logging-for-AdminTool-and-ipa-ldap-updater.patch Type: text/x-patch Size: 15394 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0118-03-Port-ipa-replica-prepare-to-the-admintool-framework.patch Type: text/x-patch Size: 45265 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0175-Make-ipapython.dogtag-log-requests-at-debug-level-no.patch Type: text/x-patch Size: 917 bytes Desc: not available URL: From pviktori at redhat.com Fri Feb 1 16:44:39 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 01 Feb 2013 17:44:39 +0100 Subject: [Freeipa-devel] [PATCHES] 0107-0114 Fix Confusing ipa tool online help organization In-Reply-To: <510AB97F.2080608@redhat.com> References: <50C9F27E.50808@redhat.com> <50CA76D6.8090404@redhat.com> <50CB0DCF.7050102@redhat.com> <510AB97F.2080608@redhat.com> Message-ID: <510BF0F7.8020102@redhat.com> On 01/31/2013 07:35 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 12/14/2012 01:46 AM, Dmitri Pal wrote: >>> On 12/13/2012 10:21 AM, Petr Viktorin wrote: >>>> https://fedorahosted.org/freeipa/ticket/3060 >>>> >>>> Here is a collection of smallish fixes to `ipa help` and `ipa >>>> --help`. >>>> This should address most of Nikolai's proposal. >>>> Additionally, it's now possible to run `ipa --help` without >>>> a Kerberos ticket. And there are some new tests. >>>> >>>> I've not included the "Often used commands" in `ipa help`; I think >>>> that is material for a manual/tutorial, not a help command. Selecting >>>> a topic from `ipa topics` and then choosing a command from `ipa help >>>> ` is a better way to use the help than the verbose `ipa help >>>> commands` or proposed incomplete "Often used commands". >>> >>> Since the ticket has a bit of discussion and you indicate that you did >>> not to address everything can you please extract what have been >>> addressed and put it into a design page. >>> I know it is not RFE but it would help to validate the changes by >>> testers. >>> Please put the wiki link into the ticket. >>> >> >> http://freeipa.org/page/V3/Help >> >> > > What is the purpose of the no-option outfile? Do you anticipate at some > point opening this up as a real option or making it easier to log while > using the api directly? On incorrect invocation (`ipa`, `ipa TOPIC`), the help command is called internally with outfile=sys.stderr. > The help for help is a little confusing: > > ----- > Purpose: Display help for a command or topic. > Usage: ipa [global-options] help [TOPIC] [options] > > Positional arguments: > TOPIC The topic or command name. > > Options: > -h, --help show this help message and exit > ----- > > Should [TOPIC] be [TOPIC | COMMAND] or something else? I'm trying to discourage `ipa help COMMAND` in favor of `ipa COMMAND --help`, that way you get the proper help for ambiguous words like "ping" (https://fedorahosted.org/freeipa/ticket/3247) I also wanted to keep the help simple and not explain this unimportant detail here. If you have better wording I can of course change it. > On my fresh F-18 install one of the new unit tests fails: > > ====================================================================== > FAIL: Test that `help user-add` & `user-add -h` are equivalent and > contain doc > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in > runTest > self.test(*self.arg) > File "/home/rcrit/redhat/freeipa/tests/test_cmdline/test_help.py", > line 111, in test_command_help > assert h_ctx.stdout == help_ctx.stdout > AssertionError This is weird, it works for me. Could you send me the two outputs so I can get some idea what's wrong? > I'm not sure the errors to stderr are working either: > > $ ipa user-show foo bar baz 2 > /dev/null > ipa: ERROR: command 'user_show' takes at most 1 argument That's just bash being evil, passing 2 as an argument and redirecting stdout. $ ipa user-show foo bar baz 2>/tmp/err $ cat /tmp/err ipa: ERROR: command 'user_show' takes at most 1 argument -- Petr? From rcritten at redhat.com Fri Feb 1 17:06:49 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 01 Feb 2013 12:06:49 -0500 Subject: [Freeipa-devel] [PATCHES] 0107-0114 Fix Confusing ipa tool online help organization In-Reply-To: <510BF0F7.8020102@redhat.com> References: <50C9F27E.50808@redhat.com> <50CA76D6.8090404@redhat.com> <50CB0DCF.7050102@redhat.com> <510AB97F.2080608@redhat.com> <510BF0F7.8020102@redhat.com> Message-ID: <510BF629.4040700@redhat.com> Petr Viktorin wrote: > On 01/31/2013 07:35 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 12/14/2012 01:46 AM, Dmitri Pal wrote: >>>> On 12/13/2012 10:21 AM, Petr Viktorin wrote: >>>>> https://fedorahosted.org/freeipa/ticket/3060 >>>>> >>>>> Here is a collection of smallish fixes to `ipa help` and `ipa >>>>> --help`. >>>>> This should address most of Nikolai's proposal. >>>>> Additionally, it's now possible to run `ipa --help` without >>>>> a Kerberos ticket. And there are some new tests. >>>>> >>>>> I've not included the "Often used commands" in `ipa help`; I think >>>>> that is material for a manual/tutorial, not a help command. Selecting >>>>> a topic from `ipa topics` and then choosing a command from `ipa help >>>>> ` is a better way to use the help than the verbose `ipa help >>>>> commands` or proposed incomplete "Often used commands". >>>> >>>> Since the ticket has a bit of discussion and you indicate that you did >>>> not to address everything can you please extract what have been >>>> addressed and put it into a design page. >>>> I know it is not RFE but it would help to validate the changes by >>>> testers. >>>> Please put the wiki link into the ticket. >>>> >>> >>> http://freeipa.org/page/V3/Help >>> >>> >> >> What is the purpose of the no-option outfile? Do you anticipate at some >> point opening this up as a real option or making it easier to log while >> using the api directly? > > On incorrect invocation (`ipa`, `ipa TOPIC`), the help command is called > internally with outfile=sys.stderr. That's true, but this is a lot of code to abstract writing to sys.stderr. I'm just trying to understand the reasoning. Do you imagine that some time in the future we'd want to control where the output goes? > >> The help for help is a little confusing: >> >> ----- >> Purpose: Display help for a command or topic. >> Usage: ipa [global-options] help [TOPIC] [options] >> >> Positional arguments: >> TOPIC The topic or command name. >> >> Options: >> -h, --help show this help message and exit >> ----- >> >> Should [TOPIC] be [TOPIC | COMMAND] or something else? > > I'm trying to discourage `ipa help COMMAND` in favor of `ipa COMMAND > --help`, that way you get the proper help for ambiguous words like > "ping" (https://fedorahosted.org/freeipa/ticket/3247) > > I also wanted to keep the help simple and not explain this unimportant > detail here. > > If you have better wording I can of course change it. Your reasoning is sound. I"m ok with gently pushing users in that direction but leaving undocumented the old behavior. Should we create a ticket to eventually return an error in that case? >> On my fresh F-18 install one of the new unit tests fails: >> >> ====================================================================== >> FAIL: Test that `help user-add` & `user-add -h` are equivalent and >> contain doc >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >> runTest >> self.test(*self.arg) >> File "/home/rcrit/redhat/freeipa/tests/test_cmdline/test_help.py", >> line 111, in test_command_help >> assert h_ctx.stdout == help_ctx.stdout >> AssertionError > > This is weird, it works for me. Could you send me the two outputs so I > can get some idea what's wrong? > >> I'm not sure the errors to stderr are working either: >> >> $ ipa user-show foo bar baz 2 > /dev/null >> ipa: ERROR: command 'user_show' takes at most 1 argument > > That's just bash being evil, passing 2 as an argument and redirecting > stdout. > > $ ipa user-show foo bar baz 2>/tmp/err > $ cat /tmp/err > ipa: ERROR: command 'user_show' takes at most 1 argument D'oh, yeah I fat-fingered it. Here is the test output: help_ctx.stdout (len 1805) type unicode Purpose: Add a new user. Usage: nosetests [global-options] user-add LOGIN [options] Positional arguments: LOGIN User login Options: -h, --help show this help message and exit --first=STR First name --last=STR Last name --cn=STR Full name --displayname=STR Display name --initials=STR Initials --homedir=STR Home directory --gecos=STR GECOS field --shell=STR Login shell --principal=STR Kerberos principal --email=STR Email address --password Prompt to set the user password --random Generate a random user password --uid=INT User ID Number (system will assign one if not provided) --gidnumber=INT Group ID Number --street=STR Street address --city=STR City --state=STR State/Province --postalcode=STR ZIP --phone=STR Telephone Number --mobile=STR Mobile Telephone Number --pager=STR Pager Number --fax=STR Fax Number --orgunit=STR Org. Unit --title=STR Job Title --manager=STR Manager --carlicense=STR Car License --sshpubkey=STR SSH public key --setattr=STR Set an attribute to a name/value pair. Format is attr=value. For multi-valued attributes, the command replaces the values already present. --addattr=STR Add an attribute/value pair. Format is attr=value. The attribute must be part of the schema. --noprivate Don't create user private group --all Retrieve and print all attributes from the server. Affects command output. --raw Print entries as stored on the server. Only affects output format. h_ctx.stdout (len 1780) type str Usage: nosetests [global-options] user-add LOGIN [options] Positional arguments: LOGIN User login Options: -h, --help show this help message and exit --first=STR First name --last=STR Last name --cn=STR Full name --displayname=STR Display name --initials=STR Initials --homedir=STR Home directory --gecos=STR GECOS field --shell=STR Login shell --principal=STR Kerberos principal --email=STR Email address --password Prompt to set the user password --random Generate a random user password --uid=INT User ID Number (system will assign one if not provided) --gidnumber=INT Group ID Number --street=STR Street address --city=STR City --state=STR State/Province --postalcode=STR ZIP --phone=STR Telephone Number --mobile=STR Mobile Telephone Number --pager=STR Pager Number --fax=STR Fax Number --orgunit=STR Org. Unit --title=STR Job Title --manager=STR Manager --carlicense=STR Car License --sshpubkey=STR SSH public key --setattr=STR Set an attribute to a name/value pair. Format is attr=value. For multi-valued attributes, the command replaces the values already present. --addattr=STR Add an attribute/value pair. Format is attr=value. The attribute must be part of the schema. --noprivate Don't create user private group --all Retrieve and print all attributes from the server. Affects command output. --raw Print entries as stored on the server. Only affects output format. From rcritten at redhat.com Fri Feb 1 19:01:54 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 01 Feb 2013 14:01:54 -0500 Subject: [Freeipa-devel] [PATCHES] 0117-0118 Port ipa-replica-prepare to the admintool framework In-Reply-To: <510BE4DB.9030406@redhat.com> References: <50E580E2.8090502@redhat.com> <50E58E01.3060402@redhat.com> <50E6DC9E.5050702@redhat.com> <51069AF1.2020705@redhat.com> <5108F115.9080902@redhat.com> <510A93C6.7050105@redhat.com> <510BE4DB.9030406@redhat.com> Message-ID: <510C1122.6090903@redhat.com> Petr Viktorin wrote: > On 01/31/2013 04:54 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 01/28/2013 04:36 PM, Petr Viktorin wrote: >>>> On 01/04/2013 02:43 PM, Petr Viktorin wrote: >>>>> On 01/03/2013 02:56 PM, John Dennis wrote: >>>>>> On 01/03/2013 08:00 AM, Petr Viktorin wrote: >>>>>>> Hello, >>>>>>> >>>>>>> The first patch implements logging-related changes to the admintool >>>>>>> framework and ipa-ldap-updater (the only thing ported to it so far). >>>>>>> The design document is at >>>>>>> http://freeipa.org/page/V3/Logging_and_output >>>>>>> >>>>>>> John, I decided to go ahead and put an explicit "logger" >>>>>>> attribute on >>>>>>> the tool class rather than adding debug, info, warn. etc methods >>>>>>> dynamically using log_mgr.get_logger. I believe it's the cleanest >>>>>>> solution. >>>>>>> We had a discussion about this in this thread: >>>>>>> https://www.redhat.com/archives/freeipa-devel/2012-July/msg00223.html; >>>>>>> >>>>>>> >>>>>>> I >>>>>>> didn't get a reaction to my conclusion so I'm letting you know in >>>>>>> case >>>>>>> you have more to say. >>>>>> >>>>>> I'm fine with not directly adding the debug, info, warn etc. methods, >>>>>> that practice was historical dating back to the days of Jason. >>>>>> However I >>>>>> do think it's useful to use a named logger and not the global >>>>>> root_logger. I'd prefer we got away from using the root_logger, it's >>>>>> continued existence is historical as well and the idea was over >>>>>> time we >>>>>> would slowly eliminate it's usage. FWIW the log_mgr.get_logger() is >>>>>> still useful for what you want to do. >>>>>> >>>>>> def get_logger(self, who, bind_logger_names=False) >>>>>> >>>>>> If you don't set bind_logger_names to True (and pass the class >>>>>> instance >>>>>> as who) you won't get the offensive debug, info, etc. methods >>>>>> added to >>>>>> the class instance. But it still does all the other bookeeping. >>>>>> >>>>>> The 'who' in this instance could be either the name of the admin >>>>>> tool or >>>>>> the class instance. >>>>>> >>>>>> Also I'd prefer using the attribute 'log' rather than 'logger'. That >>>>>> would make it consistent with code which does already use >>>>>> get_logger() >>>>>> passing a class instance because it's adds a 'log' attribute which is >>>>>> the logger. Also 'log' is twice as succinct than 'logger' (shorter >>>>>> line >>>>>> lengths). >>>>>> >>>>>> Thus if you do: >>>>>> >>>>>> log_mgr.get_logger(self) >>>>>> >>>>>> I think you'll get exactly what you want. A logger named for the >>>>>> class >>>>>> and being able to say >>>>>> >>>>>> self.log.debug() >>>>>> self.log.error() >>>>>> >>>>>> inside the class. >>>>>> >>>>>> In summary, just drop the True from the get_logger() call. >>>>>> >>>>> >>>>> Thanks! Yes, this works better. Updated patches attached. >>>>> >>>> >>>> >>>> Here is patch 117 rebased to current master. >>>> >>> >>> Rebased again. >> >> Just a few minor points. >> >> Patch 117: >> >> The n-v-r should be -14. > > Fixed, thanks for the catch. > >> >> ipa-ldap-updater is no longer runable as non-root. Was this intentional? > > `ipa-ldap-updater /some/file` works for me. You just can't --upgrade or > run the plugins as non-root (and as the help says, --plugins is implied > when no files are given). > See http://www.redhat.com/archives/freeipa-devel/2012-June/msg00109.html Yeah, I remember the agreed-on behavior. I just re-tested and it works fine, so I must've been having a really bad day yesterday. > >> Patch 118: >> >> Seems to work as it did though as a side effect of the new logging some >> things are displayed that we may want to suppress, specifically: >> >> request 'https://dart.example.com:8443/ca/ee/ca/profileSubmitSSLClient' >> >> I think changing the log level to DEBUG is probably the way to go. > > I traced that to the dogtag module. Removing it in separate patch since > it will affect other code (though I don't think it will cause trouble). Ok with me. >> While you're at it you might consider replacing the ipa_replica_prepare >> remove_file() with the one in installutils. They differ slightly in >> implementation but basically do the same thing. > > Done, thanks. > ACK. Pushed all three to master. rob From rcritten at redhat.com Fri Feb 1 19:03:37 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 01 Feb 2013 14:03:37 -0500 Subject: [Freeipa-devel] [PATCH 0028] Prevent backtrace in ipa-replica-prepare In-Reply-To: <510A6524.105@redhat.com> References: <510A4F9C.5050707@redhat.com> <510A5004.20303@redhat.com> <510A6524.105@redhat.com> Message-ID: <510C1189.80903@redhat.com> Martin Kosek wrote: > On 01/31/2013 12:05 PM, Tomas Babej wrote: >> On 01/31/2013 12:03 PM, Tomas Babej wrote: >>> Hi, >>> >>> This was a regression due to change from DatabaseError to NetworkError >>> when LDAP server is down. >>> >>> https://fedorahosted.org/freeipa/ticket/2939 >>> >>> Tomas >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> Clicking send too soon, patch attached :) >> >> Tomas > > I don't think that removing errors.DatabaseError is necessary. By the way, > would this error (and many similar errors) be solved by a server tool > refactoring that Petr Viktorin is working on? IIRC, he was about to wrap > ipa-replica-prepare in a similar framework like ipa-ldap-updater. > > With a framework like this one, we would not have to specify separate > try..catch lists in all our server manipulation tools. > > Martin Tomas, I just pushed Petr's ipa-replica-prepare framework patch and from my testing this issue is resolved. Can you confirm this and close your bugs/tickets as appropriate? It is git commit 26c498736ec8eabb8dafbc090811c92c79a8c318 in master, for reference. rob From dpal at redhat.com Sun Feb 3 01:51:11 2013 From: dpal at redhat.com (Dmitri Pal) Date: Sat, 02 Feb 2013 20:51:11 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <510AF9BF.9060901@hamada.cz> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> Message-ID: <510DC28F.1000702@redhat.com> On 01/31/2013 06:09 PM, Ondrej Hamada wrote: > Hello, > I'm starting to work on my thesis about 'More types of replicas in > FreeIPA' again. One of the main problems is the way how should the > read-only replicas deal with KDC because they're not supposed to > posses the Kerberos (krb) master key. The task was to investigate how > is this solved in Active Directory and its Read Only Domain Controllers. > > I found out that the basic of RODC behaviour is described on technet > page > (http://technet.microsoft.com/en-us/library/cc754218%28v=ws.10%29.aspx). > > Login situation: > RODC by default forwards the KRB requests to the DC. RODC then > forwards the response back to the client and also requests the > password to be replicated to RODC. Both the user and his host must be > members of 'Allowed RODC Password Replication' group in order to let > user's passwords being replicated to RODCs. > > Request services that the RODC doesn't have credentials for: > Client sends TGS-REQ to RODC. RODC can read the TGT in the request, > but doesn't have credentials for the service. So the request is > forwarded to the DC. DC can decrypt the TGT that was created by RODC > and sends back the TGS-RES that is forwarded to the client. (but it > does not trust the RODC so it recalculates the privilege attribute > certificate). RODC does not cache the credentials for the service. > > During my experiments the credentials got replicated to the RODC on > the first log on of the user. The user's KRB requests were first > forwarded to the DC. When the user got krbtgt and TGS for host, ldap > and cifs, his TGT was revoked by RODC. He run through the auth. > process again, but this time the requests were served by RODC only - > no forwarding - and not TGS for host was requested. > > Unfortunately I can not still recognize how the keys are processed. > There's barely any RPC communication - only one DCERPC packet exchange > between RODC and DC that takes place when the user sends his first TGS > request (this exchange happens also for the clients with disabled > replication). > > It looks to me like the DC knows all the RODC keys. According to > Technet, the MS implementation of Kerberos is able to recognize the > key owner from the Key Version Number value. > > I think I can't get more info from the network traffic examination. Do > you have any ideas or hints on further investigation of the problem? The page you listed shows the diagrams of the user login and TGT and then TGS acquisition. http://technet.microsoft.com/en-us/library/cc754218%28v=WS.10%29#BKMK_AuthRODC Also the following thread sheds some good light on how the authentication and caching happens in case of the RODC. http://social.technet.microsoft.com/forums/en-US/winserverDS/thread/f8d1017e-1f0e-4a9d-a241-b03b508dfe17 So they have policy driven replication of passwords. If password is allowed to be replicated by policy it is replicated if not it RODC has to always proxy to RWDC for this account. The password changes always happen on RWDC and replicated back. They can be replicated by RSO operation that allows replicating a single object. It seems that they assume that all the services are always remote thus connectivity to the central RWDC is a must. They do not seem to keep any service keys locally in the RODC. With SSSD in play on the client I am not sure we should worry about caching at least in the first implementation. So in our case it might make sense to have a proxy KDC and a RO replica with the subset of data replicated to it. I wonder if you can accomplish that by taking 389 RO replica + IAKERB I suggest you look at IAKERB and see if it can be used as a proxy for user authentication, password change and TGS requests. If not may be we can at least use it as an inspiration. The attached diagram shows what I mean. Later we can add some logic for the following: a) RODC requesting replication of a specific object b) RWDC replicating a specific object c) Policy to define for which accounts the passwords and keys can replicated d) RODC getting policy and performing local authentication for the accounts for which the keys were replicated. However with SSSD on the client it might be easier for KDC proxy just to go offline and not respond to the SSSD if it loses connection to the central server. That would force SSSD to go offline and use local password caching. I suspect that SSSD password caching would be enabled in many cases anyways so not caching all passwords in one central locating in RODC might actually be a good thing. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: RODC.png Type: image/png Size: 89945 bytes Desc: not available URL: From tbabej at redhat.com Sun Feb 3 21:12:11 2013 From: tbabej at redhat.com (Tomas Babej) Date: Sun, 03 Feb 2013 22:12:11 +0100 Subject: [Freeipa-devel] [PATCH 0028] Prevent backtrace in ipa-replica-prepare In-Reply-To: <510C1189.80903@redhat.com> References: <510A4F9C.5050707@redhat.com> <510A5004.20303@redhat.com> <510A6524.105@redhat.com> <510C1189.80903@redhat.com> Message-ID: <510ED2AB.6080405@redhat.com> On Fri 01 Feb 2013 08:03:37 PM CET, Rob Crittenden wrote: > Martin Kosek wrote: >> On 01/31/2013 12:05 PM, Tomas Babej wrote: >>> On 01/31/2013 12:03 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> This was a regression due to change from DatabaseError to NetworkError >>>> when LDAP server is down. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2939 >>>> >>>> Tomas >>>> >>>> _______________________________________________ >>>> Freeipa-devel mailing list >>>> Freeipa-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> Clicking send too soon, patch attached :) >>> >>> Tomas >> >> I don't think that removing errors.DatabaseError is necessary. By the >> way, >> would this error (and many similar errors) be solved by a server tool >> refactoring that Petr Viktorin is working on? IIRC, he was about to wrap >> ipa-replica-prepare in a similar framework like ipa-ldap-updater. >> >> With a framework like this one, we would not have to specify separate >> try..catch lists in all our server manipulation tools. >> >> Martin > > Tomas, I just pushed Petr's ipa-replica-prepare framework patch and > from my testing this issue is resolved. Can you confirm this and close > your bugs/tickets as appropriate? > > It is git commit 26c498736ec8eabb8dafbc090811c92c79a8c318 in master, > for reference. > > rob > Yes, backtrace is no longer an issue. I closed the Trac ticket, however, I do not have necessary rights to close the BZ bug. Martin, could you please close Bug 842874? https://bugzilla.redhat.com/show_bug.cgi?id=842874 Thanks, Tomas From jcholast at redhat.com Mon Feb 4 08:52:56 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 04 Feb 2013 09:52:56 +0100 Subject: [Freeipa-devel] [PATCHES] 94-96 Remove Entry and Entity classes In-Reply-To: <510BE09F.2010103@redhat.com> References: <50FEA307.7080502@redhat.com> <510A4788.2080304@redhat.com> <510BE09F.2010103@redhat.com> Message-ID: <510F76E8.4040406@redhat.com> On 1.2.2013 16:34, Rob Crittenden wrote: > Jan Cholasta wrote: >> On 22.1.2013 15:32, Jan Cholasta wrote: >>> Hi, >>> >>> these patches remove the Entry and Entity classes and move instantiation >>> of LDAPEntry objects to LDAPConnection.make_entry factory method. >>> >>> Apply on top of Petr Viktorin's LDAP code refactoring (part 1 & 2) >>> patches. >>> >>> Honza >>> >> >> Slightly changed patch 95 and rebased all the patches on top of current >> master and LDAP code refactoring part 1 & 2. >> >> Honza > > I'm curious why you chose to use __slots__ in the LDAPEntry() class. I'm > not too familiar with this directive but I always thought it was memory > management thing, or are you trying to purposely limit the capabilities > of the class (to prevent us rogue programmers from doing bad things)? > > rob > It's the first one, I tend to use __slots__ in single-purpose classes like LDAPEntry just to save some space. Honza -- Jan Cholasta From mkosek at redhat.com Mon Feb 4 10:00:58 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 04 Feb 2013 11:00:58 +0100 Subject: [Freeipa-devel] [PATCH] 1083 improve migration performance In-Reply-To: <510BDD6F.1020405@redhat.com> References: <510BDD6F.1020405@redhat.com> Message-ID: <510F86DA.3030508@redhat.com> On 02/01/2013 04:21 PM, Rob Crittenden wrote: > I did some analysis on migration and found several areas impacting performance: > > 1. We were calling user_mod to reset the magic value in description to not > create a UPG. This caused a lot of unnecessary queries to display the user. > > 2. We check the remote LDAP server to make sure that the GID is valid and added > a cache. We lacked a negative cache. > > 3. The biggest drag on performance was managing the default users group. After > about 1000 users it would take about half a second to calculate the modlist and > another half second for 389-ds to apply the change. > > This patch addresses all of these. > > For the last what I do is only do the group addition every 100 records. A query > is run to find all users who aren't in the default users group and those are > added. > > I also added a bit of logging so one can better track the progress of migration. > > I migrated 12.5k users with compat enabled in 3 1/2 hours. > > I migrated the same 12.5k users and 2k groups with compat disabled in 30 minutes. > > By contrast when I started, with compat enabled, I migrated: > > 1000 users in 7 minutes > 2000 users in 27 minutes > 3000 users in 1 hour > > rob > Good job, this should improve the migration plugin perfomance a lot. Just few minor remarks: 1) I am not native speaker, but this looks strange to me: +_krb_failed_msg = _('Unable to determine Kerberos principal %s already exists. Use \'ipa user-mod\' to set it manually.') Shouldn't it read "Unable to determine IF Kerberos principal..."? 2) In: + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % group_dn + (result, truncated) = ldap.find_entries(searchfilter, + ['member'], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, + time_limit = -1) Shouldn't we search with empty attrs_list ("attrs_list=['']")? We do not need nor use the member attribute anyway. 3) In + if migrate_cnt > 0 and migrate_cnt % 100: + api.log.info("%d %ss migrated. %s elapsed." % (migrate_cnt, ldap_obj_name, total_dur)) I think you wanted to do this condition: if migrate_cnt > 0 and migrate_cnt % 100 == 0: Otherwise, this logs INFO log level entry for every migrated user. AFAIU, this logging density should only be enabled when httpd is run with debug=true. 4) In _update_default_group: + api.log.debug('Adding users to group duration %s' % d) I would improve it this way: + mode = " (forced)" if force else "" + api.log.debug('Adding %d users to group%s, duration %s', migrate_cnt, mode, d) 5) We now print a lot of interesting migration-related information to IPA server httpd error_log. I think it may be useful to also add a note about it to "ipa help migration" I think that regular admins may not have a clue that we log information like this to this error log. Martin From pviktori at redhat.com Mon Feb 4 12:25:55 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 04 Feb 2013 13:25:55 +0100 Subject: [Freeipa-devel] [PATCHES] 0107-0114 Fix Confusing ipa tool online help organization In-Reply-To: <510BF629.4040700@redhat.com> References: <50C9F27E.50808@redhat.com> <50CA76D6.8090404@redhat.com> <50CB0DCF.7050102@redhat.com> <510AB97F.2080608@redhat.com> <510BF0F7.8020102@redhat.com> <510BF629.4040700@redhat.com> Message-ID: <510FA8D3.4000502@redhat.com> On 02/01/2013 06:06 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 01/31/2013 07:35 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 12/14/2012 01:46 AM, Dmitri Pal wrote: >>>>> On 12/13/2012 10:21 AM, Petr Viktorin wrote: >>>>>> https://fedorahosted.org/freeipa/ticket/3060 >>>>>> >>>>>> Here is a collection of smallish fixes to `ipa help` and `ipa >>>>>> --help`. >>>>>> This should address most of Nikolai's proposal. >>>>>> Additionally, it's now possible to run `ipa --help` without >>>>>> a Kerberos ticket. And there are some new tests. >>>>>> >>>>>> I've not included the "Often used commands" in `ipa help`; I think >>>>>> that is material for a manual/tutorial, not a help command. Selecting >>>>>> a topic from `ipa topics` and then choosing a command from `ipa help >>>>>> ` is a better way to use the help than the verbose `ipa help >>>>>> commands` or proposed incomplete "Often used commands". >>>>> >>>>> Since the ticket has a bit of discussion and you indicate that you did >>>>> not to address everything can you please extract what have been >>>>> addressed and put it into a design page. >>>>> I know it is not RFE but it would help to validate the changes by >>>>> testers. >>>>> Please put the wiki link into the ticket. >>>>> >>>> >>>> http://freeipa.org/page/V3/Help >>>> >>>> >>> >>> What is the purpose of the no-option outfile? Do you anticipate at some >>> point opening this up as a real option or making it easier to log while >>> using the api directly? >> >> On incorrect invocation (`ipa`, `ipa TOPIC`), the help command is called >> internally with outfile=sys.stderr. > > That's true, but this is a lot of code to abstract writing to > sys.stderr. I'm just trying to understand the reasoning. Do you imagine > that some time in the future we'd want to control where the output goes? I don't think that would be useful, I can't imagine why someone would want to log help texts, or use them to script something. Do you have another idea of how this should be done? >>> The help for help is a little confusing: >>> >>> ----- >>> Purpose: Display help for a command or topic. >>> Usage: ipa [global-options] help [TOPIC] [options] >>> >>> Positional arguments: >>> TOPIC The topic or command name. >>> >>> Options: >>> -h, --help show this help message and exit >>> ----- >>> >>> Should [TOPIC] be [TOPIC | COMMAND] or something else? >> >> I'm trying to discourage `ipa help COMMAND` in favor of `ipa COMMAND >> --help`, that way you get the proper help for ambiguous words like >> "ping" (https://fedorahosted.org/freeipa/ticket/3247) >> >> I also wanted to keep the help simple and not explain this unimportant >> detail here. >> >> If you have better wording I can of course change it. > > Your reasoning is sound. I"m ok with gently pushing users in that > direction but leaving undocumented the old behavior. Should we create a > ticket to eventually return an error in that case? Users will expect `ipa help COMMAND` to get them command help, it's pretty standard in tools with subcommands. If it was a part of the API I'd be all for enforcing proper usage, but I think a help command deserves some slack -- it's not that big a deal if you get topic help for ping instead of command help... Hm. Now I see that I should add a notice to the topic help text, to lead users to the right path. Patch attached. We will want to remove `ipa help COMMAND` from the docs, and note that "ipa help" favors topics. We can turn https://fedorahosted.org/freeipa/ticket/3247 into a doc ticket. >>> On my fresh F-18 install one of the new unit tests fails: >>> >>> ====================================================================== >>> FAIL: Test that `help user-add` & `user-add -h` are equivalent and >>> contain doc >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>> runTest >>> self.test(*self.arg) >>> File "/home/rcrit/redhat/freeipa/tests/test_cmdline/test_help.py", >>> line 111, in test_command_help >>> assert h_ctx.stdout == help_ctx.stdout >>> AssertionError >> >> This is weird, it works for me. Could you send me the two outputs so I >> can get some idea what's wrong? >> Can you check you didn't leave out patch 111 (Add command summary?)? -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0175-In-topic-help-text-mention-how-to-get-help-for-comma.patch Type: text/x-patch Size: 1053 bytes Desc: not available URL: From jcholast at redhat.com Mon Feb 4 13:25:03 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 04 Feb 2013 14:25:03 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <510BA30A.4040601@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> Message-ID: <510FB6AF.20903@redhat.com> On 1.2.2013 12:12, Petr Viktorin wrote: > On 01/31/2013 04:18 PM, Jan Cholasta wrote: >> Hi, >> >> these patches implement attribute name case preservation in LDAPEntry. >> Apply on top of Petr Viktorin's LDAP code refactoring patchset (up to >> part 5). >> >> Honza > > Patches 99 & 101 need some tests to make sure the _names work correctly. Added. > > Since you can call LDAPEntry.__init__ in different ways which don't > always correspond to the argument names, it would be nice to explain > them in a docstring. Done. > > A few nitpicks below. > >> freeipa-jcholast-99-Preserve-case-of-attribute-names-in-LDAPEntry.patch >> >> diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py >> index 20c11b4..6268ac0 100644 >> @@ -650,14 +637,48 @@ class LDAPEntry(dict): >> self._orig = self >> self._orig = deepcopy(self) >> >> + def _attr_name(self, name): >> + if not isinstance(name, (unicode, str)): > > Use basestring instead of (unicode, str). Is there any compelling reason to do so? I don't want to support arbitrary basestring subclasses in this code, just unicode and str. > > >> freeipa-jcholast-100-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch >> >> diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py >> index 6268ac0..01fa0c1 100644 >> @@ -595,8 +600,10 @@ class LDAPEntry(dict): >> _obj = {} >> orig = None >> >> + assert isinstance(_conn, IPASimpleLDAPObject) >> assert isinstance(_dn, DN) >> >> + self._conn = lambda: _conn # do not deepcopy me! > > This would be better done by overriding __deepcopy__, or just using a > custom method for the copying. > I have tried multiple different solutions and none is as elegant as this. Yes, it is a hack, but since it is an internal implementation detail of LDAPEntry, I don't see any harm in doing it. On further thought, custom method is probably better suited for this job than than deepcopy. Added. Updated patches attached. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-98.1-Use-the-dn-attribute-of-LDAPEntry-to-set-get-DNs-of-.patch Type: text/x-patch Size: 17429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-99.1-Preserve-case-of-attribute-names-in-LDAPEntry.patch Type: text/x-patch Size: 9780 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-100.1-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch Type: text/x-patch Size: 5443 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-101.1-Support-attributes-with-multiple-names-in-LDAPEntry.patch Type: text/x-patch Size: 3335 bytes Desc: not available URL: From jcholast at redhat.com Mon Feb 4 13:31:13 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 04 Feb 2013 14:31:13 +0100 Subject: [Freeipa-devel] [PATCHES] 102-104 Remove DN normalization Message-ID: <510FB821.6040805@redhat.com> Hi, this patchset removes DN normalization code from the framework. Full DNs including the configured suffix are used instead and must be used in new code. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-102-Use-full-DNs-in-plugin-code.patch Type: text/x-patch Size: 19124 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-103-Remove-DN-normalization-from-the-baseldap-plugin.patch Type: text/x-patch Size: 14125 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-104-Remove-support-for-DN-normalization-from-LDAPClient.patch Type: text/x-patch Size: 20518 bytes Desc: not available URL: From rcritten at redhat.com Mon Feb 4 13:57:07 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 04 Feb 2013 08:57:07 -0500 Subject: [Freeipa-devel] [PATCH 0028] Prevent backtrace in ipa-replica-prepare In-Reply-To: <510ED2AB.6080405@redhat.com> References: <510A4F9C.5050707@redhat.com> <510A5004.20303@redhat.com> <510A6524.105@redhat.com> <510C1189.80903@redhat.com> <510ED2AB.6080405@redhat.com> Message-ID: <510FBE33.8030100@redhat.com> Tomas Babej wrote: > On Fri 01 Feb 2013 08:03:37 PM CET, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 01/31/2013 12:05 PM, Tomas Babej wrote: >>>> On 01/31/2013 12:03 PM, Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> This was a regression due to change from DatabaseError to NetworkError >>>>> when LDAP server is down. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2939 >>>>> >>>>> Tomas >>>>> >>>>> _______________________________________________ >>>>> Freeipa-devel mailing list >>>>> Freeipa-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>>> Clicking send too soon, patch attached :) >>>> >>>> Tomas >>> >>> I don't think that removing errors.DatabaseError is necessary. By the >>> way, >>> would this error (and many similar errors) be solved by a server tool >>> refactoring that Petr Viktorin is working on? IIRC, he was about to wrap >>> ipa-replica-prepare in a similar framework like ipa-ldap-updater. >>> >>> With a framework like this one, we would not have to specify separate >>> try..catch lists in all our server manipulation tools. >>> >>> Martin >> >> Tomas, I just pushed Petr's ipa-replica-prepare framework patch and >> from my testing this issue is resolved. Can you confirm this and close >> your bugs/tickets as appropriate? >> >> It is git commit 26c498736ec8eabb8dafbc090811c92c79a8c318 in master, >> for reference. >> >> rob >> > > Yes, backtrace is no longer an issue. > I closed the Trac ticket, however, I do not have necessary rights to > close the BZ bug. > > Martin, could you please close Bug 842874? > https://bugzilla.redhat.com/show_bug.cgi?id=842874 > > Thanks, > Tomas Done From mkosek at redhat.com Mon Feb 4 14:07:19 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 04 Feb 2013 15:07:19 +0100 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer Message-ID: <510FC097.5050406@redhat.com> When ipa-client-install is run without --server option, it tries to search SRV records for IPA/LDAP server hostname, but it returns only the first record found and when the LDAP server on that hostname is not available, the whole client installation fails. Get all LDAP SRV records instead and fallback to next hostname when the current one is not available. https://fedorahosted.org/freeipa/ticket/3388 -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-362-add-ldap-server-fallback-to-client-installer.patch Type: text/x-patch Size: 7667 bytes Desc: not available URL: From pviktori at redhat.com Mon Feb 4 14:49:40 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 04 Feb 2013 15:49:40 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <510FB6AF.20903@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> Message-ID: <510FCA84.40109@redhat.com> On 02/04/2013 02:25 PM, Jan Cholasta wrote: > On 1.2.2013 12:12, Petr Viktorin wrote: >> On 01/31/2013 04:18 PM, Jan Cholasta wrote: >>> Hi, >>> >>> these patches implement attribute name case preservation in LDAPEntry. >>> Apply on top of Petr Viktorin's LDAP code refactoring patchset (up to >>> part 5). >>> >>> Honza >> >> Patches 99 & 101 need some tests to make sure the _names work correctly. > > Added. I see one of the changes is using has_key instead of `in` for a CIDict. Given that dict.has_key() is deprecated, I think a better solution would be to add __contains__ to CIDict. Is there a reason against that? >> Since you can call LDAPEntry.__init__ in different ways which don't >> always correspond to the argument names, it would be nice to explain >> them in a docstring. > > Done. > >> >> A few nitpicks below. >> >>> freeipa-jcholast-99-Preserve-case-of-attribute-names-in-LDAPEntry.patch >>> >>> diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py >>> index 20c11b4..6268ac0 100644 >>> @@ -650,14 +637,48 @@ class LDAPEntry(dict): >>> self._orig = self >>> self._orig = deepcopy(self) >>> >>> + def _attr_name(self, name): >>> + if not isinstance(name, (unicode, str)): >> >> Use basestring instead of (unicode, str). > > Is there any compelling reason to do so? I don't want to support > arbitrary basestring subclasses in this code, just unicode and str. Using basestring is the standard idiom. Since you're supporting arbitrary str and unicode subclasses anyway, I don't see your point. Besides, basestring can't even be instantiated. Someone who'd subclass basestring directly would really have to know what they're doing. >>> freeipa-jcholast-100-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch >>> >>> diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py >>> index 6268ac0..01fa0c1 100644 >>> @@ -595,8 +600,10 @@ class LDAPEntry(dict): >>> _obj = {} >>> orig = None >>> >>> + assert isinstance(_conn, IPASimpleLDAPObject) >>> assert isinstance(_dn, DN) >>> >>> + self._conn = lambda: _conn # do not deepcopy me! >> >> This would be better done by overriding __deepcopy__, or just using a >> custom method for the copying. >> > > I have tried multiple different solutions and none is as elegant as > this. Yes, it is a hack, but since it is an internal implementation > detail of LDAPEntry, I don't see any harm in doing it. > > On further thought, custom method is probably better suited for this job > than than deepcopy. Added. > > Updated patches attached. Could you also add a docstring to commit()? The function is not clear from the name alone. Other than that, the patches look good. -- Petr? From rcritten at redhat.com Mon Feb 4 15:21:26 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 04 Feb 2013 10:21:26 -0500 Subject: [Freeipa-devel] [PATCH 0027] Add checks for SELinux in install scripts In-Reply-To: <51095135.4040501@redhat.com> References: <51094671.30701@redhat.com> <51095135.4040501@redhat.com> Message-ID: <510FD1F6.60901@redhat.com> Tomas Babej wrote: > On 01/30/2013 05:12 PM, Tomas Babej wrote: >> Hi, >> >> The checks make sure that SELinux is: >> - installed and enabled (on server install) >> - installed and enabled OR not installed (on client install) >> >> Please note that client installs with SELinux not installed are >> allowed since freeipa-client package has no dependency on SELinux. >> (any objections to this approach?) >> >> The (unsupported) option --allow-no-selinux has been added. It can >> used to bypass the checks. >> >> Parts of platform-dependant code were refactored to use newly added >> is_selinux_enabled() function. >> >> https://fedorahosted.org/freeipa/ticket/3359 >> >> Tomas > > I forgot to edit the man pages. Thanks Rob! > > Updated patch attached. > > Tomas After a bit of off-line discussion I don't think we're quite ready yet to require SELinux by default on client installations (even with a flag to work around it). The feeling is this would be disruptive to existing automation. Can you still do the check but not enforce it, simply display a big warning if SELinux is disabled? rob From mkosek at redhat.com Mon Feb 4 15:22:03 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 04 Feb 2013 16:22:03 +0100 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <51011490.10101@redhat.com> References: <51011490.10101@redhat.com> Message-ID: <510FD21B.7070403@redhat.com> On 01/24/2013 12:01 PM, Martin Kosek wrote: > When user tries to perform any action requiring communication with > trusted domain, IPA server tries to retrieve a trust secret on his > behalf to be able to establish the connection. This happens for > example during group-add-member command when external user is > being resolved in the AD. > > When user is not member of Trust admins group, the retrieval crashes > and reports internal error. Catch this exception and rather report > properly formatted ACIError. > > ---- > > I hit this error after updating to the latest FreeIPA version with the AD CVE > fixed. > > Martin > I filed a ticket to not loose this fix and patch. Attaching an updated patch with ticket URL in description. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-355-2-avoid-internal-error-when-user-is-not-trust-admin.patch Type: text/x-patch Size: 2959 bytes Desc: not available URL: From rcritten at redhat.com Mon Feb 4 15:58:31 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 04 Feb 2013 10:58:31 -0500 Subject: [Freeipa-devel] Roadmap Message-ID: <510FDAA7.4010502@redhat.com> In order to keep you informed of the current and future plans for FreeIPA we've updated our Roadmap with some more details: http://www.freeipa.org/page/Roadmap rob From tbabej at redhat.com Mon Feb 4 16:23:19 2013 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 04 Feb 2013 17:23:19 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod Message-ID: <510FE077.8010801@redhat.com> Hi, When adding/modifying an ID range for a trusted domain, the newly added option --dom-name can be used. This looks up SID of the trusted domain in LDAP and therefore the user is not required to write it down in CLI. If the lookup fails, error message asking the user to specify the SID manually is shown. https://fedorahosted.org/freeipa/ticket/3133 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0030-Add-option-to-specify-SID-using-domain-name-to-idran.patch Type: text/x-patch Size: 7830 bytes Desc: not available URL: From pviktori at redhat.com Mon Feb 4 16:35:31 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 04 Feb 2013 17:35:31 +0100 Subject: [Freeipa-devel] [PATCHES] 102-104 Remove DN normalization In-Reply-To: <510FB821.6040805@redhat.com> References: <510FB821.6040805@redhat.com> Message-ID: <510FE353.5080301@redhat.com> On 02/04/2013 02:31 PM, Jan Cholasta wrote: > Hi, > > this patchset removes DN normalization code from the framework. Full DNs > including the configured suffix are used instead and must be used in new > code. > > Honza > Hooray for less magic! I've applied the patches on top of 98-101 (i.e. on top of the big refactoring). They work well, ACK. -- Petr? From pspacek at redhat.com Mon Feb 4 16:37:05 2013 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 04 Feb 2013 17:37:05 +0100 Subject: [Freeipa-devel] [PATCH 0110] Fix crash during per-zone cache flush Message-ID: <510FE3B1.2000102@redhat.com> Hello, Fix crash during per-zone cache flush. https://fedorahosted.org/bind-dyndb-ldap/ticket/107 -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0110-Fix-crash-during-per-zone-cache-flush.patch Type: text/x-patch Size: 1057 bytes Desc: not available URL: From atkac at redhat.com Mon Feb 4 16:45:26 2013 From: atkac at redhat.com (Adam Tkac) Date: Mon, 4 Feb 2013 17:45:26 +0100 Subject: [Freeipa-devel] [PATCH 0110] Fix crash during per-zone cache flush In-Reply-To: <510FE3B1.2000102@redhat.com> References: <510FE3B1.2000102@redhat.com> Message-ID: <20130204164525.GA8603@redhat.com> On Mon, Feb 04, 2013 at 05:37:05PM +0100, Petr Spacek wrote: > Hello, > > Fix crash during per-zone cache flush. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/107 Ack, thanks for the patch. Regards, Adam > From 42edc724eedd401bd1b1e06f1b1c6ef2ad878ed2 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 4 Feb 2013 17:35:24 +0100 > Subject: [PATCH] Fix crash during per-zone cache flush. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/107 > > Signed-off-by: Petr Spacek > --- > src/zone_register.c | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/src/zone_register.c b/src/zone_register.c > index 03eb1d0765de371420e4da6beb1a7b2e2e52db94..e52397357f2ef482d8dfee33f7564cd9157afa56 100644 > --- a/src/zone_register.c > +++ b/src/zone_register.c > @@ -267,8 +267,10 @@ zr_flush_all_caches(zone_register_t *zr) { > ldap_cache_t *cache; > > CHECK(dns_rbtnodechain_current(&chain, NULL, NULL, &node)); > - cache = ((zone_info_t *)(node->data))->cache; > - CHECK(flush_ldap_cache(cache)); > + if (node->data != NULL) { /* skip auxiliary RBT nodes */ > + cache = ((zone_info_t *)(node->data))->cache; > + CHECK(flush_ldap_cache(cache)); > + } > result = dns_rbtnodechain_next(&chain, NULL, NULL); > } > > -- > 1.7.11.7 > -- Adam Tkac, Red Hat, Inc. From tbabej at redhat.com Mon Feb 4 16:55:50 2013 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 04 Feb 2013 17:55:50 +0100 Subject: [Freeipa-devel] [PATCH 0027] Add checks for SELinux in install scripts In-Reply-To: <510FD1F6.60901@redhat.com> References: <51094671.30701@redhat.com> <51095135.4040501@redhat.com> <510FD1F6.60901@redhat.com> Message-ID: <510FE816.6040609@redhat.com> On 02/04/2013 04:21 PM, Rob Crittenden wrote: > Tomas Babej wrote: >> On 01/30/2013 05:12 PM, Tomas Babej wrote: >>> Hi, >>> >>> The checks make sure that SELinux is: >>> - installed and enabled (on server install) >>> - installed and enabled OR not installed (on client install) >>> >>> Please note that client installs with SELinux not installed are >>> allowed since freeipa-client package has no dependency on SELinux. >>> (any objections to this approach?) >>> >>> The (unsupported) option --allow-no-selinux has been added. It can >>> used to bypass the checks. >>> >>> Parts of platform-dependant code were refactored to use newly added >>> is_selinux_enabled() function. >>> >>> https://fedorahosted.org/freeipa/ticket/3359 >>> >>> Tomas >> >> I forgot to edit the man pages. Thanks Rob! >> >> Updated patch attached. >> >> Tomas > > After a bit of off-line discussion I don't think we're quite ready yet > to require SELinux by default on client installations (even with a > flag to work around it). The feeling is this would be disruptive to > existing automation. > > Can you still do the check but not enforce it, simply display a big > warning if SELinux is disabled? > > rob > Sure, here is the updated patch. I edited the commit message, RFE description and man pages according to the new behaviour. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0027-3-Add-checks-for-SELinux-in-install-scripts.patch Type: text/x-patch Size: 13788 bytes Desc: not available URL: From rcritten at redhat.com Mon Feb 4 16:59:18 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 04 Feb 2013 11:59:18 -0500 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <510FC097.5050406@redhat.com> References: <510FC097.5050406@redhat.com> Message-ID: <510FE8E6.4070906@redhat.com> Martin Kosek wrote: > When ipa-client-install is run without --server option, it tries to > search SRV records for IPA/LDAP server hostname, but it returns only > the first record found and when the LDAP server on that hostname is > not available, the whole client installation fails. > > Get all LDAP SRV records instead and fallback to next hostname when > the current one is not available. > > https://fedorahosted.org/freeipa/ticket/3388 I worked on the same ticket, unfortunately, but I didn't mark it as assigned which caused some duplicate effort. Sorry about that. I came up with a very similar solution but took it a bit further. This expands the treatment of the discovered servers as a list of servers rather than a single value. I do a bit more aggressive testing of all servers returned and remove any from the list that are not IPA servers. A server not responding is left in the configured list. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1084-client-failover.patch Type: text/x-diff Size: 16859 bytes Desc: not available URL: From pspacek at redhat.com Mon Feb 4 17:25:27 2013 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 04 Feb 2013 18:25:27 +0100 Subject: [Freeipa-devel] [PATCH 0110] Fix crash during per-zone cache flush In-Reply-To: <20130204164525.GA8603@redhat.com> References: <510FE3B1.2000102@redhat.com> <20130204164525.GA8603@redhat.com> Message-ID: <510FEF07.60300@redhat.com> On 4.2.2013 17:45, Adam Tkac wrote: > On Mon, Feb 04, 2013 at 05:37:05PM +0100, Petr Spacek wrote: >> >Hello, >> > >> > Fix crash during per-zone cache flush. >> > >> > https://fedorahosted.org/bind-dyndb-ldap/ticket/107 > Ack, thanks for the patch. Pushed to master and v2: 42edc724eedd401bd1b1e06f1b1c6ef2ad878ed2 I'm in the process of releasing bind-dyndb-ldap 2.5. The only change from 2.4 is this fix. -- Petr^2 Spacek From rcritten at redhat.com Mon Feb 4 19:05:38 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 04 Feb 2013 14:05:38 -0500 Subject: [Freeipa-devel] [PATCH] 1083 improve migration performance In-Reply-To: <510F86DA.3030508@redhat.com> References: <510BDD6F.1020405@redhat.com> <510F86DA.3030508@redhat.com> Message-ID: <51100682.7000200@redhat.com> Martin Kosek wrote: > On 02/01/2013 04:21 PM, Rob Crittenden wrote: >> I did some analysis on migration and found several areas impacting performance: >> >> 1. We were calling user_mod to reset the magic value in description to not >> create a UPG. This caused a lot of unnecessary queries to display the user. >> >> 2. We check the remote LDAP server to make sure that the GID is valid and added >> a cache. We lacked a negative cache. >> >> 3. The biggest drag on performance was managing the default users group. After >> about 1000 users it would take about half a second to calculate the modlist and >> another half second for 389-ds to apply the change. >> >> This patch addresses all of these. >> >> For the last what I do is only do the group addition every 100 records. A query >> is run to find all users who aren't in the default users group and those are >> added. >> >> I also added a bit of logging so one can better track the progress of migration. >> >> I migrated 12.5k users with compat enabled in 3 1/2 hours. >> >> I migrated the same 12.5k users and 2k groups with compat disabled in 30 minutes. >> >> By contrast when I started, with compat enabled, I migrated: >> >> 1000 users in 7 minutes >> 2000 users in 27 minutes >> 3000 users in 1 hour >> >> rob >> > > Good job, this should improve the migration plugin perfomance a lot. Just few > minor remarks: > > 1) I am not native speaker, but this looks strange to me: > > +_krb_failed_msg = _('Unable to determine Kerberos principal %s already exists. > Use \'ipa user-mod\' to set it manually.') Yup, typo on my part. > > Shouldn't it read "Unable to determine IF Kerberos principal..."? > > 2) In: > > + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % group_dn > + (result, truncated) = ldap.find_entries(searchfilter, > + ['member'], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, > + time_limit = -1) > > Shouldn't we search with empty attrs_list ("attrs_list=['']")? We do not need > nor use the member attribute anyway. Sure. I had it in my head that asking for an empty list returned everything. I did some quick testing and asking for a blank attribute returns no attributes, so I think will work. > > 3) In > > + if migrate_cnt > 0 and migrate_cnt % 100: > + api.log.info("%d %ss migrated. %s elapsed." % > (migrate_cnt, ldap_obj_name, total_dur)) > > I think you wanted to do this condition: > > if migrate_cnt > 0 and migrate_cnt % 100 == 0: Yup, this is what I get for adding something right before submitting the patch. Fixed. > 4) In _update_default_group: > > + api.log.debug('Adding users to group duration %s' % d) > > I would improve it this way: > > + mode = " (forced)" if force else "" > + api.log.debug('Adding %d users to group%s, duration %s', migrate_cnt, > mode, d) Sure, added. > > 5) We now print a lot of interesting migration-related information to IPA > server httpd error_log. I think it may be useful to also add a note about it to > "ipa help migration" I think that regular admins may not have a clue that we > log information like this to this error log. I added a short logging section. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1083-2-migrate.patch Type: text/x-diff Size: 10131 bytes Desc: not available URL: From jcholast at redhat.com Mon Feb 4 20:59:15 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 04 Feb 2013 21:59:15 +0100 Subject: [Freeipa-devel] [PATCHES] 102-104 Remove DN normalization In-Reply-To: <510FE353.5080301@redhat.com> References: <510FB821.6040805@redhat.com> <510FE353.5080301@redhat.com> Message-ID: <51102123.1040209@redhat.com> On 4.2.2013 17:35, Petr Viktorin wrote: > On 02/04/2013 02:31 PM, Jan Cholasta wrote: >> Hi, >> >> this patchset removes DN normalization code from the framework. Full DNs >> including the configured suffix are used instead and must be used in new >> code. >> >> Honza >> > > Hooray for less magic! > > I've applied the patches on top of 98-101 (i.e. on top of the big > refactoring). Right, I forgot to mention that, sorry. > They work well, ACK. > Thanks for the review. Honza -- Jan Cholasta From pviktori at redhat.com Tue Feb 5 12:10:19 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 05 Feb 2013 13:10:19 +0100 Subject: [Freeipa-devel] [PATCH] 0176 Don't add another nsDS5ReplicaId on updates if one already exists Message-ID: <5110F6AB.7060200@redhat.com> Hello, In the LDAP refactor I made getting single-valued attributes[1] ensure that there is really only one value. Previously we just used the first one. This exposed a bug in replica installation: see https://fedorahosted.org/freeipa/ticket/3394. Fix attached. It seems that 389 happened to return the larger value first and the "3" second, so we didn't notice the problem. Still, should be fixed in all branches. Hopefully all regressions caused by the refactor are of this kind :) [1] For those who are following: getEntry() in old code, single_value() after my patches -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0176-Don-t-add-another-nsDS5ReplicaId-on-updates-if-one-a.patch Type: text/x-patch Size: 1216 bytes Desc: not available URL: From mkosek at redhat.com Tue Feb 5 12:23:55 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 05 Feb 2013 13:23:55 +0100 Subject: [Freeipa-devel] [PATCH] 1083 improve migration performance In-Reply-To: <51100682.7000200@redhat.com> References: <510BDD6F.1020405@redhat.com> <510F86DA.3030508@redhat.com> <51100682.7000200@redhat.com> Message-ID: <5110F9DB.9060105@redhat.com> On 02/04/2013 08:05 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 02/01/2013 04:21 PM, Rob Crittenden wrote: >>> I did some analysis on migration and found several areas impacting performance: >>> >>> 1. We were calling user_mod to reset the magic value in description to not >>> create a UPG. This caused a lot of unnecessary queries to display the user. >>> >>> 2. We check the remote LDAP server to make sure that the GID is valid and added >>> a cache. We lacked a negative cache. >>> >>> 3. The biggest drag on performance was managing the default users group. After >>> about 1000 users it would take about half a second to calculate the modlist and >>> another half second for 389-ds to apply the change. >>> >>> This patch addresses all of these. >>> >>> For the last what I do is only do the group addition every 100 records. A query >>> is run to find all users who aren't in the default users group and those are >>> added. >>> >>> I also added a bit of logging so one can better track the progress of >>> migration. >>> >>> I migrated 12.5k users with compat enabled in 3 1/2 hours. >>> >>> I migrated the same 12.5k users and 2k groups with compat disabled in 30 >>> minutes. >>> >>> By contrast when I started, with compat enabled, I migrated: >>> >>> 1000 users in 7 minutes >>> 2000 users in 27 minutes >>> 3000 users in 1 hour >>> >>> rob >>> >> >> Good job, this should improve the migration plugin perfomance a lot. Just few >> minor remarks: >> >> 1) I am not native speaker, but this looks strange to me: >> >> +_krb_failed_msg = _('Unable to determine Kerberos principal %s already exists. >> Use \'ipa user-mod\' to set it manually.') > > Yup, typo on my part. > >> >> Shouldn't it read "Unable to determine IF Kerberos principal..."? >> >> 2) In: >> >> + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % >> group_dn >> + (result, truncated) = ldap.find_entries(searchfilter, >> + ['member'], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, >> + time_limit = -1) >> >> Shouldn't we search with empty attrs_list ("attrs_list=['']")? We do not need >> nor use the member attribute anyway. > > Sure. I had it in my head that asking for an empty list returned everything. I > did some quick testing and asking for a blank attribute returns no attributes, > so I think will work. > >> >> 3) In >> >> + if migrate_cnt > 0 and migrate_cnt % 100: >> + api.log.info("%d %ss migrated. %s elapsed." % >> (migrate_cnt, ldap_obj_name, total_dur)) >> >> I think you wanted to do this condition: >> >> if migrate_cnt > 0 and migrate_cnt % 100 == 0: > > Yup, this is what I get for adding something right before submitting the patch. > Fixed. > > >> 4) In _update_default_group: >> >> + api.log.debug('Adding users to group duration %s' % d) >> >> I would improve it this way: >> >> + mode = " (forced)" if force else "" >> + api.log.debug('Adding %d users to group%s, duration %s', migrate_cnt, >> mode, d) > > Sure, added. > >> >> 5) We now print a lot of interesting migration-related information to IPA >> server httpd error_log. I think it may be useful to also add a note about it to >> "ipa help migration" I think that regular admins may not have a clue that we >> log information like this to this error log. > > I added a short logging section. > > rob > Almost there! 1) Few new typos: +recommended that this be evaluated post-migration to correct or ^^ ... +/etc/ipa/defult.conf or /etc/ipa/server.conf, then an entry will be printed ^^^^^^^^^^^ 2) ipausers group update may report error for the whole migration if there is no user to update (e.g. if all migrated users already exist in IPA): # ipa migrate-ds --with-compat... ipa: ERROR: no such entry This is the misbehaving LDAP search: + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % group_dn + (result, truncated) = ldap.find_entries(searchfilter, + [''], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, + time_limit = -1) Martin From jcholast at redhat.com Tue Feb 5 12:38:58 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 05 Feb 2013 13:38:58 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <510FCA84.40109@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> Message-ID: <5110FD62.4090006@redhat.com> On 4.2.2013 15:49, Petr Viktorin wrote: > On 02/04/2013 02:25 PM, Jan Cholasta wrote: >> On 1.2.2013 12:12, Petr Viktorin wrote: >>> On 01/31/2013 04:18 PM, Jan Cholasta wrote: >>>> Hi, >>>> >>>> these patches implement attribute name case preservation in LDAPEntry. >>>> Apply on top of Petr Viktorin's LDAP code refactoring patchset (up to >>>> part 5). >>>> >>>> Honza >>> >>> Patches 99 & 101 need some tests to make sure the _names work correctly. >> >> Added. > > I see one of the changes is using has_key instead of `in` for a CIDict. > Given that dict.has_key() is deprecated, I think a better solution would > be to add __contains__ to CIDict. Is there a reason against that? I think a separate patch with this and other changes to make CIDict more like dict would be better. > >>> Since you can call LDAPEntry.__init__ in different ways which don't >>> always correspond to the argument names, it would be nice to explain >>> them in a docstring. >> >> Done. >> >>> >>> A few nitpicks below. >>> >>>> freeipa-jcholast-99-Preserve-case-of-attribute-names-in-LDAPEntry.patch >>>> >>>> diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py >>>> index 20c11b4..6268ac0 100644 >>>> @@ -650,14 +637,48 @@ class LDAPEntry(dict): >>>> self._orig = self >>>> self._orig = deepcopy(self) >>>> >>>> + def _attr_name(self, name): >>>> + if not isinstance(name, (unicode, str)): >>> >>> Use basestring instead of (unicode, str). >> >> Is there any compelling reason to do so? I don't want to support >> arbitrary basestring subclasses in this code, just unicode and str. > > Using basestring is the standard idiom. Since you're supporting > arbitrary str and unicode subclasses anyway, I don't see your point. > Besides, basestring can't even be instantiated. Someone who'd subclass > basestring directly would really have to know what they're doing. OK. I just don't like basestring I guess. Fixed. > >>>> freeipa-jcholast-100-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch >>>> >>>> diff --git a/ipaserver/ipaldap.py b/ipaserver/ipaldap.py >>>> index 6268ac0..01fa0c1 100644 >>>> @@ -595,8 +600,10 @@ class LDAPEntry(dict): >>>> _obj = {} >>>> orig = None >>>> >>>> + assert isinstance(_conn, IPASimpleLDAPObject) >>>> assert isinstance(_dn, DN) >>>> >>>> + self._conn = lambda: _conn # do not deepcopy me! >>> >>> This would be better done by overriding __deepcopy__, or just using a >>> custom method for the copying. >>> >> >> I have tried multiple different solutions and none is as elegant as >> this. Yes, it is a hack, but since it is an internal implementation >> detail of LDAPEntry, I don't see any harm in doing it. >> >> On further thought, custom method is probably better suited for this job >> than than deepcopy. Added. >> >> Updated patches attached. > > Could you also add a docstring to commit()? The function is not clear > from the name alone. Done. > > Other than that, the patches look good. > Updated patches attached. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-98.2-Use-the-dn-attribute-of-LDAPEntry-to-set-get-DNs-of-.patch Type: text/x-patch Size: 17429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-99.2-Preserve-case-of-attribute-names-in-LDAPEntry.patch Type: text/x-patch Size: 9683 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-100.2-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch Type: text/x-patch Size: 5561 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-101.2-Support-attributes-with-multiple-names-in-LDAPEntry.patch Type: text/x-patch Size: 3335 bytes Desc: not available URL: From mkosek at redhat.com Tue Feb 5 13:38:21 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 05 Feb 2013 14:38:21 +0100 Subject: [Freeipa-devel] [PATCH] 0176 Don't add another nsDS5ReplicaId on updates if one already exists In-Reply-To: <5110F6AB.7060200@redhat.com> References: <5110F6AB.7060200@redhat.com> Message-ID: <51110B4D.6070802@redhat.com> On 02/05/2013 01:10 PM, Petr Viktorin wrote: > Hello, > In the LDAP refactor I made getting single-valued attributes[1] ensure that > there is really only one value. Previously we just used the first one. > This exposed a bug in replica installation: see > https://fedorahosted.org/freeipa/ticket/3394. Fix attached. > > It seems that 389 happened to return the larger value first and the "3" second, > so we didn't notice the problem. Still, should be fixed in all branches. > > > > Hopefully all regressions caused by the refactor are of this kind :) > > [1] For those who are following: getEntry() in old code, single_value() after > my patches > ACK, thanks for catching this. Martin From rcritten at redhat.com Tue Feb 5 14:27:46 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 05 Feb 2013 09:27:46 -0500 Subject: [Freeipa-devel] [PATCH] 1083 improve migration performance In-Reply-To: <5110F9DB.9060105@redhat.com> References: <510BDD6F.1020405@redhat.com> <510F86DA.3030508@redhat.com> <51100682.7000200@redhat.com> <5110F9DB.9060105@redhat.com> Message-ID: <511116E2.7040903@redhat.com> Martin Kosek wrote: > On 02/04/2013 08:05 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 02/01/2013 04:21 PM, Rob Crittenden wrote: >>>> I did some analysis on migration and found several areas impacting performance: >>>> >>>> 1. We were calling user_mod to reset the magic value in description to not >>>> create a UPG. This caused a lot of unnecessary queries to display the user. >>>> >>>> 2. We check the remote LDAP server to make sure that the GID is valid and added >>>> a cache. We lacked a negative cache. >>>> >>>> 3. The biggest drag on performance was managing the default users group. After >>>> about 1000 users it would take about half a second to calculate the modlist and >>>> another half second for 389-ds to apply the change. >>>> >>>> This patch addresses all of these. >>>> >>>> For the last what I do is only do the group addition every 100 records. A query >>>> is run to find all users who aren't in the default users group and those are >>>> added. >>>> >>>> I also added a bit of logging so one can better track the progress of >>>> migration. >>>> >>>> I migrated 12.5k users with compat enabled in 3 1/2 hours. >>>> >>>> I migrated the same 12.5k users and 2k groups with compat disabled in 30 >>>> minutes. >>>> >>>> By contrast when I started, with compat enabled, I migrated: >>>> >>>> 1000 users in 7 minutes >>>> 2000 users in 27 minutes >>>> 3000 users in 1 hour >>>> >>>> rob >>>> >>> >>> Good job, this should improve the migration plugin perfomance a lot. Just few >>> minor remarks: >>> >>> 1) I am not native speaker, but this looks strange to me: >>> >>> +_krb_failed_msg = _('Unable to determine Kerberos principal %s already exists. >>> Use \'ipa user-mod\' to set it manually.') >> >> Yup, typo on my part. >> >>> >>> Shouldn't it read "Unable to determine IF Kerberos principal..."? >>> >>> 2) In: >>> >>> + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % >>> group_dn >>> + (result, truncated) = ldap.find_entries(searchfilter, >>> + ['member'], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, >>> + time_limit = -1) >>> >>> Shouldn't we search with empty attrs_list ("attrs_list=['']")? We do not need >>> nor use the member attribute anyway. >> >> Sure. I had it in my head that asking for an empty list returned everything. I >> did some quick testing and asking for a blank attribute returns no attributes, >> so I think will work. >> >>> >>> 3) In >>> >>> + if migrate_cnt > 0 and migrate_cnt % 100: >>> + api.log.info("%d %ss migrated. %s elapsed." % >>> (migrate_cnt, ldap_obj_name, total_dur)) >>> >>> I think you wanted to do this condition: >>> >>> if migrate_cnt > 0 and migrate_cnt % 100 == 0: >> >> Yup, this is what I get for adding something right before submitting the patch. >> Fixed. >> >> >>> 4) In _update_default_group: >>> >>> + api.log.debug('Adding users to group duration %s' % d) >>> >>> I would improve it this way: >>> >>> + mode = " (forced)" if force else "" >>> + api.log.debug('Adding %d users to group%s, duration %s', migrate_cnt, >>> mode, d) >> >> Sure, added. >> >>> >>> 5) We now print a lot of interesting migration-related information to IPA >>> server httpd error_log. I think it may be useful to also add a note about it to >>> "ipa help migration" I think that regular admins may not have a clue that we >>> log information like this to this error log. >> >> I added a short logging section. >> >> rob >> > > Almost there! > > 1) Few new typos: > +recommended that this be evaluated post-migration to correct or > ^^ I think this was correct but it was definitely awkward so I re-worded the sentence. > ... > +/etc/ipa/defult.conf or /etc/ipa/server.conf, then an entry will be printed > ^^^^^^^^^^^ Fixed > 2) ipausers group update may report error for the whole migration if there is > no user to update (e.g. if all migrated users already exist in IPA): > > # ipa migrate-ds --with-compat... > ipa: ERROR: no such entry > > This is the misbehaving LDAP search: > + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % group_dn > + (result, truncated) = ldap.find_entries(searchfilter, > + [''], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, > + time_limit = -1) Ok, wrapped in try/except. I also moved the ldap.update_entry() block so we don't try the update if we know we have no new members. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1083-3-migrate.patch Type: text/x-diff Size: 10218 bytes Desc: not available URL: From pviktori at redhat.com Tue Feb 5 14:45:48 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 05 Feb 2013 15:45:48 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <5110FD62.4090006@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> <5110FD62.4090006@redhat.com> Message-ID: <51111B1C.2010305@redhat.com> On 02/05/2013 01:38 PM, Jan Cholasta wrote: > On 4.2.2013 15:49, Petr Viktorin wrote: [...] >> >> I see one of the changes is using has_key instead of `in` for a CIDict. >> Given that dict.has_key() is deprecated, I think a better solution would >> be to add __contains__ to CIDict. Is there a reason against that? > > I think a separate patch with this and other changes to make CIDict more > like dict would be better. OK, I'll make a patch. [...] > Updated patches attached. > The changes look good but I can no longer apply the patches. Can you please rebase them? -- Petr? From mkosek at redhat.com Tue Feb 5 15:11:45 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 05 Feb 2013 16:11:45 +0100 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <510FE8E6.4070906@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> Message-ID: <51112131.3080409@redhat.com> On 02/04/2013 05:59 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> When ipa-client-install is run without --server option, it tries to >> search SRV records for IPA/LDAP server hostname, but it returns only >> the first record found and when the LDAP server on that hostname is >> not available, the whole client installation fails. >> >> Get all LDAP SRV records instead and fallback to next hostname when >> the current one is not available. >> >> https://fedorahosted.org/freeipa/ticket/3388 > > I worked on the same ticket, unfortunately, but I didn't mark it as assigned > which caused some duplicate effort. Sorry about that. > > I came up with a very similar solution but took it a bit further. This expands > the treatment of the discovered servers as a list of servers rather than a > single value. > > I do a bit more aggressive testing of all servers returned and remove any from > the list that are not IPA servers. A server not responding is left in the > configured list. > > rob 1) (minor) If I connected IPA host to other IPA server before next enrollment, there will be outdated /etc/ipa/ca.crt. This will cause all tries to connect to IPA server to fail with TLS error, but without giving any clue to user... # ipa-client-install Provide your IPA server name (ex: ipa.example.com): He would need to reach out to the log to find this line: LDAP Error: Connect error: TLS error -8054:You are attempting to import a cert with the same issuer/serial as an existing cert, but that is not the same cert. I am thinking if we should not expose some LDAP errors after all? To give some clue to user... 2) (minor) While we are touching these errors I would also fix a typo there :-) ... if isinstance(err, ldap.INAPPROPRIATE_AUTH): root_logger.debug("LDAP Error: Anonymous acces not allowed") return [NO_ACCESS_TO_LDAP] ^^^^^ ... 3) (Regression) You neither set ds.server nor add server to valid_servers when anonymous access is not enabled. The installer then does not try to connect to this server even though the installation would succeed: ... elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == NO_TLS_LDAP: ldapaccess = False ... 4) (Regression) Client will now report success in discovery even when the server is down: # ipa-client-install Unable to verify that vm-037.idm.lab.bos.redhat.com (realm IDM.LAB.BOS.REDHAT.COM) is an IPA server Discovery was successful! Hostname: vm-148.idm.lab.bos.redhat.com Realm: IDM.LAB.BOS.REDHAT.COM DNS Domain: idm.lab.bos.redhat.com IPA Server: vm-037.idm.lab.bos.redhat.com BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com Continue to configure the system with these values? [no]: y User authorized to enroll computers: admin Synchronizing time with KDC... Password for admin at IDM.LAB.BOS.REDHAT.COM: Kerberos authentication failed kinit: Generic error (see e-text) while getting initial credentials Please make sure the following ports are opened in the firewall settings: TCP: 80, 88, 389 UDP: 88 (at least one of TCP/UDP ports 88 has to be open) Also note that following ports are necessary for ipa-client working properly after enrollment: TCP: 464 UDP: 464, 123 (if NTP enabled) Installation failed. Rolling back changes. IPA client is not configured on this system. LDAP on vm-037 in this case is down. I think this would cause a regression too, because previously we simply reported that no discovered server is available and let user enter the server manually. IMO we are trying to be too smart when processing the (discovered) servers. Why do we need to process and verify _all_ discovered servers even when the list is not written anywhere in case of SRV lookup? I think it causes unnecessary traffic on network - we should connect to the first server available. 5) In ipa-client-automount: + # Now confirm that our server is an IPA server. Stop checking on the first + # success so we know we have at least one good one. + for server in servers: + root_logger.debug("Verifying that %s is an IPA server" % server) + ldapret = ds.ipacheckldap(server, api.env.realm) In case of successful autodiscovery, this looks redundant as we try to verify the servers second time even though we already did it with ds.search and ds.server should already contain a functional server. Martin From mkosek at redhat.com Tue Feb 5 16:20:55 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 05 Feb 2013 17:20:55 +0100 Subject: [Freeipa-devel] [PATCH] 1083 improve migration performance In-Reply-To: <511116E2.7040903@redhat.com> References: <510BDD6F.1020405@redhat.com> <510F86DA.3030508@redhat.com> <51100682.7000200@redhat.com> <5110F9DB.9060105@redhat.com> <511116E2.7040903@redhat.com> Message-ID: <51113167.9090208@redhat.com> On 02/05/2013 03:27 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 02/04/2013 08:05 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 02/01/2013 04:21 PM, Rob Crittenden wrote: >>>>> I did some analysis on migration and found several areas impacting >>>>> performance: >>>>> >>>>> 1. We were calling user_mod to reset the magic value in description to not >>>>> create a UPG. This caused a lot of unnecessary queries to display the user. >>>>> >>>>> 2. We check the remote LDAP server to make sure that the GID is valid and >>>>> added >>>>> a cache. We lacked a negative cache. >>>>> >>>>> 3. The biggest drag on performance was managing the default users group. >>>>> After >>>>> about 1000 users it would take about half a second to calculate the >>>>> modlist and >>>>> another half second for 389-ds to apply the change. >>>>> >>>>> This patch addresses all of these. >>>>> >>>>> For the last what I do is only do the group addition every 100 records. A >>>>> query >>>>> is run to find all users who aren't in the default users group and those are >>>>> added. >>>>> >>>>> I also added a bit of logging so one can better track the progress of >>>>> migration. >>>>> >>>>> I migrated 12.5k users with compat enabled in 3 1/2 hours. >>>>> >>>>> I migrated the same 12.5k users and 2k groups with compat disabled in 30 >>>>> minutes. >>>>> >>>>> By contrast when I started, with compat enabled, I migrated: >>>>> >>>>> 1000 users in 7 minutes >>>>> 2000 users in 27 minutes >>>>> 3000 users in 1 hour >>>>> >>>>> rob >>>>> >>>> >>>> Good job, this should improve the migration plugin perfomance a lot. Just few >>>> minor remarks: >>>> >>>> 1) I am not native speaker, but this looks strange to me: >>>> >>>> +_krb_failed_msg = _('Unable to determine Kerberos principal %s already >>>> exists. >>>> Use \'ipa user-mod\' to set it manually.') >>> >>> Yup, typo on my part. >>> >>>> >>>> Shouldn't it read "Unable to determine IF Kerberos principal..."? >>>> >>>> 2) In: >>>> >>>> + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % >>>> group_dn >>>> + (result, truncated) = ldap.find_entries(searchfilter, >>>> + ['member'], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, >>>> + time_limit = -1) >>>> >>>> Shouldn't we search with empty attrs_list ("attrs_list=['']")? We do not need >>>> nor use the member attribute anyway. >>> >>> Sure. I had it in my head that asking for an empty list returned everything. I >>> did some quick testing and asking for a blank attribute returns no attributes, >>> so I think will work. >>> >>>> >>>> 3) In >>>> >>>> + if migrate_cnt > 0 and migrate_cnt % 100: >>>> + api.log.info("%d %ss migrated. %s elapsed." % >>>> (migrate_cnt, ldap_obj_name, total_dur)) >>>> >>>> I think you wanted to do this condition: >>>> >>>> if migrate_cnt > 0 and migrate_cnt % 100 == 0: >>> >>> Yup, this is what I get for adding something right before submitting the patch. >>> Fixed. >>> >>> >>>> 4) In _update_default_group: >>>> >>>> + api.log.debug('Adding users to group duration %s' % d) >>>> >>>> I would improve it this way: >>>> >>>> + mode = " (forced)" if force else "" >>>> + api.log.debug('Adding %d users to group%s, duration %s', migrate_cnt, >>>> mode, d) >>> >>> Sure, added. >>> >>>> >>>> 5) We now print a lot of interesting migration-related information to IPA >>>> server httpd error_log. I think it may be useful to also add a note about >>>> it to >>>> "ipa help migration" I think that regular admins may not have a clue that we >>>> log information like this to this error log. >>> >>> I added a short logging section. >>> >>> rob >>> >> >> Almost there! >> >> 1) Few new typos: >> +recommended that this be evaluated post-migration to correct or >> ^^ > > I think this was correct but it was definitely awkward so I re-worded the > sentence. > >> ... >> +/etc/ipa/defult.conf or /etc/ipa/server.conf, then an entry will be printed >> ^^^^^^^^^^^ > > Fixed > >> 2) ipausers group update may report error for the whole migration if there is >> no user to update (e.g. if all migrated users already exist in IPA): >> >> # ipa migrate-ds --with-compat... >> ipa: ERROR: no such entry >> >> This is the misbehaving LDAP search: >> + searchfilter = "(&(objectclass=posixAccount)(!(memberof=%s)))" % >> group_dn >> + (result, truncated) = ldap.find_entries(searchfilter, >> + [''], api.env.container_user, scope=_ldap.SCOPE_SUBTREE, >> + time_limit = -1) > > Ok, wrapped in try/except. > > I also moved the ldap.update_entry() block so we don't try the update if we > know we have no new members. > > rob ACK. Pushed to master, ipa-3-1, ipa-3-0. Martin From mkosek at redhat.com Tue Feb 5 16:25:55 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 05 Feb 2013 17:25:55 +0100 Subject: [Freeipa-devel] [RFE] Configurable SID Blacklists Message-ID: <51113293.6010605@redhat.com> Hello, below is a design page for ticket https://fedorahosted.org/freeipa/ticket/3289: http://www.freeipa.org/page/V3/Configurable_SID_Blacklists There is one question in the text. Martin ----------------------------------------------------------------------------- = Overview = Related tickets: * [https://fedorahosted.org/freeipa/ticket/3289 3289: Make SID checks for MS-PAC filter configurable] * [https://fedorahosted.org/freeipa/ticket/3231 3231: Need to relax MS-PAC checks] Microsoft Windows 2012 slightly changed what it sends in the MS-PAC, and it sends a special SID in the ExtraSids buffer. We used to not accept this MS-PAC and raise validation error, but ticket [https://fedorahosted.org/freeipa/ticket/3231 3231] introduced a static list of SIDs that are filtered and are excluded from the MS-PAC to avoid this error. A target of this RFE is to include this list in LDAP to allow Administrator to change the list and add or remove SID to filter. = Use Cases = Windows 2012 user from a trusted AD domain tries to authenticate to IPA domain, but his ticket is refused due to MS-PAC check. Administrator may want to extend the default list of SID so that the colliding SID is filtered. = Design= == Configuration granularity == SID blacklist should be configured per-trust. Administrator should be able to configure a blacklist for both ''incoming'' MS-PAC (i.e. authentication from a trusted domain to IPA domain) and ''outgoing'' MS-PAC (i.e. for transitive authentication of a user from IPA trusted domain trying to authenticate to other domain trusted by IPA, but which is not trusted directly by this domain). == Schema updates == The feature will introduce 2 new ''attributeTypes'' which will be added to ''MAY'' list of ''ipaNTTrustedDomain'' object class: attributetypes: ( 2.16.840.1.113730.3.8.11.38 NAME 'ipaNTSIDBlacklistIncoming' DESC 'Extra SIDs filtered out from incoming MS-PAC' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 X-ORIGIN 'IPA v3' ) attributetypes: ( 2.16.840.1.113730.3.8.11.39 NAME 'ipaNTSIDBlacklistOutgoing' DESC 'Extra SIDs filtered out from outgoing MS-PAC' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 X-ORIGIN 'IPA v3' ) == ipa-kdb Changes == Currently, ipa-kdb uses mspac_well_known_sids static list of SIDs to filter SIDs from MS-PAC. Changes: * struct ipadb_mspac will be changed: ** well_known_sids will be renamed to sid_blacklist_incoming ** new sid_blacklist_outgoing will be added * ipadb_mspac_fill_well_known_sids function will be updated to read these new attributes from LDAP and if it finds these attributes, it will fill their value to mspac->sid_blacklist_incoming or mspac->sid_blacklist_outgoing respectively. If the attributes for the trust is missing, it will use the default value in mspac_well_known_sids. * Note that mspac->sid_blacklist_outgoing '''will be unused''' until the transitive trusts functionality is implemented. These new attributes should not cause high LDAP load as ipadb_reinit_mspac it is run at most once per minute. = Implementation = N/A = Feature Managment = === UI === UI will need to allow updating these new attributes in ''Settings'' tab in ''IPA Server'' -> ''Trusts'' section. === CLI === CLI should allow editing of these new attributes. They should not be displayed by default in ''trust-show'' or ''trust-find'' command, but only with ''--all'' option to keep clarity of trust entries in these commands. = Major configuration options and enablement = N/A = Replication = New attributes will be replicated. = Updates and Upgrades = The 2 new ''attributeTypes'' will be added and one ''ipaNTTrustedDomain'' object class will be updated. '''QUESTION''': We can either fill ''ipaNTSIDBlacklistIncoming'' and ''ipaNTSIDBlacklistIncoming'' for all current trusts during updates or fill them only for re-established and new trusts. The latter would avoid unnecessary update plugin. The prefilled list should be equal to mspac_well_known_sids list in ipa_kdb_mspac.c. = Dependencies = N/A = External Impact = N/A From rcritten at redhat.com Tue Feb 5 16:57:31 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 05 Feb 2013 11:57:31 -0500 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <51112131.3080409@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> Message-ID: <511139FB.3050102@redhat.com> Martin Kosek wrote: > On 02/04/2013 05:59 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> When ipa-client-install is run without --server option, it tries to >>> search SRV records for IPA/LDAP server hostname, but it returns only >>> the first record found and when the LDAP server on that hostname is >>> not available, the whole client installation fails. >>> >>> Get all LDAP SRV records instead and fallback to next hostname when >>> the current one is not available. >>> >>> https://fedorahosted.org/freeipa/ticket/3388 >> >> I worked on the same ticket, unfortunately, but I didn't mark it as assigned >> which caused some duplicate effort. Sorry about that. >> >> I came up with a very similar solution but took it a bit further. This expands >> the treatment of the discovered servers as a list of servers rather than a >> single value. >> >> I do a bit more aggressive testing of all servers returned and remove any from >> the list that are not IPA servers. A server not responding is left in the >> configured list. >> >> rob > > 1) (minor) If I connected IPA host to other IPA server before next enrollment, > there will be outdated /etc/ipa/ca.crt. This will cause all tries to connect to > IPA server to fail with TLS error, but without giving any clue to user... Yeah, this can be a problem but I'm going to leave things as-is for now. I believe we have a separate ticket to clean this up. I don't want to combine too many things into this patch, it is disruptive enough. > > # ipa-client-install > Provide your IPA server name (ex: ipa.example.com): > > He would need to reach out to the log to find this line: > LDAP Error: Connect error: TLS error -8054:You are attempting to import a cert > with the same issuer/serial as an existing cert, but that is not the same cert. > > I am thinking if we should not expose some LDAP errors after all? To give some > clue to user... Yeah, I switched the LDAP error unhandled exception back from debug to error. > > 2) (minor) While we are touching these errors I would also fix a typo there :-) > ... > if isinstance(err, ldap.INAPPROPRIATE_AUTH): > root_logger.debug("LDAP Error: Anonymous acces not allowed") > return [NO_ACCESS_TO_LDAP] ^^^^^ > ... Heh, ok. > > 3) (Regression) You neither set ds.server nor add server to valid_servers when > anonymous access is not enabled. The installer then does not try to connect to > this server even though the installation would succeed: > > ... > elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == NO_TLS_LDAP: > ldapaccess = False > ... Good point. The handling for this is done a bit later so I need to defer a little processing but it should work now. > > 4) (Regression) Client will now report success in discovery even when the > server is down: > > # ipa-client-install > Unable to verify that vm-037.idm.lab.bos.redhat.com (realm > IDM.LAB.BOS.REDHAT.COM) is an IPA server > Discovery was successful! > Hostname: vm-148.idm.lab.bos.redhat.com > Realm: IDM.LAB.BOS.REDHAT.COM > DNS Domain: idm.lab.bos.redhat.com > IPA Server: vm-037.idm.lab.bos.redhat.com > BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com > > Continue to configure the system with these values? [no]: y > User authorized to enroll computers: admin > Synchronizing time with KDC... > Password for admin at IDM.LAB.BOS.REDHAT.COM: > Kerberos authentication failed > kinit: Generic error (see e-text) while getting initial credentials > > Please make sure the following ports are opened in the firewall settings: > TCP: 80, 88, 389 > UDP: 88 (at least one of TCP/UDP ports 88 has to be open) > Also note that following ports are necessary for ipa-client working properly > after enrollment: > TCP: 464 > UDP: 464, 123 (if NTP enabled) > Installation failed. Rolling back changes. > IPA client is not configured on this system. > > > LDAP on vm-037 in this case is down. I think this would cause a regression too, > because previously we simply reported that no discovered server is available > and let user enter the server manually. Fixed. > > IMO we are trying to be too smart when processing the (discovered) servers. Why > do we need to process and verify _all_ discovered servers even when the list is > not written anywhere in case of SRV lookup? I think it causes unnecessary > traffic on network - we should connect to the first server available. That's a good point. Now we except on the first discovered server. I think for user-provided servers we still want to verify them all since they will be hardcoded in a config file. > 5) In ipa-client-automount: > > + # Now confirm that our server is an IPA server. Stop checking on the first > + # success so we know we have at least one good one. > + for server in servers: > + root_logger.debug("Verifying that %s is an IPA server" % server) > + ldapret = ds.ipacheckldap(server, api.env.realm) > > > In case of successful autodiscovery, this looks redundant as we try to verify > the servers second time even though we already did it with ds.search and > ds.server should already contain a functional server. That's true, I ripped all this out. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1084-2-client-failover.patch Type: text/x-diff Size: 17734 bytes Desc: not available URL: From pviktori at redhat.com Tue Feb 5 17:02:55 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 05 Feb 2013 18:02:55 +0100 Subject: [Freeipa-devel] [PATCHES] 0177-0179 Add missing dict methods to CIDict Message-ID: <51113B3F.3070808@redhat.com> CIDict, our case-insensitive dictionary, inherits from dict but did not reimplement the full dict interface. Calling the missing methods silently invoked case-sensitive behavior. Our code seems to avoid that, but it's a bit of a minefield for new development. Patch 119 adds the missing dict methods (except view{items,keys,values}(), which now raise errors), and adds tests. Patches 117-118 modernize the testsuite a bit (these have been sitting in my queue for a while, I think now is a good time to submit them): The first one moves some old tests from the main code tree to tests/. (The adtrust_install test wasn't run before, this move makes nose notice it). The second converts CIDict's unittest-based suite to nose. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0177-Move-tests-to-test-directories.patch Type: text/x-patch Size: 4649 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0178-Convert-test_ipautil-from-unittest-to-nose.patch Type: text/x-patch Size: 17739 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0179-Add-missing-dict-methods-to-CIDict.patch Type: text/x-patch Size: 6200 bytes Desc: not available URL: From rcritten at redhat.com Tue Feb 5 23:44:07 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 05 Feb 2013 18:44:07 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command Message-ID: <51119947.9050707@redhat.com> This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1085-cert-find.patch Type: text/x-diff Size: 31870 bytes Desc: not available URL: From mkosek at redhat.com Wed Feb 6 09:48:30 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 06 Feb 2013 10:48:30 +0100 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <511139FB.3050102@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> <511139FB.3050102@redhat.com> Message-ID: <511226EE.8060703@redhat.com> On 02/05/2013 05:57 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 02/04/2013 05:59 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> When ipa-client-install is run without --server option, it tries to >>>> search SRV records for IPA/LDAP server hostname, but it returns only >>>> the first record found and when the LDAP server on that hostname is >>>> not available, the whole client installation fails. >>>> >>>> Get all LDAP SRV records instead and fallback to next hostname when >>>> the current one is not available. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3388 >>> >>> I worked on the same ticket, unfortunately, but I didn't mark it as assigned >>> which caused some duplicate effort. Sorry about that. >>> >>> I came up with a very similar solution but took it a bit further. This expands >>> the treatment of the discovered servers as a list of servers rather than a >>> single value. >>> >>> I do a bit more aggressive testing of all servers returned and remove any from >>> the list that are not IPA servers. A server not responding is left in the >>> configured list. >>> >>> rob >> >> 1) (minor) If I connected IPA host to other IPA server before next enrollment, >> there will be outdated /etc/ipa/ca.crt. This will cause all tries to connect to >> IPA server to fail with TLS error, but without giving any clue to user... > > Yeah, this can be a problem but I'm going to leave things as-is for now. I > believe we have a separate ticket to clean this up. I don't want to combine too > many things into this patch, it is disruptive enough. > >> >> # ipa-client-install >> Provide your IPA server name (ex: ipa.example.com): >> >> He would need to reach out to the log to find this line: >> LDAP Error: Connect error: TLS error -8054:You are attempting to import a cert >> with the same issuer/serial as an existing cert, but that is not the same cert. >> >> I am thinking if we should not expose some LDAP errors after all? To give some >> clue to user... > > Yeah, I switched the LDAP error unhandled exception back from debug to error. > >> >> 2) (minor) While we are touching these errors I would also fix a typo there :-) >> ... >> if isinstance(err, ldap.INAPPROPRIATE_AUTH): >> root_logger.debug("LDAP Error: Anonymous acces not allowed") >> return [NO_ACCESS_TO_LDAP] ^^^^^ >> ... > > Heh, ok. > > >> >> 3) (Regression) You neither set ds.server nor add server to valid_servers when >> anonymous access is not enabled. The installer then does not try to connect to >> this server even though the installation would succeed: >> >> ... >> elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == NO_TLS_LDAP: >> ldapaccess = False >> ... > > Good point. The handling for this is done a bit later so I need to defer a > little processing but it should work now. > >> >> 4) (Regression) Client will now report success in discovery even when the >> server is down: >> >> # ipa-client-install >> Unable to verify that vm-037.idm.lab.bos.redhat.com (realm >> IDM.LAB.BOS.REDHAT.COM) is an IPA server >> Discovery was successful! >> Hostname: vm-148.idm.lab.bos.redhat.com >> Realm: IDM.LAB.BOS.REDHAT.COM >> DNS Domain: idm.lab.bos.redhat.com >> IPA Server: vm-037.idm.lab.bos.redhat.com >> BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com >> >> Continue to configure the system with these values? [no]: y >> User authorized to enroll computers: admin >> Synchronizing time with KDC... >> Password for admin at IDM.LAB.BOS.REDHAT.COM: >> Kerberos authentication failed >> kinit: Generic error (see e-text) while getting initial credentials >> >> Please make sure the following ports are opened in the firewall settings: >> TCP: 80, 88, 389 >> UDP: 88 (at least one of TCP/UDP ports 88 has to be open) >> Also note that following ports are necessary for ipa-client working properly >> after enrollment: >> TCP: 464 >> UDP: 464, 123 (if NTP enabled) >> Installation failed. Rolling back changes. >> IPA client is not configured on this system. >> >> >> LDAP on vm-037 in this case is down. I think this would cause a regression too, >> because previously we simply reported that no discovered server is available >> and let user enter the server manually. > > Fixed. > >> >> IMO we are trying to be too smart when processing the (discovered) servers. Why >> do we need to process and verify _all_ discovered servers even when the list is >> not written anywhere in case of SRV lookup? I think it causes unnecessary >> traffic on network - we should connect to the first server available. > > That's a good point. Now we except on the first discovered server. > > I think for user-provided servers we still want to verify them all since they > will be hardcoded in a config file. > >> 5) In ipa-client-automount: >> >> + # Now confirm that our server is an IPA server. Stop checking on the first >> + # success so we know we have at least one good one. >> + for server in servers: >> + root_logger.debug("Verifying that %s is an IPA server" % server) >> + ldapret = ds.ipacheckldap(server, api.env.realm) >> >> >> In case of successful autodiscovery, this looks redundant as we try to verify >> the servers second time even though we already did it with ds.search and >> ds.server should already contain a functional server. > > That's true, I ripped all this out. > > rob > 1) One whitespace error: git apply /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch:96: trailing whitespace. warning: 1 line adds whitespace errors. 2) When ipa-client-automount --server option is passed, the --server value is then never user. This leads to installation failure when --no-sssd and --server options are used: ... Raised exception local variable 'server' referenced before assignment ... ipa-client-install looks OK so far, I did not find any issue in my tests. Martin From jcholast at redhat.com Wed Feb 6 09:55:53 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 06 Feb 2013 10:55:53 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <51111B1C.2010305@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> <5110FD62.4090006@redhat.com> <51111B1C.2010305@redhat.com> Message-ID: <511228A9.5060303@redhat.com> On 5.2.2013 15:45, Petr Viktorin wrote: > On 02/05/2013 01:38 PM, Jan Cholasta wrote: >> On 4.2.2013 15:49, Petr Viktorin wrote: > [...] >>> >>> I see one of the changes is using has_key instead of `in` for a CIDict. >>> Given that dict.has_key() is deprecated, I think a better solution would >>> be to add __contains__ to CIDict. Is there a reason against that? >> >> I think a separate patch with this and other changes to make CIDict more >> like dict would be better. > > OK, I'll make a patch. > > [...] >> Updated patches attached. >> > > The changes look good but I can no longer apply the patches. Can you > please rebase them? > > Sure. -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-98.3-Use-the-dn-attribute-of-LDAPEntry-to-set-get-DNs-of-.patch Type: text/x-patch Size: 17429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-99.3-Preserve-case-of-attribute-names-in-LDAPEntry.patch Type: text/x-patch Size: 9685 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-100.3-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch Type: text/x-patch Size: 5562 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-101.3-Support-attributes-with-multiple-names-in-LDAPEntry.patch Type: text/x-patch Size: 3335 bytes Desc: not available URL: From pviktori at redhat.com Wed Feb 6 10:47:45 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 06 Feb 2013 11:47:45 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51119947.9050707@redhat.com> References: <51119947.9050707@redhat.com> Message-ID: <511234D1.6080605@redhat.com> On 02/06/2013 12:44 AM, Rob Crittenden wrote: > This adds a cert-find command for the dogtag backend. > > Searches can be done by serial number, by subject, revocation reason, > issue date, notbefore, notafter and revocation dates. > > I added some basic tests for this. I made it a separate test file > because the cert plugin tests do not use the declarative format and rely > on the selfsign backend by default. > > rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. The API.txt check fails: Option sizelimit? of command cert_find in ipalib, not in API file: Int('sizelimit?', default=100, minvalue=0) What are --all and --raw for? Is the plan to implement --all if/when Dogtag supports requesting additional data? The format of --validnotbefore-to and friends should be mentioned in --help text; the following is confusing: $ ipa cert-show 1 [...] Not Before: Wed Feb 06 09:32:17 2013 UTC [...] $ ipa cert-find -h [...] --validnotbefore-to=STR Valid not before to this date [...] $ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC' ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 2013 UTC' does not match format '%Y-%m-%d' Could you make the help text for --exactly more specific? Please remove the extra whitespace at the end of dogtag.py I'd welcome a link to the design page in the commit message. -- Petr? From mkosek at redhat.com Wed Feb 6 11:23:32 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 06 Feb 2013 12:23:32 +0100 Subject: [Freeipa-devel] [PATCH] 0176 Don't add another nsDS5ReplicaId on updates if one already exists In-Reply-To: <51110B4D.6070802@redhat.com> References: <5110F6AB.7060200@redhat.com> <51110B4D.6070802@redhat.com> Message-ID: <51123D34.2080604@redhat.com> On 02/05/2013 02:38 PM, Martin Kosek wrote: > On 02/05/2013 01:10 PM, Petr Viktorin wrote: >> Hello, >> In the LDAP refactor I made getting single-valued attributes[1] ensure that >> there is really only one value. Previously we just used the first one. >> This exposed a bug in replica installation: see >> https://fedorahosted.org/freeipa/ticket/3394. Fix attached. >> >> It seems that 389 happened to return the larger value first and the "3" second, >> so we didn't notice the problem. Still, should be fixed in all branches. >> >> >> >> Hopefully all regressions caused by the refactor are of this kind :) >> >> [1] For those who are following: getEntry() in old code, single_value() after >> my patches >> > > ACK, thanks for catching this. > > Martin > Pushed to master, ipa-3-1. Martin From rcritten at redhat.com Wed Feb 6 15:12:49 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 06 Feb 2013 10:12:49 -0500 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <511226EE.8060703@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> <511139FB.3050102@redhat.com> <511226EE.8060703@redhat.com> Message-ID: <511272F1.2030108@redhat.com> Martin Kosek wrote: > On 02/05/2013 05:57 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 02/04/2013 05:59 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> When ipa-client-install is run without --server option, it tries to >>>>> search SRV records for IPA/LDAP server hostname, but it returns only >>>>> the first record found and when the LDAP server on that hostname is >>>>> not available, the whole client installation fails. >>>>> >>>>> Get all LDAP SRV records instead and fallback to next hostname when >>>>> the current one is not available. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/3388 >>>> >>>> I worked on the same ticket, unfortunately, but I didn't mark it as assigned >>>> which caused some duplicate effort. Sorry about that. >>>> >>>> I came up with a very similar solution but took it a bit further. This expands >>>> the treatment of the discovered servers as a list of servers rather than a >>>> single value. >>>> >>>> I do a bit more aggressive testing of all servers returned and remove any from >>>> the list that are not IPA servers. A server not responding is left in the >>>> configured list. >>>> >>>> rob >>> >>> 1) (minor) If I connected IPA host to other IPA server before next enrollment, >>> there will be outdated /etc/ipa/ca.crt. This will cause all tries to connect to >>> IPA server to fail with TLS error, but without giving any clue to user... >> >> Yeah, this can be a problem but I'm going to leave things as-is for now. I >> believe we have a separate ticket to clean this up. I don't want to combine too >> many things into this patch, it is disruptive enough. >> >>> >>> # ipa-client-install >>> Provide your IPA server name (ex: ipa.example.com): >>> >>> He would need to reach out to the log to find this line: >>> LDAP Error: Connect error: TLS error -8054:You are attempting to import a cert >>> with the same issuer/serial as an existing cert, but that is not the same cert. >>> >>> I am thinking if we should not expose some LDAP errors after all? To give some >>> clue to user... >> >> Yeah, I switched the LDAP error unhandled exception back from debug to error. >> >>> >>> 2) (minor) While we are touching these errors I would also fix a typo there :-) >>> ... >>> if isinstance(err, ldap.INAPPROPRIATE_AUTH): >>> root_logger.debug("LDAP Error: Anonymous acces not allowed") >>> return [NO_ACCESS_TO_LDAP] ^^^^^ >>> ... >> >> Heh, ok. >> >> >>> >>> 3) (Regression) You neither set ds.server nor add server to valid_servers when >>> anonymous access is not enabled. The installer then does not try to connect to >>> this server even though the installation would succeed: >>> >>> ... >>> elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == NO_TLS_LDAP: >>> ldapaccess = False >>> ... >> >> Good point. The handling for this is done a bit later so I need to defer a >> little processing but it should work now. >> >>> >>> 4) (Regression) Client will now report success in discovery even when the >>> server is down: >>> >>> # ipa-client-install >>> Unable to verify that vm-037.idm.lab.bos.redhat.com (realm >>> IDM.LAB.BOS.REDHAT.COM) is an IPA server >>> Discovery was successful! >>> Hostname: vm-148.idm.lab.bos.redhat.com >>> Realm: IDM.LAB.BOS.REDHAT.COM >>> DNS Domain: idm.lab.bos.redhat.com >>> IPA Server: vm-037.idm.lab.bos.redhat.com >>> BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com >>> >>> Continue to configure the system with these values? [no]: y >>> User authorized to enroll computers: admin >>> Synchronizing time with KDC... >>> Password for admin at IDM.LAB.BOS.REDHAT.COM: >>> Kerberos authentication failed >>> kinit: Generic error (see e-text) while getting initial credentials >>> >>> Please make sure the following ports are opened in the firewall settings: >>> TCP: 80, 88, 389 >>> UDP: 88 (at least one of TCP/UDP ports 88 has to be open) >>> Also note that following ports are necessary for ipa-client working properly >>> after enrollment: >>> TCP: 464 >>> UDP: 464, 123 (if NTP enabled) >>> Installation failed. Rolling back changes. >>> IPA client is not configured on this system. >>> >>> >>> LDAP on vm-037 in this case is down. I think this would cause a regression too, >>> because previously we simply reported that no discovered server is available >>> and let user enter the server manually. >> >> Fixed. >> >>> >>> IMO we are trying to be too smart when processing the (discovered) servers. Why >>> do we need to process and verify _all_ discovered servers even when the list is >>> not written anywhere in case of SRV lookup? I think it causes unnecessary >>> traffic on network - we should connect to the first server available. >> >> That's a good point. Now we except on the first discovered server. >> >> I think for user-provided servers we still want to verify them all since they >> will be hardcoded in a config file. >> >>> 5) In ipa-client-automount: >>> >>> + # Now confirm that our server is an IPA server. Stop checking on the first >>> + # success so we know we have at least one good one. >>> + for server in servers: >>> + root_logger.debug("Verifying that %s is an IPA server" % server) >>> + ldapret = ds.ipacheckldap(server, api.env.realm) >>> >>> >>> In case of successful autodiscovery, this looks redundant as we try to verify >>> the servers second time even though we already did it with ds.search and >>> ds.server should already contain a functional server. >> >> That's true, I ripped all this out. >> >> rob >> > > 1) One whitespace error: > > git apply /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch > /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch:96: trailing whitespace. > > warning: 1 line adds whitespace errors. > > 2) When ipa-client-automount --server option is passed, the --server value is > then never user. This leads to installation failure when --no-sssd and --server > options are used: > > ... > Raised exception local variable 'server' referenced before assignment > ... > > > ipa-client-install looks OK so far, I did not find any issue in my tests. > > Martin > Fixed rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1084-3-client-failover.patch Type: text/x-diff Size: 17431 bytes Desc: not available URL: From mkosek at redhat.com Wed Feb 6 16:32:29 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 06 Feb 2013 17:32:29 +0100 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <511272F1.2030108@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> <511139FB.3050102@redhat.com> <511226EE.8060703@redhat.com> <511272F1.2030108@redhat.com> Message-ID: <5112859D.8020709@redhat.com> On 02/06/2013 04:12 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 02/05/2013 05:57 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 02/04/2013 05:59 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>>>> When ipa-client-install is run without --server option, it tries to >>>>>> search SRV records for IPA/LDAP server hostname, but it returns only >>>>>> the first record found and when the LDAP server on that hostname is >>>>>> not available, the whole client installation fails. >>>>>> >>>>>> Get all LDAP SRV records instead and fallback to next hostname when >>>>>> the current one is not available. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/3388 >>>>> >>>>> I worked on the same ticket, unfortunately, but I didn't mark it as assigned >>>>> which caused some duplicate effort. Sorry about that. >>>>> >>>>> I came up with a very similar solution but took it a bit further. This >>>>> expands >>>>> the treatment of the discovered servers as a list of servers rather than a >>>>> single value. >>>>> >>>>> I do a bit more aggressive testing of all servers returned and remove any >>>>> from >>>>> the list that are not IPA servers. A server not responding is left in the >>>>> configured list. >>>>> >>>>> rob >>>> >>>> 1) (minor) If I connected IPA host to other IPA server before next enrollment, >>>> there will be outdated /etc/ipa/ca.crt. This will cause all tries to >>>> connect to >>>> IPA server to fail with TLS error, but without giving any clue to user... >>> >>> Yeah, this can be a problem but I'm going to leave things as-is for now. I >>> believe we have a separate ticket to clean this up. I don't want to combine too >>> many things into this patch, it is disruptive enough. >>> >>>> >>>> # ipa-client-install >>>> Provide your IPA server name (ex: ipa.example.com): >>>> >>>> He would need to reach out to the log to find this line: >>>> LDAP Error: Connect error: TLS error -8054:You are attempting to import a cert >>>> with the same issuer/serial as an existing cert, but that is not the same >>>> cert. >>>> >>>> I am thinking if we should not expose some LDAP errors after all? To give some >>>> clue to user... >>> >>> Yeah, I switched the LDAP error unhandled exception back from debug to error. >>> >>>> >>>> 2) (minor) While we are touching these errors I would also fix a typo there >>>> :-) >>>> ... >>>> if isinstance(err, ldap.INAPPROPRIATE_AUTH): >>>> root_logger.debug("LDAP Error: Anonymous acces not allowed") >>>> return [NO_ACCESS_TO_LDAP] ^^^^^ >>>> ... >>> >>> Heh, ok. >>> >>> >>>> >>>> 3) (Regression) You neither set ds.server nor add server to valid_servers when >>>> anonymous access is not enabled. The installer then does not try to connect to >>>> this server even though the installation would succeed: >>>> >>>> ... >>>> elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == >>>> NO_TLS_LDAP: >>>> ldapaccess = False >>>> ... >>> >>> Good point. The handling for this is done a bit later so I need to defer a >>> little processing but it should work now. >>> >>>> >>>> 4) (Regression) Client will now report success in discovery even when the >>>> server is down: >>>> >>>> # ipa-client-install >>>> Unable to verify that vm-037.idm.lab.bos.redhat.com (realm >>>> IDM.LAB.BOS.REDHAT.COM) is an IPA server >>>> Discovery was successful! >>>> Hostname: vm-148.idm.lab.bos.redhat.com >>>> Realm: IDM.LAB.BOS.REDHAT.COM >>>> DNS Domain: idm.lab.bos.redhat.com >>>> IPA Server: vm-037.idm.lab.bos.redhat.com >>>> BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com >>>> >>>> Continue to configure the system with these values? [no]: y >>>> User authorized to enroll computers: admin >>>> Synchronizing time with KDC... >>>> Password for admin at IDM.LAB.BOS.REDHAT.COM: >>>> Kerberos authentication failed >>>> kinit: Generic error (see e-text) while getting initial credentials >>>> >>>> Please make sure the following ports are opened in the firewall settings: >>>> TCP: 80, 88, 389 >>>> UDP: 88 (at least one of TCP/UDP ports 88 has to be open) >>>> Also note that following ports are necessary for ipa-client working properly >>>> after enrollment: >>>> TCP: 464 >>>> UDP: 464, 123 (if NTP enabled) >>>> Installation failed. Rolling back changes. >>>> IPA client is not configured on this system. >>>> >>>> >>>> LDAP on vm-037 in this case is down. I think this would cause a regression >>>> too, >>>> because previously we simply reported that no discovered server is available >>>> and let user enter the server manually. >>> >>> Fixed. >>> >>>> >>>> IMO we are trying to be too smart when processing the (discovered) servers. >>>> Why >>>> do we need to process and verify _all_ discovered servers even when the >>>> list is >>>> not written anywhere in case of SRV lookup? I think it causes unnecessary >>>> traffic on network - we should connect to the first server available. >>> >>> That's a good point. Now we except on the first discovered server. >>> >>> I think for user-provided servers we still want to verify them all since they >>> will be hardcoded in a config file. >>> >>>> 5) In ipa-client-automount: >>>> >>>> + # Now confirm that our server is an IPA server. Stop checking on the >>>> first >>>> + # success so we know we have at least one good one. >>>> + for server in servers: >>>> + root_logger.debug("Verifying that %s is an IPA server" % server) >>>> + ldapret = ds.ipacheckldap(server, api.env.realm) >>>> >>>> >>>> In case of successful autodiscovery, this looks redundant as we try to verify >>>> the servers second time even though we already did it with ds.search and >>>> ds.server should already contain a functional server. >>> >>> That's true, I ripped all this out. >>> >>> rob >>> >> >> 1) One whitespace error: >> >> git apply /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch >> /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch:96: trailing whitespace. >> >> warning: 1 line adds whitespace errors. >> >> 2) When ipa-client-automount --server option is passed, the --server value is >> then never user. This leads to installation failure when --no-sssd and --server >> options are used: >> >> ... >> Raised exception local variable 'server' referenced before assignment >> ... >> >> >> ipa-client-install looks OK so far, I did not find any issue in my tests. >> >> Martin >> > > Fixed > > rob Looks good. Just one more remark: 1) When --server has anonymous search disabled (i.e. ret==ipadiscovery.NO_ACCESS_TO_LDAP), ipa-client-automount just exits: # ipa-client-automount --server vm-024.idm.lab.bos.redhat.com Unable to confirm that vm-024.idm.lab.bos.redhat.com is an IPA server When I do autodiscovery, the exact same server is used and installation succeeds: # ipa-client-automount Searching for IPA server... IPA server: DNS discovery Location: default Continue to configure the system with these values? [no]: y Configured /etc/nsswitch.conf Configured /etc/sysconfig/nfs Configured /etc/idmapd.conf Started rpcidmapd Started rpcgssd Restarting sssd, waiting for it to become available. Started autofs Martin From mkosek at redhat.com Wed Feb 6 16:38:28 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 06 Feb 2013 17:38:28 +0100 Subject: [Freeipa-devel] [RFE] Configurable SID Blacklists In-Reply-To: <51113293.6010605@redhat.com> References: <51113293.6010605@redhat.com> Message-ID: <51128704.40402@redhat.com> On 02/05/2013 05:25 PM, Martin Kosek wrote: > Hello, > > below is a design page for ticket https://fedorahosted.org/freeipa/ticket/3289: > > http://www.freeipa.org/page/V3/Configurable_SID_Blacklists > > There is one question in the text. > > Martin > I had to refactor "ipa-kdb Changes" section as I realized that ipadb_mspac structure is global and we want these list per-trust. Martin From akrivoka at redhat.com Wed Feb 6 17:27:26 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Wed, 06 Feb 2013 18:27:26 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains Message-ID: <5112927E.3080700@redhat.com> Hello, Below is a design page for ticket: https://fedorahosted.org/freeipa/ticket/2945. There are a couple of questions in the text. Thoughts, comments welcome! http://www.freeipa.org/page/V3/Realm_Domains -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. From tbabej at redhat.com Wed Feb 6 17:37:47 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 06 Feb 2013 18:37:47 +0100 Subject: [Freeipa-devel] [PATCHES 0031-0032] Improve HBAC rule handling in selinuxusermap-add/mod/find Message-ID: <511294EB.9050805@redhat.com> Hi, this pair of patches improves HBAC rule handling in selinuxusermap commands. Patch 0031 deals with: https://fedorahosted.org/freeipa/ticket/3349 Patch 0032 takes care of: https://fedorahosted.org/freeipa/ticket/3348 and is to be applied on top of Patch 0031. See commit messages for detailed info. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0031-Improve-HBAC-rule-handling-in-selinuxusermap-add-mod.patch Type: text/x-patch Size: 34418 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0032-Catch-TypeError-when-processing-attributes-into-dict.patch Type: text/x-patch Size: 2113 bytes Desc: not available URL: From rcritten at redhat.com Wed Feb 6 18:23:20 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 06 Feb 2013 13:23:20 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511234D1.6080605@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> Message-ID: <51129F98.7060706@redhat.com> Petr Viktorin wrote: > On 02/06/2013 12:44 AM, Rob Crittenden wrote: >> This adds a cert-find command for the dogtag backend. >> >> Searches can be done by serial number, by subject, revocation reason, >> issue date, notbefore, notafter and revocation dates. >> >> I added some basic tests for this. I made it a separate test file >> because the cert plugin tests do not use the declarative format and rely >> on the selfsign backend by default. >> >> rob > > Thanks! The code works well, but I found a few issues. > > > These tests don't work when the full test suite is run: test_cert adds > and revokes additional certs that throw the code off. > Perhaps have the tests only query valid certs? I don't see that option > but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. > > > The API.txt check fails: > Option sizelimit? of command cert_find in ipalib, not in API file: > > > Int('sizelimit?', default=100, minvalue=0) Ouch. I thought I had fixed that, obviously not. Done now. > > What are --all and --raw for? Is the plan to implement --all if/when > Dogtag supports requesting additional data? Correct, they don't do anything at the moment. I have an RFE open to return additional data from certs. Once that is done then all will make sense. I don't know that raw will ever do anything interesting here but it comes with all commands. > > The format of --validnotbefore-to and friends should be mentioned in > --help text; the following is confusing: > $ ipa cert-show 1 > [...] > Not Before: Wed Feb 06 09:32:17 2013 UTC > [...] > $ ipa cert-find -h > [...] > --validnotbefore-to=STR > Valid not before to this date > [...] > $ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC' > ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 > 2013 UTC' does not match format '%Y-%m-%d' It was listed in the top block but I added it to the usage help as well for clarity. > Could you make the help text for --exactly more specific? Done. > Please remove the extra whitespace at the end of dogtag.py > > I'd welcome a link to the design page in the commit message. > both done rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1085-2-cert-find.patch Type: text/x-diff Size: 33683 bytes Desc: not available URL: From rcritten at redhat.com Wed Feb 6 18:57:27 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 06 Feb 2013 13:57:27 -0500 Subject: [Freeipa-devel] [PATCHES 0031-0032] Improve HBAC rule handling in selinuxusermap-add/mod/find In-Reply-To: <511294EB.9050805@redhat.com> References: <511294EB.9050805@redhat.com> Message-ID: <5112A797.805@redhat.com> Tomas Babej wrote: > Hi, > > this pair of patches improves HBAC rule handling in selinuxusermap > commands. > > Patch 0031 deals with: > https://fedorahosted.org/freeipa/ticket/3349 > > Patch 0032 takes care of: > https://fedorahosted.org/freeipa/ticket/3348 > > and is to be applied on top of Patch 0031. > > See commit messages for detailed info. > > Tomas > ACK for patch 0032. For patch 0031 we can't change the data type of an existing attribute. It will break backwards compatibility. Can you test with an older client to see if it cares (it may not care about the name of the type). If older clients will work then this is probably ok. I gather that seealso detected as a DN attribute and converted into a DN class and this is blowing up the Str validator? rob From rcritten at redhat.com Wed Feb 6 21:21:54 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 06 Feb 2013 16:21:54 -0500 Subject: [Freeipa-devel] [PATCH] 0004 Take into consideration services when deleting replicas In-Reply-To: <5101741D.5090406@redhat.com> References: <51014FA2.9020503@redhat.com> <5101506F.5000108@redhat.com> <5101741D.5090406@redhat.com> Message-ID: <5112C972.20105@redhat.com> Ana Krivokapic wrote: > On 01/24/2013 04:17 PM, Rob Crittenden wrote: >> Ana Krivokapic wrote: >>> When deleting a replica from IPA domain, warn if the installation is >>> left without CA and/or DNS. >>> >>> Ticket: https://fedorahosted.org/freeipa/ticket/2879 >> >> IMHO we should not give an option to delete the last CA. >> >> DNS can be more easily re-added if needed. >> >> Should we not ask this if the --force flag is set? >> >> rob >> > > Makes sense, thanks Rob. > > I have changed the behaviour to: > * abort the deletion if the last CA is about to be deleted > * warn if the last DNS is about to be deleted (but only require user > confirmation if the --force flag is not set) > > Updated patch is attached. > Looks good. I removed the comma in the sentence about the CA then pushed to master and ipa-3-1. rob From rcritten at redhat.com Wed Feb 6 22:00:39 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 06 Feb 2013 17:00:39 -0500 Subject: [Freeipa-devel] [PATCHES] 0107-0114 Fix Confusing ipa tool online help organization In-Reply-To: <510FA8D3.4000502@redhat.com> References: <50C9F27E.50808@redhat.com> <50CA76D6.8090404@redhat.com> <50CB0DCF.7050102@redhat.com> <510AB97F.2080608@redhat.com> <510BF0F7.8020102@redhat.com> <510BF629.4040700@redhat.com> <510FA8D3.4000502@redhat.com> Message-ID: <5112D287.7010905@redhat.com> Petr Viktorin wrote: > On 02/01/2013 06:06 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 01/31/2013 07:35 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> On 12/14/2012 01:46 AM, Dmitri Pal wrote: >>>>>> On 12/13/2012 10:21 AM, Petr Viktorin wrote: >>>>>>> https://fedorahosted.org/freeipa/ticket/3060 >>>>>>> >>>>>>> Here is a collection of smallish fixes to `ipa help` and `ipa >>>>>>> --help`. >>>>>>> This should address most of Nikolai's proposal. >>>>>>> Additionally, it's now possible to run `ipa --help` >>>>>>> without >>>>>>> a Kerberos ticket. And there are some new tests. >>>>>>> >>>>>>> I've not included the "Often used commands" in `ipa help`; I think >>>>>>> that is material for a manual/tutorial, not a help command. >>>>>>> Selecting >>>>>>> a topic from `ipa topics` and then choosing a command from `ipa help >>>>>>> ` is a better way to use the help than the verbose `ipa help >>>>>>> commands` or proposed incomplete "Often used commands". >>>>>> >>>>>> Since the ticket has a bit of discussion and you indicate that you >>>>>> did >>>>>> not to address everything can you please extract what have been >>>>>> addressed and put it into a design page. >>>>>> I know it is not RFE but it would help to validate the changes by >>>>>> testers. >>>>>> Please put the wiki link into the ticket. >>>>>> >>>>> >>>>> http://freeipa.org/page/V3/Help >>>>> >>>>> >>>> >>>> What is the purpose of the no-option outfile? Do you anticipate at some >>>> point opening this up as a real option or making it easier to log while >>>> using the api directly? >>> >>> On incorrect invocation (`ipa`, `ipa TOPIC`), the help command is called >>> internally with outfile=sys.stderr. >> >> That's true, but this is a lot of code to abstract writing to >> sys.stderr. I'm just trying to understand the reasoning. Do you imagine >> that some time in the future we'd want to control where the output goes? > > I don't think that would be useful, I can't imagine why someone would > want to log help texts, or use them to script something. > Do you have another idea of how this should be done? > >>>> The help for help is a little confusing: >>>> >>>> ----- >>>> Purpose: Display help for a command or topic. >>>> Usage: ipa [global-options] help [TOPIC] [options] >>>> >>>> Positional arguments: >>>> TOPIC The topic or command name. >>>> >>>> Options: >>>> -h, --help show this help message and exit >>>> ----- >>>> >>>> Should [TOPIC] be [TOPIC | COMMAND] or something else? >>> >>> I'm trying to discourage `ipa help COMMAND` in favor of `ipa COMMAND >>> --help`, that way you get the proper help for ambiguous words like >>> "ping" (https://fedorahosted.org/freeipa/ticket/3247) >>> >>> I also wanted to keep the help simple and not explain this unimportant >>> detail here. >>> >>> If you have better wording I can of course change it. >> >> Your reasoning is sound. I"m ok with gently pushing users in that >> direction but leaving undocumented the old behavior. Should we create a >> ticket to eventually return an error in that case? > > Users will expect `ipa help COMMAND` to get them command help, it's > pretty standard in tools with subcommands. If it was a part of the API > I'd be all for enforcing proper usage, but I think a help command > deserves some slack -- it's not that big a deal if you get topic help > for ping instead of command help... > Hm. Now I see that I should add a notice to the topic help text, to lead > users to the right path. Patch attached. > > We will want to remove `ipa help COMMAND` from the docs, and note that > "ipa help" favors topics. > We can turn https://fedorahosted.org/freeipa/ticket/3247 into a doc ticket. > > >>>> On my fresh F-18 install one of the new unit tests fails: >>>> >>>> ====================================================================== >>>> FAIL: Test that `help user-add` & `user-add -h` are equivalent and >>>> contain doc >>>> ---------------------------------------------------------------------- >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>>> runTest >>>> self.test(*self.arg) >>>> File "/home/rcrit/redhat/freeipa/tests/test_cmdline/test_help.py", >>>> line 111, in test_command_help >>>> assert h_ctx.stdout == help_ctx.stdout >>>> AssertionError >>> >>> This is weird, it works for me. Could you send me the two outputs so I >>> can get some idea what's wrong? >>> > > Can you check you didn't leave out patch 111 (Add command summary?)? > > Yeah, I missed 110 and 111. Tests are passing now. I still don't understand the purpose of outfile. What do we gain by making this configurable and who or what would ever change it? rob From pvoborni at redhat.com Thu Feb 7 09:55:28 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 07 Feb 2013 10:55:28 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <5112927E.3080700@redhat.com> References: <5112927E.3080700@redhat.com> Message-ID: <51137A10.2040908@redhat.com> On 02/06/2013 06:27 PM, Ana Krivokapic wrote: > Hello, > > Below is a design page for ticket: > https://fedorahosted.org/freeipa/ticket/2945. > > There are a couple of questions in the text. > > Thoughts, comments welcome! > > http://www.freeipa.org/page/V3/Realm_Domains > Hello, I don't think we should make special commands for adding/removing of a single value of multivalued attribute. We should use similar concept as in server config or dns config. IMO the commands should be: realmdomains-show realmdomains-mod --associatedDomain={defaultdomain,domain1,domain2,..} we might introduced --add-domain and --del-domain options for adding/removing of a single value. IIRC we don't have a precedence for such options, but we've discussed it before [1]. Regarding WEB UI: Identity section is OK. > ipa host-add command can be hooked to realmdomains-add-domain, to automatically > add domain of added host to the list of domains associated with IPA realm Web UI uses DNS zones list for host-add operation. Does the proposal mean that it will have to be changed to use realmdomain list? I'm not sure which way is better. [1] https://fedorahosted.org/freeipa/ticket/3054 -- Petr Vobornik From pvoborni at redhat.com Thu Feb 7 11:32:59 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 07 Feb 2013 12:32:59 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51119947.9050707@redhat.com> References: <51119947.9050707@redhat.com> Message-ID: <511390EB.40701@redhat.com> On 02/06/2013 12:44 AM, Rob Crittenden wrote: > This adds a cert-find command for the dogtag backend. > > Searches can be done by serial number, by subject, revocation reason, > issue date, notbefore, notafter and revocation dates. > > I added some basic tests for this. I made it a separate test file > because the cert plugin tests do not use the declarative format and rely > on the selfsign backend by default. > > rob > Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 -- Petr Vobornik From sbose at redhat.com Thu Feb 7 11:48:23 2013 From: sbose at redhat.com (Sumit Bose) Date: Thu, 7 Feb 2013 12:48:23 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <51137A10.2040908@redhat.com> References: <5112927E.3080700@redhat.com> <51137A10.2040908@redhat.com> Message-ID: <20130207114823.GC2237@localhost.localdomain> On Thu, Feb 07, 2013 at 10:55:28AM +0100, Petr Vobornik wrote: > On 02/06/2013 06:27 PM, Ana Krivokapic wrote: > >Hello, > > > >Below is a design page for ticket: > >https://fedorahosted.org/freeipa/ticket/2945. > > > >There are a couple of questions in the text. > > > >Thoughts, comments welcome! > > > >http://www.freeipa.org/page/V3/Realm_Domains > > > > Hello, > > I don't think we should make special commands for adding/removing of > a single value of multivalued attribute. We should use similar > concept as in server config or dns config. > > IMO the commands should be: > > realmdomains-show > realmdomains-mod --associatedDomain={defaultdomain,domain1,domain2,..} > > we might introduced --add-domain and --del-domain options for > adding/removing of a single value. IIRC we don't have a precedence +1, I think having only --associatedDomain is cumbersome and error-prone. > for such options, but we've discussed it before [1]. > > > Regarding WEB UI: > Identity section is OK. My first guess was 'IPA Server' but I won't mind if it is put under Identity. > > >ipa host-add command can be hooked to realmdomains-add-domain, to automatically > > add domain of added host to the list of domains associated with > IPA realm > > Web UI uses DNS zones list for host-add operation. Does the proposal > mean that it will have to be changed to use realmdomain list? I'm > not sure which way is better. Does this mean you cannot add hosts managed by a different DNS server or is it just a convenience option so that the domain name must not always be specified? > > [1] https://fedorahosted.org/freeipa/ticket/3054 > -- > Petr Vobornik > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From pvoborni at redhat.com Thu Feb 7 11:55:00 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 07 Feb 2013 12:55:00 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <20130207114823.GC2237@localhost.localdomain> References: <5112927E.3080700@redhat.com> <51137A10.2040908@redhat.com> <20130207114823.GC2237@localhost.localdomain> Message-ID: <51139614.9070707@redhat.com> On 02/07/2013 12:48 PM, Sumit Bose wrote: > On Thu, Feb 07, 2013 at 10:55:28AM +0100, Petr Vobornik wrote: >> On 02/06/2013 06:27 PM, Ana Krivokapic wrote: >> >>> ipa host-add command can be hooked to realmdomains-add-domain, to automatically >>> add domain of added host to the list of domains associated with >> IPA realm >> >> Web UI uses DNS zones list for host-add operation. Does the proposal >> mean that it will have to be changed to use realmdomain list? I'm >> not sure which way is better. > > Does this mean you cannot add hosts managed by a different DNS server or > is it just a convenience option so that the domain name must not always > be specified? It just for convenience. You have to specify domain name, but you are not limited to current dns zones. -- Petr Vobornik From sbose at redhat.com Thu Feb 7 12:38:11 2013 From: sbose at redhat.com (Sumit Bose) Date: Thu, 7 Feb 2013 13:38:11 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <5112927E.3080700@redhat.com> References: <5112927E.3080700@redhat.com> Message-ID: <20130207123811.GD2237@localhost.localdomain> On Wed, Feb 06, 2013 at 06:27:26PM +0100, Ana Krivokapic wrote: > Hello, > > Below is a design page for ticket: > https://fedorahosted.org/freeipa/ticket/2945. > > There are a couple of questions in the text. about 'Do we also need to check if the domain is accessible through DNS?' I think it would be good to print a warning that no SOA or NS record was found for the domain. But I think there might be cases where the domain is added to the realmdomains first and then the DNS zone is created. So my suggestion would be either - not fail and print a warning or - fail but allow to skip the check with a --force option. I think you should discuss in 'Updates and Upgrades' if and how cn=Realm Domains,cn=ipa,cn=etc,$SUFFIX is created during updates. bye, Sumit > > Thoughts, comments welcome! > > http://www.freeipa.org/page/V3/Realm_Domains > > -- > Regards, > > Ana Krivokapic > Associate Software Engineer > FreeIPA team > Red Hat Inc. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From pspacek at redhat.com Thu Feb 7 12:57:18 2013 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 07 Feb 2013 13:57:18 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <20130207123811.GD2237@localhost.localdomain> References: <5112927E.3080700@redhat.com> <20130207123811.GD2237@localhost.localdomain> Message-ID: <5113A4AE.1090109@redhat.com> On 7.2.2013 13:38, Sumit Bose wrote: > On Wed, Feb 06, 2013 at 06:27:26PM +0100, Ana Krivokapic wrote: >> Hello, >> >> Below is a design page for ticket: >> https://fedorahosted.org/freeipa/ticket/2945. >> >> There are a couple of questions in the text. > > about 'Do we also need to check if the domain is accessible through > DNS?' I think it would be good to print a warning that no SOA or NS > record was found for the domain. But I think there might be cases where > the domain is added to the realmdomains first and then the DNS zone is > created. So my suggestion would be either > - not fail and print a warning or > - fail but allow to skip the check with a --force option. +1 for --force option I added questions about interaction with "ipa dnszone-add" to design document: http://www.freeipa.org/page/V3/Realm_Domains Should dnszone-del delete associatedDomain when whole DNS zone is being deleted? Should dnszone-add offer an option to create associatedDomain attribute for the new zone? Petr^2 Spacek > I think you should discuss in 'Updates and Upgrades' if and how cn=Realm > Domains,cn=ipa,cn=etc,$SUFFIX is created during updates. > > bye, > Sumit >> >> Thoughts, comments welcome! >> >> http://www.freeipa.org/page/V3/Realm_Domains -- Petr^2 Spacek From sbose at redhat.com Thu Feb 7 13:38:32 2013 From: sbose at redhat.com (Sumit Bose) Date: Thu, 7 Feb 2013 14:38:32 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <5113A4AE.1090109@redhat.com> References: <5112927E.3080700@redhat.com> <20130207123811.GD2237@localhost.localdomain> <5113A4AE.1090109@redhat.com> Message-ID: <20130207133832.GE2237@localhost.localdomain> On Thu, Feb 07, 2013 at 01:57:18PM +0100, Petr Spacek wrote: > On 7.2.2013 13:38, Sumit Bose wrote: > >On Wed, Feb 06, 2013 at 06:27:26PM +0100, Ana Krivokapic wrote: > >>Hello, > >> > >>Below is a design page for ticket: > >>https://fedorahosted.org/freeipa/ticket/2945. > >> > >>There are a couple of questions in the text. > > > >about 'Do we also need to check if the domain is accessible through > >DNS?' I think it would be good to print a warning that no SOA or NS > >record was found for the domain. But I think there might be cases where > >the domain is added to the realmdomains first and then the DNS zone is > >created. So my suggestion would be either > >- not fail and print a warning or > >- fail but allow to skip the check with a --force option. > +1 for --force option > > I added questions about interaction with "ipa dnszone-add" to design document: > http://www.freeipa.org/page/V3/Realm_Domains > > Should dnszone-del delete associatedDomain when whole DNS zone is being deleted? I think no, because the related host and service objects will still be available. E.g. the zone will be deleted because it will be managed by a different DNS server of the hosts are still in IPA. > > Should dnszone-add offer an option to create associatedDomain > attribute for the new zone? yes, that would be useful. Although I think the hook suggested by Ana during 'ipa host-add' is good, because at this stage the domain is really used in the sense that there is a Kerberos principal with the domain in it. bye, Sumit > > Petr^2 Spacek > > >I think you should discuss in 'Updates and Upgrades' if and how cn=Realm > >Domains,cn=ipa,cn=etc,$SUFFIX is created during updates. > > > >bye, > >Sumit > >> > >>Thoughts, comments welcome! > >> > >>http://www.freeipa.org/page/V3/Realm_Domains > > -- > Petr^2 Spacek > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From rcritten at redhat.com Thu Feb 7 14:08:27 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 07 Feb 2013 09:08:27 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511390EB.40701@redhat.com> References: <51119947.9050707@redhat.com> <511390EB.40701@redhat.com> Message-ID: <5113B55B.7030605@redhat.com> Petr Vobornik wrote: > On 02/06/2013 12:44 AM, Rob Crittenden wrote: >> This adds a cert-find command for the dogtag backend. >> >> Searches can be done by serial number, by subject, revocation reason, >> issue date, notbefore, notafter and revocation dates. >> >> I added some basic tests for this. I made it a separate test file >> because the cert plugin tests do not use the declarative format and rely >> on the selfsign backend by default. >> >> rob >> > > Should I create Web UI in scope of this ticket or a new one? > > I was also thinking if it's time to implement #191 'Web UI: specify > fields to search on' [1]. Maybe in Pilsner. > > [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob From rcritten at redhat.com Thu Feb 7 15:03:54 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 07 Feb 2013 10:03:54 -0500 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <5112859D.8020709@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> <511139FB.3050102@redhat.com> <511226EE.8060703@redhat.com> <511272F1.2030108@redhat.com> <5112859D.8020709@redhat.com> Message-ID: <5113C25A.6010001@redhat.com> Martin Kosek wrote: > On 02/06/2013 04:12 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 02/05/2013 05:57 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On 02/04/2013 05:59 PM, Rob Crittenden wrote: >>>>>> Martin Kosek wrote: >>>>>>> When ipa-client-install is run without --server option, it tries to >>>>>>> search SRV records for IPA/LDAP server hostname, but it returns only >>>>>>> the first record found and when the LDAP server on that hostname is >>>>>>> not available, the whole client installation fails. >>>>>>> >>>>>>> Get all LDAP SRV records instead and fallback to next hostname when >>>>>>> the current one is not available. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/3388 >>>>>> >>>>>> I worked on the same ticket, unfortunately, but I didn't mark it as assigned >>>>>> which caused some duplicate effort. Sorry about that. >>>>>> >>>>>> I came up with a very similar solution but took it a bit further. This >>>>>> expands >>>>>> the treatment of the discovered servers as a list of servers rather than a >>>>>> single value. >>>>>> >>>>>> I do a bit more aggressive testing of all servers returned and remove any >>>>>> from >>>>>> the list that are not IPA servers. A server not responding is left in the >>>>>> configured list. >>>>>> >>>>>> rob >>>>> >>>>> 1) (minor) If I connected IPA host to other IPA server before next enrollment, >>>>> there will be outdated /etc/ipa/ca.crt. This will cause all tries to >>>>> connect to >>>>> IPA server to fail with TLS error, but without giving any clue to user... >>>> >>>> Yeah, this can be a problem but I'm going to leave things as-is for now. I >>>> believe we have a separate ticket to clean this up. I don't want to combine too >>>> many things into this patch, it is disruptive enough. >>>> >>>>> >>>>> # ipa-client-install >>>>> Provide your IPA server name (ex: ipa.example.com): >>>>> >>>>> He would need to reach out to the log to find this line: >>>>> LDAP Error: Connect error: TLS error -8054:You are attempting to import a cert >>>>> with the same issuer/serial as an existing cert, but that is not the same >>>>> cert. >>>>> >>>>> I am thinking if we should not expose some LDAP errors after all? To give some >>>>> clue to user... >>>> >>>> Yeah, I switched the LDAP error unhandled exception back from debug to error. >>>> >>>>> >>>>> 2) (minor) While we are touching these errors I would also fix a typo there >>>>> :-) >>>>> ... >>>>> if isinstance(err, ldap.INAPPROPRIATE_AUTH): >>>>> root_logger.debug("LDAP Error: Anonymous acces not allowed") >>>>> return [NO_ACCESS_TO_LDAP] ^^^^^ >>>>> ... >>>> >>>> Heh, ok. >>>> >>>> >>>>> >>>>> 3) (Regression) You neither set ds.server nor add server to valid_servers when >>>>> anonymous access is not enabled. The installer then does not try to connect to >>>>> this server even though the installation would succeed: >>>>> >>>>> ... >>>>> elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == >>>>> NO_TLS_LDAP: >>>>> ldapaccess = False >>>>> ... >>>> >>>> Good point. The handling for this is done a bit later so I need to defer a >>>> little processing but it should work now. >>>> >>>>> >>>>> 4) (Regression) Client will now report success in discovery even when the >>>>> server is down: >>>>> >>>>> # ipa-client-install >>>>> Unable to verify that vm-037.idm.lab.bos.redhat.com (realm >>>>> IDM.LAB.BOS.REDHAT.COM) is an IPA server >>>>> Discovery was successful! >>>>> Hostname: vm-148.idm.lab.bos.redhat.com >>>>> Realm: IDM.LAB.BOS.REDHAT.COM >>>>> DNS Domain: idm.lab.bos.redhat.com >>>>> IPA Server: vm-037.idm.lab.bos.redhat.com >>>>> BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com >>>>> >>>>> Continue to configure the system with these values? [no]: y >>>>> User authorized to enroll computers: admin >>>>> Synchronizing time with KDC... >>>>> Password for admin at IDM.LAB.BOS.REDHAT.COM: >>>>> Kerberos authentication failed >>>>> kinit: Generic error (see e-text) while getting initial credentials >>>>> >>>>> Please make sure the following ports are opened in the firewall settings: >>>>> TCP: 80, 88, 389 >>>>> UDP: 88 (at least one of TCP/UDP ports 88 has to be open) >>>>> Also note that following ports are necessary for ipa-client working properly >>>>> after enrollment: >>>>> TCP: 464 >>>>> UDP: 464, 123 (if NTP enabled) >>>>> Installation failed. Rolling back changes. >>>>> IPA client is not configured on this system. >>>>> >>>>> >>>>> LDAP on vm-037 in this case is down. I think this would cause a regression >>>>> too, >>>>> because previously we simply reported that no discovered server is available >>>>> and let user enter the server manually. >>>> >>>> Fixed. >>>> >>>>> >>>>> IMO we are trying to be too smart when processing the (discovered) servers. >>>>> Why >>>>> do we need to process and verify _all_ discovered servers even when the >>>>> list is >>>>> not written anywhere in case of SRV lookup? I think it causes unnecessary >>>>> traffic on network - we should connect to the first server available. >>>> >>>> That's a good point. Now we except on the first discovered server. >>>> >>>> I think for user-provided servers we still want to verify them all since they >>>> will be hardcoded in a config file. >>>> >>>>> 5) In ipa-client-automount: >>>>> >>>>> + # Now confirm that our server is an IPA server. Stop checking on the >>>>> first >>>>> + # success so we know we have at least one good one. >>>>> + for server in servers: >>>>> + root_logger.debug("Verifying that %s is an IPA server" % server) >>>>> + ldapret = ds.ipacheckldap(server, api.env.realm) >>>>> >>>>> >>>>> In case of successful autodiscovery, this looks redundant as we try to verify >>>>> the servers second time even though we already did it with ds.search and >>>>> ds.server should already contain a functional server. >>>> >>>> That's true, I ripped all this out. >>>> >>>> rob >>>> >>> >>> 1) One whitespace error: >>> >>> git apply /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch >>> /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch:96: trailing whitespace. >>> >>> warning: 1 line adds whitespace errors. >>> >>> 2) When ipa-client-automount --server option is passed, the --server value is >>> then never user. This leads to installation failure when --no-sssd and --server >>> options are used: >>> >>> ... >>> Raised exception local variable 'server' referenced before assignment >>> ... >>> >>> >>> ipa-client-install looks OK so far, I did not find any issue in my tests. >>> >>> Martin >>> >> >> Fixed >> >> rob > > Looks good. Just one more remark: > > 1) When --server has anonymous search disabled (i.e. > ret==ipadiscovery.NO_ACCESS_TO_LDAP), ipa-client-automount just exits: > > # ipa-client-automount --server vm-024.idm.lab.bos.redhat.com > Unable to confirm that vm-024.idm.lab.bos.redhat.com is an IPA server > > When I do autodiscovery, the exact same server is used and installation succeeds: > > # ipa-client-automount > Searching for IPA server... > IPA server: DNS discovery > Location: default > Continue to configure the system with these values? [no]: y > Configured /etc/nsswitch.conf > Configured /etc/sysconfig/nfs > Configured /etc/idmapd.conf > Started rpcidmapd > Started rpcgssd > Restarting sssd, waiting for it to become available. > Started autofs > > Martin > Fixed. I think this has existed for a while. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1084-4-client-failover.patch Type: text/x-diff Size: 17724 bytes Desc: not available URL: From mkosek at redhat.com Thu Feb 7 18:13:00 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 07 Feb 2013 19:13:00 +0100 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <5113C25A.6010001@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> <511139FB.3050102@redhat.com> <511226EE.8060703@redhat.com> <511272F1.2030108@redhat.com> <5112859D.8020709@redhat.com> <5113C25A.6010001@redhat.com> Message-ID: <5113EEAC.4080604@redhat.com> On 02/07/2013 04:03 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 02/06/2013 04:12 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 02/05/2013 05:57 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>>>> On 02/04/2013 05:59 PM, Rob Crittenden wrote: >>>>>>> Martin Kosek wrote: >>>>>>>> When ipa-client-install is run without --server option, it tries to >>>>>>>> search SRV records for IPA/LDAP server hostname, but it returns only >>>>>>>> the first record found and when the LDAP server on that hostname is >>>>>>>> not available, the whole client installation fails. >>>>>>>> >>>>>>>> Get all LDAP SRV records instead and fallback to next hostname when >>>>>>>> the current one is not available. >>>>>>>> >>>>>>>> https://fedorahosted.org/freeipa/ticket/3388 >>>>>>> >>>>>>> I worked on the same ticket, unfortunately, but I didn't mark it as >>>>>>> assigned >>>>>>> which caused some duplicate effort. Sorry about that. >>>>>>> >>>>>>> I came up with a very similar solution but took it a bit further. This >>>>>>> expands >>>>>>> the treatment of the discovered servers as a list of servers rather than a >>>>>>> single value. >>>>>>> >>>>>>> I do a bit more aggressive testing of all servers returned and remove any >>>>>>> from >>>>>>> the list that are not IPA servers. A server not responding is left in the >>>>>>> configured list. >>>>>>> >>>>>>> rob >>>>>> >>>>>> 1) (minor) If I connected IPA host to other IPA server before next >>>>>> enrollment, >>>>>> there will be outdated /etc/ipa/ca.crt. This will cause all tries to >>>>>> connect to >>>>>> IPA server to fail with TLS error, but without giving any clue to user... >>>>> >>>>> Yeah, this can be a problem but I'm going to leave things as-is for now. I >>>>> believe we have a separate ticket to clean this up. I don't want to >>>>> combine too >>>>> many things into this patch, it is disruptive enough. >>>>> >>>>>> >>>>>> # ipa-client-install >>>>>> Provide your IPA server name (ex: ipa.example.com): >>>>>> >>>>>> He would need to reach out to the log to find this line: >>>>>> LDAP Error: Connect error: TLS error -8054:You are attempting to import a >>>>>> cert >>>>>> with the same issuer/serial as an existing cert, but that is not the same >>>>>> cert. >>>>>> >>>>>> I am thinking if we should not expose some LDAP errors after all? To give >>>>>> some >>>>>> clue to user... >>>>> >>>>> Yeah, I switched the LDAP error unhandled exception back from debug to error. >>>>> >>>>>> >>>>>> 2) (minor) While we are touching these errors I would also fix a typo there >>>>>> :-) >>>>>> ... >>>>>> if isinstance(err, ldap.INAPPROPRIATE_AUTH): >>>>>> root_logger.debug("LDAP Error: Anonymous acces not >>>>>> allowed") >>>>>> return [NO_ACCESS_TO_LDAP] ^^^^^ >>>>>> ... >>>>> >>>>> Heh, ok. >>>>> >>>>> >>>>>> >>>>>> 3) (Regression) You neither set ds.server nor add server to valid_servers >>>>>> when >>>>>> anonymous access is not enabled. The installer then does not try to >>>>>> connect to >>>>>> this server even though the installation would succeed: >>>>>> >>>>>> ... >>>>>> elif ldapret[0] == NO_ACCESS_TO_LDAP or ldapret[0] == >>>>>> NO_TLS_LDAP: >>>>>> ldapaccess = False >>>>>> ... >>>>> >>>>> Good point. The handling for this is done a bit later so I need to defer a >>>>> little processing but it should work now. >>>>> >>>>>> >>>>>> 4) (Regression) Client will now report success in discovery even when the >>>>>> server is down: >>>>>> >>>>>> # ipa-client-install >>>>>> Unable to verify that vm-037.idm.lab.bos.redhat.com (realm >>>>>> IDM.LAB.BOS.REDHAT.COM) is an IPA server >>>>>> Discovery was successful! >>>>>> Hostname: vm-148.idm.lab.bos.redhat.com >>>>>> Realm: IDM.LAB.BOS.REDHAT.COM >>>>>> DNS Domain: idm.lab.bos.redhat.com >>>>>> IPA Server: vm-037.idm.lab.bos.redhat.com >>>>>> BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com >>>>>> >>>>>> Continue to configure the system with these values? [no]: y >>>>>> User authorized to enroll computers: admin >>>>>> Synchronizing time with KDC... >>>>>> Password for admin at IDM.LAB.BOS.REDHAT.COM: >>>>>> Kerberos authentication failed >>>>>> kinit: Generic error (see e-text) while getting initial credentials >>>>>> >>>>>> Please make sure the following ports are opened in the firewall settings: >>>>>> TCP: 80, 88, 389 >>>>>> UDP: 88 (at least one of TCP/UDP ports 88 has to be open) >>>>>> Also note that following ports are necessary for ipa-client working properly >>>>>> after enrollment: >>>>>> TCP: 464 >>>>>> UDP: 464, 123 (if NTP enabled) >>>>>> Installation failed. Rolling back changes. >>>>>> IPA client is not configured on this system. >>>>>> >>>>>> >>>>>> LDAP on vm-037 in this case is down. I think this would cause a regression >>>>>> too, >>>>>> because previously we simply reported that no discovered server is available >>>>>> and let user enter the server manually. >>>>> >>>>> Fixed. >>>>> >>>>>> >>>>>> IMO we are trying to be too smart when processing the (discovered) servers. >>>>>> Why >>>>>> do we need to process and verify _all_ discovered servers even when the >>>>>> list is >>>>>> not written anywhere in case of SRV lookup? I think it causes unnecessary >>>>>> traffic on network - we should connect to the first server available. >>>>> >>>>> That's a good point. Now we except on the first discovered server. >>>>> >>>>> I think for user-provided servers we still want to verify them all since they >>>>> will be hardcoded in a config file. >>>>> >>>>>> 5) In ipa-client-automount: >>>>>> >>>>>> + # Now confirm that our server is an IPA server. Stop checking on the >>>>>> first >>>>>> + # success so we know we have at least one good one. >>>>>> + for server in servers: >>>>>> + root_logger.debug("Verifying that %s is an IPA server" % server) >>>>>> + ldapret = ds.ipacheckldap(server, api.env.realm) >>>>>> >>>>>> >>>>>> In case of successful autodiscovery, this looks redundant as we try to >>>>>> verify >>>>>> the servers second time even though we already did it with ds.search and >>>>>> ds.server should already contain a functional server. >>>>> >>>>> That's true, I ripped all this out. >>>>> >>>>> rob >>>>> >>>> >>>> 1) One whitespace error: >>>> >>>> git apply /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch >>>> /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch:96: trailing >>>> whitespace. >>>> >>>> warning: 1 line adds whitespace errors. >>>> >>>> 2) When ipa-client-automount --server option is passed, the --server value is >>>> then never user. This leads to installation failure when --no-sssd and >>>> --server >>>> options are used: >>>> >>>> ... >>>> Raised exception local variable 'server' referenced before assignment >>>> ... >>>> >>>> >>>> ipa-client-install looks OK so far, I did not find any issue in my tests. >>>> >>>> Martin >>>> >>> >>> Fixed >>> >>> rob >> >> Looks good. Just one more remark: >> >> 1) When --server has anonymous search disabled (i.e. >> ret==ipadiscovery.NO_ACCESS_TO_LDAP), ipa-client-automount just exits: >> >> # ipa-client-automount --server vm-024.idm.lab.bos.redhat.com >> Unable to confirm that vm-024.idm.lab.bos.redhat.com is an IPA server >> >> When I do autodiscovery, the exact same server is used and installation >> succeeds: >> >> # ipa-client-automount >> Searching for IPA server... >> IPA server: DNS discovery >> Location: default >> Continue to configure the system with these values? [no]: y >> Configured /etc/nsswitch.conf >> Configured /etc/sysconfig/nfs >> Configured /etc/idmapd.conf >> Started rpcidmapd >> Started rpcgssd >> Restarting sssd, waiting for it to become available. >> Started autofs >> >> Martin >> > > Fixed. I think this has existed for a while. > > rob Ok, sorry if I bugged you for existing bugs then :-) I think the patch is OK, I have found just one case when we did not fallback because we always used just one SRV record. This is a diff that fixed the issue for me: diff --git a/ipa-client/ipaclient/ipadiscovery.py b/ipa-client/ipaclient/ipadiscovery.py index 8338329..7fc6aae 100644 --- a/ipa-client/ipaclient/ipadiscovery.py +++ b/ipa-client/ipaclient/ipadiscovery.py @@ -201,7 +201,8 @@ class IPADiscovery(object): return NO_LDAP_SERVER else: root_logger.debug("Search for LDAP SRV record in %s", domain) - servers = self.ipadns_search_srv(domain, '_ldap._tcp', 389) + servers = self.ipadns_search_srv(domain, '_ldap._tcp', 389, + break_on_first=False) if servers: autodiscovered = True self.domain = domain ACK if you squash this patch. Martin From simo at redhat.com Thu Feb 7 21:42:26 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 07 Feb 2013 16:42:26 -0500 Subject: [Freeipa-devel] [PATCH] Add delegation info to MS-PAC Message-ID: <1360273346.2299.15.camel@willson.li.ssimo.org> This information is not strictly required but is part of the MS-PAC specification and I had some time to kill on the plane on my last trip back. I tested it briefly with cross-realm trusts and it seem to work fine. Neither IPA nor AD2012 complained when looking at PACs, do far. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-Delegation-Info-to-MS-PAC.patch Type: text/x-patch Size: 7578 bytes Desc: not available URL: From rcritten at redhat.com Thu Feb 7 21:56:06 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 07 Feb 2013 16:56:06 -0500 Subject: [Freeipa-devel] [PATCH] 362 Add LDAP server fallback to client installer In-Reply-To: <5113EEAC.4080604@redhat.com> References: <510FC097.5050406@redhat.com> <510FE8E6.4070906@redhat.com> <51112131.3080409@redhat.com> <511139FB.3050102@redhat.com> <511226EE.8060703@redhat.com> <511272F1.2030108@redhat.com> <5112859D.8020709@redhat.com> <5113C25A.6010001@redhat.com> <5113EEAC.4080604@redhat.com> Message-ID: <511422F6.1070704@redhat.com> Martin Kosek wrote: > On 02/07/2013 04:03 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 02/06/2013 04:12 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On 02/05/2013 05:57 PM, Rob Crittenden wrote: >>>>>> Martin Kosek wrote: >>>>>>> On 02/04/2013 05:59 PM, Rob Crittenden wrote: >>>>>>>> Martin Kosek wrote: >>>>>>>>> When ipa-client-install is run without --server option, it >>>>>>>>> tries to >>>>>>>>> search SRV records for IPA/LDAP server hostname, but it returns >>>>>>>>> only >>>>>>>>> the first record found and when the LDAP server on that >>>>>>>>> hostname is >>>>>>>>> not available, the whole client installation fails. >>>>>>>>> >>>>>>>>> Get all LDAP SRV records instead and fallback to next hostname >>>>>>>>> when >>>>>>>>> the current one is not available. >>>>>>>>> >>>>>>>>> https://fedorahosted.org/freeipa/ticket/3388 >>>>>>>> >>>>>>>> I worked on the same ticket, unfortunately, but I didn't mark it as >>>>>>>> assigned >>>>>>>> which caused some duplicate effort. Sorry about that. >>>>>>>> >>>>>>>> I came up with a very similar solution but took it a bit >>>>>>>> further. This >>>>>>>> expands >>>>>>>> the treatment of the discovered servers as a list of servers >>>>>>>> rather than a >>>>>>>> single value. >>>>>>>> >>>>>>>> I do a bit more aggressive testing of all servers returned and >>>>>>>> remove any >>>>>>>> from >>>>>>>> the list that are not IPA servers. A server not responding is >>>>>>>> left in the >>>>>>>> configured list. >>>>>>>> >>>>>>>> rob >>>>>>> >>>>>>> 1) (minor) If I connected IPA host to other IPA server before next >>>>>>> enrollment, >>>>>>> there will be outdated /etc/ipa/ca.crt. This will cause all tries to >>>>>>> connect to >>>>>>> IPA server to fail with TLS error, but without giving any clue to >>>>>>> user... >>>>>> >>>>>> Yeah, this can be a problem but I'm going to leave things as-is >>>>>> for now. I >>>>>> believe we have a separate ticket to clean this up. I don't want to >>>>>> combine too >>>>>> many things into this patch, it is disruptive enough. >>>>>> >>>>>>> >>>>>>> # ipa-client-install >>>>>>> Provide your IPA server name (ex: ipa.example.com): >>>>>>> >>>>>>> He would need to reach out to the log to find this line: >>>>>>> LDAP Error: Connect error: TLS error -8054:You are attempting to >>>>>>> import a >>>>>>> cert >>>>>>> with the same issuer/serial as an existing cert, but that is not >>>>>>> the same >>>>>>> cert. >>>>>>> >>>>>>> I am thinking if we should not expose some LDAP errors after all? >>>>>>> To give >>>>>>> some >>>>>>> clue to user... >>>>>> >>>>>> Yeah, I switched the LDAP error unhandled exception back from >>>>>> debug to error. >>>>>> >>>>>>> >>>>>>> 2) (minor) While we are touching these errors I would also fix a >>>>>>> typo there >>>>>>> :-) >>>>>>> ... >>>>>>> if isinstance(err, ldap.INAPPROPRIATE_AUTH): >>>>>>> root_logger.debug("LDAP Error: Anonymous acces >>>>>>> not >>>>>>> allowed") >>>>>>> return [NO_ACCESS_TO_LDAP] ^^^^^ >>>>>>> ... >>>>>> >>>>>> Heh, ok. >>>>>> >>>>>> >>>>>>> >>>>>>> 3) (Regression) You neither set ds.server nor add server to >>>>>>> valid_servers >>>>>>> when >>>>>>> anonymous access is not enabled. The installer then does not try to >>>>>>> connect to >>>>>>> this server even though the installation would succeed: >>>>>>> >>>>>>> ... >>>>>>> elif ldapret[0] == NO_ACCESS_TO_LDAP or >>>>>>> ldapret[0] == >>>>>>> NO_TLS_LDAP: >>>>>>> ldapaccess = False >>>>>>> ... >>>>>> >>>>>> Good point. The handling for this is done a bit later so I need to >>>>>> defer a >>>>>> little processing but it should work now. >>>>>> >>>>>>> >>>>>>> 4) (Regression) Client will now report success in discovery even >>>>>>> when the >>>>>>> server is down: >>>>>>> >>>>>>> # ipa-client-install >>>>>>> Unable to verify that vm-037.idm.lab.bos.redhat.com (realm >>>>>>> IDM.LAB.BOS.REDHAT.COM) is an IPA server >>>>>>> Discovery was successful! >>>>>>> Hostname: vm-148.idm.lab.bos.redhat.com >>>>>>> Realm: IDM.LAB.BOS.REDHAT.COM >>>>>>> DNS Domain: idm.lab.bos.redhat.com >>>>>>> IPA Server: vm-037.idm.lab.bos.redhat.com >>>>>>> BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com >>>>>>> >>>>>>> Continue to configure the system with these values? [no]: y >>>>>>> User authorized to enroll computers: admin >>>>>>> Synchronizing time with KDC... >>>>>>> Password for admin at IDM.LAB.BOS.REDHAT.COM: >>>>>>> Kerberos authentication failed >>>>>>> kinit: Generic error (see e-text) while getting initial credentials >>>>>>> >>>>>>> Please make sure the following ports are opened in the firewall >>>>>>> settings: >>>>>>> TCP: 80, 88, 389 >>>>>>> UDP: 88 (at least one of TCP/UDP ports 88 has to be open) >>>>>>> Also note that following ports are necessary for ipa-client >>>>>>> working properly >>>>>>> after enrollment: >>>>>>> TCP: 464 >>>>>>> UDP: 464, 123 (if NTP enabled) >>>>>>> Installation failed. Rolling back changes. >>>>>>> IPA client is not configured on this system. >>>>>>> >>>>>>> >>>>>>> LDAP on vm-037 in this case is down. I think this would cause a >>>>>>> regression >>>>>>> too, >>>>>>> because previously we simply reported that no discovered server >>>>>>> is available >>>>>>> and let user enter the server manually. >>>>>> >>>>>> Fixed. >>>>>> >>>>>>> >>>>>>> IMO we are trying to be too smart when processing the >>>>>>> (discovered) servers. >>>>>>> Why >>>>>>> do we need to process and verify _all_ discovered servers even >>>>>>> when the >>>>>>> list is >>>>>>> not written anywhere in case of SRV lookup? I think it causes >>>>>>> unnecessary >>>>>>> traffic on network - we should connect to the first server >>>>>>> available. >>>>>> >>>>>> That's a good point. Now we except on the first discovered server. >>>>>> >>>>>> I think for user-provided servers we still want to verify them all >>>>>> since they >>>>>> will be hardcoded in a config file. >>>>>> >>>>>>> 5) In ipa-client-automount: >>>>>>> >>>>>>> + # Now confirm that our server is an IPA server. Stop >>>>>>> checking on the >>>>>>> first >>>>>>> + # success so we know we have at least one good one. >>>>>>> + for server in servers: >>>>>>> + root_logger.debug("Verifying that %s is an IPA server" % >>>>>>> server) >>>>>>> + ldapret = ds.ipacheckldap(server, api.env.realm) >>>>>>> >>>>>>> >>>>>>> In case of successful autodiscovery, this looks redundant as we >>>>>>> try to >>>>>>> verify >>>>>>> the servers second time even though we already did it with >>>>>>> ds.search and >>>>>>> ds.server should already contain a functional server. >>>>>> >>>>>> That's true, I ripped all this out. >>>>>> >>>>>> rob >>>>>> >>>>> >>>>> 1) One whitespace error: >>>>> >>>>> git apply /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch >>>>> /home/mkosek/freeipa-rcrit-1084-2-client-failover.patch:96: trailing >>>>> whitespace. >>>>> >>>>> warning: 1 line adds whitespace errors. >>>>> >>>>> 2) When ipa-client-automount --server option is passed, the >>>>> --server value is >>>>> then never user. This leads to installation failure when --no-sssd and >>>>> --server >>>>> options are used: >>>>> >>>>> ... >>>>> Raised exception local variable 'server' referenced before assignment >>>>> ... >>>>> >>>>> >>>>> ipa-client-install looks OK so far, I did not find any issue in my >>>>> tests. >>>>> >>>>> Martin >>>>> >>>> >>>> Fixed >>>> >>>> rob >>> >>> Looks good. Just one more remark: >>> >>> 1) When --server has anonymous search disabled (i.e. >>> ret==ipadiscovery.NO_ACCESS_TO_LDAP), ipa-client-automount just exits: >>> >>> # ipa-client-automount --server vm-024.idm.lab.bos.redhat.com >>> Unable to confirm that vm-024.idm.lab.bos.redhat.com is an IPA server >>> >>> When I do autodiscovery, the exact same server is used and installation >>> succeeds: >>> >>> # ipa-client-automount >>> Searching for IPA server... >>> IPA server: DNS discovery >>> Location: default >>> Continue to configure the system with these values? [no]: y >>> Configured /etc/nsswitch.conf >>> Configured /etc/sysconfig/nfs >>> Configured /etc/idmapd.conf >>> Started rpcidmapd >>> Started rpcgssd >>> Restarting sssd, waiting for it to become available. >>> Started autofs >>> >>> Martin >>> >> >> Fixed. I think this has existed for a while. >> >> rob > > Ok, sorry if I bugged you for existing bugs then :-) > > I think the patch is OK, I have found just one case when we did not > fallback because we always used just one SRV record. This is a diff that > fixed the issue for me: > > diff --git a/ipa-client/ipaclient/ipadiscovery.py > b/ipa-client/ipaclient/ipadiscovery.py > index 8338329..7fc6aae 100644 > --- a/ipa-client/ipaclient/ipadiscovery.py > +++ b/ipa-client/ipaclient/ipadiscovery.py > @@ -201,7 +201,8 @@ class IPADiscovery(object): > return NO_LDAP_SERVER > else: > root_logger.debug("Search for LDAP SRV record in %s", > domain) > - servers = self.ipadns_search_srv(domain, '_ldap._tcp', > 389) > + servers = self.ipadns_search_srv(domain, '_ldap._tcp', > 389, > + break_on_first=False) > if servers: > autodiscovered = True > self.domain = domain > > > ACK if you squash this patch. > > Martin fixed and pushed to master and ipa-3-1 rob From jcholast at redhat.com Fri Feb 8 07:43:31 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 08 Feb 2013 08:43:31 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51119947.9050707@redhat.com> References: <51119947.9050707@redhat.com> Message-ID: <5114ACA3.3010301@redhat.com> Hi, On 6.2.2013 00:44, Rob Crittenden wrote: > This adds a cert-find command for the dogtag backend. > > Searches can be done by serial number, by subject, revocation reason, > issue date, notbefore, notafter and revocation dates. > > I added some basic tests for this. I made it a separate test file > because the cert plugin tests do not use the declarative format and rely > on the selfsign backend by default. > > rob > I have one design question: why do you emulate object interface with Command plugins? Wouldn't it be better to add an actual Object plugin and Method plugins? That way you would not have to duplicate the Object bits for certs and as a result, the code would be cleaner and consistent with the rest of our plugins. Honza -- Jan Cholasta From mkosek at redhat.com Fri Feb 8 09:47:28 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 08 Feb 2013 10:47:28 +0100 Subject: [Freeipa-devel] [PATCH] 363-368 Configurable SID blacklists Message-ID: <5114C9B0.6050608@redhat.com> Sending patches according to RFE: http://www.freeipa.org/page/V3/Configurable_SID_Blacklists How this works: 1) Trust is added, SID blacklist is filled with default list (by ipa-sam plugin). When SID blacklist attribute is missing (e.g. for current trusts), ipa-kdb will use the hardcoded list. # echo password | ipa trust-add MKAD2012.TEST --admin="Administrator" --password ---------------------------------------------- Re-established trust to domain "MKAD2012.TEST" ---------------------------------------------- Realm name: MKAD2012.TEST Domain NetBIOS name: MKAD2012 Domain Security Identifier: S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx SID blacklist incoming: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, S-1-5-3, S-1-5-4, S-1-5-5, S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, S-1-5-11, S-1-5-12, S-1-5-13, S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, S-1-5-19, S-1-5-20 SID blacklist outgoing: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, S-1-5-3, S-1-5-4, S-1-5-5, S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, S-1-5-11, S-1-5-12, S-1-5-13, S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, S-1-5-19, S-1-5-20 Trust direction: Two-way trust Trust type: Active Directory domain Trust status: Established and verified 2) Incoming SID blacklist is updated (I added S-1-18-1 to the list as it is included in MS-PAC when I log from AD 2012): # ipa trust-mod MKAD2012.TEST --sid-blacklist-incoming S-1-0,S-1-1,S-1-2,S-1-3,S-1-5-1,S-1-5-2,S-1-5-3,S-1-5-4,S-1-5-5,S-1-5-6,S-1-5-7,S-1-5-8,S-1-5-9,S-1-5-10,S-1-5-11,S-1-5-12,S-1-5-13,S-1-5-14,S-1-5-15,S-1-5-16,S-1-5-17,S-1-5-18,S-1-5-19,S-1-5-20,S-1-18-1 3) When I now login from AD2012 to my IPA machine, I get error message in krb5kdc.log about the filtered SID I configured in LDAP: ... Feb 08 04:11:33 ipa.linux.mkad2012.test krb5kdc[6493](Error): PAC filtering issue: SID [S-1-18-1] is not allowed from a trusted source and will be excluded. ... NOTE: When coding and testing this feature I fixed several related bugs I found in ipa-kdb, see description of patches 363-365. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-363-ipa-kdb-add-sentinel-for-ldapderefspec-allocation.patch Type: text/x-patch Size: 1502 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-364-ipa-kdb-avoid-enomem-when-all-sids-are-filtered-out.patch Type: text/x-patch Size: 2012 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-365-ipa-kdb-reinitialize-ldap-configuration-for-known-re.patch Type: text/x-patch Size: 3386 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-366-add-sid-blacklist-attributes.patch Type: text/x-patch Size: 13847 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-367-ipa-kdb-read-sid-blacklist-from-ldap.patch Type: text/x-patch Size: 10331 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-368-ipa-sam-fill-sid-blacklist-when-trust-is-added.patch Type: text/x-patch Size: 2263 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 8 12:06:54 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 08 Feb 2013 13:06:54 +0100 Subject: [Freeipa-devel] [PATCH] 363-368 Configurable SID blacklists In-Reply-To: <5114C9B0.6050608@redhat.com> References: <5114C9B0.6050608@redhat.com> Message-ID: <5114EA5E.2070800@redhat.com> On 02/08/2013 10:47 AM, Martin Kosek wrote: > Sending patches according to RFE: > http://www.freeipa.org/page/V3/Configurable_SID_Blacklists > > How this works: > > 1) Trust is added, SID blacklist is filled with default list (by ipa-sam > plugin). When SID blacklist attribute is missing (e.g. for current trusts), > ipa-kdb will use the hardcoded list. > > # echo password | ipa trust-add MKAD2012.TEST --admin="Administrator" --password > ---------------------------------------------- > Re-established trust to domain "MKAD2012.TEST" > ---------------------------------------------- > Realm name: MKAD2012.TEST > Domain NetBIOS name: MKAD2012 > Domain Security Identifier: S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx > SID blacklist incoming: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, > S-1-5-3, S-1-5-4, S-1-5-5, > S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, > S-1-5-11, S-1-5-12, S-1-5-13, > S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, > S-1-5-19, S-1-5-20 > SID blacklist outgoing: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, > S-1-5-3, S-1-5-4, S-1-5-5, > S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, > S-1-5-11, S-1-5-12, S-1-5-13, > S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, > S-1-5-19, S-1-5-20 > Trust direction: Two-way trust > Trust type: Active Directory domain > Trust status: Established and verified > > 2) Incoming SID blacklist is updated (I added S-1-18-1 to the list as it is > included in MS-PAC when I log from AD 2012): > > # ipa trust-mod MKAD2012.TEST --sid-blacklist-incoming > S-1-0,S-1-1,S-1-2,S-1-3,S-1-5-1,S-1-5-2,S-1-5-3,S-1-5-4,S-1-5-5,S-1-5-6,S-1-5-7,S-1-5-8,S-1-5-9,S-1-5-10,S-1-5-11,S-1-5-12,S-1-5-13,S-1-5-14,S-1-5-15,S-1-5-16,S-1-5-17,S-1-5-18,S-1-5-19,S-1-5-20,S-1-18-1 > > 3) When I now login from AD2012 to my IPA machine, I get error message in > krb5kdc.log about the filtered SID I configured in LDAP: > > ... > Feb 08 04:11:33 ipa.linux.mkad2012.test krb5kdc[6493](Error): PAC filtering > issue: SID [S-1-18-1] is not allowed from a trusted source and will be excluded. > ... > > NOTE: > When coding and testing this feature I fixed several related bugs I found in > ipa-kdb, see description of patches 363-365. > > Martin > I forgot to update ACI allowing Trust Admins to modify the blacklist. I also added a validator for SIDs to help catching invalid SIDs. Updated patches attached. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-363-2-ipa-kdb-add-sentinel-for-ldapderefspec-allocation.patch Type: text/x-patch Size: 1502 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-364-2-ipa-kdb-avoid-enomem-when-all-sids-are-filtered-out.patch Type: text/x-patch Size: 2012 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-365-2-ipa-kdb-reinitialize-ldap-configuration-for-known-re.patch Type: text/x-patch Size: 3386 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-366-2-add-sid-blacklist-attributes.patch Type: text/x-patch Size: 18732 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-367-2-ipa-kdb-read-sid-blacklist-from-ldap.patch Type: text/x-patch Size: 10331 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-368-2-ipa-sam-fill-sid-blacklist-when-trust-is-added.patch Type: text/x-patch Size: 2263 bytes Desc: not available URL: From pvoborni at redhat.com Fri Feb 8 13:27:36 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 08 Feb 2013 14:27:36 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE Message-ID: <5114FD48.3070104@redhat.com> Checkbox for NONE option was added. https://fedorahosted.org/freeipa/ticket/3404 Patches for master and 3.1 branch attached. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0255-3-1-Added-Web-UI-support-for-service-PAC-type-option-NON.patch Type: text/x-patch Size: 1028 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0255-master-Added-Web-UI-support-for-service-PAC-type-option-NON.patch Type: text/x-patch Size: 1088 bytes Desc: not available URL: From rcritten at redhat.com Fri Feb 8 13:55:09 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 08 Feb 2013 08:55:09 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <5114ACA3.3010301@redhat.com> References: <51119947.9050707@redhat.com> <5114ACA3.3010301@redhat.com> Message-ID: <511503BD.2050606@redhat.com> Jan Cholasta wrote: > Hi, > > On 6.2.2013 00:44, Rob Crittenden wrote: >> This adds a cert-find command for the dogtag backend. >> >> Searches can be done by serial number, by subject, revocation reason, >> issue date, notbefore, notafter and revocation dates. >> >> I added some basic tests for this. I made it a separate test file >> because the cert plugin tests do not use the declarative format and rely >> on the selfsign backend by default. >> >> rob >> > > I have one design question: why do you emulate object interface with > Command plugins? Wouldn't it be better to add an actual Object plugin > and Method plugins? That way you would not have to duplicate the Object > bits for certs and as a result, the code would be cleaner and consistent > with the rest of our plugins. > > Honza > I forget the details of the reasoning but IIRC it is because these commands aren't backed by LDAP. So the normal things we get out of Object don't apply. rob From akrivoka at redhat.com Fri Feb 8 14:05:18 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Fri, 08 Feb 2013 15:05:18 +0100 Subject: [Freeipa-devel] [RFE] List of IPA realm domains In-Reply-To: <5112927E.3080700@redhat.com> References: <5112927E.3080700@redhat.com> Message-ID: <5115061E.1080904@redhat.com> I have modified the design page as per suggestions posted on the list: * There will 2 commands (not 3): realmdomains-show and realmdomains-mod * Updates and Upgrades section updated to reflect the addition of new LDAP container http://www.freeipa.org/page/V3/Realm_Domains On 02/06/2013 06:27 PM, Ana Krivokapic wrote: > Hello, > > Below is a design page for ticket: > https://fedorahosted.org/freeipa/ticket/2945. > > There are a couple of questions in the text. > > Thoughts, comments welcome! > > http://www.freeipa.org/page/V3/Realm_Domains > -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. From mkosek at redhat.com Fri Feb 8 14:18:57 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 08 Feb 2013 15:18:57 +0100 Subject: [Freeipa-devel] [PATCH] 89 Raise ValidationError on invalid CSV values In-Reply-To: <51068B87.5030601@redhat.com> References: <50EDA4CE.1060200@redhat.com> <50F3F272.2000501@redhat.com> <5106450A.2000206@redhat.com> <51068B87.5030601@redhat.com> Message-ID: <51150951.8000206@redhat.com> On 01/28/2013 03:30 PM, Petr Viktorin wrote: > On 01/28/2013 10:29 AM, Jan Cholasta wrote: >> On 14.1.2013 12:56, Petr Viktorin wrote: >>> On 01/09/2013 06:11 PM, Jan Cholasta wrote: >>>> Hi, >>>> >>>> this patch fixes . >>>> >>>> Honza >>>> >>> >>> The patch works well, but could you also add a test to ensure we don't >>> regress in the future? >>> >>> >> >> Test added. >> > > ACK > Pushed to master, ipa-3-1. Martin From abokovoy at redhat.com Fri Feb 8 14:25:21 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 8 Feb 2013 16:25:21 +0200 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <510FE077.8010801@redhat.com> References: <510FE077.8010801@redhat.com> Message-ID: <20130208142521.GS4506@redhat.com> On Mon, 04 Feb 2013, Tomas Babej wrote: >Hi, > >When adding/modifying an ID range for a trusted domain, the newly >added option --dom-name can be used. This looks up SID of the >trusted domain in LDAP and therefore the user is not required >to write it down in CLI. If the lookup fails, error message >asking the user to specify the SID manually is shown. > >https://fedorahosted.org/freeipa/ticket/3133 > >Tomas >>From 72f8802953edaaf5b9f7c34a38601fbccd681c8e Mon Sep 17 00:00:00 2001 >From: Tomas Babej >Date: Mon, 4 Feb 2013 08:33:53 -0500 >Subject: [PATCH] Add option to specify SID using domain name to > idrange-add/mod > >When adding/modifying an ID range for a trusted domain, the newly >added option --dom-name can be used. This looks up SID of the >trusted domain in LDAP and therefore the user is not required >to write it down in CLI. If the lookup fails, error message >asking the user to specify the SID manually is shown. > >https://fedorahosted.org/freeipa/ticket/3133 >--- > ipalib/plugins/idrange.py | 78 +++++++++++++++++++++++++++++++++++++++++------ > ipaserver/dcerpc.py | 10 ++++++ > 2 files changed, 78 insertions(+), 10 deletions(-) > >diff --git a/ipalib/plugins/idrange.py b/ipalib/plugins/idrange.py >index 84e1057ac6b59b8ad99882a54e3288897338c978..77a75e4cabc18ca873be7cadcf870427d5b36ea0 100644 >--- a/ipalib/plugins/idrange.py >+++ b/ipalib/plugins/idrange.py >@@ -197,6 +197,11 @@ class idrange(LDAPObject): > cli_name='dom_sid', > label=_('Domain SID of the trusted domain'), > ), >+ Str('ipanttrusteddomainname?', >+ cli_name='dom_name', >+ flags=('no_search', 'virtual_attribute'), >+ label=_('Name of the trusted domain'), >+ ), New options is added but API.txt wasn't changed. As result, 'make rpms' does not work. Could you please fix the patch and re-send it? -- / Alexander Bokovoy From rcritten at redhat.com Fri Feb 8 14:44:46 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 08 Feb 2013 09:44:46 -0500 Subject: [Freeipa-devel] [PATCH] 1086 handle no entries migrated Message-ID: <51150F5E.2050102@redhat.com> The migration performance patch added a crash bug if no entries were successfully migrated. This should fix it. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1086-migrate.patch Type: text/x-diff Size: 1013 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 8 15:00:29 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 08 Feb 2013 16:00:29 +0100 Subject: [Freeipa-devel] [PATCH 0026] Prevent integer overflow when setting krbPasswordExpiration In-Reply-To: <1359071152.20683.510.camel@willson.li.ssimo.org> References: <50F42859.1070807@redhat.com> <1358257702.15136.124.camel@willson.li.ssimo.org> <50F5BE38.1020901@redhat.com> <50F5C1D0.6060404@redhat.com> <1358283576.4590.18.camel@willson.li.ssimo.org> <50F5D9E2.7030800@redhat.com> <1358290532.4590.21.camel@willson.li.ssimo.org> <50F6946C.3040905@redhat.com> <1358344021.4590.31.camel@willson.li.ssimo.org> <50F6DC0C.1090106@redhat.com> <1358355708.4590.47.camel@willson.li.ssimo.org> <50F6E412.7020402@redhat.com> <50F74C57.6030604@redhat.com> <50F80AD1.20408@redhat.com> <1358439519.20683.14.camel@willson.li.ssimo.org> <50FEA74A.5060908@redhat.com> <1358870269.20683.292.camel@willson.li.ssimo.org> <50FEDCF3.2050409@redhat.com> <50FFDED0.2070703@redhat.com> <1359071152.20683.510.camel@willson.li.ssimo.org> Message-ID: <5115130D.5090106@redhat.com> On 01/25/2013 12:45 AM, Simo Sorce wrote: > On Wed, 2013-01-23 at 14:00 +0100, Tomas Babej wrote: >> On 01/22/2013 07:39 PM, Dmitri Pal wrote: >>> On 01/22/2013 10:57 AM, Simo Sorce wrote: >>>> On Tue, 2013-01-22 at 15:50 +0100, Tomas Babej wrote: >>>>> Here I bring the updated version of the patch. Please note, that I >>>>> *added* a flag attribute to ipadb_ldap_attr_to_krb5_timestamp >>>>> function, that controls whether the timestamp will be checked for >>>>> overflow or not. The reasoning behind this is that some attributes >>>>> will not be set to future dates, due to their inherent nature - such >>>>> as krbLastSuccessfulAuth or krbLastAdminUnlock. >>>>> >>>>> These are all related to past dates, and it would make no sense to set >>>>> them to future dates, even manually. Therefore I'd rather represent >>>>> negative values in these attributes as past dates. They would have to >>>>> be set manually anyway, because they would represent timestamps before >>>>> the beginning of the unix epoch, however, I find this approach better >>>>> than pushing them up to year 2038 in case such things happens. >>>>> >>>>> Any objections to this approach? >>>>> >>>> I am not sure I understand what is the point of giving this option to >>>> callers. A) How does an API user know when to use one or the other >>>> option. B) What good does it make to have the same date return different >>>> results based on a flag ? >>>> >>>> What will happen later on when MIT will 'fix' the 2038 limit by changing >>>> the meaning of negative timestamps ? Keep in mind that right now >>>> negative timestamps are not really valid in the MIT code. >>>> >>>> Unless there is a 'use' for getting negative timestamps I think it is >>>> only harmful to allow it and consumers would only be confused on whether >>>> it should be used or not. >>>> >>>> So my first impression is that you are a bit overthinking here and we >>>> should instead always force the same behavior for all callers and always >>>> check and enforce endoftime dates. >>>> >>>> Simo. >>>> >>> +1 >> Ok, the patch does not distinguish between 'past' and 'future' >> timestamps anymore. >> >> Please respond if you see any issues. > > I am sorry I haven't replied yet. I meant to test the patch this time > before ACKing given how fiddly this time issue seem to be but haven't > had found the time so far. > > Just want to say I do not see any problem in the last incarnation of the > patch, so if someone beats me to testing you have my blessing for an > ACK. > > Simo. > I tested the change and it worked fine. With Simo's blessing on ipa-kdb changes I am giving a second ACK. Pushed to master, ipa-3-1. Martin From tbabej at redhat.com Fri Feb 8 15:29:08 2013 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 08 Feb 2013 16:29:08 +0100 Subject: [Freeipa-devel] [PATCHES 0031-0032] Improve HBAC rule handling in selinuxusermap-add/mod/find In-Reply-To: <5112A797.805@redhat.com> References: <511294EB.9050805@redhat.com> <5112A797.805@redhat.com> Message-ID: <511519C4.30907@redhat.com> On 02/06/2013 07:57 PM, Rob Crittenden wrote: > Tomas Babej wrote: >> Hi, >> >> this pair of patches improves HBAC rule handling in selinuxusermap >> commands. >> >> Patch 0031 deals with: >> https://fedorahosted.org/freeipa/ticket/3349 >> >> Patch 0032 takes care of: >> https://fedorahosted.org/freeipa/ticket/3348 >> >> and is to be applied on top of Patch 0031. >> >> See commit messages for detailed info. >> >> Tomas >> > > ACK for patch 0032. > > For patch 0031 we can't change the data type of an existing attribute. > It will break backwards compatibility. Can you test with an older > client to see if it cares (it may not care about the name of the > type). If older clients will work then this is probably ok. > > I gather that seealso detected as a DN attribute and converted into a > DN class and this is blowing up the Str validator? > Yes, that was exactly the case. > rob I added a workaround for older client versions, tested it with freeipa-client/admintools 2.2, works as expeceted. However, this only should be issue if there is older admintools package on the client than on the server. Outline is such as follows: I added a new flag for DNParam seelalso attribute, called 'allow_malformed' that allows any string to be passed to DNParam. Its value gets wrapped in 'malformed=yes,value='. This allows to parse out the string in selinuxusermap-add/mod pre_callback out of the DN and search for the rule with such name so that it's DN gets in LDAP instead. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0031-2-Improve-HBAC-rule-handling-in-selinuxusermap-add-mod.patch Type: text/x-patch Size: 38401 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 8 15:30:54 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 08 Feb 2013 16:30:54 +0100 Subject: [Freeipa-devel] [PATCH] 1086 handle no entries migrated In-Reply-To: <51150F5E.2050102@redhat.com> References: <51150F5E.2050102@redhat.com> Message-ID: <51151A2E.6050609@redhat.com> On 02/08/2013 03:44 PM, Rob Crittenden wrote: > The migration performance patch added a crash bug if no entries were > successfully migrated. This should fix it. > > rob > ACK. Pushed to master, ipa-3-1, ipa-3-0. Martin From tbabej at redhat.com Fri Feb 8 15:35:03 2013 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 08 Feb 2013 16:35:03 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <20130208142521.GS4506@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> Message-ID: <51151B27.8020103@redhat.com> On 02/08/2013 03:25 PM, Alexander Bokovoy wrote: > On Mon, 04 Feb 2013, Tomas Babej wrote: >> Hi, >> >> When adding/modifying an ID range for a trusted domain, the newly >> added option --dom-name can be used. This looks up SID of the >> trusted domain in LDAP and therefore the user is not required >> to write it down in CLI. If the lookup fails, error message >> asking the user to specify the SID manually is shown. >> >> https://fedorahosted.org/freeipa/ticket/3133 >> >> Tomas > >>> From 72f8802953edaaf5b9f7c34a38601fbccd681c8e Mon Sep 17 00:00:00 2001 >> From: Tomas Babej >> Date: Mon, 4 Feb 2013 08:33:53 -0500 >> Subject: [PATCH] Add option to specify SID using domain name to >> idrange-add/mod >> >> When adding/modifying an ID range for a trusted domain, the newly >> added option --dom-name can be used. This looks up SID of the >> trusted domain in LDAP and therefore the user is not required >> to write it down in CLI. If the lookup fails, error message >> asking the user to specify the SID manually is shown. >> >> https://fedorahosted.org/freeipa/ticket/3133 >> --- >> ipalib/plugins/idrange.py | 78 >> +++++++++++++++++++++++++++++++++++++++++------ >> ipaserver/dcerpc.py | 10 ++++++ >> 2 files changed, 78 insertions(+), 10 deletions(-) >> >> diff --git a/ipalib/plugins/idrange.py b/ipalib/plugins/idrange.py >> index >> 84e1057ac6b59b8ad99882a54e3288897338c978..77a75e4cabc18ca873be7cadcf870427d5b36ea0 >> 100644 >> --- a/ipalib/plugins/idrange.py >> +++ b/ipalib/plugins/idrange.py >> @@ -197,6 +197,11 @@ class idrange(LDAPObject): >> cli_name='dom_sid', >> label=_('Domain SID of the trusted domain'), >> ), >> + Str('ipanttrusteddomainname?', >> + cli_name='dom_name', >> + flags=('no_search', 'virtual_attribute'), >> + label=_('Name of the trusted domain'), >> + ), > New options is added but API.txt wasn't changed. As result, 'make rpms' > does not work. > > Could you please fix the patch and re-send it? > Sorry about that. Updated patch attached. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0030-2-Add-option-to-specify-SID-using-domain-name-to-idran.patch Type: text/x-patch Size: 10629 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 8 15:58:47 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 08 Feb 2013 16:58:47 +0100 Subject: [Freeipa-devel] [PATCH] 369 Fix permission_find test error Message-ID: <511520B7.9010702@redhat.com> Remove extraneous memberindirect_role attribute from permission_find unit test to avoid false negative test result. Pushed as a one-liner to all affected branches: master, ipa-3-1, ipa-3-0. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-369-fix-permission_find-test-error.patch Type: text/x-patch Size: 1103 bytes Desc: not available URL: From abokovoy at redhat.com Fri Feb 8 16:30:12 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 8 Feb 2013 18:30:12 +0200 Subject: [Freeipa-devel] [PATCH] 356 Add trusconfig-show and trustconfig-mod commands In-Reply-To: <510BDD3C.5030900@redhat.com> References: <5107BC0D.6070200@redhat.com> <20130201145554.GL4506@redhat.com> <510BDD3C.5030900@redhat.com> Message-ID: <20130208163012.GT4506@redhat.com> On Fri, 01 Feb 2013, Martin Kosek wrote: >On 02/01/2013 03:55 PM, Alexander Bokovoy wrote: >> On Tue, 29 Jan 2013, Martin Kosek wrote: >>> trust_output_params = ( >>> @@ -482,3 +499,158 @@ api.register(trust_mod) >>> api.register(trust_del) >>> api.register(trust_find) >>> api.register(trust_show) >>> + >>> + >>> +_trust_type_option = ( >>> + StrEnum('trust_type', >>> + cli_name='type', >>> + label=_('Trust type (ad for Active Directory, default)'), >>> + values=(u'ad',), >>> + default=u'ad', >>> + autofill=True, >>> + ), >>> +) >> We already have various trust type definitions in the same file. Maybe >> it makes sense to unify those somehow? > >Right, I unified those 2 separate trust_type option definitions. > >> >>> + def get_dn(self, *keys, **kwargs): >>> + trust_type = kwargs.get('trust_type') >>> + if trust_type is None: >>> + raise errors.RequirementError(name='trust_type') >>> + if kwargs['trust_type'] == u'ad': >> Perhaps better to define constants for the trust type values... > >I changed it a bit and now it uses a dict instead. I think its now more general >and extensible. > >> >>> + except ValueError: >>> + # The search is performed for groups with "posixgroup" objectclass >>> + # and not "ipausergroup" so that it can also match groups like >>> + # "Default SMG Group" which does not have this objectclass. >> 'Default SM_B_ Group' > >Fixed. > >> >> Thanks for the unit tests too! >> > >You are welcome! I also generated API.txt which I forgot to do last time. >Updated patch attached. ACK for the code but please add more documentation (below). Works like sharm. I tried also changing default fallback group to some IPA group, then back to Default SMB Group and it worked well. Also specifying non-existing group was noted and rejected. Please make sure to mention in the design page magic value 'Default SMB Group' and also that you can use any group with 'posixgroup' objectclass, and that 'Default SMB Group' is not visible through normal IPA tools. We need to write better documentation (online help) for trustconfig-mod. Basically, right now it helps no one to understand what is supposed to be done here. Once help is added, ACK. -- / Alexander Bokovoy From jhrozek at redhat.com Fri Feb 8 17:44:09 2013 From: jhrozek at redhat.com (Jakub Hrozek) Date: Fri, 8 Feb 2013 18:44:09 +0100 Subject: [Freeipa-devel] [PATCH] Allow ipa-replica-conncheck and ipa-adtrust-install to read krb5 includedir Message-ID: <20130208174409.GJ3804@hendrix.brq.redhat.com> The krb5 includedir uses a different delimeter (space) than the rest of the krb5 config file (equal sign). But only the ipa-client-install and ipa-server-install scripts were set with the correct delimeters. This patch also adds the delimeters to ipa-adtrust-install and ipa-replica-conncheck. -------------- next part -------------- >From 194748f492f5751ec24c527469b1a7bcfdac76fa Mon Sep 17 00:00:00 2001 From: Jakub Hrozek Date: Fri, 8 Feb 2013 18:37:43 +0100 Subject: [PATCH] Allow ipa-replica-conncheck and ipa-adtrust-install to read krb5 includedir https://fedorahosted.org/freeipa/ticket/3132 --- install/tools/ipa-replica-conncheck | 2 +- ipaserver/install/adtrustinstance.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/install/tools/ipa-replica-conncheck b/install/tools/ipa-replica-conncheck index 29c43f60bd9e38d649db3730daacb76bd45b8786..8c96136d16fbf7ea33170eee1dedb9d500ceeecd 100755 --- a/install/tools/ipa-replica-conncheck +++ b/install/tools/ipa-replica-conncheck @@ -165,7 +165,7 @@ def sigterm_handler(signum, frame): def configure_krb5_conf(realm, kdc, filename): krbconf = ipaclient.ipachangeconf.IPAChangeConf("IPA Installer") - krbconf.setOptionAssignment(" = ") + krbconf.setOptionAssignment((" = ", " ")) krbconf.setSectionNameDelimiters(("[","]")) krbconf.setSubSectionDelimiters(("{","}")) krbconf.setIndent((""," "," ")) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 16f2136a6485e6915fd5de2000e6a378d03b44aa..cba88eb065cf2764ad4548742019a9f5cc40f499 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -574,7 +574,7 @@ class ADTRUSTInstance(service.Service): self.print_msg("Cannot modify /etc/krb5.conf") krbconf = ipaclient.ipachangeconf.IPAChangeConf("IPA Installer") - krbconf.setOptionAssignment(" = ") + krbconf.setOptionAssignment((" = ", " ")) krbconf.setSectionNameDelimiters(("[", "]")) krbconf.setSubSectionDelimiters(("{", "}")) krbconf.setIndent(("", " ", " ")) -- 1.8.1 From abokovoy at redhat.com Sun Feb 10 21:22:48 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Sun, 10 Feb 2013 23:22:48 +0200 Subject: [Freeipa-devel] [PATCH] Allow ipa-replica-conncheck and ipa-adtrust-install to read krb5 includedir In-Reply-To: <20130208174409.GJ3804@hendrix.brq.redhat.com> References: <20130208174409.GJ3804@hendrix.brq.redhat.com> Message-ID: <20130210212248.GV4506@redhat.com> On Fri, 08 Feb 2013, Jakub Hrozek wrote: >The krb5 includedir uses a different delimeter (space) than the rest of the krb5 >config file (equal sign). But only the ipa-client-install and >ipa-server-install scripts were set with the correct delimeters. This >patch also adds the delimeters to ipa-adtrust-install and >ipa-replica-conncheck. ACK. Works for me on Fedora 18 with git master. I've also checked RHEL 6.4 beta but since both original fix and this one are missing there, no changes were required to have ipa-adtrust-install working with 'includedir' defined in /etc/krb5.conf. -- / Alexander Bokovoy From tbabej at redhat.com Mon Feb 11 10:17:57 2013 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 11 Feb 2013 11:17:57 +0100 Subject: [Freeipa-devel] [PATCH 0033] Prevent changing protected group's name using --setattr Message-ID: <5118C555.6090009@redhat.com> Hi, The name of any protected group now cannot be changed by modifing the cn attribute using --setattr. Unit tests have been added to make sure there is no regression. https://fedorahosted.org/freeipa/ticket/3354 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0033-Prevent-changing-protected-group-s-name-using-setatt.patch Type: text/x-patch Size: 2692 bytes Desc: not available URL: From pspacek at redhat.com Mon Feb 11 11:57:16 2013 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 11 Feb 2013 12:57:16 +0100 Subject: [Freeipa-devel] user interfaces & default values = problem Message-ID: <5118DC9C.20804@redhat.com> Hello list, I realized one general problem we have with user interfaces for IPA & default values for various configuration options. Let me use DNS dynamic update as an example: - We have "built-in" default configuration (disabled) - We have "global" configuration object (ipaDnsConfig) - We have "per-object" configuration (in each DNS zone) IMHO user interface would be more usable if user can *see* which value is effective for particular service (and more generally any other object). At the moment, command "ipa dnszone-show example.com" will not show "dynamic update" value if it is not configured in the zone itself. E.g. [root at ipa1 ~]# ipa dnszone-show example.com Zone name: example.com Authoritative nameserver: ipa1.example.com. Administrator e-mail address: hostmaster.example.com. SOA serial: 1360583295 SOA refresh: 3600 SOA retry: 900 SOA expire: 1209600 SOA minimum: 3600 Active zone: TRUE Allow query: any; Allow transfer: none; # No "Dynamic update" value is shown above ... $ ipa dnsconfig-show [root at ipa1 ~]# ipa dnsconfig-show --------------------------------- Global DNS configuration is empty --------------------------------- What is the built-in default? It is in bind-dyndb-ldap documentation ... It is hard to debug things when you can't *see* effective value. IMHO it would be good to add lines like: Dynamic update: FALSE (inherited built-in default) and Dynamic update: FALSE (inherited global configuration) + some graphical representation of same thing to WebUI. Exactly same problem applies to PAC type for each service etc, so some general solution would be nice. Microsoft has something called "Resultant Set of Policy" for this purpose. -- Petr^2 Spacek From mkosek at redhat.com Mon Feb 11 12:49:11 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 11 Feb 2013 13:49:11 +0100 Subject: [Freeipa-devel] user interfaces & default values = problem In-Reply-To: <5118DC9C.20804@redhat.com> References: <5118DC9C.20804@redhat.com> Message-ID: <5118E8C7.5050705@redhat.com> On 02/11/2013 12:57 PM, Petr Spacek wrote: > Hello list, > > I realized one general problem we have with user interfaces for IPA & default > values for various configuration options. > > Let me use DNS dynamic update as an example: > - We have "built-in" default configuration (disabled) > - We have "global" configuration object (ipaDnsConfig) > - We have "per-object" configuration (in each DNS zone) > > IMHO user interface would be more usable if user can *see* which value is > effective for particular service (and more generally any other object). > > At the moment, command "ipa dnszone-show example.com" will not show "dynamic > update" value if it is not configured in the zone itself. > > E.g. > [root at ipa1 ~]# ipa dnszone-show example.com > Zone name: example.com > Authoritative nameserver: ipa1.example.com. > Administrator e-mail address: hostmaster.example.com. > SOA serial: 1360583295 > SOA refresh: 3600 > SOA retry: 900 > SOA expire: 1209600 > SOA minimum: 3600 > Active zone: TRUE > Allow query: any; > Allow transfer: none; > > # No "Dynamic update" value is shown above ... > > $ ipa dnsconfig-show > [root at ipa1 ~]# ipa dnsconfig-show > --------------------------------- > Global DNS configuration is empty > --------------------------------- > > What is the built-in default? It is in bind-dyndb-ldap documentation ... > > > It is hard to debug things when you can't *see* effective value. IMHO it would > be good to add lines like: > Dynamic update: FALSE (inherited built-in default) > and > Dynamic update: FALSE (inherited global configuration) > > + some graphical representation of same thing to WebUI. > > Exactly same problem applies to PAC type for each service etc, so some general > solution would be nice. > > Microsoft has something called "Resultant Set of Policy" for this purpose. > Hello Petr, in case of Dynamic update, we always configure the attribute with the setting, we just do not show the attribute by default to not make the output confusing... You need to pass --all option to show all zone configuration settings, including update policy: # ipa dnszone-show example.com --all dn: idnsname=example.com,cn=dns,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com Zone name: example.com Authoritative nameserver: vm-037.idm.lab.bos.redhat.com. Administrator e-mail address: hostmaster.example.com. SOA serial: 1360583513 SOA refresh: 3600 SOA retry: 900 SOA expire: 1209600 SOA minimum: 3600 BIND update policy: grant IDM.LAB.BOS.REDHAT.COM krb5-self * A; grant IDM.LAB.BOS.REDHAT.COM krb5-self * AAAA; grant IDM.LAB.BOS.REDHAT.COM krb5-self * SSHFP; Active zone: TRUE Dynamic update: FALSE <<<<<<<< Allow query: any; Allow transfer: none; nsrecord: vm-037.idm.lab.bos.redhat.com. objectclass: top, idnsrecord, idnszone If people want to see this setting by default, we can of course show it by default. As for the description - while text comments added to values look interesting: > Dynamic update: FALSE (inherited global configuration) it is not so straightforward to be done in our framework. We try quite hard to assign appropriate LDAP data types for different attributes which are then encoded to appropriate native Python types in our framework. "Dynamic update" is a BOOL type, so framework cannot return any text attached to it. We now return this format: { "error": null, "id": null, "principal": "admin at EXAMPEL.COM", "result": { "count": 1, "results": [ { "error": null, "result": { "dn": "idnsname=example.com,cn=dns,dc=example,dc=com", "idnsallowdynupdate": [ "FALSE" ], "idnsallowquery": [ "any;" ], "idnsallowtransfer": [ "none;" ], ... }, "summary": null, "value": "@" } ] }, "version": "3.1" } To support what you propose, we would also need to to return something like "meta-data" for the resulting dictionary returned by IPA server, for example as another attribute in resulting dict: { "error": null, "id": null, "principal": "admin at EXAMPEL.COM", "result": { "count": 1, "results": [ { "error": null, "result": { "dn": "idnsname=example.com,cn=dns,dc=example,dc=com", "idnsallowdynupdate": [ "FALSE" ], "__rsop_idnsallowdynupdate": [ "inherited global configuration" ], ... }, "summary": null, "value": "@" } ] }, "version": "3.1" } ... or in a separate tree: { "error": null, "id": null, "principal": "admin at EXAMPEL.COM", "result": { ... }, "meta": { "rsop": { "results": [ { "dn": "idnsname=example.com,cn=dns,dc=example,dc=com", "idnsallowdynupdate": [ "inherited global configuration" } ], } } Martin From pviktori at redhat.com Mon Feb 11 12:59:55 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 11 Feb 2013 13:59:55 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <511228A9.5060303@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> <5110FD62.4090006@redhat.com> <51111B1C.2010305@redhat.com> <511228A9.5060303@redhat.com> Message-ID: <5118EB4B.4080507@redhat.com> On 02/06/2013 10:55 AM, Jan Cholasta wrote: > On 5.2.2013 15:45, Petr Viktorin wrote: >> On 02/05/2013 01:38 PM, Jan Cholasta wrote: >>> On 4.2.2013 15:49, Petr Viktorin wrote: >> [...] >>>> >>>> I see one of the changes is using has_key instead of `in` for a CIDict. >>>> Given that dict.has_key() is deprecated, I think a better solution >>>> would >>>> be to add __contains__ to CIDict. Is there a reason against that? >>> >>> I think a separate patch with this and other changes to make CIDict more >>> like dict would be better. >> >> OK, I'll make a patch. >> >> [...] >>> Updated patches attached. >>> >> >> The changes look good but I can no longer apply the patches. Can you >> please rebase them? >> >> > > Sure. ACK -- Petr? From pviktori at redhat.com Mon Feb 11 13:07:34 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 11 Feb 2013 14:07:34 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <510BD364.7050809@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> Message-ID: <5118ED16.6070301@redhat.com> On 02/01/2013 03:38 PM, Petr Viktorin wrote: > On 02/01/2013 10:24 AM, Jan Cholasta wrote: >> On 1.2.2013 09:47, Petr Viktorin wrote: >>> On 01/31/2013 07:01 PM, Jan Cholasta wrote: >>>> On 31.1.2013 11:00, Petr Viktorin wrote: >>>>> On 01/30/2013 10:53 AM, Petr Viktorin wrote: >>>>>> On 01/29/2013 04:39 PM, Petr Viktorin wrote: >>>>>>> On 01/28/2013 04:09 PM, Petr Viktorin wrote: >>>>>>>> On 01/28/2013 09:34 AM, Jan Cholasta wrote: >>>>>>>>> On 25.1.2013 14:54, Petr Viktorin wrote: >>>>>>>>>> On 01/24/2013 03:06 PM, Petr Viktorin wrote: >>>>>>>>>>> On 01/24/2013 10:43 AM, Petr Viktorin wrote: >>>>>>>>>>>> On 01/22/2013 04:04 PM, Petr Viktorin wrote: >>>>>>>>>>>>> On 01/21/2013 06:38 PM, Petr Viktorin wrote: >>>>>>>>>>>>>> On 01/17/2013 06:27 PM, Petr Viktorin wrote: >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> This is the first batch of changes aimed to consolidate our >>>>>>>>>>>>>>> LDAP >>>>>>>>>>>>>>> code. >>>>>>>>>>>>>>> Each should be a self-contained change that doesn't break >>>>>>>>>>>>>>> anything. >>>>>>>>>>>>>>> > [...] >>>>>>>>>> Since this patchset is becoming unwieldy, I've put it in a public >>>>>>>>>> repo >>>>>>>>>> that I'll keep updated. The following command will fetch it into >>>>>>>>>> your >>>>>>>>>> "pviktori-ldap-refactor" branch: >>>>>>>>>> >>>>>>>>>> git fetch git://github.com/encukou/freeipa >>>>>>>>>> ldap-refactor:pviktori-ldap-refactor >>>>>>>>>> >>>>>>>>>> > [...] >>>> >>>> I would prefer if you used the semantics of .get() for .get_single() as >>>> well (i.e. when no default value is provided, None is assumed) in patch >>>> 152. Or is there a reason not to? >>> >>> I think you should always have to write extra code to supress exceptions >>> (?Errors should never pass silently?). In Python, the easiest/shortest >>> getter you can write will typically raise an exception when the value is >>> missing (e.g. `d[k]` for dicts, `getattr(a, b)` for objects). >>> >> >> That is true, but I think consistency is more important here - the name >> suggests the method behaves the same way .get() does. If you insist on >> keeping the current behavior, would you at least consider renaming the >> method (perhaps to just "single" or "single_value")? >> >> (This is just a nitpick, so don't worry too much about it.) > > Alright, I renamed get_single to single_value(). > > I also rebased to current master. Updating patch 115 to take recent changes in replica-manage into account. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0155-02-Replace-getList-by-a-get_entries-method.patch Type: text/x-patch Size: 22355 bytes Desc: not available URL: From jcholast at redhat.com Mon Feb 11 14:17:30 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 11 Feb 2013 15:17:30 +0100 Subject: [Freeipa-devel] [PATCHES] 120-126 LDAP code refactoring (part 1) In-Reply-To: <50F83498.9050104@redhat.com> References: <50F83498.9050104@redhat.com> Message-ID: <5118FD7A.4020704@redhat.com> Hi, On 17.1.2013 18:27, Petr Viktorin wrote: > Hello, > This is the first batch of changes aimed to consolidate our LDAP code. > Each should be a self-contained change that doesn't break anything. > > These patches do some general cleanup (some of the changes might seem > trivial but help a lot when grepping through the code); merge the common > parts LDAPEntry, Entry and Entity classes; and move stuff that depends > on an installed server out of IPASimpleLDAPObject and SchemaCache. > > I'm posting them early so you can see where I'm going, and so you can > find out if your work will conflict with mine. > ACK part 1. Honza -- Jan Cholasta From abokovoy at redhat.com Mon Feb 11 14:34:46 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 11 Feb 2013 16:34:46 +0200 Subject: [Freeipa-devel] [PATCH] 363-368 Configurable SID blacklists In-Reply-To: <5114EA5E.2070800@redhat.com> References: <5114C9B0.6050608@redhat.com> <5114EA5E.2070800@redhat.com> Message-ID: <20130211143446.GA4506@redhat.com> On Fri, 08 Feb 2013, Martin Kosek wrote: >On 02/08/2013 10:47 AM, Martin Kosek wrote: >> Sending patches according to RFE: >> http://www.freeipa.org/page/V3/Configurable_SID_Blacklists >> >> How this works: >> >> 1) Trust is added, SID blacklist is filled with default list (by ipa-sam >> plugin). When SID blacklist attribute is missing (e.g. for current trusts), >> ipa-kdb will use the hardcoded list. >> >> # echo password | ipa trust-add MKAD2012.TEST --admin="Administrator" --password >> ---------------------------------------------- >> Re-established trust to domain "MKAD2012.TEST" >> ---------------------------------------------- >> Realm name: MKAD2012.TEST >> Domain NetBIOS name: MKAD2012 >> Domain Security Identifier: S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx >> SID blacklist incoming: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >> S-1-5-3, S-1-5-4, S-1-5-5, >> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >> S-1-5-11, S-1-5-12, S-1-5-13, >> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >> S-1-5-19, S-1-5-20 >> SID blacklist outgoing: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >> S-1-5-3, S-1-5-4, S-1-5-5, >> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >> S-1-5-11, S-1-5-12, S-1-5-13, >> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >> S-1-5-19, S-1-5-20 >> Trust direction: Two-way trust >> Trust type: Active Directory domain >> Trust status: Established and verified >> >> 2) Incoming SID blacklist is updated (I added S-1-18-1 to the list as it is >> included in MS-PAC when I log from AD 2012): >> >> # ipa trust-mod MKAD2012.TEST --sid-blacklist-incoming >> S-1-0,S-1-1,S-1-2,S-1-3,S-1-5-1,S-1-5-2,S-1-5-3,S-1-5-4,S-1-5-5,S-1-5-6,S-1-5-7,S-1-5-8,S-1-5-9,S-1-5-10,S-1-5-11,S-1-5-12,S-1-5-13,S-1-5-14,S-1-5-15,S-1-5-16,S-1-5-17,S-1-5-18,S-1-5-19,S-1-5-20,S-1-18-1 >> >> 3) When I now login from AD2012 to my IPA machine, I get error message in >> krb5kdc.log about the filtered SID I configured in LDAP: >> >> ... >> Feb 08 04:11:33 ipa.linux.mkad2012.test krb5kdc[6493](Error): PAC filtering >> issue: SID [S-1-18-1] is not allowed from a trusted source and will be excluded. >> ... >> >> NOTE: >> When coding and testing this feature I fixed several related bugs I found in >> ipa-kdb, see description of patches 363-365. >> >> Martin >> > >I forgot to update ACI allowing Trust Admins to modify the blacklist. I also >added a validator for SIDs to help catching invalid SIDs. > >Updated patches attached. Work for me fine against Windows 2012 server. However, I'd like you to rebase on top of your previous patches. VERSION file is causing conflict since your patchset for trustconfig command increments to the same version as this one. -- / Alexander Bokovoy From pspacek at redhat.com Mon Feb 11 14:37:58 2013 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 11 Feb 2013 15:37:58 +0100 Subject: [Freeipa-devel] DNS SOA serial managed by 389 DS plugin: design Message-ID: <51190246.8060900@redhat.com> Hello, please take a look at http://freeipa.org/page/V3/DNS_SOA_serial_auto-incrementation Overview & chapter IPA 3.0 summarize current state and problems. Next chapter contains proposal combined from Simo's and my ideas. User interface part is still an open question. Any ideas are welcome! I copied only 3.1 part to make e-mail shorter: Design in IPA 3.1 Move SOA serial maintenance from bind-dyndb-ldap to (new?) 389 DS plugin: IPA ticket #3347. DS plugin watches cn=dns sub-tree for changes. Any change in DNS record in this subtree will increment idnsSOAserial attribute in record's parent zone. Basic idea if objectClass is idnsConfig change is ignored and idnsSOAserial is not incremented else if objectClass is idnsZone increment idnsSOAserial in the same object else if objectClass is idnsRecord increment idnsSOAserial in object's immediate parent e.g. change in idnsName=test, idnsName=example.com, cn=dns will increment idnsSOAserial in object idnsName=example.com, cn=dns if parent's objectClass is not idnsZone log an error (This should never happen :-)) SOA serial incrementation algorithm OLDserial = actual idnsSOAserial value timestamp = actual UNIX timestamp if (OLDserial != timestamp) newSerial = timestamp else newSerial = oldSerial + 1 Write newSerial value to particular idnsSOAserial attribute Implementation details BIND does direct SOA serial update (not trigerred by serial autoincrement feature) after any dynamic update. We have to catch those attempts and ignore them: A plugin can intercept any modify and manipulate it, including suppressing changes to SOA Serial. It should be possible to catch & discard SOA serial modifications inside BIND. This will save some load from LDAP server. Possible optimization Increment serial value at most once per second. Basic idea: Write current timestamp (no incrementation) and write serial value to the database with one second delay. Problem: How to solve LDAP server crash? Write timestamp immediately and (while reading) subtract 1 if timestamp == serial? -- Petr^2 Spacek From mkosek at redhat.com Mon Feb 11 14:40:21 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 11 Feb 2013 15:40:21 +0100 Subject: [Freeipa-devel] [PATCH] 356 Add trusconfig-show and trustconfig-mod commands In-Reply-To: <20130208163012.GT4506@redhat.com> References: <5107BC0D.6070200@redhat.com> <20130201145554.GL4506@redhat.com> <510BDD3C.5030900@redhat.com> <20130208163012.GT4506@redhat.com> Message-ID: <511902D5.2030605@redhat.com> On 02/08/2013 05:30 PM, Alexander Bokovoy wrote: > On Fri, 01 Feb 2013, Martin Kosek wrote: >> On 02/01/2013 03:55 PM, Alexander Bokovoy wrote: >>> On Tue, 29 Jan 2013, Martin Kosek wrote: >>>> trust_output_params = ( >>>> @@ -482,3 +499,158 @@ api.register(trust_mod) >>>> api.register(trust_del) >>>> api.register(trust_find) >>>> api.register(trust_show) >>>> + >>>> + >>>> +_trust_type_option = ( >>>> + StrEnum('trust_type', >>>> + cli_name='type', >>>> + label=_('Trust type (ad for Active Directory, default)'), >>>> + values=(u'ad',), >>>> + default=u'ad', >>>> + autofill=True, >>>> + ), >>>> +) >>> We already have various trust type definitions in the same file. Maybe >>> it makes sense to unify those somehow? >> >> Right, I unified those 2 separate trust_type option definitions. >> >>> >>>> + def get_dn(self, *keys, **kwargs): >>>> + trust_type = kwargs.get('trust_type') >>>> + if trust_type is None: >>>> + raise errors.RequirementError(name='trust_type') >>>> + if kwargs['trust_type'] == u'ad': >>> Perhaps better to define constants for the trust type values... >> >> I changed it a bit and now it uses a dict instead. I think its now more general >> and extensible. >> >>> >>>> + except ValueError: >>>> + # The search is performed for groups with "posixgroup" >>>> objectclass >>>> + # and not "ipausergroup" so that it can also match groups like >>>> + # "Default SMG Group" which does not have this objectclass. >>> 'Default SM_B_ Group' >> >> Fixed. >> >>> >>> Thanks for the unit tests too! >>> >> >> You are welcome! I also generated API.txt which I forgot to do last time. >> Updated patch attached. > ACK for the code but please add more documentation (below). > > Works like sharm. I tried also changing default fallback group to > some IPA group, then back to Default SMB Group and it worked well. Also > specifying non-existing group was noted and rejected. > > Please make sure to mention in the design page magic value 'Default SMB > Group' and also that you can use any group with 'posixgroup' > objectclass, and that 'Default SMB Group' is not visible through normal > IPA tools. > > We need to write better documentation (online help) for trustconfig-mod. > Basically, right now it helps no one to understand what is supposed to > be done here. > > Once help is added, ACK. Thanks for the review! RFE updated with information you mentioned. I also added more info to trust online help (which you verified off-list). Pushed to master, ipa-3-1. Martin From mkosek at redhat.com Mon Feb 11 14:46:57 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 11 Feb 2013 15:46:57 +0100 Subject: [Freeipa-devel] [PATCH] 363-368 Configurable SID blacklists In-Reply-To: <20130211143446.GA4506@redhat.com> References: <5114C9B0.6050608@redhat.com> <5114EA5E.2070800@redhat.com> <20130211143446.GA4506@redhat.com> Message-ID: <51190461.3070805@redhat.com> On 02/11/2013 03:34 PM, Alexander Bokovoy wrote: > On Fri, 08 Feb 2013, Martin Kosek wrote: >> On 02/08/2013 10:47 AM, Martin Kosek wrote: >>> Sending patches according to RFE: >>> http://www.freeipa.org/page/V3/Configurable_SID_Blacklists >>> >>> How this works: >>> >>> 1) Trust is added, SID blacklist is filled with default list (by ipa-sam >>> plugin). When SID blacklist attribute is missing (e.g. for current trusts), >>> ipa-kdb will use the hardcoded list. >>> >>> # echo password | ipa trust-add MKAD2012.TEST --admin="Administrator" >>> --password >>> ---------------------------------------------- >>> Re-established trust to domain "MKAD2012.TEST" >>> ---------------------------------------------- >>> Realm name: MKAD2012.TEST >>> Domain NetBIOS name: MKAD2012 >>> Domain Security Identifier: S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx >>> SID blacklist incoming: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >>> S-1-5-3, S-1-5-4, S-1-5-5, >>> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >>> S-1-5-11, S-1-5-12, S-1-5-13, >>> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >>> S-1-5-19, S-1-5-20 >>> SID blacklist outgoing: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >>> S-1-5-3, S-1-5-4, S-1-5-5, >>> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >>> S-1-5-11, S-1-5-12, S-1-5-13, >>> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >>> S-1-5-19, S-1-5-20 >>> Trust direction: Two-way trust >>> Trust type: Active Directory domain >>> Trust status: Established and verified >>> >>> 2) Incoming SID blacklist is updated (I added S-1-18-1 to the list as it is >>> included in MS-PAC when I log from AD 2012): >>> >>> # ipa trust-mod MKAD2012.TEST --sid-blacklist-incoming >>> S-1-0,S-1-1,S-1-2,S-1-3,S-1-5-1,S-1-5-2,S-1-5-3,S-1-5-4,S-1-5-5,S-1-5-6,S-1-5-7,S-1-5-8,S-1-5-9,S-1-5-10,S-1-5-11,S-1-5-12,S-1-5-13,S-1-5-14,S-1-5-15,S-1-5-16,S-1-5-17,S-1-5-18,S-1-5-19,S-1-5-20,S-1-18-1 >>> >>> >>> 3) When I now login from AD2012 to my IPA machine, I get error message in >>> krb5kdc.log about the filtered SID I configured in LDAP: >>> >>> ... >>> Feb 08 04:11:33 ipa.linux.mkad2012.test krb5kdc[6493](Error): PAC filtering >>> issue: SID [S-1-18-1] is not allowed from a trusted source and will be >>> excluded. >>> ... >>> >>> NOTE: >>> When coding and testing this feature I fixed several related bugs I found in >>> ipa-kdb, see description of patches 363-365. >>> >>> Martin >>> >> >> I forgot to update ACI allowing Trust Admins to modify the blacklist. I also >> added a validator for SIDs to help catching invalid SIDs. >> >> Updated patches attached. > Work for me fine against Windows 2012 server. > > However, I'd like you to rebase on top of your previous patches. VERSION > file is causing conflict since your patchset for trustconfig command > increments to the same version as this one. > I pushed previous acked patch to master. Attaching patches 363-368 rebased on top of that. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-363-3-ipa-kdb-add-sentinel-for-ldapderefspec-allocation.patch Type: text/x-patch Size: 1504 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-364-3-ipa-kdb-avoid-enomem-when-all-sids-are-filtered-out.patch Type: text/x-patch Size: 2014 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-365-3-ipa-kdb-reinitialize-ldap-configuration-for-known-re.patch Type: text/x-patch Size: 3388 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-366-3-add-sid-blacklist-attributes.patch Type: text/x-patch Size: 18744 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-367-3-ipa-kdb-read-sid-blacklist-from-ldap.patch Type: text/x-patch Size: 10333 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-368-3-ipa-sam-fill-sid-blacklist-when-trust-is-added.patch Type: text/x-patch Size: 2265 bytes Desc: not available URL: From sbose at redhat.com Mon Feb 11 16:00:19 2013 From: sbose at redhat.com (Sumit Bose) Date: Mon, 11 Feb 2013 17:00:19 +0100 Subject: [Freeipa-devel] [PATCH] ipa-kdb: Free talloc autofree context when module is closed Message-ID: <20130211160019.GJ24753@localhost.localdomain> Hi, this patch should fix https://fedorahosted.org/freeipa/ticket/3410 . See commit message for details. bye, Sumit -------------- next part -------------- From 4892c10b273119e20c536285663f1ded6b6ff3fe Mon Sep 17 00:00:00 2001 From: sbose Date: Mon, 11 Feb 2013 15:39:22 +0100 Subject: [PATCH] ipa-kdb: Free talloc autofree context when module is closed Currently kdb5kdc crashes on exit if the ipadb KDB modules is loaded and trusts are configured. The reason is the talloc autofree context which get initialised during the ndr_push_union_blob() call. On exit the KDC module is unloaded an later on atexit() tries to free the context, but all related symbols are already unloaded with the module. This patch frees the talloc autofree context during the cleanup routine of the module. Since this is called only at exit and not during normal operations this is safe even if other KDC plugins use the talloc autofree context, e.g. via some Samba libraries, as well. Fixes https://fedorahosted.org/freeipa/ticket/3410 --- daemons/ipa-kdb/ipa_kdb.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/daemons/ipa-kdb/ipa_kdb.c b/daemons/ipa-kdb/ipa_kdb.c index 3527cefa10df67d3f17c730ab4483410c736a44f..0b769f7ed76488c5febf9e610450815534398f4d 100644 --- a/daemons/ipa-kdb/ipa_kdb.c +++ b/daemons/ipa-kdb/ipa_kdb.c @@ -20,6 +20,8 @@ * along with this program. If not, see . */ +#include + #include "ipa_kdb.h" struct ipadb_context *ipadb_get_context(krb5_context kcontext) @@ -452,6 +454,7 @@ static krb5_error_code ipadb_fini_module(krb5_context kcontext) ipactx = ipadb_get_context(kcontext); ipadb_context_free(kcontext, &ipactx); + talloc_free(talloc_autofree_context()); return 0; } -- 1.8.0.2 From simo at redhat.com Mon Feb 11 16:23:53 2013 From: simo at redhat.com (Simo Sorce) Date: Mon, 11 Feb 2013 11:23:53 -0500 Subject: [Freeipa-devel] DNS SOA serial managed by 389 DS plugin: design In-Reply-To: <51190246.8060900@redhat.com> References: <51190246.8060900@redhat.com> Message-ID: <1360599833.2299.100.camel@willson.li.ssimo.org> On Mon, 2013-02-11 at 15:37 +0100, Petr Spacek wrote: > Possible optimization > > Increment serial value at most once per second. > > Basic idea: Write current timestamp (no incrementation) and write > serial value > to the database with one second delay. > > Problem: How to solve LDAP server crash? Write timestamp immediately > and > (while reading) subtract 1 if timestamp == serial? > I am not sure I understand the solution here ? Maybe a simpler solution is to always write the serial as time() + 1 ? Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Mon Feb 11 17:58:05 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 11 Feb 2013 19:58:05 +0200 Subject: [Freeipa-devel] [PATCH] 363-368 Configurable SID blacklists In-Reply-To: <51190461.3070805@redhat.com> References: <5114C9B0.6050608@redhat.com> <5114EA5E.2070800@redhat.com> <20130211143446.GA4506@redhat.com> <51190461.3070805@redhat.com> Message-ID: <20130211175805.GD4506@redhat.com> On Mon, 11 Feb 2013, Martin Kosek wrote: >On 02/11/2013 03:34 PM, Alexander Bokovoy wrote: >> On Fri, 08 Feb 2013, Martin Kosek wrote: >>> On 02/08/2013 10:47 AM, Martin Kosek wrote: >>>> Sending patches according to RFE: >>>> http://www.freeipa.org/page/V3/Configurable_SID_Blacklists >>>> >>>> How this works: >>>> >>>> 1) Trust is added, SID blacklist is filled with default list (by ipa-sam >>>> plugin). When SID blacklist attribute is missing (e.g. for current trusts), >>>> ipa-kdb will use the hardcoded list. >>>> >>>> # echo password | ipa trust-add MKAD2012.TEST --admin="Administrator" >>>> --password >>>> ---------------------------------------------- >>>> Re-established trust to domain "MKAD2012.TEST" >>>> ---------------------------------------------- >>>> Realm name: MKAD2012.TEST >>>> Domain NetBIOS name: MKAD2012 >>>> Domain Security Identifier: S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx >>>> SID blacklist incoming: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >>>> S-1-5-3, S-1-5-4, S-1-5-5, >>>> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >>>> S-1-5-11, S-1-5-12, S-1-5-13, >>>> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >>>> S-1-5-19, S-1-5-20 >>>> SID blacklist outgoing: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >>>> S-1-5-3, S-1-5-4, S-1-5-5, >>>> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >>>> S-1-5-11, S-1-5-12, S-1-5-13, >>>> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >>>> S-1-5-19, S-1-5-20 >>>> Trust direction: Two-way trust >>>> Trust type: Active Directory domain >>>> Trust status: Established and verified >>>> >>>> 2) Incoming SID blacklist is updated (I added S-1-18-1 to the list as it is >>>> included in MS-PAC when I log from AD 2012): >>>> >>>> # ipa trust-mod MKAD2012.TEST --sid-blacklist-incoming >>>> S-1-0,S-1-1,S-1-2,S-1-3,S-1-5-1,S-1-5-2,S-1-5-3,S-1-5-4,S-1-5-5,S-1-5-6,S-1-5-7,S-1-5-8,S-1-5-9,S-1-5-10,S-1-5-11,S-1-5-12,S-1-5-13,S-1-5-14,S-1-5-15,S-1-5-16,S-1-5-17,S-1-5-18,S-1-5-19,S-1-5-20,S-1-18-1 >>>> >>>> >>>> 3) When I now login from AD2012 to my IPA machine, I get error message in >>>> krb5kdc.log about the filtered SID I configured in LDAP: >>>> >>>> ... >>>> Feb 08 04:11:33 ipa.linux.mkad2012.test krb5kdc[6493](Error): PAC filtering >>>> issue: SID [S-1-18-1] is not allowed from a trusted source and will be >>>> excluded. >>>> ... >>>> >>>> NOTE: >>>> When coding and testing this feature I fixed several related bugs I found in >>>> ipa-kdb, see description of patches 363-365. >>>> >>>> Martin >>>> >>> >>> I forgot to update ACI allowing Trust Admins to modify the blacklist. I also >>> added a validator for SIDs to help catching invalid SIDs. >>> >>> Updated patches attached. >> Work for me fine against Windows 2012 server. >> >> However, I'd like you to rebase on top of your previous patches. VERSION >> file is causing conflict since your patchset for trustconfig command >> increments to the same version as this one. >> > >I pushed previous acked patch to master. Attaching patches 363-368 rebased on >top of that. ACK. Thanks a lot! -- / Alexander Bokovoy From ondrej at hamada.cz Mon Feb 11 20:03:42 2013 From: ondrej at hamada.cz (Ondrej Hamada) Date: Mon, 11 Feb 2013 21:03:42 +0100 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <510DC28F.1000702@redhat.com> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> Message-ID: <51194E9E.1080909@hamada.cz> Dne 3.2.2013 02:51, Dmitri Pal napsal(a): > On 01/31/2013 06:09 PM, Ondrej Hamada wrote: >> Hello, >> I'm starting to work on my thesis about 'More types of replicas in >> FreeIPA' again. One of the main problems is the way how should the >> read-only replicas deal with KDC because they're not supposed to >> posses the Kerberos (krb) master key. The task was to investigate how >> is this solved in Active Directory and its Read Only Domain Controllers. >> >> I found out that the basic of RODC behaviour is described on technet >> page >> (http://technet.microsoft.com/en-us/library/cc754218%28v=ws.10%29.aspx). >> >> Login situation: >> RODC by default forwards the KRB requests to the DC. RODC then >> forwards the response back to the client and also requests the >> password to be replicated to RODC. Both the user and his host must be >> members of 'Allowed RODC Password Replication' group in order to let >> user's passwords being replicated to RODCs. >> >> Request services that the RODC doesn't have credentials for: >> Client sends TGS-REQ to RODC. RODC can read the TGT in the request, >> but doesn't have credentials for the service. So the request is >> forwarded to the DC. DC can decrypt the TGT that was created by RODC >> and sends back the TGS-RES that is forwarded to the client. (but it >> does not trust the RODC so it recalculates the privilege attribute >> certificate). RODC does not cache the credentials for the service. >> >> During my experiments the credentials got replicated to the RODC on >> the first log on of the user. The user's KRB requests were first >> forwarded to the DC. When the user got krbtgt and TGS for host, ldap >> and cifs, his TGT was revoked by RODC. He run through the auth. >> process again, but this time the requests were served by RODC only - >> no forwarding - and not TGS for host was requested. >> >> Unfortunately I can not still recognize how the keys are processed. >> There's barely any RPC communication - only one DCERPC packet exchange >> between RODC and DC that takes place when the user sends his first TGS >> request (this exchange happens also for the clients with disabled >> replication). >> >> It looks to me like the DC knows all the RODC keys. According to >> Technet, the MS implementation of Kerberos is able to recognize the >> key owner from the Key Version Number value. >> >> I think I can't get more info from the network traffic examination. Do >> you have any ideas or hints on further investigation of the problem? > The page you listed shows the diagrams of the user login and TGT and > then TGS acquisition. > http://technet.microsoft.com/en-us/library/cc754218%28v=WS.10%29#BKMK_AuthRODC > Also the following thread sheds some good light on how the > authentication and caching happens in case of the RODC. > http://social.technet.microsoft.com/forums/en-US/winserverDS/thread/f8d1017e-1f0e-4a9d-a241-b03b508dfe17 > So they have policy driven replication of passwords. > If password is allowed to be replicated by policy it is replicated if > not it RODC has to always proxy to RWDC for this account. > The password changes always happen on RWDC and replicated back. > They can be replicated by RSO operation that allows replicating a single > object. > > It seems that they assume that all the services are always remote thus > connectivity to the central RWDC is a must. > They do not seem to keep any service keys locally in the RODC. > > With SSSD in play on the client I am not sure we should worry about > caching at least in the first implementation. > So in our case it might make sense to have a proxy KDC and a RO replica > with the subset of data replicated to it. > I wonder if you can accomplish that by taking 389 RO replica + IAKERB > I suggest you look at IAKERB and see if it can be used as a proxy for > user authentication, password change and TGS requests. > If not may be we can at least use it as an inspiration. > > The attached diagram shows what I mean. > > Later we can add some logic for the following: > a) RODC requesting replication of a specific object > b) RWDC replicating a specific object > c) Policy to define for which accounts the passwords and keys can replicated > d) RODC getting policy and performing local authentication for the > accounts for which the keys were replicated. > > However with SSSD on the client it might be easier for KDC proxy just to > go offline and not respond to the SSSD if it loses connection to the > central server. > That would force SSSD to go offline and use local password caching. I > suspect that SSSD password caching would be enabled in many cases > anyways so not caching all passwords in one central locating in RODC > might actually be a good thing. > Thanks for hints. I was looking at IAKERB and according to RFC it should support both AS and TGS requests/replies so it might be doable. I'll try to prepare a prototype that will be using 389 RO replica and IAKERB as you proposed. From simo at redhat.com Mon Feb 11 20:21:34 2013 From: simo at redhat.com (Simo Sorce) Date: Mon, 11 Feb 2013 15:21:34 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <51194E9E.1080909@hamada.cz> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> Message-ID: <1360614094.2299.110.camel@willson.li.ssimo.org> On Mon, 2013-02-11 at 21:03 +0100, Ondrej Hamada wrote: > Dne 3.2.2013 02:51, Dmitri Pal napsal(a): > > On 01/31/2013 06:09 PM, Ondrej Hamada wrote: > >> Hello, > >> I'm starting to work on my thesis about 'More types of replicas in > >> FreeIPA' again. One of the main problems is the way how should the > >> read-only replicas deal with KDC because they're not supposed to > >> posses the Kerberos (krb) master key. The task was to investigate how > >> is this solved in Active Directory and its Read Only Domain Controllers. > >> > >> I found out that the basic of RODC behaviour is described on technet > >> page > >> (http://technet.microsoft.com/en-us/library/cc754218%28v=ws.10%29.aspx). > >> > >> Login situation: > >> RODC by default forwards the KRB requests to the DC. RODC then > >> forwards the response back to the client and also requests the > >> password to be replicated to RODC. Both the user and his host must be > >> members of 'Allowed RODC Password Replication' group in order to let > >> user's passwords being replicated to RODCs. > >> > >> Request services that the RODC doesn't have credentials for: > >> Client sends TGS-REQ to RODC. RODC can read the TGT in the request, > >> but doesn't have credentials for the service. So the request is > >> forwarded to the DC. DC can decrypt the TGT that was created by RODC > >> and sends back the TGS-RES that is forwarded to the client. (but it > >> does not trust the RODC so it recalculates the privilege attribute > >> certificate). RODC does not cache the credentials for the service. > >> > >> During my experiments the credentials got replicated to the RODC on > >> the first log on of the user. The user's KRB requests were first > >> forwarded to the DC. When the user got krbtgt and TGS for host, ldap > >> and cifs, his TGT was revoked by RODC. He run through the auth. > >> process again, but this time the requests were served by RODC only - > >> no forwarding - and not TGS for host was requested. > >> > >> Unfortunately I can not still recognize how the keys are processed. > >> There's barely any RPC communication - only one DCERPC packet exchange > >> between RODC and DC that takes place when the user sends his first TGS > >> request (this exchange happens also for the clients with disabled > >> replication). > >> > >> It looks to me like the DC knows all the RODC keys. According to > >> Technet, the MS implementation of Kerberos is able to recognize the > >> key owner from the Key Version Number value. > >> > >> I think I can't get more info from the network traffic examination. Do > >> you have any ideas or hints on further investigation of the problem? > > The page you listed shows the diagrams of the user login and TGT and > > then TGS acquisition. > > http://technet.microsoft.com/en-us/library/cc754218%28v=WS.10%29#BKMK_AuthRODC > > Also the following thread sheds some good light on how the > > authentication and caching happens in case of the RODC. > > http://social.technet.microsoft.com/forums/en-US/winserverDS/thread/f8d1017e-1f0e-4a9d-a241-b03b508dfe17 > > So they have policy driven replication of passwords. > > If password is allowed to be replicated by policy it is replicated if > > not it RODC has to always proxy to RWDC for this account. > > The password changes always happen on RWDC and replicated back. > > They can be replicated by RSO operation that allows replicating a single > > object. > > > > It seems that they assume that all the services are always remote thus > > connectivity to the central RWDC is a must. > > They do not seem to keep any service keys locally in the RODC. > > > > With SSSD in play on the client I am not sure we should worry about > > caching at least in the first implementation. > > So in our case it might make sense to have a proxy KDC and a RO replica > > with the subset of data replicated to it. > > I wonder if you can accomplish that by taking 389 RO replica + IAKERB > > I suggest you look at IAKERB and see if it can be used as a proxy for > > user authentication, password change and TGS requests. > > If not may be we can at least use it as an inspiration. > > > > The attached diagram shows what I mean. > > > > Later we can add some logic for the following: > > a) RODC requesting replication of a specific object > > b) RWDC replicating a specific object > > c) Policy to define for which accounts the passwords and keys can replicated > > d) RODC getting policy and performing local authentication for the > > accounts for which the keys were replicated. > > > > However with SSSD on the client it might be easier for KDC proxy just to > > go offline and not respond to the SSSD if it loses connection to the > > central server. > > That would force SSSD to go offline and use local password caching. I > > suspect that SSSD password caching would be enabled in many cases > > anyways so not caching all passwords in one central locating in RODC > > might actually be a good thing. > > > Thanks for hints. I was looking at IAKERB and according to RFC it should > support both AS and TGS requests/replies so it might be doable. I'll try > to prepare a prototype that will be using 389 RO replica and IAKERB as > you proposed. IAKERB is a GSSAPI level interface, it is not something you can use as a KDC Proxy on its own. An actual KDC proxy is what is needed because you cannot change applications at will and there are a number of applications that just go and talk straight to a KDC server to do AS and TGS requests. A KDC proxy is possible to build, it just require some work in the Actual KDC and probably some custom protocol to talk between KDCs. We should look into what is the simplest technical way to achieve this and perhaps propose it for standardization in IETF. Simo. -- Simo Sorce * Red Hat, Inc * New York From dpal at redhat.com Tue Feb 12 01:30:39 2013 From: dpal at redhat.com (Dmitri Pal) Date: Mon, 11 Feb 2013 20:30:39 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <1360614094.2299.110.camel@willson.li.ssimo.org> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> Message-ID: <51199B3F.2080808@redhat.com> On 02/11/2013 03:21 PM, Simo Sorce wrote: > On Mon, 2013-02-11 at 21:03 +0100, Ondrej Hamada wrote: >> Dne 3.2.2013 02:51, Dmitri Pal napsal(a): >>> On 01/31/2013 06:09 PM, Ondrej Hamada wrote: >>>> Hello, >>>> I'm starting to work on my thesis about 'More types of replicas in >>>> FreeIPA' again. One of the main problems is the way how should the >>>> read-only replicas deal with KDC because they're not supposed to >>>> posses the Kerberos (krb) master key. The task was to investigate how >>>> is this solved in Active Directory and its Read Only Domain Controllers. >>>> >>>> I found out that the basic of RODC behaviour is described on technet >>>> page >>>> (http://technet.microsoft.com/en-us/library/cc754218%28v=ws.10%29.aspx). >>>> >>>> Login situation: >>>> RODC by default forwards the KRB requests to the DC. RODC then >>>> forwards the response back to the client and also requests the >>>> password to be replicated to RODC. Both the user and his host must be >>>> members of 'Allowed RODC Password Replication' group in order to let >>>> user's passwords being replicated to RODCs. >>>> >>>> Request services that the RODC doesn't have credentials for: >>>> Client sends TGS-REQ to RODC. RODC can read the TGT in the request, >>>> but doesn't have credentials for the service. So the request is >>>> forwarded to the DC. DC can decrypt the TGT that was created by RODC >>>> and sends back the TGS-RES that is forwarded to the client. (but it >>>> does not trust the RODC so it recalculates the privilege attribute >>>> certificate). RODC does not cache the credentials for the service. >>>> >>>> During my experiments the credentials got replicated to the RODC on >>>> the first log on of the user. The user's KRB requests were first >>>> forwarded to the DC. When the user got krbtgt and TGS for host, ldap >>>> and cifs, his TGT was revoked by RODC. He run through the auth. >>>> process again, but this time the requests were served by RODC only - >>>> no forwarding - and not TGS for host was requested. >>>> >>>> Unfortunately I can not still recognize how the keys are processed. >>>> There's barely any RPC communication - only one DCERPC packet exchange >>>> between RODC and DC that takes place when the user sends his first TGS >>>> request (this exchange happens also for the clients with disabled >>>> replication). >>>> >>>> It looks to me like the DC knows all the RODC keys. According to >>>> Technet, the MS implementation of Kerberos is able to recognize the >>>> key owner from the Key Version Number value. >>>> >>>> I think I can't get more info from the network traffic examination. Do >>>> you have any ideas or hints on further investigation of the problem? >>> The page you listed shows the diagrams of the user login and TGT and >>> then TGS acquisition. >>> http://technet.microsoft.com/en-us/library/cc754218%28v=WS.10%29#BKMK_AuthRODC >>> Also the following thread sheds some good light on how the >>> authentication and caching happens in case of the RODC. >>> http://social.technet.microsoft.com/forums/en-US/winserverDS/thread/f8d1017e-1f0e-4a9d-a241-b03b508dfe17 >>> So they have policy driven replication of passwords. >>> If password is allowed to be replicated by policy it is replicated if >>> not it RODC has to always proxy to RWDC for this account. >>> The password changes always happen on RWDC and replicated back. >>> They can be replicated by RSO operation that allows replicating a single >>> object. >>> >>> It seems that they assume that all the services are always remote thus >>> connectivity to the central RWDC is a must. >>> They do not seem to keep any service keys locally in the RODC. >>> >>> With SSSD in play on the client I am not sure we should worry about >>> caching at least in the first implementation. >>> So in our case it might make sense to have a proxy KDC and a RO replica >>> with the subset of data replicated to it. >>> I wonder if you can accomplish that by taking 389 RO replica + IAKERB >>> I suggest you look at IAKERB and see if it can be used as a proxy for >>> user authentication, password change and TGS requests. >>> If not may be we can at least use it as an inspiration. >>> >>> The attached diagram shows what I mean. >>> >>> Later we can add some logic for the following: >>> a) RODC requesting replication of a specific object >>> b) RWDC replicating a specific object >>> c) Policy to define for which accounts the passwords and keys can replicated >>> d) RODC getting policy and performing local authentication for the >>> accounts for which the keys were replicated. >>> >>> However with SSSD on the client it might be easier for KDC proxy just to >>> go offline and not respond to the SSSD if it loses connection to the >>> central server. >>> That would force SSSD to go offline and use local password caching. I >>> suspect that SSSD password caching would be enabled in many cases >>> anyways so not caching all passwords in one central locating in RODC >>> might actually be a good thing. >>> >> Thanks for hints. I was looking at IAKERB and according to RFC it should >> support both AS and TGS requests/replies so it might be doable. I'll try >> to prepare a prototype that will be using 389 RO replica and IAKERB as >> you proposed. > IAKERB is a GSSAPI level interface, it is not something you can use as a > KDC Proxy on its own. > An actual KDC proxy is what is needed because you cannot change > applications at will and there are a number of applications that just go > and talk straight to a KDC server to do AS and TGS requests. > > A KDC proxy is possible to build, it just require some work in the > Actual KDC and probably some custom protocol to talk between KDCs. > We should look into what is the simplest technical way to achieve this > and perhaps propose it for standardization in IETF. > > Simo. > Really? I was hoping that IAKERB is exactly what you have described. Are you saying that IAKERB requires changes to the client? That sounds strange. -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Tue Feb 12 07:52:08 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 12 Feb 2013 08:52:08 +0100 Subject: [Freeipa-devel] [PATCH] ipa-kdb: Free talloc autofree context when module is closed In-Reply-To: <20130211160019.GJ24753@localhost.localdomain> References: <20130211160019.GJ24753@localhost.localdomain> Message-ID: <5119F4A8.1060307@redhat.com> On 02/11/2013 05:00 PM, Sumit Bose wrote: > Hi, > > this patch should fix https://fedorahosted.org/freeipa/ticket/3410 . See > commit message for details. > > bye, > Sumit > This fixed the ipa-kdb crashes for me, so ACK. Martin From mkosek at redhat.com Tue Feb 12 09:55:42 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 12 Feb 2013 10:55:42 +0100 Subject: [Freeipa-devel] [PATCH] 363-368 Configurable SID blacklists In-Reply-To: <20130211175805.GD4506@redhat.com> References: <5114C9B0.6050608@redhat.com> <5114EA5E.2070800@redhat.com> <20130211143446.GA4506@redhat.com> <51190461.3070805@redhat.com> <20130211175805.GD4506@redhat.com> Message-ID: <511A119E.4070608@redhat.com> On 02/11/2013 06:58 PM, Alexander Bokovoy wrote: > On Mon, 11 Feb 2013, Martin Kosek wrote: >> On 02/11/2013 03:34 PM, Alexander Bokovoy wrote: >>> On Fri, 08 Feb 2013, Martin Kosek wrote: >>>> On 02/08/2013 10:47 AM, Martin Kosek wrote: >>>>> Sending patches according to RFE: >>>>> http://www.freeipa.org/page/V3/Configurable_SID_Blacklists >>>>> >>>>> How this works: >>>>> >>>>> 1) Trust is added, SID blacklist is filled with default list (by ipa-sam >>>>> plugin). When SID blacklist attribute is missing (e.g. for current trusts), >>>>> ipa-kdb will use the hardcoded list. >>>>> >>>>> # echo password | ipa trust-add MKAD2012.TEST --admin="Administrator" >>>>> --password >>>>> ---------------------------------------------- >>>>> Re-established trust to domain "MKAD2012.TEST" >>>>> ---------------------------------------------- >>>>> Realm name: MKAD2012.TEST >>>>> Domain NetBIOS name: MKAD2012 >>>>> Domain Security Identifier: S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx >>>>> SID blacklist incoming: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >>>>> S-1-5-3, S-1-5-4, S-1-5-5, >>>>> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >>>>> S-1-5-11, S-1-5-12, S-1-5-13, >>>>> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >>>>> S-1-5-19, S-1-5-20 >>>>> SID blacklist outgoing: S-1-0, S-1-1, S-1-2, S-1-3, S-1-5-1, S-1-5-2, >>>>> S-1-5-3, S-1-5-4, S-1-5-5, >>>>> S-1-5-6, S-1-5-7, S-1-5-8, S-1-5-9, S-1-5-10, >>>>> S-1-5-11, S-1-5-12, S-1-5-13, >>>>> S-1-5-14, S-1-5-15, S-1-5-16, S-1-5-17, S-1-5-18, >>>>> S-1-5-19, S-1-5-20 >>>>> Trust direction: Two-way trust >>>>> Trust type: Active Directory domain >>>>> Trust status: Established and verified >>>>> >>>>> 2) Incoming SID blacklist is updated (I added S-1-18-1 to the list as it is >>>>> included in MS-PAC when I log from AD 2012): >>>>> >>>>> # ipa trust-mod MKAD2012.TEST --sid-blacklist-incoming >>>>> S-1-0,S-1-1,S-1-2,S-1-3,S-1-5-1,S-1-5-2,S-1-5-3,S-1-5-4,S-1-5-5,S-1-5-6,S-1-5-7,S-1-5-8,S-1-5-9,S-1-5-10,S-1-5-11,S-1-5-12,S-1-5-13,S-1-5-14,S-1-5-15,S-1-5-16,S-1-5-17,S-1-5-18,S-1-5-19,S-1-5-20,S-1-18-1 >>>>> >>>>> >>>>> >>>>> 3) When I now login from AD2012 to my IPA machine, I get error message in >>>>> krb5kdc.log about the filtered SID I configured in LDAP: >>>>> >>>>> ... >>>>> Feb 08 04:11:33 ipa.linux.mkad2012.test krb5kdc[6493](Error): PAC filtering >>>>> issue: SID [S-1-18-1] is not allowed from a trusted source and will be >>>>> excluded. >>>>> ... >>>>> >>>>> NOTE: >>>>> When coding and testing this feature I fixed several related bugs I found in >>>>> ipa-kdb, see description of patches 363-365. >>>>> >>>>> Martin >>>>> >>>> >>>> I forgot to update ACI allowing Trust Admins to modify the blacklist. I also >>>> added a validator for SIDs to help catching invalid SIDs. >>>> >>>> Updated patches attached. >>> Work for me fine against Windows 2012 server. >>> >>> However, I'd like you to rebase on top of your previous patches. VERSION >>> file is causing conflict since your patchset for trustconfig command >>> increments to the same version as this one. >>> >> >> I pushed previous acked patch to master. Attaching patches 363-368 rebased on >> top of that. > ACK. > > Thanks a lot! > Thanks for review! Pushed to master, ipa-3-1. Martin From pspacek at redhat.com Tue Feb 12 09:57:52 2013 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 12 Feb 2013 10:57:52 +0100 Subject: [Freeipa-devel] DNS SOA serial managed by 389 DS plugin: design In-Reply-To: <1360599833.2299.100.camel@willson.li.ssimo.org> References: <51190246.8060900@redhat.com> <1360599833.2299.100.camel@willson.li.ssimo.org> Message-ID: <511A1220.3070204@redhat.com> On 11.2.2013 17:23, Simo Sorce wrote: > On Mon, 2013-02-11 at 15:37 +0100, Petr Spacek wrote: >> Possible optimization >> >> Increment serial value at most once per second. >> >> Basic idea: Write current timestamp (no incrementation) and write >> serial value >> to the database with one second delay. >> >> Problem: How to solve LDAP server crash? Write timestamp immediately >> and >> (while reading) subtract 1 if timestamp == serial? >> > I am not sure I understand the solution here ? > > Maybe a simpler solution is to always write the serial as time() + 1 ? I tried to solve following problem: (numbers represent time in "second.millisecond" format) 1.000 : new_serial = time() + 1 1.100 : record test.example.com. updated 1.100 : zone serial overwritten with new_serial 1.500 : zone transfer started 1.500 : search result for all records and zone serial stored 1.500 : search result is transferred to slaves 1.700 : record test2.example.com. updated 1.700 : zone serial overwritten with new_serial Result: Zones on master and it's slave servers have serial = "new_serial" but the zone content is different (records under test2.example.com. are not equal). What I tried to describe above is that search for zone serial returns: if (serial value == time()) return (serial value - 1) else return (serial value) Example: 1.000 : new_serial = time() 1.100 : record test.example.com. updated 1.100 : zone serial overwritten with new_serial 1.500 : zone transfer started 1.500 : search result for all records and zone serial stored 1.500 : zone serial in search result is (new_serial - 1) 1.500 : snapshot is transferred to slaves 1.700 : record test2.example.com. updated 1.700 : zone serial overwritten with new_serial 2.000 : now the search for serial value returns new_serial, i.e. slaves see value incremented by one from last zone transfer (1.500) 9.000 : serial value is unchanged from last search Expected result: Zone data can be inconsistent between master and slaves for only one second. Data will be consistent if directory server crashed at 1.701 - new zone transfer can be initiated after server restart. Requirement: DS plugin have to modify serial value during reads. Does getimeofday() cost too much to do it for all read-serial operations? Makes it sense? -- Petr^2 Spacek From mkosek at redhat.com Tue Feb 12 11:24:48 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 12 Feb 2013 12:24:48 +0100 Subject: [Freeipa-devel] [PATCH] 370 ipa-kdb: remove memory leaks Message-ID: <511A2680.1030300@redhat.com> All known memory leaks caused by unfreed allocated memory or unfreed LDAP results (which should be also done after unsuccessful searches) are fixed. One ipadb_need_retry result check was fixed as this function returns trust in case of a need for retry and not a zero. https://fedorahosted.org/freeipa/ticket/3413 --- This patch should be combined with Sumit's patch 0093 which would prevent kdb5kdc from crashing when it is being terminted. If you want to run krb5kdc in valgrind, check the command in the ticket which would let valgrind also show symbols in dlopen'ed ipa-kdb plugin. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-370-ipa-kdb-remove-memory-leaks.patch Type: text/x-patch Size: 6378 bytes Desc: not available URL: From pspacek at redhat.com Tue Feb 12 11:57:44 2013 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 12 Feb 2013 12:57:44 +0100 Subject: [Freeipa-devel] [PATCH 0111] Automatically reload invalid zone after each change in zone data Message-ID: <511A2E38.10006@redhat.com> Hello, Automatically reload invalid zone after each change in zone data. https://fedorahosted.org/bind-dyndb-ldap/ticket/102 How to test: # create a invalid zone, e.g. zone without A records for names in NS records ipa dnszone-add zone.test --admin-email=blah.nonsense --name-server=ns.zone.test. --force # now dig ns.zone.test. should return SERVFAIL because zone doesn't have proper NS+A/AAAA records dig ns.zone.test. # addition of arbitrary record should not crash the server ipa dnsrecord-add zone.test ns --txt-rec=blah # after this modification some error message should appear in log and dig ns.zone.test. should still return SERVFAIL dig ns.zone.test. # addition of valid A record should fix the problem ipa dnsrecord-add zone.test ns --a-rec=127.0.0.1 # now dig -t ANY ns.zone.test. should return NOERROR and zone should work # TXT and also A record should be visible dig -t ANY ns.zone.test. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0111-Automatically-reload-invalid-zone-after-each-change-.patch Type: text/x-patch Size: 3553 bytes Desc: not available URL: From pspacek at redhat.com Tue Feb 12 12:58:44 2013 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 12 Feb 2013 13:58:44 +0100 Subject: [Freeipa-devel] [PATCH 0112] Make log messages related to Kerberos more verbose Message-ID: <511A3C84.4020205@redhat.com> Hello, Make log messages related to Kerberos more verbose. This change should help people supporting bind-dyndb-ldap to figure out what is happening under covers. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0112-Make-log-messages-related-to-Kerberos-more-verbose.patch Type: text/x-patch Size: 4166 bytes Desc: not available URL: From sbose at redhat.com Tue Feb 12 13:18:50 2013 From: sbose at redhat.com (Sumit Bose) Date: Tue, 12 Feb 2013 14:18:50 +0100 Subject: [Freeipa-devel] [PATCH] 370 ipa-kdb: remove memory leaks In-Reply-To: <511A2680.1030300@redhat.com> References: <511A2680.1030300@redhat.com> Message-ID: <20130212131850.GM24753@localhost.localdomain> On Tue, Feb 12, 2013 at 12:24:48PM +0100, Martin Kosek wrote: > All known memory leaks caused by unfreed allocated memory or unfreed > LDAP results (which should be also done after unsuccessful searches) > are fixed. > > One ipadb_need_retry result check was fixed as this function returns > trust in case of a need for retry and not a zero. > > https://fedorahosted.org/freeipa/ticket/3413 > > --- > > This patch should be combined with Sumit's patch 0093 which would prevent > kdb5kdc from crashing when it is being terminted. > > If you want to run krb5kdc in valgrind, check the command in the ticket which > would let valgrind also show symbols in dlopen'ed ipa-kdb plugin. > > Martin I haven't look closely so far, but I see the following compiler warning: ipa_kdb_common.c: In function 'ipadb_simple_search': ipa_kdb_common.c:175:9: warning: passing argument 1 of 'ldap_msgfree' from incompatible pointer type [enabled by default] In file included from ipa_kdb.h:35:0, from ipa_kdb_common.c:23: /usr/include/ldap.h:1848:1: note: expected 'struct LDAPMessage *' but argument is of type 'struct LDAPMessage **' bye, Sumit From simo at redhat.com Tue Feb 12 13:20:54 2013 From: simo at redhat.com (Simo Sorce) Date: Tue, 12 Feb 2013 08:20:54 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <51199B3F.2080808@redhat.com> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> Message-ID: <1360675254.3838.8.camel@willson.li.ssimo.org> On Mon, 2013-02-11 at 20:30 -0500, Dmitri Pal wrote: > On 02/11/2013 03:21 PM, Simo Sorce wrote: > > On Mon, 2013-02-11 at 21:03 +0100, Ondrej Hamada wrote: > >> Dne 3.2.2013 02:51, Dmitri Pal napsal(a): > >>> On 01/31/2013 06:09 PM, Ondrej Hamada wrote: > >>>> Hello, > >>>> I'm starting to work on my thesis about 'More types of replicas in > >>>> FreeIPA' again. One of the main problems is the way how should the > >>>> read-only replicas deal with KDC because they're not supposed to > >>>> posses the Kerberos (krb) master key. The task was to investigate how > >>>> is this solved in Active Directory and its Read Only Domain Controllers. > >>>> > >>>> I found out that the basic of RODC behaviour is described on technet > >>>> page > >>>> (http://technet.microsoft.com/en-us/library/cc754218%28v=ws.10%29.aspx). > >>>> > >>>> Login situation: > >>>> RODC by default forwards the KRB requests to the DC. RODC then > >>>> forwards the response back to the client and also requests the > >>>> password to be replicated to RODC. Both the user and his host must be > >>>> members of 'Allowed RODC Password Replication' group in order to let > >>>> user's passwords being replicated to RODCs. > >>>> > >>>> Request services that the RODC doesn't have credentials for: > >>>> Client sends TGS-REQ to RODC. RODC can read the TGT in the request, > >>>> but doesn't have credentials for the service. So the request is > >>>> forwarded to the DC. DC can decrypt the TGT that was created by RODC > >>>> and sends back the TGS-RES that is forwarded to the client. (but it > >>>> does not trust the RODC so it recalculates the privilege attribute > >>>> certificate). RODC does not cache the credentials for the service. > >>>> > >>>> During my experiments the credentials got replicated to the RODC on > >>>> the first log on of the user. The user's KRB requests were first > >>>> forwarded to the DC. When the user got krbtgt and TGS for host, ldap > >>>> and cifs, his TGT was revoked by RODC. He run through the auth. > >>>> process again, but this time the requests were served by RODC only - > >>>> no forwarding - and not TGS for host was requested. > >>>> > >>>> Unfortunately I can not still recognize how the keys are processed. > >>>> There's barely any RPC communication - only one DCERPC packet exchange > >>>> between RODC and DC that takes place when the user sends his first TGS > >>>> request (this exchange happens also for the clients with disabled > >>>> replication). > >>>> > >>>> It looks to me like the DC knows all the RODC keys. According to > >>>> Technet, the MS implementation of Kerberos is able to recognize the > >>>> key owner from the Key Version Number value. > >>>> > >>>> I think I can't get more info from the network traffic examination. Do > >>>> you have any ideas or hints on further investigation of the problem? > >>> The page you listed shows the diagrams of the user login and TGT and > >>> then TGS acquisition. > >>> http://technet.microsoft.com/en-us/library/cc754218%28v=WS.10%29#BKMK_AuthRODC > >>> Also the following thread sheds some good light on how the > >>> authentication and caching happens in case of the RODC. > >>> http://social.technet.microsoft.com/forums/en-US/winserverDS/thread/f8d1017e-1f0e-4a9d-a241-b03b508dfe17 > >>> So they have policy driven replication of passwords. > >>> If password is allowed to be replicated by policy it is replicated if > >>> not it RODC has to always proxy to RWDC for this account. > >>> The password changes always happen on RWDC and replicated back. > >>> They can be replicated by RSO operation that allows replicating a single > >>> object. > >>> > >>> It seems that they assume that all the services are always remote thus > >>> connectivity to the central RWDC is a must. > >>> They do not seem to keep any service keys locally in the RODC. > >>> > >>> With SSSD in play on the client I am not sure we should worry about > >>> caching at least in the first implementation. > >>> So in our case it might make sense to have a proxy KDC and a RO replica > >>> with the subset of data replicated to it. > >>> I wonder if you can accomplish that by taking 389 RO replica + IAKERB > >>> I suggest you look at IAKERB and see if it can be used as a proxy for > >>> user authentication, password change and TGS requests. > >>> If not may be we can at least use it as an inspiration. > >>> > >>> The attached diagram shows what I mean. > >>> > >>> Later we can add some logic for the following: > >>> a) RODC requesting replication of a specific object > >>> b) RWDC replicating a specific object > >>> c) Policy to define for which accounts the passwords and keys can replicated > >>> d) RODC getting policy and performing local authentication for the > >>> accounts for which the keys were replicated. > >>> > >>> However with SSSD on the client it might be easier for KDC proxy just to > >>> go offline and not respond to the SSSD if it loses connection to the > >>> central server. > >>> That would force SSSD to go offline and use local password caching. I > >>> suspect that SSSD password caching would be enabled in many cases > >>> anyways so not caching all passwords in one central locating in RODC > >>> might actually be a good thing. > >>> > >> Thanks for hints. I was looking at IAKERB and according to RFC it should > >> support both AS and TGS requests/replies so it might be doable. I'll try > >> to prepare a prototype that will be using 389 RO replica and IAKERB as > >> you proposed. > > IAKERB is a GSSAPI level interface, it is not something you can use as a > > KDC Proxy on its own. > > An actual KDC proxy is what is needed because you cannot change > > applications at will and there are a number of applications that just go > > and talk straight to a KDC server to do AS and TGS requests. > > > > A KDC proxy is possible to build, it just require some work in the > > Actual KDC and probably some custom protocol to talk between KDCs. > > We should look into what is the simplest technical way to achieve this > > and perhaps propose it for standardization in IETF. > > > > Simo. > > > Really? I was hoping that IAKERB is exactly what you have described. > Are you saying that IAKERB requires changes to the client? > That sounds strange. Clients using GSSAPI may not require may not require changes, but not all Kerberos clients use a modern GSSAPI library. Even SSSD uses direct Kerberos calls,a s traditionally there is no way to get initial tickets via GSSAPI. That is slowly changing but it will be a long time before that happens. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Feb 12 14:16:03 2013 From: simo at redhat.com (Simo Sorce) Date: Tue, 12 Feb 2013 09:16:03 -0500 Subject: [Freeipa-devel] [PATCH] 370 ipa-kdb: remove memory leaks In-Reply-To: <511A2680.1030300@redhat.com> References: <511A2680.1030300@redhat.com> Message-ID: <1360678563.3838.31.camel@willson.li.ssimo.org> On Tue, 2013-02-12 at 12:24 +0100, Martin Kosek wrote: Comments inline. > --- a/daemons/ipa-kdb/ipa_kdb_common.c > +++ b/daemons/ipa-kdb/ipa_kdb_common.c > @@ -172,7 +172,7 @@ krb5_error_code ipadb_simple_search(struct > ipadb_context *ipactx, > /* first test if we need to retry to connect */ > if (ret != 0 && > ipadb_need_retry(ipactx, ret)) { > - > + ldap_msgfree(res); Why do we need this ? We get in this branch only if the previous search failed, so res should always be empty. What am I missing ? > ret = ldap_search_ext_s(ipactx->lcontext, basedn, scope, > filter, attrs, 0, NULL, NULL, > &std_timeout, LDAP_NO_LIMIT, > @@ -283,6 +283,7 @@ krb5_error_code ipadb_deref_search(struct > ipadb_context *ipactx, > int times; > int ret; > int c, i; > + bool retry; > > for (c = 0; deref_attr_names[c]; c++) { > /* count */ ; > @@ -315,7 +316,8 @@ krb5_error_code ipadb_deref_search(struct > ipadb_context *ipactx, > /* retry once if connection errors (tot. max. 2 tries) */ > times = 2; > ret = LDAP_SUCCESS; > - while (!ipadb_need_retry(ipactx, ret) && times > 0) { > + retry = true; > + while (retry) { > times--; > ret = ldap_search_ext_s(ipactx->lcontext, base_dn, > scope, filter, > @@ -323,11 +325,18 @@ krb5_error_code ipadb_deref_search(struct > ipadb_context *ipactx, > ctrl, NULL, > &std_timeout, LDAP_NO_LIMIT, > res); > + retry = ipadb_need_retry(ipactx, ret) && times > 0; > + > + if (retry) { > + /* Free result before next try */ > + ldap_msgfree(*res); > + } > } This snippet seem to change the logic. Before the condition was !ipadb_need_retry() and no you change it to ipadb_need_retry() so it looks reversed, was this on purpose ? > kerr = ipadb_simple_ldap_to_kerr(ret); > > done: > + ldap_control_free(ctrl[0]); > ldap_memfree(derefval.bv_val); > free(ds); > return kerr; > diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c > b/daemons/ipa-kdb/ipa_kdb_mspac.c > index > 0780e81cb5507ed590cc9b0646ba5919c0084523..6abd51ec019ef45dd52d317b85dcb9584b49f06f 100644 > --- a/daemons/ipa-kdb/ipa_kdb_mspac.c > +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c > @@ -944,6 +944,7 @@ static int map_groups(TALLOC_CTX *memctx, > krb5_context kcontext, > goto done; > } > > + ldap_msgfree(results); > kerr = ipadb_deref_search(ipactx, basedn, LDAP_SCOPE_ONE, > filter, > entry_attrs, deref_search_attrs, > memberof_pac_attrs, &results); > @@ -1636,14 +1637,24 @@ krb5_error_code > ipadb_sign_authdata(krb5_context context, > /* put in signed data */ > ad.magic = KV5M_AUTHDATA; > ad.ad_type = KRB5_AUTHDATA_WIN2K_PAC; > - ad.contents = (krb5_octet *)pac_data.data; > + ad.contents = malloc(pac_data.length); > + if (ad.contents == NULL) { > + krb5_free_data_contents(context, &pac_data); > + ad.length = 0; > + kerr = ENOMEM; > + goto done; > + } > + memcpy(ad.contents, pac_data.data, pac_data.length); > ad.length = pac_data.length; > + krb5_free_data_contents(context, &pac_data); > + Why all this copying around ? It should be sufficient to call krb5_free_data_contents() at the end at most. > authdata[0] = &ad; > > kerr = krb5_encode_authdata_container(context, > KRB5_AUTHDATA_IF_RELEVANT, > authdata, > signed_auth_data); > + free(ad.contents); > if (kerr != 0) { > goto done; > } The rest looks good. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Tue Feb 12 14:25:57 2013 From: simo at redhat.com (Simo Sorce) Date: Tue, 12 Feb 2013 09:25:57 -0500 Subject: [Freeipa-devel] DNS SOA serial managed by 389 DS plugin: design In-Reply-To: <511A1220.3070204@redhat.com> References: <51190246.8060900@redhat.com> <1360599833.2299.100.camel@willson.li.ssimo.org> <511A1220.3070204@redhat.com> Message-ID: <1360679157.3838.32.camel@willson.li.ssimo.org> On Tue, 2013-02-12 at 10:57 +0100, Petr Spacek wrote: > On 11.2.2013 17:23, Simo Sorce wrote: > > On Mon, 2013-02-11 at 15:37 +0100, Petr Spacek wrote: > >> Possible optimization > >> > >> Increment serial value at most once per second. > >> > >> Basic idea: Write current timestamp (no incrementation) and write > >> serial value > >> to the database with one second delay. > >> > >> Problem: How to solve LDAP server crash? Write timestamp immediately > >> and > >> (while reading) subtract 1 if timestamp == serial? > >> > > I am not sure I understand the solution here ? > > > > Maybe a simpler solution is to always write the serial as time() + 1 ? > > I tried to solve following problem: > (numbers represent time in "second.millisecond" format) > > 1.000 : new_serial = time() + 1 > 1.100 : record test.example.com. updated > 1.100 : zone serial overwritten with new_serial > 1.500 : zone transfer started > 1.500 : search result for all records and zone serial stored > 1.500 : search result is transferred to slaves > 1.700 : record test2.example.com. updated > 1.700 : zone serial overwritten with new_serial > > > Result: > Zones on master and it's slave servers have serial = "new_serial" but the zone > content is different (records under test2.example.com. are not equal). > > > > What I tried to describe above is that search for zone serial returns: > if (serial value == time()) > return (serial value - 1) > else > return (serial value) > > Example: > 1.000 : new_serial = time() > 1.100 : record test.example.com. updated > 1.100 : zone serial overwritten with new_serial > 1.500 : zone transfer started > 1.500 : search result for all records and zone serial stored > 1.500 : zone serial in search result is (new_serial - 1) > 1.500 : snapshot is transferred to slaves > 1.700 : record test2.example.com. updated > 1.700 : zone serial overwritten with new_serial > > 2.000 : now the search for serial value returns new_serial, i.e. slaves see > value incremented by one from last zone transfer (1.500) > 9.000 : serial value is unchanged from last search > > Expected result: > Zone data can be inconsistent between master and slaves for only one second. > Data will be consistent if directory server crashed at 1.701 - new zone > transfer can be initiated after server restart. > > Requirement: > DS plugin have to modify serial value during reads. Does getimeofday() cost > too much to do it for all read-serial operations? > > Makes it sense? Thanks, makes a lot of sense. Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Tue Feb 12 15:14:36 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 12 Feb 2013 16:14:36 +0100 Subject: [Freeipa-devel] [PATCH] 370 ipa-kdb: remove memory leaks In-Reply-To: <1360678563.3838.31.camel@willson.li.ssimo.org> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> Message-ID: <511A5C5C.3030807@redhat.com> On 02/12/2013 03:16 PM, Simo Sorce wrote: > On Tue, 2013-02-12 at 12:24 +0100, Martin Kosek wrote: > > Comments inline. > >> --- a/daemons/ipa-kdb/ipa_kdb_common.c >> +++ b/daemons/ipa-kdb/ipa_kdb_common.c >> @@ -172,7 +172,7 @@ krb5_error_code ipadb_simple_search(struct >> ipadb_context *ipactx, >> /* first test if we need to retry to connect */ >> if (ret != 0 && >> ipadb_need_retry(ipactx, ret)) { >> - >> + ldap_msgfree(res); > > Why do we need this ? > We get in this branch only if the previous search failed, so res should > always be empty. What am I missing ? Explained in the commit description - this may not be super-critical, I just followed info in ldap_search_ext() man page: ... Note that res parameter of ldap_search_ext_s() and ldap_search_s() should be freed with ldap_msgfree() regardless of return value of these functions. ... > >> ret = ldap_search_ext_s(ipactx->lcontext, basedn, scope, >> filter, attrs, 0, NULL, NULL, >> &std_timeout, LDAP_NO_LIMIT, >> @@ -283,6 +283,7 @@ krb5_error_code ipadb_deref_search(struct >> ipadb_context *ipactx, >> int times; >> int ret; >> int c, i; >> + bool retry; >> >> for (c = 0; deref_attr_names[c]; c++) { >> /* count */ ; >> @@ -315,7 +316,8 @@ krb5_error_code ipadb_deref_search(struct >> ipadb_context *ipactx, >> /* retry once if connection errors (tot. max. 2 tries) */ >> times = 2; >> ret = LDAP_SUCCESS; >> - while (!ipadb_need_retry(ipactx, ret) && times > 0) { >> + retry = true; >> + while (retry) { >> times--; >> ret = ldap_search_ext_s(ipactx->lcontext, base_dn, >> scope, filter, >> @@ -323,11 +325,18 @@ krb5_error_code ipadb_deref_search(struct >> ipadb_context *ipactx, >> ctrl, NULL, >> &std_timeout, LDAP_NO_LIMIT, >> res); >> + retry = ipadb_need_retry(ipactx, ret) && times > 0; >> + >> + if (retry) { >> + /* Free result before next try */ >> + ldap_msgfree(*res); >> + } >> } > > This snippet seem to change the logic. > > Before the condition was !ipadb_need_retry() and no you change it to > ipadb_need_retry() so it looks reversed, was this on purpose ? As noted in commit description, this check was wrong and it was causing the function to _always_ retry at least once because ipadb_need_retry returns true when we need to retry and not 0 -> thus no "!" is needed. > >> kerr = ipadb_simple_ldap_to_kerr(ret); >> >> done: >> + ldap_control_free(ctrl[0]); >> ldap_memfree(derefval.bv_val); >> free(ds); >> return kerr; >> diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c >> b/daemons/ipa-kdb/ipa_kdb_mspac.c >> index >> 0780e81cb5507ed590cc9b0646ba5919c0084523..6abd51ec019ef45dd52d317b85dcb9584b49f06f 100644 >> --- a/daemons/ipa-kdb/ipa_kdb_mspac.c >> +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c >> @@ -944,6 +944,7 @@ static int map_groups(TALLOC_CTX *memctx, >> krb5_context kcontext, >> goto done; >> } >> >> + ldap_msgfree(results); >> kerr = ipadb_deref_search(ipactx, basedn, LDAP_SCOPE_ONE, >> filter, >> entry_attrs, deref_search_attrs, >> memberof_pac_attrs, &results); >> @@ -1636,14 +1637,24 @@ krb5_error_code >> ipadb_sign_authdata(krb5_context context, >> /* put in signed data */ >> ad.magic = KV5M_AUTHDATA; >> ad.ad_type = KRB5_AUTHDATA_WIN2K_PAC; >> - ad.contents = (krb5_octet *)pac_data.data; >> + ad.contents = malloc(pac_data.length); >> + if (ad.contents == NULL) { >> + krb5_free_data_contents(context, &pac_data); >> + ad.length = 0; >> + kerr = ENOMEM; >> + goto done; >> + } >> + memcpy(ad.contents, pac_data.data, pac_data.length); >> ad.length = pac_data.length; >> + krb5_free_data_contents(context, &pac_data); >> + > > Why all this copying around ? > It should be sufficient to call krb5_free_data_contents() at the end at > most. You are right, I overengineered this one. Fixed. > > >> authdata[0] = &ad; >> >> kerr = krb5_encode_authdata_container(context, >> KRB5_AUTHDATA_IF_RELEVANT, >> authdata, >> signed_auth_data); >> + free(ad.contents); >> if (kerr != 0) { >> goto done; >> } > > The rest looks good. > > Simo. > Attaching a patch fixing this one + an issue found by Sumit (sigh, quick-fix-before-send type of error). Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-370-2-ipa-kdb-remove-memory-leaks.patch Type: text/x-patch Size: 6021 bytes Desc: not available URL: From simo at redhat.com Tue Feb 12 15:26:28 2013 From: simo at redhat.com (Simo Sorce) Date: Tue, 12 Feb 2013 10:26:28 -0500 Subject: [Freeipa-devel] [PATCH] 370 ipa-kdb: remove memory leaks In-Reply-To: <511A5C5C.3030807@redhat.com> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> <511A5C5C.3030807@redhat.com> Message-ID: <1360682788.3838.45.camel@willson.li.ssimo.org> On Tue, 2013-02-12 at 16:14 +0100, Martin Kosek wrote: > Explained in the commit description - this may not be super-critical, I just > followed info in ldap_search_ext() man page: > > ... > > Note that res parameter of ldap_search_ext_s() and ldap_search_s() > should be freed with > ldap_msgfree() regardless of return value of these functions. > ... OK. > > This snippet seem to change the logic. > > > > Before the condition was !ipadb_need_retry() and no you change it to > > ipadb_need_retry() so it looks reversed, was this on purpose ? > > As noted in commit description, this check was wrong and it was causing the > function to _always_ retry at least once because ipadb_need_retry returns true > when we need to retry and not 0 -> thus no "!" is needed. I do not like much 'sneaking' this kind of change in the same patch that fixes memory leaks, so if you want to plit in 2 patches it would be nice, but I am of course ok with the change. > > Why all this copying around ? > > It should be sufficient to call krb5_free_data_contents() at the end at > > most. > > You are right, I overengineered this one. Fixed. Much better now, thanks. > Attaching a patch fixing this one + an issue found by Sumit (sigh, > quick-fix-before-send type of error). Looks good to me now. Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Tue Feb 12 15:42:52 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 12 Feb 2013 16:42:52 +0100 Subject: [Freeipa-devel] [PATCH] 370-371 ipa-kdb: remove memory leaks In-Reply-To: <1360682788.3838.45.camel@willson.li.ssimo.org> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> <511A5C5C.3030807@redhat.com> <1360682788.3838.45.camel@willson.li.ssimo.org> Message-ID: <511A62FC.5080907@redhat.com> On 02/12/2013 04:26 PM, Simo Sorce wrote: > On Tue, 2013-02-12 at 16:14 +0100, Martin Kosek wrote: >> Explained in the commit description - this may not be super-critical, I just >> followed info in ldap_search_ext() man page: >> >> ... >> >> Note that res parameter of ldap_search_ext_s() and ldap_search_s() >> should be freed with >> ldap_msgfree() regardless of return value of these functions. >> ... > > OK. > >>> This snippet seem to change the logic. >>> >>> Before the condition was !ipadb_need_retry() and no you change it to >>> ipadb_need_retry() so it looks reversed, was this on purpose ? >> >> As noted in commit description, this check was wrong and it was causing the >> function to _always_ retry at least once because ipadb_need_retry returns true >> when we need to retry and not 0 -> thus no "!" is needed. > > I do not like much 'sneaking' this kind of change in the same patch that > fixes memory leaks, so if you want to plit in 2 patches it would be > nice, but I am of course ok with the change. I agree with that. Lets do it right as we have not pushed it yet. I moved this change to a separate patch (attached). Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-370-3-ipa-kdb-remove-memory-leaks.patch Type: text/x-patch Size: 5906 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-371-ipa-kdb-fix-retry-logic-in-ipadb_deref_search.patch Type: text/x-patch Size: 1221 bytes Desc: not available URL: From abokovoy at redhat.com Tue Feb 12 15:48:42 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 12 Feb 2013 17:48:42 +0200 Subject: [Freeipa-devel] [PATCH] 361 ipa-adtrust-install should ask for SID generation In-Reply-To: <510B720E.7070404@redhat.com> References: <510A7C14.4080701@redhat.com> <20130131152910.GG4506@redhat.com> <510A94A9.8040708@redhat.com> <20130131180611.GI4506@redhat.com> <510B720E.7070404@redhat.com> Message-ID: <20130212154842.GA476@redhat.com> On Fri, 01 Feb 2013, Martin Kosek wrote: >On 01/31/2013 07:06 PM, Alexander Bokovoy wrote: >> On Thu, 31 Jan 2013, Martin Kosek wrote: >>> On 01/31/2013 04:29 PM, Alexander Bokovoy wrote: >>>> On Thu, 31 Jan 2013, Martin Kosek wrote: >>>>> When ipa-adtrust-install is run, check if there are any objects >>>>> that need to have SID generated. If yes, interactively ask the user >>>>> if the sidgen task should be run. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/3195 >>>> >>> ... >>>> I would still run this check in options.unattended mode and reported >>>> warning, for accounting purposes. >>>> >>>> Could you please make so? >>>> >>> >>> Sure! Updated patch attached. >> Thanks! I have only small addition: >> >>> + object_count = len(entries) >>> + if object_count > 0: >>> + print "" >>> + print "WARNING: %d existing users or groups do not have a >>> SID identifier assigned." \ >>> + % len(entries) >>> + print "Installer can run a task to have ipa-sidgen Directory >>> Server plugin generate" >>> + print "the SID identifier for all these users. Please note, >>> the in case of a high" >>> + print "number of users and groups, the operation might lead >>> to high replication" >>> + print "traffic and performance degradation. Refer to >>> ipa-adtrust-install(1) man page" >>> + print "for details." >>> + print "" >>> + if not options.unattended: >>> + if ipautil.user_input("Do you want to run the ipa-sidgen >>> task?", default=False, >>> + allow_empty=False): >>> + options.add_sids = True >> ... to make the text of warning consistent it would be good to add >> + else: >> + print "Unattended mode was selected, installer will *not* >> run ipa-sidgen task!" >> > >And here is the updated patch. ACK. I actually tested it already with other patches just forgot to reply to this email. -- / Alexander Bokovoy From edewata at redhat.com Tue Feb 12 16:14:12 2013 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 12 Feb 2013 10:14:12 -0600 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <5114FD48.3070104@redhat.com> References: <5114FD48.3070104@redhat.com> Message-ID: <511A6A54.1040307@redhat.com> On 2/8/2013 7:27 AM, Petr Vobornik wrote: > Checkbox for NONE option was added. > > https://fedorahosted.org/freeipa/ticket/3404 > > Patches for master and 3.1 branch attached. ACK. -- Endi S. Dewata From mkosek at redhat.com Tue Feb 12 16:30:22 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 12 Feb 2013 17:30:22 +0100 Subject: [Freeipa-devel] [PATCH] 361 ipa-adtrust-install should ask for SID generation In-Reply-To: <20130212154842.GA476@redhat.com> References: <510A7C14.4080701@redhat.com> <20130131152910.GG4506@redhat.com> <510A94A9.8040708@redhat.com> <20130131180611.GI4506@redhat.com> <510B720E.7070404@redhat.com> <20130212154842.GA476@redhat.com> Message-ID: <511A6E1E.6000008@redhat.com> On 02/12/2013 04:48 PM, Alexander Bokovoy wrote: > On Fri, 01 Feb 2013, Martin Kosek wrote: >> On 01/31/2013 07:06 PM, Alexander Bokovoy wrote: >>> On Thu, 31 Jan 2013, Martin Kosek wrote: >>>> On 01/31/2013 04:29 PM, Alexander Bokovoy wrote: >>>>> On Thu, 31 Jan 2013, Martin Kosek wrote: >>>>>> When ipa-adtrust-install is run, check if there are any objects >>>>>> that need to have SID generated. If yes, interactively ask the user >>>>>> if the sidgen task should be run. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/3195 >>>>> >>>> ... >>>>> I would still run this check in options.unattended mode and reported >>>>> warning, for accounting purposes. >>>>> >>>>> Could you please make so? >>>>> >>>> >>>> Sure! Updated patch attached. >>> Thanks! I have only small addition: >>> >>>> + object_count = len(entries) >>>> + if object_count > 0: >>>> + print "" >>>> + print "WARNING: %d existing users or groups do not have a >>>> SID identifier assigned." \ >>>> + % len(entries) >>>> + print "Installer can run a task to have ipa-sidgen Directory >>>> Server plugin generate" >>>> + print "the SID identifier for all these users. Please note, >>>> the in case of a high" >>>> + print "number of users and groups, the operation might lead >>>> to high replication" >>>> + print "traffic and performance degradation. Refer to >>>> ipa-adtrust-install(1) man page" >>>> + print "for details." >>>> + print "" >>>> + if not options.unattended: >>>> + if ipautil.user_input("Do you want to run the ipa-sidgen >>>> task?", default=False, >>>> + allow_empty=False): >>>> + options.add_sids = True >>> ... to make the text of warning consistent it would be good to add >>> + else: >>> + print "Unattended mode was selected, installer will *not* >>> run ipa-sidgen task!" >>> >> >> And here is the updated patch. > ACK. > > I actually tested it already with other patches just forgot to reply to > this email. > Pushed to master, ipa-3-1. Martin From tbabej at redhat.com Tue Feb 12 16:50:33 2013 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 12 Feb 2013 17:50:33 +0100 Subject: [Freeipa-devel] [PATCH 0034] Deny LDAP binds for user accounts with expired principal Message-ID: <511A72D9.6090506@redhat.com> Hi, This patch adds a check for krbprincipalexpiration attribute to pre_bind operation in ipa-pwd-extop dirsrv plugin. If the principal is expired, auth is denied and LDAP_INVALID_CREDENTIALS along with the error message is sent back to the client. Since krbprincipalexpiration attribute is not mandatory, if there is no value set, the check is passed. https://fedorahosted.org/freeipa/ticket/3305 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0034-Deny-LDAP-binds-for-user-accounts-with-expired-princ.patch Type: text/x-patch Size: 3962 bytes Desc: not available URL: From pvoborni at redhat.com Tue Feb 12 16:56:23 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 12 Feb 2013 17:56:23 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <511A6A54.1040307@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> Message-ID: <511A7437.8000207@redhat.com> On 02/12/2013 05:14 PM, Endi Sukma Dewata wrote: > On 2/8/2013 7:27 AM, Petr Vobornik wrote: >> Checkbox for NONE option was added. >> >> https://fedorahosted.org/freeipa/ticket/3404 >> >> Patches for master and 3.1 branch attached. > > ACK. > We were discussing to NACK this approach. The implementation should be improved because of the mutually exclusive nature of NONE option with [MS-PAC, PAD] options. I think we should add spec definition (to Web UI only, or into server plugin as well) of mutex options. Something like: mutex_groups: [[['NONE'],['MS-PAC', 'PAD']], [/*another array of groups*/]]; basically an array of group arrays where group array would contain arrays of mutually exclusive option. Is it over-complicated? Would one array or a pair of groups be enough? That way we can disable and uncheck(should not happen, but to keep data consistency) checkboxes of options from other groups when user selects a value of some group. As a separate issue, we might want to fetch the options from metadata, when we want to show the values and don't care about creating different labels. -- Petr Vobornik From abokovoy at redhat.com Tue Feb 12 17:00:13 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 12 Feb 2013 19:00:13 +0200 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <51151B27.8020103@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> Message-ID: <20130212170013.GB476@redhat.com> On Fri, 08 Feb 2013, Tomas Babej wrote: > On 02/08/2013 03:25 PM, Alexander Bokovoy wrote: >> On Mon, 04 Feb 2013, Tomas Babej wrote: >>> Hi, >>> >>> When adding/modifying an ID range for a trusted domain, the newly >>> added option --dom-name can be used. This looks up SID of the >>> trusted domain in LDAP and therefore the user is not required >>> to write it down in CLI. If the lookup fails, error message >>> asking the user to specify the SID manually is shown. >>> >>> https://fedorahosted.org/freeipa/ticket/3133 >>> >>> Tomas >> >>>> From 72f8802953edaaf5b9f7c34a38601fbccd681c8e Mon Sep 17 00:00:00 2001 >>> From: Tomas Babej >>> Date: Mon, 4 Feb 2013 08:33:53 -0500 >>> Subject: [PATCH] Add option to specify SID using domain name to >>> idrange-add/mod >>> >>> When adding/modifying an ID range for a trusted domain, the newly >>> added option --dom-name can be used. This looks up SID of the >>> trusted domain in LDAP and therefore the user is not required >>> to write it down in CLI. If the lookup fails, error message >>> asking the user to specify the SID manually is shown. >>> >>> https://fedorahosted.org/freeipa/ticket/3133 >>> --- >>> ipalib/plugins/idrange.py | 78 >>> +++++++++++++++++++++++++++++++++++++++++------ >>> ipaserver/dcerpc.py | 10 ++++++ >>> 2 files changed, 78 insertions(+), 10 deletions(-) >>> >>> diff --git a/ipalib/plugins/idrange.py b/ipalib/plugins/idrange.py >>> index 84e1057ac6b59b8ad99882a54e3288897338c978..77a75e4cabc18ca873be7cadcf870427d5b36ea0 >>> 100644 >>> --- a/ipalib/plugins/idrange.py >>> +++ b/ipalib/plugins/idrange.py >>> @@ -197,6 +197,11 @@ class idrange(LDAPObject): >>> cli_name='dom_sid', >>> label=_('Domain SID of the trusted domain'), >>> ), >>> + Str('ipanttrusteddomainname?', >>> + cli_name='dom_name', >>> + flags=('no_search', 'virtual_attribute'), >>> + label=_('Name of the trusted domain'), >>> + ), >> New options is added but API.txt wasn't changed. As result, 'make rpms' >> does not work. >> >> Could you please fix the patch and re-send it? >> > Sorry about that. > > Updated patch attached. I have one small question regarding use of dom_sid/dom_name. If both dom_sid and dom_name were specified, failing to resolve dom_name would force command to raise exception. I'm not sure this is right behavior. Probably we should detect that both dom_sid and dom_name were specified and bail out earlier so that only one of them is accepted. That would be clearer to users, wouldn't it? -- / Alexander Bokovoy From tbabej at redhat.com Tue Feb 12 17:03:44 2013 From: tbabej at redhat.com (Tomas Babej) Date: Tue, 12 Feb 2013 18:03:44 +0100 Subject: [Freeipa-devel] [PATCH 0034] Deny LDAP binds for user accounts with expired principal In-Reply-To: <511A72D9.6090506@redhat.com> References: <511A72D9.6090506@redhat.com> Message-ID: <511A75F0.7070801@redhat.com> On 02/12/2013 05:50 PM, Tomas Babej wrote: > Hi, > > This patch adds a check for krbprincipalexpiration attribute to > pre_bind operation > in ipa-pwd-extop dirsrv plugin. If the principal is expired, auth is > denied and LDAP_INVALID_CREDENTIALS along with the error message is > sent back to the client. Since krbprincipalexpiration attribute is not > mandatory, if there is no value set, the check is passed. > > https://fedorahosted.org/freeipa/ticket/3305 > > Tomas > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel I just self-reviewed the patch and noticed a memory leak. It's fixed now. Updated patch attached. Tomas -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0034-2-Deny-LDAP-binds-for-user-accounts-with-expired-princ.patch Type: text/x-patch Size: 4146 bytes Desc: not available URL: From pvoborni at redhat.com Tue Feb 12 17:13:51 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 12 Feb 2013 18:13:51 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <511A7437.8000207@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> Message-ID: <511A784F.4000204@redhat.com> On 02/12/2013 05:56 PM, Petr Vobornik wrote: > On 02/12/2013 05:14 PM, Endi Sukma Dewata wrote: >> On 2/8/2013 7:27 AM, Petr Vobornik wrote: >>> Checkbox for NONE option was added. >>> >>> https://fedorahosted.org/freeipa/ticket/3404 >>> >>> Patches for master and 3.1 branch attached. >> >> ACK. >> > > We were discussing to NACK this approach. > > The implementation should be improved because of the mutually exclusive > nature of NONE option with [MS-PAC, PAD] options. > > I think we should add spec definition (to Web UI only, or into server > plugin as well) of mutex options. Something like: > > mutex_groups: [[['NONE'],['MS-PAC', 'PAD']], [/*another array of > groups*/]]; > > basically an array of group arrays where group array would contain > arrays of mutually exclusive option. Is it over-complicated? Would one > array or a pair of groups be enough? > > That way we can disable and uncheck(should not happen, but to keep data > consistency) checkboxes of options from other groups when user selects a > value of some group. Or just dim (no disable) and uncheck. That way there would still be visual distinction and one don't have to uncheck all the options from one group when he wants to select options of mutex group. > > > As a separate issue, we might want to fetch the options from metadata, > when we want to show the values and don't care about creating different > labels. -- Petr Vobornik From simo at redhat.com Tue Feb 12 17:23:11 2013 From: simo at redhat.com (Simo Sorce) Date: Tue, 12 Feb 2013 12:23:11 -0500 Subject: [Freeipa-devel] [PATCH 0034] Deny LDAP binds for user accounts with expired principal In-Reply-To: <511A75F0.7070801@redhat.com> References: <511A72D9.6090506@redhat.com> <511A75F0.7070801@redhat.com> Message-ID: <1360689791.3838.56.camel@willson.li.ssimo.org> On Tue, 2013-02-12 at 18:03 +0100, Tomas Babej wrote: > On 02/12/2013 05:50 PM, Tomas Babej wrote: > > Hi, > > > > This patch adds a check for krbprincipalexpiration attribute to > > pre_bind operation > > in ipa-pwd-extop dirsrv plugin. If the principal is expired, auth is > > denied and LDAP_INVALID_CREDENTIALS along with the error message is > > sent back to the client. Since krbprincipalexpiration attribute is > not > > mandatory, if there is no value set, the check is passed. > > > > https://fedorahosted.org/freeipa/ticket/3305 Comments inline. > @@ -1166,6 +1173,29 @@ static int ipapwd_pre_bind(Slapi_PBlock *pb) > goto done; > } > > + /* check the krbPrincipalExpiration attribute is present */ > + ret = slapi_entry_attr_find(entry, "krbPrincipalExpiration", > &attr); > + if (!ret) { ret is not a boolean so probably better ti use (ret != 0) > + /* if it is, check whether the principal has not expired */ > + > + principal_expire = slapi_entry_attr_get_charptr(entry, > + "krbprincipalexpiration"); > + memset(&expire_tm, 0, sizeof (expire_tm)); > + > + if (strptime(principal_expire, "%Y%m%d%H%M%SZ", &expire_tm)){ style issue missing space between ) and { > + expire_time = mktime(&expire_tm); > + /* this might have overflown on 32-bit system */ This comment does not help to point out what you want to put in evidence. if there is an overflow what is the consequence ? Or does it mean the next condition is taking that into account ? > + if (current_time > expire_time && expire_time > 0){ style issue missing space between ) and { > + LOG_FATAL("kerberos principal has expired in user > entry: %s\n", > + dn); I think a better phrasing would be: "The kerberos principal on %s is expired\n" > + errMesg = "Kerberos principal has expired."; s/has/is/ The rest looks good to me. Simo. -- Simo Sorce * Red Hat, Inc * New York From akrivoka at redhat.com Tue Feb 12 16:27:12 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Tue, 12 Feb 2013 17:27:12 +0100 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc Message-ID: <511A6D60.6030300@redhat.com> Add new LDAP container to store the list of domains associated with IPA realm. Add two new ipa commands (ipa realmdomains-show and ipa realmdomains-mod) to allow manipulation of the list of realm domains. Unit test file covering these new commands was added. https://fedorahosted.org/freeipa/ticket/2945 -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akrivoka-0005-Add-list-of-domains-associated-to-our-realm-to-cn-et.patch Type: text/x-patch Size: 18903 bytes Desc: not available URL: From pvoborni at redhat.com Tue Feb 12 17:58:49 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 12 Feb 2013 18:58:49 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <510FE077.8010801@redhat.com> References: <510FE077.8010801@redhat.com> Message-ID: <511A82D9.3030202@redhat.com> On 02/04/2013 05:23 PM, Tomas Babej wrote: > Hi, > > When adding/modifying an ID range for a trusted domain, the newly > added option --dom-name can be used. This looks up SID of the > trusted domain in LDAP and therefore the user is not required > to write it down in CLI. If the lookup fails, error message > asking the user to specify the SID manually is shown. > > https://fedorahosted.org/freeipa/ticket/3133 > > Tomas > > Just wondering: How bad would it be to not introduce new virtual attribute and just use the ipanttrusteddomainsid. On add and mod (when ipanttrusteddomainsid is set) we would check if ipanttrusteddomainsid is SID. If not, it would be treated as domain name and get_trusted_domain_sid_from_name method will be used to get the SID. I'm asking because I don't really like virtual and standard attributes for the same ldap attribute in a mod command. In WEB UI details page we have to display only one field - ipanttrusteddomainsid. So we are left with options: 1) do not use this feature for mod operations in Web UI 2) enter domain name in ipanttrusteddomainsid field, implement the aforementioned check in Web UI and fill the correct option in RPC request 3) add special action into action list which will open new dialog, user will enter domain name, mod command with ipanttrusteddomainname set will be executed on confirmation 4) some other method I don't really like any of the options. If a SID check is an easy operation, we can go with #2, but I would still rather see this logic in server plugin. -- Petr Vobornik From edewata at redhat.com Tue Feb 12 22:15:59 2013 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 12 Feb 2013 16:15:59 -0600 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <511A7437.8000207@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> Message-ID: <511ABF1F.70106@redhat.com> On 2/12/2013 10:56 AM, Petr Vobornik wrote: > We were discussing to NACK this approach. > > The implementation should be improved because of the mutually exclusive > nature of NONE option with [MS-PAC, PAD] options. > > I think we should add spec definition (to Web UI only, or into server > plugin as well) of mutex options. Something like: > > mutex_groups: [[['NONE'],['MS-PAC', 'PAD']], [/*another array of > groups*/]]; > > basically an array of group arrays where group array would contain > arrays of mutually exclusive option. Is it over-complicated? Would one > array or a pair of groups be enough? > > That way we can disable and uncheck(should not happen, but to keep data > consistency) checkboxes of options from other groups when user selects a > value of some group. If they are mutually exclusive, they probably should be separated using radio buttons like this: PAC: ( ) None (o) Type: [x] MS-PAC [ ] PAD It might be better to use a composite widget of radio buttons and checkboxes so we can reuse the code. Probably the definition will look something like this: { name: 'ipakrbauthzdata', type: 'radio', label: ..., options: [ { label: ..., value: 'NONE' }, { label: ..., type: 'checkboxes', options: [ { label: ..., value: 'MS-PAC' }, { label: ..., value: 'PAD' } ] } ] } The composite widget will handle setting the appropriate widgets when the value of the field on load. It will handle enabling/disabling the checkboxes when the radio button is selected. It will also compute the final value of the field from selected radio button/checkboxes on save. > Or just dim (no disable) and uncheck. That way there would still be > visual distinction and one don't have to uncheck all the options from > one group when he wants to select options of mutex group. That is also possible, but it changes the normal behavior of checkboxes, so probably users would have to play around with it to understand the grouping. They could also get confused, e.g. is dimmed checkbox disabled? > As a separate issue, we might want to fetch the options from metadata, > when we want to show the values and don't care about creating different > labels. If we use radio buttons like above, new labels are necessary to describe the different groups of checkboxes. For all radio buttons & checkboxes in general, if we don't want to use labels we probably could simplify the definition such that we can specify the string values directly (without nested object): { name: 'ipakrbauthzdata', type: 'radio', options: [ 'NONE', { type: 'checkboxes', options: ['MS-PAC', 'PAD'] } ] } -- Endi S. Dewata From dpal at redhat.com Wed Feb 13 00:30:04 2013 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 12 Feb 2013 19:30:04 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <1360675254.3838.8.camel@willson.li.ssimo.org> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> <1360675254.3838.8.camel@willson.li.ssimo.org> Message-ID: <511ADE8C.2000008@redhat.com> On 02/12/2013 08:20 AM, Simo Sorce wrote: > On Mon, 2013-02-11 at 20:30 -0500, Dmitri Pal wrote: >> On 02/11/2013 03:21 PM, Simo Sorce wrote: >>> On Mon, 2013-02-11 at 21:03 +0100, Ondrej Hamada wrote: >>>> Dne 3.2.2013 02:51, Dmitri Pal napsal(a): >>>>> On 01/31/2013 06:09 PM, Ondrej Hamada wrote: >>>>>> Hello, >>>>>> I'm starting to work on my thesis about 'More types of replicas in >>>>>> FreeIPA' again. One of the main problems is the way how should the >>>>>> read-only replicas deal with KDC because they're not supposed to >>>>>> posses the Kerberos (krb) master key. The task was to investigate how >>>>>> is this solved in Active Directory and its Read Only Domain Controllers. >>>>>> >>>>>> I found out that the basic of RODC behaviour is described on technet >>>>>> page >>>>>> (http://technet.microsoft.com/en-us/library/cc754218%28v=ws.10%29.aspx). >>>>>> >>>>>> Login situation: >>>>>> RODC by default forwards the KRB requests to the DC. RODC then >>>>>> forwards the response back to the client and also requests the >>>>>> password to be replicated to RODC. Both the user and his host must be >>>>>> members of 'Allowed RODC Password Replication' group in order to let >>>>>> user's passwords being replicated to RODCs. >>>>>> >>>>>> Request services that the RODC doesn't have credentials for: >>>>>> Client sends TGS-REQ to RODC. RODC can read the TGT in the request, >>>>>> but doesn't have credentials for the service. So the request is >>>>>> forwarded to the DC. DC can decrypt the TGT that was created by RODC >>>>>> and sends back the TGS-RES that is forwarded to the client. (but it >>>>>> does not trust the RODC so it recalculates the privilege attribute >>>>>> certificate). RODC does not cache the credentials for the service. >>>>>> >>>>>> During my experiments the credentials got replicated to the RODC on >>>>>> the first log on of the user. The user's KRB requests were first >>>>>> forwarded to the DC. When the user got krbtgt and TGS for host, ldap >>>>>> and cifs, his TGT was revoked by RODC. He run through the auth. >>>>>> process again, but this time the requests were served by RODC only - >>>>>> no forwarding - and not TGS for host was requested. >>>>>> >>>>>> Unfortunately I can not still recognize how the keys are processed. >>>>>> There's barely any RPC communication - only one DCERPC packet exchange >>>>>> between RODC and DC that takes place when the user sends his first TGS >>>>>> request (this exchange happens also for the clients with disabled >>>>>> replication). >>>>>> >>>>>> It looks to me like the DC knows all the RODC keys. According to >>>>>> Technet, the MS implementation of Kerberos is able to recognize the >>>>>> key owner from the Key Version Number value. >>>>>> >>>>>> I think I can't get more info from the network traffic examination. Do >>>>>> you have any ideas or hints on further investigation of the problem? >>>>> The page you listed shows the diagrams of the user login and TGT and >>>>> then TGS acquisition. >>>>> http://technet.microsoft.com/en-us/library/cc754218%28v=WS.10%29#BKMK_AuthRODC >>>>> Also the following thread sheds some good light on how the >>>>> authentication and caching happens in case of the RODC. >>>>> http://social.technet.microsoft.com/forums/en-US/winserverDS/thread/f8d1017e-1f0e-4a9d-a241-b03b508dfe17 >>>>> So they have policy driven replication of passwords. >>>>> If password is allowed to be replicated by policy it is replicated if >>>>> not it RODC has to always proxy to RWDC for this account. >>>>> The password changes always happen on RWDC and replicated back. >>>>> They can be replicated by RSO operation that allows replicating a single >>>>> object. >>>>> >>>>> It seems that they assume that all the services are always remote thus >>>>> connectivity to the central RWDC is a must. >>>>> They do not seem to keep any service keys locally in the RODC. >>>>> >>>>> With SSSD in play on the client I am not sure we should worry about >>>>> caching at least in the first implementation. >>>>> So in our case it might make sense to have a proxy KDC and a RO replica >>>>> with the subset of data replicated to it. >>>>> I wonder if you can accomplish that by taking 389 RO replica + IAKERB >>>>> I suggest you look at IAKERB and see if it can be used as a proxy for >>>>> user authentication, password change and TGS requests. >>>>> If not may be we can at least use it as an inspiration. >>>>> >>>>> The attached diagram shows what I mean. >>>>> >>>>> Later we can add some logic for the following: >>>>> a) RODC requesting replication of a specific object >>>>> b) RWDC replicating a specific object >>>>> c) Policy to define for which accounts the passwords and keys can replicated >>>>> d) RODC getting policy and performing local authentication for the >>>>> accounts for which the keys were replicated. >>>>> >>>>> However with SSSD on the client it might be easier for KDC proxy just to >>>>> go offline and not respond to the SSSD if it loses connection to the >>>>> central server. >>>>> That would force SSSD to go offline and use local password caching. I >>>>> suspect that SSSD password caching would be enabled in many cases >>>>> anyways so not caching all passwords in one central locating in RODC >>>>> might actually be a good thing. >>>>> >>>> Thanks for hints. I was looking at IAKERB and according to RFC it should >>>> support both AS and TGS requests/replies so it might be doable. I'll try >>>> to prepare a prototype that will be using 389 RO replica and IAKERB as >>>> you proposed. >>> IAKERB is a GSSAPI level interface, it is not something you can use as a >>> KDC Proxy on its own. >>> An actual KDC proxy is what is needed because you cannot change >>> applications at will and there are a number of applications that just go >>> and talk straight to a KDC server to do AS and TGS requests. >>> >>> A KDC proxy is possible to build, it just require some work in the >>> Actual KDC and probably some custom protocol to talk between KDCs. >>> We should look into what is the simplest technical way to achieve this >>> and perhaps propose it for standardization in IETF. >>> >>> Simo. >>> >> Really? I was hoping that IAKERB is exactly what you have described. >> Are you saying that IAKERB requires changes to the client? >> That sounds strange. > Clients using GSSAPI may not require may not require changes, but not > all Kerberos clients use a modern GSSAPI library. > Even SSSD uses direct Kerberos calls,a s traditionally there is no way > to get initial tickets via GSSAPI. That is slowly changing but it will > be a long time before that happens. > > Simo. > It looks like thinks are starting to boil down to building a Kerberos proxy. Is this something that fits within your thesis agenda Ondra? -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pviktori at redhat.com Wed Feb 13 09:21:41 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 10:21:41 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51129F98.7060706@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> Message-ID: <511B5B25.7040607@redhat.com> On 02/06/2013 07:23 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>> This adds a cert-find command for the dogtag backend. >>> >>> Searches can be done by serial number, by subject, revocation reason, >>> issue date, notbefore, notafter and revocation dates. >>> >>> I added some basic tests for this. I made it a separate test file >>> because the cert plugin tests do not use the declarative format and rely >>> on the selfsign backend by default. >>> >>> rob >> >> Thanks! The code works well, but I found a few issues. >> >> >> These tests don't work when the full test suite is run: test_cert adds >> and revokes additional certs that throw the code off. >> Perhaps have the tests only query valid certs? I don't see that option >> but I think it would be helpful to support. > > I added some rather nasty hacks to the test to make things pass. I limit > the search to 10 certificates, which is the number start with by > default. There is an open dogtag ticket to return only VALID > certificates so we should be able to drop this eventually. > > I had to go further on one of the revocation tests, limiting it to a > sizelimit of 1. The count changes every time the suite runs so this is > the safest for now. It also means that one test will fail if this is the > only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. > >> >> >> The API.txt check fails: >> Option sizelimit? of command cert_find in ipalib, not in API file: >> >> >> Int('sizelimit?', default=100, minvalue=0) > > Ouch. I thought I had fixed that, obviously not. Done now. > >> >> What are --all and --raw for? Is the plan to implement --all if/when >> Dogtag supports requesting additional data? > > Correct, they don't do anything at the moment. I have an RFE open to > return additional data from certs. Once that is done then all will make > sense. I don't know that raw will ever do anything interesting here but > it comes with all commands. > >> >> The format of --validnotbefore-to and friends should be mentioned in >> --help text; the following is confusing: >> $ ipa cert-show 1 >> [...] >> Not Before: Wed Feb 06 09:32:17 2013 UTC >> [...] >> $ ipa cert-find -h >> [...] >> --validnotbefore-to=STR >> Valid not before to this date >> [...] >> $ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC' >> ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 >> 2013 UTC' does not match format '%Y-%m-%d' > > It was listed in the top block but I added it to the usage help as well > for clarity. > >> Could you make the help text for --exactly more specific? > > Done. > > >> Please remove the extra whitespace at the end of dogtag.py >> >> I'd welcome a link to the design page in the commit message. >> > > both done > > rob -- Petr? From mkosek at redhat.com Wed Feb 13 09:48:42 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 13 Feb 2013 10:48:42 +0100 Subject: [Freeipa-devel] Announcing FreeIPA 2.2.2 Message-ID: <511B617A.4010602@redhat.com> The FreeIPA team is proud to announce version FreeIPA v2.2.2 This release contains Security Updates. It can be downloaded from http://www.freeipa.org/page/Downloads. A build is currently on the way to updates-testing for Fedora 17. == Highlights == This release contains a Security Advisory: * CVE-2012-5484: MITM Attack during Join process The FreeIPA Team would like to thank the Red Hat Security Response Team and in particular Vincent Danen for the invaluable assistance provided for the assessment and resolution of these issues. For CVE-2012-5484 we would like to thank Petr Men??k for reporting the issue. == Upgrading == Please consult each CVE announcement for related Upgrading instructions. An IPA server can be upgraded simply by installing updated rpms. The server does not need to be shut down in advance. If you have multiple servers you may upgrade them one at a time. It is expected that all servers will be upgraded in a relatively short period (days or weeks not months). They should be able to co-exist peacefully but new features will not be available on old servers and enrolling a new client against an old server will result in the SSH keys not being uploaded. Downgrading a server once upgraded is not supported. == Feedback == Please provide comments, bugs and other feedback via the freeipa-devel mailing list: http://www.redhat.com/mailman/listinfo/freeipa-devel == Detailed Changelog since 2.2.1 == Alexander Bokovoy (1): * Update plugin to upload CA certificate to LDAP Jan Cholasta (1): * Pylint cleanup John Dennis (1): * Use secure method to acquire IPA CA certificate Martin Kosek (3): * Run index task for new indexes * Filter suffix in replication management tools * Become IPA 2.2.2 Rob Crittenden (1): * Do SSL CA verification and hostname validation. Simo Sorce (1): * Upload CA cert in the directory on install ################################################################################ == CVE-2012-5484: MITM Attack during Join process == A weakness was found in the way an IPA client communicates with an IPA server when attempting to join an IPA domain. When an IPA client attempts to join an IPA domain an attacker could run a Man in The Middle Attack to try to intercept and hijack initial communication. A join initiated by an administrative user would grant the attacker administrative rights to the IPA server, whereas a join initiated by an unprivileged user would only grant the attacker limited privilege (typically just the ability to join the domain). The weakness is caused by the way the CA certificate is retrieved from the server. The following SSL communication may then be intercepted and subverted. Note that no credentials are exposed through this attack and it is effective only if performed during the join procedure and network traffic can be redirected or intercepted. Mere observation of the network traffic is not sufficient to grant an attacker any privilege. == Affected Versions == All 2.x and 3.x versions == Impact == Low == Acknowledgements == The FreeIPA team would like to thank Petr Men??k for reporting this issue. == Upgrade instructions == The resolution for this issue consist in allowing clients to download the CA certificate exclusively via a mutually authenticated LDAP connection or by providing the CA cert via an external method to the client. At least one IPA server in a domain need to be updated using the provided patches, so that the CA certificate is made available via LDAP. All client should be upgraded to use the updated ipa-client-install script that downloads the CA cert via an authenticated LDAP connection. From pviktori at redhat.com Wed Feb 13 10:18:28 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 11:18:28 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <5107F377.5020007@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> Message-ID: <511B6874.5060106@redhat.com> On 01/29/2013 05:06 PM, Petr Viktorin wrote: > On 01/04/2013 07:20 PM, Petr Viktorin wrote: >> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>> It's time to give this to another set of eyes :) >>>>>> >>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>> >>>>>> More info is in commit messages. >>>>>> >>>>>> >>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>> change the >>>>>> design document: when the client doesn't send the API version, it is >>>>>> assumed >>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>> The client still gets a warning if the version is missing. Except for >>>>>> those >>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>> the >>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>> our client >>>>>> is so far the only one that validates so strictly.) >>>>> >>>>> I did a basic test of this patch and also quickly read through the >>>>> patches and >>>>> besides nitpicks (like unused inspect module in >>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>> obvious >>>>> errors in the Python code. >>>> >>>> Noted, will fix in future versions of the patch. >>>> >>>>> >>>>> However, this patch breaks WebUI badly, I did not even get to a log in >>>>> screen. >>>>> Cooperation with Petr Vobornik will be needed. In my case, I got blank >>>>> screen >>>>> and Javascript error: >>>>> >>>>> TypeError: IPA.messages.dialogs is undefined >>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>> Line 1460 >>>>> >>>>> I assume this is related to the Internal Error that was returned in >>>>> the JSON call >>>>> >>>>> { >>>>> "error": null, >>>>> "id": null, >>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>> "result": { >>>>> "count": 5, >>>>> "results": [ >>>>> { >>>>> "error": "an internal error has occurred", >>>>> "error_code": 903, >>>>> "error_name": "InternalError" >>>>> }, >>>>> { >>>>> ... >>>>> >>>>> This can be reproduced with: >>>>> >>>>> # curl -v -H "Content-Type:application/json" -H >>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>> --negotiate -u : >>>>> --cacert /etc/ipa/ca.crt -d >>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>> https://`hostname`/ipa/json >>>> >>>> Good catch! The i18n_messages plugin already defines a "messages" >>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>> check for clashes. >>>> Since i18n_messages is an internal command only used by the Web UI, we >>>> can rename its output to "texts" without breaking compatibility. >>>> >>>> I'm attaching a preliminary fix (for both backend and UI), but >>>> hopefully >>>> it won't be necessary, see below. >>>> >>>>> I am also not sure I like the requirement of a specific version option >>>>> to be >>>>> always passed. I would prefer that missing version option would mean >>>>> "I use the >>>>> most recent version of API" instead - it would make the custom >>>>> JSONRPC/XMLRPC >>>>> calls easier to use. >>>>> >>>>> But since the version option was not being sent for some commands, we >>>>> may not >>>>> have a choice anyway if we do not want to break old clients in case we >>>>> add some >>>>> capabilities to these commands. >>>>> >>>> >>>> I see three other options, all worse: >>>> - Do not use capabilities for the affected commands, meaning no new >>>> functionality can be added to them (and by extension, no new >>>> functionality common to all commands can be added). >>>> - Treat a missing version as the current version >>>> - Break backwards compatibility >>>> >>>> And one possibly better (thanks to Petr? and Martin for opening my eyes >>>> off-list!): >>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>> >>>> This would also allow us to solve the above name-clashing problem >>>> elegantly. Here is a reminder of what a JSON response looks like: >>>> >>>> { >>>> "error": null, >>>> "id": 0, >>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>> "result": { >>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>> 2.47" >>>> }, >>>> "version": "3.1.0GIT2e4bd02" >>>> } >>>> >>>> A XML-RPC response only contains the "result" part of that. >>>> So with JSON, we can put the messages in the top level, which is much >>>> better design. >> >> Custom info in the "top level" seems to be a violation of the JSON-RPC >> spec. I'd rather not do more of those, so I'm withdrawing this idea. >> >>>> >>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>> inability to attach extra info to errors (commit >>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>> PublicError). >>>> >>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>> >>> >>> +1, XML-RPC sucks. This should have been done a long time ago. >>> >>> Honza >>> >> >> Here are new patches. >> >> XML-RPC requests with missing version are assumed to be old (the version >> before capabilities are introduced, 2.47). This takes care of backcompat >> with clients with bug 3294. >> JSON-RPC requests with missing version are assumed to be testing calls >> (e.g. curl), they get behavior of the latest version but also a warning. >> I've also added this info to the design doc. >> >> It turns out that these patches don't depend on whether our client uses >> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >> extra unit tests. >> > > Patch 106 had a minor conflict with master, attaching fixed version. > Patches 106 & 115 need an API version bump. I also noticed that `makeapi --validate` wasn't checking capabilities properly. Fixed. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0106-04-Add-client-capabilities-enable-messages.patch Type: text/x-patch Size: 20543 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0115-03-Rename-the-messages-Output-of-the-i18n_messages-comm.patch Type: text/x-patch Size: 3739 bytes Desc: not available URL: From jcholast at redhat.com Wed Feb 13 10:34:54 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 13 Feb 2013 11:34:54 +0100 Subject: [Freeipa-devel] [PATCHES] 127-136 LDAP code refactoring (Part 2) In-Reply-To: <51010252.1030200@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> Message-ID: <511B6C4E.508@redhat.com> On 24.1.2013 10:43, Petr Viktorin wrote: > On 01/22/2013 04:04 PM, Petr Viktorin wrote: >> On 01/21/2013 06:38 PM, Petr Viktorin wrote: >>> On 01/17/2013 06:27 PM, Petr Viktorin wrote: >>>> Hello, >>>> This is the first batch of changes aimed to consolidate our LDAP code. >>>> Each should be a self-contained change that doesn't break anything. >>>> >>>> These patches do some general cleanup (some of the changes might seem >>>> trivial but help a lot when grepping through the code); merge the >>>> common >>>> parts LDAPEntry, Entry and Entity classes; and move stuff that depends >>>> on an installed server out of IPASimpleLDAPObject and SchemaCache. >>>> >>>> I'm posting them early so you can see where I'm going, and so you can >>>> find out if your work will conflict with mine. > > > Patch 0120 grew a conflict with master, attaching a rebased version. > ACK part 2. Honza -- Jan Cholasta From mkosek at redhat.com Wed Feb 13 12:09:26 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 13 Feb 2013 13:09:26 +0100 Subject: [Freeipa-devel] [PATCH] 349 Test NetBIOS name clash before creating a trust In-Reply-To: <50F98614.3040505@redhat.com> References: <50F03223.8070802@redhat.com> <50F815AD.2030908@redhat.com> <50F98614.3040505@redhat.com> Message-ID: <511B8276.4090301@redhat.com> On 01/18/2013 06:27 PM, Martin Kosek wrote: > On 01/17/2013 04:15 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> Give a clear message about what is wrong with current Trust settings >>> before letting AD to return a confusing error message. >>> >>> https://fedorahosted.org/freeipa/ticket/3193 >> >> I hate being picky over wording but... >> >> I think it would read better if you replaced 'this' with 'The IPA server' or >> 'IPA' or something like that. >> >> rob >> > > No worries, attaching a better worded version. > > Martin > ACKed by Alexander on #freeipa. Pushed to master, ipa-3-1. Martin From mkosek at redhat.com Wed Feb 13 12:16:51 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 13 Feb 2013 13:16:51 +0100 Subject: [Freeipa-devel] [PATCH] 352-354 Add support for AD users to hbactest command In-Reply-To: <510BB677.1060201@redhat.com> References: <50F98561.1040503@redhat.com> <1358620528.20683.98.camel@willson.li.ssimo.org> <50FF9B05.3020207@redhat.com> <1358947395.20683.365.camel@willson.li.ssimo.org> <5100DF88.8010502@redhat.com> <1359036276.20683.387.camel@willson.li.ssimo.org> <510BB677.1060201@redhat.com> Message-ID: <511B8433.1040307@redhat.com> On 02/01/2013 01:35 PM, Martin Kosek wrote: > On 01/24/2013 03:04 PM, Simo Sorce wrote: >> On Thu, 2013-01-24 at 08:15 +0100, Martin Kosek wrote: >>> On 01/23/2013 02:23 PM, Simo Sorce wrote: >>>> On Wed, 2013-01-23 at 09:10 +0100, Martin Kosek wrote: >>>>> On 01/19/2013 07:35 PM, Simo Sorce wrote: >>>>>> On Fri, 2013-01-18 at 18:24 +0100, Martin Kosek wrote: >>>>>>> How this works: >>>>>>> 1. When a trusted domain user is tested, AD GC is searched >>>>>>> for the user entry Distinguished Name >>>>>> >>>>>> My head is not clear today but it looks to me you are doing 2 searches. >>>>>> One to go from samAccountName -> DNa dn then a second for DN -> SID. >>>>>> >>>>>> Why are you doing 2 searches ? The first one can return you the >>>>>> ObjectSid already. >>>>>> >>>>>> Simo. >>>>> >>>>> I had to do 2 searches because GC refuses to give me tokenGroups attribute >>>>> content when I do not search with exact DN and LDAP SCOPE_BASE. So I have to do >>>>> the first search to find out the DN of the searched user and then a second >>>>> query to get the tokenGroups (and ObjectSid). >>>> >>>> I see, yes that makes sense, would you mind adding a comment to this >>>> effect so we do not try to 'optimize' at some point ? >>>> I have no additional concerns then. >>>> >>>> Simo. >>>> >>> >>> Hello Simo, >>> >>> Thanks for review. Anyway, there is already a relevant comment in dcerpc.py, >>> where the double search is performed: >>> >>> ... >>> def get_trusted_domain_user_and_groups(self, object_name): >>> ... >>> entries = self.get_trusted_domain_objects(components.get('domain'), >>> components.get('flatname'), filter, attrs, _ldap.SCOPE_SUBTREE) >>> >>> # Get SIDs of user object and it's groups >>> # tokenGroups attribute must be read with scope BASE to avoid search error >>> attrs = ['objectSID', 'tokenGroups'] >>> ... >>> >>> I think it's enough to avoid "optimizing" this process - we would find out the >>> "optimization" soon anyway, as the tokenGroups search would return error :-) >> >> Perfect! >> >> /me just had an eye vision exam, will complain to his doctor :-) >> > > I enhanced the hbactest to also support user SID, not only trusted domain user > name. Updated set of patches attached. > > How it works: > > # ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 --host > `hostname` --service sshd > -------------------- > Access granted: True > -------------------- > Matched rules: can_login > > Martin Attaching a rebased set of patches. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-352-3-generalize-ad-gc-search.patch Type: text/x-patch Size: 12565 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-353-3-do-not-hide-sid-resolver-error-in-group-add-member.patch Type: text/x-patch Size: 1377 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-354-3-add-support-for-ad-users-to-hbactest-command.patch Type: text/x-patch Size: 8857 bytes Desc: not available URL: From abokovoy at redhat.com Wed Feb 13 13:14:58 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 13 Feb 2013 15:14:58 +0200 Subject: [Freeipa-devel] [PATCH] 352-354 Add support for AD users to hbactest command In-Reply-To: <511B8433.1040307@redhat.com> References: <50F98561.1040503@redhat.com> <1358620528.20683.98.camel@willson.li.ssimo.org> <50FF9B05.3020207@redhat.com> <1358947395.20683.365.camel@willson.li.ssimo.org> <5100DF88.8010502@redhat.com> <1359036276.20683.387.camel@willson.li.ssimo.org> <510BB677.1060201@redhat.com> <511B8433.1040307@redhat.com> Message-ID: <20130213131458.GC476@redhat.com> On Wed, 13 Feb 2013, Martin Kosek wrote: >On 02/01/2013 01:35 PM, Martin Kosek wrote: >> On 01/24/2013 03:04 PM, Simo Sorce wrote: >>> On Thu, 2013-01-24 at 08:15 +0100, Martin Kosek wrote: >>>> On 01/23/2013 02:23 PM, Simo Sorce wrote: >>>>> On Wed, 2013-01-23 at 09:10 +0100, Martin Kosek wrote: >>>>>> On 01/19/2013 07:35 PM, Simo Sorce wrote: >>>>>>> On Fri, 2013-01-18 at 18:24 +0100, Martin Kosek wrote: >>>>>>>> How this works: >>>>>>>> 1. When a trusted domain user is tested, AD GC is searched >>>>>>>> for the user entry Distinguished Name >>>>>>> >>>>>>> My head is not clear today but it looks to me you are doing 2 searches. >>>>>>> One to go from samAccountName -> DNa dn then a second for DN -> SID. >>>>>>> >>>>>>> Why are you doing 2 searches ? The first one can return you the >>>>>>> ObjectSid already. >>>>>>> >>>>>>> Simo. >>>>>> >>>>>> I had to do 2 searches because GC refuses to give me tokenGroups attribute >>>>>> content when I do not search with exact DN and LDAP SCOPE_BASE. So I have to do >>>>>> the first search to find out the DN of the searched user and then a second >>>>>> query to get the tokenGroups (and ObjectSid). >>>>> >>>>> I see, yes that makes sense, would you mind adding a comment to this >>>>> effect so we do not try to 'optimize' at some point ? >>>>> I have no additional concerns then. >>>>> >>>>> Simo. >>>>> >>>> >>>> Hello Simo, >>>> >>>> Thanks for review. Anyway, there is already a relevant comment in dcerpc.py, >>>> where the double search is performed: >>>> >>>> ... >>>> def get_trusted_domain_user_and_groups(self, object_name): >>>> ... >>>> entries = self.get_trusted_domain_objects(components.get('domain'), >>>> components.get('flatname'), filter, attrs, _ldap.SCOPE_SUBTREE) >>>> >>>> # Get SIDs of user object and it's groups >>>> # tokenGroups attribute must be read with scope BASE to avoid search error >>>> attrs = ['objectSID', 'tokenGroups'] >>>> ... >>>> >>>> I think it's enough to avoid "optimizing" this process - we would find out the >>>> "optimization" soon anyway, as the tokenGroups search would return error :-) >>> >>> Perfect! >>> >>> /me just had an eye vision exam, will complain to his doctor :-) >>> >> >> I enhanced the hbactest to also support user SID, not only trusted domain user >> name. Updated set of patches attached. >> >> How it works: >> >> # ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 --host >> `hostname` --service sshd >> -------------------- >> Access granted: True >> -------------------- >> Matched rules: can_login >> >> Martin > >Attaching a rebased set of patches. Thanks. I think we need to account for few more conditions. There are following cases: - user exists in AD - user does not exist in AD - group from AD specified as user The latter case is interesting because we can have groups from AD in external group mappings and people may specify them by mistake in hbactest. In such case I think we could have stated that group/user unknown or unmapped. Right now I'm getting less than helpful message: --------------------------------------------------------------------------- [root at jano ~]# ipa group-show extbar Group name: extbar Description: Ext bar Member of groups: foobar External member: S-1-5-21-3502988750-125904550-3683905862-512, S-1-5-21-3502988750-125904550-3683905862-500 [root at jano ~]# ipa group-show foobar Group name: foobar Description: foo bar GID: 944600004 Member groups: extbar [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-512 --host `hostname` --service sshd ipa: ERROR: trusted domain object not found [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-500 --host `hostname` --service sshd -------------------- Access granted: True -------------------- Matched rules: allow_all [root at jano ~]# ipa hbactest --user ADX\\Administrator --host `hostname` --service sshd -------------------- Access granted: True -------------------- Matched rules: allow_all [root at jano ~]# ipa hbactest --user ADX\\Domain\ Admins --host `hostname` --service sshd ipa: ERROR: trusted domain object not found [root at jano ~]# ipa hbactest --user ADX\\Enterprise\ Administrator --host `hostname` --service sshd ipa: ERROR: trusted domain object not found [root at jano ~]# ipa hbactest --user ADX\\DoesNotExist --host `hostname` --service sshd ipa: ERROR: trusted domain object not found ----------------------------------------------------------------------------- As you can see, group-as-user, unmapped group/user and non-existing AD user all raise error but the message is quite unclear -- does it talk about trusted domain itself or some object within the domain? Perhaps specifying real reason would be more helpful? If we are expecting only a user object, then I think we can change message a bit to state it more clear. >- return False >- return False >+ if not found_flatname: >+ raise errors.ValidationError(name=_('trusted domain object'), >+ error= _('no trusted domain matched the specified flat name')) >+ if not entries: >+ raise errors.NotFound(reason=_('trusted domain object not found')) >+ >+ return entries -- / Alexander Bokovoy From tbabej at redhat.com Wed Feb 13 12:25:22 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 13 Feb 2013 13:25:22 +0100 Subject: [Freeipa-devel] [PATCH 0034] Deny LDAP binds for user accounts with expired principal In-Reply-To: <1360689791.3838.56.camel@willson.li.ssimo.org> References: <511A72D9.6090506@redhat.com> <511A75F0.7070801@redhat.com> <1360689791.3838.56.camel@willson.li.ssimo.org> Message-ID: <511B8632.7000709@redhat.com> On 02/12/2013 06:23 PM, Simo Sorce wrote: > On Tue, 2013-02-12 at 18:03 +0100, Tomas Babej wrote: >> On 02/12/2013 05:50 PM, Tomas Babej wrote: >>> Hi, >>> >>> This patch adds a check for krbprincipalexpiration attribute to >>> pre_bind operation >>> in ipa-pwd-extop dirsrv plugin. If the principal is expired, auth is >>> denied and LDAP_INVALID_CREDENTIALS along with the error message is >>> sent back to the client. Since krbprincipalexpiration attribute is >> not >>> mandatory, if there is no value set, the check is passed. >>> >>> https://fedorahosted.org/freeipa/ticket/3305 > > Comments inline. >> @@ -1166,6 +1173,29 @@ static int ipapwd_pre_bind(Slapi_PBlock *pb) >> goto done; >> } >> >> + /* check the krbPrincipalExpiration attribute is present */ >> + ret = slapi_entry_attr_find(entry, "krbPrincipalExpiration", >> &attr); >> + if (!ret) { > ret is not a boolean so probably better ti use (ret != 0) > >> + /* if it is, check whether the principal has not expired */ >> + >> + principal_expire = slapi_entry_attr_get_charptr(entry, >> + "krbprincipalexpiration"); >> + memset(&expire_tm, 0, sizeof (expire_tm)); >> + >> + if (strptime(principal_expire, "%Y%m%d%H%M%SZ", &expire_tm)){ > style issue missing space between ) and { > >> + expire_time = mktime(&expire_tm); >> + /* this might have overflown on 32-bit system */ > This comment does not help to point out what you want to put in > evidence. > if there is an overflow what is the consequence ? Or does it mean the > next condition is taking that into account ? Yeah, this was rather ill-worded. Replaced by a rather verbose comment that hopefully clears things out. > >> + if (current_time > expire_time && expire_time > 0){ > style issue missing space between ) and { > >> + LOG_FATAL("kerberos principal has expired in user >> entry: %s\n", >> + dn); > I think a better phrasing would be: "The kerberos principal on %s is > expired\n" > >> + errMesg = "Kerberos principal has expired."; > s/has/is/ > > The rest looks good to me. > > Simo. > Styling issues fixed and updated patch attached :) Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0034-3-Deny-LDAP-binds-for-user-accounts-with-expired-princ.patch Type: text/x-patch Size: 4419 bytes Desc: not available URL: From simo at redhat.com Wed Feb 13 13:36:05 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 08:36:05 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <511ADE8C.2000008@redhat.com> References: <4FC8CF97.8000202@redhat.com> <1338828433.8230.235.camel@willson.li.ssimo.org> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> <1360675254.3838.8.camel@willson.li.ssimo.org> <511ADE8C.2000008@redhat.com> Message-ID: <1360762565.12328.24.camel@willson.li.ssimo.org> On Tue, 2013-02-12 at 19:30 -0500, Dmitri Pal wrote: > It looks like thinks are starting to boil down to building a Kerberos proxy. > Is this something that fits within your thesis agenda Ondra? I guess that's for Ondrej to say, if it is too much we can simply start working on the LDAP/replication side with rekeying and what not, and deal with the KDC part at a later time. Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Wed Feb 13 14:12:21 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 13 Feb 2013 16:12:21 +0200 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc In-Reply-To: <511A6D60.6030300@redhat.com> References: <511A6D60.6030300@redhat.com> Message-ID: <20130213141221.GD476@redhat.com> On Tue, 12 Feb 2013, Ana Krivokapic wrote: >Add new LDAP container to store the list of domains associated with IPA >realm. >Add two new ipa commands (ipa realmdomains-show and ipa >realmdomains-mod) to allow manipulation of the list of realm domains. >Unit test file covering these new commands was added. > >https://fedorahosted.org/freeipa/ticket/2945 ACK, works perfectly. We need to decide on the questions still open in the http://www.freeipa.org/page/V3/Realm_Domains but the decision should not prevent this work from being committed. Thanks! -- / Alexander Bokovoy From simo at redhat.com Wed Feb 13 14:53:44 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 09:53:44 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP Message-ID: <1360767224.12328.41.camel@willson.li.ssimo.org> Hello list, with recently seen a few requests to add FreeIPA users via LDAP directly. This is a common method supported by many meta-directory/HR systems, however so far we cannot really recommend it because we add quite a number of attributes automatically in our framework code when we create users, and those functions can change in future versions. However these external tools are usually not very flexible and supporting them as they are would make for a much better experience for integrators. I had a brief discussion with Rob on IRC on how to address this situation. If we limit ourselves to users we could probably resolve this problem with a relatively simple 389ds plugin that intercept add operations that try to add a user. The idea is that the remote system would be allowed to set a minimum of attributes (even incomplete according to our schema). But as long as a specific set of objectclasses is set (say person and posixaccount) the operation would be recognized as an attempt to create a user account. In this case the plugin would take over the operation and perform a call against our framework using json. The call would send a reformatted request using the data we got in input so that any custom objectclass /attribute can be preserved. The call would also add a special flag so the framework knows this is coming from 389ds itself. The framework would treat this requests in a slightly special way, it would use all the common code currently used to properly format a user entry adding all the ancillary data we need, but instead of trying to ldapadd the entry, it would instead return it back to the caller. 389ds at this point gets back a json reply, convert it into an ldap add operation and proceeds with this new 'augmented' operation instead of the original one. What do people think about this option ? I think it would be extremely valuable for admins, as it would allow them to drive user 'synchronization' in a very simple way. It could also be used to properly augment winsync users so we can allow full creation when syncing from AD with all the proper attributes created through the json request. So I see a lot of optential here. The only doubt is the json back and forth communication. What do people on the framework side think ? Is there going to be any big problem in adapting the framework so we can use common code to prepare the object but then change the last stp and return a json reply instead of perfroming an ldap add operation ? Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Wed Feb 13 15:08:44 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 13 Feb 2013 08:08:44 -0700 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360767224.12328.41.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> Message-ID: <511BAC7C.2090807@redhat.com> On 02/13/2013 07:53 AM, Simo Sorce wrote: > Hello list, > > with recently seen a few requests to add FreeIPA users via LDAP > directly. This is a common method supported by many meta-directory/HR > systems, however so far we cannot really recommend it because we add > quite a number of attributes automatically in our framework code when we > create users, and those functions can change in future versions. > > However these external tools are usually not very flexible and > supporting them as they are would make for a much better experience for > integrators. > > I had a brief discussion with Rob on IRC on how to address this > situation. > > If we limit ourselves to users we could probably resolve this problem > with a relatively simple 389ds plugin that intercept add operations that > try to add a user. > > The idea is that the remote system would be allowed to set a minimum of > attributes (even incomplete according to our schema). But as long as a > specific set of objectclasses is set (say person and posixaccount) the > operation would be recognized as an attempt to create a user account. > > In this case the plugin would take over the operation and perform a call > against our framework using json. So the 389 plugin would make an http/json call to the framework? > The call would send a reformatted > request using the data we got in input so that any custom > objectclass /attribute can be preserved. The call would also add a > special flag so the framework knows this is coming from 389ds itself. > > The framework would treat this requests in a slightly special way, it > would use all the common code currently used to properly format a user > entry adding all the ancillary data we need, but instead of trying to > ldapadd the entry, it would instead return it back to the caller. > > 389ds at this point gets back a json reply, convert it into an ldap add > operation and proceeds with this new 'augmented' operation instead of > the original one. > > What do people think about this option ? > I think it would be extremely valuable for admins, as it would allow > them to drive user 'synchronization' in a very simple way. > It could also be used to properly augment winsync users so we can allow > full creation when syncing from AD with all the proper attributes > created through the json request. So I see a lot of optential here. > > The only doubt is the json back and forth communication. > > What do people on the framework side think ? Is there going to be any > big problem in adapting the framework so we can use common code to > prepare the object but then change the last stp and return a json reply > instead of perfroming an ldap add operation ? > > Simo. > From pviktori at redhat.com Wed Feb 13 15:12:35 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 16:12:35 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360767224.12328.41.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> Message-ID: <511BAD63.9050005@redhat.com> On 02/13/2013 03:53 PM, Simo Sorce wrote: > Hello list, > > with recently seen a few requests to add FreeIPA users via LDAP > directly. This is a common method supported by many meta-directory/HR > systems, however so far we cannot really recommend it because we add > quite a number of attributes automatically in our framework code when we > create users, and those functions can change in future versions. > > However these external tools are usually not very flexible and > supporting them as they are would make for a much better experience for > integrators. > > I had a brief discussion with Rob on IRC on how to address this > situation. > > If we limit ourselves to users we could probably resolve this problem > with a relatively simple 389ds plugin that intercept add operations that > try to add a user. > > The idea is that the remote system would be allowed to set a minimum of > attributes (even incomplete according to our schema). But as long as a > specific set of objectclasses is set (say person and posixaccount) the > operation would be recognized as an attempt to create a user account. > > In this case the plugin would take over the operation and perform a call > against our framework using json. The call would send a reformatted > request using the data we got in input so that any custom > objectclass /attribute can be preserved. The call would also add a > special flag so the framework knows this is coming from 389ds itself. > > The framework would treat this requests in a slightly special way, it > would use all the common code currently used to properly format a user > entry adding all the ancillary data we need, but instead of trying to > ldapadd the entry, it would instead return it back to the caller. Why return the data as JSON, instead of writing it directly to LDAP? Our own post-callback assumes the user is already in LDAP, and who knows what user-supplied callbacks will do. Keep in mind IPA is plugable; at least for outside plugins' sake (if not our own sanity's) we should keep the number of code paths to a minimum. > 389ds at this point gets back a json reply, convert it into an ldap add > operation and proceeds with this new 'augmented' operation instead of > the original one. > > What do people think about this option ? > I think it would be extremely valuable for admins, as it would allow > them to drive user 'synchronization' in a very simple way. > It could also be used to properly augment winsync users so we can allow > full creation when syncing from AD with all the proper attributes > created through the json request. So I see a lot of optential here. > > The only doubt is the json back and forth communication. > > What do people on the framework side think ? Is there going to be any > big problem in adapting the framework so we can use common code to > prepare the object but then change the last stp and return a json reply > instead of perfroming an ldap add operation ? > > Simo. > -- Petr? From pspacek at redhat.com Wed Feb 13 15:15:20 2013 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 13 Feb 2013 16:15:20 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360767224.12328.41.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> Message-ID: <511BAE08.5040108@redhat.com> On 13.2.2013 15:53, Simo Sorce wrote: > Hello list, > > with recently seen a few requests to add FreeIPA users via LDAP > directly. This is a common method supported by many meta-directory/HR > systems, however so far we cannot really recommend it because we add > quite a number of attributes automatically in our framework code when we > create users, and those functions can change in future versions. > > However these external tools are usually not very flexible and > supporting them as they are would make for a much better experience for > integrators. > > I had a brief discussion with Rob on IRC on how to address this > situation. > > If we limit ourselves to users we could probably resolve this problem > with a relatively simple 389ds plugin that intercept add operations that > try to add a user. > > The idea is that the remote system would be allowed to set a minimum of > attributes (even incomplete according to our schema). But as long as a > specific set of objectclasses is set (say person and posixaccount) the > operation would be recognized as an attempt to create a user account. > > In this case the plugin would take over the operation and perform a call > against our framework using json. The call would send a reformatted > request using the data we got in input so that any custom > objectclass /attribute can be preserved. The call would also add a > special flag so the framework knows this is coming from 389ds itself. > > The framework would treat this requests in a slightly special way, it > would use all the common code currently used to properly format a user > entry adding all the ancillary data we need, but instead of trying to > ldapadd the entry, it would instead return it back to the caller. > > 389ds at this point gets back a json reply, convert it into an ldap add > operation and proceeds with this new 'augmented' operation instead of > the original one. > > What do people think about this option ? > I think it would be extremely valuable for admins, as it would allow > them to drive user 'synchronization' in a very simple way. > It could also be used to properly augment winsync users so we can allow > full creation when syncing from AD with all the proper attributes > created through the json request. So I see a lot of optential here. > > The only doubt is the json back and forth communication. > > What do people on the framework side think ? Is there going to be any > big problem in adapting the framework so we can use common code to > prepare the object but then change the last stp and return a json reply > instead of perfroming an ldap add operation ? Personally, I don't like the idea of JSON requests & response parsing *inside* 389. I see a lot of opportunities for crash or security hole. What about small (optional) separate daemon? A) Variant with separate sub-tree 1. create some new subtree, e.g. cn=useradd-playground,dc=example,dc=com 2. watch this sub-tree with persistent search 3. catch new objects and run IPA commands as necessary 4. replace "old" objects with referrals to IPA objects (or do something different) I'm not sure that various meta-directory systems can handle situation where user disappeared/object is moved/object is morphed to referal. That can be a problem. B) More crazy variant with LDAP "proxy" (Now I'm on very thin ice, I don't know how complex it would be. I only found some examples like [1].) Prepare "fake" (non-IPA) LDAP server with same LDAP "structure" and let separate daemon watch server's content and run IPA commands as necessary. It would be best if the small daemon can pretend to be LDAP server and catch LDAP operations directly, as shown in [1]. [1] http://tonthon.blogspot.cz/2011/02/ldaptor-ldap-with-twisted-server-side.html -- Petr^2 Spacek From rcritten at redhat.com Wed Feb 13 15:30:26 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 13 Feb 2013 10:30:26 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BAD63.9050005@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> Message-ID: <511BB192.5090000@redhat.com> Petr Viktorin wrote: > On 02/13/2013 03:53 PM, Simo Sorce wrote: >> Hello list, >> >> with recently seen a few requests to add FreeIPA users via LDAP >> directly. This is a common method supported by many meta-directory/HR >> systems, however so far we cannot really recommend it because we add >> quite a number of attributes automatically in our framework code when we >> create users, and those functions can change in future versions. >> >> However these external tools are usually not very flexible and >> supporting them as they are would make for a much better experience for >> integrators. >> >> I had a brief discussion with Rob on IRC on how to address this >> situation. >> >> If we limit ourselves to users we could probably resolve this problem >> with a relatively simple 389ds plugin that intercept add operations that >> try to add a user. >> >> The idea is that the remote system would be allowed to set a minimum of >> attributes (even incomplete according to our schema). But as long as a >> specific set of objectclasses is set (say person and posixaccount) the >> operation would be recognized as an attempt to create a user account. >> >> In this case the plugin would take over the operation and perform a call >> against our framework using json. The call would send a reformatted >> request using the data we got in input so that any custom >> objectclass /attribute can be preserved. The call would also add a >> special flag so the framework knows this is coming from 389ds itself. >> >> The framework would treat this requests in a slightly special way, it >> would use all the common code currently used to properly format a user >> entry adding all the ancillary data we need, but instead of trying to >> ldapadd the entry, it would instead return it back to the caller. > > Why return the data as JSON, instead of writing it directly to LDAP? > Our own post-callback assumes the user is already in LDAP, and who knows > what user-supplied callbacks will do. Keep in mind IPA is plugable; at > least for outside plugins' sake (if not our own sanity's) we should keep > the number of code paths to a minimum. Simo's concern was, I believe, with permissions. My concern is with moving the logic of adding users directly into 389-ds. I think Simo's concern was with the credentials used to add the user. His idea was to use the 389-ds service principal to call into JSON. This principal normally does not have user add permission but if all we were doing is determining what the add should look it it should be adequate. Then the formatted user would be returned and 389-ds base would attempt the ADD, authenticated as the user who did the BIND (which we probably need to assume is a simple bind). Otherwise we will need to find some way to connect to the framework, authenticate as the user who made the 389-ds BIND, and add the user. rob From pviktori at redhat.com Wed Feb 13 15:33:41 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 16:33:41 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BAE08.5040108@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> Message-ID: <511BB255.7090104@redhat.com> On 02/13/2013 04:15 PM, Petr Spacek wrote: > On 13.2.2013 15:53, Simo Sorce wrote: >> Hello list, >> >> with recently seen a few requests to add FreeIPA users via LDAP >> directly. This is a common method supported by many meta-directory/HR >> systems, however so far we cannot really recommend it because we add >> quite a number of attributes automatically in our framework code when we >> create users, and those functions can change in future versions. >> >> However these external tools are usually not very flexible and >> supporting them as they are would make for a much better experience for >> integrators. >> >> I had a brief discussion with Rob on IRC on how to address this >> situation. >> >> If we limit ourselves to users we could probably resolve this problem >> with a relatively simple 389ds plugin that intercept add operations that >> try to add a user. >> >> The idea is that the remote system would be allowed to set a minimum of >> attributes (even incomplete according to our schema). But as long as a >> specific set of objectclasses is set (say person and posixaccount) the >> operation would be recognized as an attempt to create a user account. >> >> In this case the plugin would take over the operation and perform a call >> against our framework using json. The call would send a reformatted >> request using the data we got in input so that any custom >> objectclass /attribute can be preserved. The call would also add a >> special flag so the framework knows this is coming from 389ds itself. >> >> The framework would treat this requests in a slightly special way, it >> would use all the common code currently used to properly format a user >> entry adding all the ancillary data we need, but instead of trying to >> ldapadd the entry, it would instead return it back to the caller. >> >> 389ds at this point gets back a json reply, convert it into an ldap add >> operation and proceeds with this new 'augmented' operation instead of >> the original one. >> >> What do people think about this option ? >> I think it would be extremely valuable for admins, as it would allow >> them to drive user 'synchronization' in a very simple way. >> It could also be used to properly augment winsync users so we can allow >> full creation when syncing from AD with all the proper attributes >> created through the json request. So I see a lot of optential here. >> >> The only doubt is the json back and forth communication. >> >> What do people on the framework side think ? Is there going to be any >> big problem in adapting the framework so we can use common code to >> prepare the object but then change the last stp and return a json reply >> instead of perfroming an ldap add operation ? > > Personally, I don't like the idea of JSON requests & response parsing > *inside* 389. I see a lot of opportunities for crash or security hole. It seems Petr is reading my mind here. I was thinking about these solutions too; they seem cleaner but I don't know if they're feasible. > What about small (optional) separate daemon? > > A) Variant with separate sub-tree > > 1. create some new subtree, e.g. cn=useradd-playground,dc=example,dc=com > 2. watch this sub-tree with persistent search > 3. catch new objects and run IPA commands as necessary > 4. replace "old" objects with referrals to IPA objects (or do something > different) > > I'm not sure that various meta-directory systems can handle situation > where user disappeared/object is moved/object is morphed to referal. > That can be a problem. A separate tree would mean we're not committed to supporting this particular kind of magic forever. We could have different "compatibility hacks" in the future, each with its own semantics. > B) More crazy variant with LDAP "proxy" > (Now I'm on very thin ice, I don't know how complex it would be. I only > found some examples like [1].) > Prepare "fake" (non-IPA) LDAP server with same LDAP "structure" and let > separate daemon watch server's content and run IPA commands as > necessary. It would be best if the small daemon can pretend to be LDAP > server and catch LDAP operations directly, as shown in [1]. An adaptor would certainly be the proper solution when we need to interop with an inflexible foreign interface. The question is if building one is worth it. In both cases rather than core IPA functionality this would be something external, something the users have to explicitly install and use, something that doesn't necessarily have to work right with user-supplied plugins, something limited (say, to users only), and something that'd always use existing code paths. I'm really worried about scope creep here; after a few years of adding features like this IPA would become unmaintainable. Better to have a focused core and add on top. -- Petr? From pviktori at redhat.com Wed Feb 13 16:16:32 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 17:16:32 +0100 Subject: [Freeipa-devel] [PATCH 0033] Prevent changing protected group's name using --setattr In-Reply-To: <5118C555.6090009@redhat.com> References: <5118C555.6090009@redhat.com> Message-ID: <511BBC60.8070601@redhat.com> On 02/11/2013 11:17 AM, Tomas Babej wrote: > Hi, > > The name of any protected group now cannot be changed by modifing > the cn attribute using --setattr. Unit tests have been added to > make sure there is no regression. > > https://fedorahosted.org/freeipa/ticket/3354 > > Tomas > We need a better general way to validate getattr/setattr on known atributes, but works well for now. ACK -- Petr? From mkosek at redhat.com Wed Feb 13 16:17:42 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 13 Feb 2013 17:17:42 +0100 Subject: [Freeipa-devel] [PATCH] 352-354 Add support for AD users to hbactest command In-Reply-To: <20130213131458.GC476@redhat.com> References: <50F98561.1040503@redhat.com> <1358620528.20683.98.camel@willson.li.ssimo.org> <50FF9B05.3020207@redhat.com> <1358947395.20683.365.camel@willson.li.ssimo.org> <5100DF88.8010502@redhat.com> <1359036276.20683.387.camel@willson.li.ssimo.org> <510BB677.1060201@redhat.com> <511B8433.1040307@redhat.com> <20130213131458.GC476@redhat.com> Message-ID: <511BBCA6.6000906@redhat.com> On 02/13/2013 02:14 PM, Alexander Bokovoy wrote: > On Wed, 13 Feb 2013, Martin Kosek wrote: >> On 02/01/2013 01:35 PM, Martin Kosek wrote: >>> On 01/24/2013 03:04 PM, Simo Sorce wrote: >>>> On Thu, 2013-01-24 at 08:15 +0100, Martin Kosek wrote: >>>>> On 01/23/2013 02:23 PM, Simo Sorce wrote: >>>>>> On Wed, 2013-01-23 at 09:10 +0100, Martin Kosek wrote: >>>>>>> On 01/19/2013 07:35 PM, Simo Sorce wrote: >>>>>>>> On Fri, 2013-01-18 at 18:24 +0100, Martin Kosek wrote: >>>>>>>>> How this works: >>>>>>>>> 1. When a trusted domain user is tested, AD GC is searched >>>>>>>>> for the user entry Distinguished Name >>>>>>>> >>>>>>>> My head is not clear today but it looks to me you are doing 2 searches. >>>>>>>> One to go from samAccountName -> DNa dn then a second for DN -> SID. >>>>>>>> >>>>>>>> Why are you doing 2 searches ? The first one can return you the >>>>>>>> ObjectSid already. >>>>>>>> >>>>>>>> Simo. >>>>>>> >>>>>>> I had to do 2 searches because GC refuses to give me tokenGroups attribute >>>>>>> content when I do not search with exact DN and LDAP SCOPE_BASE. So I >>>>>>> have to do >>>>>>> the first search to find out the DN of the searched user and then a second >>>>>>> query to get the tokenGroups (and ObjectSid). >>>>>> >>>>>> I see, yes that makes sense, would you mind adding a comment to this >>>>>> effect so we do not try to 'optimize' at some point ? >>>>>> I have no additional concerns then. >>>>>> >>>>>> Simo. >>>>>> >>>>> >>>>> Hello Simo, >>>>> >>>>> Thanks for review. Anyway, there is already a relevant comment in dcerpc.py, >>>>> where the double search is performed: >>>>> >>>>> ... >>>>> def get_trusted_domain_user_and_groups(self, object_name): >>>>> ... >>>>> entries = self.get_trusted_domain_objects(components.get('domain'), >>>>> components.get('flatname'), filter, attrs, >>>>> _ldap.SCOPE_SUBTREE) >>>>> >>>>> # Get SIDs of user object and it's groups >>>>> # tokenGroups attribute must be read with scope BASE to avoid >>>>> search error >>>>> attrs = ['objectSID', 'tokenGroups'] >>>>> ... >>>>> >>>>> I think it's enough to avoid "optimizing" this process - we would find out >>>>> the >>>>> "optimization" soon anyway, as the tokenGroups search would return error :-) >>>> >>>> Perfect! >>>> >>>> /me just had an eye vision exam, will complain to his doctor :-) >>>> >>> >>> I enhanced the hbactest to also support user SID, not only trusted domain user >>> name. Updated set of patches attached. >>> >>> How it works: >>> >>> # ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 --host >>> `hostname` --service sshd >>> -------------------- >>> Access granted: True >>> -------------------- >>> Matched rules: can_login >>> >>> Martin >> >> Attaching a rebased set of patches. > Thanks. > > I think we need to account for few more conditions. > > There are following cases: > > - user exists in AD > - user does not exist in AD > - group from AD specified as user > > The latter case is interesting because we can have groups from AD in > external group mappings and people may specify them by mistake in > hbactest. In such case I think we could have stated that group/user > unknown or unmapped. > > Right now I'm getting less than helpful message: > --------------------------------------------------------------------------- > [root at jano ~]# ipa group-show extbar > Group name: extbar > Description: Ext bar > Member of groups: foobar > External member: S-1-5-21-3502988750-125904550-3683905862-512, > S-1-5-21-3502988750-125904550-3683905862-500 > [root at jano ~]# ipa group-show foobar > Group name: foobar > Description: foo bar > GID: 944600004 > Member groups: extbar > [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-512 > --host `hostname` --service sshd > ipa: ERROR: trusted domain object not found > [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-500 > --host `hostname` --service sshd -------------------- > Access granted: True > -------------------- > Matched rules: allow_all > [root at jano ~]# ipa hbactest --user ADX\\Administrator --host `hostname` > --service sshd > -------------------- > Access granted: True > -------------------- > Matched rules: allow_all > [root at jano ~]# ipa hbactest --user ADX\\Domain\ Admins --host `hostname` > --service sshd > ipa: ERROR: trusted domain object not found > [root at jano ~]# ipa hbactest --user ADX\\Enterprise\ Administrator --host > `hostname` --service sshd > ipa: ERROR: trusted domain object not found > [root at jano ~]# ipa hbactest --user ADX\\DoesNotExist --host `hostname` > --service sshd > ipa: ERROR: trusted domain object not found > ----------------------------------------------------------------------------- > > As you can see, group-as-user, unmapped group/user and non-existing AD > user all raise error but the message is quite unclear -- does it talk > about trusted domain itself or some object within the domain? Perhaps > specifying real reason would be more helpful? > > If we are expecting only a user object, then I think we can change > message a bit to state it more clear. > >> - return False >> - return False >> + if not found_flatname: >> + raise errors.ValidationError(name=_('trusted domain object'), >> + error= _('no trusted domain matched the specified >> flat name')) >> + if not entries: >> + raise errors.NotFound(reason=_('trusted domain object not found')) >> + >> + return entries > > Fixed. This is quite straightforward as the errors are now returned via specific exceptions and not with False/None result: # ipa hbactest --user "AD\Doesnotexist" --host `hostname` --service sshdipa: ERROR: trusted domain user not found I am not convinced we should create a specific error when a group is passed instead of user. To fix this, we would have to search GC with different filter and also allow (|(objectclass=group)(objectclass=user)) instead of (objectclass=user) which is there now and then check if the resulting object is user or group. I am not convinced we want to do that, the error is now pretty clear that user could not be found... I also extended hbactest help and added a section about testing trusted domain users, I forgot to add it before. I am also attaching a related simple patch fixing formatting for other hbactest examples. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-352-4-generalize-ad-gc-search.patch Type: text/x-patch Size: 12565 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-353-4-do-not-hide-sid-resolver-error-in-group-add-member.patch Type: text/x-patch Size: 1377 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-354-4-add-support-for-ad-users-to-hbactest-command.patch Type: text/x-patch Size: 11842 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-hbachelp-examples-formatting.patch Type: text/x-patch Size: 3794 bytes Desc: not available URL: From simo at redhat.com Wed Feb 13 16:27:10 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 11:27:10 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BB255.7090104@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> Message-ID: <1360772830.12328.75.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 16:33 +0100, Petr Viktorin wrote: > On 02/13/2013 04:15 PM, Petr Spacek wrote: > > On 13.2.2013 15:53, Simo Sorce wrote: > >> Hello list, > >> > >> with recently seen a few requests to add FreeIPA users via LDAP > >> directly. This is a common method supported by many meta-directory/HR > >> systems, however so far we cannot really recommend it because we add > >> quite a number of attributes automatically in our framework code when we > >> create users, and those functions can change in future versions. > >> > >> However these external tools are usually not very flexible and > >> supporting them as they are would make for a much better experience for > >> integrators. > >> > >> I had a brief discussion with Rob on IRC on how to address this > >> situation. > >> > >> If we limit ourselves to users we could probably resolve this problem > >> with a relatively simple 389ds plugin that intercept add operations that > >> try to add a user. > >> > >> The idea is that the remote system would be allowed to set a minimum of > >> attributes (even incomplete according to our schema). But as long as a > >> specific set of objectclasses is set (say person and posixaccount) the > >> operation would be recognized as an attempt to create a user account. > >> > >> In this case the plugin would take over the operation and perform a call > >> against our framework using json. The call would send a reformatted > >> request using the data we got in input so that any custom > >> objectclass /attribute can be preserved. The call would also add a > >> special flag so the framework knows this is coming from 389ds itself. > >> > >> The framework would treat this requests in a slightly special way, it > >> would use all the common code currently used to properly format a user > >> entry adding all the ancillary data we need, but instead of trying to > >> ldapadd the entry, it would instead return it back to the caller. > >> > >> 389ds at this point gets back a json reply, convert it into an ldap add > >> operation and proceeds with this new 'augmented' operation instead of > >> the original one. > >> > >> What do people think about this option ? > >> I think it would be extremely valuable for admins, as it would allow > >> them to drive user 'synchronization' in a very simple way. > >> It could also be used to properly augment winsync users so we can allow > >> full creation when syncing from AD with all the proper attributes > >> created through the json request. So I see a lot of optential here. > >> > >> The only doubt is the json back and forth communication. > >> > >> What do people on the framework side think ? Is there going to be any > >> big problem in adapting the framework so we can use common code to > >> prepare the object but then change the last stp and return a json reply > >> instead of perfroming an ldap add operation ? > > > > Personally, I don't like the idea of JSON requests & response parsing > > *inside* 389. I see a lot of opportunities for crash or security hole. Petr, p[lease substantiate what is your worry, how is using json in 389ds a security issue ? It is well confined, and the only server you contact is the http server on localhost, not any random external server, so what makes you think of security holes ? What would cause crashes ? Please be precise and substantiate a threat model, I currently see nothing special in this scenario. > It seems Petr is reading my mind here. I was thinking about these > solutions too; they seem cleaner but I don't know if they're feasible. I am sorry, but 'cleaner' is really the last word I'd use, 'hack' is what comes to mind here to be honest. > > What about small (optional) separate daemon? One more moving part one additional process, that does very little, it looks to me as a big hammer to yield. > > A) Variant with separate sub-tree > > > > 1. create some new subtree, e.g. cn=useradd-playground,dc=example,dc=com This has more consequences than you may think. I do not like the separate field idea because you need to treat it in a special way. We would probably need special ACIs to anoty allow any other client to see this subtree otherwise they may see incomplete objects. Yet we need to allow some user to write to it. We need to decide case by case which plugins to enable, not all DS plugins can use filters or exclude subtrees so we may have issues with some plugins if we do not want them to operate on these special entries and so on. > > 2. watch this sub-tree with persistent search Persistent search are scarce, and user creation is rare, it would be over blown, plus it would be a race condition if the daemon dies for whatever reason. > > 3. catch new objects and run IPA commands as necessary With what user identity ? Keep in mind that one of the reasons we use delegation is to make sure that ACIs are meaningful *and* auditing reflects the actual user that did operations. With an external daemon you have to cases: 1) you use a fixed identity for the external daemon to create these users, that means the daemon now have to do additional checks to see if the user used to create the entry is authorized or not and what attributes it is authorized to create. This is a nightmare, as you do not want to start parsing ACIs in the external deamons, also prone to race conditions if ACIs are being changed. I *really* don't like this. 2) make the daemon all powerful so it can do s4u2self and s4u2proxy for any user in the system. This way the daemon can connect to LDAP as the original user and ACIs are applied as usual. This means the daemon is *all powerful* as it can impersonate any users. I *really* don't like this either. Also keep in mind that this 2 step method means you cannot easily relay back errors to the original client. So you can end up with the original system believing all is fine, but the actual real user object creation failed. This in turn requires additional monitoring by admins they will need to monitor the playground area to see if some objects are 'stranded' there. They need some way to find out what went wrong in the 2 step procedure which is obscure to them. Then they need to find a way to re-trigger the original system to re-create the user, even though the original system was told the creation as ok, so this may be difficult depending on the system. It quickly becomes a nightmare handling errors with this 2 step system. > > 4. replace "old" objects with referrals to IPA objects (or do something > > different) uh ? > > I'm not sure that various meta-directory systems can handle situation > > where user disappeared/object is moved/object is morphed to referal. > > That can be a problem. Yes and not the only one. > A separate tree would mean we're not committed to supporting this > particular kind of magic forever. We could have different "compatibility > hacks" in the future, each with its own semantics. Yes a separate tree would really be much more work, sorry but I do not like this 'solution; one bit for the above reasons. > > B) More crazy variant with LDAP "proxy" > > (Now I'm on very thin ice, I don't know how complex it would be. I only > > found some examples like [1].) > > Prepare "fake" (non-IPA) LDAP server with same LDAP "structure" and let > > separate daemon watch server's content and run IPA commands as > > necessary. It would be best if the small daemon can pretend to be LDAP > > server and catch LDAP operations directly, as shown in [1]. > > An adaptor would certainly be the proper solution when we need to > interop with an inflexible foreign interface. The question is if > building one is worth it. It has all the same issues as the above daemon, *plus* all the complexity of a full LDAP server, because you do not know what checking operations the remote system may be doing, so you can't just fake up some parts, you need a full staging proxy. So now we go from a simple plugin that does some json, to a full blown mapping proxy. With delayed operations. The complexity scale is 1:100 or more if you ask me (where 1 is the plugin I described and 100 is plan B). > In both cases rather than core IPA functionality this would be something > external, something the users have to explicitly install and use, > something that doesn't necessarily have to work right with user-supplied > plugins, something limited (say, to users only), and something that'd > always use existing code paths. > I'm really worried about scope creep here; after a few years of adding > features like this IPA would become unmaintainable. Better to have a > focused core and add on top. This is why I proposed a plugin that is limited to users and calls the framework so we can use common code. The *simpler* way would be to simply replicate the core framework login in the 389ds plugin or even *move* it there. But we want to keep the logic in the framework as it is more flexible and easier to work with and extend, so I proposed a 389ds plugin that just *asks* the framwrok for the data. This keeps the busienss loginc in the python framewrok, yet it allows an LDAP driver to add users properly in IPA just using LDAP calls. I do not see this as a slippery slope, as it would be limited to user creation by definition. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 16:34:28 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 11:34:28 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BAD63.9050005@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> Message-ID: <1360773268.12328.81.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: > Our own post-callback assumes the user is already in LDAP, and who > knows what user-supplied callbacks will do. Keep in mind IPA is > plugable; at least for outside plugins' sake (if not our own sanity's) > we should keep the number of code paths to a minimum. True which is why my proposal is to not use the standard user-add RPC call, but have a separate one. This separate call would only call the core business logic to create the user account add operation, but none of the external plumbing. Ideally we spit the framework flow like this: Normal user -> Real user-add --- . . . . . . . . . --- LDAP add \ / -- common logic -- / \ 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply custom plugins should be called in the custom logic an operate on the object before the ADD is attempted. If we do it this way then most of the code path will be in common which is what we want, and only the mechanical operation of adding the actual object to ldap will be different. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 16:40:54 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 11:40:54 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360772830.12328.75.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> Message-ID: <1360773654.12328.84.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 11:27 -0500, Simo Sorce wrote: > This is why I proposed a plugin that is limited to users and calls the > framework so we can use common code. > The *simpler* way would be to simply replicate the core framework > login > in the 389ds plugin or even *move* it there. > > But we want to keep the logic in the framework as it is more flexible > and easier to work with and extend, so I proposed a 389ds plugin that > just *asks* the framwrok for the data. This keeps the busienss loginc > in the python framewrok, yet it allows an LDAP driver to add users > properly in IPA just using LDAP calls. > > I do not see this as a slippery slope, as it would be limited to user > creation by definition. > Btw as a bonus you get an API that user can use to 'test' user creation w/o actually creating users, I think this may come useful as well for people experimenting with creating plugins, as they can tweak their plugin configuration rapidly w/o actually touching LDAP. And only use the real creation path once they are happy with the results returned by the new API. It's not a reason in itself to add this stuff but it is a nice byproduct. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 16:53:51 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 11:53:51 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BAC7C.2090807@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAC7C.2090807@redhat.com> Message-ID: <1360774431.12328.94.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 08:08 -0700, Rich Megginson wrote: > On 02/13/2013 07:53 AM, Simo Sorce wrote: > > Hello list, > > > > with recently seen a few requests to add FreeIPA users via LDAP > > directly. This is a common method supported by many meta-directory/HR > > systems, however so far we cannot really recommend it because we add > > quite a number of attributes automatically in our framework code when we > > create users, and those functions can change in future versions. > > > > However these external tools are usually not very flexible and > > supporting them as they are would make for a much better experience for > > integrators. > > > > I had a brief discussion with Rob on IRC on how to address this > > situation. > > > > If we limit ourselves to users we could probably resolve this problem > > with a relatively simple 389ds plugin that intercept add operations that > > try to add a user. > > > > The idea is that the remote system would be allowed to set a minimum of > > attributes (even incomplete according to our schema). But as long as a > > specific set of objectclasses is set (say person and posixaccount) the > > operation would be recognized as an attempt to create a user account. > > > > In this case the plugin would take over the operation and perform a call > > against our framework using json. > > So the 389 plugin would make an http/json call to the framework? This i the idea. 1. add operation comes in For example: dn: uid=foo,cn=users,cn=accounts,dc=freeipa,dc=org changetype: add objectclass: ipa-add-user objectclass: custom-stuff uid: foo custom-attr: bar ... (other attributes we require absolutely) 2. we match on special objectclass 'IPA-ADD-USER' 3. plugin converts incoming entry into json request and calls framework 4. framework does magic and creates a full ldap request 5. framework converts it back into a json object 6. plugin converts json object into an add request and replaces the original add with the new data 7. plugins tries to commit the new data as the original user. 8. if there are any errors they are sent back to the client. This plugin would need to operate before any other so that all other plugins will see the data as modified by the call to the framework, not the original request. Also the special objectclass is just an idea, I am open to other suggestions. The only important thing is that we have a way to distinguish between an complete add operation that should just go thorough and a an add operation that needs to be sent to the framework for adjustment. We could also decide to always intercept, but it seem to be a bit wasteful for regular framework add operations as it would go through it twice: http -> framewrok -> ldap add -> framework (recheck) -> add Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 16:57:26 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 11:57:26 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BC2F7.5040707@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> Message-ID: <1360774646.12328.96.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 11:44 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: > >> Our own post-callback assumes the user is already in LDAP, and who > >> knows what user-supplied callbacks will do. Keep in mind IPA is > >> plugable; at least for outside plugins' sake (if not our own sanity's) > >> we should keep the number of code paths to a minimum. > > > > True which is why my proposal is to not use the standard user-add RPC > > call, but have a separate one. > > > > This separate call would only call the core business logic to create the > > user account add operation, but none of the external plumbing. > > > > Ideally we spit the framework flow like this: > > > > Normal user -> Real user-add --- . . . . . . . . . --- LDAP add > > \ / > > -- common logic -- > > / \ > > 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply > > > > > > custom plugins should be called in the custom logic an operate on the > > object before the ADD is attempted. > > > > If we do it this way then most of the code path will be in common which > > is what we want, and only the mechanical operation of adding the actual > > object to ldap will be different. > > > > Simo. > > > > There is one missing a few steps. A plugin execution looks like: > > Normal user -> Real user-add --- pre-op call(s) --- execute (LPAP add > record) --- post-op call(s) which may do additional add/modify > > It is the postop calls that would be the problem. They assume that the > entry has already been written (so, for example, it has a valid > UID/GID/ipaUniqueId, etc). Why are they done after the add ? It seem dangerous. What happens id the first ldap add succeed and the post op fails ? We should exceute the ldap call after the post ops are perfomed imho. Simo. -- Simo Sorce * Red Hat, Inc * New York From pviktori at redhat.com Wed Feb 13 17:11:50 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 18:11:50 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360772830.12328.75.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> Message-ID: <511BC956.5010307@redhat.com> On 02/13/2013 05:27 PM, Simo Sorce wrote: [...] > > I am sorry, but 'cleaner' is really the last word I'd use, 'hack' is > what comes to mind here to be honest. Then I'm missing something. Thanks for your explanations. >>> What about small (optional) separate daemon? > > One more moving part one additional process, that does very little, it > looks to me as a big hammer to yield. > >>> A) Variant with separate sub-tree >>> >>> 1. create some new subtree, e.g. cn=useradd-playground,dc=example,dc=com > > This has more consequences than you may think. > I do not like the separate field idea because you need to treat it in a > special way. We would probably need special ACIs to anoty allow any > other client to see this subtree otherwise they may see incomplete > objects. Yet we need to allow some user to write to it. > We need to decide case by case which plugins to enable, not all DS > plugins can use filters or exclude subtrees so we may have issues with > some plugins if we do not want them to operate on these special entries > and so on. Is it possible to use read ACIs of the original tree? >>> 2. watch this sub-tree with persistent search > > Persistent search are scarce, and user creation is rare, it would be > over blown, plus it would be a race condition if the daemon dies for > whatever reason. > >>> 3. catch new objects and run IPA commands as necessary > > With what user identity ? > Keep in mind that one of the reasons we use delegation is to make sure > that ACIs are meaningful *and* auditing reflects the actual user that > did operations. Thanks. Now your proposal starts making sense to me. [...] > >> In both cases rather than core IPA functionality this would be something >> external, something the users have to explicitly install and use, >> something that doesn't necessarily have to work right with user-supplied >> plugins, something limited (say, to users only), and something that'd >> always use existing code paths. >> I'm really worried about scope creep here; after a few years of adding >> features like this IPA would become unmaintainable. Better to have a >> focused core and add on top. > > This is why I proposed a plugin that is limited to users and calls the > framework so we can use common code. > The *simpler* way would be to simply replicate the core framework login > in the 389ds plugin or even *move* it there. > > But we want to keep the logic in the framework as it is more flexible > and easier to work with and extend, so I proposed a 389ds plugin that > just *asks* the framwrok for the data. This keeps the busienss loginc in > the python framewrok, yet it allows an LDAP driver to add users properly > in IPA just using LDAP calls. The problem is that the framework may do more than LDAP operations. It supports user-written plugins that can literally do anything. We'd need to limit what IPA plugins can do, or how they do it, otherwise it's impossible to just return the result. So I can see a DS plugin calling IPA, but IPA really has to write the results itself. > I do not see this as a slippery slope, as it would be limited to user > creation by definition. I'd be extremely surprised if all of these inflexible external HR systems happened to limit themselves to user creation. -- Petr? From rmeggins at redhat.com Wed Feb 13 17:13:08 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 13 Feb 2013 10:13:08 -0700 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360774431.12328.94.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAC7C.2090807@redhat.com> <1360774431.12328.94.camel@willson.li.ssimo.org> Message-ID: <511BC9A4.30106@redhat.com> On 02/13/2013 09:53 AM, Simo Sorce wrote: > On Wed, 2013-02-13 at 08:08 -0700, Rich Megginson wrote: >> On 02/13/2013 07:53 AM, Simo Sorce wrote: >>> Hello list, >>> >>> with recently seen a few requests to add FreeIPA users via LDAP >>> directly. This is a common method supported by many meta-directory/HR >>> systems, however so far we cannot really recommend it because we add >>> quite a number of attributes automatically in our framework code when we >>> create users, and those functions can change in future versions. >>> >>> However these external tools are usually not very flexible and >>> supporting them as they are would make for a much better experience for >>> integrators. >>> >>> I had a brief discussion with Rob on IRC on how to address this >>> situation. >>> >>> If we limit ourselves to users we could probably resolve this problem >>> with a relatively simple 389ds plugin that intercept add operations that >>> try to add a user. >>> >>> The idea is that the remote system would be allowed to set a minimum of >>> attributes (even incomplete according to our schema). But as long as a >>> specific set of objectclasses is set (say person and posixaccount) the >>> operation would be recognized as an attempt to create a user account. >>> >>> In this case the plugin would take over the operation and perform a call >>> against our framework using json. >> So the 389 plugin would make an http/json call to the framework? > This i the idea. > > 1. add operation comes in > For example: > dn: uid=foo,cn=users,cn=accounts,dc=freeipa,dc=org > changetype: add > objectclass: ipa-add-user > objectclass: custom-stuff > uid: foo > custom-attr: bar > ... (other attributes we require absolutely) > 2. we match on special objectclass 'IPA-ADD-USER' > 3. plugin converts incoming entry into json request and calls framework > 4. framework does magic and creates a full ldap request > 5. framework converts it back into a json object > 6. plugin converts json object into an add request and replaces the > original add with the new data > 7. plugins tries to commit the new data as the original user. > 8. if there are any errors they are sent back to the client. Ok. That's a lot of http/json code to link into 389. > > This plugin would need to operate before any other so that all other > plugins will see the data as modified by the call to the framework, not > the original request. Should be ok to do in an "early" pre-op plugin. > > Also the special objectclass is just an idea, I am open to other > suggestions. The only important thing is that we have a way to > distinguish between an complete add operation that should just go > thorough and a an add operation that needs to be sent to the framework > for adjustment. We could also decide to always intercept, but it seem to > be a bit wasteful for regular framework add operations as it would go > through it twice: http -> framewrok -> ldap add -> framework (recheck) > -> add Yeah, I think it would kill general performance to do that. There should be a way to distinguish with some combination of objectclasses/attributes. > > > Simo. > From rmeggins at redhat.com Wed Feb 13 17:16:07 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 13 Feb 2013 10:16:07 -0700 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360774646.12328.96.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> Message-ID: <511BCA56.6090005@redhat.com> On 02/13/2013 09:57 AM, Simo Sorce wrote: > On Wed, 2013-02-13 at 11:44 -0500, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: >>>> Our own post-callback assumes the user is already in LDAP, and who >>>> knows what user-supplied callbacks will do. Keep in mind IPA is >>>> plugable; at least for outside plugins' sake (if not our own sanity's) >>>> we should keep the number of code paths to a minimum. >>> True which is why my proposal is to not use the standard user-add RPC >>> call, but have a separate one. >>> >>> This separate call would only call the core business logic to create the >>> user account add operation, but none of the external plumbing. >>> >>> Ideally we spit the framework flow like this: >>> >>> Normal user -> Real user-add --- . . . . . . . . . --- LDAP add >>> \ / >>> -- common logic -- >>> / \ >>> 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply >>> >>> >>> custom plugins should be called in the custom logic an operate on the >>> object before the ADD is attempted. >>> >>> If we do it this way then most of the code path will be in common which >>> is what we want, and only the mechanical operation of adding the actual >>> object to ldap will be different. >>> >>> Simo. >>> >> There is one missing a few steps. A plugin execution looks like: >> >> Normal user -> Real user-add --- pre-op call(s) --- execute (LPAP add >> record) --- post-op call(s) which may do additional add/modify >> >> It is the postop calls that would be the problem. They assume that the >> entry has already been written (so, for example, it has a valid >> UID/GID/ipaUniqueId, etc). > Why are they done after the add ? It seem dangerous. > What happens id the first ldap add succeed and the post op fails ? Are you talking about 389 plugins? If so, then in 1.3.0 and later, if you do all of the pre-op/post-op as betxn plugins, then they all take place in the same transaction, and they all succeed or all fail. > > We should exceute the ldap call after the post ops are perfomed imho. > > Simo. > From pviktori at redhat.com Wed Feb 13 17:16:57 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 13 Feb 2013 18:16:57 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360774646.12328.96.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> Message-ID: <511BCA89.9070804@redhat.com> On 02/13/2013 05:57 PM, Simo Sorce wrote: > On Wed, 2013-02-13 at 11:44 -0500, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: >>>> Our own post-callback assumes the user is already in LDAP, and who >>>> knows what user-supplied callbacks will do. Keep in mind IPA is >>>> plugable; at least for outside plugins' sake (if not our own sanity's) >>>> we should keep the number of code paths to a minimum. >>> >>> True which is why my proposal is to not use the standard user-add RPC >>> call, but have a separate one. >>> >>> This separate call would only call the core business logic to create the >>> user account add operation, but none of the external plumbing. >>> >>> Ideally we spit the framework flow like this: >>> >>> Normal user -> Real user-add --- . . . . . . . . . --- LDAP add >>> \ / >>> -- common logic -- >>> / \ >>> 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply >>> >>> >>> custom plugins should be called in the custom logic an operate on the >>> object before the ADD is attempted. >>> >>> If we do it this way then most of the code path will be in common which >>> is what we want, and only the mechanical operation of adding the actual >>> object to ldap will be different. >>> >>> Simo. >>> >> >> There is one missing a few steps. A plugin execution looks like: >> >> Normal user -> Real user-add --- pre-op call(s) --- execute (LPAP add >> record) --- post-op call(s) which may do additional add/modify >> >> It is the postop calls that would be the problem. They assume that the >> entry has already been written (so, for example, it has a valid >> UID/GID/ipaUniqueId, etc). > > Why are they done after the add ? It seem dangerous. This comment from the user-add post-op plugin should explain: # Fetch the entry again to update memberof, mep data, etc updated # at the end of the transaction. Some processing happens in DS plugins, we can't get the data if the object isn't added yet. > What happens id the first ldap add succeed and the post op fails ? > > We should exceute the ldap call after the post ops are perfomed imho. > -- Petr? From rcritten at redhat.com Wed Feb 13 16:44:39 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 13 Feb 2013 11:44:39 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360773268.12328.81.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> Message-ID: <511BC2F7.5040707@redhat.com> Simo Sorce wrote: > On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: >> Our own post-callback assumes the user is already in LDAP, and who >> knows what user-supplied callbacks will do. Keep in mind IPA is >> plugable; at least for outside plugins' sake (if not our own sanity's) >> we should keep the number of code paths to a minimum. > > True which is why my proposal is to not use the standard user-add RPC > call, but have a separate one. > > This separate call would only call the core business logic to create the > user account add operation, but none of the external plumbing. > > Ideally we spit the framework flow like this: > > Normal user -> Real user-add --- . . . . . . . . . --- LDAP add > \ / > -- common logic -- > / \ > 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply > > > custom plugins should be called in the custom logic an operate on the > object before the ADD is attempted. > > If we do it this way then most of the code path will be in common which > is what we want, and only the mechanical operation of adding the actual > object to ldap will be different. > > Simo. > There is one missing a few steps. A plugin execution looks like: Normal user -> Real user-add --- pre-op call(s) --- execute (LPAP add record) --- post-op call(s) which may do additional add/modify It is the postop calls that would be the problem. They assume that the entry has already been written (so, for example, it has a valid UID/GID/ipaUniqueId, etc). rob From simo at redhat.com Wed Feb 13 17:35:13 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 12:35:13 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BCA89.9070804@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> Message-ID: <1360776913.12328.108.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 18:16 +0100, Petr Viktorin wrote: > On 02/13/2013 05:57 PM, Simo Sorce wrote: > > On Wed, 2013-02-13 at 11:44 -0500, Rob Crittenden wrote: > >> Simo Sorce wrote: > >>> On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: > >>>> Our own post-callback assumes the user is already in LDAP, and who > >>>> knows what user-supplied callbacks will do. Keep in mind IPA is > >>>> plugable; at least for outside plugins' sake (if not our own sanity's) > >>>> we should keep the number of code paths to a minimum. > >>> > >>> True which is why my proposal is to not use the standard user-add RPC > >>> call, but have a separate one. > >>> > >>> This separate call would only call the core business logic to create the > >>> user account add operation, but none of the external plumbing. > >>> > >>> Ideally we spit the framework flow like this: > >>> > >>> Normal user -> Real user-add --- . . . . . . . . . --- LDAP add > >>> \ / > >>> -- common logic -- > >>> / \ > >>> 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply > >>> > >>> > >>> custom plugins should be called in the custom logic an operate on the > >>> object before the ADD is attempted. > >>> > >>> If we do it this way then most of the code path will be in common which > >>> is what we want, and only the mechanical operation of adding the actual > >>> object to ldap will be different. > >>> > >>> Simo. > >>> > >> > >> There is one missing a few steps. A plugin execution looks like: > >> > >> Normal user -> Real user-add --- pre-op call(s) --- execute (LPAP add > >> record) --- post-op call(s) which may do additional add/modify > >> > >> It is the postop calls that would be the problem. They assume that the > >> entry has already been written (so, for example, it has a valid > >> UID/GID/ipaUniqueId, etc). > > > > Why are they done after the add ? It seem dangerous. > > This comment from the user-add post-op plugin should explain: > > # Fetch the entry again to update memberof, mep data, etc updated > # at the end of the transaction. > > Some processing happens in DS plugins, we can't get the data if the > object isn't added yet. Is this data actually needed for the post-op plugins ? Also what happens if a post-op plugin fails ? Please don't tell me the framework deletes the user object that would be really bad. Simo. -- Simo Sorce * Red Hat, Inc * New York From jdennis at redhat.com Wed Feb 13 17:40:53 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 13 Feb 2013 12:40:53 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360773268.12328.81.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> Message-ID: <511BD025.7000201@redhat.com> I appreciate Simo's concern for authorization and audit in this process, we must solve that problem. If I understand the proposal correctly it's akin to recording a macro that can be replayed. The framework executes as normal but instead of issuing the LDAP modify commands we record them. Then after the entire command completes we send the recorded operations back to 389DS in some form Did I understand this correctly? If so I'm very much against the idea of sending JSON back to 389DS to execute the totality of the operation. Why? It either breaks or has the potential to break our entire processing model, pre and post operations, validity checks (e.g. querying the current state) user supplied plugins, etc. I could see this working in some limited cases which might give you the illusion it would work. But the only robust general solution I think we can sign up for supporting is to use the API commands we designed, period. Anything else just seems like a nightmare scenario of corner cases. Therefore I think the proposal of watching something (yet to be determined), calling our API commands, and then cleaning up the watched entity afterwards is the best approach. Figuring out how to authenticate/authorize/audit this is the primary challenge, a challenge far more manageable then trying to subvert the framework with every known and unknown risk that introduces. It's hard enough as it is assuring our documented API works correctly. Our API is the only thing I think we can realistically commit to supporting. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From simo at redhat.com Wed Feb 13 17:50:47 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 12:50:47 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BC956.5010307@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> Message-ID: <1360777847.12328.124.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 18:11 +0100, Petr Viktorin wrote: > >>> 1. create some new subtree, e.g. cn=useradd-playground,dc=example,dc=com > > > > This has more consequences than you may think. > > I do not like the separate field idea because you need to treat it in a > > special way. We would probably need special ACIs to anoty allow any > > other client to see this subtree otherwise they may see incomplete > > objects. Yet we need to allow some user to write to it. > > We need to decide case by case which plugins to enable, not all DS > > plugins can use filters or exclude subtrees so we may have issues with > > some plugins if we do not want them to operate on these special entries > > and so on. > > Is it possible to use read ACIs of the original tree? Not sure what's the question here. Care to elaborate ? > The problem is that the framework may do more than LDAP operations. It > supports user-written plugins that can literally do anything. > We'd need to limit what IPA plugins can do, or how they do it, otherwise > it's impossible to just return the result. Ok, then maybe we can have a 'core' common logic part that has some more limitations. > So I can see a DS plugin calling IPA, but IPA really has to write the > results itself. This is harder, but could be done too. The way it would work would be to make 389ds able to perform s4u2self for the case a user bound to LDAP with a simple bind, and then s4u2proxy with the evidence ticket to get HTTP/server ticket, then we can connect to the framework using the user's identity. In turn the framework will perform an additional s4u2proxy delegation to get a ticket back to ldap/server to perform the add operation. This means the flow would be: client --(LDAP)--> 389DS --(HTTP/json)--> framework --(LDAP)--> add The problem with this solution is that there is a risk of loops, so we need a mechanism that makes it impossible for the 389ds plugin to recurse into the framework. Also we need to make sure the add operation is not blocking add operations by keeping a transaction open causing a deadlock. And we cannot do that by immediately returning to the client because we need to wait for the framework reply to know if the operation was successful or not. I see some more low level risks doing it this way which is why I made my proposal to avoid this potential loop. Rich, is there potential from deadlocking here due to the new transaction stuff ? Or can we single out this plugin to run before *any* transaction is started ? > > I do not see this as a slippery slope, as it would be limited to user > > creation by definition. > > I'd be extremely surprised if all of these inflexible external HR > systems happened to limit themselves to user creation. HR systems only deal with hiring and firing employees, they *may* deal with adding users to groups, but that doesn't require special operations for us, it is just a matter of adding a 'member' attribute to a group object. I do not think we need to offer anything else here. We already have proof this is sufficient, because we have experience with the AD winsync plugin which has a similar function, the only differnce is that the driver is an AD server instead of an HR system directly. And with that plugin we also only create users by default, and we haven't heard many requests to do anything more. The only request we really had was to make sure we could sync also posix attrs from AD, but that's it, still just basic user info. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 17:53:21 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 12:53:21 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BD025.7000201@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> Message-ID: <1360778001.12328.126.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 12:40 -0500, John Dennis wrote: > I appreciate Simo's concern for authorization and audit in this process, > we must solve that problem. If I understand the proposal correctly it's > akin to recording a macro that can be replayed. The framework executes > as normal but instead of issuing the LDAP modify commands we record > them. Then after the entire command completes we send the recorded > operations back to 389DS in some form Did I understand this correctly? > If so I'm very much against the idea of sending JSON back to 389DS to > execute the totality of the operation. Why? It either breaks or has the > potential to break our entire processing model, pre and post operations, > validity checks (e.g. querying the current state) user supplied plugins, > etc. I could see this working in some limited cases which might give you > the illusion it would work. But the only robust general solution I think > we can sign up for supporting is to use the API commands we designed, > period. Anything else just seems like a nightmare scenario of corner cases. > > Therefore I think the proposal of watching something (yet to be > determined), calling our API commands, and then cleaning up the watched > entity afterwards is the best approach. Figuring out how to > authenticate/authorize/audit this is the primary challenge, a challenge > far more manageable then trying to subvert the framework with every > known and unknown risk that introduces. It's hard enough as it is > assuring our documented API works correctly. Our API is the only thing I > think we can realistically commit to supporting. See my reply to Petr, it can be done the way you ask (ie framework does the actual ldap add), the only concern I have is looping and deadlocks. If we can solve the looping and potential deadlocking concerns I think we can avoid the json reply and let the framework do the actual final ldap add. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Wed Feb 13 17:57:26 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 13 Feb 2013 10:57:26 -0700 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360777847.12328.124.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> Message-ID: <511BD406.9090302@redhat.com> On 02/13/2013 10:50 AM, Simo Sorce wrote: > On Wed, 2013-02-13 at 18:11 +0100, Petr Viktorin wrote: >>>>> 1. create some new subtree, e.g. cn=useradd-playground,dc=example,dc=com >>> This has more consequences than you may think. >>> I do not like the separate field idea because you need to treat it in a >>> special way. We would probably need special ACIs to anoty allow any >>> other client to see this subtree otherwise they may see incomplete >>> objects. Yet we need to allow some user to write to it. >>> We need to decide case by case which plugins to enable, not all DS >>> plugins can use filters or exclude subtrees so we may have issues with >>> some plugins if we do not want them to operate on these special entries >>> and so on. >> Is it possible to use read ACIs of the original tree? > Not sure what's the question here. Care to elaborate ? > >> The problem is that the framework may do more than LDAP operations. It >> supports user-written plugins that can literally do anything. >> We'd need to limit what IPA plugins can do, or how they do it, otherwise >> it's impossible to just return the result. > Ok, then maybe we can have a 'core' common logic part that has some more > limitations. > >> So I can see a DS plugin calling IPA, but IPA really has to write the >> results itself. > This is harder, but could be done too. The way it would work would be to > make 389ds able to perform s4u2self for the case a user bound to LDAP > with a simple bind, and then s4u2proxy with the evidence ticket to get > HTTP/server ticket, then we can connect to the framework using the > user's identity. In turn the framework will perform an additional > s4u2proxy delegation to get a ticket back to ldap/server to perform the > add operation. > > This means the flow would be: > > client --(LDAP)--> 389DS --(HTTP/json)--> framework --(LDAP)--> add > > The problem with this solution is that there is a risk of loops, so we > need a mechanism that makes it impossible for the 389ds plugin to > recurse into the framework. Also we need to make sure the add operation > is not blocking add operations by keeping a transaction open causing a > deadlock. And we cannot do that by immediately returning to the client > because we need to wait for the framework reply to know if the operation > was successful or not. > I see some more low level risks doing it this way which is why I made my > proposal to avoid this potential loop. > > Rich, > is there potential from deadlocking here due to the new transaction > stuff ? Or can we single out this plugin to run before *any* transaction > is started ? If you do this in a "regular" pre-op, not a "betxn" pre-op, then it should be fine. > >>> I do not see this as a slippery slope, as it would be limited to user >>> creation by definition. >> I'd be extremely surprised if all of these inflexible external HR >> systems happened to limit themselves to user creation. > HR systems only deal with hiring and firing employees, they *may* deal > with adding users to groups, but that doesn't require special operations > for us, it is just a matter of adding a 'member' attribute to a group > object. > I do not think we need to offer anything else here. We already have > proof this is sufficient, because we have experience with the AD winsync > plugin which has a similar function, the only differnce is that the > driver is an AD server instead of an HR system directly. And with that > plugin we also only create users by default, and we haven't heard many > requests to do anything more. The only request we really had was to make > sure we could sync also posix attrs from AD, but that's it, still just > basic user info. > > Simo. > From jdennis at redhat.com Wed Feb 13 17:59:26 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 13 Feb 2013 12:59:26 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360778001.12328.126.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> Message-ID: <511BD47E.3000507@redhat.com> On 02/13/2013 12:53 PM, Simo Sorce wrote: > If we can solve the looping and potential deadlocking concerns I think > we can avoid the json reply and let the framework do the actual final > ldap add. Could you elaborate on your looping and deadlock concerns? I don't see where they would arise if what we're watching is entirely independent of our LDAP tree. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From simo at redhat.com Wed Feb 13 18:04:50 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 13:04:50 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BD47E.3000507@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> Message-ID: <1360778690.12328.127.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: > On 02/13/2013 12:53 PM, Simo Sorce wrote: > > > If we can solve the looping and potential deadlocking concerns I think > > we can avoid the json reply and let the framework do the actual final > > ldap add. > > Could you elaborate on your looping and deadlock concerns? I don't see > where they would arise if what we're watching is entirely independent of > our LDAP tree. I do not understand what you are 'watching' ? Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 18:11:48 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 13:11:48 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BD406.9090302@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> Message-ID: <1360779108.12328.133.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 10:57 -0700, Rich Megginson wrote: > > Rich, > > is there potential from deadlocking here due to the new transaction > > stuff ? Or can we single out this plugin to run before *any* > transaction > > is started ? > If you do this in a "regular" pre-op, not a "betxn" pre-op, then it > should be fine. Ok in this case we should be able to create a regular pre-op plugin to intercept the ldap add call and then use the following flow: client --(LDAP)--> 389DS --(HTTP/json)--> framework --(LDAP)--> add So no deadlocks will happen, the remaining issue is how to make sure we do not loop by mistake in the second add. One way could be to have loop detection so that if more then two (1. original, 2. framework) adds for the same DN come in we just return errors. Another way is to use a special objectclass as I proposed in the thread and make sure the framework explictly blacklists it so that it can never try to send an add with the special oc even by mistake or user misconfiguration. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Feb 13 18:30:24 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 13 Feb 2013 13:30:24 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360778690.12328.127.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> Message-ID: <511BDBC0.8070805@redhat.com> Simo Sorce wrote: > On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >> On 02/13/2013 12:53 PM, Simo Sorce wrote: >> >>> If we can solve the looping and potential deadlocking concerns I think >>> we can avoid the json reply and let the framework do the actual final >>> ldap add. >> >> Could you elaborate on your looping and deadlock concerns? I don't see >> where they would arise if what we're watching is entirely independent of >> our LDAP tree. > > I do not understand what you are 'watching' ? > > Simo. > I think he means have a persistent search to watch for new entries and then act upon them. rob From ondrej at hamada.cz Wed Feb 13 18:34:00 2013 From: ondrej at hamada.cz (Ondrej Hamada) Date: Wed, 13 Feb 2013 19:34:00 +0100 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <1360762565.12328.24.camel@willson.li.ssimo.org> References: <4FC8CF97.8000202@redhat.com> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> <1360675254.3838.8.camel@willson.li.ssimo.org> <511ADE8C.2000008@redhat.com> <1360762565.12328.24.camel@willson.li.ssimo.org> Message-ID: <511BDC98.5040305@hamada.cz> Dne 13.2.2013 14:36, Simo Sorce napsal(a): > On Tue, 2013-02-12 at 19:30 -0500, Dmitri Pal wrote: > >> It looks like thinks are starting to boil down to building a Kerberos proxy. >> Is this something that fits within your thesis agenda Ondra? > I guess that's for Ondrej to say, if it is too much we can simply start > working on the LDAP/replication side with rekeying and what not, and > deal with the KDC part at a later time. > > Simo. > Working on the LDAP/repl side fits the thesis agenda better, so I would like to go that way. Rekeying - do you mean some sort of plugin for transporting the krb keys from masters to consumers? Besides securing transport of keys what else should be done in ldap? I've only partial replication in my mind - I mean replication of entries selected by some kind of ldap filters. From abokovoy at redhat.com Wed Feb 13 18:38:46 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 13 Feb 2013 20:38:46 +0200 Subject: [Freeipa-devel] [PATCH] 352-354 Add support for AD users to hbactest command In-Reply-To: <511BBCA6.6000906@redhat.com> References: <50F98561.1040503@redhat.com> <1358620528.20683.98.camel@willson.li.ssimo.org> <50FF9B05.3020207@redhat.com> <1358947395.20683.365.camel@willson.li.ssimo.org> <5100DF88.8010502@redhat.com> <1359036276.20683.387.camel@willson.li.ssimo.org> <510BB677.1060201@redhat.com> <511B8433.1040307@redhat.com> <20130213131458.GC476@redhat.com> <511BBCA6.6000906@redhat.com> Message-ID: <20130213183846.GE476@redhat.com> On Wed, 13 Feb 2013, Martin Kosek wrote: >On 02/13/2013 02:14 PM, Alexander Bokovoy wrote: >> On Wed, 13 Feb 2013, Martin Kosek wrote: >>> On 02/01/2013 01:35 PM, Martin Kosek wrote: >>>> On 01/24/2013 03:04 PM, Simo Sorce wrote: >>>>> On Thu, 2013-01-24 at 08:15 +0100, Martin Kosek wrote: >>>>>> On 01/23/2013 02:23 PM, Simo Sorce wrote: >>>>>>> On Wed, 2013-01-23 at 09:10 +0100, Martin Kosek wrote: >>>>>>>> On 01/19/2013 07:35 PM, Simo Sorce wrote: >>>>>>>>> On Fri, 2013-01-18 at 18:24 +0100, Martin Kosek wrote: >>>>>>>>>> How this works: >>>>>>>>>> 1. When a trusted domain user is tested, AD GC is searched >>>>>>>>>> for the user entry Distinguished Name >>>>>>>>> >>>>>>>>> My head is not clear today but it looks to me you are doing 2 searches. >>>>>>>>> One to go from samAccountName -> DNa dn then a second for DN -> SID. >>>>>>>>> >>>>>>>>> Why are you doing 2 searches ? The first one can return you the >>>>>>>>> ObjectSid already. >>>>>>>>> >>>>>>>>> Simo. >>>>>>>> >>>>>>>> I had to do 2 searches because GC refuses to give me tokenGroups attribute >>>>>>>> content when I do not search with exact DN and LDAP SCOPE_BASE. So I >>>>>>>> have to do >>>>>>>> the first search to find out the DN of the searched user and then a second >>>>>>>> query to get the tokenGroups (and ObjectSid). >>>>>>> >>>>>>> I see, yes that makes sense, would you mind adding a comment to this >>>>>>> effect so we do not try to 'optimize' at some point ? >>>>>>> I have no additional concerns then. >>>>>>> >>>>>>> Simo. >>>>>>> >>>>>> >>>>>> Hello Simo, >>>>>> >>>>>> Thanks for review. Anyway, there is already a relevant comment in dcerpc.py, >>>>>> where the double search is performed: >>>>>> >>>>>> ... >>>>>> def get_trusted_domain_user_and_groups(self, object_name): >>>>>> ... >>>>>> entries = self.get_trusted_domain_objects(components.get('domain'), >>>>>> components.get('flatname'), filter, attrs, >>>>>> _ldap.SCOPE_SUBTREE) >>>>>> >>>>>> # Get SIDs of user object and it's groups >>>>>> # tokenGroups attribute must be read with scope BASE to avoid >>>>>> search error >>>>>> attrs = ['objectSID', 'tokenGroups'] >>>>>> ... >>>>>> >>>>>> I think it's enough to avoid "optimizing" this process - we would find out >>>>>> the >>>>>> "optimization" soon anyway, as the tokenGroups search would return error :-) >>>>> >>>>> Perfect! >>>>> >>>>> /me just had an eye vision exam, will complain to his doctor :-) >>>>> >>>> >>>> I enhanced the hbactest to also support user SID, not only trusted domain user >>>> name. Updated set of patches attached. >>>> >>>> How it works: >>>> >>>> # ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 --host >>>> `hostname` --service sshd >>>> -------------------- >>>> Access granted: True >>>> -------------------- >>>> Matched rules: can_login >>>> >>>> Martin >>> >>> Attaching a rebased set of patches. >> Thanks. >> >> I think we need to account for few more conditions. >> >> There are following cases: >> >> - user exists in AD >> - user does not exist in AD >> - group from AD specified as user >> >> The latter case is interesting because we can have groups from AD in >> external group mappings and people may specify them by mistake in >> hbactest. In such case I think we could have stated that group/user >> unknown or unmapped. >> >> Right now I'm getting less than helpful message: >> --------------------------------------------------------------------------- >> [root at jano ~]# ipa group-show extbar >> Group name: extbar >> Description: Ext bar >> Member of groups: foobar >> External member: S-1-5-21-3502988750-125904550-3683905862-512, >> S-1-5-21-3502988750-125904550-3683905862-500 >> [root at jano ~]# ipa group-show foobar >> Group name: foobar >> Description: foo bar >> GID: 944600004 >> Member groups: extbar >> [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-512 >> --host `hostname` --service sshd >> ipa: ERROR: trusted domain object not found >> [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-500 >> --host `hostname` --service sshd -------------------- >> Access granted: True >> -------------------- >> Matched rules: allow_all >> [root at jano ~]# ipa hbactest --user ADX\\Administrator --host `hostname` >> --service sshd >> -------------------- >> Access granted: True >> -------------------- >> Matched rules: allow_all >> [root at jano ~]# ipa hbactest --user ADX\\Domain\ Admins --host `hostname` >> --service sshd >> ipa: ERROR: trusted domain object not found >> [root at jano ~]# ipa hbactest --user ADX\\Enterprise\ Administrator --host >> `hostname` --service sshd >> ipa: ERROR: trusted domain object not found >> [root at jano ~]# ipa hbactest --user ADX\\DoesNotExist --host `hostname` >> --service sshd >> ipa: ERROR: trusted domain object not found >> ----------------------------------------------------------------------------- >> >> As you can see, group-as-user, unmapped group/user and non-existing AD >> user all raise error but the message is quite unclear -- does it talk >> about trusted domain itself or some object within the domain? Perhaps >> specifying real reason would be more helpful? >> >> If we are expecting only a user object, then I think we can change >> message a bit to state it more clear. >> >>> - return False >>> - return False >>> + if not found_flatname: >>> + raise errors.ValidationError(name=_('trusted domain object'), >>> + error= _('no trusted domain matched the specified >>> flat name')) >>> + if not entries: >>> + raise errors.NotFound(reason=_('trusted domain object not found')) >>> + >>> + return entries >> >> > >Fixed. This is quite straightforward as the errors are now returned via >specific exceptions and not with False/None result: > ># ipa hbactest --user "AD\Doesnotexist" --host `hostname` --service sshdipa: >ERROR: trusted domain user not found > >I am not convinced we should create a specific error when a group is passed >instead of user. To fix this, we would have to search GC with different filter >and also allow (|(objectclass=group)(objectclass=user)) instead of >(objectclass=user) which is there now and then check if the resulting object is >user or group. I am not convinced we want to do that, the error is now pretty >clear that user could not be found... > >I also extended hbactest help and added a section about testing trusted domain >users, I forgot to add it before. I am also attaching a related simple patch >fixing formatting for other hbactest examples. One more fix and we are done (see below). Thanks for refinements! >+HBACTEST AND TRUSTED DOMAINS >+ >+When an external trusted domain is configured in IPA, HBAC rules are also applied >+on users accessing IPA resources from the trusted domain. Trusted domain users and >+groups (and their SIDs) can be then assigned to external groups which can be >+members of POSIX groups in IPA which can be used in HBAC rules and thus allowing >+access to resources protected by the HBAC system. >+ >+hbactest plugin is capable of testing access for both local IPA users and users >+from the trusted domains, either by a fully qualified user name or by user SID. >+Such user names need to have a trusted domain specified as a short name >+(DOMAIN\Administrator) or with a user principal name (UPN), Administrator at ad.test. >+ >+Please note that hbactest executed with a trusted domain user as --user parameter >+can be only run by members of "trust admins" group. >+ >+EXAMPLES: >+ >+ 1. Test if a user from a trusted domain specified by its shortname matches any >+ rule: >+ >+ $ ipa hbactest --user "DOMAIN\Administrator" --host `hostname` --service sshd "Domain\Administrator" would not give working \ in most shells. You need to use 'Domain\Administrator' for that. >+ 4. Test if other user from a trusted domain specified by its SID matches any rule: >+ >+ $ ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 \\ >+ --host `hostname` --service sshd >+ -------------------- >+ Access granted: True >+ -------------------- >+ Matched rules: allow_all >+ Matched rules: can_login >+ >+ 5. Test if other user from a trusted domain specified by its shortname matches >+ any rule: >+ >+ $ ipa hbactest --user "DOMAIN\Otheruser" --host `hostname` --service sshd Same here. Once these fixed, please commit. -- / Alexander Bokovoy From jdennis at redhat.com Wed Feb 13 18:55:45 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 13 Feb 2013 13:55:45 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BDBC0.8070805@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> Message-ID: <511BE1B1.7000704@redhat.com> On 02/13/2013 01:30 PM, Rob Crittenden wrote: > Simo Sorce wrote: >> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>> >>>> If we can solve the looping and potential deadlocking concerns I think >>>> we can avoid the json reply and let the framework do the actual final >>>> ldap add. >>> >>> Could you elaborate on your looping and deadlock concerns? I don't see >>> where they would arise if what we're watching is entirely independent of >>> our LDAP tree. >> >> I do not understand what you are 'watching' ? >> >> Simo. >> > > I think he means have a persistent search to watch for new entries and > then act upon them. Yes, it could either be a persistent search or an external (cron) process that periodically polls LDAP. In each case there is an LDAP tree used as a "staging area". The staging area is completely independent of IPA's LDAP area. If something shows up in the staging area the contents of the staging area are used to drive our IPA commands. The staging area is then cleaned of all the entries which succeeded. Under this scenario what are the looping and deadlock concerns? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Wed Feb 13 18:59:04 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 13 Feb 2013 13:59:04 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BE1B1.7000704@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <511BE1B1.7000704@redhat.com> Message-ID: <511BE278.8050801@redhat.com> John Dennis wrote: > On 02/13/2013 01:30 PM, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>>> >>>>> If we can solve the looping and potential deadlocking concerns I think >>>>> we can avoid the json reply and let the framework do the actual final >>>>> ldap add. >>>> >>>> Could you elaborate on your looping and deadlock concerns? I don't see >>>> where they would arise if what we're watching is entirely >>>> independent of >>>> our LDAP tree. >>> >>> I do not understand what you are 'watching' ? >>> >>> Simo. >>> >> >> I think he means have a persistent search to watch for new entries and >> then act upon them. > > Yes, it could either be a persistent search or an external (cron) > process that periodically polls LDAP. In each case there is an LDAP tree > used as a "staging area". The staging area is completely independent of > IPA's LDAP area. If something shows up in the staging area the contents > of the staging area are used to drive our IPA commands. The staging area > is then cleaned of all the entries which succeeded. > > Under this scenario what are the looping and deadlock concerns? > > I don't think any. Simo's concern was doing an ADD within an ADD via his proposed 389-ds method. rob From simo at redhat.com Wed Feb 13 19:08:50 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 14:08:50 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BDBC0.8070805@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> Message-ID: <1360782530.12328.135.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: > Simo Sorce wrote: > > On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: > >> On 02/13/2013 12:53 PM, Simo Sorce wrote: > >> > >>> If we can solve the looping and potential deadlocking concerns I think > >>> we can avoid the json reply and let the framework do the actual final > >>> ldap add. > >> > >> Could you elaborate on your looping and deadlock concerns? I don't see > >> where they would arise if what we're watching is entirely independent of > >> our LDAP tree. > > > > I do not understand what you are 'watching' ? > > > > Simo. > > > > I think he means have a persistent search to watch for new entries and > then act upon them. I think a 2 step approach is misguided, so I've written it off from the start. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 13 19:10:21 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 13 Feb 2013 14:10:21 -0500 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <511BDC98.5040305@hamada.cz> References: <4FC8CF97.8000202@redhat.com> <4FCDE22B.5090000@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> <1360675254.3838.8.camel@willson.li.ssimo.org> <511ADE8C.2000008@redhat.com> <1360762565.12328.24.camel@willson.li.ssimo.org> <511BDC98.5040305@hamada.cz> Message-ID: <1360782621.12328.137.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 19:34 +0100, Ondrej Hamada wrote: > Dne 13.2.2013 14:36, Simo Sorce napsal(a): > > On Tue, 2013-02-12 at 19:30 -0500, Dmitri Pal wrote: > > > >> It looks like thinks are starting to boil down to building a Kerberos proxy. > >> Is this something that fits within your thesis agenda Ondra? > > I guess that's for Ondrej to say, if it is too much we can simply start > > working on the LDAP/replication side with rekeying and what not, and > > deal with the KDC part at a later time. > > > > Simo. > > > Working on the LDAP/repl side fits the thesis agenda better, so I would > like to go that way. > > Rekeying - do you mean some sort of plugin for transporting the krb keys > from masters to consumers? > > Besides securing transport of keys what else should be done in ldap? > I've only partial replication in my mind - I mean replication of entries > selected by some kind of ldap filters. We would need to re-encrypt keys so that we do not need to hand off to remote KDCs the same master key. This way a compromise in a branch office replica would not compromise the central infrastructure, but only affect the remote branch. Simo. -- Simo Sorce * Red Hat, Inc * New York From bcook at redhat.com Wed Feb 13 19:59:31 2013 From: bcook at redhat.com (Brian Cook) Date: Wed, 13 Feb 2013 11:59:31 -0800 Subject: [Freeipa-devel] installing from built RPMs on F18 Message-ID: <3074482B-3F26-44C3-8D64-46E3C16B37B8@redhat.com> Hello, When I try install from RPMs created from 'make rpms' on F18 I get the following error: 2013-02-13T19:49:27Z INFO The ipa-server-install command failed, exception: IndexError: list index out of range Here are the few log entries before it: 2013-02-13T19:49:27Z INFO File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 617, in run_script return_value = main_function() File "/sbin/ipa-server-install", line 986, in main dm_password, subject_base=options.subject) File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 623, in configure_instance self.start_creation(runtime=210) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 358, in start_creation method() File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1225, in __request_ra_certificate self.requestId = item_node[0].childNodes[0].data 2013-02-13T19:49:27Z INFO The ipa-server-install command failed, exception: IndexError: list index out of range Any idea what I am doing wrong? Thanks, Brian From jdennis at redhat.com Wed Feb 13 20:11:29 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 13 Feb 2013 15:11:29 -0500 Subject: [Freeipa-devel] installing from built RPMs on F18 In-Reply-To: <3074482B-3F26-44C3-8D64-46E3C16B37B8@redhat.com> References: <3074482B-3F26-44C3-8D64-46E3C16B37B8@redhat.com> Message-ID: <511BF371.50501@redhat.com> On 02/13/2013 02:59 PM, Brian Cook wrote: > Hello, > > When I try install from RPMs created from 'make rpms' on F18 I get the following error: > 2013-02-13T19:49:27Z INFO The ipa-server-install command failed, exception: IndexError: list index out of range > > > Here are the few log entries before it: > > 2013-02-13T19:49:27Z INFO File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 617, in run_script > return_value = main_function() > > File "/sbin/ipa-server-install", line 986, in main > dm_password, subject_base=options.subject) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 623, in configure_instance > self.start_creation(runtime=210) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 358, in start_creation > method() > > File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1225, in __request_ra_certificate > self.requestId = item_node[0].childNodes[0].data > > 2013-02-13T19:49:27Z INFO The ipa-server-install command failed, exception: IndexError: list index out of range > > > Any idea what I am doing wrong? You're not doing anything wrong. Caused by a bug in the nss package, see this thread https://www.redhat.com/archives/freeipa-users/2013-February/msg00195.html downgrade nss install upgrade nss -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Wed Feb 13 20:11:37 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 13 Feb 2013 15:11:37 -0500 Subject: [Freeipa-devel] installing from built RPMs on F18 In-Reply-To: <3074482B-3F26-44C3-8D64-46E3C16B37B8@redhat.com> References: <3074482B-3F26-44C3-8D64-46E3C16B37B8@redhat.com> Message-ID: <511BF379.2070301@redhat.com> Brian Cook wrote: > Hello, > > When I try install from RPMs created from 'make rpms' on F18 I get the following error: > 2013-02-13T19:49:27Z INFO The ipa-server-install command failed, exception: IndexError: list index out of range > > > Here are the few log entries before it: > > 2013-02-13T19:49:27Z INFO File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 617, in run_script > return_value = main_function() > > File "/sbin/ipa-server-install", line 986, in main > dm_password, subject_base=options.subject) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 623, in configure_instance > self.start_creation(runtime=210) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 358, in start_creation > method() > > File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1225, in __request_ra_certificate > self.requestId = item_node[0].childNodes[0].data > > 2013-02-13T19:49:27Z INFO The ipa-server-install command failed, exception: IndexError: list index out of range > > > Any idea what I am doing wrong? You aren't doing anything wrong. This is caused by a bug in the nss package, see this thread https://www.redhat.com/archives/freeipa-users/2013-February/msg00195.html rob From bcook at redhat.com Wed Feb 13 21:39:32 2013 From: bcook at redhat.com (Brian Cook) Date: Wed, 13 Feb 2013 13:39:32 -0800 Subject: [Freeipa-devel] patch for trac 2575 Message-ID: <9DD1D1BB-6B86-4EA1-B61B-B208E6BC7152@redhat.com> This is a patch for ticket 2575 on trac: [RFE] Installer wizard should prompt for DNS. This is my first time submitting a patch so I was looking for something that seemed relatively easy? Thanks, Brian diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install index 1559107..d8c4ae5 100755 --- a/install/tools/ipa-server-install +++ b/install/tools/ipa-server-install @@ -564,6 +564,7 @@ def main(): global pw_name global uninstalling global installation_cleanup + ds = None safe_options, options = parse_options() @@ -740,8 +741,18 @@ def main(): admin_password = "" reverse_zone = None - # check bind packages are installed + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection + setup_dns=False if options.setup_dns: + setup_dns=True + + # Ask user if they want to install DNS + if ipautil.user_input("Do you want to cnfigure integrated DNS (bind)?", false): + setup_dns=True + + + # check bind packages are installed + if setup_dns: if not bindinstance.check_inst(options.unattended): sys.exit("Aborting installation") @@ -827,7 +838,7 @@ def main(): else: admin_password = options.admin_password - if options.setup_dns: + if setup_dns: if options.no_forwarders: dns_forwarders = () elif options.forwarders: @@ -858,7 +869,7 @@ def main(): print "Realm name: %s" % realm_name print - if options.setup_dns: + if setup_dns: print "BIND DNS server will be configured to serve IPA domain with:" print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ else ", ".join([str(ip) for ip in dns_forwarders])) @@ -1102,7 +1113,7 @@ def main(): persistent_search=options.persistent_search, serial_autoincrement=options.serial_autoincrement, ca_configured=not options.selfsign) - if options.setup_dns: + if setup_dns: api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) bind.create_instance() @@ -1147,11 +1158,11 @@ def main(): print "\t\t * 80, 443: HTTP/HTTPS" print "\t\t * 389, 636: LDAP/LDAPS" print "\t\t * 88, 464: kerberos" - if options.setup_dns: + if setup_dns: print "\t\t * 53: bind" print "\t\tUDP Ports:" print "\t\t * 88, 464: kerberos" - if options.setup_dns: + if setup_dns: print "\t\t * 53: bind" if options.conf_ntp: print "\t\t * 123: ntp" -------------- next part -------------- An HTML attachment was scrubbed... URL: From nkinder at redhat.com Wed Feb 13 22:04:04 2013 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 13 Feb 2013 14:04:04 -0800 Subject: [Freeipa-devel] Nightly 389 Directory Server development yum repositories Message-ID: <511C0DD4.5090408@redhat.com> Hi, I have set up a nightly development yum repository for 389 Directory Server builds from "master". There are currently builds available for Fedora 18 (x86_64 and i686). I will work on adding builds for rawhide (F19) in the near future. New builds are made nightly from the "master" branch in Git, which will be updated to the repo around 12:00 UTC-08:00. To set a system up to use the development repository, download the following repo file and place it in the /etc/yum.repos.d directory: http://nkinder.fedorapeople.org/389-devel/389-devel-fedora.repo It is very important to understand what the purpose of this repo is (and what it is NOT). These are test builds that are provided for convenience for use in development and testing. The builds will very likely be unstable at times. You should NOT try to run these builds for anything important. I expect these builds to be used by the 389 development team for testing as we approach the release of new official packages. I also expect that these builds will be useful for other open source projects (such as FreeIPA) who want early access to upcoming 389 Directory Server versions during their own development cycles. It is also perfectly fine to use these builds to preview upcoming features and play with them, as long don't expect everything to always be stable and working. If you do encounter issues with the development builds that seem like bugs, please let us know about them. You can bring them up on our IRC channel (#389 on FreeNode), or file a ticket for it in our Trac instance here: https://fedorahosted.org/389/ Please let me know if you encounter any problems with the repository. Thanks, -NGK From bcook at redhat.com Wed Feb 13 22:20:56 2013 From: bcook at redhat.com (Brian Cook) Date: Wed, 13 Feb 2013 14:20:56 -0800 Subject: [Freeipa-devel] patch for trac 2575 In-Reply-To: References: Message-ID: <3E4F9F3A-594B-4BDE-B9A1-A069847B997E@redhat.com> Please disregard the first patch as it still asked the user if they want to install DNS even if --setup-dns was passed, this one is fixed. Brian diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install index 1559107..96ef802 100755 --- a/install/tools/ipa-server-install +++ b/install/tools/ipa-server-install @@ -564,6 +564,7 @@ def main(): global pw_name global uninstalling global installation_cleanup + ds = None safe_options, options = parse_options() @@ -740,8 +741,18 @@ def main(): admin_password = "" reverse_zone = None - # check bind packages are installed + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection + setup_dns=False if options.setup_dns: + setup_dns=True + else: + # Ask user if they want to install DNS + if ipautil.user_input("Do you want to configure integrated DNS (bind)?", False): + setup_dns=True + + + # check bind packages are installed + if setup_dns: if not bindinstance.check_inst(options.unattended): sys.exit("Aborting installation") @@ -827,7 +838,7 @@ def main(): else: admin_password = options.admin_password - if options.setup_dns: + if setup_dns: if options.no_forwarders: dns_forwarders = () elif options.forwarders: @@ -858,7 +869,7 @@ def main(): print "Realm name: %s" % realm_name print - if options.setup_dns: + if setup_dns: print "BIND DNS server will be configured to serve IPA domain with:" print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ else ", ".join([str(ip) for ip in dns_forwarders])) @@ -1102,7 +1113,7 @@ def main(): persistent_search=options.persistent_search, serial_autoincrement=options.serial_autoincrement, ca_configured=not options.selfsign) - if options.setup_dns: + if setup_dns: api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) bind.create_instance() @@ -1147,11 +1158,11 @@ def main(): print "\t\t * 80, 443: HTTP/HTTPS" print "\t\t * 389, 636: LDAP/LDAPS" print "\t\t * 88, 464: kerberos" - if options.setup_dns: + if setup_dns: print "\t\t * 53: bind" print "\t\tUDP Ports:" print "\t\t * 88, 464: kerberos" - if options.setup_dns: + if setup_dns: print "\t\t * 53: bind" if options.conf_ntp: print "\t\t * 123: ntp" > > Message: 8 > Date: Wed, 13 Feb 2013 13:39:32 -0800 > From: Brian Cook > To: "freeipa-devel at redhat.com" > Subject: [Freeipa-devel] patch for trac 2575 > Message-ID: <9DD1D1BB-6B86-4EA1-B61B-B208E6BC7152 at redhat.com> > Content-Type: text/plain; charset="windows-1252" > > This is a patch for ticket 2575 on trac: [RFE] Installer wizard should prompt for DNS. This is my first time submitting a patch so I was looking for something that seemed relatively easy? > > Thanks, > Brian > > > diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install > index 1559107..d8c4ae5 100755 > --- a/install/tools/ipa-server-install > +++ b/install/tools/ipa-server-install > @@ -564,6 +564,7 @@ def main(): > global pw_name > global uninstalling > global installation_cleanup > + > ds = None > > safe_options, options = parse_options() > @@ -740,8 +741,18 @@ def main(): > admin_password = "" > reverse_zone = None > > - # check bind packages are installed > + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection > + setup_dns=False > if options.setup_dns: > + setup_dns=True > + > + # Ask user if they want to install DNS > + if ipautil.user_input("Do you want to cnfigure integrated DNS (bind)?", false): > + setup_dns=True > + > + > + # check bind packages are installed > + if setup_dns: > if not bindinstance.check_inst(options.unattended): > sys.exit("Aborting installation") > > @@ -827,7 +838,7 @@ def main(): > else: > admin_password = options.admin_password > > - if options.setup_dns: > + if setup_dns: > if options.no_forwarders: > dns_forwarders = () > elif options.forwarders: > @@ -858,7 +869,7 @@ def main(): > print "Realm name: %s" % realm_name > print > > - if options.setup_dns: > + if setup_dns: > print "BIND DNS server will be configured to serve IPA domain with:" > print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ > else ", ".join([str(ip) for ip in dns_forwarders])) > @@ -1102,7 +1113,7 @@ def main(): > persistent_search=options.persistent_search, > serial_autoincrement=options.serial_autoincrement, > ca_configured=not options.selfsign) > - if options.setup_dns: > + if setup_dns: > api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) > > bind.create_instance() > @@ -1147,11 +1158,11 @@ def main(): > print "\t\t * 80, 443: HTTP/HTTPS" > print "\t\t * 389, 636: LDAP/LDAPS" > print "\t\t * 88, 464: kerberos" > - if options.setup_dns: > + if setup_dns: > print "\t\t * 53: bind" > print "\t\tUDP Ports:" > print "\t\t * 88, 464: kerberos" > - if options.setup_dns: > + if setup_dns: > print "\t\t * 53: bind" > if options.conf_ntp: > print "\t\t * 123: ntp" > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > End of Freeipa-devel Digest, Vol 69, Issue 49 > ********************************************* From dpal at redhat.com Thu Feb 14 02:18:16 2013 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 13 Feb 2013 21:18:16 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360782530.12328.135.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> Message-ID: <511C4968.5060606@redhat.com> On 02/13/2013 02:08 PM, Simo Sorce wrote: > On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>>> >>>>> If we can solve the looping and potential deadlocking concerns I think >>>>> we can avoid the json reply and let the framework do the actual final >>>>> ldap add. >>>> Could you elaborate on your looping and deadlock concerns? I don't see >>>> where they would arise if what we're watching is entirely independent of >>>> our LDAP tree. >>> I do not understand what you are 'watching' ? >>> >>> Simo. >>> >> I think he means have a persistent search to watch for new entries and >> then act upon them. > I think a 2 step approach is misguided, so I've written it off from the > start. > > Simo. > This all thread smells like rewriting winsync by using internal plugin and IPA framework. Is there any chance we can use existing winsync solution to do what we need but sync from staging instance rather than AD? Then the flow will be: HR system -> staging DS instance -> dirsync -> IPA Couple assumptions: a) We are satisfied with the security of the existing winsync solution and authentication used there. I do not see why it should be different here as it is a very similar use case. b) I expect we can sync from 389 to 389 with minor config changes and effectively no code changes c) The users created via winsync now are created in a proper way so IPA accepts then as IPA users This solves the problem of add and delete of the users. I know that winsync already supports group add/delete but we never qualified it in IPA. So may be it is time, at least for this use case. I would really think that this might be a lower cost solution than writing yet another custom plugin. Let us try to reuse what we already have. -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From nkinder at redhat.com Thu Feb 14 02:48:18 2013 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 13 Feb 2013 18:48:18 -0800 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C4968.5060606@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> Message-ID: <511C5072.80100@redhat.com> On 02/13/2013 06:18 PM, Dmitri Pal wrote: > On 02/13/2013 02:08 PM, Simo Sorce wrote: >> On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: >>> Simo Sorce wrote: >>>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>>>> >>>>>> If we can solve the looping and potential deadlocking concerns I think >>>>>> we can avoid the json reply and let the framework do the actual final >>>>>> ldap add. >>>>> Could you elaborate on your looping and deadlock concerns? I don't see >>>>> where they would arise if what we're watching is entirely independent of >>>>> our LDAP tree. >>>> I do not understand what you are 'watching' ? >>>> >>>> Simo. >>>> >>> I think he means have a persistent search to watch for new entries and >>> then act upon them. >> I think a 2 step approach is misguided, so I've written it off from the >> start. >> >> Simo. >> > This all thread smells like rewriting winsync by using internal plugin > and IPA framework. > Is there any chance we can use existing winsync solution to do what we > need but sync from staging instance rather than AD? > Then the flow will be: > HR system -> staging DS instance -> dirsync -> IPA > Couple assumptions: > a) We are satisfied with the security of the existing winsync solution > and authentication used there. I do not see why it should be different > here as it is a very similar use case. > b) I expect we can sync from 389 to 389 with minor config changes and > effectively no code changes This is not the case. Winsync uses the AD Dirsync control, which 389 does not support. The Winsync code also expects the AD schema on the entries coming in from the Dirsync response. Even if 389 supported the Dirsync control, there would be a fair amount of code changes to deal with whatever schema we would need to expect. > c) The users created via winsync now are created in a proper way so IPA > accepts then as IPA users > > This solves the problem of add and delete of the users. > I know that winsync already supports group add/delete but we never > qualified it in IPA. So may be it is time, at least for this use case. > I would really think that this might be a lower cost solution than > writing yet another custom plugin. > Let us try to reuse what we already have. I think a custom plug-in would be easier (though I do have concerns about the performance impact of the JSON approach). -NGK > From dpal at redhat.com Thu Feb 14 02:49:07 2013 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 13 Feb 2013 21:49:07 -0500 Subject: [Freeipa-devel] patch for trac 2575 In-Reply-To: <3E4F9F3A-594B-4BDE-B9A1-A069847B997E@redhat.com> References: <3E4F9F3A-594B-4BDE-B9A1-A069847B997E@redhat.com> Message-ID: <511C50A3.3040109@redhat.com> On 02/13/2013 05:20 PM, Brian Cook wrote: > Please disregard the first patch as it still asked the user if they want to install DNS even if --setup-dns was passed, this one is fixed. > > Brian Brian, Thanks for the patch. Can you please format it following these guidelines: https://fedorahosted.org/freeipa/wiki/PatchFormat Thanks Dmitri > > > > diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install > index 1559107..96ef802 100755 > --- a/install/tools/ipa-server-install > +++ b/install/tools/ipa-server-install > @@ -564,6 +564,7 @@ def main(): > global pw_name > global uninstalling > global installation_cleanup > + > ds = None > > safe_options, options = parse_options() > @@ -740,8 +741,18 @@ def main(): > admin_password = "" > reverse_zone = None > > - # check bind packages are installed > + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection > + setup_dns=False > if options.setup_dns: > + setup_dns=True > + else: > + # Ask user if they want to install DNS > + if ipautil.user_input("Do you want to configure integrated DNS (bind)?", False): > + setup_dns=True > + > + > + # check bind packages are installed > + if setup_dns: > if not bindinstance.check_inst(options.unattended): > sys.exit("Aborting installation") > > @@ -827,7 +838,7 @@ def main(): > else: > admin_password = options.admin_password > > - if options.setup_dns: > + if setup_dns: > if options.no_forwarders: > dns_forwarders = () > elif options.forwarders: > @@ -858,7 +869,7 @@ def main(): > print "Realm name: %s" % realm_name > print > > - if options.setup_dns: > + if setup_dns: > print "BIND DNS server will be configured to serve IPA domain with:" > print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ > else ", ".join([str(ip) for ip in dns_forwarders])) > @@ -1102,7 +1113,7 @@ def main(): > persistent_search=options.persistent_search, > serial_autoincrement=options.serial_autoincrement, > ca_configured=not options.selfsign) > - if options.setup_dns: > + if setup_dns: > api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) > > bind.create_instance() > @@ -1147,11 +1158,11 @@ def main(): > print "\t\t * 80, 443: HTTP/HTTPS" > print "\t\t * 389, 636: LDAP/LDAPS" > print "\t\t * 88, 464: kerberos" > - if options.setup_dns: > + if setup_dns: > print "\t\t * 53: bind" > print "\t\tUDP Ports:" > print "\t\t * 88, 464: kerberos" > - if options.setup_dns: > + if setup_dns: > print "\t\t * 53: bind" > if options.conf_ntp: > print "\t\t * 123: ntp" > > > > >> Message: 8 >> Date: Wed, 13 Feb 2013 13:39:32 -0800 >> From: Brian Cook >> To: "freeipa-devel at redhat.com" >> Subject: [Freeipa-devel] patch for trac 2575 >> Message-ID: <9DD1D1BB-6B86-4EA1-B61B-B208E6BC7152 at redhat.com> >> Content-Type: text/plain; charset="windows-1252" >> >> This is a patch for ticket 2575 on trac: [RFE] Installer wizard should prompt for DNS. This is my first time submitting a patch so I was looking for something that seemed relatively easy? >> >> Thanks, >> Brian >> >> >> diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install >> index 1559107..d8c4ae5 100755 >> --- a/install/tools/ipa-server-install >> +++ b/install/tools/ipa-server-install >> @@ -564,6 +564,7 @@ def main(): >> global pw_name >> global uninstalling >> global installation_cleanup >> + >> ds = None >> >> safe_options, options = parse_options() >> @@ -740,8 +741,18 @@ def main(): >> admin_password = "" >> reverse_zone = None >> >> - # check bind packages are installed >> + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection >> + setup_dns=False >> if options.setup_dns: >> + setup_dns=True >> + >> + # Ask user if they want to install DNS >> + if ipautil.user_input("Do you want to cnfigure integrated DNS (bind)?", false): >> + setup_dns=True >> + >> + >> + # check bind packages are installed >> + if setup_dns: >> if not bindinstance.check_inst(options.unattended): >> sys.exit("Aborting installation") >> >> @@ -827,7 +838,7 @@ def main(): >> else: >> admin_password = options.admin_password >> >> - if options.setup_dns: >> + if setup_dns: >> if options.no_forwarders: >> dns_forwarders = () >> elif options.forwarders: >> @@ -858,7 +869,7 @@ def main(): >> print "Realm name: %s" % realm_name >> print >> >> - if options.setup_dns: >> + if setup_dns: >> print "BIND DNS server will be configured to serve IPA domain with:" >> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >> else ", ".join([str(ip) for ip in dns_forwarders])) >> @@ -1102,7 +1113,7 @@ def main(): >> persistent_search=options.persistent_search, >> serial_autoincrement=options.serial_autoincrement, >> ca_configured=not options.selfsign) >> - if options.setup_dns: >> + if setup_dns: >> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) >> >> bind.create_instance() >> @@ -1147,11 +1158,11 @@ def main(): >> print "\t\t * 80, 443: HTTP/HTTPS" >> print "\t\t * 389, 636: LDAP/LDAPS" >> print "\t\t * 88, 464: kerberos" >> - if options.setup_dns: >> + if setup_dns: >> print "\t\t * 53: bind" >> print "\t\tUDP Ports:" >> print "\t\t * 88, 464: kerberos" >> - if options.setup_dns: >> + if setup_dns: >> print "\t\t * 53: bind" >> if options.conf_ntp: >> print "\t\t * 123: ntp" >> >> >> >> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> >> ------------------------------ >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel >> >> End of Freeipa-devel Digest, Vol 69, Issue 49 >> ********************************************* > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Thu Feb 14 02:55:24 2013 From: dpal at redhat.com (Dmitri Pal) Date: Wed, 13 Feb 2013 21:55:24 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C5072.80100@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> Message-ID: <511C521C.1070807@redhat.com> On 02/13/2013 09:48 PM, Nathan Kinder wrote: > On 02/13/2013 06:18 PM, Dmitri Pal wrote: >> On 02/13/2013 02:08 PM, Simo Sorce wrote: >>> On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: >>>> Simo Sorce wrote: >>>>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>>>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>>>>> >>>>>>> If we can solve the looping and potential deadlocking concerns I >>>>>>> think >>>>>>> we can avoid the json reply and let the framework do the actual >>>>>>> final >>>>>>> ldap add. >>>>>> Could you elaborate on your looping and deadlock concerns? I >>>>>> don't see >>>>>> where they would arise if what we're watching is entirely >>>>>> independent of >>>>>> our LDAP tree. >>>>> I do not understand what you are 'watching' ? >>>>> >>>>> Simo. >>>>> >>>> I think he means have a persistent search to watch for new entries and >>>> then act upon them. >>> I think a 2 step approach is misguided, so I've written it off from the >>> start. >>> >>> Simo. >>> >> This all thread smells like rewriting winsync by using internal plugin >> and IPA framework. >> Is there any chance we can use existing winsync solution to do what we >> need but sync from staging instance rather than AD? >> Then the flow will be: >> HR system -> staging DS instance -> dirsync -> IPA >> Couple assumptions: >> a) We are satisfied with the security of the existing winsync solution >> and authentication used there. I do not see why it should be different >> here as it is a very similar use case. >> b) I expect we can sync from 389 to 389 with minor config changes and >> effectively no code changes > This is not the case. Winsync uses the AD Dirsync control, which 389 > does not support. The Winsync code also expects the AD schema on the > entries coming in from the Dirsync response. Even if 389 supported > the Dirsync control, there would be a fair amount of code changes to > deal with whatever schema we would need to expect. >> c) The users created via winsync now are created in a proper way so IPA >> accepts then as IPA users >> >> This solves the problem of add and delete of the users. >> I know that winsync already supports group add/delete but we never >> qualified it in IPA. So may be it is time, at least for this use case. >> I would really think that this might be a lower cost solution than >> writing yet another custom plugin. >> Let us try to reuse what we already have. > I think a custom plug-in would be easier (though I do have concerns > about the performance impact of the JSON approach). > I really think that we rather have an external solution and not the one hot wired into the IPA internals via the proposed plugin. Something like an external proxy or gateway rather than the approach Simo proposed even if it would be more work. This work would be reusable. We already have several use cases for the LDAP proxy. This can be one more. A bit more and there will be enough need to build one anyways. The main value is that it is optional and external and acts as an ecosystem level solution that can be developed and tested on a separate schedule rather than being internal to IPA. > -NGK >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From nkinder at redhat.com Thu Feb 14 03:40:37 2013 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 13 Feb 2013 19:40:37 -0800 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511BCA89.9070804@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> Message-ID: <511C5CB5.2050101@redhat.com> On 02/13/2013 09:16 AM, Petr Viktorin wrote: > On 02/13/2013 05:57 PM, Simo Sorce wrote: >> On Wed, 2013-02-13 at 11:44 -0500, Rob Crittenden wrote: >>> Simo Sorce wrote: >>>> On Wed, 2013-02-13 at 16:12 +0100, Petr Viktorin wrote: >>>>> Our own post-callback assumes the user is already in LDAP, and who >>>>> knows what user-supplied callbacks will do. Keep in mind IPA is >>>>> plugable; at least for outside plugins' sake (if not our own >>>>> sanity's) >>>>> we should keep the number of code paths to a minimum. >>>> >>>> True which is why my proposal is to not use the standard user-add RPC >>>> call, but have a separate one. >>>> >>>> This separate call would only call the core business logic to >>>> create the >>>> user account add operation, but none of the external plumbing. >>>> >>>> Ideally we spit the framework flow like this: >>>> >>>> Normal user -> Real user-add --- . . . . . . . . . --- LDAP add >>>> \ / >>>> -- common logic -- >>>> / \ >>>> 389ds plugin -> Mock user-add -- . . . . . . . . . --- json reply >>>> >>>> >>>> custom plugins should be called in the custom logic an operate on the >>>> object before the ADD is attempted. >>>> >>>> If we do it this way then most of the code path will be in common >>>> which >>>> is what we want, and only the mechanical operation of adding the >>>> actual >>>> object to ldap will be different. >>>> >>>> Simo. >>>> >>> >>> There is one missing a few steps. A plugin execution looks like: >>> >>> Normal user -> Real user-add --- pre-op call(s) --- execute (LPAP add >>> record) --- post-op call(s) which may do additional add/modify >>> >>> It is the postop calls that would be the problem. They assume that the >>> entry has already been written (so, for example, it has a valid >>> UID/GID/ipaUniqueId, etc). >> >> Why are they done after the add ? It seem dangerous. > > This comment from the user-add post-op plugin should explain: > > # Fetch the entry again to update memberof, mep data, etc updated > # at the end of the transaction. > > Some processing happens in DS plugins, we can't get the data if the > object isn't added yet. You are referring to the fact that an 'ipa user-add' is actually multiple LDAP operations performed by the framework in IPA, right? I know at a minimum, the IPA framework performs 3 operations against DS when adding a user: - ADD operation to create the user entry (triggers a bunch of stuff in DS plug-ins) - SRCH operation to read the user entry with all of the dynamic stuff that was performed by the DS plug-ins - MOD operation to 'ipausers' group to add the new user as a group member With 389-ds-base 1.3.0, the pre/post-op DS plug-ins that IPA is concerned with are now betxn plug-ins, so that at least guarantees that the search operation will find everything it needs. The search could be eliminated if we add support for the 'Post-Read' control in DS as described in RFC 4527, which might be a good idea in the future (though I don't think it would really help us here). That still leaves the problem of an 'ipa user-add' being composed of multiple write operations against LDAP. With the DS plug-in approach that calls into the IPA framework with a 'mock add' to form the resulting entry at the pre-op stage, we could take care of the initial ADD operation of the user entry. We would still need to have a way to trigger the additional write operations that the IPA framework performs. The proposed DS plug-in could do an internal search operation after the add, then it could perform the additional write operations that are needed. This logic would need to be in the DS plug-in, or we would need another way to get the details from the IPA framework via JSON. The approach begins to get a bit messy. > >> What happens id the first ldap add succeed and the post op fails ? >> >> We should exceute the ldap call after the post ops are perfomed imho. >> > > From jdennis at redhat.com Thu Feb 14 04:30:46 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 13 Feb 2013 23:30:46 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C5CB5.2050101@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> Message-ID: <511C6876.20706@redhat.com> On 02/13/2013 10:40 PM, Nathan Kinder wrote: > With the DS plug-in approach that calls into the IPA framework with a > 'mock add' to form the resulting entry at the pre-op stage, we could > take care of the initial ADD operation of the user entry. We would > still need to have a way to trigger the additional write operations that > the IPA framework performs. The proposed DS plug-in could do an > internal search operation after the add, then it could perform the > additional write operations that are needed. This logic would need to > be in the DS plug-in, or we would need another way to get the details > from the IPA framework via JSON. The approach begins to get a bit messy. Messy is one word for it :-) Herein lies the problem. user-add is actually a simple case where in this particular version of our code we know the LDAP operations that occur and in what order. But there is nothing in our coding guidelines that guarantees this, it just happens to be true today. Pre and post callbacks are free to do as they please. The logic in our ldap module can change (in fact it's undergoing a rewrite as we speak). We've committed to supporting an extension mechanism (plugins) that ties into the framework operations and who knows what might occur in that code. At the moment it's not used, but hopefully it will be. Then comes the question, where do we draw the line? Do we say only user-add with no additional options is supported because we know what occurs during it's processing and hence we feel safe emulating user-add outside the framework? We all know this is a slippery slope, as soon as you support user-add this way we'll be getting requests to support other commands or other options. And after we make framework changes (which we do frequently) are we going to test these out-of-band operations continue to emulate the ever changing framework? It's a slippery slope that can expose us to problems we don't need. Jenny, do you want to test an entirely different mechanism or do you want to limit it to our defined API? We have a defined API, I really think that's the only thing we should support. Backdoor shortcuts that sidestep our framework should be off the table IMHO, it's just inviting too many problems. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From nkinder at redhat.com Thu Feb 14 05:16:58 2013 From: nkinder at redhat.com (Nathan Kinder) Date: Wed, 13 Feb 2013 21:16:58 -0800 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C6876.20706@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> Message-ID: <511C734A.2000203@redhat.com> On 02/13/2013 08:30 PM, John Dennis wrote: > On 02/13/2013 10:40 PM, Nathan Kinder wrote: >> With the DS plug-in approach that calls into the IPA framework with a >> 'mock add' to form the resulting entry at the pre-op stage, we could >> take care of the initial ADD operation of the user entry. We would >> still need to have a way to trigger the additional write operations that >> the IPA framework performs. The proposed DS plug-in could do an >> internal search operation after the add, then it could perform the >> additional write operations that are needed. This logic would need to >> be in the DS plug-in, or we would need another way to get the details >> from the IPA framework via JSON. The approach begins to get a bit >> messy. > > Messy is one word for it :-) Herein lies the problem. user-add is > actually a simple case where in this particular version of our code we > know the LDAP operations that occur and in what order. But there is > nothing in our coding guidelines that guarantees this, it just happens > to be true today. Pre and post callbacks are free to do as they > please. The logic in our ldap module can change (in fact it's > undergoing a rewrite as we speak). We've committed to supporting an > extension mechanism (plugins) that ties into the framework operations > and who knows what might occur in that code. At the moment it's not > used, but hopefully it will be. > > Then comes the question, where do we draw the line? Do we say only > user-add with no additional options is supported because we know what > occurs during it's processing and hence we feel safe emulating > user-add outside the framework? > > We all know this is a slippery slope, as soon as you support user-add > this way we'll be getting requests to support other commands or other > options. And after we make framework changes (which we do frequently) > are we going to test these out-of-band operations continue to emulate > the ever changing framework? > > It's a slippery slope that can expose us to problems we don't need. > > Jenny, do you want to test an entirely different mechanism or do you > want to limit it to our defined API? > > We have a defined API, I really think that's the only thing we should > support. Backdoor shortcuts that sidestep our framework should be off > the table IMHO, it's just inviting too many problems. This is why I think that the most sane approach is to put something in front of 'ipa user-add'. This is not without it's own set of difficulties, but if we need a basic LDAP interface that can be used to create IPA users, it should end up calling into the same code that 'ipa user-add' uses. This would ensure that any extensions (plugins) for the framework are called. I actually see this as very similar to the LDAP migration case. For migration, we basically take entries in LDIF format, extract the relevant data, then feed it into 'ipa user-add', right? That's more or less what we need to do here for HR systems that provision users, but the user addition needs to be pushed to IPA via an LDAP client instead of IPA pulling from an LDAP server. > > From jdennis at redhat.com Thu Feb 14 05:34:48 2013 From: jdennis at redhat.com (John Dennis) Date: Thu, 14 Feb 2013 00:34:48 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C734A.2000203@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> Message-ID: <511C7778.9070102@redhat.com> On 02/14/2013 12:16 AM, Nathan Kinder wrote: > On 02/13/2013 08:30 PM, John Dennis wrote: >> On 02/13/2013 10:40 PM, Nathan Kinder wrote: >>> With the DS plug-in approach that calls into the IPA framework with a >>> 'mock add' to form the resulting entry at the pre-op stage, we could >>> take care of the initial ADD operation of the user entry. We would >>> still need to have a way to trigger the additional write operations that >>> the IPA framework performs. The proposed DS plug-in could do an >>> internal search operation after the add, then it could perform the >>> additional write operations that are needed. This logic would need to >>> be in the DS plug-in, or we would need another way to get the details >>> from the IPA framework via JSON. The approach begins to get a bit >>> messy. >> >> Messy is one word for it :-) Herein lies the problem. user-add is >> actually a simple case where in this particular version of our code we >> know the LDAP operations that occur and in what order. But there is >> nothing in our coding guidelines that guarantees this, it just happens >> to be true today. Pre and post callbacks are free to do as they >> please. The logic in our ldap module can change (in fact it's >> undergoing a rewrite as we speak). We've committed to supporting an >> extension mechanism (plugins) that ties into the framework operations >> and who knows what might occur in that code. At the moment it's not >> used, but hopefully it will be. >> >> Then comes the question, where do we draw the line? Do we say only >> user-add with no additional options is supported because we know what >> occurs during it's processing and hence we feel safe emulating >> user-add outside the framework? >> >> We all know this is a slippery slope, as soon as you support user-add >> this way we'll be getting requests to support other commands or other >> options. And after we make framework changes (which we do frequently) >> are we going to test these out-of-band operations continue to emulate >> the ever changing framework? >> >> It's a slippery slope that can expose us to problems we don't need. >> >> Jenny, do you want to test an entirely different mechanism or do you >> want to limit it to our defined API? >> >> We have a defined API, I really think that's the only thing we should >> support. Backdoor shortcuts that sidestep our framework should be off >> the table IMHO, it's just inviting too many problems. > This is why I think that the most sane approach is to put something in > front of 'ipa user-add'. This is not without it's own set of > difficulties, but if we need a basic LDAP interface that can be used to > create IPA users, it should end up calling into the same code that 'ipa > user-add' uses. This would ensure that any extensions (plugins) for the > framework are called. > I actually see this as very similar to the LDAP migration case. For > migration, we basically take entries in LDIF format, extract the > relevant data, then feed it into 'ipa user-add', right? That's more or > less what we need to do here for HR systems that provision users, but > the user addition needs to be pushed to IPA via an LDAP client instead > of IPA pulling from an LDAP server. Yes, this is exactly what I'm saying and we concur. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Thu Feb 14 07:16:51 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 14 Feb 2013 08:16:51 +0100 Subject: [Freeipa-devel] patch for trac 2575 In-Reply-To: <511C50A3.3040109@redhat.com> References: <3E4F9F3A-594B-4BDE-B9A1-A069847B997E@redhat.com> <511C50A3.3040109@redhat.com> Message-ID: <511C8F63.4070307@redhat.com> On 02/14/2013 03:49 AM, Dmitri Pal wrote: > On 02/13/2013 05:20 PM, Brian Cook wrote: >> Please disregard the first patch as it still asked the user if they want to install DNS even if --setup-dns was passed, this one is fixed. >> >> Brian > > Brian, > > Thanks for the patch. > Can you please format it following these guidelines: > https://fedorahosted.org/freeipa/wiki/PatchFormat > > Thanks > Dmitri Hello Brian, Thanks for the patch! Also few technical notes: 1) There is no need to invent the new variable, you can ask and set options.setup_dns to True. We already to this in other parts incode 2) This patch would --unattended mode when no --setup-dns is passed Martin >> >> >> >> diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install >> index 1559107..96ef802 100755 >> --- a/install/tools/ipa-server-install >> +++ b/install/tools/ipa-server-install >> @@ -564,6 +564,7 @@ def main(): >> global pw_name >> global uninstalling >> global installation_cleanup >> + >> ds = None >> >> safe_options, options = parse_options() >> @@ -740,8 +741,18 @@ def main(): >> admin_password = "" >> reverse_zone = None >> >> - # check bind packages are installed >> + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection >> + setup_dns=False >> if options.setup_dns: >> + setup_dns=True >> + else: >> + # Ask user if they want to install DNS >> + if ipautil.user_input("Do you want to configure integrated DNS (bind)?", False): >> + setup_dns=True >> + >> + >> + # check bind packages are installed >> + if setup_dns: >> if not bindinstance.check_inst(options.unattended): >> sys.exit("Aborting installation") >> >> @@ -827,7 +838,7 @@ def main(): >> else: >> admin_password = options.admin_password >> >> - if options.setup_dns: >> + if setup_dns: >> if options.no_forwarders: >> dns_forwarders = () >> elif options.forwarders: >> @@ -858,7 +869,7 @@ def main(): >> print "Realm name: %s" % realm_name >> print >> >> - if options.setup_dns: >> + if setup_dns: >> print "BIND DNS server will be configured to serve IPA domain with:" >> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >> else ", ".join([str(ip) for ip in dns_forwarders])) >> @@ -1102,7 +1113,7 @@ def main(): >> persistent_search=options.persistent_search, >> serial_autoincrement=options.serial_autoincrement, >> ca_configured=not options.selfsign) >> - if options.setup_dns: >> + if setup_dns: >> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) >> >> bind.create_instance() >> @@ -1147,11 +1158,11 @@ def main(): >> print "\t\t * 80, 443: HTTP/HTTPS" >> print "\t\t * 389, 636: LDAP/LDAPS" >> print "\t\t * 88, 464: kerberos" >> - if options.setup_dns: >> + if setup_dns: >> print "\t\t * 53: bind" >> print "\t\tUDP Ports:" >> print "\t\t * 88, 464: kerberos" >> - if options.setup_dns: >> + if setup_dns: >> print "\t\t * 53: bind" >> if options.conf_ntp: >> print "\t\t * 123: ntp" >> >> >> >> >>> Message: 8 >>> Date: Wed, 13 Feb 2013 13:39:32 -0800 >>> From: Brian Cook >>> To: "freeipa-devel at redhat.com" >>> Subject: [Freeipa-devel] patch for trac 2575 >>> Message-ID: <9DD1D1BB-6B86-4EA1-B61B-B208E6BC7152 at redhat.com> >>> Content-Type: text/plain; charset="windows-1252" >>> >>> This is a patch for ticket 2575 on trac: [RFE] Installer wizard should prompt for DNS. This is my first time submitting a patch so I was looking for something that seemed relatively easy? >>> >>> Thanks, >>> Brian >>> >>> >>> diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install >>> index 1559107..d8c4ae5 100755 >>> --- a/install/tools/ipa-server-install >>> +++ b/install/tools/ipa-server-install >>> @@ -564,6 +564,7 @@ def main(): >>> global pw_name >>> global uninstalling >>> global installation_cleanup >>> + >>> ds = None >>> >>> safe_options, options = parse_options() >>> @@ -740,8 +741,18 @@ def main(): >>> admin_password = "" >>> reverse_zone = None >>> >>> - # check bind packages are installed >>> + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection >>> + setup_dns=False >>> if options.setup_dns: >>> + setup_dns=True >>> + >>> + # Ask user if they want to install DNS >>> + if ipautil.user_input("Do you want to cnfigure integrated DNS (bind)?", false): >>> + setup_dns=True >>> + >>> + >>> + # check bind packages are installed >>> + if setup_dns: >>> if not bindinstance.check_inst(options.unattended): >>> sys.exit("Aborting installation") >>> >>> @@ -827,7 +838,7 @@ def main(): >>> else: >>> admin_password = options.admin_password >>> >>> - if options.setup_dns: >>> + if setup_dns: >>> if options.no_forwarders: >>> dns_forwarders = () >>> elif options.forwarders: >>> @@ -858,7 +869,7 @@ def main(): >>> print "Realm name: %s" % realm_name >>> print >>> >>> - if options.setup_dns: >>> + if setup_dns: >>> print "BIND DNS server will be configured to serve IPA domain with:" >>> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >>> else ", ".join([str(ip) for ip in dns_forwarders])) >>> @@ -1102,7 +1113,7 @@ def main(): >>> persistent_search=options.persistent_search, >>> serial_autoincrement=options.serial_autoincrement, >>> ca_configured=not options.selfsign) >>> - if options.setup_dns: >>> + if setup_dns: >>> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) >>> >>> bind.create_instance() >>> @@ -1147,11 +1158,11 @@ def main(): >>> print "\t\t * 80, 443: HTTP/HTTPS" >>> print "\t\t * 389, 636: LDAP/LDAPS" >>> print "\t\t * 88, 464: kerberos" >>> - if options.setup_dns: >>> + if setup_dns: >>> print "\t\t * 53: bind" >>> print "\t\tUDP Ports:" >>> print "\t\t * 88, 464: kerberos" >>> - if options.setup_dns: >>> + if setup_dns: >>> print "\t\t * 53: bind" >>> if options.conf_ntp: >>> print "\t\t * 123: ntp" >>> >>> >>> >>> >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> >>> End of Freeipa-devel Digest, Vol 69, Issue 49 >>> ********************************************* >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > From mkosek at redhat.com Thu Feb 14 07:44:49 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 14 Feb 2013 08:44:49 +0100 Subject: [Freeipa-devel] [PATCH] 352-354 Add support for AD users to hbactest command In-Reply-To: <20130213183846.GE476@redhat.com> References: <50F98561.1040503@redhat.com> <1358620528.20683.98.camel@willson.li.ssimo.org> <50FF9B05.3020207@redhat.com> <1358947395.20683.365.camel@willson.li.ssimo.org> <5100DF88.8010502@redhat.com> <1359036276.20683.387.camel@willson.li.ssimo.org> <510BB677.1060201@redhat.com> <511B8433.1040307@redhat.com> <20130213131458.GC476@redhat.com> <511BBCA6.6000906@redhat.com> <20130213183846.GE476@redhat.com> Message-ID: <511C95F1.2000906@redhat.com> On 02/13/2013 07:38 PM, Alexander Bokovoy wrote: > On Wed, 13 Feb 2013, Martin Kosek wrote: >> On 02/13/2013 02:14 PM, Alexander Bokovoy wrote: >>> On Wed, 13 Feb 2013, Martin Kosek wrote: >>>> On 02/01/2013 01:35 PM, Martin Kosek wrote: >>>>> On 01/24/2013 03:04 PM, Simo Sorce wrote: >>>>>> On Thu, 2013-01-24 at 08:15 +0100, Martin Kosek wrote: >>>>>>> On 01/23/2013 02:23 PM, Simo Sorce wrote: >>>>>>>> On Wed, 2013-01-23 at 09:10 +0100, Martin Kosek wrote: >>>>>>>>> On 01/19/2013 07:35 PM, Simo Sorce wrote: >>>>>>>>>> On Fri, 2013-01-18 at 18:24 +0100, Martin Kosek wrote: >>>>>>>>>>> How this works: >>>>>>>>>>> 1. When a trusted domain user is tested, AD GC is searched >>>>>>>>>>> for the user entry Distinguished Name >>>>>>>>>> >>>>>>>>>> My head is not clear today but it looks to me you are doing 2 searches. >>>>>>>>>> One to go from samAccountName -> DNa dn then a second for DN -> SID. >>>>>>>>>> >>>>>>>>>> Why are you doing 2 searches ? The first one can return you the >>>>>>>>>> ObjectSid already. >>>>>>>>>> >>>>>>>>>> Simo. >>>>>>>>> >>>>>>>>> I had to do 2 searches because GC refuses to give me tokenGroups >>>>>>>>> attribute >>>>>>>>> content when I do not search with exact DN and LDAP SCOPE_BASE. So I >>>>>>>>> have to do >>>>>>>>> the first search to find out the DN of the searched user and then a >>>>>>>>> second >>>>>>>>> query to get the tokenGroups (and ObjectSid). >>>>>>>> >>>>>>>> I see, yes that makes sense, would you mind adding a comment to this >>>>>>>> effect so we do not try to 'optimize' at some point ? >>>>>>>> I have no additional concerns then. >>>>>>>> >>>>>>>> Simo. >>>>>>>> >>>>>>> >>>>>>> Hello Simo, >>>>>>> >>>>>>> Thanks for review. Anyway, there is already a relevant comment in >>>>>>> dcerpc.py, >>>>>>> where the double search is performed: >>>>>>> >>>>>>> ... >>>>>>> def get_trusted_domain_user_and_groups(self, object_name): >>>>>>> ... >>>>>>> entries = self.get_trusted_domain_objects(components.get('domain'), >>>>>>> components.get('flatname'), filter, attrs, >>>>>>> _ldap.SCOPE_SUBTREE) >>>>>>> >>>>>>> # Get SIDs of user object and it's groups >>>>>>> # tokenGroups attribute must be read with scope BASE to avoid >>>>>>> search error >>>>>>> attrs = ['objectSID', 'tokenGroups'] >>>>>>> ... >>>>>>> >>>>>>> I think it's enough to avoid "optimizing" this process - we would find out >>>>>>> the >>>>>>> "optimization" soon anyway, as the tokenGroups search would return error >>>>>>> :-) >>>>>> >>>>>> Perfect! >>>>>> >>>>>> /me just had an eye vision exam, will complain to his doctor :-) >>>>>> >>>>> >>>>> I enhanced the hbactest to also support user SID, not only trusted domain >>>>> user >>>>> name. Updated set of patches attached. >>>>> >>>>> How it works: >>>>> >>>>> # ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 --host >>>>> `hostname` --service sshd >>>>> -------------------- >>>>> Access granted: True >>>>> -------------------- >>>>> Matched rules: can_login >>>>> >>>>> Martin >>>> >>>> Attaching a rebased set of patches. >>> Thanks. >>> >>> I think we need to account for few more conditions. >>> >>> There are following cases: >>> >>> - user exists in AD >>> - user does not exist in AD >>> - group from AD specified as user >>> >>> The latter case is interesting because we can have groups from AD in >>> external group mappings and people may specify them by mistake in >>> hbactest. In such case I think we could have stated that group/user >>> unknown or unmapped. >>> >>> Right now I'm getting less than helpful message: >>> --------------------------------------------------------------------------- >>> [root at jano ~]# ipa group-show extbar >>> Group name: extbar >>> Description: Ext bar >>> Member of groups: foobar >>> External member: S-1-5-21-3502988750-125904550-3683905862-512, >>> S-1-5-21-3502988750-125904550-3683905862-500 >>> [root at jano ~]# ipa group-show foobar >>> Group name: foobar >>> Description: foo bar >>> GID: 944600004 >>> Member groups: extbar >>> [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-512 >>> --host `hostname` --service sshd >>> ipa: ERROR: trusted domain object not found >>> [root at jano ~]# ipa hbactest --user S-1-5-21-3502988750-125904550-3683905862-500 >>> --host `hostname` --service sshd >>> -------------------- >>> Access granted: True >>> -------------------- >>> Matched rules: allow_all >>> [root at jano ~]# ipa hbactest --user ADX\\Administrator --host `hostname` >>> --service sshd >>> -------------------- >>> Access granted: True >>> -------------------- >>> Matched rules: allow_all >>> [root at jano ~]# ipa hbactest --user ADX\\Domain\ Admins --host `hostname` >>> --service sshd >>> ipa: ERROR: trusted domain object not found >>> [root at jano ~]# ipa hbactest --user ADX\\Enterprise\ Administrator --host >>> `hostname` --service sshd >>> ipa: ERROR: trusted domain object not found >>> [root at jano ~]# ipa hbactest --user ADX\\DoesNotExist --host `hostname` >>> --service sshd >>> ipa: ERROR: trusted domain object not found >>> ----------------------------------------------------------------------------- >>> >>> As you can see, group-as-user, unmapped group/user and non-existing AD >>> user all raise error but the message is quite unclear -- does it talk >>> about trusted domain itself or some object within the domain? Perhaps >>> specifying real reason would be more helpful? >>> >>> If we are expecting only a user object, then I think we can change >>> message a bit to state it more clear. >>> >>>> - return False >>>> - return False >>>> + if not found_flatname: >>>> + raise errors.ValidationError(name=_('trusted domain object'), >>>> + error= _('no trusted domain matched the specified >>>> flat name')) >>>> + if not entries: >>>> + raise errors.NotFound(reason=_('trusted domain object not >>>> found')) >>>> + >>>> + return entries >>> >>> >> >> Fixed. This is quite straightforward as the errors are now returned via >> specific exceptions and not with False/None result: >> >> # ipa hbactest --user "AD\Doesnotexist" --host `hostname` --service sshdipa: >> ERROR: trusted domain user not found >> >> I am not convinced we should create a specific error when a group is passed >> instead of user. To fix this, we would have to search GC with different filter >> and also allow (|(objectclass=group)(objectclass=user)) instead of >> (objectclass=user) which is there now and then check if the resulting object is >> user or group. I am not convinced we want to do that, the error is now pretty >> clear that user could not be found... >> >> I also extended hbactest help and added a section about testing trusted domain >> users, I forgot to add it before. I am also attaching a related simple patch >> fixing formatting for other hbactest examples. > One more fix and we are done (see below). > > Thanks for refinements! > >> +HBACTEST AND TRUSTED DOMAINS >> + >> +When an external trusted domain is configured in IPA, HBAC rules are also >> applied >> +on users accessing IPA resources from the trusted domain. Trusted domain >> users and >> +groups (and their SIDs) can be then assigned to external groups which can be >> +members of POSIX groups in IPA which can be used in HBAC rules and thus >> allowing >> +access to resources protected by the HBAC system. >> + >> +hbactest plugin is capable of testing access for both local IPA users and users >> +from the trusted domains, either by a fully qualified user name or by user SID. >> +Such user names need to have a trusted domain specified as a short name >> +(DOMAIN\Administrator) or with a user principal name (UPN), >> Administrator at ad.test. >> + >> +Please note that hbactest executed with a trusted domain user as --user >> parameter >> +can be only run by members of "trust admins" group. >> + >> +EXAMPLES: >> + >> + 1. Test if a user from a trusted domain specified by its shortname >> matches any >> + rule: >> + >> + $ ipa hbactest --user "DOMAIN\Administrator" --host `hostname` --service >> sshd > "Domain\Administrator" would not give working \ in most shells. You need > to use 'Domain\Administrator' for that. > >> + 4. Test if other user from a trusted domain specified by its SID matches >> any rule: >> + >> + $ ipa hbactest --user S-1-5-21-3035198329-144811719-1378114514-500 \\ >> + --host `hostname` --service sshd >> + -------------------- >> + Access granted: True >> + -------------------- >> + Matched rules: allow_all >> + Matched rules: can_login >> + >> + 5. Test if other user from a trusted domain specified by its shortname >> matches >> + any rule: >> + >> + $ ipa hbactest --user "DOMAIN\Otheruser" --host `hostname` --service sshd > Same here. > > Once these fixed, please commit. Fixed. Pushed to master, ipa-3-1. Thanks for review both of you! Martin From pviktori at redhat.com Thu Feb 14 08:59:17 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 09:59:17 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360779108.12328.133.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> Message-ID: <511CA765.1010704@redhat.com> On 02/13/2013 07:11 PM, Simo Sorce wrote: > On Wed, 2013-02-13 at 10:57 -0700, Rich Megginson wrote: > >>> Rich, >>> is there potential from deadlocking here due to the new transaction >>> stuff ? Or can we single out this plugin to run before *any* >> transaction >>> is started ? > >> If you do this in a "regular" pre-op, not a "betxn" pre-op, then it >> should be fine. > > Ok in this case we should be able to create a regular pre-op plugin to > intercept the ldap add call and then use the following flow: > client --(LDAP)--> 389DS --(HTTP/json)--> framework --(LDAP)--> add > > So no deadlocks will happen, the remaining issue is how to make sure we > do not loop by mistake in the second add. > > One way could be to have loop detection so that if more then two (1. > original, 2. framework) adds for the same DN come in we just return > errors. Another way is to use a special objectclass as I proposed in the > thread and make sure the framework explictly blacklists it so that it > can never try to send an add with the special oc even by mistake or user > misconfiguration. > And a third way is a separate LDAP tree. I'm not familiar with what DS plugins can do. Can we craft one that intercepts all read operations on "cn=HR tree,$SUFFIX" and does them on "cn=users,cn=accounts,$SUFFIX" instead (using that tree's permissions), and forwards all write operations on "cn=HR tree" to IPA via JSON? A fourth way is a proxy/gateway, essentially the same but with a separate server. -- Petr? From tbabej at redhat.com Thu Feb 14 09:05:30 2013 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 14 Feb 2013 10:05:30 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <511A82D9.3030202@redhat.com> References: <510FE077.8010801@redhat.com> <511A82D9.3030202@redhat.com> Message-ID: <511CA8DA.1060900@redhat.com> On 02/12/2013 06:58 PM, Petr Vobornik wrote: > On 02/04/2013 05:23 PM, Tomas Babej wrote: >> Hi, >> >> When adding/modifying an ID range for a trusted domain, the newly >> added option --dom-name can be used. This looks up SID of the >> trusted domain in LDAP and therefore the user is not required >> to write it down in CLI. If the lookup fails, error message >> asking the user to specify the SID manually is shown. >> >> https://fedorahosted.org/freeipa/ticket/3133 >> >> Tomas >> >> > > Just wondering: How bad would it be to not introduce new virtual > attribute and just use the ipanttrusteddomainsid. On add and mod (when > ipanttrusteddomainsid is set) we would check if ipanttrusteddomainsid > is SID. If not, it would be treated as domain name and > get_trusted_domain_sid_from_name method will be used to get the SID. > > I'm asking because I don't really like virtual and standard attributes > for the same ldap attribute in a mod command. In WEB UI details page > we have to display only one field - ipanttrusteddomainsid. > > So we are left with options: > 1) do not use this feature for mod operations in Web UI > 2) enter domain name in ipanttrusteddomainsid field, implement the > aforementioned check in Web UI and fill the correct option in RPC request > 3) add special action into action list which will open new dialog, > user will enter domain name, mod command with ipanttrusteddomainname > set will be executed on confirmation > 4) some other method > > I don't really like any of the options. If a SID check is an easy > operation, we can go with #2, but I would still rather see this logic > in server plugin. Just for the record, after a short discussion with Petr we decided to keep the virtual attribute ipatrustedomainname as it is. The idea of having ipatrusteddomainsid do two different things seems rather confusing, and in the end would be probably less user-friendly. In the WebUI, user should be able to enter SID using either domain name or domain SID. The proposal is that user would be able to either modify domain SID or enter domain name (and therefore modify SID). Tomas From sbose at redhat.com Thu Feb 14 09:07:20 2013 From: sbose at redhat.com (Sumit Bose) Date: Thu, 14 Feb 2013 10:07:20 +0100 Subject: [Freeipa-devel] [PATCH] 370-371 ipa-kdb: remove memory leaks In-Reply-To: <511A62FC.5080907@redhat.com> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> <511A5C5C.3030807@redhat.com> <1360682788.3838.45.camel@willson.li.ssimo.org> <511A62FC.5080907@redhat.com> Message-ID: <20130214090719.GO24753@localhost.localdomain> On Tue, Feb 12, 2013 at 04:42:52PM +0100, Martin Kosek wrote: > On 02/12/2013 04:26 PM, Simo Sorce wrote: > > On Tue, 2013-02-12 at 16:14 +0100, Martin Kosek wrote: > >> Explained in the commit description - this may not be super-critical, I just > >> followed info in ldap_search_ext() man page: > >> > >> ... > >> > >> Note that res parameter of ldap_search_ext_s() and ldap_search_s() > >> should be freed with > >> ldap_msgfree() regardless of return value of these functions. > >> ... > > > > OK. > > > >>> This snippet seem to change the logic. > >>> > >>> Before the condition was !ipadb_need_retry() and no you change it to > >>> ipadb_need_retry() so it looks reversed, was this on purpose ? > >> > >> As noted in commit description, this check was wrong and it was causing the > >> function to _always_ retry at least once because ipadb_need_retry returns true > >> when we need to retry and not 0 -> thus no "!" is needed. > > > > I do not like much 'sneaking' this kind of change in the same patch that > > fixes memory leaks, so if you want to plit in 2 patches it would be > > nice, but I am of course ok with the change. > > I agree with that. Lets do it right as we have not pushed it yet. I moved this > change to a separate patch (attached). I didn't found any issues while testing, so ACK. When I run kinit in a endless loop I still see a constant increase in the resident memory size, but it looks like it is not related to the ipadb code. I will investigate further. bye, Sumit > > Martin From mkosek at redhat.com Thu Feb 14 09:16:44 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 14 Feb 2013 10:16:44 +0100 Subject: [Freeipa-devel] [PATCH] 370-371 ipa-kdb: remove memory leaks In-Reply-To: <20130214090719.GO24753@localhost.localdomain> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> <511A5C5C.3030807@redhat.com> <1360682788.3838.45.camel@willson.li.ssimo.org> <511A62FC.5080907@redhat.com> <20130214090719.GO24753@localhost.localdomain> Message-ID: <511CAB7C.3040504@redhat.com> On 02/14/2013 10:07 AM, Sumit Bose wrote: > On Tue, Feb 12, 2013 at 04:42:52PM +0100, Martin Kosek wrote: >> On 02/12/2013 04:26 PM, Simo Sorce wrote: >>> On Tue, 2013-02-12 at 16:14 +0100, Martin Kosek wrote: >>>> Explained in the commit description - this may not be super-critical, I just >>>> followed info in ldap_search_ext() man page: >>>> >>>> ... >>>> >>>> Note that res parameter of ldap_search_ext_s() and ldap_search_s() >>>> should be freed with >>>> ldap_msgfree() regardless of return value of these functions. >>>> ... >>> >>> OK. >>> >>>>> This snippet seem to change the logic. >>>>> >>>>> Before the condition was !ipadb_need_retry() and no you change it to >>>>> ipadb_need_retry() so it looks reversed, was this on purpose ? >>>> >>>> As noted in commit description, this check was wrong and it was causing the >>>> function to _always_ retry at least once because ipadb_need_retry returns true >>>> when we need to retry and not 0 -> thus no "!" is needed. >>> >>> I do not like much 'sneaking' this kind of change in the same patch that >>> fixes memory leaks, so if you want to plit in 2 patches it would be >>> nice, but I am of course ok with the change. >> >> I agree with that. Lets do it right as we have not pushed it yet. I moved this >> change to a separate patch (attached). > > I didn't found any issues while testing, so ACK. > > When I run kinit in a endless loop I still see a constant increase in > the resident memory size, but it looks like it is not related to the > ipadb code. I will investigate further. > > bye, > Sumit > Ok, thanks. Pushed to master, ipa-3-1. Martin From mkosek at redhat.com Thu Feb 14 09:17:16 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 14 Feb 2013 10:17:16 +0100 Subject: [Freeipa-devel] [PATCH] ipa-kdb: Free talloc autofree context when module is closed In-Reply-To: <5119F4A8.1060307@redhat.com> References: <20130211160019.GJ24753@localhost.localdomain> <5119F4A8.1060307@redhat.com> Message-ID: <511CAB9C.9060408@redhat.com> On 02/12/2013 08:52 AM, Martin Kosek wrote: > On 02/11/2013 05:00 PM, Sumit Bose wrote: >> Hi, >> >> this patch should fix https://fedorahosted.org/freeipa/ticket/3410 . See >> commit message for details. >> >> bye, >> Sumit >> > > This fixed the ipa-kdb crashes for me, so ACK. > > Martin > Pushed to master, ipa-3-1. Martin From pspacek at redhat.com Thu Feb 14 09:36:32 2013 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 14 Feb 2013 10:36:32 +0100 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <1360782621.12328.137.camel@willson.li.ssimo.org> References: <4FC8CF97.8000202@redhat.com> <1338899122.8230.253.camel@willson.li.ssimo.org> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> <1360675254.3838.8.camel@willson.li.ssimo.org> <511ADE8C.2000008@redhat.com> <1360762565.12328.24.camel@willson.li.ssimo.org> <511BDC98.5040305@hamada.cz> <1360782621.12328.137.camel@willson.li.ssimo.! org> Message-ID: <511CB020.5060608@redhat.com> On 13.2.2013 20:10, Simo Sorce wrote: > On Wed, 2013-02-13 at 19:34 +0100, Ondrej Hamada wrote: >> Dne 13.2.2013 14:36, Simo Sorce napsal(a): >>> On Tue, 2013-02-12 at 19:30 -0500, Dmitri Pal wrote: >>> >>>> It looks like thinks are starting to boil down to building a Kerberos proxy. >>>> Is this something that fits within your thesis agenda Ondra? >>> I guess that's for Ondrej to say, if it is too much we can simply start >>> working on the LDAP/replication side with rekeying and what not, and >>> deal with the KDC part at a later time. >>> >>> Simo. >>> >> Working on the LDAP/repl side fits the thesis agenda better, so I would >> like to go that way. >> >> Rekeying - do you mean some sort of plugin for transporting the krb keys >> from masters to consumers? >> >> Besides securing transport of keys what else should be done in ldap? >> I've only partial replication in my mind - I mean replication of entries >> selected by some kind of ldap filters. > > We would need to re-encrypt keys so that we do not need to hand off to > remote KDCs the same master key. > This way a compromise in a branch office replica would not compromise > the central infrastructure, but only affect the remote branch. Generally, this feature could be nice for "plain" Kerberos. (I mean some re-keying integration with classical "kprop" mechanism.) -- Petr^2 Spacek From pviktori at redhat.com Thu Feb 14 09:45:21 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 10:45:21 +0100 Subject: [Freeipa-devel] [PATCH] 90 Run interactive_prompt callbacks after CSV values are split In-Reply-To: <50EDA525.1090104@redhat.com> References: <50EDA525.1090104@redhat.com> Message-ID: <511CB231.7030906@redhat.com> On 01/09/2013 06:13 PM, Jan Cholasta wrote: > Hi, > > this patch fixes . > > Honza > This needs a test; here one I used to check it. Otherwise it works well, ACK if the test is added. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: Add-a-test-for-interactive_prompt-on-DNS-record-with.patch Type: text/x-patch Size: 1981 bytes Desc: not available URL: From pviktori at redhat.com Thu Feb 14 10:03:13 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 11:03:13 +0100 Subject: [Freeipa-devel] [PATCHES] 0107-0114 Fix Confusing ipa tool online help organization In-Reply-To: <5112D287.7010905@redhat.com> References: <50C9F27E.50808@redhat.com> <50CA76D6.8090404@redhat.com> <50CB0DCF.7050102@redhat.com> <510AB97F.2080608@redhat.com> <510BF0F7.8020102@redhat.com> <510BF629.4040700@redhat.com> <510FA8D3.4000502@redhat.com> <5112D287.7010905@redhat.com> Message-ID: <511CB661.7010300@redhat.com> On 02/06/2013 11:00 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 02/01/2013 06:06 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 01/31/2013 07:35 PM, Rob Crittenden wrote: >>>>> Petr Viktorin wrote: >>>>>> On 12/14/2012 01:46 AM, Dmitri Pal wrote: >>>>>>> On 12/13/2012 10:21 AM, Petr Viktorin wrote: >>>>>>>> https://fedorahosted.org/freeipa/ticket/3060 >>>>>>>> >>>>>>>> Here is a collection of smallish fixes to `ipa help` and `ipa >>>>>>>> --help`. >>>>>>>> This should address most of Nikolai's proposal. >>>>>>>> Additionally, it's now possible to run `ipa --help` >>>>>>>> without >>>>>>>> a Kerberos ticket. And there are some new tests. >>>>>>>> >>>>>>>> I've not included the "Often used commands" in `ipa help`; I think >>>>>>>> that is material for a manual/tutorial, not a help command. >>>>>>>> Selecting >>>>>>>> a topic from `ipa topics` and then choosing a command from `ipa >>>>>>>> help >>>>>>>> ` is a better way to use the help than the verbose `ipa help >>>>>>>> commands` or proposed incomplete "Often used commands". >>>>>>> >>>>>>> Since the ticket has a bit of discussion and you indicate that you >>>>>>> did >>>>>>> not to address everything can you please extract what have been >>>>>>> addressed and put it into a design page. >>>>>>> I know it is not RFE but it would help to validate the changes by >>>>>>> testers. >>>>>>> Please put the wiki link into the ticket. >>>>>>> >>>>>> >>>>>> http://freeipa.org/page/V3/Help >>>>>> >>>>>> >>>>> >>>>> What is the purpose of the no-option outfile? Do you anticipate at >>>>> some >>>>> point opening this up as a real option or making it easier to log >>>>> while >>>>> using the api directly? >>>> >>>> On incorrect invocation (`ipa`, `ipa TOPIC`), the help command is >>>> called >>>> internally with outfile=sys.stderr. >>> >>> That's true, but this is a lot of code to abstract writing to >>> sys.stderr. I'm just trying to understand the reasoning. Do you imagine >>> that some time in the future we'd want to control where the output goes? >> >> I don't think that would be useful, I can't imagine why someone would >> want to log help texts, or use them to script something. >> Do you have another idea of how this should be done? >> >>>>> The help for help is a little confusing: >>>>> >>>>> ----- >>>>> Purpose: Display help for a command or topic. >>>>> Usage: ipa [global-options] help [TOPIC] [options] >>>>> >>>>> Positional arguments: >>>>> TOPIC The topic or command name. >>>>> >>>>> Options: >>>>> -h, --help show this help message and exit >>>>> ----- >>>>> >>>>> Should [TOPIC] be [TOPIC | COMMAND] or something else? >>>> >>>> I'm trying to discourage `ipa help COMMAND` in favor of `ipa COMMAND >>>> --help`, that way you get the proper help for ambiguous words like >>>> "ping" (https://fedorahosted.org/freeipa/ticket/3247) >>>> >>>> I also wanted to keep the help simple and not explain this unimportant >>>> detail here. >>>> >>>> If you have better wording I can of course change it. >>> >>> Your reasoning is sound. I"m ok with gently pushing users in that >>> direction but leaving undocumented the old behavior. Should we create a >>> ticket to eventually return an error in that case? >> >> Users will expect `ipa help COMMAND` to get them command help, it's >> pretty standard in tools with subcommands. If it was a part of the API >> I'd be all for enforcing proper usage, but I think a help command >> deserves some slack -- it's not that big a deal if you get topic help >> for ping instead of command help... >> Hm. Now I see that I should add a notice to the topic help text, to lead >> users to the right path. Patch attached. >> >> We will want to remove `ipa help COMMAND` from the docs, and note that >> "ipa help" favors topics. >> We can turn https://fedorahosted.org/freeipa/ticket/3247 into a doc >> ticket. >> >> >>>>> On my fresh F-18 install one of the new unit tests fails: >>>>> >>>>> ====================================================================== >>>>> FAIL: Test that `help user-add` & `user-add -h` are equivalent and >>>>> contain doc >>>>> ---------------------------------------------------------------------- >>>>> Traceback (most recent call last): >>>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>>>> runTest >>>>> self.test(*self.arg) >>>>> File "/home/rcrit/redhat/freeipa/tests/test_cmdline/test_help.py", >>>>> line 111, in test_command_help >>>>> assert h_ctx.stdout == help_ctx.stdout >>>>> AssertionError >>>> >>>> This is weird, it works for me. Could you send me the two outputs so I >>>> can get some idea what's wrong? >>>> >> >> Can you check you didn't leave out patch 111 (Add command summary?)? >> >> > > Yeah, I missed 110 and 111. Tests are passing now. > > I still don't understand the purpose of outfile. What do we gain by > making this configurable and who or what would ever change it? > The mechanism in the patch changes it. For `ipa help`, the output goes to stdout. For `ipa` (invalid invocation), there's an error and the help is written to stderr. So the place where the output ends up needs to be configurable. -- Petr? From pviktori at redhat.com Thu Feb 14 11:10:28 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 12:10:28 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <5118ED16.6070301@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> <5118ED16.6070301@redhat.com> Message-ID: <511CC624.7000300@redhat.com> On 02/11/2013 02:07 PM, Petr Viktorin wrote: > On 02/01/2013 03:38 PM, Petr Viktorin wrote: >> On 02/01/2013 10:24 AM, Jan Cholasta wrote: >>> On 1.2.2013 09:47, Petr Viktorin wrote: >>>> On 01/31/2013 07:01 PM, Jan Cholasta wrote: >>>>> On 31.1.2013 11:00, Petr Viktorin wrote: >>>>>> On 01/30/2013 10:53 AM, Petr Viktorin wrote: >>>>>>> On 01/29/2013 04:39 PM, Petr Viktorin wrote: >>>>>>>> On 01/28/2013 04:09 PM, Petr Viktorin wrote: >>>>>>>>> On 01/28/2013 09:34 AM, Jan Cholasta wrote: >>>>>>>>>> On 25.1.2013 14:54, Petr Viktorin wrote: >>>>>>>>>>> On 01/24/2013 03:06 PM, Petr Viktorin wrote: >>>>>>>>>>>> On 01/24/2013 10:43 AM, Petr Viktorin wrote: >>>>>>>>>>>>> On 01/22/2013 04:04 PM, Petr Viktorin wrote: >>>>>>>>>>>>>> On 01/21/2013 06:38 PM, Petr Viktorin wrote: >>>>>>>>>>>>>>> On 01/17/2013 06:27 PM, Petr Viktorin wrote: >>>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>>> This is the first batch of changes aimed to consolidate our >>>>>>>>>>>>>>>> LDAP >>>>>>>>>>>>>>>> code. >>>>>>>>>>>>>>>> Each should be a self-contained change that doesn't break >>>>>>>>>>>>>>>> anything. >>>>>>>>>>>>>>>> >> [...] >>>>>>>>>>> Since this patchset is becoming unwieldy, I've put it in a >>>>>>>>>>> public >>>>>>>>>>> repo >>>>>>>>>>> that I'll keep updated. The following command will fetch it into >>>>>>>>>>> your >>>>>>>>>>> "pviktori-ldap-refactor" branch: >>>>>>>>>>> >>>>>>>>>>> git fetch git://github.com/encukou/freeipa >>>>>>>>>>> ldap-refactor:pviktori-ldap-refactor >>>>>>>>>>> >>>>>>>>>>> >> [...] >>>>> >>>>> I would prefer if you used the semantics of .get() for >>>>> .get_single() as >>>>> well (i.e. when no default value is provided, None is assumed) in >>>>> patch >>>>> 152. Or is there a reason not to? >>>> >>>> I think you should always have to write extra code to supress >>>> exceptions >>>> (?Errors should never pass silently?). In Python, the easiest/shortest >>>> getter you can write will typically raise an exception when the >>>> value is >>>> missing (e.g. `d[k]` for dicts, `getattr(a, b)` for objects). >>>> >>> >>> That is true, but I think consistency is more important here - the name >>> suggests the method behaves the same way .get() does. If you insist on >>> keeping the current behavior, would you at least consider renaming the >>> method (perhaps to just "single" or "single_value")? >>> >>> (This is just a nitpick, so don't worry too much about it.) >> >> Alright, I renamed get_single to single_value(). >> >> I also rebased to current master. > > Updating patch 115 to take recent changes in replica-manage into account. > Patches 161 & 167 needed a rebase. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0161-03-replace-getEntry-with-get_entry-or-get_entries-if-sc.patch Type: text/x-patch Size: 20132 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0167-02-Remove-IPAdmin.sasl_interactive_bind_s.patch Type: text/x-patch Size: 2545 bytes Desc: not available URL: From tbabej at redhat.com Thu Feb 14 11:35:42 2013 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 14 Feb 2013 12:35:42 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <20130212170013.GB476@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> <20130212170013.GB476@redhat.com> Message-ID: <511CCC0E.5060409@redhat.com> On 02/12/2013 06:00 PM, Alexander Bokovoy wrote: > On Fri, 08 Feb 2013, Tomas Babej wrote: >> On 02/08/2013 03:25 PM, Alexander Bokovoy wrote: >>> On Mon, 04 Feb 2013, Tomas Babej wrote: >>>> Hi, >>>> >>>> When adding/modifying an ID range for a trusted domain, the newly >>>> added option --dom-name can be used. This looks up SID of the >>>> trusted domain in LDAP and therefore the user is not required >>>> to write it down in CLI. If the lookup fails, error message >>>> asking the user to specify the SID manually is shown. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3133 >>>> >>>> Tomas >>> >>>>> From 72f8802953edaaf5b9f7c34a38601fbccd681c8e Mon Sep 17 00:00:00 >>>>> 2001 >>>> From: Tomas Babej >>>> Date: Mon, 4 Feb 2013 08:33:53 -0500 >>>> Subject: [PATCH] Add option to specify SID using domain name to >>>> idrange-add/mod >>>> >>>> When adding/modifying an ID range for a trusted domain, the newly >>>> added option --dom-name can be used. This looks up SID of the >>>> trusted domain in LDAP and therefore the user is not required >>>> to write it down in CLI. If the lookup fails, error message >>>> asking the user to specify the SID manually is shown. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3133 >>>> --- >>>> ipalib/plugins/idrange.py | 78 >>>> +++++++++++++++++++++++++++++++++++++++++------ >>>> ipaserver/dcerpc.py | 10 ++++++ >>>> 2 files changed, 78 insertions(+), 10 deletions(-) >>>> >>>> diff --git a/ipalib/plugins/idrange.py b/ipalib/plugins/idrange.py >>>> index >>>> 84e1057ac6b59b8ad99882a54e3288897338c978..77a75e4cabc18ca873be7cadcf870427d5b36ea0 >>>> 100644 >>>> --- a/ipalib/plugins/idrange.py >>>> +++ b/ipalib/plugins/idrange.py >>>> @@ -197,6 +197,11 @@ class idrange(LDAPObject): >>>> cli_name='dom_sid', >>>> label=_('Domain SID of the trusted domain'), >>>> ), >>>> + Str('ipanttrusteddomainname?', >>>> + cli_name='dom_name', >>>> + flags=('no_search', 'virtual_attribute'), >>>> + label=_('Name of the trusted domain'), >>>> + ), >>> New options is added but API.txt wasn't changed. As result, 'make rpms' >>> does not work. >>> >>> Could you please fix the patch and re-send it? >>> >> Sorry about that. >> >> Updated patch attached. > I have one small question regarding use of dom_sid/dom_name. > > If both dom_sid and dom_name were specified, failing to resolve dom_name > would force command to raise exception. > > I'm not sure this is right behavior. Probably we should detect that both > dom_sid and dom_name were specified and bail out earlier so that only > one of them is accepted. That would be clearer to users, wouldn't it Sure, I definitely agree on that point. I added the check to idrange-add and idrange-mod. Also, the patch needed a rebase to apply on the current master. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0030-3-Add-option-to-specify-SID-using-domain-name-to-idran.patch Type: text/x-patch Size: 11141 bytes Desc: not available URL: From pvoborni at redhat.com Thu Feb 14 11:42:57 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 14 Feb 2013 12:42:57 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <5113B55B.7030605@redhat.com> References: <51119947.9050707@redhat.com> <511390EB.40701@redhat.com> <5113B55B.7030605@redhat.com> Message-ID: <511CCDC1.5030908@redhat.com> On 02/07/2013 03:08 PM, Rob Crittenden wrote: > Petr Vobornik wrote: >> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>> This adds a cert-find command for the dogtag backend. >>> >>> Searches can be done by serial number, by subject, revocation reason, >>> issue date, notbefore, notafter and revocation dates. >>> >>> I added some basic tests for this. I made it a separate test file >>> because the cert plugin tests do not use the declarative format and rely >>> on the selfsign backend by default. >>> >>> rob >>> >> >> Should I create Web UI in scope of this ticket or a new one? >> >> I was also thinking if it's time to implement #191 'Web UI: specify >> fields to search on' [1]. Maybe in Pilsner. >> >> [1] https://fedorahosted.org/freeipa/ticket/191 > > I'm going to open a UI ticket once the API is finalized. I didn't want > to give you a moving target to work against. > > rob > I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. -- Petr Vobornik From pvoborni at redhat.com Thu Feb 14 12:30:46 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 14 Feb 2013 13:30:46 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <511ABF1F.70106@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> Message-ID: <511CD8F6.9040509@redhat.com> On 02/12/2013 11:15 PM, Endi Sukma Dewata wrote: > On 2/12/2013 10:56 AM, Petr Vobornik wrote: >> We were discussing to NACK this approach. >> >> The implementation should be improved because of the mutually exclusive >> nature of NONE option with [MS-PAC, PAD] options. >> >> I think we should add spec definition (to Web UI only, or into server >> plugin as well) of mutex options. Something like: >> >> mutex_groups: [[['NONE'],['MS-PAC', 'PAD']], [/*another array of >> groups*/]]; >> >> basically an array of group arrays where group array would contain >> arrays of mutually exclusive option. Is it over-complicated? Would one >> array or a pair of groups be enough? >> >> That way we can disable and uncheck(should not happen, but to keep data >> consistency) checkboxes of options from other groups when user selects a >> value of some group. > > If they are mutually exclusive, they probably should be separated using > radio buttons like this: > > PAC: ( ) None > (o) Type: > [x] MS-PAC > [ ] PAD You missed one option: nothing selected. It can be solved by adding '( ) Inherited' radio. Anyway, this design seems more user friendly for more general audience than mine so I will implement it. The only problem with it is that one have to come with new label for each group and empty value - can't be inferred from metadata. > > It might be better to use a composite widget of radio buttons and > checkboxes so we can reuse the code. Probably the definition will look > something like this: > > { > name: 'ipakrbauthzdata', > type: 'radio', Not sure if it should be radio, more like something new. > label: ..., > options: [ > { > label: ..., > value: 'NONE' > }, > { > label: ..., > type: 'checkboxes', Do you expect to be there something different than checkboxes, or do you want it to do it this way for possible future customization. > options: [ > { > label: ..., > value: 'MS-PAC' > }, > { > label: ..., > value: 'PAD' > } > ] > } > ] > } > > The composite widget will handle setting the appropriate widgets when > the value of the field on load. It will handle enabling/disabling the > checkboxes when the radio button is selected. It will also compute the > final value of the field from selected radio button/checkboxes on save. > >> Or just dim (no disable) and uncheck. That way there would still be >> visual distinction and one don't have to uncheck all the options from >> one group when he wants to select options of mutex group. > > That is also possible, but it changes the normal behavior of checkboxes, > so probably users would have to play around with it to understand the > grouping. They could also get confused, e.g. is dimmed checkbox disabled? > >> As a separate issue, we might want to fetch the options from metadata, >> when we want to show the values and don't care about creating different >> labels. > > If we use radio buttons like above, new labels are necessary to describe > the different groups of checkboxes. For all radio buttons & checkboxes > in general, if we don't want to use labels we probably could simplify > the definition such that we can specify the string values directly > (without nested object): > > { > name: 'ipakrbauthzdata', > type: 'radio', > options: [ > 'NONE', > { > type: 'checkboxes', > options: ['MS-PAC', 'PAD'] > } > ] > } > +1 for classic checkboxes or radios. Problem with this example is that, it is missing label for the checkbox group which can't be inferred from anything. -- Petr Vobornik From pspacek at redhat.com Thu Feb 14 13:26:35 2013 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 14 Feb 2013 14:26:35 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C521C.1070807@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> Message-ID: <511CE60B.5060106@redhat.com> On 14.2.2013 03:55, Dmitri Pal wrote: > On 02/13/2013 09:48 PM, Nathan Kinder wrote: >> On 02/13/2013 06:18 PM, Dmitri Pal wrote: >>> On 02/13/2013 02:08 PM, Simo Sorce wrote: >>>> On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: >>>>> Simo Sorce wrote: >>>>>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>>>>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>>>>>> >>>>>>>> If we can solve the looping and potential deadlocking concerns I >>>>>>>> think >>>>>>>> we can avoid the json reply and let the framework do the actual >>>>>>>> final >>>>>>>> ldap add. >>>>>>> Could you elaborate on your looping and deadlock concerns? I >>>>>>> don't see >>>>>>> where they would arise if what we're watching is entirely >>>>>>> independent of >>>>>>> our LDAP tree. >>>>>> I do not understand what you are 'watching' ? >>>>>> >>>>>> Simo. >>>>>> >>>>> I think he means have a persistent search to watch for new entries and >>>>> then act upon them. >>>> I think a 2 step approach is misguided, so I've written it off from the >>>> start. >>>> >>>> Simo. >>>> >>> This all thread smells like rewriting winsync by using internal plugin >>> and IPA framework. >>> Is there any chance we can use existing winsync solution to do what we >>> need but sync from staging instance rather than AD? >>> Then the flow will be: >>> HR system -> staging DS instance -> dirsync -> IPA >>> Couple assumptions: >>> a) We are satisfied with the security of the existing winsync solution >>> and authentication used there. I do not see why it should be different >>> here as it is a very similar use case. >>> b) I expect we can sync from 389 to 389 with minor config changes and >>> effectively no code changes >> This is not the case. Winsync uses the AD Dirsync control, which 389 >> does not support. The Winsync code also expects the AD schema on the >> entries coming in from the Dirsync response. Even if 389 supported >> the Dirsync control, there would be a fair amount of code changes to >> deal with whatever schema we would need to expect. >>> c) The users created via winsync now are created in a proper way so IPA >>> accepts then as IPA users >>> >>> This solves the problem of add and delete of the users. >>> I know that winsync already supports group add/delete but we never >>> qualified it in IPA. So may be it is time, at least for this use case. >>> I would really think that this might be a lower cost solution than >>> writing yet another custom plugin. >>> Let us try to reuse what we already have. >> I think a custom plug-in would be easier (though I do have concerns >> about the performance impact of the JSON approach). >> > I really think that we rather have an external solution and not the one > hot wired into the IPA internals via the proposed plugin. > Something like an external proxy or gateway rather than the approach > Simo proposed even if it would be more work. This work would be reusable. > We already have several use cases for the LDAP proxy. This can be one > more. A bit more and there will be enough need to build one anyways. > The main value is that it is optional and external and acts as an > ecosystem level solution that can be developed and tested on a separate > schedule rather than being internal to IPA. In my Fedora 17 I found package python-ldaptor. It seems to offer nice support for writing own event-based LDAP servers. For simple LDAP proxy it could be enough. $ yum install python-ldaptor $ python import ldaptor.protocols.ldap.ldapserver help(ldaptor.protocols.ldap.ldapserver) -- Petr^2 Spacek From simo at redhat.com Thu Feb 14 14:01:11 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 09:01:11 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C521C.1070807@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> Message-ID: <1360850471.12328.172.camel@willson.li.ssimo.org> On Wed, 2013-02-13 at 21:55 -0500, Dmitri Pal wrote: > On 02/13/2013 09:48 PM, Nathan Kinder wrote: > > On 02/13/2013 06:18 PM, Dmitri Pal wrote: > >> On 02/13/2013 02:08 PM, Simo Sorce wrote: > >>> On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: > >>>> Simo Sorce wrote: > >>>>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: > >>>>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: > >>>>>> > >>>>>>> If we can solve the looping and potential deadlocking concerns I > >>>>>>> think > >>>>>>> we can avoid the json reply and let the framework do the actual > >>>>>>> final > >>>>>>> ldap add. > >>>>>> Could you elaborate on your looping and deadlock concerns? I > >>>>>> don't see > >>>>>> where they would arise if what we're watching is entirely > >>>>>> independent of > >>>>>> our LDAP tree. > >>>>> I do not understand what you are 'watching' ? > >>>>> > >>>>> Simo. > >>>>> > >>>> I think he means have a persistent search to watch for new entries and > >>>> then act upon them. > >>> I think a 2 step approach is misguided, so I've written it off from the > >>> start. > >>> > >>> Simo. > >>> > >> This all thread smells like rewriting winsync by using internal plugin > >> and IPA framework. > >> Is there any chance we can use existing winsync solution to do what we > >> need but sync from staging instance rather than AD? > >> Then the flow will be: > >> HR system -> staging DS instance -> dirsync -> IPA > >> Couple assumptions: > >> a) We are satisfied with the security of the existing winsync solution > >> and authentication used there. I do not see why it should be different > >> here as it is a very similar use case. > >> b) I expect we can sync from 389 to 389 with minor config changes and > >> effectively no code changes > > This is not the case. Winsync uses the AD Dirsync control, which 389 > > does not support. The Winsync code also expects the AD schema on the > > entries coming in from the Dirsync response. Even if 389 supported > > the Dirsync control, there would be a fair amount of code changes to > > deal with whatever schema we would need to expect. > >> c) The users created via winsync now are created in a proper way so IPA > >> accepts then as IPA users > >> > >> This solves the problem of add and delete of the users. > >> I know that winsync already supports group add/delete but we never > >> qualified it in IPA. So may be it is time, at least for this use case. > >> I would really think that this might be a lower cost solution than > >> writing yet another custom plugin. > >> Let us try to reuse what we already have. > > I think a custom plug-in would be easier (though I do have concerns > > about the performance impact of the JSON approach). > > > I really think that we rather have an external solution and not the one > hot wired into the IPA internals via the proposed plugin. > Something like an external proxy or gateway rather than the approach > Simo proposed even if it would be more work. This work would be reusable. > We already have several use cases for the LDAP proxy. This can be one > more. A bit more and there will be enough need to build one anyways. > The main value is that it is optional and external and acts as an > ecosystem level solution that can be developed and tested on a separate > schedule rather than being internal to IPA. Penrose. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Feb 14 14:05:32 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 09:05:32 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C7778.9070102@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> Message-ID: <1360850732.12328.174.camel@willson.li.ssimo.org> On Thu, 2013-02-14 at 00:34 -0500, John Dennis wrote: > On 02/14/2013 12:16 AM, Nathan Kinder wrote: > > On 02/13/2013 08:30 PM, John Dennis wrote: > >> On 02/13/2013 10:40 PM, Nathan Kinder wrote: > >>> With the DS plug-in approach that calls into the IPA framework with a > >>> 'mock add' to form the resulting entry at the pre-op stage, we could > >>> take care of the initial ADD operation of the user entry. We would > >>> still need to have a way to trigger the additional write operations that > >>> the IPA framework performs. The proposed DS plug-in could do an > >>> internal search operation after the add, then it could perform the > >>> additional write operations that are needed. This logic would need to > >>> be in the DS plug-in, or we would need another way to get the details > >>> from the IPA framework via JSON. The approach begins to get a bit > >>> messy. > >> > >> Messy is one word for it :-) Herein lies the problem. user-add is > >> actually a simple case where in this particular version of our code we > >> know the LDAP operations that occur and in what order. But there is > >> nothing in our coding guidelines that guarantees this, it just happens > >> to be true today. Pre and post callbacks are free to do as they > >> please. The logic in our ldap module can change (in fact it's > >> undergoing a rewrite as we speak). We've committed to supporting an > >> extension mechanism (plugins) that ties into the framework operations > >> and who knows what might occur in that code. At the moment it's not > >> used, but hopefully it will be. > >> > >> Then comes the question, where do we draw the line? Do we say only > >> user-add with no additional options is supported because we know what > >> occurs during it's processing and hence we feel safe emulating > >> user-add outside the framework? > >> > >> We all know this is a slippery slope, as soon as you support user-add > >> this way we'll be getting requests to support other commands or other > >> options. And after we make framework changes (which we do frequently) > >> are we going to test these out-of-band operations continue to emulate > >> the ever changing framework? > >> > >> It's a slippery slope that can expose us to problems we don't need. > >> > >> Jenny, do you want to test an entirely different mechanism or do you > >> want to limit it to our defined API? > >> > >> We have a defined API, I really think that's the only thing we should > >> support. Backdoor shortcuts that sidestep our framework should be off > >> the table IMHO, it's just inviting too many problems. > > > This is why I think that the most sane approach is to put something in > > front of 'ipa user-add'. This is not without it's own set of > > difficulties, but if we need a basic LDAP interface that can be used to > > create IPA users, it should end up calling into the same code that 'ipa > > user-add' uses. This would ensure that any extensions (plugins) for the > > framework are called. > > > I actually see this as very similar to the LDAP migration case. For > > migration, we basically take entries in LDIF format, extract the > > relevant data, then feed it into 'ipa user-add', right? That's more or > > less what we need to do here for HR systems that provision users, but > > the user addition needs to be pushed to IPA via an LDAP client instead > > of IPA pulling from an LDAP server. > > Yes, this is exactly what I'm saying and we concur. So as I proposed we can call ipa user-add from LDAP from a non-transactional pre-op plugin. We just need to be careful about when we allow that to avoid loops, but besides that problem it seem relatively easy and does not require crazy things like playgrounds or even full LDAP proxies. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Feb 14 14:06:40 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 09:06:40 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CE60B.5060106@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> <511CE60B.5060106@redhat.com> Message-ID: <1360850800.12328.175.camel@willson.li.ssimo.org> On Thu, 2013-02-14 at 14:26 +0100, Petr Spacek wrote: > In my Fedora 17 I found package python-ldaptor. It seems to offer nice support > for writing own event-based LDAP servers. For simple LDAP proxy it could be > enough. > > $ yum install python-ldaptor > $ python > import ldaptor.protocols.ldap.ldapserver > help(ldaptor.protocols.ldap.ldapserver) No. LDAP proxies are *not* simple. Ask Endi he's worked on a meta-directory for years. Simo. -- Simo Sorce * Red Hat, Inc * New York From pviktori at redhat.com Thu Feb 14 14:25:23 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 15:25:23 +0100 Subject: [Freeipa-devel] [PATCH] 0119 Switch client to JSON-RPC In-Reply-To: <50F53F2B.4020406@redhat.com> References: <50F53F2B.4020406@redhat.com> Message-ID: <511CF3D3.80107@redhat.com> On 01/15/2013 12:36 PM, Petr Viktorin wrote: > I meant to hold this patch a while longer to let it mature, but from > what Brian Smith asked on the user list it seems it could help him. > > Design: http://freeipa.org/page/V3/JSON-RPC > Ticket: https://fedorahosted.org/freeipa/ticket/3299 > > See the design page for what the patch does. > > > As much as I've tried to avoid them, the code includes some workarounds: > It extends xmlrpclib to also support JSON. This is rather intrusive, but > to not do that I'd need to write a parallel stack for JSON, without the > help of a standard library. > The registration of either jsonclient or xmlclient as "rpcclient" in the > API also needs a bit of magic, since the framework requires the class > name to match the attribute. > > > To prevent backwards compatibility problems, we need to ensure that all > official JSON clients send the API version, so this patch should be > applied after my patches 0104-0106. > Updating to current master. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0119-02-Switch-client-to-JSON-RPC.patch Type: text/x-patch Size: 34691 bytes Desc: not available URL: From rcritten at redhat.com Thu Feb 14 14:29:06 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 14 Feb 2013 09:29:06 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511C7778.9070102@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> Message-ID: <511CF4B2.1010904@redhat.com> John Dennis wrote: > On 02/14/2013 12:16 AM, Nathan Kinder wrote: >> On 02/13/2013 08:30 PM, John Dennis wrote: >>> On 02/13/2013 10:40 PM, Nathan Kinder wrote: >>>> With the DS plug-in approach that calls into the IPA framework with a >>>> 'mock add' to form the resulting entry at the pre-op stage, we could >>>> take care of the initial ADD operation of the user entry. We would >>>> still need to have a way to trigger the additional write operations >>>> that >>>> the IPA framework performs. The proposed DS plug-in could do an >>>> internal search operation after the add, then it could perform the >>>> additional write operations that are needed. This logic would need to >>>> be in the DS plug-in, or we would need another way to get the details >>>> from the IPA framework via JSON. The approach begins to get a bit >>>> messy. >>> >>> Messy is one word for it :-) Herein lies the problem. user-add is >>> actually a simple case where in this particular version of our code we >>> know the LDAP operations that occur and in what order. But there is >>> nothing in our coding guidelines that guarantees this, it just happens >>> to be true today. Pre and post callbacks are free to do as they >>> please. The logic in our ldap module can change (in fact it's >>> undergoing a rewrite as we speak). We've committed to supporting an >>> extension mechanism (plugins) that ties into the framework operations >>> and who knows what might occur in that code. At the moment it's not >>> used, but hopefully it will be. >>> >>> Then comes the question, where do we draw the line? Do we say only >>> user-add with no additional options is supported because we know what >>> occurs during it's processing and hence we feel safe emulating >>> user-add outside the framework? >>> >>> We all know this is a slippery slope, as soon as you support user-add >>> this way we'll be getting requests to support other commands or other >>> options. And after we make framework changes (which we do frequently) >>> are we going to test these out-of-band operations continue to emulate >>> the ever changing framework? >>> >>> It's a slippery slope that can expose us to problems we don't need. >>> >>> Jenny, do you want to test an entirely different mechanism or do you >>> want to limit it to our defined API? >>> >>> We have a defined API, I really think that's the only thing we should >>> support. Backdoor shortcuts that sidestep our framework should be off >>> the table IMHO, it's just inviting too many problems. > >> This is why I think that the most sane approach is to put something in >> front of 'ipa user-add'. This is not without it's own set of >> difficulties, but if we need a basic LDAP interface that can be used to >> create IPA users, it should end up calling into the same code that 'ipa >> user-add' uses. This would ensure that any extensions (plugins) for the >> framework are called. > >> I actually see this as very similar to the LDAP migration case. For >> migration, we basically take entries in LDIF format, extract the >> relevant data, then feed it into 'ipa user-add', right? That's more or >> less what we need to do here for HR systems that provision users, but >> the user addition needs to be pushed to IPA via an LDAP client instead >> of IPA pulling from an LDAP server. > > Yes, this is exactly what I'm saying and we concur. In reality we don't actually pass migrated users to user-add for performance reasons. rob From jdennis at redhat.com Thu Feb 14 14:33:52 2013 From: jdennis at redhat.com (John Dennis) Date: Thu, 14 Feb 2013 09:33:52 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360850732.12328.174.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <1360850732.12328.174.camel@willson.li.ssimo.org> Message-ID: <511CF5D0.9080803@redhat.com> On 02/14/2013 09:05 AM, Simo Sorce wrote: > So as I proposed we can call ipa user-add from LDAP from a > non-transactional pre-op plugin. We just need to be careful about when > we allow that to avoid loops, but besides that problem it seem > relatively easy and does not require crazy things like playgrounds or > even full LDAP proxies. I think I need a clarification because perhaps I didn't fully understand your proposal. Is the idea with a non-transactional pre-op plugin it invokes user-add and then the pre-op returns *without* having modified ldap? In effect it acts as a trigger? That still implies there has to be a separate tree where the foreign entity writes (and the pre-op plugin watches) because otherwise how could the pre-op plugin distinguish between framework writes and foreign writes? If there is a separate tree where is the looping issue? You still haven't explained this. Also, under the scenario that a foreign entity writes something into LDAP (somewhere) and it triggers us to call user-add via some mechanism then what happens when errors occur? The foreign entity will not know we rejected the operation nor why. Also, don't forget they want to delete users, remove group membership, add group membership, add groups, remove groups etc. Some of these operations are dependent upon logic in our framework. I don't see how some of these operations can be reliably managed by a foreign entity simultaneously performing LDAP operations. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu Feb 14 14:34:57 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 14 Feb 2013 09:34:57 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511CCDC1.5030908@redhat.com> References: <51119947.9050707@redhat.com> <511390EB.40701@redhat.com> <5113B55B.7030605@redhat.com> <511CCDC1.5030908@redhat.com> Message-ID: <511CF611.1020805@redhat.com> Petr Vobornik wrote: > On 02/07/2013 03:08 PM, Rob Crittenden wrote: >> Petr Vobornik wrote: >>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>> This adds a cert-find command for the dogtag backend. >>>> >>>> Searches can be done by serial number, by subject, revocation reason, >>>> issue date, notbefore, notafter and revocation dates. >>>> >>>> I added some basic tests for this. I made it a separate test file >>>> because the cert plugin tests do not use the declarative format and >>>> rely >>>> on the selfsign backend by default. >>>> >>>> rob >>>> >>> >>> Should I create Web UI in scope of this ticket or a new one? >>> >>> I was also thinking if it's time to implement #191 'Web UI: specify >>> fields to search on' [1]. Maybe in Pilsner. >>> >>> [1] https://fedorahosted.org/freeipa/ticket/191 >> >> I'm going to open a UI ticket once the API is finalized. I didn't want >> to give you a moving target to work against. >> >> rob >> > > I see that the search requires to specify options for attributes to > search on. There is no general CRITERIA positional argument as in other > find commands or am I mistaken? > > Is it possible to add the CRITERIA argument? Is the no 'OR' search an > obstacle for it? > > If so we would really need to push the ticket #191 because UI doesn't > support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob From jdennis at redhat.com Thu Feb 14 14:35:18 2013 From: jdennis at redhat.com (John Dennis) Date: Thu, 14 Feb 2013 09:35:18 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CF4B2.1010904@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <511CF4B2.1010904@redhat.com> Message-ID: <511CF626.2030206@redhat.com> On 02/14/2013 09:29 AM, Rob Crittenden wrote: > In reality we don't actually pass migrated users to user-add for > performance reasons. What parts of user-add are we bypassing? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu Feb 14 14:40:33 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 14 Feb 2013 09:40:33 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CF626.2030206@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <511CF4B2.1010904@redhat.com> <511CF626.2030206@redhat.com> Message-ID: <511CF761.6000409@redhat.com> John Dennis wrote: > On 02/14/2013 09:29 AM, Rob Crittenden wrote: >> In reality we don't actually pass migrated users to user-add for >> performance reasons. > > What parts of user-add are we bypassing? > > Everything. It doesn't call into it at all. rob From rcritten at redhat.com Thu Feb 14 14:47:18 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 14 Feb 2013 09:47:18 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CF5D0.9080803@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <1360850732.12328.174.camel@willson.li.ssimo.org> <511CF5D0.9080803@redhat.com> Message-ID: <511CF8F6.5070309@redhat.com> John Dennis wrote: > On 02/14/2013 09:05 AM, Simo Sorce wrote: >> So as I proposed we can call ipa user-add from LDAP from a >> non-transactional pre-op plugin. We just need to be careful about when >> we allow that to avoid loops, but besides that problem it seem >> relatively easy and does not require crazy things like playgrounds or >> even full LDAP proxies. > > I think I need a clarification because perhaps I didn't fully understand > your proposal. > > Is the idea with a non-transactional pre-op plugin it invokes user-add > and then the pre-op returns *without* having modified ldap? In effect it > acts as a trigger? > > That still implies there has to be a separate tree where the foreign > entity writes (and the pre-op plugin watches) because otherwise how > could the pre-op plugin distinguish between framework writes and foreign > writes? The proposal is that we write a 389-ds plugin that intercepts LDAP ADD operations. I don't think Simo proposed where in the tree this would be written, or whether we would be able to distinguish between a normal ADD and this special case (loop 1). For the purposes of modification you probably want it to write into the current user container. It will need to be able to push MOD requests for names, addresses, etc. Most LDAP apps take a single basedn against which to do things. We can't expect to be able to provide one set for adds and one set for mods. And of course this leads into deletes. We want users gone when they leave the company. Feature creep already. > If there is a separate tree where is the looping issue? You still > haven't explained this. The looping is in telling whether the add is from the external HR source or a "normal" ADD from IPA. If we get it wrong, ever, then it will ADD, then call ipa user-add, ADD, call ipa user-add, ADD, ... > Also, under the scenario that a foreign entity writes something into > LDAP (somewhere) and it triggers us to call user-add via some mechanism > then what happens when errors occur? The foreign entity will not know we > rejected the operation nor why. I don't know about that. We'd get the exceptions back from JSON so would have to translate them back into LDAP errors. For the cases there isn't a 1-1 match we'd have to do our best and add good logging. > Also, don't forget they want to delete users, remove group membership, > add group membership, add groups, remove groups etc. Some of these > operations are dependent upon logic in our framework. I don't see how > some of these operations can be reliably managed by a foreign entity > simultaneously performing LDAP operations. Yup, we're on the same page here. rob From simo at redhat.com Thu Feb 14 15:11:57 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 10:11:57 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CF5D0.9080803@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <1360850732.12328.174.camel@willson.li.ssimo.org> <511CF5D0.9080803@redhat.com> Message-ID: <1360854717.12328.181.camel@willson.li.ssimo.org> On Thu, 2013-02-14 at 09:33 -0500, John Dennis wrote: > On 02/14/2013 09:05 AM, Simo Sorce wrote: > > So as I proposed we can call ipa user-add from LDAP from a > > non-transactional pre-op plugin. We just need to be careful about when > > we allow that to avoid loops, but besides that problem it seem > > relatively easy and does not require crazy things like playgrounds or > > even full LDAP proxies. > > I think I need a clarification because perhaps I didn't fully understand > your proposal. > > Is the idea with a non-transactional pre-op plugin it invokes user-add > and then the pre-op returns *without* having modified ldap? In effect it > acts as a trigger? Yes. > That still implies there has to be a separate tree where the foreign > entity writes (and the pre-op plugin watches) No. > because otherwise how > could the pre-op plugin distinguish between framework writes and foreign > writes? The idea I flaunted around is that the pluggin triggers only if a special objectclass is used. The framewrok would never try to create a user with this special objectclass so loops are avoided. > If there is a separate tree where is the looping issue? You still > haven't explained this. No separate tree, the plugin intercepts the Add operations, and calls the framework. The framework does the real user creation and then returns. At that point the pre-op plugin returns immediately with the error code from the framework discarding the actual add operation. > Also, under the scenario that a foreign entity writes something into > LDAP (somewhere) and it triggers us to call user-add via some mechanism > then what happens when errors occur? We return back an error. > The foreign entity will not know we rejected the operation nor why. This is the problem if you use a staged approach, and one of the reasons I do not consider it as viable, and therefore reject any proposal that imply staged writes and separate operations. > Also, don't forget they want to delete users, remove group membership, > add group membership, add groups, remove groups etc. Some of these > operations are dependent upon logic in our framework. Which ones. Only user creation is really special. Deleting a user should not depend on the framework, nor adding group members. Adding a user is a special case, and should be limited to that. > I don't see how > some of these operations can be reliably managed by a foreign entity > simultaneously performing LDAP operations. I do not see what your concerns are. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Feb 14 15:19:08 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 10:19:08 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CF8F6.5070309@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <1360850732.12328.174.camel@willson.li.ssimo.org> <511CF5D0.9080803@redhat.com> <511CF8F6.5070309@redhat.com> Message-ID: <1360855148.12328.189.camel@willson.li.ssimo.org> On Thu, 2013-02-14 at 09:47 -0500, Rob Crittenden wrote: > John Dennis wrote: > > On 02/14/2013 09:05 AM, Simo Sorce wrote: > >> So as I proposed we can call ipa user-add from LDAP from a > >> non-transactional pre-op plugin. We just need to be careful about when > >> we allow that to avoid loops, but besides that problem it seem > >> relatively easy and does not require crazy things like playgrounds or > >> even full LDAP proxies. > > > > I think I need a clarification because perhaps I didn't fully understand > > your proposal. > > > > Is the idea with a non-transactional pre-op plugin it invokes user-add > > and then the pre-op returns *without* having modified ldap? In effect it > > acts as a trigger? > > > > That still implies there has to be a separate tree where the foreign > > entity writes (and the pre-op plugin watches) because otherwise how > > could the pre-op plugin distinguish between framework writes and foreign > > writes? > > The proposal is that we write a 389-ds plugin that intercepts LDAP ADD > operations. I don't think Simo proposed where in the tree this would be > written, or whether we would be able to distinguish between a normal ADD > and this special case (loop 1). I did, please see my other emails in the thread. > For the purposes of modification you probably want it to write into the > current user container. It will need to be able to push MOD requests for > names, addresses, etc. Most LDAP apps take a single basedn against which > to do things. We can't expect to be able to provide one set for adds and > one set for mods. We do not intercept mods, period. The only *real* issue is creating users, because the framework adds a lot of stuff to properly format users, but modifications from LDAP clients is fair game. We cannot consider the framework as the only gatekeeper beyond object creation, if this is not clear we need to discuss. > And of course this leads into deletes. We want users gone when they > leave the company. Feature creep already. Do we do anything special in the framework for user deletes now ? I am not aware of any special ops, and I would be against making a delete operation a special operation. > > If there is a separate tree where is the looping issue? You still > > haven't explained this. > > The looping is in telling whether the add is from the external HR source > or a "normal" ADD from IPA. If we get it wrong, ever, then it will ADD, > then call ipa user-add, ADD, call ipa user-add, ADD, ... Yes, this is the only concern, I made one proposal to avoid the issue, there are also other ways, for example the framework could use a special control to tell our server 'I am the framework, do not loop'. We just need to decide what's the best way to do it. For example we already need to avoid any interference with replication, but that is easy to do, as we know if an operation is coming via replication. The other is winsync, but I would argue that winsync maybe should actually use this plugin as it would make it more consistent with the framework. Simo. -- Simo Sorce * Red Hat, Inc * New York From rmeggins at redhat.com Thu Feb 14 15:22:14 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 14 Feb 2013 08:22:14 -0700 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511CA765.1010704@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> <511CA765.1010704@redhat.com> Message-ID: <511D0126.9090105@redhat.com> On 02/14/2013 01:59 AM, Petr Viktorin wrote: > On 02/13/2013 07:11 PM, Simo Sorce wrote: >> On Wed, 2013-02-13 at 10:57 -0700, Rich Megginson wrote: >> >>>> Rich, >>>> is there potential from deadlocking here due to the new transaction >>>> stuff ? Or can we single out this plugin to run before *any* >>> transaction >>>> is started ? >> >>> If you do this in a "regular" pre-op, not a "betxn" pre-op, then it >>> should be fine. >> >> Ok in this case we should be able to create a regular pre-op plugin to >> intercept the ldap add call and then use the following flow: >> client --(LDAP)--> 389DS --(HTTP/json)--> framework --(LDAP)--> add >> >> So no deadlocks will happen, the remaining issue is how to make sure we >> do not loop by mistake in the second add. >> >> One way could be to have loop detection so that if more then two (1. >> original, 2. framework) adds for the same DN come in we just return >> errors. Another way is to use a special objectclass as I proposed in the >> thread and make sure the framework explictly blacklists it so that it >> can never try to send an add with the special oc even by mistake or user >> misconfiguration. >> > > And a third way is a separate LDAP tree. > > I'm not familiar with what DS plugins can do. Can we craft one that > intercepts all read operations on "cn=HR tree,$SUFFIX" and does them > on "cn=users,cn=accounts,$SUFFIX" instead (using that tree's > permissions), and forwards all write operations on "cn=HR tree" to IPA > via JSON? Yes. > > > A fourth way is a proxy/gateway, essentially the same but with a > separate server. > From rcritten at redhat.com Thu Feb 14 15:28:41 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 14 Feb 2013 10:28:41 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360855148.12328.189.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BC2F7.5040707@redhat.com> <1360774646.12328.96.camel@willson.li.ssimo.org> <511BCA89.9070804@redhat.com> <511C5CB5.2050101@redhat.com> <511C6876.20706@redhat.com> <511C734A.2000203@redhat.com> <511C7778.9070102@redhat.com> <1360850732.12328.174.camel@willson.li.ssimo.org> <511CF5D0.9080803@redhat.com> <511CF8F6.5070309@redhat.com> <1360855148.12328.189.camel@willson.li.ssimo.org> Message-ID: <511D02A9.9090906@redhat.com> Simo Sorce wrote: > On Thu, 2013-02-14 at 09:47 -0500, Rob Crittenden wrote: >> John Dennis wrote: >>> On 02/14/2013 09:05 AM, Simo Sorce wrote: >>>> So as I proposed we can call ipa user-add from LDAP from a >>>> non-transactional pre-op plugin. We just need to be careful about when >>>> we allow that to avoid loops, but besides that problem it seem >>>> relatively easy and does not require crazy things like playgrounds or >>>> even full LDAP proxies. >>> >>> I think I need a clarification because perhaps I didn't fully understand >>> your proposal. >>> >>> Is the idea with a non-transactional pre-op plugin it invokes user-add >>> and then the pre-op returns *without* having modified ldap? In effect it >>> acts as a trigger? >>> >>> That still implies there has to be a separate tree where the foreign >>> entity writes (and the pre-op plugin watches) because otherwise how >>> could the pre-op plugin distinguish between framework writes and foreign >>> writes? >> >> The proposal is that we write a 389-ds plugin that intercepts LDAP ADD >> operations. I don't think Simo proposed where in the tree this would be >> written, or whether we would be able to distinguish between a normal ADD >> and this special case (loop 1). > > I did, please see my other emails in the thread. > >> For the purposes of modification you probably want it to write into the >> current user container. It will need to be able to push MOD requests for >> names, addresses, etc. Most LDAP apps take a single basedn against which >> to do things. We can't expect to be able to provide one set for adds and >> one set for mods. > > We do not intercept mods, period. > The only *real* issue is creating users, because the framework adds a > lot of stuff to properly format users, but modifications from LDAP > clients is fair game. We are in agreement here. My point is that if we try to write to a special part of the tree then move the entry then MODS will always fail. > We cannot consider the framework as the only gatekeeper beyond object > creation, if this is not clear we need to discuss. That was actually the point of the framework from the beginning. >> And of course this leads into deletes. We want users gone when they >> leave the company. Feature creep already. > > Do we do anything special in the framework for user deletes now ? > I am not aware of any special ops, and I would be against making a > delete operation a special operation. Any operation can have a pre/post operation. One RFE we have is to add triggers so that shell scripts (or whatever) can be executed at times. This is is the sort of thing that would be missed. > >> > If there is a separate tree where is the looping issue? You still >>> haven't explained this. >> >> The looping is in telling whether the add is from the external HR source >> or a "normal" ADD from IPA. If we get it wrong, ever, then it will ADD, >> then call ipa user-add, ADD, call ipa user-add, ADD, ... > > Yes, this is the only concern, I made one proposal to avoid the issue, > there are also other ways, for example the framework could use a special > control to tell our server 'I am the framework, do not loop'. We just > need to decide what's the best way to do it. Right. I don't know enough about how much flexibility these external systems have when creating LDAP objects so they may not be able to add additional objectclasses. A control may be the way to go. > For example we already need to avoid any interference with replication, > but that is easy to do, as we know if an operation is coming via > replication. The other is winsync, but I would argue that winsync maybe > should actually use this plugin as it would make it more consistent with > the framework. Yes, if we could solve this without too much performance issues that should work. rob From pviktori at redhat.com Thu Feb 14 15:29:35 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 16:29:35 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511D0126.9090105@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> <511CA765.1010704@redhat.com> <511D0126.9090105@redhat.com> Message-ID: <511D02DF.90302@redhat.com> On 02/14/2013 04:22 PM, Rich Megginson wrote: > On 02/14/2013 01:59 AM, Petr Viktorin wrote: >> On 02/13/2013 07:11 PM, Simo Sorce wrote: >>> On Wed, 2013-02-13 at 10:57 -0700, Rich Megginson wrote: >>> >>>>> Rich, >>>>> is there potential from deadlocking here due to the new transaction >>>>> stuff ? Or can we single out this plugin to run before *any* >>>> transaction >>>>> is started ? >>> >>>> If you do this in a "regular" pre-op, not a "betxn" pre-op, then it >>>> should be fine. >>> >>> Ok in this case we should be able to create a regular pre-op plugin to >>> intercept the ldap add call and then use the following flow: >>> client --(LDAP)--> 389DS --(HTTP/json)--> framework --(LDAP)--> add >>> >>> So no deadlocks will happen, the remaining issue is how to make sure we >>> do not loop by mistake in the second add. >>> >>> One way could be to have loop detection so that if more then two (1. >>> original, 2. framework) adds for the same DN come in we just return >>> errors. Another way is to use a special objectclass as I proposed in the >>> thread and make sure the framework explictly blacklists it so that it >>> can never try to send an add with the special oc even by mistake or user >>> misconfiguration. >>> >> >> And a third way is a separate LDAP tree. >> >> I'm not familiar with what DS plugins can do. Can we craft one that >> intercepts all read operations on "cn=HR tree,$SUFFIX" and does them >> on "cn=users,cn=accounts,$SUFFIX" instead (using that tree's >> permissions), and forwards all write operations on "cn=HR tree" to IPA >> via JSON? > Yes. Then I recommend doing this. It avoids problems with delegation (the "real" tree permissions are used) and looping (plugin and IPA write to separate trees). Other operations (deletes, mods) can be either similarly delegated to the "real" tree, or passed through IPA if we want to do that in the future. Problems with replication can be solved by just not replicating the separate tree. It also doesn't pollute core IPA with special cases, which is what worries me. -- Petr? From edewata at redhat.com Thu Feb 14 15:56:44 2013 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 14 Feb 2013 09:56:44 -0600 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <511CD8F6.9040509@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> <511CD8F6.9040509@redhat.com> Message-ID: <511D093C.5090602@redhat.com> On 2/14/2013 6:30 AM, Petr Vobornik wrote: >> If they are mutually exclusive, they probably should be separated using >> radio buttons like this: >> >> PAC: ( ) None >> (o) Type: >> [x] MS-PAC >> [ ] PAD > > You missed one option: nothing selected. It can be solved by adding '( > ) Inherited' radio. I wouldn't have guessed that :) I agree we should add the 'Inherited' option. > Anyway, this design seems more user friendly for more general audience > than mine so I will implement it. The only problem with it is that one > have to come with new label for each group and empty value - can't be > inferred from metadata. Is there any issue adding new labels at this point? Worst case we could hard code the label now and add a translation later. >> It might be better to use a composite widget of radio buttons and >> checkboxes so we can reuse the code. Probably the definition will look >> something like this: >> >> { >> name: 'ipakrbauthzdata', >> type: 'radio', > > Not sure if it should be radio, more like something new. Right, probably the current radio widget can't do this. So either we improve the radio widget or create something new. >> label: ..., >> options: [ >> { >> label: ..., >> value: 'NONE' >> }, >> { >> label: ..., >> type: 'checkboxes', > > Do you expect to be there something different than checkboxes, or do you > want it to do it this way for possible future customization. Ideally it should be generic enough to combine any widgets. This might be a common scenario somewhere else: Something: ( ) Option 1 ( ) Option 2 (o) Other: [something else ] -- Endi S. Dewata From simo at redhat.com Thu Feb 14 16:01:23 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 11:01:23 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511D02DF.90302@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> <511CA765.1010704@redhat.com> <511D0126.9090105@redhat.com> <511D02DF.90302@redhat.com> Message-ID: <1360857683.12328.207.camel@willson.li.ssimo.org> On Thu, 2013-02-14 at 16:29 +0100, Petr Viktorin wrote: > Then I recommend doing this. It avoids problems with delegation (the > "real" tree permissions are used) and looping (plugin and IPA write to > separate trees). Virtual objects are not free of issues, you are just trading some issues for others here, and I do not think you are properly evaluating the trade here. I am dead set against using staging areas, and I want to see proof they are a good idea, not handwaving 'fears' of using json in the server to forward operations directly to the framework. > Other operations (deletes, mods) can be either > similarly delegated to the "real" tree, And *this* is a slippery slope. you are trading delegating one single operation to the framework directly, with proper error propagation back to the client, with now implementing full virtualized operations for mod and delete, and bind ? and search and then ? You are now basically paving the way for a virtual directory in tree. Sorry, but no. > or passed through IPA if we want to do that in the future. > Problems with replication can be solved by just not replicating the > separate tree. More and more hacks, for this supposedly 'cleaner' or 'simpler' solution ... sorry if I don't bite. > It also doesn't pollute core IPA with special cases, which is what > worries me. What does this mean ? We *have* a special case, and we are discussing how to handle it. The situation here (I do not want to call it a 'problem') is that we decided to put business logic around user creation in the framework because we thought that would be easier to prototype and develop in python code rather than in a plugin. However this special handling *must* be limited. LDAP is our main storage, and we must allow LDAP operations against it. Special cases needs to be limited to things that are really harder done in plugins rather then python code. For example if we need triggers for some operations in LDAP, they *must* be done through a 389ds plugin. Otherwise LDAP quickly becomes a read-only interface and interoperability quickly goes out of the window. I always treated the framework as a *management interface* on top. We can do things that are 'convenient', and 'help' admins avoid mistakes, but we cannot move core functionality in the framework, it would be a grave mistake. User creation *is* a special case, but should remain one of very few special exceptions. This very thread and the need for the interface proposed in this thread is a clear example of why we need to be extremely careful not to be too liberal with what business logic we move in the framework. LDAP keeps us honest, so we need to limit what we do in the framework, otherwise we'll keep taking shortcuts and soon enough it goes out of control and we loose interoperability with anything that is not purpose-built to talk to our framework. This should be an unacceptable scenario because it is like putting ourselves in a getto. We trade greater interoperability and acceptance for small conveniences here and there. We must thread *very* carefully along this line. Simo. -- Simo Sorce * Red Hat, Inc * New York From pvoborni at redhat.com Thu Feb 14 16:02:40 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 14 Feb 2013 17:02:40 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511CF611.1020805@redhat.com> References: <51119947.9050707@redhat.com> <511390EB.40701@redhat.com> <5113B55B.7030605@redhat.com> <511CCDC1.5030908@redhat.com> <511CF611.1020805@redhat.com> Message-ID: <511D0AA0.3020209@redhat.com> On 02/14/2013 03:34 PM, Rob Crittenden wrote: > Petr Vobornik wrote: >> On 02/07/2013 03:08 PM, Rob Crittenden wrote: >>> Petr Vobornik wrote: >>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>> This adds a cert-find command for the dogtag backend. >>>>> >>>>> Searches can be done by serial number, by subject, revocation reason, >>>>> issue date, notbefore, notafter and revocation dates. >>>>> >>>>> I added some basic tests for this. I made it a separate test file >>>>> because the cert plugin tests do not use the declarative format and >>>>> rely >>>>> on the selfsign backend by default. >>>>> >>>>> rob >>>>> >>>> >>>> Should I create Web UI in scope of this ticket or a new one? >>>> >>>> I was also thinking if it's time to implement #191 'Web UI: specify >>>> fields to search on' [1]. Maybe in Pilsner. >>>> >>>> [1] https://fedorahosted.org/freeipa/ticket/191 >>> >>> I'm going to open a UI ticket once the API is finalized. I didn't want >>> to give you a moving target to work against. >>> >>> rob >>> >> >> I see that the search requires to specify options for attributes to >> search on. There is no general CRITERIA positional argument as in other >> find commands or am I mistaken? >> >> Is it possible to add the CRITERIA argument? Is the no 'OR' search an >> obstacle for it? >> >> If so we would really need to push the ticket #191 because UI doesn't >> support search by only specifying specific attributes yet. > > Your analysis is correct. > > It may be considered a hack but what if I treat subject as the CRITERIA > argument? > > rob Better that than nothing. Just a confirmation: when user does not set any option, it will return all certificates? Or it will return nothing? I see Web UI implementation this way: 1) implement simple search with the hack now 2) if there will be time before release (after the refactoring and other tickets) implement #191 (will require UXD input) to implement this the proper way. We can move #191 to triage to decide it. -- Petr Vobornik From rmeggins at redhat.com Thu Feb 14 16:26:52 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Thu, 14 Feb 2013 09:26:52 -0700 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360857683.12328.207.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> <511CA765.1010704@redhat.com> <511D0126.9090105@redhat.com> <511D02DF.90302@redhat.com> <1360857683.12328.207.camel@willson.li.ssimo.org> Message-ID: <511D104C.9000306@redhat.com> On 02/14/2013 09:01 AM, Simo Sorce wrote: > On Thu, 2013-02-14 at 16:29 +0100, Petr Viktorin wrote: >> Then I recommend doing this. It avoids problems with delegation (the >> "real" tree permissions are used) and looping (plugin and IPA write to >> separate trees). > Virtual objects are not free of issues, you are just trading some issues > for others here, and I do not think you are properly evaluating the > trade here. > > I am dead set against using staging areas, and I want to see proof they > are a good idea, not handwaving 'fears' of using json in the server to > forward operations directly to the framework. > >> Other operations (deletes, mods) can be either >> similarly delegated to the "real" tree, > And *this* is a slippery slope. you are trading delegating one single > operation to the framework directly, with proper error propagation back > to the client, with now implementing full virtualized operations for mod > and delete, and bind ? and search and then ? > > You are now basically paving the way for a virtual directory in tree. > > Sorry, but no. > >> or passed through IPA if we want to do that in the future. >> Problems with replication can be solved by just not replicating the >> separate tree. > More and more hacks, for this supposedly 'cleaner' or 'simpler' > solution ... sorry if I don't bite. > >> It also doesn't pollute core IPA with special cases, which is what >> worries me. > What does this mean ? > > We *have* a special case, and we are discussing how to handle it. > > The situation here (I do not want to call it a 'problem') is that we > decided to put business logic around user creation in the framework > because we thought that would be easier to prototype and develop in > python code rather than in a plugin. > > However this special handling *must* be limited. LDAP is our main > storage, and we must allow LDAP operations against it. Special cases > needs to be limited to things that are really harder done in plugins > rather then python code. > > For example if we need triggers for some operations in LDAP, they *must* > be done through a 389ds plugin. Otherwise LDAP quickly becomes a > read-only interface and interoperability quickly goes out of the window. > > I always treated the framework as a *management interface* on top. We > can do things that are 'convenient', and 'help' admins avoid mistakes, > but we cannot move core functionality in the framework, it would be a > grave mistake. User creation *is* a special case, but should remain one > of very few special exceptions. > > This very thread and the need for the interface proposed in this thread > is a clear example of why we need to be extremely careful not to be too > liberal with what business logic we move in the framework. > > LDAP keeps us honest, so we need to limit what we do in the framework, > otherwise we'll keep taking shortcuts and soon enough it goes out of > control and we loose interoperability with anything that is not > purpose-built to talk to our framework. > > This should be an unacceptable scenario because it is like putting > ourselves in a getto. > > We trade greater interoperability and acceptance for small conveniences > here and there. > > We must thread *very* carefully along this line. +1 - virtual trees usually end up being rat holes with no end What is the problem we're trying to solve? To be able to call python code in response to an LDAP operation? What if we added a python interpreter to 389, like OpenLDAP back-python? > > Simo. > From rcritten at redhat.com Thu Feb 14 16:27:08 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 14 Feb 2013 11:27:08 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511D0AA0.3020209@redhat.com> References: <51119947.9050707@redhat.com> <511390EB.40701@redhat.com> <5113B55B.7030605@redhat.com> <511CCDC1.5030908@redhat.com> <511CF611.1020805@redhat.com> <511D0AA0.3020209@redhat.com> Message-ID: <511D105C.90204@redhat.com> Petr Vobornik wrote: > On 02/14/2013 03:34 PM, Rob Crittenden wrote: >> Petr Vobornik wrote: >>> On 02/07/2013 03:08 PM, Rob Crittenden wrote: >>>> Petr Vobornik wrote: >>>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>>> This adds a cert-find command for the dogtag backend. >>>>>> >>>>>> Searches can be done by serial number, by subject, revocation reason, >>>>>> issue date, notbefore, notafter and revocation dates. >>>>>> >>>>>> I added some basic tests for this. I made it a separate test file >>>>>> because the cert plugin tests do not use the declarative format and >>>>>> rely >>>>>> on the selfsign backend by default. >>>>>> >>>>>> rob >>>>>> >>>>> >>>>> Should I create Web UI in scope of this ticket or a new one? >>>>> >>>>> I was also thinking if it's time to implement #191 'Web UI: specify >>>>> fields to search on' [1]. Maybe in Pilsner. >>>>> >>>>> [1] https://fedorahosted.org/freeipa/ticket/191 >>>> >>>> I'm going to open a UI ticket once the API is finalized. I didn't want >>>> to give you a moving target to work against. >>>> >>>> rob >>>> >>> >>> I see that the search requires to specify options for attributes to >>> search on. There is no general CRITERIA positional argument as in other >>> find commands or am I mistaken? >>> >>> Is it possible to add the CRITERIA argument? Is the no 'OR' search an >>> obstacle for it? >>> >>> If so we would really need to push the ticket #191 because UI doesn't >>> support search by only specifying specific attributes yet. >> >> Your analysis is correct. >> >> It may be considered a hack but what if I treat subject as the CRITERIA >> argument? >> >> rob > > Better that than nothing. > > Just a confirmation: when user does not set any option, it will return > all certificates? Or it will return nothing? > > I see Web UI implementation this way: > 1) implement simple search with the hack now > 2) if there will be time before release (after the refactoring and > other tickets) implement #191 (will require UXD input) to implement this > the proper way. We can move #191 to triage to decide it. The thing is we have to live with whatever API choice we decide on. It is more correct to have no positional args and stick with options, to mimik what the remote API provides. If you search on nothing you get everything, up to the default sizelimit. rob From abokovoy at redhat.com Thu Feb 14 16:37:19 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 14 Feb 2013 18:37:19 +0200 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <511CCC0E.5060409@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> <20130212170013.GB476@redhat.com> <511CCC0E.5060409@redhat.com> Message-ID: <20130214163719.GG476@redhat.com> On Thu, 14 Feb 2013, Tomas Babej wrote: >>>>> + Str('ipanttrusteddomainname?', >>>>> + cli_name='dom_name', >>>>> + flags=('no_search', 'virtual_attribute'), >>>>> + label=_('Name of the trusted domain'), >>>>> + ), >>>> New options is added but API.txt wasn't changed. As result, 'make rpms' >>>> does not work. >>>> >>>> Could you please fix the patch and re-send it? >>>> >>> Sorry about that. >>> >>> Updated patch attached. >> I have one small question regarding use of dom_sid/dom_name. >> >> If both dom_sid and dom_name were specified, failing to resolve dom_name >> would force command to raise exception. >> >> I'm not sure this is right behavior. Probably we should detect that both >> dom_sid and dom_name were specified and bail out earlier so that only >> one of them is accepted. That would be clearer to users, wouldn't it > Sure, I definitely agree on that point. I added the check to idrange-add and > idrange-mod. Also, the patch needed a rebase to apply on the current master. I tried to play with various scenarious and one thing I noticed is that we refer dom_sid and dom_name in the error messages as they are named internally. CLI replaces underscore by hyphen (_ -> -) and therefore this error message becomes wrong -- you cannot specify --dom_sid, this option is unknown to CLI. In Web UI this would also mean an error message pointing to non-existing option. Perhaps it would be reasonable to name options '--name' and '--sid'? We don't have any clash there: ------------------------------------------------------------------------- # ipa idrange-mod --help Usage: ipa [global-options] idrange-mod NAME [options] Positional arguments: NAME Range name Options: -h, --help show this help message and exit --base-id=INT First Posix ID of the range --range-size=INT Number of IDs in the range --rid-base=INT First RID of the corresponding RID range --secondary-rid-base=INT First RID of the secondary RID range --dom-sid=STR Domain SID of the trusted domain --dom-name=STR Name of the trusted domain --setattr=STR Set an attribute to a name/value pair. Format is attr=value. For multi-valued attributes, the command replaces the values already present. --addattr=STR Add an attribute/value pair. Format is attr=value. The attribute must be part of the schema. --delattr=STR Delete an attribute/value pair. The option willbe evaluated last, after all sets and adds. --rights Display the access rights of this entry(requires --all). See ipa man page for details. --all Retrieve and print all attributes from the server. Affects command output. --raw Print entries as stored on the server. Only affects output format. ------------------------------------------------------------------------- So, if --name and --sid were used, an error message would become ---------------------------------------------------------------------- # ipa idrange-mod AD.LAN_id_range --dom-name foo.bar ipa: ERROR: invalid 'ID Range setup': SID for the specified trusted domain name could not be found. Please specify the SID directly using --sid option. ---------------------------------------------------------------------- Additionally, there is an error when SID for an object within the domain is specified. Last RID of the SID represents an object within the domain and we generally need to be careful allowing it in the place where domain SID is specified: # ipa idrange-mod AD.LAN_id_range --dom-sid S-1-5-21-3502988750-125904550-3683905862-1 ----------------------------------- Modified ID range "AD.LAN_id_range" ----------------------------------- Range name: AD.LAN_id_range First Posix ID of the range: 1442800000 Number of IDs in the range: 200000 First RID of the corresponding RID range: 0 Domain SID of the trusted domain: S-1-5-21-3502988750-125904550-3683905862-1 Range type: Active Directory domain range Now this range is completely unusable due to the fact that there is no way to match the domain SID against the range. I think we need to make the check against established trusts more strict and only allow exact match. -- / Alexander Bokovoy From pvoborni at redhat.com Thu Feb 14 16:42:14 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 14 Feb 2013 17:42:14 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511D105C.90204@redhat.com> References: <51119947.9050707@redhat.com> <511390EB.40701@redhat.com> <5113B55B.7030605@redhat.com> <511CCDC1.5030908@redhat.com> <511CF611.1020805@redhat.com> <511D0AA0.3020209@redhat.com> <511D105C.90204@redhat.com> Message-ID: <511D13E6.1020501@redhat.com> On 02/14/2013 05:27 PM, Rob Crittenden wrote: > Petr Vobornik wrote: >> On 02/14/2013 03:34 PM, Rob Crittenden wrote: >>> Petr Vobornik wrote: >>>> On 02/07/2013 03:08 PM, Rob Crittenden wrote: >>>>> Petr Vobornik wrote: >>>>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>>>> This adds a cert-find command for the dogtag backend. >>>>>>> >>>>>>> Searches can be done by serial number, by subject, revocation >>>>>>> reason, >>>>>>> issue date, notbefore, notafter and revocation dates. >>>>>>> >>>>>>> I added some basic tests for this. I made it a separate test file >>>>>>> because the cert plugin tests do not use the declarative format and >>>>>>> rely >>>>>>> on the selfsign backend by default. >>>>>>> >>>>>>> rob >>>>>>> >>>>>> >>>>>> Should I create Web UI in scope of this ticket or a new one? >>>>>> >>>>>> I was also thinking if it's time to implement #191 'Web UI: specify >>>>>> fields to search on' [1]. Maybe in Pilsner. >>>>>> >>>>>> [1] https://fedorahosted.org/freeipa/ticket/191 >>>>> >>>>> I'm going to open a UI ticket once the API is finalized. I didn't want >>>>> to give you a moving target to work against. >>>>> >>>>> rob >>>>> >>>> >>>> I see that the search requires to specify options for attributes to >>>> search on. There is no general CRITERIA positional argument as in other >>>> find commands or am I mistaken? >>>> >>>> Is it possible to add the CRITERIA argument? Is the no 'OR' search an >>>> obstacle for it? >>>> >>>> If so we would really need to push the ticket #191 because UI doesn't >>>> support search by only specifying specific attributes yet. >>> >>> Your analysis is correct. >>> >>> It may be considered a hack but what if I treat subject as the CRITERIA >>> argument? >>> >>> rob >> >> Better that than nothing. >> >> Just a confirmation: when user does not set any option, it will return >> all certificates? Or it will return nothing? >> >> I see Web UI implementation this way: >> 1) implement simple search with the hack now >> 2) if there will be time before release (after the refactoring and >> other tickets) implement #191 (will require UXD input) to implement this >> the proper way. We can move #191 to triage to decide it. > > The thing is we have to live with whatever API choice we decide on. > > It is more correct to have no positional args and stick with options, to > mimik what the remote API provides. It's not a problem to use the --subject option, so we don't have to implement the positional argument when it has the same meaning. What I meant by use hack is 'filter by subject or nothing'. --> This patch is OK from UI perspective (considering the limitations). > > If you search on nothing you get everything, up to the default sizelimit. > > rob -- Petr Vobornik From simo at redhat.com Thu Feb 14 16:53:32 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 14 Feb 2013 11:53:32 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511D114D.3040904@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> <511CA765.1010704@redhat.com> <511D0126.9090105@redhat.com> <511D02DF.90302@redhat.com> <1360857683.12328.207.camel@willson.li.ssimo.org> <511D114D.3040904@redhat.com> Message-ID: <1360860812.12328.218.camel@willson.li.ssimo.org> On Thu, 2013-02-14 at 17:31 +0100, Petr Viktorin wrote: > On 02/14/2013 05:01 PM, Simo Sorce wrote: > > On Thu, 2013-02-14 at 16:29 +0100, Petr Viktorin wrote: > >> Then I recommend doing this. It avoids problems with delegation (the > >> "real" tree permissions are used) and looping (plugin and IPA write to > >> separate trees). > > > > Virtual objects are not free of issues, you are just trading some issues > > for others here, and I do not think you are properly evaluating the > > trade here. > > > > I am dead set against using staging areas, and I want to see proof they > > are a good idea, not handwaving 'fears' of using json in the server to > > forward operations directly to the framework. > > > >> Other operations (deletes, mods) can be either > >> similarly delegated to the "real" tree, > > > > And *this* is a slippery slope. you are trading delegating one single > > operation to the framework directly, with proper error propagation back > > to the client, with now implementing full virtualized operations for mod > > and delete, and bind ? and search and then ? > > > > You are now basically paving the way for a virtual directory in tree. > > > > Sorry, but no. > > > >> or passed through IPA if we want to do that in the future. > > > >> Problems with replication can be solved by just not replicating the > >> separate tree. > > > > More and more hacks, for this supposedly 'cleaner' or 'simpler' > > solution ... sorry if I don't bite. > > > >> It also doesn't pollute core IPA with special cases, which is what > >> worries me. > > > > What does this mean ? > > > > We *have* a special case, and we are discussing how to handle it. > > > > The situation here (I do not want to call it a 'problem') is that we > > decided to put business logic around user creation in the framework > > because we thought that would be easier to prototype and develop in > > python code rather than in a plugin. > > > > However this special handling *must* be limited. LDAP is our main > > storage, and we must allow LDAP operations against it. Special cases > > needs to be limited to things that are really harder done in plugins > > rather then python code. > > > > For example if we need triggers for some operations in LDAP, they *must* > > be done through a 389ds plugin. Otherwise LDAP quickly becomes a > > read-only interface and interoperability quickly goes out of the window. > > > > I always treated the framework as a *management interface* on top. We > > can do things that are 'convenient', and 'help' admins avoid mistakes, > > but we cannot move core functionality in the framework, it would be a > > grave mistake. User creation *is* a special case, but should remain one > > of very few special exceptions. > > > > This very thread and the need for the interface proposed in this thread > > is a clear example of why we need to be extremely careful not to be too > > liberal with what business logic we move in the framework. > > > > LDAP keeps us honest, so we need to limit what we do in the framework, > > otherwise we'll keep taking shortcuts and soon enough it goes out of > > control and we loose interoperability with anything that is not > > purpose-built to talk to our framework. > > > > This should be an unacceptable scenario because it is like putting > > ourselves in a getto. > > > > We trade greater interoperability and acceptance for small conveniences > > here and there. > > > > We must thread *very* carefully along this line. > > > > Simo. > > > Ah! This is a point we need to stress more, I didn't get it. All this > time I thought of it the other way -- IPA being the main interface, with > LDAP being more of an implementation detail -- the storage, and a > read-only interface for LDAP consumers. > > Thanks for explaining, your solution makes perfect sense then. > > However, if there are more people like me, then there's code in IPA that > assumes IPA is the only interface modifications, and raw LDAP operations > are hacks that happen to work for now. > IPA's command plugins also seem misguided now, as you pointed out. > > > I don't like this view much but, well, I can adapt. > Allowing writes by tools that aren't purpose-built to talk to our > framework, (for example ones that don't honor our schema), still seems > like something to be wary of. It's a balancing act. The schema is always honored, that's why there are MUST and MAY attributes. If the schema is build with the wrong specification of MUST and MAYs that's a bug on our part. Also, there are some parts of the tree that are more sensitive than others. Some things are pretty IPA specific, and for those cases relying a bit more on the framework is ok for now. But in general LDAP is a primary interface, it's where replication happens, and where all components converge. So it is the center of all things. Think about triggers for example. One of the triggers that gets often requested is to be create home directories when users are created. Now, besides the point of whether we want or can do that, if you were to implement it you would have to do it at the 389ds level. You cannot do it in the framework at all for the simple reason that the creation may need to happen on a completely different server than the one on which the admin is creating the users, so acting at the framework level would be meaningless. This information is transported to all other servers via LDAP replication. So the only sane way to trigger an operation is via a 389ds plugin so that it can trigger on the right destination server. This is just an example, but shows that LDAP is the core. Now the framework is crucial, I do not want to diminish the framework role or importance, but it is has its place and function, and must not be expanded where it shouldn't go. Simo. -- Simo Sorce * Red Hat, Inc * New York From pviktori at redhat.com Thu Feb 14 16:31:09 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 14 Feb 2013 17:31:09 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360857683.12328.207.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAE08.5040108@redhat.com> <511BB255.7090104@redhat.com> <1360772830.12328.75.camel@willson.li.ssimo.org> <511BC956.5010307@redhat.com> <1360777847.12328.124.camel@willson.li.ssimo.org> <511BD406.9090302@redhat.com> <1360779108.12328.133.camel@willson.li.ssimo.org> <511CA765.1010704@redhat.com> <511D0126.9090105@redhat.com> <511D02DF.90302@redhat.com> <1360857683.12328.207.camel@willson.li.ssimo.org> Message-ID: <511D114D.3040904@redhat.com> On 02/14/2013 05:01 PM, Simo Sorce wrote: > On Thu, 2013-02-14 at 16:29 +0100, Petr Viktorin wrote: >> Then I recommend doing this. It avoids problems with delegation (the >> "real" tree permissions are used) and looping (plugin and IPA write to >> separate trees). > > Virtual objects are not free of issues, you are just trading some issues > for others here, and I do not think you are properly evaluating the > trade here. > > I am dead set against using staging areas, and I want to see proof they > are a good idea, not handwaving 'fears' of using json in the server to > forward operations directly to the framework. > >> Other operations (deletes, mods) can be either >> similarly delegated to the "real" tree, > > And *this* is a slippery slope. you are trading delegating one single > operation to the framework directly, with proper error propagation back > to the client, with now implementing full virtualized operations for mod > and delete, and bind ? and search and then ? > > You are now basically paving the way for a virtual directory in tree. > > Sorry, but no. > >> or passed through IPA if we want to do that in the future. > >> Problems with replication can be solved by just not replicating the >> separate tree. > > More and more hacks, for this supposedly 'cleaner' or 'simpler' > solution ... sorry if I don't bite. > >> It also doesn't pollute core IPA with special cases, which is what >> worries me. > > What does this mean ? > > We *have* a special case, and we are discussing how to handle it. > > The situation here (I do not want to call it a 'problem') is that we > decided to put business logic around user creation in the framework > because we thought that would be easier to prototype and develop in > python code rather than in a plugin. > > However this special handling *must* be limited. LDAP is our main > storage, and we must allow LDAP operations against it. Special cases > needs to be limited to things that are really harder done in plugins > rather then python code. > > For example if we need triggers for some operations in LDAP, they *must* > be done through a 389ds plugin. Otherwise LDAP quickly becomes a > read-only interface and interoperability quickly goes out of the window. > > I always treated the framework as a *management interface* on top. We > can do things that are 'convenient', and 'help' admins avoid mistakes, > but we cannot move core functionality in the framework, it would be a > grave mistake. User creation *is* a special case, but should remain one > of very few special exceptions. > > This very thread and the need for the interface proposed in this thread > is a clear example of why we need to be extremely careful not to be too > liberal with what business logic we move in the framework. > > LDAP keeps us honest, so we need to limit what we do in the framework, > otherwise we'll keep taking shortcuts and soon enough it goes out of > control and we loose interoperability with anything that is not > purpose-built to talk to our framework. > > This should be an unacceptable scenario because it is like putting > ourselves in a getto. > > We trade greater interoperability and acceptance for small conveniences > here and there. > > We must thread *very* carefully along this line. > > Simo. Ah! This is a point we need to stress more, I didn't get it. All this time I thought of it the other way -- IPA being the main interface, with LDAP being more of an implementation detail -- the storage, and a read-only interface for LDAP consumers. Thanks for explaining, your solution makes perfect sense then. However, if there are more people like me, then there's code in IPA that assumes IPA is the only interface modifications, and raw LDAP operations are hacks that happen to work for now. IPA's command plugins also seem misguided now, as you pointed out. I don't like this view much but, well, I can adapt. Allowing writes by tools that aren't purpose-built to talk to our framework, (for example ones that don't honor our schema), still seems like something to be wary of. -- Petr? From edewata at redhat.com Thu Feb 14 17:33:32 2013 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 14 Feb 2013 11:33:32 -0600 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360850800.12328.175.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> <511CE60B.5060106@redhat.com> <1360850800.12328.175.camel@willson.li.ssimo.org> Message-ID: <511D1FEC.3090003@redhat.com> On 2/14/2013 8:06 AM, Simo Sorce wrote: > On Thu, 2013-02-14 at 14:26 +0100, Petr Spacek wrote: > >> In my Fedora 17 I found package python-ldaptor. It seems to offer nice support >> for writing own event-based LDAP servers. For simple LDAP proxy it could be >> enough. >> >> $ yum install python-ldaptor >> $ python >> import ldaptor.protocols.ldap.ldapserver >> help(ldaptor.protocols.ldap.ldapserver) > > No. > LDAP proxies are *not* simple. > > Ask Endi he's worked on a meta-directory for years. > > Simo. It depends on what you want to do with the proxy. If it's only a thin layer which converts the LDAP ADD to IPA user-add it might not be that complicated. Penrose virtual directory consists of a frontend LDAP interface, a transformation engine, and backends which may include an LDAP server as well. The front-end LDAP interface is the proxy we're talking about here, it's only used to receive LDAP requests and pass them to the transformation engine. The transformation engine is where the complexity occurs. In IPA this is already handled by the framework. In Penrose it's quite complex because it aims to provide a generic way to map an LDAP request to multiple backends which involves dealing with different types of backends, joining the backends, transforming the DN & attributes back and forth, etc. So I'd say implementing an LDAP frontend for IPA using Python is something worth exploring. That way it can run in the same process so there's no concern about JSON performance/stability. -- Endi S. Dewata From bcook at redhat.com Fri Feb 15 02:07:21 2013 From: bcook at redhat.com (Brian Cook) Date: Thu, 14 Feb 2013 21:07:21 -0500 (EST) Subject: [Freeipa-devel] patch for trac 2575 In-Reply-To: <511C8F63.4070307@redhat.com> Message-ID: <478729079.1780206.1360894041401.JavaMail.root@redhat.com> Thanks Martin and Dmitri. I have attached a patch that I -think- is formatted correctly... I removed the new variable and added check for --unattended. Thanks, Brian ----- Original Message ----- From: "Martin Kosek" To: dpal at redhat.com Cc: freeipa-devel at redhat.com Sent: Wednesday, February 13, 2013 11:16:51 PM Subject: Re: [Freeipa-devel] patch for trac 2575 On 02/14/2013 03:49 AM, Dmitri Pal wrote: > On 02/13/2013 05:20 PM, Brian Cook wrote: >> Please disregard the first patch as it still asked the user if they want to install DNS even if --setup-dns was passed, this one is fixed. >> >> Brian > > Brian, > > Thanks for the patch. > Can you please format it following these guidelines: > https://fedorahosted.org/freeipa/wiki/PatchFormat > > Thanks > Dmitri Hello Brian, Thanks for the patch! Also few technical notes: 1) There is no need to invent the new variable, you can ask and set options.setup_dns to True. We already to this in other parts incode 2) This patch would --unattended mode when no --setup-dns is passed Martin >> >> >> >> diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install >> index 1559107..96ef802 100755 >> --- a/install/tools/ipa-server-install >> +++ b/install/tools/ipa-server-install >> @@ -564,6 +564,7 @@ def main(): >> global pw_name >> global uninstalling >> global installation_cleanup >> + >> ds = None >> >> safe_options, options = parse_options() >> @@ -740,8 +741,18 @@ def main(): >> admin_password = "" >> reverse_zone = None >> >> - # check bind packages are installed >> + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection >> + setup_dns=False >> if options.setup_dns: >> + setup_dns=True >> + else: >> + # Ask user if they want to install DNS >> + if ipautil.user_input("Do you want to configure integrated DNS (bind)?", False): >> + setup_dns=True >> + >> + >> + # check bind packages are installed >> + if setup_dns: >> if not bindinstance.check_inst(options.unattended): >> sys.exit("Aborting installation") >> >> @@ -827,7 +838,7 @@ def main(): >> else: >> admin_password = options.admin_password >> >> - if options.setup_dns: >> + if setup_dns: >> if options.no_forwarders: >> dns_forwarders = () >> elif options.forwarders: >> @@ -858,7 +869,7 @@ def main(): >> print "Realm name: %s" % realm_name >> print >> >> - if options.setup_dns: >> + if setup_dns: >> print "BIND DNS server will be configured to serve IPA domain with:" >> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >> else ", ".join([str(ip) for ip in dns_forwarders])) >> @@ -1102,7 +1113,7 @@ def main(): >> persistent_search=options.persistent_search, >> serial_autoincrement=options.serial_autoincrement, >> ca_configured=not options.selfsign) >> - if options.setup_dns: >> + if setup_dns: >> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) >> >> bind.create_instance() >> @@ -1147,11 +1158,11 @@ def main(): >> print "\t\t * 80, 443: HTTP/HTTPS" >> print "\t\t * 389, 636: LDAP/LDAPS" >> print "\t\t * 88, 464: kerberos" >> - if options.setup_dns: >> + if setup_dns: >> print "\t\t * 53: bind" >> print "\t\tUDP Ports:" >> print "\t\t * 88, 464: kerberos" >> - if options.setup_dns: >> + if setup_dns: >> print "\t\t * 53: bind" >> if options.conf_ntp: >> print "\t\t * 123: ntp" >> >> >> >> >>> Message: 8 >>> Date: Wed, 13 Feb 2013 13:39:32 -0800 >>> From: Brian Cook >>> To: "freeipa-devel at redhat.com" >>> Subject: [Freeipa-devel] patch for trac 2575 >>> Message-ID: <9DD1D1BB-6B86-4EA1-B61B-B208E6BC7152 at redhat.com> >>> Content-Type: text/plain; charset="windows-1252" >>> >>> This is a patch for ticket 2575 on trac: [RFE] Installer wizard should prompt for DNS. This is my first time submitting a patch so I was looking for something that seemed relatively easy? >>> >>> Thanks, >>> Brian >>> >>> >>> diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install >>> index 1559107..d8c4ae5 100755 >>> --- a/install/tools/ipa-server-install >>> +++ b/install/tools/ipa-server-install >>> @@ -564,6 +564,7 @@ def main(): >>> global pw_name >>> global uninstalling >>> global installation_cleanup >>> + >>> ds = None >>> >>> safe_options, options = parse_options() >>> @@ -740,8 +741,18 @@ def main(): >>> admin_password = "" >>> reverse_zone = None >>> >>> - # check bind packages are installed >>> + # Setup a variable to use instead of options.setup_dns to enable interactive DNS selection >>> + setup_dns=False >>> if options.setup_dns: >>> + setup_dns=True >>> + >>> + # Ask user if they want to install DNS >>> + if ipautil.user_input("Do you want to cnfigure integrated DNS (bind)?", false): >>> + setup_dns=True >>> + >>> + >>> + # check bind packages are installed >>> + if setup_dns: >>> if not bindinstance.check_inst(options.unattended): >>> sys.exit("Aborting installation") >>> >>> @@ -827,7 +838,7 @@ def main(): >>> else: >>> admin_password = options.admin_password >>> >>> - if options.setup_dns: >>> + if setup_dns: >>> if options.no_forwarders: >>> dns_forwarders = () >>> elif options.forwarders: >>> @@ -858,7 +869,7 @@ def main(): >>> print "Realm name: %s" % realm_name >>> print >>> >>> - if options.setup_dns: >>> + if setup_dns: >>> print "BIND DNS server will be configured to serve IPA domain with:" >>> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >>> else ", ".join([str(ip) for ip in dns_forwarders])) >>> @@ -1102,7 +1113,7 @@ def main(): >>> persistent_search=options.persistent_search, >>> serial_autoincrement=options.serial_autoincrement, >>> ca_configured=not options.selfsign) >>> - if options.setup_dns: >>> + if setup_dns: >>> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=dm_password) >>> >>> bind.create_instance() >>> @@ -1147,11 +1158,11 @@ def main(): >>> print "\t\t * 80, 443: HTTP/HTTPS" >>> print "\t\t * 389, 636: LDAP/LDAPS" >>> print "\t\t * 88, 464: kerberos" >>> - if options.setup_dns: >>> + if setup_dns: >>> print "\t\t * 53: bind" >>> print "\t\tUDP Ports:" >>> print "\t\t * 88, 464: kerberos" >>> - if options.setup_dns: >>> + if setup_dns: >>> print "\t\t * 53: bind" >>> if options.conf_ntp: >>> print "\t\t * 123: ntp" >>> >>> >>> >>> >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> >>> End of Freeipa-devel Digest, Vol 69, Issue 49 >>> ********************************************* >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > _______________________________________________ Freeipa-devel mailing list Freeipa-devel at redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-bcook-0001-Add-DNS-Setup-Prompt-to-Install.patch Type: text/x-patch Size: 1574 bytes Desc: not available URL: From pspacek at redhat.com Fri Feb 15 08:04:38 2013 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 15 Feb 2013 09:04:38 +0100 Subject: [Freeipa-devel] [PATCH] 370-371 ipa-kdb: remove memory leaks In-Reply-To: <20130214090719.GO24753@localhost.localdomain> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> <511A5C5C.3030807@redhat.com> <1360682788.3838.45.camel@willson.li.ssimo.org> <511A62FC.5080907@redhat.com> <20130214090719.GO24753@localhost.localdomain> Message-ID: <511DEC16.3030306@redhat.com> On 14.2.2013 10:07, Sumit Bose wrote: > On Tue, Feb 12, 2013 at 04:42:52PM +0100, Martin Kosek wrote: >> On 02/12/2013 04:26 PM, Simo Sorce wrote: >>> On Tue, 2013-02-12 at 16:14 +0100, Martin Kosek wrote: >>>> Explained in the commit description - this may not be super-critical, I just >>>> followed info in ldap_search_ext() man page: >>>> >>>> ... >>>> >>>> Note that res parameter of ldap_search_ext_s() and ldap_search_s() >>>> should be freed with >>>> ldap_msgfree() regardless of return value of these functions. >>>> ... >>> >>> OK. >>> >>>>> This snippet seem to change the logic. >>>>> >>>>> Before the condition was !ipadb_need_retry() and no you change it to >>>>> ipadb_need_retry() so it looks reversed, was this on purpose ? >>>> >>>> As noted in commit description, this check was wrong and it was causing the >>>> function to _always_ retry at least once because ipadb_need_retry returns true >>>> when we need to retry and not 0 -> thus no "!" is needed. >>> >>> I do not like much 'sneaking' this kind of change in the same patch that >>> fixes memory leaks, so if you want to plit in 2 patches it would be >>> nice, but I am of course ok with the change. >> >> I agree with that. Lets do it right as we have not pushed it yet. I moved this >> change to a separate patch (attached). > > I didn't found any issues while testing, so ACK. > > When I run kinit in a endless loop I still see a constant increase in > the resident memory size, but it looks like it is not related to the > ipadb code. I will investigate further. > > bye, > Sumit KDC has built-in in-memory "lookaside cache", see http://mailman.mit.edu/pipermail/krbdev/2012-June/010985.html You can recompile KDC without lookaside cache or run test for more than 120 seconds. Memory consumption should be roughly stable (if request rate is stable all the time). -- Petr^2 Spacek From sbose at redhat.com Fri Feb 15 08:24:27 2013 From: sbose at redhat.com (Sumit Bose) Date: Fri, 15 Feb 2013 09:24:27 +0100 Subject: [Freeipa-devel] [PATCH] 370-371 ipa-kdb: remove memory leaks In-Reply-To: <511DEC16.3030306@redhat.com> References: <511A2680.1030300@redhat.com> <1360678563.3838.31.camel@willson.li.ssimo.org> <511A5C5C.3030807@redhat.com> <1360682788.3838.45.camel@willson.li.ssimo.org> <511A62FC.5080907@redhat.com> <20130214090719.GO24753@localhost.localdomain> <511DEC16.3030306@redhat.com> Message-ID: <20130215082427.GQ24753@localhost.localdomain> On Fri, Feb 15, 2013 at 09:04:38AM +0100, Petr Spacek wrote: > On 14.2.2013 10:07, Sumit Bose wrote: > >On Tue, Feb 12, 2013 at 04:42:52PM +0100, Martin Kosek wrote: > >>On 02/12/2013 04:26 PM, Simo Sorce wrote: > >>>On Tue, 2013-02-12 at 16:14 +0100, Martin Kosek wrote: > >>>>Explained in the commit description - this may not be super-critical, I just > >>>>followed info in ldap_search_ext() man page: > >>>> > >>>>... > >>>> > >>>> Note that res parameter of ldap_search_ext_s() and ldap_search_s() > >>>>should be freed with > >>>> ldap_msgfree() regardless of return value of these functions. > >>>>... > >>> > >>>OK. > >>> > >>>>>This snippet seem to change the logic. > >>>>> > >>>>>Before the condition was !ipadb_need_retry() and no you change it to > >>>>>ipadb_need_retry() so it looks reversed, was this on purpose ? > >>>> > >>>>As noted in commit description, this check was wrong and it was causing the > >>>>function to _always_ retry at least once because ipadb_need_retry returns true > >>>>when we need to retry and not 0 -> thus no "!" is needed. > >>> > >>>I do not like much 'sneaking' this kind of change in the same patch that > >>>fixes memory leaks, so if you want to plit in 2 patches it would be > >>>nice, but I am of course ok with the change. > >> > >>I agree with that. Lets do it right as we have not pushed it yet. I moved this > >>change to a separate patch (attached). > > > >I didn't found any issues while testing, so ACK. > > > >When I run kinit in a endless loop I still see a constant increase in > >the resident memory size, but it looks like it is not related to the > >ipadb code. I will investigate further. > > > >bye, > >Sumit > > KDC has built-in in-memory "lookaside cache", see > http://mailman.mit.edu/pipermail/krbdev/2012-June/010985.html > > You can recompile KDC without lookaside cache or run test for more > than 120 seconds. Memory consumption should be roughly stable (if > request rate is stable all the time). ah, thanks for the hint. The memory consumption really saturates after 2-3 minutes. bye, Sumit > > -- > Petr^2 Spacek > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From pspacek at redhat.com Fri Feb 15 08:53:46 2013 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 15 Feb 2013 09:53:46 +0100 Subject: [Freeipa-devel] [PATCH 0113] Periodically reconnect if Kerberos KDC is unavailable Message-ID: <511DF79A.4010804@redhat.com> Hello, Periodically reconnect if Kerberos KDC is unavailable. https://fedorahosted.org/bind-dyndb-ldap/ticket/100 At the moment, Kerberos libraries contain a memory leak which will be triggered by periodical reconnecting implemented in this ticket. https://bugzilla.redhat.com/show_bug.cgi?id=911110 https://bugzilla.redhat.com/show_bug.cgi?id=911147 -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0113-Periodically-reconnect-if-Kerberos-KDC-is-unavailabl.patch Type: text/x-patch Size: 884 bytes Desc: not available URL: From tbabej at redhat.com Fri Feb 15 11:29:49 2013 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 15 Feb 2013 12:29:49 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <20130214163719.GG476@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> <20130212170013.GB476@redhat.com> <511CCC0E.5060409@redhat.com> <20130214163719.GG476@redhat.com> Message-ID: <511E1C2D.9080701@redhat.com> On 02/14/2013 05:37 PM, Alexander Bokovoy wrote: > On Thu, 14 Feb 2013, Tomas Babej wrote: >>>>>> + Str('ipanttrusteddomainname?', >>>>>> + cli_name='dom_name', >>>>>> + flags=('no_search', 'virtual_attribute'), >>>>>> + label=_('Name of the trusted domain'), >>>>>> + ), >>>>> New options is added but API.txt wasn't changed. As result, 'make >>>>> rpms' >>>>> does not work. >>>>> >>>>> Could you please fix the patch and re-send it? >>>>> >>>> Sorry about that. >>>> >>>> Updated patch attached. >>> I have one small question regarding use of dom_sid/dom_name. >>> >>> If both dom_sid and dom_name were specified, failing to resolve >>> dom_name >>> would force command to raise exception. >>> >>> I'm not sure this is right behavior. Probably we should detect that >>> both >>> dom_sid and dom_name were specified and bail out earlier so that only >>> one of them is accepted. That would be clearer to users, wouldn't it >> Sure, I definitely agree on that point. I added the check to >> idrange-add and >> idrange-mod. Also, the patch needed a rebase to apply on the current >> master. > I tried to play with various scenarious and one thing I noticed is > that we refer dom_sid and dom_name in the error messages as they are > named internally. CLI replaces underscore by hyphen (_ -> -) and > therefore > this error message becomes wrong -- you cannot specify --dom_sid, this > option is unknown to CLI. > > In Web UI this would also mean an error message pointing to non-existing > option. Perhaps it would be reasonable to name options '--name' and > '--sid'? We don't have any clash there: > ------------------------------------------------------------------------- > # ipa idrange-mod --help > Usage: ipa [global-options] idrange-mod NAME [options] > > Positional arguments: > NAME Range name > > Options: > -h, --help show this help message and exit > --base-id=INT First Posix ID of the range > --range-size=INT Number of IDs in the range > --rid-base=INT First RID of the corresponding RID range > --secondary-rid-base=INT > First RID of the secondary RID range > --dom-sid=STR Domain SID of the trusted domain > --dom-name=STR Name of the trusted domain > --setattr=STR Set an attribute to a name/value pair. Format is > attr=value. For multi-valued attributes, the > command > replaces the values already present. > --addattr=STR Add an attribute/value pair. Format is > attr=value. The > attribute must be part of the schema. > --delattr=STR Delete an attribute/value pair. The option willbe > evaluated last, after all sets and adds. > --rights Display the access rights of this entry(requires > --all). See ipa man page for details. > --all Retrieve and print all attributes from the > server. > Affects command output. > --raw Print entries as stored on the server. Only > affects > output format. > ------------------------------------------------------------------------- > > So, if --name and --sid were used, an error message would become > ---------------------------------------------------------------------- > # ipa idrange-mod AD.LAN_id_range --dom-name foo.bar ipa: ERROR: > invalid 'ID Range setup': SID for the specified trusted > domain name could not be found. Please specify the SID directly using > --sid option. > ---------------------------------------------------------------------- > > > Additionally, there is an error when SID for an object within the domain > is specified. Last RID of the SID represents an object within the domain > and we generally need to be careful allowing it in the place where > domain SID is specified: > > # ipa idrange-mod AD.LAN_id_range --dom-sid > S-1-5-21-3502988750-125904550-3683905862-1 > ----------------------------------- > Modified ID range "AD.LAN_id_range" > ----------------------------------- > Range name: AD.LAN_id_range > First Posix ID of the range: 1442800000 > Number of IDs in the range: 200000 > First RID of the corresponding RID range: 0 > Domain SID of the trusted domain: > S-1-5-21-3502988750-125904550-3683905862-1 > Range type: Active Directory domain range > > Now this range is completely unusable due to the fact that there is no > way to match the domain SID against the range. > > I think we need to make the check against established trusts more > strict and only allow exact match. > 1.) Regarding dom_sid and dom_name options, we do not have to change their internal names to get more user-friendly error messages. These are hardcoded strings, and not generated from internal names of the options. I followed the naming convention already set in the file, but you're right, the current state is somewhat confusing to the end user. I changed all the error messages so that they refer to hyphen-versions of the options (not only dom_sid/dom_name but also rid_base, etc.). I considered renaming the options to --sid and --name. However, although --sid is pretty self-explanatory, --name could be quite confusing, as ID range has name of its own. Furthermore, this would break some documentation / other references, and since refactoring of the error messages described above solves our issue here, I'd vote for that solution. 2.) Exact match against estabilished trusts - this is a nice catch! However, as far as I understand this is not something introduced by my patch, and it would not be transparent to include the fix here. If you agree, I'll create a new ticket in Trac. Updated patch attached (error messages refactored). Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0030-4-Add-option-to-specify-SID-using-domain-name-to-idran.patch Type: text/x-patch Size: 11844 bytes Desc: not available URL: From akrivoka at redhat.com Fri Feb 15 14:22:33 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Fri, 15 Feb 2013 15:22:33 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation Message-ID: <511E44A9.9090509@redhat.com> Hello, The .isalpha() check in validate_domain_name() was too strict, causing some commands like ipa dnsrecord-add to fail. https://fedorahosted.org/freeipa/ticket/3385 -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akrivoka-0006-Remove-check-for-alphabetic-only-characters-from-dom.patch Type: text/x-patch Size: 1079 bytes Desc: not available URL: From pviktori at redhat.com Fri Feb 15 15:15:44 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 15 Feb 2013 16:15:44 +0100 Subject: [Freeipa-devel] [PATCH] 0180 Check SSH connection in ipa-replica-conncheck Message-ID: <511E5120.6050604@redhat.com> ipa-replica-conncheck ran SSH in quiet mode, probably to suppress a message about connecting to an unknown host. This made it hard to debug connection errors. I didn't find a way to separate SSH output from the output of the called command, I decided to try an additional SSH connection before calling conncheck. SSH is set to verbose and if it fails the errors get printed out. Also, the host is added to a temporary known_hosts file. The second SSH is called without the -q flag so errors/warnings are not lost even if the command fails. The temporary known_hosts file is used so the unknown host warning doesn't appear here. https://fedorahosted.org/freeipa/ticket/3402 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0180-Check-SSH-connection-in-ipa-replica-conncheck.patch Type: text/x-patch Size: 3235 bytes Desc: not available URL: From rcritten at redhat.com Fri Feb 15 15:38:35 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 15 Feb 2013 10:38:35 -0500 Subject: [Freeipa-devel] [PATCH] 0180 Check SSH connection in ipa-replica-conncheck In-Reply-To: <511E5120.6050604@redhat.com> References: <511E5120.6050604@redhat.com> Message-ID: <511E567B.3070307@redhat.com> Petr Viktorin wrote: > ipa-replica-conncheck ran SSH in quiet mode, probably to suppress a > message about connecting to an unknown host. This made it hard to debug > connection errors. > > I didn't find a way to separate SSH output from the output of the called > command, I decided to try an additional SSH connection before calling > conncheck. SSH is set to verbose and if it fails the errors get printed > out. Also, the host is added to a temporary known_hosts file. > The second SSH is called without the -q flag so errors/warnings are not > lost even if the command fails. The temporary known_hosts file is used > so the unknown host warning doesn't appear here. > > https://fedorahosted.org/freeipa/ticket/3402 The general procedure looks good, I don't think we should hardcode the path to ssh. ipautil.run() overrides the current environment so we should be able to safely run just 'ssh'. We eventually need a cross-platform way of locating system binaries. The hardcoded path to ipa-replica-conncheck is probably ok since we provide that binary ourselves. rob From pviktori at redhat.com Fri Feb 15 16:40:45 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 15 Feb 2013 17:40:45 +0100 Subject: [Freeipa-devel] [PATCH] 0180 Check SSH connection in ipa-replica-conncheck In-Reply-To: <511E567B.3070307@redhat.com> References: <511E5120.6050604@redhat.com> <511E567B.3070307@redhat.com> Message-ID: <511E650D.3050100@redhat.com> On 02/15/2013 04:38 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> ipa-replica-conncheck ran SSH in quiet mode, probably to suppress a >> message about connecting to an unknown host. This made it hard to debug >> connection errors. >> >> I didn't find a way to separate SSH output from the output of the called >> command, I decided to try an additional SSH connection before calling >> conncheck. SSH is set to verbose and if it fails the errors get printed >> out. Also, the host is added to a temporary known_hosts file. >> The second SSH is called without the -q flag so errors/warnings are not >> lost even if the command fails. The temporary known_hosts file is used >> so the unknown host warning doesn't appear here. >> >> https://fedorahosted.org/freeipa/ticket/3402 > > The general procedure looks good, I don't think we should hardcode the > path to ssh. ipautil.run() overrides the current environment so we > should be able to safely run just 'ssh'. > > We eventually need a cross-platform way of locating system binaries. > > The hardcoded path to ipa-replica-conncheck is probably ok since we > provide that binary ourselves. > > rob Changed, thanks. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0180-02-Check-SSH-connection-in-ipa-replica-conncheck.patch Type: text/x-patch Size: 3226 bytes Desc: not available URL: From pviktori at redhat.com Fri Feb 15 16:56:06 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 15 Feb 2013 17:56:06 +0100 Subject: [Freeipa-devel] [RFE] Drop CSV support in CLI Message-ID: <511E68A6.8010409@redhat.com> Design document here: http://freeipa.org/page/V3/Drop_CSV Since this is a rather drastic change, I think it would be nice if the client tried to detect the old syntax and provide a message about the change. The detection doesn't have to be perfect, of course. To make that possible we can use JSON-RPC to provide extra data about the ValidationError. See the design doc for details. -- Petr? From rcritten at redhat.com Fri Feb 15 19:18:32 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 15 Feb 2013 14:18:32 -0500 Subject: [Freeipa-devel] [PATCH] 0180 Check SSH connection in ipa-replica-conncheck In-Reply-To: <511E650D.3050100@redhat.com> References: <511E5120.6050604@redhat.com> <511E567B.3070307@redhat.com> <511E650D.3050100@redhat.com> Message-ID: <511E8A08.2050002@redhat.com> Petr Viktorin wrote: > On 02/15/2013 04:38 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> ipa-replica-conncheck ran SSH in quiet mode, probably to suppress a >>> message about connecting to an unknown host. This made it hard to debug >>> connection errors. >>> >>> I didn't find a way to separate SSH output from the output of the called >>> command, I decided to try an additional SSH connection before calling >>> conncheck. SSH is set to verbose and if it fails the errors get printed >>> out. Also, the host is added to a temporary known_hosts file. >>> The second SSH is called without the -q flag so errors/warnings are not >>> lost even if the command fails. The temporary known_hosts file is used >>> so the unknown host warning doesn't appear here. >>> >>> https://fedorahosted.org/freeipa/ticket/3402 >> >> The general procedure looks good, I don't think we should hardcode the >> path to ssh. ipautil.run() overrides the current environment so we >> should be able to safely run just 'ssh'. >> >> We eventually need a cross-platform way of locating system binaries. >> >> The hardcoded path to ipa-replica-conncheck is probably ok since we >> provide that binary ourselves. >> >> rob > > Changed, thanks. > Looks and works well. I just have one final question. Should remote_addr and temp_known_hosts be passed in as args? They are basically globals but it is obvious where they came from, so not really a NAK, just a question. rob From rcritten at redhat.com Fri Feb 15 21:43:41 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 15 Feb 2013 16:43:41 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511B5B25.7040607@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> <511B5B25.7040607@redhat.com> Message-ID: <511EAC0D.1020905@redhat.com> Petr Viktorin wrote: > On 02/06/2013 07:23 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>> This adds a cert-find command for the dogtag backend. >>>> >>>> Searches can be done by serial number, by subject, revocation reason, >>>> issue date, notbefore, notafter and revocation dates. >>>> >>>> I added some basic tests for this. I made it a separate test file >>>> because the cert plugin tests do not use the declarative format and >>>> rely >>>> on the selfsign backend by default. >>>> >>>> rob >>> >>> Thanks! The code works well, but I found a few issues. >>> >>> >>> These tests don't work when the full test suite is run: test_cert adds >>> and revokes additional certs that throw the code off. >>> Perhaps have the tests only query valid certs? I don't see that option >>> but I think it would be helpful to support. >> >> I added some rather nasty hacks to the test to make things pass. I limit >> the search to 10 certificates, which is the number start with by >> default. There is an open dogtag ticket to return only VALID >> certificates so we should be able to drop this eventually. >> >> I had to go further on one of the revocation tests, limiting it to a >> sizelimit of 1. The count changes every time the suite runs so this is >> the safest for now. It also means that one test will fail if this is the >> only part of the suite executed. > > This gets rid of most of the failures, but it still fails the "certs for > this IPA server/short name" tests if the cert from ./make-cert is > present (creating it is part of `make test`). > If make-cert is run more times, it'll revoke the previous cert, so the > test for revocation reason 4 fails as well. > > It looks like when using sizelimit Dogtag will always discard *newer* > certs, ones with higher serials. Is it documented behavior or does > Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns > I wonder how other people run their tests. This solution looks like it > could break easily if people do something differently :( > > I'm not sure how to solve this properly. Perhaps not using Declarative, > and checking "by hand" that the wanted certs are in the response and the > unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1085-3-cert-find.patch Type: text/x-patch Size: 26530 bytes Desc: not available URL: From dpal at redhat.com Fri Feb 15 21:54:12 2013 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 15 Feb 2013 16:54:12 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360850471.12328.172.camel@willson.li.ssimo.org> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> <1360850471.12328.172.camel@willson.li.ssimo.org> Message-ID: <511EAE84.3080104@redhat.com> On 02/14/2013 09:01 AM, Simo Sorce wrote: > On Wed, 2013-02-13 at 21:55 -0500, Dmitri Pal wrote: >> On 02/13/2013 09:48 PM, Nathan Kinder wrote: >>> On 02/13/2013 06:18 PM, Dmitri Pal wrote: >>>> On 02/13/2013 02:08 PM, Simo Sorce wrote: >>>>> On Wed, 2013-02-13 at 13:30 -0500, Rob Crittenden wrote: >>>>>> Simo Sorce wrote: >>>>>>> On Wed, 2013-02-13 at 12:59 -0500, John Dennis wrote: >>>>>>>> On 02/13/2013 12:53 PM, Simo Sorce wrote: >>>>>>>> >>>>>>>>> If we can solve the looping and potential deadlocking concerns I >>>>>>>>> think >>>>>>>>> we can avoid the json reply and let the framework do the actual >>>>>>>>> final >>>>>>>>> ldap add. >>>>>>>> Could you elaborate on your looping and deadlock concerns? I >>>>>>>> don't see >>>>>>>> where they would arise if what we're watching is entirely >>>>>>>> independent of >>>>>>>> our LDAP tree. >>>>>>> I do not understand what you are 'watching' ? >>>>>>> >>>>>>> Simo. >>>>>>> >>>>>> I think he means have a persistent search to watch for new entries and >>>>>> then act upon them. >>>>> I think a 2 step approach is misguided, so I've written it off from the >>>>> start. >>>>> >>>>> Simo. >>>>> >>>> This all thread smells like rewriting winsync by using internal plugin >>>> and IPA framework. >>>> Is there any chance we can use existing winsync solution to do what we >>>> need but sync from staging instance rather than AD? >>>> Then the flow will be: >>>> HR system -> staging DS instance -> dirsync -> IPA >>>> Couple assumptions: >>>> a) We are satisfied with the security of the existing winsync solution >>>> and authentication used there. I do not see why it should be different >>>> here as it is a very similar use case. >>>> b) I expect we can sync from 389 to 389 with minor config changes and >>>> effectively no code changes >>> This is not the case. Winsync uses the AD Dirsync control, which 389 >>> does not support. The Winsync code also expects the AD schema on the >>> entries coming in from the Dirsync response. Even if 389 supported >>> the Dirsync control, there would be a fair amount of code changes to >>> deal with whatever schema we would need to expect. >>>> c) The users created via winsync now are created in a proper way so IPA >>>> accepts then as IPA users >>>> >>>> This solves the problem of add and delete of the users. >>>> I know that winsync already supports group add/delete but we never >>>> qualified it in IPA. So may be it is time, at least for this use case. >>>> I would really think that this might be a lower cost solution than >>>> writing yet another custom plugin. >>>> Let us try to reuse what we already have. >>> I think a custom plug-in would be easier (though I do have concerns >>> about the performance impact of the JSON approach). >>> >> I really think that we rather have an external solution and not the one >> hot wired into the IPA internals via the proposed plugin. >> Something like an external proxy or gateway rather than the approach >> Simo proposed even if it would be more work. This work would be reusable. >> We already have several use cases for the LDAP proxy. This can be one >> more. A bit more and there will be enough need to build one anyways. >> The main value is that it is optional and external and acts as an >> ecosystem level solution that can be developed and tested on a separate >> schedule rather than being internal to IPA. > Penrose. > > Simo. > Neah... Count this plant dead. -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From dpal at redhat.com Fri Feb 15 22:28:32 2013 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 15 Feb 2013 17:28:32 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511D1FEC.3090003@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> <511CE60B.5060106@redhat.com> <1360850800.12328.175.camel@willson.li.ssimo.org> <511D1FEC.3090003@redhat.com> Message-ID: <511EB690.5030209@redhat.com> On 02/14/2013 12:33 PM, Endi Sukma Dewata wrote: > On 2/14/2013 8:06 AM, Simo Sorce wrote: >> On Thu, 2013-02-14 at 14:26 +0100, Petr Spacek wrote: >> >>> In my Fedora 17 I found package python-ldaptor. It seems to offer >>> nice support >>> for writing own event-based LDAP servers. For simple LDAP proxy it >>> could be >>> enough. >>> >>> $ yum install python-ldaptor >>> $ python >>> import ldaptor.protocols.ldap.ldapserver >>> help(ldaptor.protocols.ldap.ldapserver) >> >> No. >> LDAP proxies are *not* simple. >> >> Ask Endi he's worked on a meta-directory for years. >> >> Simo. > > It depends on what you want to do with the proxy. If it's only a thin > layer which converts the LDAP ADD to IPA user-add it might not be that > complicated. > > Penrose virtual directory consists of a frontend LDAP interface, a > transformation engine, and backends which may include an LDAP server > as well. The front-end LDAP interface is the proxy we're talking about > here, it's only used to receive LDAP requests and pass them to the > transformation engine. > > The transformation engine is where the complexity occurs. In IPA this > is already handled by the framework. In Penrose it's quite complex > because it aims to provide a generic way to map an LDAP request to > multiple backends which involves dealing with different types of > backends, joining the backends, transforming the DN & attributes back > and forth, etc. > > So I'd say implementing an LDAP frontend for IPA using Python is > something worth exploring. That way it can run in the same process so > there's no concern about JSON performance/stability. > +1 I would rather go this way. Simo, Your point about how things should have been bight be righteous but Petr confusion about the role framework shows the reality. Currently all framework is developed as if it is the primary interface for creation/modification. LDAP is exposed but can be consumed by external clients as read only. This is how we decided to implement things. I clearly remember a lengthy discussion about this 4 years ago when the framework architecture was discussed. The main reason is plugability. Since we need to provide extensible plugability framework it is the only supportable option. For good or for bad it is how things are now and Petr is correct because this is what was decided and this is what he was told. It might have been a wrong decision (the interaction with the DS team was not at the same level as it is now) but under circumstances it was the one we made. And we now need to live with it. I do not think we have time to reevaluate things and move them into DS plugins is we find the framework abusing its power. I generally disagree with the notion that we can and should view add-user as the only special operation. Because of the framework extensibility any operation can be special. So I think that external LDAP proxy makes more sense. It is a bit more complex than what you propose but has several significant benefits: * We need LDAP proxy for other purposes so there will be a reuse of code and framework * We already have expertise * It allows us to keep what we already have done without any conceptual changes and continue with the approach we have been developing framework for last 4 years * It can be developed independently and incrementally without affecting the core IPA and its release schedule * It does not require and knowledge of the internals of IPA framework or DS plugins and can be developed by an external contributor I agree that virtual entries and staging areas are not the right approach. I also do not think that penrose at least in its current incarnation is the right approach either. Leveraging a python based LDAP proxy sounds like a good starting point. With that I suggest to move this conversation from what to who and how. Do we have an RFE logged? -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From simo at redhat.com Fri Feb 15 22:59:58 2013 From: simo at redhat.com (Simo Sorce) Date: Fri, 15 Feb 2013 17:59:58 -0500 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <511EB690.5030209@redhat.com> References: <1360767224.12328.41.camel@willson.li.ssimo.org> <511BAD63.9050005@redhat.com> <1360773268.12328.81.camel@willson.li.ssimo.org> <511BD025.7000201@redhat.com> <1360778001.12328.126.camel@willson.li.ssimo.org> <511BD47E.3000507@redhat.com> <1360778690.12328.127.camel@willson.li.ssimo.org> <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> <511CE60B.5060106@redhat.com> <1360850800.12328.175.camel@willson.li.ssimo.org> <511D1FEC.3090003@redhat.com> <511EB690.5030209@redhat.com> Message-ID: <1360969198.12328.309.camel@willson.li.ssimo.org> On Fri, 2013-02-15 at 17:28 -0500, Dmitri Pal wrote: > On 02/14/2013 12:33 PM, Endi Sukma Dewata wrote: > > On 2/14/2013 8:06 AM, Simo Sorce wrote: > >> On Thu, 2013-02-14 at 14:26 +0100, Petr Spacek wrote: > >> > >>> In my Fedora 17 I found package python-ldaptor. It seems to offer > >>> nice support > >>> for writing own event-based LDAP servers. For simple LDAP proxy it > >>> could be > >>> enough. > >>> > >>> $ yum install python-ldaptor > >>> $ python > >>> import ldaptor.protocols.ldap.ldapserver > >>> help(ldaptor.protocols.ldap.ldapserver) > >> > >> No. > >> LDAP proxies are *not* simple. > >> > >> Ask Endi he's worked on a meta-directory for years. > >> > >> Simo. > > > > It depends on what you want to do with the proxy. If it's only a thin > > layer which converts the LDAP ADD to IPA user-add it might not be that > > complicated. > > > > Penrose virtual directory consists of a frontend LDAP interface, a > > transformation engine, and backends which may include an LDAP server > > as well. The front-end LDAP interface is the proxy we're talking about > > here, it's only used to receive LDAP requests and pass them to the > > transformation engine. > > > > The transformation engine is where the complexity occurs. In IPA this > > is already handled by the framework. In Penrose it's quite complex > > because it aims to provide a generic way to map an LDAP request to > > multiple backends which involves dealing with different types of > > backends, joining the backends, transforming the DN & attributes back > > and forth, etc. > > > > So I'd say implementing an LDAP frontend for IPA using Python is > > something worth exploring. That way it can run in the same process so > > there's no concern about JSON performance/stability. > > > +1 I would rather go this way. > > Simo, > > Your point about how things should have been bight be righteous but Petr > confusion about the role framework shows the reality. Currently all > framework is developed as if it is the primary interface for > creation/modification. LDAP is exposed but can be consumed by external > clients as read only. This is how we decided to implement things. I > clearly remember a lengthy discussion about this 4 years ago when the > framework architecture was discussed. The main reason is plugability. > Since we need to provide extensible plugability framework it is the only > supportable option. For good or for bad it is how things are now and > Petr is correct because this is what was decided and this is what he was > told. It might have been a wrong decision (the interaction with the DS > team was not at the same level as it is now) but under circumstances it > was the one we made. And we now need to live with it. I do not think we > have time to reevaluate things and move them into DS plugins is we find > the framework abusing its power. > > I generally disagree with the notion that we can and should view > add-user as the only special operation. Because of the framework > extensibility any operation can be special. I do not have time to explain why I do not agree with your analysis here. I just don't, I think I will re-raise this later. > So I think that external LDAP proxy makes more sense. It is a bit more > complex than what you propose but has several significant benefits: > * We need LDAP proxy for other purposes so there will be a reuse of code > and framework What other purposes ? > * We already have expertise > * It allows us to keep what we already have done without any conceptual > changes and continue with the approach we have been developing framework > for last 4 years I do not think there is any need to make substantial changes to the framework with either proposal, and both would cause similar changes anyway. > * It can be developed independently and incrementally without affecting > the core IPA and its release schedule Same for both approaches. > * It does not require and knowledge of the internals of IPA framework or > DS plugins and can be developed by an external contributor Same level of knowledge is required for both approaches. > I agree that virtual entries and staging areas are not the right > approach. I also do not think that penrose at least in its current > incarnation is the right approach either. > Leveraging a python based LDAP proxy sounds like a good starting point. I think it is a bad idea. > With that I suggest to move this conversation from what to who and how. > Do we have an RFE logged? I suggest we do nothing, I think there is a substantial under-estimation of what it means to introduce an LDAP proxy on top of the framework, or on the side, or whatever, it is not even clear at what level this proxy would operate and with what semantics, and it is a substantially larger project than the plugin I proposed. It would divert substantial resources, and take a long time to be made useful. NAK. Simo. -- Simo Sorce * Red Hat, Inc * New York From ondrej at hamada.cz Fri Feb 15 23:39:46 2013 From: ondrej at hamada.cz (Ondrej Hamada) Date: Sat, 16 Feb 2013 00:39:46 +0100 Subject: [Freeipa-devel] More types of replicas in FreeIPA In-Reply-To: <1360782621.12328.137.camel@willson.li.ssimo.org> References: <4FC8CF97.8000202@redhat.com> <4FCE003A.800@redhat.com> <1338901087.8230.258.camel@willson.li.ssimo.org> <4FCE08D6.4080005@redhat.com> <1338905200.8230.270.camel@willson.li.ssimo.org> <4FCE1CA7.6010308@redhat.com> <1338909050.8230.278.camel@willson.li.ssimo.org> <4FCE3B63.1040806@redhat.com> <4FCE3DB1.7020002@redhat.com> <1338918705.8230.280.camel@willson.li.ssimo.org> <4FCF54A2.8080706@redhat.com> <1338990180.8230.304.camel@willson.li.ssimo.org> <50127FEC.8020908@redhat.com> <1343394730.2666.31.camel@willson.li.ssimo.org> <50167111.9090605@redhat.com> <510AF9BF.9060901@hamada.cz> <510DC28F.1000702@redhat.com> <51194E9E.1080909@hamada.cz> <1360614094.2299.110.camel@willson.li.ssimo.org> <51199B3F.2080808@redhat.com> <1360675254.3838.8.camel@willson.li.ssimo.org> <511ADE8C.2000008@redhat.com> <1360762565.12328.24.camel@willson.li.ssimo.org> <511BDC98.5040305@hamada.cz> <1360782621.12328.137.camel@willson.li.ssimo.org> Message-ID: <511EC742.8090104@hamada.cz> Dne 13.2.2013 20:10, Simo Sorce napsal(a): > On Wed, 2013-02-13 at 19:34 +0100, Ondrej Hamada wrote: >> Dne 13.2.2013 14:36, Simo Sorce napsal(a): >>> On Tue, 2013-02-12 at 19:30 -0500, Dmitri Pal wrote: >>> >>>> It looks like thinks are starting to boil down to building a Kerberos proxy. >>>> Is this something that fits within your thesis agenda Ondra? >>> I guess that's for Ondrej to say, if it is too much we can simply start >>> working on the LDAP/replication side with rekeying and what not, and >>> deal with the KDC part at a later time. >>> >>> Simo. >>> >> Working on the LDAP/repl side fits the thesis agenda better, so I would >> like to go that way. >> >> Rekeying - do you mean some sort of plugin for transporting the krb keys >> from masters to consumers? >> >> Besides securing transport of keys what else should be done in ldap? >> I've only partial replication in my mind - I mean replication of entries >> selected by some kind of ldap filters. > We would need to re-encrypt keys so that we do not need to hand off to > remote KDCs the same master key. > This way a compromise in a branch office replica would not compromise > the central infrastructure, but only affect the remote branch. > > Simo. > Just a small summary from IRC chat: Master servers must posses all RO replicas master keys in order to able to re-encrypt the keys. Probably replica long-term key (or any other key known only to replica) could be used. If the hubs are used, then they must also posses the RO replicas keys. Under such circumstances the compromised hub is almost the same threat as compromised master. Additionally the hub does not bring much functionality we can not supply by master server. So it seems as a better idea to don't create special hub replicas and use the master server instead (just don't expose it to clients). As a next step I'm going to look at the ipa password plugin for dirsrv - do you think it is a good source of inspiration for the rekeying plugin? From sbose at redhat.com Mon Feb 18 09:29:35 2013 From: sbose at redhat.com (Sumit Bose) Date: Mon, 18 Feb 2013 10:29:35 +0100 Subject: [Freeipa-devel] [RFC] Creating a new plugin to make it simpler to add users via LDAP In-Reply-To: <1360969198.12328.309.camel@willson.li.ssimo.org> References: <511BDBC0.8070805@redhat.com> <1360782530.12328.135.camel@willson.li.ssimo.org> <511C4968.5060606@redhat.com> <511C5072.80100@redhat.com> <511C521C.1070807@redhat.com> <511CE60B.5060106@redhat.com> <1360850800.12328.175.camel@willson.li.ssimo.org> <511D1FEC.3090003@redhat.com> <511EB690.5030209@redhat.com> <1360969198.12328.309.camel@willson.li.ssimo.org> Message-ID: <20130218092935.GA27418@localhost.localdomain> On Fri, Feb 15, 2013 at 05:59:58PM -0500, Simo Sorce wrote: > On Fri, 2013-02-15 at 17:28 -0500, Dmitri Pal wrote: > > On 02/14/2013 12:33 PM, Endi Sukma Dewata wrote: > > > On 2/14/2013 8:06 AM, Simo Sorce wrote: > > >> On Thu, 2013-02-14 at 14:26 +0100, Petr Spacek wrote: > > >> > > >>> In my Fedora 17 I found package python-ldaptor. It seems to offer > > >>> nice support > > >>> for writing own event-based LDAP servers. For simple LDAP proxy it > > >>> could be > > >>> enough. > > >>> > > >>> $ yum install python-ldaptor > > >>> $ python > > >>> import ldaptor.protocols.ldap.ldapserver > > >>> help(ldaptor.protocols.ldap.ldapserver) > > >> > > >> No. > > >> LDAP proxies are *not* simple. > > >> > > >> Ask Endi he's worked on a meta-directory for years. > > >> > > >> Simo. > > > > > > It depends on what you want to do with the proxy. If it's only a thin > > > layer which converts the LDAP ADD to IPA user-add it might not be that > > > complicated. > > > > > > Penrose virtual directory consists of a frontend LDAP interface, a > > > transformation engine, and backends which may include an LDAP server > > > as well. The front-end LDAP interface is the proxy we're talking about > > > here, it's only used to receive LDAP requests and pass them to the > > > transformation engine. > > > > > > The transformation engine is where the complexity occurs. In IPA this > > > is already handled by the framework. In Penrose it's quite complex > > > because it aims to provide a generic way to map an LDAP request to > > > multiple backends which involves dealing with different types of > > > backends, joining the backends, transforming the DN & attributes back > > > and forth, etc. > > > > > > So I'd say implementing an LDAP frontend for IPA using Python is > > > something worth exploring. That way it can run in the same process so > > > there's no concern about JSON performance/stability. > > > > > +1 I would rather go this way. > > > > Simo, > > > > Your point about how things should have been bight be righteous but Petr > > confusion about the role framework shows the reality. Currently all > > framework is developed as if it is the primary interface for > > creation/modification. LDAP is exposed but can be consumed by external > > clients as read only. This is how we decided to implement things. I > > clearly remember a lengthy discussion about this 4 years ago when the > > framework architecture was discussed. The main reason is plugability. > > Since we need to provide extensible plugability framework it is the only > > supportable option. For good or for bad it is how things are now and > > Petr is correct because this is what was decided and this is what he was > > told. It might have been a wrong decision (the interaction with the DS > > team was not at the same level as it is now) but under circumstances it > > was the one we made. And we now need to live with it. I do not think we > > have time to reevaluate things and move them into DS plugins is we find > > the framework abusing its power. > > > > I generally disagree with the notion that we can and should view > > add-user as the only special operation. Because of the framework > > extensibility any operation can be special. > > I do not have time to explain why I do not agree with your analysis > here. I just don't, I think I will re-raise this later. > > > So I think that external LDAP proxy makes more sense. It is a bit more > > complex than what you propose but has several significant benefits: > > * We need LDAP proxy for other purposes so there will be a reuse of code > > and framework > > What other purposes ? > > > * We already have expertise > > * It allows us to keep what we already have done without any conceptual > > changes and continue with the approach we have been developing framework > > for last 4 years > > I do not think there is any need to make substantial changes to the > framework with either proposal, and both would cause similar changes > anyway. > > > * It can be developed independently and incrementally without affecting > > the core IPA and its release schedule > > Same for both approaches. > > > * It does not require and knowledge of the internals of IPA framework or > > DS plugins and can be developed by an external contributor > > Same level of knowledge is required for both approaches. > > > I agree that virtual entries and staging areas are not the right > > approach. I also do not think that penrose at least in its current > > incarnation is the right approach either. > > Leveraging a python based LDAP proxy sounds like a good starting point. > > I think it is a bad idea. I agree with Simo here, besides other just two important aspects here. First, since all LDAP request will go through the proxy it needs a good security audit if a new project is used here, if 389ds or OpenLDAP are used here this would be different because they already have some history in this respect. Second, the proxy must support Kerberos/GSSAPI for authentication and must the able to delegate the credentials to the following processes. > > > With that I suggest to move this conversation from what to who and how. > > Do we have an RFE logged? > > I suggest we do nothing, I think there is a substantial under-estimation > of what it means to introduce an LDAP proxy on top of the framework, or > on the side, or whatever, it is not even clear at what level this proxy > would operate and with what semantics, and it is a substantially larger > project than the plugin I proposed. It would divert substantial > resources, and take a long time to be made useful. Although I like the proxy idea in general, I agree. A general purposes proxy as it was suggested is a major research project and adding and modifying users, groups, host etc with plain LDAP operations (i.e. something like the FreeIPA API via LDAP) might be the key feature for FreeIPA v4 or v5, if we really want it. bye, Sumit > > NAK. > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From pviktori at redhat.com Mon Feb 18 11:18:22 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 18 Feb 2013 12:18:22 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <511EAC0D.1020905@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> <511B5B25.7040607@redhat.com> <511EAC0D.1020905@redhat.com> Message-ID: <51220DFE.3040209@redhat.com> On 02/15/2013 10:43 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 02/06/2013 07:23 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>> This adds a cert-find command for the dogtag backend. >>>>> >>>>> Searches can be done by serial number, by subject, revocation reason, >>>>> issue date, notbefore, notafter and revocation dates. >>>>> >>>>> I added some basic tests for this. I made it a separate test file >>>>> because the cert plugin tests do not use the declarative format and >>>>> rely >>>>> on the selfsign backend by default. >>>>> >>>>> rob >>>> >>>> Thanks! The code works well, but I found a few issues. >>>> >>>> >>>> These tests don't work when the full test suite is run: test_cert adds >>>> and revokes additional certs that throw the code off. >>>> Perhaps have the tests only query valid certs? I don't see that option >>>> but I think it would be helpful to support. >>> >>> I added some rather nasty hacks to the test to make things pass. I limit >>> the search to 10 certificates, which is the number start with by >>> default. There is an open dogtag ticket to return only VALID >>> certificates so we should be able to drop this eventually. >>> >>> I had to go further on one of the revocation tests, limiting it to a >>> sizelimit of 1. The count changes every time the suite runs so this is >>> the safest for now. It also means that one test will fail if this is the >>> only part of the suite executed. >> >> This gets rid of most of the failures, but it still fails the "certs for >> this IPA server/short name" tests if the cert from ./make-cert is >> present (creating it is part of `make test`). >> If make-cert is run more times, it'll revoke the previous cert, so the >> test for revocation reason 4 fails as well. >> >> It looks like when using sizelimit Dogtag will always discard *newer* >> certs, ones with higher serials. Is it documented behavior or does >> Dogtag just happen to do that? > > It isn't documented anywhere I could find, it is just what dogtag returns > >> I wonder how other people run their tests. This solution looks like it >> could break easily if people do something differently :( >> >> I'm not sure how to solve this properly. Perhaps not using Declarative, >> and checking "by hand" that the wanted certs are in the response and the >> unwanted ones are not, would work better. > > I ended up switching the test class. It is not a very fine-grained set > of tests, mostly searching with limits and verifying that we fall within > a reasonable range, but it is better than nothing. > > rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py -- Petr? From pspacek at redhat.com Mon Feb 18 11:27:35 2013 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 18 Feb 2013 12:27:35 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <511E44A9.9090509@redhat.com> References: <511E44A9.9090509@redhat.com> Message-ID: <51221027.7050803@redhat.com> On 15.2.2013 15:22, Ana Krivokapic wrote: > Hello, > > The .isalpha() check in validate_domain_name() was too strict, > causing some commands like ipa dnsrecord-add to fail. > > https://fedorahosted.org/freeipa/ticket/3385 I would add --force option rather than removing whole check, if it's possible. Would it be possible to mention RFC in the error message? Something like _('top level domain label must be alphabetic (RFC 1123 section 2.1)') ? IMHO it is handy, because it educates users. -- Petr^2 Spacek From abokovoy at redhat.com Mon Feb 18 11:36:40 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 18 Feb 2013 13:36:40 +0200 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <511E1C2D.9080701@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> <20130212170013.GB476@redhat.com> <511CCC0E.5060409@redhat.com> <20130214163719.GG476@redhat.com> <511E1C2D.9080701@redhat.com> Message-ID: <20130218113640.GI476@redhat.com> On Fri, 15 Feb 2013, Tomas Babej wrote: >On 02/14/2013 05:37 PM, Alexander Bokovoy wrote: >>On Thu, 14 Feb 2013, Tomas Babej wrote: >>>>>>>+ Str('ipanttrusteddomainname?', >>>>>>>+ cli_name='dom_name', >>>>>>>+ flags=('no_search', 'virtual_attribute'), >>>>>>>+ label=_('Name of the trusted domain'), >>>>>>>+ ), >>>>>>New options is added but API.txt wasn't changed. As result, >>>>>>'make rpms' >>>>>>does not work. >>>>>> >>>>>>Could you please fix the patch and re-send it? >>>>>> >>>>>Sorry about that. >>>>> >>>>>Updated patch attached. >>>>I have one small question regarding use of dom_sid/dom_name. >>>> >>>>If both dom_sid and dom_name were specified, failing to resolve >>>>dom_name >>>>would force command to raise exception. >>>> >>>>I'm not sure this is right behavior. Probably we should detect >>>>that both >>>>dom_sid and dom_name were specified and bail out earlier so that only >>>>one of them is accepted. That would be clearer to users, wouldn't it >>>Sure, I definitely agree on that point. I added the check to >>>idrange-add and >>>idrange-mod. Also, the patch needed a rebase to apply on the >>>current master. >>I tried to play with various scenarious and one thing I noticed is >>that we refer dom_sid and dom_name in the error messages as they >>are >>named internally. CLI replaces underscore by hyphen (_ -> -) and >>therefore >>this error message becomes wrong -- you cannot specify --dom_sid, this >>option is unknown to CLI. >> >>In Web UI this would also mean an error message pointing to non-existing >>option. Perhaps it would be reasonable to name options '--name' and >>'--sid'? We don't have any clash there: >>------------------------------------------------------------------------- >># ipa idrange-mod --help >>Usage: ipa [global-options] idrange-mod NAME [options] >> >>Positional arguments: >> NAME Range name >> >>Options: >> -h, --help show this help message and exit >> --base-id=INT First Posix ID of the range >> --range-size=INT Number of IDs in the range >> --rid-base=INT First RID of the corresponding RID range >> --secondary-rid-base=INT >> First RID of the secondary RID range >> --dom-sid=STR Domain SID of the trusted domain >> --dom-name=STR Name of the trusted domain >> --setattr=STR Set an attribute to a name/value pair. Format is >> attr=value. For multi-valued attributes, >>the command >> replaces the values already present. >> --addattr=STR Add an attribute/value pair. Format is >>attr=value. The >> attribute must be part of the schema. >> --delattr=STR Delete an attribute/value pair. The option willbe >> evaluated last, after all sets and adds. >> --rights Display the access rights of this entry(requires >> --all). See ipa man page for details. >> --all Retrieve and print all attributes from the >>server. >> Affects command output. >> --raw Print entries as stored on the server. Only >>affects >> output format. >>------------------------------------------------------------------------- >> >>So, if --name and --sid were used, an error message would become >>---------------------------------------------------------------------- >># ipa idrange-mod AD.LAN_id_range --dom-name foo.bar ipa: ERROR: >>invalid 'ID Range setup': SID for the specified trusted >>domain name could not be found. Please specify the SID directly using >>--sid option. >>---------------------------------------------------------------------- >> >> >>Additionally, there is an error when SID for an object within the domain >>is specified. Last RID of the SID represents an object within the domain >>and we generally need to be careful allowing it in the place where >>domain SID is specified: >> >># ipa idrange-mod AD.LAN_id_range --dom-sid >>S-1-5-21-3502988750-125904550-3683905862-1 >>----------------------------------- >>Modified ID range "AD.LAN_id_range" >>----------------------------------- >> Range name: AD.LAN_id_range >> First Posix ID of the range: 1442800000 >> Number of IDs in the range: 200000 >> First RID of the corresponding RID range: 0 >> Domain SID of the trusted domain: >>S-1-5-21-3502988750-125904550-3683905862-1 >> Range type: Active Directory domain range >> >>Now this range is completely unusable due to the fact that there is no >>way to match the domain SID against the range. >> >>I think we need to make the check against established trusts more >>strict and only allow exact match. >> >1.) Regarding dom_sid and dom_name options, we do not have to change >their internal names >to get more user-friendly error messages. These are hardcoded >strings, and not generated from >internal names of the options. I followed the naming convention >already set in the file, but you're right, >the current state is somewhat confusing to the end user. I changed >all the error messages so that >they refer to hyphen-versions of the options (not only >dom_sid/dom_name but also rid_base, etc.). Ok, thanks. >I considered renaming the options to --sid and --name. However, >although --sid is pretty self-explanatory, >--name could be quite confusing, as ID range has name of its own. >Furthermore, this would break some >documentation / other references, and since refactoring of the error >messages described above solves our >issue here, I'd vote for that solution. Agreed. >2.) Exact match against estabilished trusts - this is a nice catch! >However, as far as I understand this is >not something introduced by my patch, and it would not be transparent >to include the fix here. If you agree, >I'll create a new ticket in Trac. Please do. >Updated patch attached (error messages refactored). ACK. -- / Alexander Bokovoy From sbose at redhat.com Mon Feb 18 11:47:50 2013 From: sbose at redhat.com (Sumit Bose) Date: Mon, 18 Feb 2013 12:47:50 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <51221027.7050803@redhat.com> References: <511E44A9.9090509@redhat.com> <51221027.7050803@redhat.com> Message-ID: <20130218114750.GC27418@localhost.localdomain> On Mon, Feb 18, 2013 at 12:27:35PM +0100, Petr Spacek wrote: > On 15.2.2013 15:22, Ana Krivokapic wrote: > >Hello, > > > >The .isalpha() check in validate_domain_name() was too strict, > >causing some commands like ipa dnsrecord-add to fail. > > > >https://fedorahosted.org/freeipa/ticket/3385 > > I would add --force option rather than removing whole check, if it's possible. > > Would it be possible to mention RFC in the error message? Something > like _('top level domain label must be alphabetic (RFC 1123 section > 2.1)') > ? > > IMHO it is handy, because it educates users. The problem is that this check is always done on the last component of the domain_name even if it is just a sub-domain of the FreeIPA domain, where e.g. numbers are valid characters. At the beginning of validate_domain_name() a trailing '.' is stripped away. iirc the trailing '.' is an indication for a complete, fully qualified name. Would it work if the presence of the trailing '.' is saved and the check is only done if there was a '.'? bye, Sumit > > -- > Petr^2 Spacek > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From tbabej at redhat.com Mon Feb 18 11:58:58 2013 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 18 Feb 2013 12:58:58 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <20130218113640.GI476@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> <20130212170013.GB476@redhat.com> <511CCC0E.5060409@redhat.com> <20130214163719.GG476@redhat.com> <511E1C2D.9080701@redhat.com> <20130218113640.GI476@redhat.com> Message-ID: <51221782.1080405@redhat.com> On 02/18/2013 12:36 PM, Alexander Bokovoy wrote: > On Fri, 15 Feb 2013, Tomas Babej wrote: >> On 02/14/2013 05:37 PM, Alexander Bokovoy wrote: >>> On Thu, 14 Feb 2013, Tomas Babej wrote: >>>>>>>> + Str('ipanttrusteddomainname?', >>>>>>>> + cli_name='dom_name', >>>>>>>> + flags=('no_search', 'virtual_attribute'), >>>>>>>> + label=_('Name of the trusted domain'), >>>>>>>> + ), >>>>>>> New options is added but API.txt wasn't changed. As result, >>>>>>> 'make rpms' >>>>>>> does not work. >>>>>>> >>>>>>> Could you please fix the patch and re-send it? >>>>>>> >>>>>> Sorry about that. >>>>>> >>>>>> Updated patch attached. >>>>> I have one small question regarding use of dom_sid/dom_name. >>>>> >>>>> If both dom_sid and dom_name were specified, failing to resolve >>>>> dom_name >>>>> would force command to raise exception. >>>>> >>>>> I'm not sure this is right behavior. Probably we should detect >>>>> that both >>>>> dom_sid and dom_name were specified and bail out earlier so that only >>>>> one of them is accepted. That would be clearer to users, wouldn't it >>>> Sure, I definitely agree on that point. I added the check to >>>> idrange-add and >>>> idrange-mod. Also, the patch needed a rebase to apply on the >>>> current master. >>> I tried to play with various scenarious and one thing I noticed is >>> that we refer dom_sid and dom_name in the error messages as they are >>> named internally. CLI replaces underscore by hyphen (_ -> -) and >>> therefore >>> this error message becomes wrong -- you cannot specify --dom_sid, this >>> option is unknown to CLI. >>> >>> In Web UI this would also mean an error message pointing to >>> non-existing >>> option. Perhaps it would be reasonable to name options '--name' and >>> '--sid'? We don't have any clash there: >>> ------------------------------------------------------------------------- >>> >>> # ipa idrange-mod --help >>> Usage: ipa [global-options] idrange-mod NAME [options] >>> >>> Positional arguments: >>> NAME Range name >>> >>> Options: >>> -h, --help show this help message and exit >>> --base-id=INT First Posix ID of the range >>> --range-size=INT Number of IDs in the range >>> --rid-base=INT First RID of the corresponding RID range >>> --secondary-rid-base=INT >>> First RID of the secondary RID range >>> --dom-sid=STR Domain SID of the trusted domain >>> --dom-name=STR Name of the trusted domain >>> --setattr=STR Set an attribute to a name/value pair. Format is >>> attr=value. For multi-valued attributes, the >>> command >>> replaces the values already present. >>> --addattr=STR Add an attribute/value pair. Format is >>> attr=value. The >>> attribute must be part of the schema. >>> --delattr=STR Delete an attribute/value pair. The option >>> willbe >>> evaluated last, after all sets and adds. >>> --rights Display the access rights of this entry(requires >>> --all). See ipa man page for details. >>> --all Retrieve and print all attributes from the >>> server. >>> Affects command output. >>> --raw Print entries as stored on the server. Only >>> affects >>> output format. >>> ------------------------------------------------------------------------- >>> >>> >>> So, if --name and --sid were used, an error message would become >>> ---------------------------------------------------------------------- >>> # ipa idrange-mod AD.LAN_id_range --dom-name foo.bar ipa: ERROR: >>> invalid 'ID Range setup': SID for the specified trusted >>> domain name could not be found. Please specify the SID directly using >>> --sid option. >>> ---------------------------------------------------------------------- >>> >>> >>> Additionally, there is an error when SID for an object within the >>> domain >>> is specified. Last RID of the SID represents an object within the >>> domain >>> and we generally need to be careful allowing it in the place where >>> domain SID is specified: >>> >>> # ipa idrange-mod AD.LAN_id_range --dom-sid >>> S-1-5-21-3502988750-125904550-3683905862-1 >>> ----------------------------------- >>> Modified ID range "AD.LAN_id_range" >>> ----------------------------------- >>> Range name: AD.LAN_id_range >>> First Posix ID of the range: 1442800000 >>> Number of IDs in the range: 200000 >>> First RID of the corresponding RID range: 0 >>> Domain SID of the trusted domain: >>> S-1-5-21-3502988750-125904550-3683905862-1 >>> Range type: Active Directory domain range >>> >>> Now this range is completely unusable due to the fact that there is no >>> way to match the domain SID against the range. >>> >>> I think we need to make the check against established trusts more >>> strict and only allow exact match. >>> >> 1.) Regarding dom_sid and dom_name options, we do not have to change >> their internal names >> to get more user-friendly error messages. These are hardcoded >> strings, and not generated from >> internal names of the options. I followed the naming convention >> already set in the file, but you're right, >> the current state is somewhat confusing to the end user. I changed >> all the error messages so that >> they refer to hyphen-versions of the options (not only >> dom_sid/dom_name but also rid_base, etc.). > Ok, thanks. > >> I considered renaming the options to --sid and --name. However, >> although --sid is pretty self-explanatory, >> --name could be quite confusing, as ID range has name of its own. >> Furthermore, this would break some >> documentation / other references, and since refactoring of the error >> messages described above solves our >> issue here, I'd vote for that solution. > Agreed. > >> 2.) Exact match against estabilished trusts - this is a nice catch! >> However, as far as I understand this is >> not something introduced by my patch, and it would not be transparent >> to include the fix here. If you agree, >> I'll create a new ticket in Trac. > Please do. For the record: https://fedorahosted.org/freeipa/ticket/3432 > > >> Updated patch attached (error messages refactored). > ACK. > Tomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkosek at redhat.com Mon Feb 18 12:08:55 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 18 Feb 2013 13:08:55 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <20130218114750.GC27418@localhost.localdomain> References: <511E44A9.9090509@redhat.com> <51221027.7050803@redhat.com> <20130218114750.GC27418@localhost.localdomain> Message-ID: <512219D7.9000909@redhat.com> On 02/18/2013 12:47 PM, Sumit Bose wrote: > On Mon, Feb 18, 2013 at 12:27:35PM +0100, Petr Spacek wrote: >> On 15.2.2013 15:22, Ana Krivokapic wrote: >>> Hello, >>> >>> The .isalpha() check in validate_domain_name() was too strict, >>> causing some commands like ipa dnsrecord-add to fail. >>> >>> https://fedorahosted.org/freeipa/ticket/3385 >> >> I would add --force option rather than removing whole check, if it's possible. >> >> Would it be possible to mention RFC in the error message? Something >> like _('top level domain label must be alphabetic (RFC 1123 section >> 2.1)') >> ? >> >> IMHO it is handy, because it educates users. > > The problem is that this check is always done on the last component of > the domain_name even if it is just a sub-domain of the FreeIPA domain, > where e.g. numbers are valid characters. > > At the beginning of validate_domain_name() a trailing '.' is stripped > away. iirc the trailing '.' is an indication for a complete, fully > qualified name. Would it work if the presence of the trailing '.' is > saved and the check is only done if there was a '.'? > > bye, > Sumit > Sure. Though I am now not 100% sure that some IPA functions do not use this validator with a fqdn hostname without trailing dot. If not, I am for fixing this function as Sumit and Petr suggested. Martin From pviktori at redhat.com Mon Feb 18 12:44:03 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 18 Feb 2013 13:44:03 +0100 Subject: [Freeipa-devel] [PATCH] 0180 Check SSH connection in ipa-replica-conncheck In-Reply-To: <511E8A08.2050002@redhat.com> References: <511E5120.6050604@redhat.com> <511E567B.3070307@redhat.com> <511E650D.3050100@redhat.com> <511E8A08.2050002@redhat.com> Message-ID: <51222213.7080204@redhat.com> On 02/15/2013 08:18 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 02/15/2013 04:38 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> ipa-replica-conncheck ran SSH in quiet mode, probably to suppress a >>>> message about connecting to an unknown host. This made it hard to debug >>>> connection errors. >>>> >>>> I didn't find a way to separate SSH output from the output of the >>>> called >>>> command, I decided to try an additional SSH connection before calling >>>> conncheck. SSH is set to verbose and if it fails the errors get printed >>>> out. Also, the host is added to a temporary known_hosts file. >>>> The second SSH is called without the -q flag so errors/warnings are not >>>> lost even if the command fails. The temporary known_hosts file is used >>>> so the unknown host warning doesn't appear here. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3402 >>> >>> The general procedure looks good, I don't think we should hardcode the >>> path to ssh. ipautil.run() overrides the current environment so we >>> should be able to safely run just 'ssh'. >>> >>> We eventually need a cross-platform way of locating system binaries. >>> >>> The hardcoded path to ipa-replica-conncheck is probably ok since we >>> provide that binary ourselves. >>> >>> rob >> >> Changed, thanks. >> > > Looks and works well. I just have one final question. Should remote_addr > and temp_known_hosts be passed in as args? They are basically globals > but it is obvious where they came from, so not really a NAK, just a > question. > > rob Well, it's a style issue so I only have a bunch of rule-of-thumb handwaving to justify the decision... Here goes: Closing over the variables in main() doesn't have drawbacks of globals: nothing outside main() can see/modify them and it doesn't keep main() from (indirectly) calling itself (this particular main() probably won't need to do that, but in general it's good to avoid this kind of surprises). Passing the variables in as arguments would only make sense if run_ssh was a top-level function, otherwise you have to worry about the right order/names of arguments and gain nothing. The nested function makes it clear that this is just a quick way to keep things DRY, not a full-featured generic SSH caller. A top-level "run_ssh" helper shouldn't unconditionally disable StrictHostKeychecking for example. The code is tied to the caller functionally, so I kept it tied syntactically as well. On the other hand, nested functions are "magic" that you need a good reason for. I think the above is just reason enough, but I don't feel that strongly about it. Here's a version with top-level run_ssh, push it if it seems better to you. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0180-02b-Check-SSH-connection-in-ipa-replica-conncheck.patch Type: text/x-patch Size: 3367 bytes Desc: not available URL: From pviktori at redhat.com Mon Feb 18 12:57:08 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 18 Feb 2013 13:57:08 +0100 Subject: [Freeipa-devel] [PATCH] Allow ipa-replica-conncheck and ipa-adtrust-install to read krb5 includedir In-Reply-To: <20130210212248.GV4506@redhat.com> References: <20130208174409.GJ3804@hendrix.brq.redhat.com> <20130210212248.GV4506@redhat.com> Message-ID: <51222524.3060401@redhat.com> On 02/10/2013 10:22 PM, Alexander Bokovoy wrote: > On Fri, 08 Feb 2013, Jakub Hrozek wrote: >> The krb5 includedir uses a different delimeter (space) than the rest >> of the krb5 >> config file (equal sign). But only the ipa-client-install and >> ipa-server-install scripts were set with the correct delimeters. This >> patch also adds the delimeters to ipa-adtrust-install and >> ipa-replica-conncheck. > ACK. Works for me on Fedora 18 with git master. > > I've also checked RHEL 6.4 beta but since both original fix and this one > are missing there, no changes were required to have ipa-adtrust-install > working with 'includedir' defined in /etc/krb5.conf. > This has been ACKed for a week, could someone with commit rights push it? -- Petr? From mkosek at redhat.com Mon Feb 18 15:48:11 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 18 Feb 2013 16:48:11 +0100 Subject: [Freeipa-devel] [PATCH 0030] Add option to specify SID using domain name to idrange-add/mod In-Reply-To: <20130218113640.GI476@redhat.com> References: <510FE077.8010801@redhat.com> <20130208142521.GS4506@redhat.com> <51151B27.8020103@redhat.com> <20130212170013.GB476@redhat.com> <511CCC0E.5060409@redhat.com> <20130214163719.GG476@redhat.com> <511E1C2D.9080701@redhat.com> <20130218113640.GI476@redhat.com> Message-ID: <51224D3B.5060607@redhat.com> On 02/18/2013 12:36 PM, Alexander Bokovoy wrote: > On Fri, 15 Feb 2013, Tomas Babej wrote: >> On 02/14/2013 05:37 PM, Alexander Bokovoy wrote: >>> On Thu, 14 Feb 2013, Tomas Babej wrote: >>>>>>>> + Str('ipanttrusteddomainname?', >>>>>>>> + cli_name='dom_name', >>>>>>>> + flags=('no_search', 'virtual_attribute'), >>>>>>>> + label=_('Name of the trusted domain'), >>>>>>>> + ), >>>>>>> New options is added but API.txt wasn't changed. As result, 'make rpms' >>>>>>> does not work. >>>>>>> >>>>>>> Could you please fix the patch and re-send it? >>>>>>> >>>>>> Sorry about that. >>>>>> >>>>>> Updated patch attached. >>>>> I have one small question regarding use of dom_sid/dom_name. >>>>> >>>>> If both dom_sid and dom_name were specified, failing to resolve dom_name >>>>> would force command to raise exception. >>>>> >>>>> I'm not sure this is right behavior. Probably we should detect that both >>>>> dom_sid and dom_name were specified and bail out earlier so that only >>>>> one of them is accepted. That would be clearer to users, wouldn't it >>>> Sure, I definitely agree on that point. I added the check to idrange-add and >>>> idrange-mod. Also, the patch needed a rebase to apply on the current master. >>> I tried to play with various scenarious and one thing I noticed is that we >>> refer dom_sid and dom_name in the error messages as they are >>> named internally. CLI replaces underscore by hyphen (_ -> -) and therefore >>> this error message becomes wrong -- you cannot specify --dom_sid, this >>> option is unknown to CLI. >>> >>> In Web UI this would also mean an error message pointing to non-existing >>> option. Perhaps it would be reasonable to name options '--name' and >>> '--sid'? We don't have any clash there: >>> ------------------------------------------------------------------------- >>> # ipa idrange-mod --help >>> Usage: ipa [global-options] idrange-mod NAME [options] >>> >>> Positional arguments: >>> NAME Range name >>> >>> Options: >>> -h, --help show this help message and exit >>> --base-id=INT First Posix ID of the range >>> --range-size=INT Number of IDs in the range >>> --rid-base=INT First RID of the corresponding RID range >>> --secondary-rid-base=INT >>> First RID of the secondary RID range >>> --dom-sid=STR Domain SID of the trusted domain >>> --dom-name=STR Name of the trusted domain >>> --setattr=STR Set an attribute to a name/value pair. Format is >>> attr=value. For multi-valued attributes, the command >>> replaces the values already present. >>> --addattr=STR Add an attribute/value pair. Format is >>> attr=value. The >>> attribute must be part of the schema. >>> --delattr=STR Delete an attribute/value pair. The option willbe >>> evaluated last, after all sets and adds. >>> --rights Display the access rights of this entry(requires >>> --all). See ipa man page for details. >>> --all Retrieve and print all attributes from the server. >>> Affects command output. >>> --raw Print entries as stored on the server. Only affects >>> output format. >>> ------------------------------------------------------------------------- >>> >>> So, if --name and --sid were used, an error message would become >>> ---------------------------------------------------------------------- >>> # ipa idrange-mod AD.LAN_id_range --dom-name foo.bar ipa: ERROR: invalid 'ID >>> Range setup': SID for the specified trusted >>> domain name could not be found. Please specify the SID directly using >>> --sid option. >>> ---------------------------------------------------------------------- >>> >>> >>> Additionally, there is an error when SID for an object within the domain >>> is specified. Last RID of the SID represents an object within the domain >>> and we generally need to be careful allowing it in the place where >>> domain SID is specified: >>> >>> # ipa idrange-mod AD.LAN_id_range --dom-sid >>> S-1-5-21-3502988750-125904550-3683905862-1 >>> ----------------------------------- >>> Modified ID range "AD.LAN_id_range" >>> ----------------------------------- >>> Range name: AD.LAN_id_range >>> First Posix ID of the range: 1442800000 >>> Number of IDs in the range: 200000 >>> First RID of the corresponding RID range: 0 >>> Domain SID of the trusted domain: S-1-5-21-3502988750-125904550-3683905862-1 >>> Range type: Active Directory domain range >>> >>> Now this range is completely unusable due to the fact that there is no >>> way to match the domain SID against the range. >>> >>> I think we need to make the check against established trusts more >>> strict and only allow exact match. >>> >> 1.) Regarding dom_sid and dom_name options, we do not have to change their >> internal names >> to get more user-friendly error messages. These are hardcoded strings, and >> not generated from >> internal names of the options. I followed the naming convention already set >> in the file, but you're right, >> the current state is somewhat confusing to the end user. I changed all the >> error messages so that >> they refer to hyphen-versions of the options (not only dom_sid/dom_name but >> also rid_base, etc.). > Ok, thanks. > >> I considered renaming the options to --sid and --name. However, although >> --sid is pretty self-explanatory, >> --name could be quite confusing, as ID range has name of its own. >> Furthermore, this would break some >> documentation / other references, and since refactoring of the error messages >> described above solves our >> issue here, I'd vote for that solution. > Agreed. > >> 2.) Exact match against estabilished trusts - this is a nice catch! However, >> as far as I understand this is >> not something introduced by my patch, and it would not be transparent to >> include the fix here. If you agree, >> I'll create a new ticket in Trac. > Please do. > > >> Updated patch attached (error messages refactored). > ACK. > Pushed to master, ipa-3-1 (with a mild rebase). Martin From mkosek at redhat.com Mon Feb 18 15:50:56 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 18 Feb 2013 16:50:56 +0100 Subject: [Freeipa-devel] [PATCH] Allow ipa-replica-conncheck and ipa-adtrust-install to read krb5 includedir In-Reply-To: <20130210212248.GV4506@redhat.com> References: <20130208174409.GJ3804@hendrix.brq.redhat.com> <20130210212248.GV4506@redhat.com> Message-ID: <51224DE0.2040902@redhat.com> On 02/10/2013 10:22 PM, Alexander Bokovoy wrote: > On Fri, 08 Feb 2013, Jakub Hrozek wrote: >> The krb5 includedir uses a different delimeter (space) than the rest of the krb5 >> config file (equal sign). But only the ipa-client-install and >> ipa-server-install scripts were set with the correct delimeters. This >> patch also adds the delimeters to ipa-adtrust-install and >> ipa-replica-conncheck. > ACK. Works for me on Fedora 18 with git master. > > I've also checked RHEL 6.4 beta but since both original fix and this one > are missing there, no changes were required to have ipa-adtrust-install > working with 'includedir' defined in /etc/krb5.conf. > Pushed to master, ipa-3-1. Martin From pvoborni at redhat.com Mon Feb 18 16:29:05 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 18 Feb 2013 17:29:05 +0100 Subject: [Freeipa-devel] [PATCH] 256-258 Web UI: removed build dependency errors Message-ID: <512256D1.8040101@redhat.com> First patch adds just a devel tool usable in browser. Second is foundation for specifying which dependency errors can be ignored in Web UI build. Third is an usage of second. These patches were develop as part of ongoing refactoring work but they are independent and beneficial for all UI work so I'm sending them now. 1) [PATCH] Develop.js extended Develop.js contains code useful only for debugging. It is not part of FreeIPA release. Is loaded by typing require(['freeipa/develop']); in browser JS console. It adds IPA global variable and provide easier way of loading AMD modules into window.ipadev[providedNameOrModuleName] variable. 2) [PATCH] Allow to specify modules for which builder doesn't raise dependency error One can specify module ids provided by other means (already built layer file) in providedMids array of build profile file's package section. Builder then ignores dependency errors for specified modules. This allows to build layers without source codes of their dependencies, with no expected errors raised. Example: packages:[ { name: "freeipa", location: "freeipa", providedMids: [ 'dojo/_base/declare', 'dojo/_base/lang', 'dojo/_base/array', 'dojo/Stateful' //etc ] } ], [PATCH] Web UI build profile updated freeipa.profile was updated accordingly to contain all modules in dojo layer. This change removes expected errors during the build and therefore it won't confuse others during rpm build. It also helps during development because developer will notice real dependency errors (those not specified this way). -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0256-Develop.js-extended.patch Type: text/x-patch Size: 2938 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0257-Allow-to-specify-modules-for-which-builder-doesn-t-r.patch Type: text/x-patch Size: 315559 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0258-Web-UI-build-profile-updated.patch Type: text/x-patch Size: 3403 bytes Desc: not available URL: From pvoborni at redhat.com Mon Feb 18 16:31:45 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 18 Feb 2013 17:31:45 +0100 Subject: [Freeipa-devel] [PATCH] 259 Combobox keyboard support Message-ID: <51225771.3040809@redhat.com> Combobox can be controlled just by using keyboard. When value list is closed, user can: * use UP and DOWN error to open list, it will focus the list and select previous/next value * when CB is non-editable, user can start typing, first character will open list, second will be entered into search input. Note: I wanted to copy the first char to the search box as well, but I did not figure out reliable method for converting keycode to char for non ASCII keyboard layouts * ESCAPE, ENTER, TAB keys are handled to allow keyboard operations in a container When value list is opened: * CB tries to keep focus on either search input or a select * when focus is lost, the value list is closed. So user can click anywhere on a page to close it - two comboboxes can't be opened on the same time * hitting TAB key switches between search and select * if CB is not searchable, hitting TAB will close the value list and select input textbox * hitting ESCAPE on will close the value list * hitting ENTER on search input will invoke search operation * hitting ENTER on select will close the value list * hitting UP/DOWN arrows will select previous/next values Additional modifications: * opening arrow and search button were made non-focusable. It fixes the 'wrong focus area' bug and simplifies keyboard usage. It doesn't affect mouse usage. https://fedorahosted.org/freeipa/ticket/3324 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0259-Combobox-keyboard-support.patch Type: text/x-patch Size: 12071 bytes Desc: not available URL: From pvoborni at redhat.com Mon Feb 18 16:36:09 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 18 Feb 2013 17:36:09 +0100 Subject: [Freeipa-devel] [PATCH] 260-262 Global trust configuration page Message-ID: <51225879.9020608@redhat.com> First two patches are bug fixes which are required for third patch. Depends on my patch #259 (Combobox keyboard support) 1) [PATCH] Fix dirty state update of editable combobox Editable combobox didn't update it's dirty state correctly. CB had it's own internal value changed event, which was incorrectly used. It was removed and widget's value_changed event was used instead. 2) [PATCH] Fix handling of no_update flag in Web UI There was an incorrect check for no_update flag. Check was performed as if the flag was an attribute of object not an item of array. Hence, the flag never caused any effect. 3) [PATCH] Global trust config page https://fedorahosted.org/freeipa/ticket/3333 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0260-Fix-dirty-state-update-of-editable-combobox.patch Type: text/x-patch Size: 3061 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0261-Fix-handling-of-no_update-flag-in-Web-UI.patch Type: text/x-patch Size: 1373 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0262-Global-trust-config-page.patch Type: text/x-patch Size: 5472 bytes Desc: not available URL: From pvoborni at redhat.com Mon Feb 18 16:37:29 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 18 Feb 2013 17:37:29 +0100 Subject: [Freeipa-devel] [PATCH] 263 Web UI: configurable SID blacklists Message-ID: <512258C9.5000803@redhat.com> Added blacklists section, with ipantsidblacklistincoming and ipantsidblacklistoutgoing multivalued textbox fields, into trust details page. https://fedorahosted.org/freeipa/ticket/3289 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0263-Web-UI-configurable-SID-blacklists.patch Type: text/x-patch Size: 3000 bytes Desc: not available URL: From pvoborni at redhat.com Mon Feb 18 16:52:06 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 18 Feb 2013 17:52:06 +0100 Subject: [Freeipa-devel] [PATCH] 260-262 Global trust configuration page In-Reply-To: <51225879.9020608@redhat.com> References: <51225879.9020608@redhat.com> Message-ID: <51225C36.1000406@redhat.com> Just two notes: ipantfallbackprimarygroup requires a posix group. Our API currently doesn't support search based on object classes therefore the entity select widget incorrectly offers non posix groups as well. Another problem is that hidden 'Default SMB Group' is not listed. Hence it couldn't be set again after a modification. I made the combobox editable (first usage, so it revealed a bug) to avoid this problem. User can enter garbage, but the framework should handle that. On 02/18/2013 05:36 PM, Petr Vobornik wrote: > First two patches are bug fixes which are required for third patch. > Depends on my patch #259 (Combobox keyboard support) > > 1) [PATCH] Fix dirty state update of editable combobox > > Editable combobox didn't update it's dirty state correctly. CB had it's own > internal value changed event, which was incorrectly used. It was removed > and > widget's value_changed event was used instead. > > 2) [PATCH] Fix handling of no_update flag in Web UI > > There was an incorrect check for no_update flag. Check was performed as > if the flag was an attribute of object not an item of array. Hence, the > flag never caused any effect. > > 3) [PATCH] Global trust config page > > https://fedorahosted.org/freeipa/ticket/3333 -- Petr Vobornik From rcritten at redhat.com Mon Feb 18 18:22:34 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 18 Feb 2013 13:22:34 -0500 Subject: [Freeipa-devel] [PATCHES] 0107-0114 Fix Confusing ipa tool online help organization In-Reply-To: <511CB661.7010300@redhat.com> References: <50C9F27E.50808@redhat.com> <50CA76D6.8090404@redhat.com> <50CB0DCF.7050102@redhat.com> <510AB97F.2080608@redhat.com> <510BF0F7.8020102@redhat.com> <510BF629.4040700@redhat.com> <510FA8D3.4000502@redhat.com> <5112D287.7010905@redhat.com> <511CB661.7010300@redhat.com> Message-ID: <5122716A.3030106@redhat.com> Petr Viktorin wrote: > On 02/06/2013 11:00 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 02/01/2013 06:06 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> On 01/31/2013 07:35 PM, Rob Crittenden wrote: >>>>>> Petr Viktorin wrote: >>>>>>> On 12/14/2012 01:46 AM, Dmitri Pal wrote: >>>>>>>> On 12/13/2012 10:21 AM, Petr Viktorin wrote: >>>>>>>>> https://fedorahosted.org/freeipa/ticket/3060 >>>>>>>>> >>>>>>>>> Here is a collection of smallish fixes to `ipa help` and `ipa >>>>>>>>> --help`. >>>>>>>>> This should address most of Nikolai's proposal. >>>>>>>>> Additionally, it's now possible to run `ipa --help` >>>>>>>>> without >>>>>>>>> a Kerberos ticket. And there are some new tests. >>>>>>>>> >>>>>>>>> I've not included the "Often used commands" in `ipa help`; I think >>>>>>>>> that is material for a manual/tutorial, not a help command. >>>>>>>>> Selecting >>>>>>>>> a topic from `ipa topics` and then choosing a command from `ipa >>>>>>>>> help >>>>>>>>> ` is a better way to use the help than the verbose `ipa >>>>>>>>> help >>>>>>>>> commands` or proposed incomplete "Often used commands". >>>>>>>> >>>>>>>> Since the ticket has a bit of discussion and you indicate that you >>>>>>>> did >>>>>>>> not to address everything can you please extract what have been >>>>>>>> addressed and put it into a design page. >>>>>>>> I know it is not RFE but it would help to validate the changes by >>>>>>>> testers. >>>>>>>> Please put the wiki link into the ticket. >>>>>>>> >>>>>>> >>>>>>> http://freeipa.org/page/V3/Help >>>>>>> >>>>>>> >>>>>> >>>>>> What is the purpose of the no-option outfile? Do you anticipate at >>>>>> some >>>>>> point opening this up as a real option or making it easier to log >>>>>> while >>>>>> using the api directly? >>>>> >>>>> On incorrect invocation (`ipa`, `ipa TOPIC`), the help command is >>>>> called >>>>> internally with outfile=sys.stderr. >>>> >>>> That's true, but this is a lot of code to abstract writing to >>>> sys.stderr. I'm just trying to understand the reasoning. Do you imagine >>>> that some time in the future we'd want to control where the output >>>> goes? >>> >>> I don't think that would be useful, I can't imagine why someone would >>> want to log help texts, or use them to script something. >>> Do you have another idea of how this should be done? >>> >>>>>> The help for help is a little confusing: >>>>>> >>>>>> ----- >>>>>> Purpose: Display help for a command or topic. >>>>>> Usage: ipa [global-options] help [TOPIC] [options] >>>>>> >>>>>> Positional arguments: >>>>>> TOPIC The topic or command name. >>>>>> >>>>>> Options: >>>>>> -h, --help show this help message and exit >>>>>> ----- >>>>>> >>>>>> Should [TOPIC] be [TOPIC | COMMAND] or something else? >>>>> >>>>> I'm trying to discourage `ipa help COMMAND` in favor of `ipa COMMAND >>>>> --help`, that way you get the proper help for ambiguous words like >>>>> "ping" (https://fedorahosted.org/freeipa/ticket/3247) >>>>> >>>>> I also wanted to keep the help simple and not explain this unimportant >>>>> detail here. >>>>> >>>>> If you have better wording I can of course change it. >>>> >>>> Your reasoning is sound. I"m ok with gently pushing users in that >>>> direction but leaving undocumented the old behavior. Should we create a >>>> ticket to eventually return an error in that case? >>> >>> Users will expect `ipa help COMMAND` to get them command help, it's >>> pretty standard in tools with subcommands. If it was a part of the API >>> I'd be all for enforcing proper usage, but I think a help command >>> deserves some slack -- it's not that big a deal if you get topic help >>> for ping instead of command help... >>> Hm. Now I see that I should add a notice to the topic help text, to lead >>> users to the right path. Patch attached. >>> >>> We will want to remove `ipa help COMMAND` from the docs, and note that >>> "ipa help" favors topics. >>> We can turn https://fedorahosted.org/freeipa/ticket/3247 into a doc >>> ticket. >>> >>> >>>>>> On my fresh F-18 install one of the new unit tests fails: >>>>>> >>>>>> ====================================================================== >>>>>> >>>>>> FAIL: Test that `help user-add` & `user-add -h` are equivalent and >>>>>> contain doc >>>>>> ---------------------------------------------------------------------- >>>>>> >>>>>> Traceback (most recent call last): >>>>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >>>>>> runTest >>>>>> self.test(*self.arg) >>>>>> File "/home/rcrit/redhat/freeipa/tests/test_cmdline/test_help.py", >>>>>> line 111, in test_command_help >>>>>> assert h_ctx.stdout == help_ctx.stdout >>>>>> AssertionError >>>>> >>>>> This is weird, it works for me. Could you send me the two outputs so I >>>>> can get some idea what's wrong? >>>>> >>> >>> Can you check you didn't leave out patch 111 (Add command summary?)? >>> >>> >> >> Yeah, I missed 110 and 111. Tests are passing now. >> >> I still don't understand the purpose of outfile. What do we gain by >> making this configurable and who or what would ever change it? >> > > The mechanism in the patch changes it. For `ipa help`, the output goes > to stdout. For `ipa` (invalid invocation), there's an error and the help > is written to stderr. So the place where the output ends up needs to be > configurable. > Ok, ACK, pushed these 8 plus 175 to master and ipa-3-1 rob From rcritten at redhat.com Mon Feb 18 19:33:59 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 18 Feb 2013 14:33:59 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51220DFE.3040209@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> <511B5B25.7040607@redhat.com> <511EAC0D.1020905@redhat.com> <51220DFE.3040209@redhat.com> Message-ID: <51228227.30904@redhat.com> Petr Viktorin wrote: > On 02/15/2013 10:43 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 02/06/2013 07:23 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>>> This adds a cert-find command for the dogtag backend. >>>>>> >>>>>> Searches can be done by serial number, by subject, revocation reason, >>>>>> issue date, notbefore, notafter and revocation dates. >>>>>> >>>>>> I added some basic tests for this. I made it a separate test file >>>>>> because the cert plugin tests do not use the declarative format and >>>>>> rely >>>>>> on the selfsign backend by default. >>>>>> >>>>>> rob >>>>> >>>>> Thanks! The code works well, but I found a few issues. >>>>> >>>>> >>>>> These tests don't work when the full test suite is run: test_cert adds >>>>> and revokes additional certs that throw the code off. >>>>> Perhaps have the tests only query valid certs? I don't see that option >>>>> but I think it would be helpful to support. >>>> >>>> I added some rather nasty hacks to the test to make things pass. I >>>> limit >>>> the search to 10 certificates, which is the number start with by >>>> default. There is an open dogtag ticket to return only VALID >>>> certificates so we should be able to drop this eventually. >>>> >>>> I had to go further on one of the revocation tests, limiting it to a >>>> sizelimit of 1. The count changes every time the suite runs so this is >>>> the safest for now. It also means that one test will fail if this is >>>> the >>>> only part of the suite executed. >>> >>> This gets rid of most of the failures, but it still fails the "certs for >>> this IPA server/short name" tests if the cert from ./make-cert is >>> present (creating it is part of `make test`). >>> If make-cert is run more times, it'll revoke the previous cert, so the >>> test for revocation reason 4 fails as well. >>> >>> It looks like when using sizelimit Dogtag will always discard *newer* >>> certs, ones with higher serials. Is it documented behavior or does >>> Dogtag just happen to do that? >> >> It isn't documented anywhere I could find, it is just what dogtag returns >> >>> I wonder how other people run their tests. This solution looks like it >>> could break easily if people do something differently :( >>> >>> I'm not sure how to solve this properly. Perhaps not using Declarative, >>> and checking "by hand" that the wanted certs are in the response and the >>> unwanted ones are not, would work better. >> >> I ended up switching the test class. It is not a very fine-grained set >> of tests, mostly searching with limits and verifying that we fall within >> a reasonable range, but it is better than nothing. >> >> rob > > This works much better, thanks! Just two nitpicks now. > > The patch doesn't apply well, there's a conflict in VERSION and some > added trailing whitespace, > > AFAIK this would be the only (first?) test that relies on Nose's > ordering of test modules -- tests 0011 and 0030 rely on the other cert > tests running first. Please at least mention that in a comment. Or > better, move class test_cert_find to test_cert.py > > Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob From rcritten at redhat.com Mon Feb 18 19:39:42 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 18 Feb 2013 14:39:42 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51228227.30904@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> <511B5B25.7040607@redhat.com> <511EAC0D.1020905@redhat.com> <51220DFE.3040209@redhat.com> <51228227.30904@redhat.com> Message-ID: <5122837E.5090104@redhat.com> Rob Crittenden wrote: > Petr Viktorin wrote: >> On 02/15/2013 10:43 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 02/06/2013 07:23 PM, Rob Crittenden wrote: >>>>> Petr Viktorin wrote: >>>>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>>>> This adds a cert-find command for the dogtag backend. >>>>>>> >>>>>>> Searches can be done by serial number, by subject, revocation >>>>>>> reason, >>>>>>> issue date, notbefore, notafter and revocation dates. >>>>>>> >>>>>>> I added some basic tests for this. I made it a separate test file >>>>>>> because the cert plugin tests do not use the declarative format and >>>>>>> rely >>>>>>> on the selfsign backend by default. >>>>>>> >>>>>>> rob >>>>>> >>>>>> Thanks! The code works well, but I found a few issues. >>>>>> >>>>>> >>>>>> These tests don't work when the full test suite is run: test_cert >>>>>> adds >>>>>> and revokes additional certs that throw the code off. >>>>>> Perhaps have the tests only query valid certs? I don't see that >>>>>> option >>>>>> but I think it would be helpful to support. >>>>> >>>>> I added some rather nasty hacks to the test to make things pass. I >>>>> limit >>>>> the search to 10 certificates, which is the number start with by >>>>> default. There is an open dogtag ticket to return only VALID >>>>> certificates so we should be able to drop this eventually. >>>>> >>>>> I had to go further on one of the revocation tests, limiting it to a >>>>> sizelimit of 1. The count changes every time the suite runs so this is >>>>> the safest for now. It also means that one test will fail if this is >>>>> the >>>>> only part of the suite executed. >>>> >>>> This gets rid of most of the failures, but it still fails the "certs >>>> for >>>> this IPA server/short name" tests if the cert from ./make-cert is >>>> present (creating it is part of `make test`). >>>> If make-cert is run more times, it'll revoke the previous cert, so the >>>> test for revocation reason 4 fails as well. >>>> >>>> It looks like when using sizelimit Dogtag will always discard *newer* >>>> certs, ones with higher serials. Is it documented behavior or does >>>> Dogtag just happen to do that? >>> >>> It isn't documented anywhere I could find, it is just what dogtag >>> returns >>> >>>> I wonder how other people run their tests. This solution looks like it >>>> could break easily if people do something differently :( >>>> >>>> I'm not sure how to solve this properly. Perhaps not using Declarative, >>>> and checking "by hand" that the wanted certs are in the response and >>>> the >>>> unwanted ones are not, would work better. >>> >>> I ended up switching the test class. It is not a very fine-grained set >>> of tests, mostly searching with limits and verifying that we fall within >>> a reasonable range, but it is better than nothing. >>> >>> rob >> >> This works much better, thanks! Just two nitpicks now. >> >> The patch doesn't apply well, there's a conflict in VERSION and some >> added trailing whitespace, >> >> AFAIK this would be the only (first?) test that relies on Nose's >> ordering of test modules -- tests 0011 and 0030 rely on the other cert >> tests running first. Please at least mention that in a comment. Or >> better, move class test_cert_find to test_cert.py >> >> > > Rebased the patch and removed whitespace. > > I went ahead and combined this with the existing test_cert file. > Originally test_cert was only tested against lite-server but since it > works against a live dogtag server too it makes sense to combine them. > > I improved the set up documentation a little bit and tried to handle all > the different configurations that one might see so that this should be > runnable against either a live server or the lite-server for both the > selfsign and dogtag backends. > > This relies on the user configuring ~/.ipa/default.conf to match the > remote server. There is no way from a client to know what kind of CA > backend a server is running. > > rob And the patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1085-4-cert-find.patch Type: text/x-diff Size: 30574 bytes Desc: not available URL: From rcritten at redhat.com Mon Feb 18 21:00:11 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 18 Feb 2013 16:00:11 -0500 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades Message-ID: <5122965B.3030909@redhat.com> An objectclass and attribute are not being added on upgrades. Missing these causes the UI to not work. I also noticed a typo in the ordering of a number of the trust attributes so fix those as well. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1087-schema.patch Type: text/x-diff Size: 19272 bytes Desc: not available URL: From edewata at redhat.com Tue Feb 19 03:12:40 2013 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 18 Feb 2013 21:12:40 -0600 Subject: [Freeipa-devel] [PATCH] 256-258 Web UI: removed build dependency errors In-Reply-To: <512256D1.8040101@redhat.com> References: <512256D1.8040101@redhat.com> Message-ID: <5122EDA8.7030308@redhat.com> ACK. -- Endi S. Dewata On 2/18/2013 10:29 AM, Petr Vobornik wrote: > First patch adds just a devel tool usable in browser. Second is > foundation for specifying which dependency errors can be ignored in Web > UI build. Third is an usage of second. > > These patches were develop as part of ongoing refactoring work but they > are independent and beneficial for all UI work so I'm sending them now. > > 1) [PATCH] Develop.js extended > > Develop.js contains code useful only for debugging. It is not part of > FreeIPA release. > > Is loaded by typing require(['freeipa/develop']); in browser JS console. > It adds IPA global variable and provide easier way of loading AMD > modules into window.ipadev[providedNameOrModuleName] variable. > > > 2) [PATCH] Allow to specify modules for which builder doesn't raise > dependency error > > One can specify module ids provided by other means (already built layer > file) in providedMids array of build profile file's package section. > Builder then ignores dependency errors for specified modules. This > allows to build layers without source codes of their dependencies, with > no expected errors raised. > > Example: > > packages:[ > { > name: "freeipa", > location: "freeipa", > providedMids: [ > 'dojo/_base/declare', > 'dojo/_base/lang', > 'dojo/_base/array', > 'dojo/Stateful' > //etc > ] > } > ], > > [PATCH] Web UI build profile updated > > freeipa.profile was updated accordingly to contain all modules in dojo > layer. > > This change removes expected errors during the build and therefore it > won't confuse others during rpm build. It also helps during development > because developer will notice real dependency errors (those not > specified this way). From pviktori at redhat.com Tue Feb 19 11:15:32 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 19 Feb 2013 12:15:32 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <511B6874.5060106@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> <511B6874.5060106@redhat.com> Message-ID: <51235ED4.2000801@redhat.com> On 02/13/2013 11:18 AM, Petr Viktorin wrote: > On 01/29/2013 05:06 PM, Petr Viktorin wrote: >> On 01/04/2013 07:20 PM, Petr Viktorin wrote: >>> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>>> It's time to give this to another set of eyes :) >>>>>>> >>>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>>> >>>>>>> More info is in commit messages. >>>>>>> >>>>>>> >>>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>>> change the >>>>>>> design document: when the client doesn't send the API version, it is >>>>>>> assumed >>>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>>> The client still gets a warning if the version is missing. Except >>>>>>> for >>>>>>> those >>>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>>> the >>>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>>> our client >>>>>>> is so far the only one that validates so strictly.) >>>>>> >>>>>> I did a basic test of this patch and also quickly read through the >>>>>> patches and >>>>>> besides nitpicks (like unused inspect module in >>>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>>> obvious >>>>>> errors in the Python code. >>>>> >>>>> Noted, will fix in future versions of the patch. >>>>> >>>>>> >>>>>> However, this patch breaks WebUI badly, I did not even get to a >>>>>> log in >>>>>> screen. >>>>>> Cooperation with Petr Vobornik will be needed. In my case, I got >>>>>> blank >>>>>> screen >>>>>> and Javascript error: >>>>>> >>>>>> TypeError: IPA.messages.dialogs is undefined >>>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>>> Line 1460 >>>>>> >>>>>> I assume this is related to the Internal Error that was returned in >>>>>> the JSON call >>>>>> >>>>>> { >>>>>> "error": null, >>>>>> "id": null, >>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>> "result": { >>>>>> "count": 5, >>>>>> "results": [ >>>>>> { >>>>>> "error": "an internal error has occurred", >>>>>> "error_code": 903, >>>>>> "error_name": "InternalError" >>>>>> }, >>>>>> { >>>>>> ... >>>>>> >>>>>> This can be reproduced with: >>>>>> >>>>>> # curl -v -H "Content-Type:application/json" -H >>>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>>> --negotiate -u : >>>>>> --cacert /etc/ipa/ca.crt -d >>>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>>> https://`hostname`/ipa/json >>>>> >>>>> Good catch! The i18n_messages plugin already defines a "messages" >>>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>>> check for clashes. >>>>> Since i18n_messages is an internal command only used by the Web UI, we >>>>> can rename its output to "texts" without breaking compatibility. >>>>> >>>>> I'm attaching a preliminary fix (for both backend and UI), but >>>>> hopefully >>>>> it won't be necessary, see below. >>>>> >>>>>> I am also not sure I like the requirement of a specific version >>>>>> option >>>>>> to be >>>>>> always passed. I would prefer that missing version option would mean >>>>>> "I use the >>>>>> most recent version of API" instead - it would make the custom >>>>>> JSONRPC/XMLRPC >>>>>> calls easier to use. >>>>>> >>>>>> But since the version option was not being sent for some commands, we >>>>>> may not >>>>>> have a choice anyway if we do not want to break old clients in >>>>>> case we >>>>>> add some >>>>>> capabilities to these commands. >>>>>> >>>>> >>>>> I see three other options, all worse: >>>>> - Do not use capabilities for the affected commands, meaning no new >>>>> functionality can be added to them (and by extension, no new >>>>> functionality common to all commands can be added). >>>>> - Treat a missing version as the current version >>>>> - Break backwards compatibility >>>>> >>>>> And one possibly better (thanks to Petr? and Martin for opening my >>>>> eyes >>>>> off-list!): >>>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>>> >>>>> This would also allow us to solve the above name-clashing problem >>>>> elegantly. Here is a reminder of what a JSON response looks like: >>>>> >>>>> { >>>>> "error": null, >>>>> "id": 0, >>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>> "result": { >>>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>>> 2.47" >>>>> }, >>>>> "version": "3.1.0GIT2e4bd02" >>>>> } >>>>> >>>>> A XML-RPC response only contains the "result" part of that. >>>>> So with JSON, we can put the messages in the top level, which is much >>>>> better design. >>> >>> Custom info in the "top level" seems to be a violation of the JSON-RPC >>> spec. I'd rather not do more of those, so I'm withdrawing this idea. >>> >>>>> >>>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>>> inability to attach extra info to errors (commit >>>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>>> PublicError). >>>>> >>>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>>> >>>> >>>> +1, XML-RPC sucks. This should have been done a long time ago. >>>> >>>> Honza >>>> >>> >>> Here are new patches. >>> >>> XML-RPC requests with missing version are assumed to be old (the version >>> before capabilities are introduced, 2.47). This takes care of backcompat >>> with clients with bug 3294. >>> JSON-RPC requests with missing version are assumed to be testing calls >>> (e.g. curl), they get behavior of the latest version but also a warning. >>> I've also added this info to the design doc. >>> >>> It turns out that these patches don't depend on whether our client uses >>> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >>> extra unit tests. >>> >> >> Patch 106 had a minor conflict with master, attaching fixed version. >> > > Patches 106 & 115 need an API version bump. > I also noticed that `makeapi --validate` wasn't checking capabilities > properly. Fixed. > > Rebasing patch 104 to current master. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0104-02-Add-the-version-option-to-all-Commands.patch Type: text/x-patch Size: 68082 bytes Desc: not available URL: From abokovoy at redhat.com Tue Feb 19 12:20:37 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 19 Feb 2013 14:20:37 +0200 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc In-Reply-To: <20130213141221.GD476@redhat.com> References: <511A6D60.6030300@redhat.com> <20130213141221.GD476@redhat.com> Message-ID: <20130219122037.GM476@redhat.com> On Wed, 13 Feb 2013, Alexander Bokovoy wrote: >On Tue, 12 Feb 2013, Ana Krivokapic wrote: >>Add new LDAP container to store the list of domains associated with IPA >>realm. >>Add two new ipa commands (ipa realmdomains-show and ipa >>realmdomains-mod) to allow manipulation of the list of realm domains. >>Unit test file covering these new commands was added. >> >>https://fedorahosted.org/freeipa/ticket/2945 >ACK, works perfectly. > >We need to decide on the questions still open in the >http://www.freeipa.org/page/V3/Realm_Domains but the decision should not >prevent this work from being committed. Committed to master along with a one-liner that limits ipasam to look only at $SUFFIX with base scope when searching for its own domain name as now we use domainRelatedObject in two different places and previous subtree search now gives too wide answer. -- / Alexander Bokovoy From sbose at redhat.com Tue Feb 19 12:40:10 2013 From: sbose at redhat.com (Sumit Bose) Date: Tue, 19 Feb 2013 13:40:10 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <511D093C.5090602@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> <511CD8F6.9040509@redhat.com> <511D093C.5090602@redhat.com> Message-ID: <20130219124010.GJ27418@localhost.localdomain> On Thu, Feb 14, 2013 at 09:56:44AM -0600, Endi Sukma Dewata wrote: > On 2/14/2013 6:30 AM, Petr Vobornik wrote: > >>If they are mutually exclusive, they probably should be separated using > >>radio buttons like this: > >> > >> PAC: ( ) None > >> (o) Type: > >> [x] MS-PAC > >> [ ] PAD > > > >You missed one option: nothing selected. It can be solved by adding '( > >) Inherited' radio. > > I wouldn't have guessed that :) I agree we should add the > 'Inherited' option. > Would it be possible to print the inherited values here, i.e. the default values given in the ipakrbauthzdata attribute of ipaGuiConfig object? I think this would improve the user experience because you do not have to remember/look up the default values. bye, Sumit From pviktori at redhat.com Tue Feb 19 12:46:28 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 19 Feb 2013 13:46:28 +0100 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <5122837E.5090104@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> <511B5B25.7040607@redhat.com> <511EAC0D.1020905@redhat.com> <51220DFE.3040209@redhat.com> <51228227.30904@redhat.com> <5122837E.5090104@redhat.com> Message-ID: <51237424.9020708@redhat.com> On 02/18/2013 08:39 PM, Rob Crittenden wrote: > Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 02/15/2013 10:43 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> On 02/06/2013 07:23 PM, Rob Crittenden wrote: >>>>>> Petr Viktorin wrote: >>>>>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>>>>> This adds a cert-find command for the dogtag backend. >>>>>>>> >>>>>>>> Searches can be done by serial number, by subject, revocation >>>>>>>> reason, >>>>>>>> issue date, notbefore, notafter and revocation dates. >>>>>>>> >>>>>>>> I added some basic tests for this. I made it a separate test file >>>>>>>> because the cert plugin tests do not use the declarative format and >>>>>>>> rely >>>>>>>> on the selfsign backend by default. >>>>>>>> >>>>>>>> rob >>>>>>> >>>>>>> Thanks! The code works well, but I found a few issues. >>>>>>> >>>>>>> >>>>>>> These tests don't work when the full test suite is run: test_cert >>>>>>> adds >>>>>>> and revokes additional certs that throw the code off. >>>>>>> Perhaps have the tests only query valid certs? I don't see that >>>>>>> option >>>>>>> but I think it would be helpful to support. >>>>>> >>>>>> I added some rather nasty hacks to the test to make things pass. I >>>>>> limit >>>>>> the search to 10 certificates, which is the number start with by >>>>>> default. There is an open dogtag ticket to return only VALID >>>>>> certificates so we should be able to drop this eventually. >>>>>> >>>>>> I had to go further on one of the revocation tests, limiting it to a >>>>>> sizelimit of 1. The count changes every time the suite runs so >>>>>> this is >>>>>> the safest for now. It also means that one test will fail if this is >>>>>> the >>>>>> only part of the suite executed. >>>>> >>>>> This gets rid of most of the failures, but it still fails the "certs >>>>> for >>>>> this IPA server/short name" tests if the cert from ./make-cert is >>>>> present (creating it is part of `make test`). >>>>> If make-cert is run more times, it'll revoke the previous cert, so the >>>>> test for revocation reason 4 fails as well. >>>>> >>>>> It looks like when using sizelimit Dogtag will always discard *newer* >>>>> certs, ones with higher serials. Is it documented behavior or does >>>>> Dogtag just happen to do that? >>>> >>>> It isn't documented anywhere I could find, it is just what dogtag >>>> returns >>>> >>>>> I wonder how other people run their tests. This solution looks like it >>>>> could break easily if people do something differently :( >>>>> >>>>> I'm not sure how to solve this properly. Perhaps not using >>>>> Declarative, >>>>> and checking "by hand" that the wanted certs are in the response and >>>>> the >>>>> unwanted ones are not, would work better. >>>> >>>> I ended up switching the test class. It is not a very fine-grained set >>>> of tests, mostly searching with limits and verifying that we fall >>>> within >>>> a reasonable range, but it is better than nothing. >>>> >>>> rob >>> >>> This works much better, thanks! Just two nitpicks now. >>> >>> The patch doesn't apply well, there's a conflict in VERSION and some >>> added trailing whitespace, >>> >>> AFAIK this would be the only (first?) test that relies on Nose's >>> ordering of test modules -- tests 0011 and 0030 rely on the other cert >>> tests running first. Please at least mention that in a comment. Or >>> better, move class test_cert_find to test_cert.py >>> >>> >> >> Rebased the patch and removed whitespace. >> >> I went ahead and combined this with the existing test_cert file. >> Originally test_cert was only tested against lite-server but since it >> works against a live dogtag server too it makes sense to combine them. >> >> I improved the set up documentation a little bit and tried to handle all >> the different configurations that one might see so that this should be >> runnable against either a live server or the lite-server for both the >> selfsign and dogtag backends. >> >> This relies on the user configuring ~/.ipa/default.conf to match the >> remote server. There is no way from a client to know what kind of CA >> backend a server is running. >> >> rob > > And the patch. > > rob > ACK, thank you! -- Petr? From jcholast at redhat.com Tue Feb 19 12:51:49 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 19 Feb 2013 13:51:49 +0100 Subject: [Freeipa-devel] [PATCHES] 0177-0179 Add missing dict methods to CIDict In-Reply-To: <51113B3F.3070808@redhat.com> References: <51113B3F.3070808@redhat.com> Message-ID: <51237565.1040404@redhat.com> Hi, On 5.2.2013 18:02, Petr Viktorin wrote: > CIDict, our case-insensitive dictionary, inherits from dict but did not > reimplement the full dict interface. Calling the missing methods > silently invoked case-sensitive behavior. Our code seems to avoid that, > but it's a bit of a minefield for new development. > > Patch 119 adds the missing dict methods (except > view{items,keys,values}(), which now raise errors), and adds tests. Can you please also add the (obj, **kwargs) and (**kwargs) variants of __init__ and update? > > > Patches 117-118 modernize the testsuite a bit (these have been sitting > in my queue for a while, I think now is a good time to submit them): > The first one moves some old tests from the main code tree to tests/. > (The adtrust_install test wasn't run before, this move makes nose notice > it). > The second converts CIDict's unittest-based suite to nose. > Honza -- Jan Cholasta From jcholast at redhat.com Tue Feb 19 12:57:37 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 19 Feb 2013 13:57:37 +0100 Subject: [Freeipa-devel] [PATCH] 90 Run interactive_prompt callbacks after CSV values are split In-Reply-To: <511CB231.7030906@redhat.com> References: <50EDA525.1090104@redhat.com> <511CB231.7030906@redhat.com> Message-ID: <512376C1.8080903@redhat.com> On 14.2.2013 10:45, Petr Viktorin wrote: > This needs a test; here one I used to check it. > Otherwise it works well, ACK if the test is added. > Thank you, test added. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-90.1-Run-interactive_prompt-callbacks-after-CSV-values-ar.patch Type: text/x-patch Size: 3057 bytes Desc: not available URL: From pvoborni at redhat.com Tue Feb 19 13:01:24 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 19 Feb 2013 14:01:24 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <20130219124010.GJ27418@localhost.localdomain> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> <511CD8F6.9040509@redhat.com> <511D093C.5090602@redhat.com> <20130219124010.GJ27418@localhost.localdomain> Message-ID: <512377A4.7010809@redhat.com> On 02/19/2013 01:40 PM, Sumit Bose wrote: > On Thu, Feb 14, 2013 at 09:56:44AM -0600, Endi Sukma Dewata wrote: >> On 2/14/2013 6:30 AM, Petr Vobornik wrote: >>>> If they are mutually exclusive, they probably should be separated using >>>> radio buttons like this: >>>> >>>> PAC: ( ) None >>>> (o) Type: >>>> [x] MS-PAC >>>> [ ] PAD >>> >>> You missed one option: nothing selected. It can be solved by adding '( >>> ) Inherited' radio. >> >> I wouldn't have guessed that :) I agree we should add the >> 'Inherited' option. >> > > Would it be possible to print the inherited values here, i.e. the > default values given in the ipakrbauthzdata attribute of ipaGuiConfig > object? I think this would improve the user experience because you do > not have to remember/look up the default values. > > bye, > Sumit > It is possible, but not straightforward. I see two options: 1) Currently Web UI loads config at start so we can use it. The disadvantage is that the value might be changed and so the displayed value might be incorrect. 2) Other option is to run config-show along with service-show in a batch to get the up-to-date value. I'm not a big fan of #2 but it is probably better solution. -- Petr Vobornik From sbose at redhat.com Tue Feb 19 13:08:37 2013 From: sbose at redhat.com (Sumit Bose) Date: Tue, 19 Feb 2013 14:08:37 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <512377A4.7010809@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> <511CD8F6.9040509@redhat.com> <511D093C.5090602@redhat.com> <20130219124010.GJ27418@localhost.localdomain> <512377A4.7010809@redhat.com> Message-ID: <20130219130837.GK27418@localhost.localdomain> On Tue, Feb 19, 2013 at 02:01:24PM +0100, Petr Vobornik wrote: > On 02/19/2013 01:40 PM, Sumit Bose wrote: > >On Thu, Feb 14, 2013 at 09:56:44AM -0600, Endi Sukma Dewata wrote: > >>On 2/14/2013 6:30 AM, Petr Vobornik wrote: > >>>>If they are mutually exclusive, they probably should be separated using > >>>>radio buttons like this: > >>>> > >>>> PAC: ( ) None > >>>> (o) Type: > >>>> [x] MS-PAC > >>>> [ ] PAD > >>> > >>>You missed one option: nothing selected. It can be solved by adding '( > >>>) Inherited' radio. > >> > >>I wouldn't have guessed that :) I agree we should add the > >>'Inherited' option. > >> > > > >Would it be possible to print the inherited values here, i.e. the > >default values given in the ipakrbauthzdata attribute of ipaGuiConfig > >object? I think this would improve the user experience because you do > >not have to remember/look up the default values. > > > >bye, > >Sumit > > > > It is possible, but not straightforward. > > I see two options: > 1) Currently Web UI loads config at start so we can use it. The > disadvantage is that the value might be changed and so the displayed > value might be incorrect. > 2) Other option is to run config-show along with service-show in a > batch to get the up-to-date value. > > I'm not a big fan of #2 but it is probably better solution. I agree with both :-). Since it is not straightforward I guess it would be better not to add this with this ticket but open a new one. Do you agree? bye, Sumit > -- > Petr Vobornik From pvoborni at redhat.com Tue Feb 19 13:12:24 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 19 Feb 2013 14:12:24 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <20130219130837.GK27418@localhost.localdomain> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> <511CD8F6.9040509@redhat.com> <511D093C.5090602@redhat.com> <20130219124010.GJ27418@localhost.localdomain> <512377A4.7010809@redhat.com> <20130219130837.GK27418@localhost.localdomain> Message-ID: <51237A38.9090201@redhat.com> On 02/19/2013 02:08 PM, Sumit Bose wrote: > On Tue, Feb 19, 2013 at 02:01:24PM +0100, Petr Vobornik wrote: >> On 02/19/2013 01:40 PM, Sumit Bose wrote: >>> On Thu, Feb 14, 2013 at 09:56:44AM -0600, Endi Sukma Dewata wrote: >>>> On 2/14/2013 6:30 AM, Petr Vobornik wrote: >>>>>> If they are mutually exclusive, they probably should be separated using >>>>>> radio buttons like this: >>>>>> >>>>>> PAC: ( ) None >>>>>> (o) Type: >>>>>> [x] MS-PAC >>>>>> [ ] PAD >>>>> >>>>> You missed one option: nothing selected. It can be solved by adding '( >>>>> ) Inherited' radio. >>>> >>>> I wouldn't have guessed that :) I agree we should add the >>>> 'Inherited' option. >>>> >>> >>> Would it be possible to print the inherited values here, i.e. the >>> default values given in the ipakrbauthzdata attribute of ipaGuiConfig >>> object? I think this would improve the user experience because you do >>> not have to remember/look up the default values. >>> >>> bye, >>> Sumit >>> >> >> It is possible, but not straightforward. >> >> I see two options: >> 1) Currently Web UI loads config at start so we can use it. The >> disadvantage is that the value might be changed and so the displayed >> value might be incorrect. >> 2) Other option is to run config-show along with service-show in a >> batch to get the up-to-date value. >> >> I'm not a big fan of #2 but it is probably better solution. > I agree with both :-). Since it is not straightforward I guess it would > be better not to add this with this ticket but open a new one. Do you > agree? Yes > > bye, > Sumit > >> -- >> Petr Vobornik -- Petr Vobornik From jcholast at redhat.com Tue Feb 19 13:17:24 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 19 Feb 2013 14:17:24 +0100 Subject: [Freeipa-devel] [PATCHES] 137-144 LDAP code refactoring (Part 3) In-Reply-To: <5107948C.40105@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5107948C.40105@redhat.com> Message-ID: <51237B64.2040600@redhat.com> On 29.1.2013 10:21, Jan Cholasta wrote: > A patch from this patchset (part 3) causes some of the dns plugin tests > to fail (idnsallowdynupdate is missing in dnszone_add output). > > Honza > Patch 143: + assert isinstance(entry_or_dn, DN) + if normalize is None or normalize: + entry_or_dn = self.normalize_dn(entry_or_dn) + entry_attrs = dict(entry_attrs) Can you please return LDAPEntry here as well, i.e. replace dict(entry_attrs) with self.make_entry(entry_or_dn, entry_attrs)? + def delete_entry(self, entry, normalize=None): + """Delete entry. + + The `normalize` argument does nothing when called with a LDAPEntry. + + The legacy variant is: + delete_entry(dn, normalize=True) + """ I don't think this is right. We don't need to know any of the attributes of an entry to delete it, just its DN. I think we should keep the DN variant of delete_entry as the primary one. Honza -- Jan Cholasta From sbose at redhat.com Tue Feb 19 13:28:58 2013 From: sbose at redhat.com (Sumit Bose) Date: Tue, 19 Feb 2013 14:28:58 +0100 Subject: [Freeipa-devel] [PATCH] 255 Added Web UI support for service PAC type option: NONE In-Reply-To: <51237A38.9090201@redhat.com> References: <5114FD48.3070104@redhat.com> <511A6A54.1040307@redhat.com> <511A7437.8000207@redhat.com> <511ABF1F.70106@redhat.com> <511CD8F6.9040509@redhat.com> <511D093C.5090602@redhat.com> <20130219124010.GJ27418@localhost.localdomain> <512377A4.7010809@redhat.com> <20130219130837.GK27418@localhost.localdomain> <51237A38.9090201@redhat.com> Message-ID: <20130219132858.GL27418@localhost.localdomain> On Tue, Feb 19, 2013 at 02:12:24PM +0100, Petr Vobornik wrote: > On 02/19/2013 02:08 PM, Sumit Bose wrote: > >On Tue, Feb 19, 2013 at 02:01:24PM +0100, Petr Vobornik wrote: > >>On 02/19/2013 01:40 PM, Sumit Bose wrote: > >>>On Thu, Feb 14, 2013 at 09:56:44AM -0600, Endi Sukma Dewata wrote: > >>>>On 2/14/2013 6:30 AM, Petr Vobornik wrote: > >>>>>>If they are mutually exclusive, they probably should be separated using > >>>>>>radio buttons like this: > >>>>>> > >>>>>> PAC: ( ) None > >>>>>> (o) Type: > >>>>>> [x] MS-PAC > >>>>>> [ ] PAD > >>>>> > >>>>>You missed one option: nothing selected. It can be solved by adding '( > >>>>>) Inherited' radio. > >>>> > >>>>I wouldn't have guessed that :) I agree we should add the > >>>>'Inherited' option. > >>>> > >>> > >>>Would it be possible to print the inherited values here, i.e. the > >>>default values given in the ipakrbauthzdata attribute of ipaGuiConfig > >>>object? I think this would improve the user experience because you do > >>>not have to remember/look up the default values. > >>> > >>>bye, > >>>Sumit > >>> > >> > >>It is possible, but not straightforward. > >> > >>I see two options: > >> 1) Currently Web UI loads config at start so we can use it. The > >>disadvantage is that the value might be changed and so the displayed > >>value might be incorrect. > >> 2) Other option is to run config-show along with service-show in a > >>batch to get the up-to-date value. > >> > >>I'm not a big fan of #2 but it is probably better solution. > >I agree with both :-). Since it is not straightforward I guess it would > >be better not to add this with this ticket but open a new one. Do you > >agree? > > Yes Thanks, I've opened https://fedorahosted.org/freeipa/ticket/3436 . bye, Sumit > > > > >bye, > >Sumit > > > >>-- > >>Petr Vobornik > > > -- > Petr Vobornik From pspacek at redhat.com Tue Feb 19 13:34:49 2013 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 19 Feb 2013 14:34:49 +0100 Subject: [Freeipa-devel] [PATCH 0077] Refactor settings subsystem In-Reply-To: <5072D551.10703@redhat.com> References: <5072D551.10703@redhat.com> Message-ID: <51237F79.4020006@redhat.com> On 8.10.2012 15:29, Petr Spacek wrote: > Hello, > > this patch refactors setting subsystem. After some experimenting I chosen > simple implementation with static arrays, no RBT trees involved. Speed can be > improved by reordering items in arrays. > > Commit message: > > Refactor settings subsystem. > > Settings are stored in tree of settings_set_t structures. > All settings should be accessed only through setting* functions. > Mutual exclusion during write is done by switching to single > thread mode. > > Setting_get() function doesn't copy strings, so changing the > original string can lead to obscure bugs. > This way is okay as long as strings are not changed dynamically > at run-time. > > Unknown setting in configuration file leads to failure rather than > silent ignoring it. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/53 > https://fedorahosted.org/bind-dyndb-ldap/ticket/81 Re-based & amended & tested patch is attached. The main idea is the same. > Adam, I'm still looking for way how to handle strings in settings. We have to > prevent string change/deallocation as long as somebody has a pointer to the > string (I mean pointer obtained through setting_get("name_of_setting") call). > > The only way which I can see is returning setting_string structure like > > setting_string struct { > isc_refcount_t counter; > isc_mem_t *mctx; > char *str; > }; After some poking to isc_task_beginexclusive() I consider the "struct setting_string" idea redundant. > Caller has to call setting_string_free() when string can be freed. > Setting_string_free() will decrement counter by one and free whole structure > if and only if counter reaches 0. > > Is it meaningful? Should I separate setting_get_* for each datatype? Or just > for setting_get_string() and let setting_get() universal for integers and > booleans? I separated setting_get_bool/uint/str from the universal setting_get() to enable type checks. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0077-2-Refactor-settings-subsystem.patch Type: text/x-patch Size: 92978 bytes Desc: not available URL: From jcholast at redhat.com Tue Feb 19 14:10:25 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 19 Feb 2013 15:10:25 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <510BD364.7050809@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> Message-ID: <512387D1.60603@redhat.com> On 1.2.2013 15:38, Petr Viktorin wrote: > Alright, I renamed get_single to single_value(). > > I also rebased to current master. > Patch 152: + def single_value(self, name, default=_missing): + values = self.get(name, [default]) + if len(values) != 1: + raise ValueError( + '%s has %s values, one expected' % (name, len(values))) + if values[0] is _missing: + raise KeyError(name) + return values[0] I would prefer if you used __getitem__() instead of get() and re-raised KeyError if default is _missing. Also, until we disallow non-lists, we need to check if values actually is list. I.e., do something like this: + def single_value(self, name, default=_missing): + try: + values = super(LDAPEntry, self).__getitem__(self._get_attr_name(name)) + except KeyError: + if default is _missing: + raise + return default + if not isinstance(values, list): + return values + if len(values) != 1: + raise ValueError( + '%s has %s values, one expected' % (name, len(values))) + return values[0] Patch 159: Like I said in my review of your patch 143, I think we should use DNs instead of entries in delete_entry, so I think it would make sense to do delete_entry(entry.dn) in this patch. Patch 160: I think you should do this (replace % with ,): + root_logger.debug( + "failed to find mapping tree entry for %s", self.suffix) Honza -- Jan Cholasta From pviktori at redhat.com Tue Feb 19 15:22:19 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 19 Feb 2013 16:22:19 +0100 Subject: [Freeipa-devel] [PATCH] 90 Run interactive_prompt callbacks after CSV values are split In-Reply-To: <512376C1.8080903@redhat.com> References: <50EDA525.1090104@redhat.com> <511CB231.7030906@redhat.com> <512376C1.8080903@redhat.com> Message-ID: <512398AB.8040600@redhat.com> On 02/19/2013 01:57 PM, Jan Cholasta wrote: > On 14.2.2013 10:45, Petr Viktorin wrote: >> This needs a test; here one I used to check it. >> Otherwise it works well, ACK if the test is added. >> > > Thank you, test added. > > Honza > ACK -- Petr? From pviktori at redhat.com Tue Feb 19 15:56:58 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 19 Feb 2013 16:56:58 +0100 Subject: [Freeipa-devel] [PATCHES] 137-144 LDAP code refactoring (Part 3) In-Reply-To: <51237B64.2040600@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5107948C.40105@redhat.com> <51237B64.2040600@redhat.com> Message-ID: <5123A0CA.5010206@redhat.com> On 02/19/2013 02:17 PM, Jan Cholasta wrote: > On 29.1.2013 10:21, Jan Cholasta wrote: >> A patch from this patchset (part 3) causes some of the dns plugin tests >> to fail (idnsallowdynupdate is missing in dnszone_add output). >> >> Honza >> > > Patch 143: > > + assert isinstance(entry_or_dn, DN) > + if normalize is None or normalize: > + entry_or_dn = self.normalize_dn(entry_or_dn) > + entry_attrs = dict(entry_attrs) > > Can you please return LDAPEntry here as well, i.e. replace > dict(entry_attrs) with self.make_entry(entry_or_dn, entry_attrs)? Sure. I tried to keep the old behavior with old usage, but you're right, using LDAPEntry here will be better. > + def delete_entry(self, entry, normalize=None): > + """Delete entry. > + > + The `normalize` argument does nothing when called with a > LDAPEntry. > + > + The legacy variant is: > + delete_entry(dn, normalize=True) > + """ > > I don't think this is right. We don't need to know any of the attributes > of an entry to delete it, just its DN. I think we should keep the DN > variant of delete_entry as the primary one. > Makes sense. I made them both primary. I didn't bother documenting normalize since your patch will remove that. Updated patch attached; I'll update my repo when I respond to your comments on part 4. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0143-04-Change-add-update-delete-_entry-to-take-LDAPEntries.patch Type: text/x-patch Size: 4715 bytes Desc: not available URL: From rcritten at redhat.com Tue Feb 19 16:09:23 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 11:09:23 -0500 Subject: [Freeipa-devel] [PATCH] 90 Run interactive_prompt callbacks after CSV values are split In-Reply-To: <512398AB.8040600@redhat.com> References: <50EDA525.1090104@redhat.com> <511CB231.7030906@redhat.com> <512376C1.8080903@redhat.com> <512398AB.8040600@redhat.com> Message-ID: <5123A3B3.7020706@redhat.com> Petr Viktorin wrote: > On 02/19/2013 01:57 PM, Jan Cholasta wrote: >> On 14.2.2013 10:45, Petr Viktorin wrote: >>> This needs a test; here one I used to check it. >>> Otherwise it works well, ACK if the test is added. >>> >> >> Thank you, test added. >> >> Honza >> > > ACK > Pushed to master and ipa-3-1 rob From rcritten at redhat.com Tue Feb 19 16:14:34 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 11:14:34 -0500 Subject: [Freeipa-devel] [PATCH] 360 Add autodiscovery section in ipa-client-install man pages In-Reply-To: <510A95B3.4060504@redhat.com> References: <510A6117.80507@redhat.com> <510A7533.50903@redhat.com> <510A90B2.902@redhat.com> <510A95B3.4060504@redhat.com> Message-ID: <5123A4EA.5070800@redhat.com> Martin Kosek wrote: > On 01/31/2013 04:41 PM, Martin Kosek wrote: >> On 01/31/2013 02:44 PM, Petr Spacek wrote: >>> On 31.1.2013 13:18, Martin Kosek wrote: >>>> Explain how autodiscovery and failover works and which options >>>> are important for these elements. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3383 >>> >>> Could you add some note about "how ipa-client installer will be confused by >>> AD"? One paragraph with some explanation could help. >>> >> >> Sure, makes sense. Updated patch attached. >> >> Martin >> > > Petr noticed a typo in the updated section. Fixed version attached. > > Martin ACK, pushed to master and ipa-3-1 rob From jcholast at redhat.com Tue Feb 19 16:26:15 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 19 Feb 2013 17:26:15 +0100 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades In-Reply-To: <5122965B.3030909@redhat.com> References: <5122965B.3030909@redhat.com> Message-ID: <5123A7A7.1020100@redhat.com> Hi, On 18.2.2013 22:00, Rob Crittenden wrote: > An objectclass and attribute are not being added on upgrades. Missing > these causes the UI to not work. > > I also noticed a typo in the ordering of a number of the trust > attributes so fix those as well. > > rob > The patch looks good, but I think errors like this will pop up from time to time, because we have to maintain the same thing in two places - the installation LDIFs and update files. Maybe we should start thinking about merging these two somehow, e.g. using the LDIFs for both installation and updates, with directives for the updater in specially formatted comments. Honza -- Jan Cholasta From rcritten at redhat.com Tue Feb 19 16:30:58 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 11:30:58 -0500 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <5118EB4B.4080507@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> <5110FD62.4090006@redhat.com> <51111B1C.2010305@redhat.com> <511228A9.5060303@redhat.com> <5118EB4B.4080507@redhat.com> Message-ID: <5123A8C2.5000504@redhat.com> Petr Viktorin wrote: > On 02/06/2013 10:55 AM, Jan Cholasta wrote: >> On 5.2.2013 15:45, Petr Viktorin wrote: >>> On 02/05/2013 01:38 PM, Jan Cholasta wrote: >>>> On 4.2.2013 15:49, Petr Viktorin wrote: >>> [...] >>>>> >>>>> I see one of the changes is using has_key instead of `in` for a >>>>> CIDict. >>>>> Given that dict.has_key() is deprecated, I think a better solution >>>>> would >>>>> be to add __contains__ to CIDict. Is there a reason against that? >>>> >>>> I think a separate patch with this and other changes to make CIDict >>>> more >>>> like dict would be better. >>> >>> OK, I'll make a patch. >>> >>> [...] >>>> Updated patches attached. >>>> >>> >>> The changes look good but I can no longer apply the patches. Can you >>> please rebase them? >>> >>> >> >> Sure. > > ACK > I think dropping raw=True in patch 99.3 is going to break a check later where we look at the managedby attribute. Without raw this will be managedby_host. rob From rcritten at redhat.com Tue Feb 19 16:53:36 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 11:53:36 -0500 Subject: [Freeipa-devel] [PATCH] 1085 cert-find command In-Reply-To: <51237424.9020708@redhat.com> References: <51119947.9050707@redhat.com> <511234D1.6080605@redhat.com> <51129F98.7060706@redhat.com> <511B5B25.7040607@redhat.com> <511EAC0D.1020905@redhat.com> <51220DFE.3040209@redhat.com> <51228227.30904@redhat.com> <5122837E.5090104@redhat.com> <51237424.9020708@redhat.com> Message-ID: <5123AE10.3060102@redhat.com> Petr Viktorin wrote: > On 02/18/2013 08:39 PM, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 02/15/2013 10:43 PM, Rob Crittenden wrote: >>>>> Petr Viktorin wrote: >>>>>> On 02/06/2013 07:23 PM, Rob Crittenden wrote: >>>>>>> Petr Viktorin wrote: >>>>>>>> On 02/06/2013 12:44 AM, Rob Crittenden wrote: >>>>>>>>> This adds a cert-find command for the dogtag backend. >>>>>>>>> >>>>>>>>> Searches can be done by serial number, by subject, revocation >>>>>>>>> reason, >>>>>>>>> issue date, notbefore, notafter and revocation dates. >>>>>>>>> >>>>>>>>> I added some basic tests for this. I made it a separate test file >>>>>>>>> because the cert plugin tests do not use the declarative format >>>>>>>>> and >>>>>>>>> rely >>>>>>>>> on the selfsign backend by default. >>>>>>>>> >>>>>>>>> rob >>>>>>>> >>>>>>>> Thanks! The code works well, but I found a few issues. >>>>>>>> >>>>>>>> >>>>>>>> These tests don't work when the full test suite is run: test_cert >>>>>>>> adds >>>>>>>> and revokes additional certs that throw the code off. >>>>>>>> Perhaps have the tests only query valid certs? I don't see that >>>>>>>> option >>>>>>>> but I think it would be helpful to support. >>>>>>> >>>>>>> I added some rather nasty hacks to the test to make things pass. I >>>>>>> limit >>>>>>> the search to 10 certificates, which is the number start with by >>>>>>> default. There is an open dogtag ticket to return only VALID >>>>>>> certificates so we should be able to drop this eventually. >>>>>>> >>>>>>> I had to go further on one of the revocation tests, limiting it to a >>>>>>> sizelimit of 1. The count changes every time the suite runs so >>>>>>> this is >>>>>>> the safest for now. It also means that one test will fail if this is >>>>>>> the >>>>>>> only part of the suite executed. >>>>>> >>>>>> This gets rid of most of the failures, but it still fails the "certs >>>>>> for >>>>>> this IPA server/short name" tests if the cert from ./make-cert is >>>>>> present (creating it is part of `make test`). >>>>>> If make-cert is run more times, it'll revoke the previous cert, so >>>>>> the >>>>>> test for revocation reason 4 fails as well. >>>>>> >>>>>> It looks like when using sizelimit Dogtag will always discard *newer* >>>>>> certs, ones with higher serials. Is it documented behavior or does >>>>>> Dogtag just happen to do that? >>>>> >>>>> It isn't documented anywhere I could find, it is just what dogtag >>>>> returns >>>>> >>>>>> I wonder how other people run their tests. This solution looks >>>>>> like it >>>>>> could break easily if people do something differently :( >>>>>> >>>>>> I'm not sure how to solve this properly. Perhaps not using >>>>>> Declarative, >>>>>> and checking "by hand" that the wanted certs are in the response and >>>>>> the >>>>>> unwanted ones are not, would work better. >>>>> >>>>> I ended up switching the test class. It is not a very fine-grained set >>>>> of tests, mostly searching with limits and verifying that we fall >>>>> within >>>>> a reasonable range, but it is better than nothing. >>>>> >>>>> rob >>>> >>>> This works much better, thanks! Just two nitpicks now. >>>> >>>> The patch doesn't apply well, there's a conflict in VERSION and some >>>> added trailing whitespace, >>>> >>>> AFAIK this would be the only (first?) test that relies on Nose's >>>> ordering of test modules -- tests 0011 and 0030 rely on the other cert >>>> tests running first. Please at least mention that in a comment. Or >>>> better, move class test_cert_find to test_cert.py >>>> >>>> >>> >>> Rebased the patch and removed whitespace. >>> >>> I went ahead and combined this with the existing test_cert file. >>> Originally test_cert was only tested against lite-server but since it >>> works against a live dogtag server too it makes sense to combine them. >>> >>> I improved the set up documentation a little bit and tried to handle all >>> the different configurations that one might see so that this should be >>> runnable against either a live server or the lite-server for both the >>> selfsign and dogtag backends. >>> >>> This relies on the user configuring ~/.ipa/default.conf to match the >>> remote server. There is no way from a client to know what kind of CA >>> backend a server is running. >>> >>> rob >> >> And the patch. >> >> rob >> > > ACK, thank you! > pushed to master and ipa-3-1 From rcritten at redhat.com Tue Feb 19 16:51:19 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 11:51:19 -0500 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc In-Reply-To: <20130219122037.GM476@redhat.com> References: <511A6D60.6030300@redhat.com> <20130213141221.GD476@redhat.com> <20130219122037.GM476@redhat.com> Message-ID: <5123AD87.2090800@redhat.com> Alexander Bokovoy wrote: > On Wed, 13 Feb 2013, Alexander Bokovoy wrote: >> On Tue, 12 Feb 2013, Ana Krivokapic wrote: >>> Add new LDAP container to store the list of domains associated with IPA >>> realm. >>> Add two new ipa commands (ipa realmdomains-show and ipa >>> realmdomains-mod) to allow manipulation of the list of realm domains. >>> Unit test file covering these new commands was added. >>> >>> https://fedorahosted.org/freeipa/ticket/2945 >> ACK, works perfectly. >> >> We need to decide on the questions still open in the >> http://www.freeipa.org/page/V3/Realm_Domains but the decision should not >> prevent this work from being committed. > Committed to master along with a one-liner that limits ipasam to look > only at $SUFFIX with base scope when searching for its own domain name > as now we use domainRelatedObject in two different places and previous > subtree search now gives too wide answer. > Pushed patch (and 1-liner to ipa-3-1) rob From rcritten at redhat.com Tue Feb 19 18:32:27 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 13:32:27 -0500 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades In-Reply-To: <5123A7A7.1020100@redhat.com> References: <5122965B.3030909@redhat.com> <5123A7A7.1020100@redhat.com> Message-ID: <5123C53B.7030002@redhat.com> Jan Cholasta wrote: > Hi, > > On 18.2.2013 22:00, Rob Crittenden wrote: >> An objectclass and attribute are not being added on upgrades. Missing >> these causes the UI to not work. >> >> I also noticed a typo in the ordering of a number of the trust >> attributes so fix those as well. >> >> rob >> > > The patch looks good, but I think errors like this will pop up from time > to time, because we have to maintain the same thing in two places - the > installation LDIFs and update files. Maybe we should start thinking > about merging these two somehow, e.g. using the LDIFs for both > installation and updates, with directives for the updater in specially > formatted comments. > > Honza > This idea came up long, long ago when we first added the updater very early in v2. The problem, as I recall, is that some schema is needed during the install so we need to ship it in ldif format, and the idea of splitting it didn't appeal to us. So perhaps what we should endeavor to do is add all new schema via updates and only update the schema files themselves if the schema is needed for a fresh install (since updates are done last). This also puts more schema into 99user.ldif which may or may not be desirable. rob From simo at redhat.com Tue Feb 19 19:23:17 2013 From: simo at redhat.com (Simo Sorce) Date: Tue, 19 Feb 2013 14:23:17 -0500 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades In-Reply-To: <5123C53B.7030002@redhat.com> References: <5122965B.3030909@redhat.com> <5123A7A7.1020100@redhat.com> <5123C53B.7030002@redhat.com> Message-ID: <1361301797.12328.382.camel@willson.li.ssimo.org> On Tue, 2013-02-19 at 13:32 -0500, Rob Crittenden wrote: > Jan Cholasta wrote: > > Hi, > > > > On 18.2.2013 22:00, Rob Crittenden wrote: > >> An objectclass and attribute are not being added on upgrades. Missing > >> these causes the UI to not work. > >> > >> I also noticed a typo in the ordering of a number of the trust > >> attributes so fix those as well. > >> > >> rob > >> > > > > The patch looks good, but I think errors like this will pop up from time > > to time, because we have to maintain the same thing in two places - the > > installation LDIFs and update files. Maybe we should start thinking > > about merging these two somehow, e.g. using the LDIFs for both > > installation and updates, with directives for the updater in specially > > formatted comments. > > > > Honza > > > > This idea came up long, long ago when we first added the updater very > early in v2. The problem, as I recall, is that some schema is needed > during the install so we need to ship it in ldif format, and the idea of > splitting it didn't appeal to us. > > So perhaps what we should endeavor to do is add all new schema via > updates and only update the schema files themselves if the schema is > needed for a fresh install (since updates are done last). > > This also puts more schema into 99user.ldif which may or may not be > desirable. Ron another option is to keep putting all updates only in schema files, and then have the updater "validate" the schema files. Validation would be: 1. Download schema from server (we already do this in the framework so it comes for free) 2. parse the schema files and check if each attribute and objectclass is present and in the correct form. 3. if any attribute is missing, we add it 4. if any attribute has been changed, we change it 5. same for object classes. This would allow us to keep everything just in schema files, and for now only updates would end up in 99.ldif I know there is also work in 389ds to improve schema validation and handling, so there is a chance in future we will have online interfaces to put data in multiple files w/o lumping everything in 99.ldif So by keeping stuff in schema files rather than arbitrary update files we are also sort of future proof. Finally keeping data in schema files instead of spreading it in updates should make it easier to keep an eye on the whole schema. The main issue I see is that this approach needs new code to analyze and compare schema files, however that shouldn't be overly hard. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Tue Feb 19 19:37:26 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 14:37:26 -0500 Subject: [Freeipa-devel] [PATCH 0027] Add checks for SELinux in install scripts In-Reply-To: <510FE816.6040609@redhat.com> References: <51094671.30701@redhat.com> <51095135.4040501@redhat.com> <510FD1F6.60901@redhat.com> <510FE816.6040609@redhat.com> Message-ID: <5123D476.5000108@redhat.com> Tomas Babej wrote: > On 02/04/2013 04:21 PM, Rob Crittenden wrote: >> Tomas Babej wrote: >>> On 01/30/2013 05:12 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> The checks make sure that SELinux is: >>>> - installed and enabled (on server install) >>>> - installed and enabled OR not installed (on client install) >>>> >>>> Please note that client installs with SELinux not installed are >>>> allowed since freeipa-client package has no dependency on SELinux. >>>> (any objections to this approach?) >>>> >>>> The (unsupported) option --allow-no-selinux has been added. It can >>>> used to bypass the checks. >>>> >>>> Parts of platform-dependant code were refactored to use newly added >>>> is_selinux_enabled() function. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3359 >>>> >>>> Tomas >>> >>> I forgot to edit the man pages. Thanks Rob! >>> >>> Updated patch attached. >>> >>> Tomas >> >> After a bit of off-line discussion I don't think we're quite ready yet >> to require SELinux by default on client installations (even with a >> flag to work around it). The feeling is this would be disruptive to >> existing automation. >> >> Can you still do the check but not enforce it, simply display a big >> warning if SELinux is disabled? >> >> rob >> > > Sure, here is the updated patch. > > I edited the commit message, RFE description and man pages according to > the new behaviour. > > Tomas The patch looks good, I'm just wondering about one thing. The default value for is_selinux_enabled() is True in ipapython/services.py.in. So this means that any non-Red Hat/non-Fedora system, by default, is going to assume that SELinux is enabled. My hesitation has to when we call check_selinux_status(). It may incorrectly error out. I suspect that the user would have to work around this using --allow-selinux-disabled but this wouldn't make a lot of sense since they actually do have SELinux disabled. What do you think? rob From rcritten at redhat.com Tue Feb 19 20:46:45 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 15:46:45 -0500 Subject: [Freeipa-devel] [PATCH] 0002 Add missing error message when adding duplicate external member to group In-Reply-To: <50EEE184.6080804@redhat.com> References: <50EEE184.6080804@redhat.com> Message-ID: <5123E4B5.2060702@redhat.com> Ana Krivokapic wrote: > When adding a duplicate member to a group, an error message is issued, > informing the user that the entry is already a member of the group. This > message was missing in case of an external member. > > Ticket: https://fedorahosted.org/freeipa/ticket/3254 This works ok but the sister command, group-remove-member, has the same problem. Can you add a fix there as well? I don't know if there is a way to add a unit test for this since the external member is validated meaning we'd need to set up trusts as well. It might be nice to have an optional test that can be run when a trust is configured to avoid regressions. rob From rcritten at redhat.com Tue Feb 19 21:19:05 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 16:19:05 -0500 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <510FD21B.7070403@redhat.com> References: <51011490.10101@redhat.com> <510FD21B.7070403@redhat.com> Message-ID: <5123EC49.3070508@redhat.com> Martin Kosek wrote: > On 01/24/2013 12:01 PM, Martin Kosek wrote: >> When user tries to perform any action requiring communication with >> trusted domain, IPA server tries to retrieve a trust secret on his >> behalf to be able to establish the connection. This happens for >> example during group-add-member command when external user is >> being resolved in the AD. >> >> When user is not member of Trust admins group, the retrieval crashes >> and reports internal error. Catch this exception and rather report >> properly formatted ACIError. >> >> ---- >> >> I hit this error after updating to the latest FreeIPA version with the AD CVE >> fixed. >> >> Martin >> > > I filed a ticket to not loose this fix and patch. Attaching an updated patch > with ticket URL in description. > > Martin > The patch fixes the problem but the error is untranslated: member group: AD\Domain Admins: Insufficient access: Gettext('communication with trusted domains is allowed for Trusts administrator group members only', domain='ipa', localedir=None) rob From rcritten at redhat.com Tue Feb 19 21:33:59 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 16:33:59 -0500 Subject: [Freeipa-devel] [PATCHES 0031-0032] Improve HBAC rule handling in selinuxusermap-add/mod/find In-Reply-To: <511519C4.30907@redhat.com> References: <511294EB.9050805@redhat.com> <5112A797.805@redhat.com> <511519C4.30907@redhat.com> Message-ID: <5123EFC7.9020708@redhat.com> Tomas Babej wrote: > On 02/06/2013 07:57 PM, Rob Crittenden wrote: >> Tomas Babej wrote: >>> Hi, >>> >>> this pair of patches improves HBAC rule handling in selinuxusermap >>> commands. >>> >>> Patch 0031 deals with: >>> https://fedorahosted.org/freeipa/ticket/3349 >>> >>> Patch 0032 takes care of: >>> https://fedorahosted.org/freeipa/ticket/3348 >>> >>> and is to be applied on top of Patch 0031. >>> >>> See commit messages for detailed info. >>> >>> Tomas >>> >> >> ACK for patch 0032. >> >> For patch 0031 we can't change the data type of an existing attribute. >> It will break backwards compatibility. Can you test with an older >> client to see if it cares (it may not care about the name of the >> type). If older clients will work then this is probably ok. >> >> I gather that seealso detected as a DN attribute and converted into a >> DN class and this is blowing up the Str validator? >> > Yes, that was exactly the case. >> rob > > I added a workaround for older client versions, tested it with > freeipa-client/admintools 2.2, works as expeceted. > However, this only should be issue if there is older admintools package > on the client than on the server. > > Outline is such as follows: I added a new flag for DNParam seelalso > attribute, called 'allow_malformed' that allows any string to be passed > to DNParam. Its value gets wrapped in 'malformed=yes,value='. > This allows to parse out the string in selinuxusermap-add/mod > pre_callback out of the DN and search for the rule with such name so > that it's DN gets in LDAP instead. > > Updated patch attached. > > Tomas I like where you're going with this, just a couple of comments: 1. Should we come up with a more universal name for allow_malformed? Is this something that we should allow at a higher level? I was thinking allow_raw, or allow_non_dn, or something like that. 2. I think that if a bad dn is passed in as a Str the conversion into a DN won't be handled: + if 'allow_malformed' in self.flags: + dn = DN(('malformed','yes'),('value',value)) Should this be wrapped in a try/except to raise a ConversionError if it fails? rob From rcritten at redhat.com Tue Feb 19 21:57:48 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 16:57:48 -0500 Subject: [Freeipa-devel] [PATCH 0033] Prevent changing protected group's name using --setattr In-Reply-To: <511BBC60.8070601@redhat.com> References: <5118C555.6090009@redhat.com> <511BBC60.8070601@redhat.com> Message-ID: <5123F55C.7070509@redhat.com> Petr Viktorin wrote: > On 02/11/2013 11:17 AM, Tomas Babej wrote: >> Hi, >> >> The name of any protected group now cannot be changed by modifing >> the cn attribute using --setattr. Unit tests have been added to >> make sure there is no regression. >> >> https://fedorahosted.org/freeipa/ticket/3354 >> >> Tomas >> > > We need a better general way to validate getattr/setattr on known > atributes, but works well for now. ACK > > Pushed to master and ipa-3-1 rob From rcritten at redhat.com Tue Feb 19 22:06:58 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 17:06:58 -0500 Subject: [Freeipa-devel] [PATCH] 0180 Check SSH connection in ipa-replica-conncheck In-Reply-To: <51222213.7080204@redhat.com> References: <511E5120.6050604@redhat.com> <511E567B.3070307@redhat.com> <511E650D.3050100@redhat.com> <511E8A08.2050002@redhat.com> <51222213.7080204@redhat.com> Message-ID: <5123F782.6080900@redhat.com> Petr Viktorin wrote: > On 02/15/2013 08:18 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 02/15/2013 04:38 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> ipa-replica-conncheck ran SSH in quiet mode, probably to suppress a >>>>> message about connecting to an unknown host. This made it hard to >>>>> debug >>>>> connection errors. >>>>> >>>>> I didn't find a way to separate SSH output from the output of the >>>>> called >>>>> command, I decided to try an additional SSH connection before calling >>>>> conncheck. SSH is set to verbose and if it fails the errors get >>>>> printed >>>>> out. Also, the host is added to a temporary known_hosts file. >>>>> The second SSH is called without the -q flag so errors/warnings are >>>>> not >>>>> lost even if the command fails. The temporary known_hosts file is used >>>>> so the unknown host warning doesn't appear here. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/3402 >>>> >>>> The general procedure looks good, I don't think we should hardcode the >>>> path to ssh. ipautil.run() overrides the current environment so we >>>> should be able to safely run just 'ssh'. >>>> >>>> We eventually need a cross-platform way of locating system binaries. >>>> >>>> The hardcoded path to ipa-replica-conncheck is probably ok since we >>>> provide that binary ourselves. >>>> >>>> rob >>> >>> Changed, thanks. >>> >> >> Looks and works well. I just have one final question. Should remote_addr >> and temp_known_hosts be passed in as args? They are basically globals >> but it is obvious where they came from, so not really a NAK, just a >> question. >> >> rob > > Well, it's a style issue so I only have a bunch of rule-of-thumb > handwaving to justify the decision... Here goes: > > Closing over the variables in main() doesn't have drawbacks of globals: > nothing outside main() can see/modify them and it doesn't keep main() > from (indirectly) calling itself (this particular main() probably won't > need to do that, but in general it's good to avoid this kind of surprises). > > Passing the variables in as arguments would only make sense if run_ssh > was a top-level function, otherwise you have to worry about the right > order/names of arguments and gain nothing. > > The nested function makes it clear that this is just a quick way to keep > things DRY, not a full-featured generic SSH caller. A top-level > "run_ssh" helper shouldn't unconditionally disable StrictHostKeychecking > for example. The code is tied to the caller functionally, so I kept it > tied syntactically as well. > > > On the other hand, nested functions are "magic" that you need a good > reason for. I think the above is just reason enough, but I don't feel > that strongly about it. Here's a version with top-level run_ssh, push it > if it seems better to you. > Works for me. I pushed the second patch to master and ipa-3-1. It is clear enough where the variables come from. We can revisit in the future if this gets messy. rob From rcritten at redhat.com Wed Feb 20 03:43:43 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 19 Feb 2013 22:43:43 -0500 Subject: [Freeipa-devel] Backup and Restore design Message-ID: <5124466F.9030800@redhat.com> I've looked into some basic backup and restore procedures for IPA. My findings are here: http://freeipa.org/page/V3/Backup_and_Restore This is in support of ticket https://fedorahosted.org/freeipa/ticket/3128 rob From jdennis at redhat.com Wed Feb 20 04:12:00 2013 From: jdennis at redhat.com (John Dennis) Date: Tue, 19 Feb 2013 23:12:00 -0500 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <5124466F.9030800@redhat.com> References: <5124466F.9030800@redhat.com> Message-ID: <51244D10.8020700@redhat.com> On 02/19/2013 10:43 PM, Rob Crittenden wrote: > I've looked into some basic backup and restore procedures for IPA. My > findings are here: http://freeipa.org/page/V3/Backup_and_Restore Good write up Rob! It seems to me there are two critical sub-issues to solve first that could benefit us in the long run anyway. 1) Replacing certs. This has been a pain point for many admins who for various reasons now have invalid certs and need to redeploy all certs to bring back up our integrated web of services. Can this be a separate work item? It would be generally useful in contexts other than backups. 2) Tracking every file we modify. Aren't we already a good way there with the sysrestore functionality already in use in the install tools? It is just a matter of enhancing it a bit and being diligent about making sure it's used in every modification we make? Actually I'm not thinking we track every modification, rather we keep a manifest of everything we've touched and then just snapshot those files during backup, this would capture any modifications not made by us but perhaps are critical to restoring a working system (i.e. puppet modifications). The sysrestore module could provide the manifest, right? John -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Wed Feb 20 08:15:40 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 09:15:40 +0100 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <5123EC49.3070508@redhat.com> References: <51011490.10101@redhat.com> <510FD21B.7070403@redhat.com> <5123EC49.3070508@redhat.com> Message-ID: <5124862C.5040400@redhat.com> On 02/19/2013 10:19 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 01/24/2013 12:01 PM, Martin Kosek wrote: >>> When user tries to perform any action requiring communication with >>> trusted domain, IPA server tries to retrieve a trust secret on his >>> behalf to be able to establish the connection. This happens for >>> example during group-add-member command when external user is >>> being resolved in the AD. >>> >>> When user is not member of Trust admins group, the retrieval crashes >>> and reports internal error. Catch this exception and rather report >>> properly formatted ACIError. >>> >>> ---- >>> >>> I hit this error after updating to the latest FreeIPA version with the AD CVE >>> fixed. >>> >>> Martin >>> >> >> I filed a ticket to not loose this fix and patch. Attaching an updated patch >> with ticket URL in description. >> >> Martin >> > > > The patch fixes the problem but the error is untranslated: > > member group: AD\Domain Admins: Insufficient access: Gettext('communication > with trusted domains is allowed for Trusts administrator group members only', > domain='ipa', localedir=None) > > rob I think this is just because this string is not in our ipa.pot file yet (will be when we do Transifex refresh"). Martin From tbabej at redhat.com Wed Feb 20 09:31:25 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 20 Feb 2013 10:31:25 +0100 Subject: [Freeipa-devel] [PATCH 0035] Use default.conf as flag of IPA client being installed Message-ID: <512497ED.6080200@redhat.com> Hi, When installing / uninstalling IPA client, the checks that determine whether IPA client is installed now take the existence of /etc/ipa/default.conf into consideration. The client will not uninstall unless either something is backed up or /etc/ipa/default.conf file does exist. The client will not install if something is backed up or default.conf file does exist (unless it's installation on master). https://fedorahosted.org/freeipa/ticket/3331 Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0035-Use-default.conf-as-flag-of-IPA-client-being-install.patch Type: text/x-patch Size: 2102 bytes Desc: not available URL: From mkosek at redhat.com Wed Feb 20 09:37:24 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 10:37:24 +0100 Subject: [Freeipa-devel] [PATCH 0024] Make options checks in idrange-add/mod consistent In-Reply-To: <50C71A55.4040107@redhat.com> References: <50C71A55.4040107@redhat.com> Message-ID: <51249954.4050304@redhat.com> On 12/11/2012 12:34 PM, Tomas Babej wrote: > Hi, > > **NOTE**: This is to be applied on top of my PATCH 0021 > > Option checks in idrange-add/mod have been made consistent. > Both now enforce the following checks: > - dom_sid and secondary_rid_base cannot be used together > - rid_base must be used together if dom_rid is set > - secondary_rid_base and rid_base must be used together > if dom_rid is not set > cat > Unit test for third check has been added. > > http://fedorahosted.org/freeipa/ticket/3170 > > Tomas I think this needs a rebase. Martin From akrivoka at redhat.com Wed Feb 20 10:03:26 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Wed, 20 Feb 2013 11:03:26 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <512219D7.9000909@redhat.com> References: <511E44A9.9090509@redhat.com> <51221027.7050803@redhat.com> <20130218114750.GC27418@localhost.localdomain> <512219D7.9000909@redhat.com> Message-ID: <51249F6E.7030005@redhat.com> On 02/18/2013 01:08 PM, Martin Kosek wrote: > On 02/18/2013 12:47 PM, Sumit Bose wrote: >> On Mon, Feb 18, 2013 at 12:27:35PM +0100, Petr Spacek wrote: >>> On 15.2.2013 15:22, Ana Krivokapic wrote: >>>> Hello, >>>> >>>> The .isalpha() check in validate_domain_name() was too strict, >>>> causing some commands like ipa dnsrecord-add to fail. >>>> >>>> https://fedorahosted.org/freeipa/ticket/3385 >>> I would add --force option rather than removing whole check, if it's possible. >>> >>> Would it be possible to mention RFC in the error message? Something >>> like _('top level domain label must be alphabetic (RFC 1123 section >>> 2.1)') >>> ? >>> >>> IMHO it is handy, because it educates users. >> The problem is that this check is always done on the last component of >> the domain_name even if it is just a sub-domain of the FreeIPA domain, >> where e.g. numbers are valid characters. >> >> At the beginning of validate_domain_name() a trailing '.' is stripped >> away. iirc the trailing '.' is an indication for a complete, fully >> qualified name. Would it work if the presence of the trailing '.' is >> saved and the check is only done if there was a '.'? >> >> bye, >> Sumit >> > Sure. Though I am now not 100% sure that some IPA functions do not use this > validator with a fqdn hostname without trailing dot. If not, I am for fixing > this function as Sumit and Petr suggested. > > Martin > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel After some thought, I decided to change the approach. As pointed out by Sumit, the problem was that the validate_domain_name() function did not distinguish between fqdn and non-fqdn domains (subdomains of the IPA domain). The trailing dot is not a clear indication either, because some IPA functions use this validator with an fqdn without the trailing dot. To fix this, I introduced an additional parameter to this function - a flag which indicates whether the domain name is an fqdn or not. The is .isalpha() check is then performed only in the case of an fqdn. I also improved the error message to mention the relevant RFC, as suggested by Petr. -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akrivoka-0006-02-Improve-domain-name-validation.patch Type: text/x-patch Size: 2293 bytes Desc: not available URL: From tbabej at redhat.com Wed Feb 20 10:20:28 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 20 Feb 2013 11:20:28 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <50D444E1.9050502@redhat.com> References: <50D444E1.9050502@redhat.com> Message-ID: <5124A36C.70501@redhat.com> On 12/21/2012 12:15 PM, Tomas Babej wrote: > Hi, > > Sending updated and rebased versions of patches 0024 and 0025. > > Tomas > > Sending rebased version, these got quite rotten. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0024-4-Make-options-checks-in-idrange-add-mod-consistent.patch Type: text/x-patch Size: 8990 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0025-4-Add-trusted-domain-range-objectclass-when-using-idra.patch Type: text/x-patch Size: 1325 bytes Desc: not available URL: From tbabej at redhat.com Wed Feb 20 10:49:19 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 20 Feb 2013 11:49:19 +0100 Subject: [Freeipa-devel] [PATCH 0027] Add checks for SELinux in install scripts In-Reply-To: <5123D476.5000108@redhat.com> References: <51094671.30701@redhat.com> <51095135.4040501@redhat.com> <510FD1F6.60901@redhat.com> <510FE816.6040609@redhat.com> <5123D476.5000108@redhat.com> Message-ID: <5124AA2F.1040009@redhat.com> On Tue 19 Feb 2013 08:37:26 PM CET, Rob Crittenden wrote: > Tomas Babej wrote: >> On 02/04/2013 04:21 PM, Rob Crittenden wrote: >>> Tomas Babej wrote: >>>> On 01/30/2013 05:12 PM, Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> The checks make sure that SELinux is: >>>>> - installed and enabled (on server install) >>>>> - installed and enabled OR not installed (on client install) >>>>> >>>>> Please note that client installs with SELinux not installed are >>>>> allowed since freeipa-client package has no dependency on SELinux. >>>>> (any objections to this approach?) >>>>> >>>>> The (unsupported) option --allow-no-selinux has been added. It can >>>>> used to bypass the checks. >>>>> >>>>> Parts of platform-dependant code were refactored to use newly added >>>>> is_selinux_enabled() function. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/3359 >>>>> >>>>> Tomas >>>> >>>> I forgot to edit the man pages. Thanks Rob! >>>> >>>> Updated patch attached. >>>> >>>> Tomas >>> >>> After a bit of off-line discussion I don't think we're quite ready yet >>> to require SELinux by default on client installations (even with a >>> flag to work around it). The feeling is this would be disruptive to >>> existing automation. >>> >>> Can you still do the check but not enforce it, simply display a big >>> warning if SELinux is disabled? >>> >>> rob >>> >> >> Sure, here is the updated patch. >> >> I edited the commit message, RFE description and man pages according to >> the new behaviour. >> >> Tomas > > The patch looks good, I'm just wondering about one thing. The default > value for is_selinux_enabled() is True in ipapython/services.py.in. > > So this means that any non-Red Hat/non-Fedora system, by default, is > going to assume that SELinux is enabled. > > My hesitation has to when we call check_selinux_status(). It may > incorrectly error out. I suspect that the user would have to work > around this using --allow-selinux-disabled but this wouldn't make a > lot of sense since they actually do have SELinux disabled. Yes, you're right. And the error message would not even be helpful since it would tell the user to install policycoreutils package. This would be the case both with server and client installs when selinux would not be installed at all. > What do you think? > > rob Well we have 2 options as I see it: 1.) We can either return None as default, and add checks to check_selinux_status, restore_context and install scripts that would ensure that we behave properly when is_selinux_enabled() is not implemented. 2.) We can remove the default value, since it would cause forementioned crash and add comment that this function needs to be implemented properly in every platform file. I'm probably for option 2, there's no need to clutter the code with checks that compensate for improper platform file implementations. Tomas From pviktori at redhat.com Wed Feb 20 11:30:48 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 20 Feb 2013 12:30:48 +0100 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <5124862C.5040400@redhat.com> References: <51011490.10101@redhat.com> <510FD21B.7070403@redhat.com> <5123EC49.3070508@redhat.com> <5124862C.5040400@redhat.com> Message-ID: <5124B3E8.20709@redhat.com> On 02/20/2013 09:15 AM, Martin Kosek wrote: > On 02/19/2013 10:19 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 01/24/2013 12:01 PM, Martin Kosek wrote: >>>> When user tries to perform any action requiring communication with >>>> trusted domain, IPA server tries to retrieve a trust secret on his >>>> behalf to be able to establish the connection. This happens for >>>> example during group-add-member command when external user is >>>> being resolved in the AD. >>>> >>>> When user is not member of Trust admins group, the retrieval crashes >>>> and reports internal error. Catch this exception and rather report >>>> properly formatted ACIError. >>>> >>>> ---- >>>> >>>> I hit this error after updating to the latest FreeIPA version with the AD CVE >>>> fixed. >>>> >>>> Martin >>>> >>> >>> I filed a ticket to not loose this fix and patch. Attaching an updated patch >>> with ticket URL in description. >>> >>> Martin >>> >> >> >> The patch fixes the problem but the error is untranslated: >> >> member group: AD\Domain Admins: Insufficient access: Gettext('communication >> with trusted domains is allowed for Trusts administrator group members only', >> domain='ipa', localedir=None) >> >> rob > > I think this is just because this string is not in our ipa.pot file yet (will > be when we do Transifex refresh"). > > Martin > I don't have AD so I can't investigate, but this problem is usually due to the error being converted to string instead of using the strerror attribute. -- Petr? From tbabej at redhat.com Wed Feb 20 11:31:10 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 20 Feb 2013 12:31:10 +0100 Subject: [Freeipa-devel] [PATCHES 0031-0032] Improve HBAC rule handling in selinuxusermap-add/mod/find In-Reply-To: <5123EFC7.9020708@redhat.com> References: <511294EB.9050805@redhat.com> <5112A797.805@redhat.com> <511519C4.30907@redhat.com> <5123EFC7.9020708@redhat.com> Message-ID: <5124B3FE.7000407@redhat.com> On 02/19/2013 10:33 PM, Rob Crittenden wrote: > Tomas Babej wrote: >> On 02/06/2013 07:57 PM, Rob Crittenden wrote: >>> Tomas Babej wrote: >>>> Hi, >>>> >>>> this pair of patches improves HBAC rule handling in selinuxusermap >>>> commands. >>>> >>>> Patch 0031 deals with: >>>> https://fedorahosted.org/freeipa/ticket/3349 >>>> >>>> Patch 0032 takes care of: >>>> https://fedorahosted.org/freeipa/ticket/3348 >>>> >>>> and is to be applied on top of Patch 0031. >>>> >>>> See commit messages for detailed info. >>>> >>>> Tomas >>>> >>> >>> ACK for patch 0032. >>> >>> For patch 0031 we can't change the data type of an existing attribute. >>> It will break backwards compatibility. Can you test with an older >>> client to see if it cares (it may not care about the name of the >>> type). If older clients will work then this is probably ok. >>> >>> I gather that seealso detected as a DN attribute and converted into a >>> DN class and this is blowing up the Str validator? >>> >> Yes, that was exactly the case. >>> rob >> >> I added a workaround for older client versions, tested it with >> freeipa-client/admintools 2.2, works as expeceted. >> However, this only should be issue if there is older admintools package >> on the client than on the server. >> >> Outline is such as follows: I added a new flag for DNParam seelalso >> attribute, called 'allow_malformed' that allows any string to be passed >> to DNParam. Its value gets wrapped in 'malformed=yes,value='. >> This allows to parse out the string in selinuxusermap-add/mod >> pre_callback out of the DN and search for the rule with such name so >> that it's DN gets in LDAP instead. >> >> Updated patch attached. >> >> Tomas > > I like where you're going with this, just a couple of comments: > > 1. Should we come up with a more universal name for allow_malformed? > Is this something that we should allow at a higher level? I was > thinking allow_raw, or allow_non_dn, or something like that. To me, allow_non_dn sounds is just as specific as allow_malformed, they both refer to DN specifically. I'd go with allow_raw, if need for such pattern may eventually arise for other parameter classes than DNParam. What do you mean by higher level, turning this hack into a feature Param class? I don't see how this would work, each parameter class that implements its own type validation as DNParam needs to override _convert_scalar(). And in every such class we would need to wrap our raw value so that it is represented in the type of this parameter, as we do with DN(('malformed','yes'),('value',value)) now. Maybe we could skip type validation in _convert_scalar default implementation or catch the error raised somehow, and let the type be invalid, but I'm not aware of the consenquences. I would need to investigate. Wouldn't it cause failure deeper in the framework? Or did you by higher level mean simply picking a more general name for the flag so it can be reused in other parameter classes with the same name? > > 2. I think that if a bad dn is passed in as a Str the conversion into > a DN won't be handled: > > + if 'allow_malformed' in self.flags: > + dn = DN(('malformed','yes'),('value',value)) > > Should this be wrapped in a try/except to raise a ConversionError if > it fails? Yes, thanks for that catch. > > rob Tomas From mkosek at redhat.com Wed Feb 20 11:46:09 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 12:46:09 +0100 Subject: [Freeipa-devel] [PATCHES] 0022, 0115-0116 Make Sudo commands case-sensitive In-Reply-To: <50CF3583.3060700@redhat.com> References: <4F58D6E8.7040202@redhat.com> <50CF3583.3060700@redhat.com> Message-ID: <5124B781.5030304@redhat.com> On 12/17/2012 04:08 PM, Petr Viktorin wrote: > https://fedorahosted.org/freeipa/ticket/2482 > > The first two patches are rebased from what I sent back in March; the third > fixes ACIs using targetfilter. > I finally got to your patches. Generally, everything worked like charm, I have just few minor comments: 0022: - patch needs a rebase - patch description is confusing, we are talking about RDN "sudocmd" and not "CN" 0115: I would optimize the LDAP calls a little: 1) Use sudorule base DN as a base for the LDAP search 2) Do not call LDAP search twice, but just once and then collect the result. Now you use 2 LDAP searches with following filters: (&(objectClass=ipasudorule)(memberallowcmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)) (&(objectClass=ipasudorule)(memberdenycmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)) We can do just one LDAP search with this filter: (&(objectClass=ipasudorule)(|(memberallowcmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)(memberdenycmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test))) 0116: - patch description needs amending: s/CN/SUDOCMD/ Martin From pviktori at redhat.com Wed Feb 20 12:03:27 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 20 Feb 2013 13:03:27 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <512387D1.60603@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> <512387D1.60603@redhat.com> Message-ID: <5124BB8F.80803@redhat.com> On 02/19/2013 03:10 PM, Jan Cholasta wrote: > On 1.2.2013 15:38, Petr Viktorin wrote: >> Alright, I renamed get_single to single_value(). >> >> I also rebased to current master. >> > > Patch 152: > > + def single_value(self, name, default=_missing): > + values = self.get(name, [default]) > + if len(values) != 1: > + raise ValueError( > + '%s has %s values, one expected' % (name, len(values))) > + if values[0] is _missing: > + raise KeyError(name) > + return values[0] > > I would prefer if you used __getitem__() instead of get() and re-raised > KeyError if default is _missing. Also, until we disallow non-lists, we > need to check if values actually is list. I.e., do something like this: > > + def single_value(self, name, default=_missing): > + try: > + values = super(LDAPEntry, > self).__getitem__(self._get_attr_name(name)) > + except KeyError: > + if default is _missing: > + raise > + return default > + if not isinstance(values, list): > + return values > + if len(values) != 1: > + raise ValueError( > + '%s has %s values, one expected' % (name, len(values))) > + return values[0] Updated, thanks. _get_attr_name is only added in your patch 99, so I used _attr_name here and modified your patch. I wrote some tests for single_value, and while I was at it I added tests for a few other LDAPEntry methods as well. I've also split things up into more testcases. Including as patch 0181. > Patch 159: > > Like I said in my review of your patch 143, I think we should use DNs > instead of entries in delete_entry, so I think it would make sense to do > delete_entry(entry.dn) in this patch. I think that for symmetry with add_entry and update_entry, delete_entry should take entries too. If you already have the entry, having to type the extra ".dn" is not very intuitive. The proper thing to do would be a separate delete_by_dn(), but delete_entry() taking either entries or DNs works fine IMO. > Patch 160: > > I think you should do this (replace % with ,): > > + root_logger.debug( > + "failed to find mapping tree entry for %s", self.suffix) > Fixed, thanks. I've also rebased 174 and your patch 104 to current master. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0152-03-Introduce-LDAPEntry.single_value-for-getting-single-.patch Type: text/x-patch Size: 1851 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0160-02-Fix-typo-and-traceback-suppression-in-replication.py.patch Type: text/x-patch Size: 1172 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0174-03-Remove-some-uses-of-raw-python-ldap.patch Type: text/x-patch Size: 37062 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-104pviktori-Remove-DN-normalization-from-the-baseldap-plugin.patch Type: text/x-patch Size: 14615 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-99pviktori-Preserve-case-of-attribute-names-in-LDAPEntry.patch Type: text/x-patch Size: 10565 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0181-Improve-LDAPEntry-tests.patch Type: text/x-patch Size: 4950 bytes Desc: not available URL: From abokovoy at redhat.com Wed Feb 20 13:24:03 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 20 Feb 2013 15:24:03 +0200 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <5124A36C.70501@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> Message-ID: <20130220132403.GN476@redhat.com> On Wed, 20 Feb 2013, Tomas Babej wrote: >On 12/21/2012 12:15 PM, Tomas Babej wrote: >>Hi, >> >>Sending updated and rebased versions of patches 0024 and 0025. >> >>Tomas >> >> >Sending rebased version, these got quite rotten. Thanks for updating them. >@@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): > 'not be found. Please specify the SID directly ' > 'using dom-sid option.')) > >- try: >- (old_dn, old_attrs) = ldap.get_entry(dn, >- ['ipabaseid', >- 'ipaidrangesize', >- 'ipabaserid', >- 'ipasecondarybaserid']) >- except errors.NotFound: >- self.obj.handle_not_found(*keys) >+ if in_updated_attrs('ipanttrusteddomainsid'): >+ if in_updated_attrs('ipasecondarybaserid'): >+ raise errors.ValidationError(name='ID Range setup', >+ error=_('Options dom_sid and secondary_rid_base cannot ' >+ 'be used together')) Since we agreed to refer to options by their CLI name (--dom-sid and --secondary-rid-base) in the other patch, it makes sense to use it here too. >- if is_set('ipanttrusteddomainsid'): >- # Validate SID as the one of trusted domains >- self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >+ if not in_updated_attrs('ipabaserid'): >+ raise errors.ValidationError(name='ID Range setup', >+ error=_('Options dom_sid and rid_base must ' >+ 'be used together')) Same here. >+ # secondary base rid must be set if and only if base rid is set >+ if in_updated_attrs('ipasecondarybaserid') !=\ >+ in_updated_attrs('ipabaserid'): >+ raise errors.ValidationError(name='ID Range setup', >+ error=_('Options secondary_rid_base and rid_base must ' >+ 'be used together')) Same here. >+ dict( >+ desc='Try to modify ID range %r so it has only primary rid range set' % (testrange8), >+ command=('idrange_mod', [testrange8], >+ dict(ipabaserid=testrange8_base_rid)), >+ expected=errors.ValidationError( >+ name='ID Range setup', error='Options secondary_rid_base and rid_base must be used together'), >+ ), And synchronize error message here too. -- / Alexander Bokovoy From mkosek at redhat.com Wed Feb 20 14:12:28 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 15:12:28 +0100 Subject: [Freeipa-devel] [PATCH] Add delegation info to MS-PAC In-Reply-To: <1360273346.2299.15.camel@willson.li.ssimo.org> References: <1360273346.2299.15.camel@willson.li.ssimo.org> Message-ID: <5124D9CC.2070205@redhat.com> On 02/07/2013 10:42 PM, Simo Sorce wrote: > This information is not strictly required but is part of the MS-PAC > specification and I had some time to kill on the plane on my last trip > back. > > I tested it briefly with cross-realm trusts and it seem to work fine. > Neither IPA nor AD2012 complained when looking at PACs, do far. > > Simo. > Can we make a ticket for this one, so that it does not get lost in the swarm of freeipa-devel patches? (and so that we can triage it properly) Martin From simo at redhat.com Wed Feb 20 14:16:58 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 20 Feb 2013 09:16:58 -0500 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <5124466F.9030800@redhat.com> References: <5124466F.9030800@redhat.com> Message-ID: <1361369818.12328.428.camel@willson.li.ssimo.org> On Tue, 2013-02-19 at 22:43 -0500, Rob Crittenden wrote: > I've looked into some basic backup and restore procedures for IPA. My > findings are here: http://freeipa.org/page/V3/Backup_and_Restore Great summary! For the catastrofic failure scenario, should we mention how to put back a full failed and restored machine online ? I am thinking the restored server may be behind (even if only by a few entries) in the replication, so the CSNs in the other replicas will not match. I guess we should mention a full resync may be needed ? Or is there a way to bring back CSNs on replicas ? In the 'Returning to a good state' case, can we consider some split brain approach, were we sever replication and rebuild one server at a time ? Maybe we can think of a way to 'mark' all server as 'bad' so that on restore the replication agreements do not need to be changed but changes from 'bad' servers will not be accepted ? I guess this crosses also a request by someone to be able to 'pause' replication, would use the same mechanism I guess. Full system backup: in the first part it is said the process is offline, but in the 'LDAP' section you say ldapi is used, but that would mean DS is running ? Also are we sure we can get all data we need via ldapi ? Are we going to miss any "operational" attribute ? For restore are we sure we can reload data w/o alterations ? What about plugins ? will we have to disable all plugins during a restore ? For the open questions. Size of backup: I think we should make it easy to configure the system to use a custom directory to dump the backups. This way admins can make sure it is on a different partition/disk or even over NFS and that the backup will not fill up the disk on which DS is running. We should definitely allow to encrypt backup files, a gpg public key would be sufficient. For replica cases, maybe we can create a command that dumps the changelog from a good replica and then allow us to reply all changes that are missing from the backup to bring the server up to the last minute ? Simo. -- Simo Sorce * Red Hat, Inc * New York From tbabej at redhat.com Wed Feb 20 14:19:43 2013 From: tbabej at redhat.com (Tomas Babej) Date: Wed, 20 Feb 2013 15:19:43 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <20130220132403.GN476@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> <20130220132403.GN476@redhat.com> Message-ID: <5124DB7F.3020101@redhat.com> On Wed 20 Feb 2013 02:24:03 PM CET, Alexander Bokovoy wrote: > On Wed, 20 Feb 2013, Tomas Babej wrote: >> On 12/21/2012 12:15 PM, Tomas Babej wrote: >>> Hi, >>> >>> Sending updated and rebased versions of patches 0024 and 0025. >>> >>> Tomas >>> >>> >> Sending rebased version, these got quite rotten. > Thanks for updating them. > >> @@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): >> 'not be found. Please specify the SID >> directly ' >> 'using dom-sid option.')) >> >> - try: >> - (old_dn, old_attrs) = ldap.get_entry(dn, >> - ['ipabaseid', >> - 'ipaidrangesize', >> - 'ipabaserid', >> - 'ipasecondarybaserid']) >> - except errors.NotFound: >> - self.obj.handle_not_found(*keys) >> + if in_updated_attrs('ipanttrusteddomainsid'): >> + if in_updated_attrs('ipasecondarybaserid'): >> + raise errors.ValidationError(name='ID Range setup', >> + error=_('Options dom_sid and secondary_rid_base >> cannot ' >> + 'be used together')) > Since we agreed to refer to options by their CLI name (--dom-sid and > --secondary-rid-base) in the other patch, it makes sense to use it > here too. > > >> - if is_set('ipanttrusteddomainsid'): >> - # Validate SID as the one of trusted domains >> - >> self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >> >> + if not in_updated_attrs('ipabaserid'): >> + raise errors.ValidationError(name='ID Range setup', >> + error=_('Options dom_sid and rid_base must ' >> + 'be used together')) > Same here. > >> + # secondary base rid must be set if and only if base rid >> is set >> + if in_updated_attrs('ipasecondarybaserid') !=\ >> + in_updated_attrs('ipabaserid'): >> + raise errors.ValidationError(name='ID Range setup', >> + error=_('Options secondary_rid_base and rid_base >> must ' >> + 'be used together')) > Same here. > >> + dict( >> + desc='Try to modify ID range %r so it has only primary >> rid range set' % (testrange8), >> + command=('idrange_mod', [testrange8], >> + dict(ipabaserid=testrange8_base_rid)), >> + expected=errors.ValidationError( >> + name='ID Range setup', error='Options >> secondary_rid_base and rid_base must be used together'), >> + ), > And synchronize error message here too. > Thanks! Sending the updated patch 0024. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0024-5-Make-options-checks-in-idrange-add-mod-consistent.patch Type: text/x-patch Size: 8990 bytes Desc: not available URL: From simo at redhat.com Wed Feb 20 14:51:12 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 20 Feb 2013 09:51:12 -0500 Subject: [Freeipa-devel] [PATCH] Add delegation info to MS-PAC In-Reply-To: <5124D9CC.2070205@redhat.com> References: <1360273346.2299.15.camel@willson.li.ssimo.org> <5124D9CC.2070205@redhat.com> Message-ID: <1361371872.12328.433.camel@willson.li.ssimo.org> On Wed, 2013-02-20 at 15:12 +0100, Martin Kosek wrote: > On 02/07/2013 10:42 PM, Simo Sorce wrote: > > This information is not strictly required but is part of the MS-PAC > > specification and I had some time to kill on the plane on my last trip > > back. > > > > I tested it briefly with cross-realm trusts and it seem to work fine. > > Neither IPA nor AD2012 complained when looking at PACs, do far. > > > > Simo. > > > > Can we make a ticket for this one, so that it does not get lost in the swarm of > freeipa-devel patches? (and so that we can triage it properly) https://fedorahosted.org/freeipa/ticket/3442 Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Feb 20 15:20:04 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 20 Feb 2013 10:20:04 -0500 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <51244D10.8020700@redhat.com> References: <5124466F.9030800@redhat.com> <51244D10.8020700@redhat.com> Message-ID: <5124E9A4.9030200@redhat.com> John Dennis wrote: > On 02/19/2013 10:43 PM, Rob Crittenden wrote: >> I've looked into some basic backup and restore procedures for IPA. My >> findings are here: http://freeipa.org/page/V3/Backup_and_Restore > > Good write up Rob! > > It seems to me there are two critical sub-issues to solve first that > could benefit us in the long run anyway. > > 1) Replacing certs. This has been a pain point for many admins who for > various reasons now have invalid certs and need to redeploy all certs to > bring back up our integrated web of services. Can this be a separate > work item? It would be generally useful in contexts other than backups. This is definitely something we need, but yeah, it's a bit out of my scope here. There are lots of reasons to replace the CA but we don't really have any way to do that currently. I did find a sort of poor-man's way of replacing the CA and documented that, but I'm not sure I'd want to do that with live data. It was sort of a "gee, I wonder what would happen..." sort of thing. > 2) Tracking every file we modify. Aren't we already a good way there > with the sysrestore functionality already in use in the install tools? > It is just a matter of enhancing it a bit and being diligent about > making sure it's used in every modification we make? Actually I'm not > thinking we track every modification, rather we keep a manifest of > everything we've touched and then just snapshot those files during > backup, this would capture any modifications not made by us but perhaps > are critical to restoring a working system (i.e. puppet modifications). > The sysrestore module could provide the manifest, right? The problem is we also need to back up a lot of files we don't otherwise track (or probably want to). For example, the entire CA instance, the entire 389-ds instances, etc. My goal for the data backup was to be able to drop the file onto a brand new machine, with the right rpms installed and be able to do a restore and start everything up. To do that we have to restore a whole ton of things that we don't necessarily touch in our installer so is beyond the scope of sysrestore. That and we already herd cats. It would require us to be more responsible for files provided by other apps. So if 389-ds or dogtag decided to ship a new file, we'd have to update our sysrestore stuff to capture that, or we'd have to loop over the directories to make sure we had everything tracked. Thanks for the feedback. rob From rcritten at redhat.com Wed Feb 20 15:38:01 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 20 Feb 2013 10:38:01 -0500 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <1361369818.12328.428.camel@willson.li.ssimo.org> References: <5124466F.9030800@redhat.com> <1361369818.12328.428.camel@willson.li.ssimo.org> Message-ID: <5124EDD9.7050900@redhat.com> Simo Sorce wrote: > On Tue, 2013-02-19 at 22:43 -0500, Rob Crittenden wrote: >> I've looked into some basic backup and restore procedures for IPA. My >> findings are here: http://freeipa.org/page/V3/Backup_and_Restore > > Great summary! > > For the catastrofic failure scenario, should we mention how to put back > a full failed and restored machine online ? I am thinking the restored > server may be behind (even if only by a few entries) in the replication, > so the CSNs in the other replicas will not match. > I guess we should mention a full resync may be needed ? Or is there a > way to bring back CSNs on replicas ? Good questions. It depends on how long the machine was down and how many changes have happened. It is possible that one would want to do a full re-init. I'll add that to the design. > In the 'Returning to a good state' case, can we consider some split > brain approach, were we sever replication and rebuild one server at a > time ? Perhaps using a firewall, but then you run the risk of each of those servers accepting changes during the rebuild and you could end up with a lot of collisions, which sort of goes against the point of restoring to a known good state. The changelog is the key here. I'll have to ponder this one a bit, I'm a bit conflicted on the right approach. > Maybe we can think of a way to 'mark' all server as 'bad' so that on > restore the replication agreements do not need to be changed but changes > from 'bad' servers will not be accepted ? > I guess this crosses also a request by someone to be able to 'pause' > replication, would use the same mechanism I guess. AFAIK there is an option to pause replication now (at least in 1.3). What you can't do is drop the changelog AFAIK. That is the real problem. If you want to restore to a known state you need to drop all the changelog entries since that time. I'll check with the 389-ds team to see if that is possible. Since we know the time of the backup, we might be able to drop newer entries than that. > Full system backup: > in the first part it is said the process is offline, but in the 'LDAP' > section you say ldapi is used, but that would mean DS is running ? > Also are we sure we can get all data we need via ldapi ? Are we going to > miss any "operational" attribute ? The full backup is offline because it is just using tar. This is sort of a brute-force backup, copying bits from A to B. The data backup is online and creates a task in 389-ds to write the data and changelog to a file. It should write everything completely. We don't do an ldapsearch. I chose to not back up in ldif because this would back up just the data and not the changelog. The other advantage is that the db2bak format includes the indexes and ldif restore would require a rebuild. > For restore are we sure we can reload data w/o alterations ? What about > plugins ? will we have to disable all plugins during a restore ? Yes, it should be fine. I'm hoping that the version will help us with this, to prevent someone from restoring an ancient backup on a new system, for example (or the reverse). > For the open questions. > > Size of backup: > I think we should make it easy to configure the system to use a custom > directory to dump the backups. This way admins can make sure it is on a > different partition/disk or even over NFS and that the backup will not > fill up the disk on which DS is running. That's a good idea. I'll have to think about where we would configure that. Perhaps as an optional argument to the backup command. > We should definitely allow to encrypt backup files, a gpg public key > would be sufficient. Ok. I wasn't sure if there would be any corruption concerns. > For replica cases, maybe we can create a command that dumps the > changelog from a good replica and then allow us to reply all changes > that are missing from the backup to bring the server up to the last > minute ? This would happen when we went online anyway though, at least for the entries currently in the changelog. I guess this would have the advantage of doing it in bulk and not over a (potentially) slow link. rob From pspacek at redhat.com Wed Feb 20 15:57:13 2013 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 20 Feb 2013 16:57:13 +0100 Subject: [Freeipa-devel] [PATCH 0114] Log name of the zone if zone cannot be created Message-ID: <5124F259.8050203@redhat.com> Hello, Log name of the zone if zone cannot be created. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0114-Log-name-of-the-zone-if-zone-cannot-be-created.patch Type: text/x-patch Size: 1367 bytes Desc: not available URL: From pviktori at redhat.com Wed Feb 20 16:00:29 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 20 Feb 2013 17:00:29 +0100 Subject: [Freeipa-devel] [PATCHES] 0022, 0115-0116 Make Sudo commands case-sensitive In-Reply-To: <5124B781.5030304@redhat.com> References: <4F58D6E8.7040202@redhat.com> <50CF3583.3060700@redhat.com> <5124B781.5030304@redhat.com> Message-ID: <5124F31D.10302@redhat.com> On 02/20/2013 12:46 PM, Martin Kosek wrote: > On 12/17/2012 04:08 PM, Petr Viktorin wrote: >> https://fedorahosted.org/freeipa/ticket/2482 >> >> The first two patches are rebased from what I sent back in March; the third >> fixes ACIs using targetfilter. >> > > I finally got to your patches. Generally, everything worked like charm, I have > just few minor comments: > > 0022: > - patch needs a rebase > - patch description is confusing, we are talking about RDN "sudocmd" and not "CN" > > 0115: > I would optimize the LDAP calls a little: > 1) Use sudorule base DN as a base for the LDAP search > 2) Do not call LDAP search twice, but just once and then collect the result. > Now you use 2 LDAP searches with following filters: > > (&(objectClass=ipasudorule)(memberallowcmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)) > > (&(objectClass=ipasudorule)(memberdenycmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)) > > We can do just one LDAP search with this filter: > > (&(objectClass=ipasudorule)(|(memberallowcmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)(memberdenycmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test))) > > 0116: > - patch description needs amending: s/CN/SUDOCMD/ > > Martin > Thanks. Attaching updated patches. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0022-04-Use-ipauniqueid-for-the-RDN-of-sudo-commands.patch Type: text/x-patch Size: 14440 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0115-02-Prevent-a-sudo-command-from-being-deleted-if-it-is-a.patch Type: text/x-patch Size: 6140 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0116-02-Update-sudocmd-ACIs-to-use-targetfilter.patch Type: text/x-patch Size: 4540 bytes Desc: not available URL: From mkosek at redhat.com Wed Feb 20 16:17:33 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 17:17:33 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <51235ED4.2000801@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> <511B6874.5060106@redhat.com> <51235ED4.2000801@redhat.com> Message-ID: <5124F71D.701@redhat.com> On 02/19/2013 12:15 PM, Petr Viktorin wrote: > On 02/13/2013 11:18 AM, Petr Viktorin wrote: >> On 01/29/2013 05:06 PM, Petr Viktorin wrote: >>> On 01/04/2013 07:20 PM, Petr Viktorin wrote: >>>> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>>>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>>>> It's time to give this to another set of eyes :) >>>>>>>> >>>>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>>>> >>>>>>>> More info is in commit messages. >>>>>>>> >>>>>>>> >>>>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>>>> change the >>>>>>>> design document: when the client doesn't send the API version, it is >>>>>>>> assumed >>>>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>>>> The client still gets a warning if the version is missing. Except >>>>>>>> for >>>>>>>> those >>>>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>>>> the >>>>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>>>> our client >>>>>>>> is so far the only one that validates so strictly.) >>>>>>> >>>>>>> I did a basic test of this patch and also quickly read through the >>>>>>> patches and >>>>>>> besides nitpicks (like unused inspect module in >>>>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>>>> obvious >>>>>>> errors in the Python code. >>>>>> >>>>>> Noted, will fix in future versions of the patch. >>>>>> >>>>>>> >>>>>>> However, this patch breaks WebUI badly, I did not even get to a >>>>>>> log in >>>>>>> screen. >>>>>>> Cooperation with Petr Vobornik will be needed. In my case, I got >>>>>>> blank >>>>>>> screen >>>>>>> and Javascript error: >>>>>>> >>>>>>> TypeError: IPA.messages.dialogs is undefined >>>>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>>>> Line 1460 >>>>>>> >>>>>>> I assume this is related to the Internal Error that was returned in >>>>>>> the JSON call >>>>>>> >>>>>>> { >>>>>>> "error": null, >>>>>>> "id": null, >>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>> "result": { >>>>>>> "count": 5, >>>>>>> "results": [ >>>>>>> { >>>>>>> "error": "an internal error has occurred", >>>>>>> "error_code": 903, >>>>>>> "error_name": "InternalError" >>>>>>> }, >>>>>>> { >>>>>>> ... >>>>>>> >>>>>>> This can be reproduced with: >>>>>>> >>>>>>> # curl -v -H "Content-Type:application/json" -H >>>>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>>>> --negotiate -u : >>>>>>> --cacert /etc/ipa/ca.crt -d >>>>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>>>> https://`hostname`/ipa/json >>>>>> >>>>>> Good catch! The i18n_messages plugin already defines a "messages" >>>>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>>>> check for clashes. >>>>>> Since i18n_messages is an internal command only used by the Web UI, we >>>>>> can rename its output to "texts" without breaking compatibility. >>>>>> >>>>>> I'm attaching a preliminary fix (for both backend and UI), but >>>>>> hopefully >>>>>> it won't be necessary, see below. >>>>>> >>>>>>> I am also not sure I like the requirement of a specific version >>>>>>> option >>>>>>> to be >>>>>>> always passed. I would prefer that missing version option would mean >>>>>>> "I use the >>>>>>> most recent version of API" instead - it would make the custom >>>>>>> JSONRPC/XMLRPC >>>>>>> calls easier to use. >>>>>>> >>>>>>> But since the version option was not being sent for some commands, we >>>>>>> may not >>>>>>> have a choice anyway if we do not want to break old clients in >>>>>>> case we >>>>>>> add some >>>>>>> capabilities to these commands. >>>>>>> >>>>>> >>>>>> I see three other options, all worse: >>>>>> - Do not use capabilities for the affected commands, meaning no new >>>>>> functionality can be added to them (and by extension, no new >>>>>> functionality common to all commands can be added). >>>>>> - Treat a missing version as the current version >>>>>> - Break backwards compatibility >>>>>> >>>>>> And one possibly better (thanks to Petr? and Martin for opening my >>>>>> eyes >>>>>> off-list!): >>>>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>>>> >>>>>> This would also allow us to solve the above name-clashing problem >>>>>> elegantly. Here is a reminder of what a JSON response looks like: >>>>>> >>>>>> { >>>>>> "error": null, >>>>>> "id": 0, >>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>> "result": { >>>>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>>>> 2.47" >>>>>> }, >>>>>> "version": "3.1.0GIT2e4bd02" >>>>>> } >>>>>> >>>>>> A XML-RPC response only contains the "result" part of that. >>>>>> So with JSON, we can put the messages in the top level, which is much >>>>>> better design. >>>> >>>> Custom info in the "top level" seems to be a violation of the JSON-RPC >>>> spec. I'd rather not do more of those, so I'm withdrawing this idea. >>>> >>>>>> >>>>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>>>> inability to attach extra info to errors (commit >>>>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>>>> PublicError). >>>>>> >>>>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>>>> >>>>> >>>>> +1, XML-RPC sucks. This should have been done a long time ago. >>>>> >>>>> Honza >>>>> >>>> >>>> Here are new patches. >>>> >>>> XML-RPC requests with missing version are assumed to be old (the version >>>> before capabilities are introduced, 2.47). This takes care of backcompat >>>> with clients with bug 3294. >>>> JSON-RPC requests with missing version are assumed to be testing calls >>>> (e.g. curl), they get behavior of the latest version but also a warning. >>>> I've also added this info to the design doc. >>>> >>>> It turns out that these patches don't depend on whether our client uses >>>> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >>>> extra unit tests. >>>> >>> >>> Patch 106 had a minor conflict with master, attaching fixed version. >>> >> >> Patches 106 & 115 need an API version bump. >> I also noticed that `makeapi --validate` wasn't checking capabilities >> properly. Fixed. >> >> > > Rebasing patch 104 to current master. > Patch 104 and 106 needs a rebase. Generally I think this patchset is OK and we will need it as a foundation for other features. I may have done my custom rebasing wrong, but my WebUI stopped working with these patches. I see this in error_log: [Wed Feb 20 10:38:22.351845 2013] [auth_kerb:error] [pid 22172] [client 10.34.4.72:40777] gss_display_name() failed: A required input parameter could not be read: An invalid name was supplied (, Unknown error), referer: https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ I also saw one failed test case: ====================================================================== FAIL: Try user_show with no version ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/root/freeipa-master/tests/test_xmlrpc/test_user_plugin.py", line 1760, in test_user_show_without_version assert res['messages'] == (expected_message.to_dict(), ) AssertionError Martin From rmeggins at redhat.com Wed Feb 20 16:27:54 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 20 Feb 2013 09:27:54 -0700 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <5124EDD9.7050900@redhat.com> References: <5124466F.9030800@redhat.com> <1361369818.12328.428.camel@willson.li.ssimo.org> <5124EDD9.7050900@redhat.com> Message-ID: <5124F98A.20106@redhat.com> On 02/20/2013 08:38 AM, Rob Crittenden wrote: > Simo Sorce wrote: >> On Tue, 2013-02-19 at 22:43 -0500, Rob Crittenden wrote: >>> I've looked into some basic backup and restore procedures for IPA. My >>> findings are here: http://freeipa.org/page/V3/Backup_and_Restore >> >> Great summary! >> >> For the catastrofic failure scenario, should we mention how to put back >> a full failed and restored machine online ? I am thinking the restored >> server may be behind (even if only by a few entries) in the replication, >> so the CSNs in the other replicas will not match. >> I guess we should mention a full resync may be needed ? Or is there a >> way to bring back CSNs on replicas ? > > Good questions. It depends on how long the machine was down and how > many changes have happened. It is possible that one would want to do a > full re-init. I'll add that to the design. The replication protocol will detect if a replica is too out of date to bring up to date with an incremental update, and requires a re-init. > >> In the 'Returning to a good state' case, can we consider some split >> brain approach, were we sever replication and rebuild one server at a >> time ? > > Perhaps using a firewall, but then you run the risk of each of those > servers accepting changes during the rebuild and you could end up with > a lot of collisions, which sort of goes against the point of restoring > to a known good state. > > The changelog is the key here. I'll have to ponder this one a bit, I'm > a bit conflicted on the right approach. > >> Maybe we can think of a way to 'mark' all server as 'bad' so that on >> restore the replication agreements do not need to be changed but changes >> from 'bad' servers will not be accepted ? >> I guess this crosses also a request by someone to be able to 'pause' >> replication, would use the same mechanism I guess. > > AFAIK there is an option to pause replication now (at least in 1.3). > > What you can't do is drop the changelog AFAIK. That is the real > problem. If you want to restore to a known state you need to drop all > the changelog entries since that time. I'll check with the 389-ds team > to see if that is possible. Since we know the time of the backup, we > might be able to drop newer entries than that. Not sure what you mean - what exactly do you want to do with the changelog? > >> Full system backup: >> in the first part it is said the process is offline, but in the 'LDAP' >> section you say ldapi is used, but that would mean DS is running ? >> Also are we sure we can get all data we need via ldapi ? Are we going to >> miss any "operational" attribute ? > > The full backup is offline because it is just using tar. This is sort > of a brute-force backup, copying bits from A to B. > > The data backup is online and creates a task in 389-ds to write the > data and changelog to a file. It should write everything completely. > We don't do an ldapsearch. > > I chose to not back up in ldif because this would back up just the > data and not the changelog. The other advantage is that the db2bak > format includes the indexes and ldif restore would require a rebuild. It is good to have a long term backup in LDIF format - no matter what happens to the database, if you have an LDIF backup, you can always recreate your data. So it's good to have both - db2bak format for shorter term/frequent backups, and LDIF for longer term/infrequent backups. > >> For restore are we sure we can reload data w/o alterations ? What about >> plugins ? will we have to disable all plugins during a restore ? > > Yes, it should be fine. I'm hoping that the version will help us with > this, to prevent someone from restoring an ancient backup on a new > system, for example (or the reverse). > >> For the open questions. >> >> Size of backup: >> I think we should make it easy to configure the system to use a custom >> directory to dump the backups. This way admins can make sure it is on a >> different partition/disk or even over NFS and that the backup will not >> fill up the disk on which DS is running. > > That's a good idea. I'll have to think about where we would configure > that. Perhaps as an optional argument to the backup command. You'll have to figure out a way around selinux, or add some sort of selinux magic that allows db2bak to write there. > >> We should definitely allow to encrypt backup files, a gpg public key >> would be sufficient. > > Ok. I wasn't sure if there would be any corruption concerns. > >> For replica cases, maybe we can create a command that dumps the >> changelog from a good replica and then allow us to reply all changes >> that are missing from the backup to bring the server up to the last >> minute ? > > This would happen when we went online anyway though, at least for the > entries currently in the changelog. I guess this would have the > advantage of doing it in bulk and not over a (potentially) slow link. That should happen automatically with the replication protocol - it will attempt to bring older replicas up-to-date or, if they are too far out of date, will complain that they need a re-init. > > rob > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From pviktori at redhat.com Wed Feb 20 16:37:15 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 20 Feb 2013 17:37:15 +0100 Subject: [Freeipa-devel] [PATCHES] 0177-0179 Add missing dict methods to CIDict In-Reply-To: <51237565.1040404@redhat.com> References: <51113B3F.3070808@redhat.com> <51237565.1040404@redhat.com> Message-ID: <5124FBBB.6080801@redhat.com> On 02/19/2013 01:51 PM, Jan Cholasta wrote: > Hi, > > On 5.2.2013 18:02, Petr Viktorin wrote: >> CIDict, our case-insensitive dictionary, inherits from dict but did not >> reimplement the full dict interface. Calling the missing methods >> silently invoked case-sensitive behavior. Our code seems to avoid that, >> but it's a bit of a minefield for new development. >> >> Patch 119 adds the missing dict methods (except >> view{items,keys,values}(), which now raise errors), and adds tests. > > Can you please also add the (obj, **kwargs) and (**kwargs) variants of > __init__ and update? Added, thanks for the catch. > >> >> >> Patches 117-118 modernize the testsuite a bit (these have been sitting >> in my queue for a while, I think now is a good time to submit them): >> The first one moves some old tests from the main code tree to tests/. >> (The adtrust_install test wasn't run before, this move makes nose notice >> it). >> The second converts CIDict's unittest-based suite to nose. >> > > Honza > -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0179-02-Add-missing-dict-methods-to-CIDict.patch Type: text/x-patch Size: 8750 bytes Desc: not available URL: From mkosek at redhat.com Wed Feb 20 16:39:27 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 17:39:27 +0100 Subject: [Freeipa-devel] [PATCHES] 0022, 0115-0116 Make Sudo commands case-sensitive In-Reply-To: <5124F31D.10302@redhat.com> References: <4F58D6E8.7040202@redhat.com> <50CF3583.3060700@redhat.com> <5124B781.5030304@redhat.com> <5124F31D.10302@redhat.com> Message-ID: <5124FC3F.7040709@redhat.com> On 02/20/2013 05:00 PM, Petr Viktorin wrote: > On 02/20/2013 12:46 PM, Martin Kosek wrote: >> On 12/17/2012 04:08 PM, Petr Viktorin wrote: >>> https://fedorahosted.org/freeipa/ticket/2482 >>> >>> The first two patches are rebased from what I sent back in March; the third >>> fixes ACIs using targetfilter. >>> >> >> I finally got to your patches. Generally, everything worked like charm, I have >> just few minor comments: >> >> 0022: >> - patch needs a rebase >> - patch description is confusing, we are talking about RDN "sudocmd" and not >> "CN" >> >> 0115: >> I would optimize the LDAP calls a little: >> 1) Use sudorule base DN as a base for the LDAP search >> 2) Do not call LDAP search twice, but just once and then collect the result. >> Now you use 2 LDAP searches with following filters: >> >> (&(objectClass=ipasudorule)(memberallowcmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)) >> >> >> (&(objectClass=ipasudorule)(memberdenycmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)) >> >> >> We can do just one LDAP search with this filter: >> >> (&(objectClass=ipasudorule)(|(memberallowcmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test)(memberdenycmd=sudocmd=/usr/bin/less,cn=sudocmds,cn=sudo,dc=linux,dc=ad,dc=test))) >> >> >> 0116: >> - patch description needs amending: s/CN/SUDOCMD/ >> >> Martin >> > > Thanks. Attaching updated patches. > ACK. Pushed to master, ipa-3-1. Martin From rcritten at redhat.com Wed Feb 20 16:44:19 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 20 Feb 2013 11:44:19 -0500 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <5124F98A.20106@redhat.com> References: <5124466F.9030800@redhat.com> <1361369818.12328.428.camel@willson.li.ssimo.org> <5124EDD9.7050900@redhat.com> <5124F98A.20106@redhat.com> Message-ID: <5124FD63.4050004@redhat.com> Rich Megginson wrote: > On 02/20/2013 08:38 AM, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Tue, 2013-02-19 at 22:43 -0500, Rob Crittenden wrote: >>>> I've looked into some basic backup and restore procedures for IPA. My >>>> findings are here: http://freeipa.org/page/V3/Backup_and_Restore >>> >>> Great summary! >>> >>> For the catastrofic failure scenario, should we mention how to put back >>> a full failed and restored machine online ? I am thinking the restored >>> server may be behind (even if only by a few entries) in the replication, >>> so the CSNs in the other replicas will not match. >>> I guess we should mention a full resync may be needed ? Or is there a >>> way to bring back CSNs on replicas ? >> >> Good questions. It depends on how long the machine was down and how >> many changes have happened. It is possible that one would want to do a >> full re-init. I'll add that to the design. > > The replication protocol will detect if a replica is too out of date to > bring up to date with an incremental update, and requires a re-init. Ok, I'll update the design with this, thanks. > >> >>> In the 'Returning to a good state' case, can we consider some split >>> brain approach, were we sever replication and rebuild one server at a >>> time ? >> >> Perhaps using a firewall, but then you run the risk of each of those >> servers accepting changes during the rebuild and you could end up with >> a lot of collisions, which sort of goes against the point of restoring >> to a known good state. >> >> The changelog is the key here. I'll have to ponder this one a bit, I'm >> a bit conflicted on the right approach. >> >>> Maybe we can think of a way to 'mark' all server as 'bad' so that on >>> restore the replication agreements do not need to be changed but changes >>> from 'bad' servers will not be accepted ? >>> I guess this crosses also a request by someone to be able to 'pause' >>> replication, would use the same mechanism I guess. >> >> AFAIK there is an option to pause replication now (at least in 1.3). >> >> What you can't do is drop the changelog AFAIK. That is the real >> problem. If you want to restore to a known state you need to drop all >> the changelog entries since that time. I'll check with the 389-ds team >> to see if that is possible. Since we know the time of the backup, we >> might be able to drop newer entries than that. > > Not sure what you mean - what exactly do you want to do with the changelog? As an example, if we have 2 IPA masters and we restore the data on one of them, as soon as it comes back upon the other is going to push the changelog onto it (as it should) so they are in sync again. So the question is, how do we restore several masters at the same time without apply changings from the changelog? What I was going to ask is, can we delete all changelog entries from Time X until now? That would prevent the sync issues, but it would retain the part of the changelog we care about. > >> >>> Full system backup: >>> in the first part it is said the process is offline, but in the 'LDAP' >>> section you say ldapi is used, but that would mean DS is running ? >>> Also are we sure we can get all data we need via ldapi ? Are we going to >>> miss any "operational" attribute ? >> >> The full backup is offline because it is just using tar. This is sort >> of a brute-force backup, copying bits from A to B. >> >> The data backup is online and creates a task in 389-ds to write the >> data and changelog to a file. It should write everything completely. >> We don't do an ldapsearch. >> >> I chose to not back up in ldif because this would back up just the >> data and not the changelog. The other advantage is that the db2bak >> format includes the indexes and ldif restore would require a rebuild. > > It is good to have a long term backup in LDIF format - no matter what > happens to the database, if you have an LDIF backup, you can always > recreate your data. So it's good to have both - db2bak format for > shorter term/frequent backups, and LDIF for longer term/infrequent backups. Ok, so maybe when we do a full backup first we snapshot the database to ldif via db2ldif.pl, then back up the whole shebang? >> >>> For restore are we sure we can reload data w/o alterations ? What about >>> plugins ? will we have to disable all plugins during a restore ? >> >> Yes, it should be fine. I'm hoping that the version will help us with >> this, to prevent someone from restoring an ancient backup on a new >> system, for example (or the reverse). >> >>> For the open questions. >>> >>> Size of backup: >>> I think we should make it easy to configure the system to use a custom >>> directory to dump the backups. This way admins can make sure it is on a >>> different partition/disk or even over NFS and that the backup will not >>> fill up the disk on which DS is running. >> >> That's a good idea. I'll have to think about where we would configure >> that. Perhaps as an optional argument to the backup command. > You'll have to figure out a way around selinux, or add some sort of > selinux magic that allows db2bak to write there. >> >>> We should definitely allow to encrypt backup files, a gpg public key >>> would be sufficient. >> >> Ok. I wasn't sure if there would be any corruption concerns. >> >>> For replica cases, maybe we can create a command that dumps the >>> changelog from a good replica and then allow us to reply all changes >>> that are missing from the backup to bring the server up to the last >>> minute ? >> >> This would happen when we went online anyway though, at least for the >> entries currently in the changelog. I guess this would have the >> advantage of doing it in bulk and not over a (potentially) slow link. > > That should happen automatically with the replication protocol - it will > attempt to bring older replicas up-to-date or, if they are too far out > of date, will complain that they need a re-init. Yeah, I guess one way or another the bits are going to need to go over that slow link. I was thinking about a sneakernet to move the changelog, but yeah, in retrospect that doesn't make a lot of sense. thanks rob From rmeggins at redhat.com Wed Feb 20 17:19:02 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Wed, 20 Feb 2013 10:19:02 -0700 Subject: [Freeipa-devel] Backup and Restore design In-Reply-To: <5124FD63.4050004@redhat.com> References: <5124466F.9030800@redhat.com> <1361369818.12328.428.camel@willson.li.ssimo.org> <5124EDD9.7050900@redhat.com> <5124F98A.20106@redhat.com> <5124FD63.4050004@redhat.com> Message-ID: <51250586.1020305@redhat.com> On 02/20/2013 09:44 AM, Rob Crittenden wrote: > Rich Megginson wrote: >> On 02/20/2013 08:38 AM, Rob Crittenden wrote: >>> Simo Sorce wrote: >>>> On Tue, 2013-02-19 at 22:43 -0500, Rob Crittenden wrote: >>>>> I've looked into some basic backup and restore procedures for IPA. My >>>>> findings are here: http://freeipa.org/page/V3/Backup_and_Restore >>>> >>>> Great summary! >>>> >>>> For the catastrofic failure scenario, should we mention how to put >>>> back >>>> a full failed and restored machine online ? I am thinking the restored >>>> server may be behind (even if only by a few entries) in the >>>> replication, >>>> so the CSNs in the other replicas will not match. >>>> I guess we should mention a full resync may be needed ? Or is there a >>>> way to bring back CSNs on replicas ? >>> >>> Good questions. It depends on how long the machine was down and how >>> many changes have happened. It is possible that one would want to do a >>> full re-init. I'll add that to the design. >> >> The replication protocol will detect if a replica is too out of date to >> bring up to date with an incremental update, and requires a re-init. > > Ok, I'll update the design with this, thanks. > >> >>> >>>> In the 'Returning to a good state' case, can we consider some split >>>> brain approach, were we sever replication and rebuild one server at a >>>> time ? >>> >>> Perhaps using a firewall, but then you run the risk of each of those >>> servers accepting changes during the rebuild and you could end up with >>> a lot of collisions, which sort of goes against the point of restoring >>> to a known good state. >>> >>> The changelog is the key here. I'll have to ponder this one a bit, I'm >>> a bit conflicted on the right approach. >>> >>>> Maybe we can think of a way to 'mark' all server as 'bad' so that on >>>> restore the replication agreements do not need to be changed but >>>> changes >>>> from 'bad' servers will not be accepted ? >>>> I guess this crosses also a request by someone to be able to 'pause' >>>> replication, would use the same mechanism I guess. >>> >>> AFAIK there is an option to pause replication now (at least in 1.3). >>> >>> What you can't do is drop the changelog AFAIK. That is the real >>> problem. If you want to restore to a known state you need to drop all >>> the changelog entries since that time. I'll check with the 389-ds team >>> to see if that is possible. Since we know the time of the backup, we >>> might be able to drop newer entries than that. >> >> Not sure what you mean - what exactly do you want to do with the >> changelog? > > As an example, if we have 2 IPA masters and we restore the data on one > of them, as soon as it comes back upon the other is going to push the > changelog onto it (as it should) so they are in sync again. > > So the question is, how do we restore several masters at the same time > without apply changings from the changelog? > > What I was going to ask is, can we delete all changelog entries from > Time X until now? That would prevent the sync issues, but it would > retain the part of the changelog we care about. Is this the problem you are trying to solve? You have a situation where some bogus data was introduced into your system, and that bogus data has now been replicated everywhere. You want to rollback the state of everything to before the bogus data was introduced. Let's assume you want to delete the bogus data and everything that happened after that. The first step is to pick a server to restore, and restore that server from a backup. The first problem is that this server will need to reject any replicated updates, but still allow regular client updates, after the restore process is complete (the db is in read-only mode during the restore). The only way to do this now would be to first disable all replication agreements on all other replicas going to this server, which would be quite painful. Alternately - during the restore process, change the replica generation of the restored server - other servers would see the different replica generation and would refuse to send updates (and report lots of replication errors). Once the first server is restored, you would just use the online or offline replica init procedure. > >> >>> >>>> Full system backup: >>>> in the first part it is said the process is offline, but in the 'LDAP' >>>> section you say ldapi is used, but that would mean DS is running ? >>>> Also are we sure we can get all data we need via ldapi ? Are we >>>> going to >>>> miss any "operational" attribute ? >>> >>> The full backup is offline because it is just using tar. This is sort >>> of a brute-force backup, copying bits from A to B. >>> >>> The data backup is online and creates a task in 389-ds to write the >>> data and changelog to a file. It should write everything completely. >>> We don't do an ldapsearch. >>> >>> I chose to not back up in ldif because this would back up just the >>> data and not the changelog. The other advantage is that the db2bak >>> format includes the indexes and ldif restore would require a rebuild. >> >> It is good to have a long term backup in LDIF format - no matter what >> happens to the database, if you have an LDIF backup, you can always >> recreate your data. So it's good to have both - db2bak format for >> shorter term/frequent backups, and LDIF for longer term/infrequent >> backups. > > Ok, so maybe when we do a full backup first we snapshot the database > to ldif via db2ldif.pl, then back up the whole shebang? > >>> >>>> For restore are we sure we can reload data w/o alterations ? What >>>> about >>>> plugins ? will we have to disable all plugins during a restore ? >>> >>> Yes, it should be fine. I'm hoping that the version will help us with >>> this, to prevent someone from restoring an ancient backup on a new >>> system, for example (or the reverse). >>> >>>> For the open questions. >>>> >>>> Size of backup: >>>> I think we should make it easy to configure the system to use a custom >>>> directory to dump the backups. This way admins can make sure it is >>>> on a >>>> different partition/disk or even over NFS and that the backup will not >>>> fill up the disk on which DS is running. >>> >>> That's a good idea. I'll have to think about where we would configure >>> that. Perhaps as an optional argument to the backup command. >> You'll have to figure out a way around selinux, or add some sort of >> selinux magic that allows db2bak to write there. >>> >>>> We should definitely allow to encrypt backup files, a gpg public key >>>> would be sufficient. >>> >>> Ok. I wasn't sure if there would be any corruption concerns. >>> >>>> For replica cases, maybe we can create a command that dumps the >>>> changelog from a good replica and then allow us to reply all changes >>>> that are missing from the backup to bring the server up to the last >>>> minute ? >>> >>> This would happen when we went online anyway though, at least for the >>> entries currently in the changelog. I guess this would have the >>> advantage of doing it in bulk and not over a (potentially) slow link. >> >> That should happen automatically with the replication protocol - it will >> attempt to bring older replicas up-to-date or, if they are too far out >> of date, will complain that they need a re-init. > > Yeah, I guess one way or another the bits are going to need to go over > that slow link. I was thinking about a sneakernet to move the > changelog, but yeah, in retrospect that doesn't make a lot of sense. > > thanks > > rob > From mkosek at redhat.com Wed Feb 20 17:03:30 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 20 Feb 2013 18:03:30 +0100 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <5124B3E8.20709@redhat.com> References: <51011490.10101@redhat.com> <510FD21B.7070403@redhat.com> <5123EC49.3070508@redhat.com> <5124862C.5040400@redhat.com> <5124B3E8.20709@redhat.com> Message-ID: <512501E2.8070005@redhat.com> On 02/20/2013 12:30 PM, Petr Viktorin wrote: > On 02/20/2013 09:15 AM, Martin Kosek wrote: >> On 02/19/2013 10:19 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 01/24/2013 12:01 PM, Martin Kosek wrote: >>>>> When user tries to perform any action requiring communication with >>>>> trusted domain, IPA server tries to retrieve a trust secret on his >>>>> behalf to be able to establish the connection. This happens for >>>>> example during group-add-member command when external user is >>>>> being resolved in the AD. >>>>> >>>>> When user is not member of Trust admins group, the retrieval crashes >>>>> and reports internal error. Catch this exception and rather report >>>>> properly formatted ACIError. >>>>> >>>>> ---- >>>>> >>>>> I hit this error after updating to the latest FreeIPA version with the AD CVE >>>>> fixed. >>>>> >>>>> Martin >>>>> >>>> >>>> I filed a ticket to not loose this fix and patch. Attaching an updated patch >>>> with ticket URL in description. >>>> >>>> Martin >>>> >>> >>> >>> The patch fixes the problem but the error is untranslated: >>> >>> member group: AD\Domain Admins: Insufficient access: >>> Gettext('communication >>> with trusted domains is allowed for Trusts administrator group members only', >>> domain='ipa', localedir=None) >>> >>> rob >> >> I think this is just because this string is not in our ipa.pot file yet (will >> be when we do Transifex refresh"). >> >> Martin >> > > I don't have AD so I can't investigate, but this problem is usually due to the > error being converted to string instead of using the strerror attribute. > You are right, attaching a patch which fixes it for group-add-member. But just with using a quick grep, I see we do not use strerror on a lot of other places, we may want to open a ticket to fix that too. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-355-3-avoid-internal-error-when-user-is-not-trust-admin.patch Type: text/x-patch Size: 3764 bytes Desc: not available URL: From rcritten at redhat.com Wed Feb 20 18:25:15 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 20 Feb 2013 13:25:15 -0500 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <512501E2.8070005@redhat.com> References: <51011490.10101@redhat.com> <510FD21B.7070403@redhat.com> <5123EC49.3070508@redhat.com> <5124862C.5040400@redhat.com> <5124B3E8.20709@redhat.com> <512501E2.8070005@redhat.com> Message-ID: <5125150B.2020905@redhat.com> Martin Kosek wrote: > On 02/20/2013 12:30 PM, Petr Viktorin wrote: >> On 02/20/2013 09:15 AM, Martin Kosek wrote: >>> On 02/19/2013 10:19 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On 01/24/2013 12:01 PM, Martin Kosek wrote: >>>>>> When user tries to perform any action requiring communication with >>>>>> trusted domain, IPA server tries to retrieve a trust secret on his >>>>>> behalf to be able to establish the connection. This happens for >>>>>> example during group-add-member command when external user is >>>>>> being resolved in the AD. >>>>>> >>>>>> When user is not member of Trust admins group, the retrieval crashes >>>>>> and reports internal error. Catch this exception and rather report >>>>>> properly formatted ACIError. >>>>>> >>>>>> ---- >>>>>> >>>>>> I hit this error after updating to the latest FreeIPA version with the AD CVE >>>>>> fixed. >>>>>> >>>>>> Martin >>>>>> >>>>> >>>>> I filed a ticket to not loose this fix and patch. Attaching an updated patch >>>>> with ticket URL in description. >>>>> >>>>> Martin >>>>> >>>> >>>> >>>> The patch fixes the problem but the error is untranslated: >>>> >>>> member group: AD\Domain Admins: Insufficient access: >>>> Gettext('communication >>>> with trusted domains is allowed for Trusts administrator group members only', >>>> domain='ipa', localedir=None) >>>> >>>> rob >>> >>> I think this is just because this string is not in our ipa.pot file yet (will >>> be when we do Transifex refresh"). >>> >>> Martin >>> >> >> I don't have AD so I can't investigate, but this problem is usually due to the >> error being converted to string instead of using the strerror attribute. >> > > You are right, attaching a patch which fixes it for group-add-member. But just > with using a quick grep, I see we do not use strerror on a lot of other places, > we may want to open a ticket to fix that too. > > Martin > ACK, pushed to master and ipa-3-1 I think another ticket for your grep findings would be a good idea. rob From jdennis at redhat.com Wed Feb 20 20:20:39 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 20 Feb 2013 15:20:39 -0500 Subject: [Freeipa-devel] NSS 3.14.3 Release, this fixes the certutil bug encountered during install In-Reply-To: <18fa15637c7ccb2fd0c5a1cb04b14f05.squirrel@webmail.dreamhost.com> References: <18fa15637c7ccb2fd0c5a1cb04b14f05.squirrel@webmail.dreamhost.com> Message-ID: <51253017.3080701@redhat.com> NSS 3.14.3 was just released by the NSS team. This is critical for us because it fixes https://bugzilla.mozilla.org/show_bug.cgi?id=840714 which was causing install failures as reported on this list. I expect a new RPM will follow shortly. John -------- Original Message -------- Subject: [ANNOUNCE] NSS 3.14.3 Release Date: Wed, 20 Feb 2013 12:07:38 -0800 From: Ryan Sleevi Reply-To: mozilla's crypto code discussion list To: dev-tech-crypto at lists.mozilla.org The NSS Development Team is pleased to announce the release of NSS 3.14.3. The official release notes are available at https://developer.mozilla.org/en-US/docs/NSS/NSS_3.14.3_release_notes , and are reproduced at the end of this message. This release includes mitigations for recently discussed "Lucky Thirteen" attack (CVE-2013-1620). However, please note the limitations of the mitigations discussed in the release notes below. Introduction: Network Security Services (NSS) 3.14.3 is a patch release for NSS 3.14. The bug fixes in NSS 3.14.3 are described in the "Bugs Fixed" section below. Distribution Information * The CVS tag is NSS_3_14_3_RTM. NSS 3.14.3 requires NSPR 4.9.5 or newer. * NSS 3.14.3 source distributions are also available on ftp.mozilla.org for secure HTTPS download: - Source tarballs: https://ftp.mozilla.org/pub/mozilla.org/security/nss/releases/NSS_3_14_3_RTM/src/ New in NSS 3.14.3 * No new major functionality is introduced in this release. This release is a patch release to address CVE-2013-1620. New Functions * in pk11pub.h - PK11_SignWithSymKey - Similar to PK11_Sign, performs a signing operation in a single operation. However, unlike PK11_Sign, which uses a SECKEYPrivateKey, PK11_SignWithSymKey performs the signature using a symmetric key, such as commonly used for generating MACs. New Types * CK_NSS_MAC_CONSTANT_TIME_PARAMS - Parameters for use with CKM_NSS_HMAC_CONSTANT_TIME and CKM_NSS_SSL3_MAC_CONSTANT_TIME. New PKCS #11 Mechanisms * CKM_NSS_HMAC_CONSTANT_TIME - Constant-time HMAC operation for use when verifying a padded, MAC-then-encrypted block of data. CKM_NSS_SSL3_MAC_CONSTANT_TIME - Constant-time MAC operation for use when verifying a padded, MAC-then-encrypted block of data using the SSLv3 MAC. Notable Changes in NSS 3.14.3 * CVE-2013-1620 Recent research by Nadhem AlFardan and Kenny Patterson has highlighted a weakness in the handling of CBC padding as used in SSL, TLS, and DTLS that allows an attacker to exploit timing differences in MAC processing. The details of their research and the attack can be found at http://www.isg.rhul.ac.uk/tls/, and has been referred to as "Lucky Thirteen". NSS 3.14.3 includes changes to the softoken and ssl libraries to address and mitigate these attacks, contributed by Adam Langley of Google. This attack is mitigated when using NSS 3.14.3 with an NSS Cryptographic Module ("softoken") version 3.14.3 or later. However, this attack is only partially mitigated if NSS 3.14.3 is used with the current FIPS validated NSS Cryptographic Module, version 3.12.9.1. * Bug 840714 - "certutil -a" was not correctly producing ASCII output as requested. * Bug 837799 - NSS 3.14.2 broke compilation with older versions of sqlite that lacked the SQLITE_FCNTL_TEMPFILENAME file control. NSS 3.14.3 now properly compiles when used with older versions of sqlite. Acknowledgements * The NSS development team would like to thank Nadhem AlFardan and Kenny Patterson (Royal Holloway, University of London) for responsibly disclosing the issue by providing advance copies of their research. In addition, thanks to Adam Langley (Google) for the development of a mitigation for the issues raised in the paper, along with Emilia Kasper and Bodo M?ller (Google) for assisting in the review and improvements to the initial patches. Bugs fixed in NSS 3.14.3 * https://bugzilla.mozilla.org/buglist.cgi?list_id=5689256;resolution=FIXED;classification=Components;query_format=advanced;target_milestone=3.14.3;product=NSS Compatibility * NSS 3.14.3 shared libraries are backward compatible with all older NSS 3.x shared libraries. A program linked with older NSS 3.x shared libraries will work with NSS 3.14.3 shared libraries without recompiling or relinking. Furthermore, applications that restrict their use of NSS APIs to the functions listed in NSS Public Functions will remain compatible with future versions of the NSS shared libraries. Feedback * Bugs discovered should be reported by filing a bug report with bugzilla.mozilla.org (product NSS). -- dev-tech-crypto mailing list dev-tech-crypto at lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto From mkosek at redhat.com Thu Feb 21 07:56:33 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 08:56:33 +0100 Subject: [Freeipa-devel] [PATCH] 355 Avoid internal error when user is not Trust admin In-Reply-To: <5125150B.2020905@redhat.com> References: <51011490.10101@redhat.com> <510FD21B.7070403@redhat.com> <5123EC49.3070508@redhat.com> <5124862C.5040400@redhat.com> <5124B3E8.20709@redhat.com> <512501E2.8070005@redhat.com> <5125150B.2020905@redhat.com> Message-ID: <5125D331.7070003@redhat.com> On 02/20/2013 07:25 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 02/20/2013 12:30 PM, Petr Viktorin wrote: >>> On 02/20/2013 09:15 AM, Martin Kosek wrote: >>>> On 02/19/2013 10:19 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>>>> On 01/24/2013 12:01 PM, Martin Kosek wrote: >>>>>>> When user tries to perform any action requiring communication with >>>>>>> trusted domain, IPA server tries to retrieve a trust secret on his >>>>>>> behalf to be able to establish the connection. This happens for >>>>>>> example during group-add-member command when external user is >>>>>>> being resolved in the AD. >>>>>>> >>>>>>> When user is not member of Trust admins group, the retrieval crashes >>>>>>> and reports internal error. Catch this exception and rather report >>>>>>> properly formatted ACIError. >>>>>>> >>>>>>> ---- >>>>>>> >>>>>>> I hit this error after updating to the latest FreeIPA version with the >>>>>>> AD CVE >>>>>>> fixed. >>>>>>> >>>>>>> Martin >>>>>>> >>>>>> >>>>>> I filed a ticket to not loose this fix and patch. Attaching an updated patch >>>>>> with ticket URL in description. >>>>>> >>>>>> Martin >>>>>> >>>>> >>>>> >>>>> The patch fixes the problem but the error is untranslated: >>>>> >>>>> member group: AD\Domain Admins: Insufficient access: >>>>> Gettext('communication >>>>> with trusted domains is allowed for Trusts administrator group members only', >>>>> domain='ipa', localedir=None) >>>>> >>>>> rob >>>> >>>> I think this is just because this string is not in our ipa.pot file yet (will >>>> be when we do Transifex refresh"). >>>> >>>> Martin >>>> >>> >>> I don't have AD so I can't investigate, but this problem is usually due to the >>> error being converted to string instead of using the strerror attribute. >>> >> >> You are right, attaching a patch which fixes it for group-add-member. But just >> with using a quick grep, I see we do not use strerror on a lot of other places, >> we may want to open a ticket to fix that too. >> >> Martin >> > > ACK, pushed to master and ipa-3-1 > > I think another ticket for your grep findings would be a good idea. > > rob Ok, here it is: https://fedorahosted.org/freeipa/ticket/3445 Martin From jcholast at redhat.com Thu Feb 21 09:06:47 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 21 Feb 2013 10:06:47 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <5123A8C2.5000504@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> <5110FD62.4090006@redhat.com> <51111B1C.2010305@redhat.com> <511228A9.5060303@redhat.com> <5118EB4B.4080507@redhat.com> <5123A8C2.5000504@redhat.com> Message-ID: <5125E3A7.1090205@redhat.com> On 19.2.2013 17:30, Rob Crittenden wrote: > I think dropping raw=True in patch 99.3 is going to break a check later > where we look at the managedby attribute. Without raw this will be > managedby_host. > Fixed, thanks for the catch. I have also made 2 changes to patch 100 (made sure the entry returned by ldap2.get_ipa_config is using the correct IPASimpleLDAPObject and changed LDAPEntry.clone to be less fragile). Updated (and rebased) patches attached. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-98.4-Use-the-dn-attribute-of-LDAPEntry-to-set-get-DNs-of-.patch Type: text/x-patch Size: 17429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-99.4-Preserve-case-of-attribute-names-in-LDAPEntry.patch Type: text/x-patch Size: 10750 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-100.4-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch Type: text/x-patch Size: 5957 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-101.4-Support-attributes-with-multiple-names-in-LDAPEntry.patch Type: text/x-patch Size: 3335 bytes Desc: not available URL: From jcholast at redhat.com Thu Feb 21 11:39:30 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 21 Feb 2013 12:39:30 +0100 Subject: [Freeipa-devel] [PATCHES] 98-101 Preserve case of LDAP attribute names In-Reply-To: <5125E3A7.1090205@redhat.com> References: <510A8B55.70109@redhat.com> <510BA30A.4040601@redhat.com> <510FB6AF.20903@redhat.com> <510FCA84.40109@redhat.com> <5110FD62.4090006@redhat.com> <51111B1C.2010305@redhat.com> <511228A9.5060303@redhat.com> <5118EB4B.4080507@redhat.com> <5123A8C2.5000504@redhat.com> <5125E3A7.1090205@redhat.com> Message-ID: <51260772.6090208@redhat.com> On 21.2.2013 10:06, Jan Cholasta wrote: > On 19.2.2013 17:30, Rob Crittenden wrote: >> I think dropping raw=True in patch 99.3 is going to break a check later >> where we look at the managedby attribute. Without raw this will be >> managedby_host. >> > > Fixed, thanks for the catch. > > I have also made 2 changes to patch 100 (made sure the entry returned by > ldap2.get_ipa_config is using the correct IPASimpleLDAPObject and > changed LDAPEntry.clone to be less fragile). > > Updated (and rebased) patches attached. > > Honza > Whoops, wrong patches. Correct patches attached. Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-98.5-Use-the-dn-attribute-of-LDAPEntry-to-set-get-DNs-of-.patch Type: text/x-patch Size: 17429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-99.5-Preserve-case-of-attribute-names-in-LDAPEntry.patch Type: text/x-patch Size: 10750 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-100.5-Aggregate-IPASimpleLDAPObject-in-LDAPEntry.patch Type: text/x-patch Size: 6039 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-101.5-Support-attributes-with-multiple-names-in-LDAPEntry.patch Type: text/x-patch Size: 3335 bytes Desc: not available URL: From mkosek at redhat.com Thu Feb 21 11:47:26 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 12:47:26 +0100 Subject: [Freeipa-devel] [PATCH 0035] Use default.conf as flag of IPA client being installed In-Reply-To: <512497ED.6080200@redhat.com> References: <512497ED.6080200@redhat.com> Message-ID: <5126094E.1040503@redhat.com> On 02/20/2013 10:31 AM, Tomas Babej wrote: > Hi, > > When installing / uninstalling IPA client, the checks that > determine whether IPA client is installed now take the existence > of /etc/ipa/default.conf into consideration. > > The client will not uninstall unless either something is backed > up or /etc/ipa/default.conf file does exist. > > The client will not install if something is backed up or > default.conf file does exist (unless it's installation on master). > > https://fedorahosted.org/freeipa/ticket/3331 > > Tomas > > Can we create a function testing if ipa client is installed to avoid duplication of the decision logic? Something like is_ipa_configured present in ipaserver/install/installutils.py. Keep in mind that we cannot use ipaserver package as it may not be present on client. Martin From pviktori at redhat.com Thu Feb 21 11:50:33 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 21 Feb 2013 12:50:33 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <5124F71D.701@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> <511B6874.5060106@redhat.com> <51235ED4.2000801@redhat.com> <5124F71D.701@redhat.com> Message-ID: <51260A09.3050001@redhat.com> On 02/20/2013 05:17 PM, Martin Kosek wrote: > On 02/19/2013 12:15 PM, Petr Viktorin wrote: >> On 02/13/2013 11:18 AM, Petr Viktorin wrote: >>> On 01/29/2013 05:06 PM, Petr Viktorin wrote: >>>> On 01/04/2013 07:20 PM, Petr Viktorin wrote: >>>>> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>>>>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>>>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>>>>> It's time to give this to another set of eyes :) >>>>>>>>> >>>>>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>>>>> >>>>>>>>> More info is in commit messages. >>>>>>>>> >>>>>>>>> >>>>>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>>>>> change the >>>>>>>>> design document: when the client doesn't send the API version, it is >>>>>>>>> assumed >>>>>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>>>>> The client still gets a warning if the version is missing. Except >>>>>>>>> for >>>>>>>>> those >>>>>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>>>>> the >>>>>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>>>>> our client >>>>>>>>> is so far the only one that validates so strictly.) >>>>>>>> >>>>>>>> I did a basic test of this patch and also quickly read through the >>>>>>>> patches and >>>>>>>> besides nitpicks (like unused inspect module in >>>>>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>>>>> obvious >>>>>>>> errors in the Python code. >>>>>>> >>>>>>> Noted, will fix in future versions of the patch. >>>>>>> >>>>>>>> >>>>>>>> However, this patch breaks WebUI badly, I did not even get to a >>>>>>>> log in >>>>>>>> screen. >>>>>>>> Cooperation with Petr Vobornik will be needed. In my case, I got >>>>>>>> blank >>>>>>>> screen >>>>>>>> and Javascript error: >>>>>>>> >>>>>>>> TypeError: IPA.messages.dialogs is undefined >>>>>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>>>>> Line 1460 >>>>>>>> >>>>>>>> I assume this is related to the Internal Error that was returned in >>>>>>>> the JSON call >>>>>>>> >>>>>>>> { >>>>>>>> "error": null, >>>>>>>> "id": null, >>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>> "result": { >>>>>>>> "count": 5, >>>>>>>> "results": [ >>>>>>>> { >>>>>>>> "error": "an internal error has occurred", >>>>>>>> "error_code": 903, >>>>>>>> "error_name": "InternalError" >>>>>>>> }, >>>>>>>> { >>>>>>>> ... >>>>>>>> >>>>>>>> This can be reproduced with: >>>>>>>> >>>>>>>> # curl -v -H "Content-Type:application/json" -H >>>>>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>>>>> --negotiate -u : >>>>>>>> --cacert /etc/ipa/ca.crt -d >>>>>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>>>>> https://`hostname`/ipa/json >>>>>>> >>>>>>> Good catch! The i18n_messages plugin already defines a "messages" >>>>>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>>>>> check for clashes. >>>>>>> Since i18n_messages is an internal command only used by the Web UI, we >>>>>>> can rename its output to "texts" without breaking compatibility. >>>>>>> >>>>>>> I'm attaching a preliminary fix (for both backend and UI), but >>>>>>> hopefully >>>>>>> it won't be necessary, see below. >>>>>>> >>>>>>>> I am also not sure I like the requirement of a specific version >>>>>>>> option >>>>>>>> to be >>>>>>>> always passed. I would prefer that missing version option would mean >>>>>>>> "I use the >>>>>>>> most recent version of API" instead - it would make the custom >>>>>>>> JSONRPC/XMLRPC >>>>>>>> calls easier to use. >>>>>>>> >>>>>>>> But since the version option was not being sent for some commands, we >>>>>>>> may not >>>>>>>> have a choice anyway if we do not want to break old clients in >>>>>>>> case we >>>>>>>> add some >>>>>>>> capabilities to these commands. >>>>>>>> >>>>>>> >>>>>>> I see three other options, all worse: >>>>>>> - Do not use capabilities for the affected commands, meaning no new >>>>>>> functionality can be added to them (and by extension, no new >>>>>>> functionality common to all commands can be added). >>>>>>> - Treat a missing version as the current version >>>>>>> - Break backwards compatibility >>>>>>> >>>>>>> And one possibly better (thanks to Petr? and Martin for opening my >>>>>>> eyes >>>>>>> off-list!): >>>>>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>>>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>>>>> >>>>>>> This would also allow us to solve the above name-clashing problem >>>>>>> elegantly. Here is a reminder of what a JSON response looks like: >>>>>>> >>>>>>> { >>>>>>> "error": null, >>>>>>> "id": 0, >>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>> "result": { >>>>>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>>>>> 2.47" >>>>>>> }, >>>>>>> "version": "3.1.0GIT2e4bd02" >>>>>>> } >>>>>>> >>>>>>> A XML-RPC response only contains the "result" part of that. >>>>>>> So with JSON, we can put the messages in the top level, which is much >>>>>>> better design. >>>>> >>>>> Custom info in the "top level" seems to be a violation of the JSON-RPC >>>>> spec. I'd rather not do more of those, so I'm withdrawing this idea. >>>>> >>>>>>> >>>>>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>>>>> inability to attach extra info to errors (commit >>>>>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>>>>> PublicError). >>>>>>> >>>>>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>>>>> >>>>>> >>>>>> +1, XML-RPC sucks. This should have been done a long time ago. >>>>>> >>>>>> Honza >>>>>> >>>>> >>>>> Here are new patches. >>>>> >>>>> XML-RPC requests with missing version are assumed to be old (the version >>>>> before capabilities are introduced, 2.47). This takes care of backcompat >>>>> with clients with bug 3294. >>>>> JSON-RPC requests with missing version are assumed to be testing calls >>>>> (e.g. curl), they get behavior of the latest version but also a warning. >>>>> I've also added this info to the design doc. >>>>> >>>>> It turns out that these patches don't depend on whether our client uses >>>>> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >>>>> extra unit tests. >>>>> >>>> >>>> Patch 106 had a minor conflict with master, attaching fixed version. >>>> >>> >>> Patches 106 & 115 need an API version bump. >>> I also noticed that `makeapi --validate` wasn't checking capabilities >>> properly. Fixed. >>> >>> >> >> Rebasing patch 104 to current master. >> > > Patch 104 and 106 needs a rebase. > > Generally I think this patchset is OK and we will need it as a foundation for > other features. > > I may have done my custom rebasing wrong, but my WebUI stopped working with > these patches. I see this in error_log: > > [Wed Feb 20 10:38:22.351845 2013] [auth_kerb:error] [pid 22172] [client > 10.34.4.72:40777] gss_display_name() failed: A required input > parameter could not be read: An invalid name was supplied (, Unknown error), > referer: https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ > > > I also saw one failed test case: > ====================================================================== > FAIL: Try user_show with no version > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest > self.test(*self.arg) > File "/root/freeipa-master/tests/test_xmlrpc/test_user_plugin.py", line 1760, > in test_user_show_without_version > assert res['messages'] == (expected_message.to_dict(), ) > AssertionError > > Martin > Here are the rebased patches. Please test from a clean tree, the UI stopped working here when I had some stale files around. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0104-03-Add-the-version-option-to-all-Commands.patch Type: text/x-patch Size: 68325 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0105-03-Add-ipalib.messages.patch Type: text/x-patch Size: 17068 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0106-05-Add-client-capabilities-enable-messages.patch Type: text/x-patch Size: 20543 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0115-04-Rename-the-messages-Output-of-the-i18n_messages-comm.patch Type: text/x-patch Size: 3739 bytes Desc: not available URL: From pvoborni at redhat.com Thu Feb 21 12:43:57 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 21 Feb 2013 13:43:57 +0100 Subject: [Freeipa-devel] [PATCHES] 228-237 Confirmation of dialogs by keyboard, better password dialogs In-Reply-To: <50EAB726.7080902@redhat.com> References: <50AA65E7.4070301@redhat.com> <50AA6926.6000804@redhat.com> <50AB7D76.6010809@redhat.com> <50B4E163.5080707@redhat.com> <50EAB726.7080902@redhat.com> Message-ID: <5126168D.6080709@redhat.com> On 01/07/2013 12:53 PM, Petr Vobornik wrote: > On 11/27/2012 04:50 PM, Endi Sukma Dewata wrote: >> On 11/20/2012 6:54 AM, Petr Vobornik wrote: >>> New design page: >>> http://www.freeipa.org/page/V3/WebUI_keyboard_confirmation >>> >>> Link to design page was added to tickets #3200 and #2910. >>> >>> In the ticket list of previous mail is a mistake. This effort is related >>> to tickets #3200, #2910 and #2884. Probably commit messages should be >>> amended. >> >> ACK. > > Pushed to master. > Pushed to ipa-3-1 -- Petr Vobornik From pvoborni at redhat.com Thu Feb 21 12:44:03 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 21 Feb 2013 13:44:03 +0100 Subject: [Freeipa-devel] [PATCH] 253 Enable mod_deflate In-Reply-To: <50F8259F.2020308@redhat.com> References: <50EC4DE0.1070100@redhat.com> <50EC5026.7050109@redhat.com> <50EC51D8.7060609@redhat.com> <50ED1F47.2050405@redhat.com> <50ED4D75.9040103@redhat.com> <50ED6B6B.5010406@redhat.com> <50ED983F.6040705@redhat.com> <50F8259F.2020308@redhat.com> Message-ID: <51261693.7070304@redhat.com> On 01/17/2013 05:23 PM, Petr Vobornik wrote: > On 01/09/2013 05:18 PM, Petr Viktorin wrote: >> On 01/09/2013 02:06 PM, Petr Vobornik wrote: >>> On 01/09/2013 11:59 AM, Petr Viktorin wrote: >>>> On 01/09/2013 08:41 AM, Martin Kosek wrote: >>>>> On 01/08/2013 06:05 PM, Petr Vobornik wrote: >>>>>> On 01/08/2013 05:58 PM, Rob Crittenden wrote: >>>>>>> Petr Vobornik wrote: 8><------ >>>> >>> >>> Thanks. Compression for application/x-font-woff removed. Updated patch >>> attached. >>> >> >> ACK >> >> > Pushed to master > Pushed to ipa-3-1 -- Petr Vobornik From pvoborni at redhat.com Thu Feb 21 12:44:05 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 21 Feb 2013 13:44:05 +0100 Subject: [Freeipa-devel] [PATCH] 254 Fix BuildRequires: rhino replaced with java-1.7.0-openjdk In-Reply-To: <50FEB932.4000208@redhat.com> References: <50FE715F.6080007@redhat.com> <50FEB833.5080000@redhat.com> <50FEB932.4000208@redhat.com> Message-ID: <51261695.7040106@redhat.com> On 01/22/2013 05:07 PM, Petr Vobornik wrote: > On 01/22/2013 05:02 PM, Rob Crittenden wrote: >> Petr Vobornik wrote: >>> Rhino is needed for Web UI build. Rhino needs java, but from package >>> perspective java-1.7.0-openjdk requires rhino. So the correct >>> BuildRequires is java-1.7.0-openjdk. >> >> ACK >> > Pushed to master. > Pushed to ipa-3-1 -- Petr Vobornik From pvoborni at redhat.com Thu Feb 21 12:44:07 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 21 Feb 2013 13:44:07 +0100 Subject: [Freeipa-devel] [PATCH] 240-252 AMD modules and Web UI build In-Reply-To: <50F95924.5070800@redhat.com> References: <50EC4D5B.5080501@redhat.com> <50F76F07.6030405@redhat.com> <50F7F635.6020609@redhat.com> <50F8AF62.7030809@redhat.com> <50F95924.5070800@redhat.com> Message-ID: <51261697.7040305@redhat.com> On 01/18/2013 03:16 PM, Petr Vobornik wrote: > On 01/18/2013 03:11 AM, Endi Sukma Dewata wrote: >> On 1/17/2013 8:01 PM, Petr Vobornik wrote: >>> On 01/17/2013 04:24 AM, Endi Sukma Dewata wrote: >>>> Nice work! They seem to be working fine so it's ACKed. >>> >>> I found a little error - there is a jsl problem in dojo.profile:86 - >>> comma at the end of a list. Updated patch 243 attached. >> >> ACK. > > Pushed to master. > Pushed to ipa-3-1 -- Petr Vobornik From pvoborni at redhat.com Thu Feb 21 12:48:07 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 21 Feb 2013 13:48:07 +0100 Subject: [Freeipa-devel] Web UI: ipa-3-1 and master are equal Message-ID: <51261787.1040202@redhat.com> I pushed Web UI patches which were in master but not in ipa-3-1 branch to ipa-3-1 branch. > 2808a5ac01483275a75c0c0cbf9836a7e21a2041 Fix BuildRequires: rhino replaced with java-1.7.0-openjdk > 95a191e98177d56bee32d5844f34241d5b987663 Change tests to use AMD loader > aec35465fed1771e45f9755c80781bf478087fa9 Updated makefiles to build FreeIPA Web UI layer > 591041d54d22703153ff2d7793d2a45a634152ff Change Web UI sources to simple AMD modules > e472a3419e60cda05978994db3915962a241a86f AMD config file > 193d0ac9d23b0a8873846e2797e001fe83e1d2f3 Update JavaScript Lint configuration file > 5ae3f90500cac3bbe26210635a1fbd304def474c Move of core Web UI files to AMD directory > 27d9348139ef7b603bc515cf5bbbac78f1e416a2 Move of Web UI non AMD dep. libs to libs subdirectory > 16f9038b3e105d9ac6099be25404c3e6cf31d89c Web UI Sync development utility > 146f3a73b00cbbafc734578c767b932030bc1aec Web UI development environment directory structure and configuration > 2ae4e649a72838cbbfa68818c183d20fcf11151c Minimal Dojo layer > 4a4cbdecc2c1fec6666377fe6839a1897d2c6dd3 Config files for builder of FreeIPA UI layer > e8db24fdb989e6cb68f7a6eb8cdbca1b77c35721 Dojo Builder > d7b25b0bd34568d28f74a3dfc27f520bc7f094b9 Use Uglify.js for JS optimization > 69cbbd10a11736754e5137e394b3f87291cb0fd2 Enable mod_deflate > 06a6ef87d81a0300c716dac47a998dcbc38fbcc2 Focus first input element after 'Add and Add another' > 8e3b09310e615036570ab29fb7c2d1f3009aa2b2 Standardize login password reset, user reset password and host set OTP dialogs > d68dd2138d546673c1dae90e6dfc977ed0b82cd4 Confirm association dialogs by enter > b1632ffa52b2a2f30f3d83aaabc958e4a9928c0e Focus last dialog when some is closed > 69185978eb8b9b206fda13af49e9885cc4a150b7 Confirm error dialog by enter > 40a5d90cfe2077d847841ce46a859ca9ade97a61 Confirm adder dialog by enter > c3de70c77bae2b184b52b6029c4146b8312be784 Confirm mixin > c66c8202919f10a65fddf3b97355a87c39b8ba2b Make confirm_dialog a base class for message_dialog > 92aa635be56fe6e991e902ece369a7bbf57a018d Make confirm_dialog a base class for deleter dialog > 9806eae286764e35a218d9e3f832a6c85a7ca929 Make confirm_dialog a base class of revoke and restore certificate dialogs > b238fc13ef98afb0bae4c479be1fc2c7fa94468d Avoid internal error when user is not Trust admin -- Petr Vobornik From tbabej at redhat.com Thu Feb 21 12:29:49 2013 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 21 Feb 2013 13:29:49 +0100 Subject: [Freeipa-devel] [PATCH 0035] Use default.conf as flag of IPA client being installed In-Reply-To: <5126094E.1040503@redhat.com> References: <512497ED.6080200@redhat.com> <5126094E.1040503@redhat.com> Message-ID: <5126133D.1080001@redhat.com> On 02/21/2013 12:47 PM, Martin Kosek wrote: > On 02/20/2013 10:31 AM, Tomas Babej wrote: >> Hi, >> >> When installing / uninstalling IPA client, the checks that >> determine whether IPA client is installed now take the existence >> of /etc/ipa/default.conf into consideration. >> >> The client will not uninstall unless either something is backed >> up or /etc/ipa/default.conf file does exist. >> >> The client will not install if something is backed up or >> default.conf file does exist (unless it's installation on master). >> >> https://fedorahosted.org/freeipa/ticket/3331 >> >> Tomas >> >> > Can we create a function testing if ipa client is installed to avoid > duplication of the decision logic? Something like is_ipa_configured present in > ipaserver/install/installutils.py. > > Keep in mind that we cannot use ipaserver package as it may not be present on > client. > > Martin Moved to is_ipa_client_installed function to ipautils.py Updated patch attached. Tomas -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0035-2-Use-default.conf-as-flag-of-IPA-client-being-install.patch Type: text/x-patch Size: 3487 bytes Desc: not available URL: From mkosek at redhat.com Thu Feb 21 12:50:40 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 13:50:40 +0100 Subject: [Freeipa-devel] [PATCH 0035] Use default.conf as flag of IPA client being installed In-Reply-To: <5126133D.1080001@redhat.com> References: <512497ED.6080200@redhat.com> <5126094E.1040503@redhat.com> <5126133D.1080001@redhat.com> Message-ID: <51261820.10809@redhat.com> On 02/21/2013 01:29 PM, Tomas Babej wrote: > On 02/21/2013 12:47 PM, Martin Kosek wrote: >> On 02/20/2013 10:31 AM, Tomas Babej wrote: >>> Hi, >>> >>> When installing / uninstalling IPA client, the checks that >>> determine whether IPA client is installed now take the existence >>> of /etc/ipa/default.conf into consideration. >>> >>> The client will not uninstall unless either something is backed >>> up or /etc/ipa/default.conf file does exist. >>> >>> The client will not install if something is backed up or >>> default.conf file does exist (unless it's installation on master). >>> >>> https://fedorahosted.org/freeipa/ticket/3331 >>> >>> Tomas >>> >>> >> Can we create a function testing if ipa client is installed to avoid >> duplication of the decision logic? Something like is_ipa_configured present in >> ipaserver/install/installutils.py. >> >> Keep in mind that we cannot use ipaserver package as it may not be present on >> client. >> >> Martin > Moved to is_ipa_client_installed function to ipautils.py > > Updated patch attached. > > Tomas > You just created a nice import loop: # make rpms ... if [ "" != "yes" ]; then \ ./makeapi --validate; \ fi Traceback (most recent call last): File "./makeapi", line 457, in sys.exit(main()) File "./makeapi", line 428, in main api.finalize() File "/root/freeipa-master/ipalib/plugable.py", line 674, in finalize self.__do_if_not_done('load_plugins') File "/root/freeipa-master/ipalib/plugable.py", line 454, in __do_if_not_done getattr(self, name)() File "/root/freeipa-master/ipalib/plugable.py", line 613, in load_plugins self.import_plugins('ipalib') File "/root/freeipa-master/ipalib/plugable.py", line 655, in import_plugins __import__(fullname) File "/root/freeipa-master/ipalib/plugins/cert.py", line 30, in from ipalib import pkcs10 File "/root/freeipa-master/ipalib/pkcs10.py", line 24, in from ipapython import ipautil File "/root/freeipa-master/ipapython/ipautil.py", line 52, in from ipapython import sysrestore File "/root/freeipa-master/ipapython/sysrestore.py", line 34, in from ipapython import ipautil ImportError: cannot import name ipautil make: *** [version-update] Error 1 Martin From pviktori at redhat.com Thu Feb 21 12:58:20 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 21 Feb 2013 13:58:20 +0100 Subject: [Freeipa-devel] [PATCH] 0182 Fix permission validation and normalization in aci.py Message-ID: <512619EC.3010601@redhat.com> This patch fixes an internal error in the permission plugin that would become more noticeable when CSV is dropped. https://fedorahosted.org/freeipa/ticket/3420 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0182-Fix-permission-validation-and-normalization-in-aci.p.patch Type: text/x-patch Size: 1939 bytes Desc: not available URL: From mkosek at redhat.com Thu Feb 21 13:06:49 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 14:06:49 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <51260A09.3050001@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> <511B6874.5060106@redhat.com> <51235ED4.2000801@redhat.com> <5124F71D.701@redhat.com> <51260A09.3050001@redhat.com> Message-ID: <51261BE9.4050107@redhat.com> On 02/21/2013 12:50 PM, Petr Viktorin wrote: > On 02/20/2013 05:17 PM, Martin Kosek wrote: >> On 02/19/2013 12:15 PM, Petr Viktorin wrote: >>> On 02/13/2013 11:18 AM, Petr Viktorin wrote: >>>> On 01/29/2013 05:06 PM, Petr Viktorin wrote: >>>>> On 01/04/2013 07:20 PM, Petr Viktorin wrote: >>>>>> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>>>>>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>>>>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>>>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>>>>>> It's time to give this to another set of eyes :) >>>>>>>>>> >>>>>>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>>>>>> >>>>>>>>>> More info is in commit messages. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>>>>>> change the >>>>>>>>>> design document: when the client doesn't send the API version, it is >>>>>>>>>> assumed >>>>>>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>>>>>> The client still gets a warning if the version is missing. Except >>>>>>>>>> for >>>>>>>>>> those >>>>>>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>>>>>> the >>>>>>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>>>>>> our client >>>>>>>>>> is so far the only one that validates so strictly.) >>>>>>>>> >>>>>>>>> I did a basic test of this patch and also quickly read through the >>>>>>>>> patches and >>>>>>>>> besides nitpicks (like unused inspect module in >>>>>>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>>>>>> obvious >>>>>>>>> errors in the Python code. >>>>>>>> >>>>>>>> Noted, will fix in future versions of the patch. >>>>>>>> >>>>>>>>> >>>>>>>>> However, this patch breaks WebUI badly, I did not even get to a >>>>>>>>> log in >>>>>>>>> screen. >>>>>>>>> Cooperation with Petr Vobornik will be needed. In my case, I got >>>>>>>>> blank >>>>>>>>> screen >>>>>>>>> and Javascript error: >>>>>>>>> >>>>>>>>> TypeError: IPA.messages.dialogs is undefined >>>>>>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>>>>>> Line 1460 >>>>>>>>> >>>>>>>>> I assume this is related to the Internal Error that was returned in >>>>>>>>> the JSON call >>>>>>>>> >>>>>>>>> { >>>>>>>>> "error": null, >>>>>>>>> "id": null, >>>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>>> "result": { >>>>>>>>> "count": 5, >>>>>>>>> "results": [ >>>>>>>>> { >>>>>>>>> "error": "an internal error has occurred", >>>>>>>>> "error_code": 903, >>>>>>>>> "error_name": "InternalError" >>>>>>>>> }, >>>>>>>>> { >>>>>>>>> ... >>>>>>>>> >>>>>>>>> This can be reproduced with: >>>>>>>>> >>>>>>>>> # curl -v -H "Content-Type:application/json" -H >>>>>>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>>>>>> --negotiate -u : >>>>>>>>> --cacert /etc/ipa/ca.crt -d >>>>>>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>>>>>> https://`hostname`/ipa/json >>>>>>>> >>>>>>>> Good catch! The i18n_messages plugin already defines a "messages" >>>>>>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>>>>>> check for clashes. >>>>>>>> Since i18n_messages is an internal command only used by the Web UI, we >>>>>>>> can rename its output to "texts" without breaking compatibility. >>>>>>>> >>>>>>>> I'm attaching a preliminary fix (for both backend and UI), but >>>>>>>> hopefully >>>>>>>> it won't be necessary, see below. >>>>>>>> >>>>>>>>> I am also not sure I like the requirement of a specific version >>>>>>>>> option >>>>>>>>> to be >>>>>>>>> always passed. I would prefer that missing version option would mean >>>>>>>>> "I use the >>>>>>>>> most recent version of API" instead - it would make the custom >>>>>>>>> JSONRPC/XMLRPC >>>>>>>>> calls easier to use. >>>>>>>>> >>>>>>>>> But since the version option was not being sent for some commands, we >>>>>>>>> may not >>>>>>>>> have a choice anyway if we do not want to break old clients in >>>>>>>>> case we >>>>>>>>> add some >>>>>>>>> capabilities to these commands. >>>>>>>>> >>>>>>>> >>>>>>>> I see three other options, all worse: >>>>>>>> - Do not use capabilities for the affected commands, meaning no new >>>>>>>> functionality can be added to them (and by extension, no new >>>>>>>> functionality common to all commands can be added). >>>>>>>> - Treat a missing version as the current version >>>>>>>> - Break backwards compatibility >>>>>>>> >>>>>>>> And one possibly better (thanks to Petr? and Martin for opening my >>>>>>>> eyes >>>>>>>> off-list!): >>>>>>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>>>>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>>>>>> >>>>>>>> This would also allow us to solve the above name-clashing problem >>>>>>>> elegantly. Here is a reminder of what a JSON response looks like: >>>>>>>> >>>>>>>> { >>>>>>>> "error": null, >>>>>>>> "id": 0, >>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>> "result": { >>>>>>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>>>>>> 2.47" >>>>>>>> }, >>>>>>>> "version": "3.1.0GIT2e4bd02" >>>>>>>> } >>>>>>>> >>>>>>>> A XML-RPC response only contains the "result" part of that. >>>>>>>> So with JSON, we can put the messages in the top level, which is much >>>>>>>> better design. >>>>>> >>>>>> Custom info in the "top level" seems to be a violation of the JSON-RPC >>>>>> spec. I'd rather not do more of those, so I'm withdrawing this idea. >>>>>> >>>>>>>> >>>>>>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>>>>>> inability to attach extra info to errors (commit >>>>>>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>>>>>> PublicError). >>>>>>>> >>>>>>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>>>>>> >>>>>>> >>>>>>> +1, XML-RPC sucks. This should have been done a long time ago. >>>>>>> >>>>>>> Honza >>>>>>> >>>>>> >>>>>> Here are new patches. >>>>>> >>>>>> XML-RPC requests with missing version are assumed to be old (the version >>>>>> before capabilities are introduced, 2.47). This takes care of backcompat >>>>>> with clients with bug 3294. >>>>>> JSON-RPC requests with missing version are assumed to be testing calls >>>>>> (e.g. curl), they get behavior of the latest version but also a warning. >>>>>> I've also added this info to the design doc. >>>>>> >>>>>> It turns out that these patches don't depend on whether our client uses >>>>>> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >>>>>> extra unit tests. >>>>>> >>>>> >>>>> Patch 106 had a minor conflict with master, attaching fixed version. >>>>> >>>> >>>> Patches 106 & 115 need an API version bump. >>>> I also noticed that `makeapi --validate` wasn't checking capabilities >>>> properly. Fixed. >>>> >>>> >>> >>> Rebasing patch 104 to current master. >>> >> >> Patch 104 and 106 needs a rebase. >> >> Generally I think this patchset is OK and we will need it as a foundation for >> other features. >> >> I may have done my custom rebasing wrong, but my WebUI stopped working with >> these patches. I see this in error_log: >> >> [Wed Feb 20 10:38:22.351845 2013] [auth_kerb:error] [pid 22172] [client >> 10.34.4.72:40777] gss_display_name() failed: A required input >> parameter could not be read: An invalid name was supplied (, Unknown error), >> referer: https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ >> >> >> I also saw one failed test case: >> ====================================================================== >> FAIL: Try user_show with no version >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >> self.test(*self.arg) >> File "/root/freeipa-master/tests/test_xmlrpc/test_user_plugin.py", line 1760, >> in test_user_show_without_version >> assert res['messages'] == (expected_message.to_dict(), ) >> AssertionError >> >> Martin >> > > Here are the rebased patches. > Please test from a clean tree, the UI stopped working here when I had some > stale files around. > Thanks, everything looks good except our first message (VersionMissing) which is sent when I run commands via JSON-RPC which do not have any version attached. Now, I get this error: "messages": [ { "code": 13001, "message": "API Version number was not sent. Assuming client version 2.47. Current API version is 2.53", "name": "VersionMissing", "type": "warning" } ], There are 2 issues with this message wording: 1) 2.47 is deprecated, I assume you wanted VERSION_WITHOUT_CAPABILITIES 2) We assume API_VERSION (current API version) and not VERSION_WITHOUT_CAPABILITIES Martin From pvoborni at redhat.com Thu Feb 21 13:13:13 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 21 Feb 2013 14:13:13 +0100 Subject: [Freeipa-devel] [PATCH] 256-258 Web UI: removed build dependency errors In-Reply-To: <5122EDA8.7030308@redhat.com> References: <512256D1.8040101@redhat.com> <5122EDA8.7030308@redhat.com> Message-ID: <51261D69.7040308@redhat.com> On 02/19/2013 04:12 AM, Endi Sukma Dewata wrote: > ACK. > Pushed to master, ipa-3-1 -- Petr Vobornik From mkosek at redhat.com Thu Feb 21 13:22:25 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 14:22:25 +0100 Subject: [Freeipa-devel] [PATCH] 372 Use fixed test domain in realmdomains test Message-ID: <51261F91.1060802@redhat.com> Fixes test unit broken on some systems. Pushed as a one-liner. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-372-use-fixed-test-domain-in-realmdomains-test.patch Type: text/x-patch Size: 1332 bytes Desc: not available URL: From mkosek at redhat.com Thu Feb 21 13:22:59 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 14:22:59 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <5124DB7F.3020101@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> <20130220132403.GN476@redhat.com> <5124DB7F.3020101@redhat.com> Message-ID: <51261FB3.8030005@redhat.com> On 02/20/2013 03:19 PM, Tomas Babej wrote: > On Wed 20 Feb 2013 02:24:03 PM CET, Alexander Bokovoy wrote: >> On Wed, 20 Feb 2013, Tomas Babej wrote: >>> On 12/21/2012 12:15 PM, Tomas Babej wrote: >>>> Hi, >>>> >>>> Sending updated and rebased versions of patches 0024 and 0025. >>>> >>>> Tomas >>>> >>>> >>> Sending rebased version, these got quite rotten. >> Thanks for updating them. >> >>> @@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): >>> 'not be found. Please specify the SID >>> directly ' >>> 'using dom-sid option.')) >>> >>> - try: >>> - (old_dn, old_attrs) = ldap.get_entry(dn, >>> - ['ipabaseid', >>> - 'ipaidrangesize', >>> - 'ipabaserid', >>> - 'ipasecondarybaserid']) >>> - except errors.NotFound: >>> - self.obj.handle_not_found(*keys) >>> + if in_updated_attrs('ipanttrusteddomainsid'): >>> + if in_updated_attrs('ipasecondarybaserid'): >>> + raise errors.ValidationError(name='ID Range setup', >>> + error=_('Options dom_sid and secondary_rid_base >>> cannot ' >>> + 'be used together')) >> Since we agreed to refer to options by their CLI name (--dom-sid and >> --secondary-rid-base) in the other patch, it makes sense to use it >> here too. >> >> >>> - if is_set('ipanttrusteddomainsid'): >>> - # Validate SID as the one of trusted domains >>> - >>> self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >>> >>> + if not in_updated_attrs('ipabaserid'): >>> + raise errors.ValidationError(name='ID Range setup', >>> + error=_('Options dom_sid and rid_base must ' >>> + 'be used together')) >> Same here. >> >>> + # secondary base rid must be set if and only if base rid >>> is set >>> + if in_updated_attrs('ipasecondarybaserid') !=\ >>> + in_updated_attrs('ipabaserid'): >>> + raise errors.ValidationError(name='ID Range setup', >>> + error=_('Options secondary_rid_base and rid_base >>> must ' >>> + 'be used together')) >> Same here. >> >>> + dict( >>> + desc='Try to modify ID range %r so it has only primary >>> rid range set' % (testrange8), >>> + command=('idrange_mod', [testrange8], >>> + dict(ipabaserid=testrange8_base_rid)), >>> + expected=errors.ValidationError( >>> + name='ID Range setup', error='Options >>> secondary_rid_base and rid_base must be used together'), >>> + ), >> And synchronize error message here too. >> > > Thanks! > > Sending the updated patch 0024. > > Tomas > In patch 0024 your intention is OK, but the checking functions are not: is_set = lambda x: (x in entry_attrs) and (x is not None) + in_updated_attrs = lambda x: any((x in attrs and x is not None) + for attrs in (entry_attrs, old_attrs)) They return True even when the attribute is None because they check if *x* is None and not if *attrs[x]* is None. Example: # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 --secondary-rid-base=1000000 local_range ---------------------------- Added ID range "local_range" ---------------------------- Range name: local_range First Posix ID of the range: 1200000 Number of IDs in the range: 200000 First RID of the corresponding RID range: 1000 First RID of the secondary RID range: 1000000 Range type: local domain range This command should be NOOP, but is not: # ipa idrange-mod local_range --dom-sid= ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base cannot be used together Martin From mkosek at redhat.com Thu Feb 21 13:39:40 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 14:39:40 +0100 Subject: [Freeipa-devel] patch for trac 2575 In-Reply-To: <478729079.1780206.1360894041401.JavaMail.root@redhat.com> References: <478729079.1780206.1360894041401.JavaMail.root@redhat.com> Message-ID: <5126239C.2050507@redhat.com> Thanks Brian. I still see few issues though: 1) The patch adds a whitespace error: $ git apply ~/freeipa-bcook-0001-Add-DNS-Setup-Prompt-to-Install.patch /home/mkosek/freeipa-bcook-0001-Add-DNS-Setup-Prompt-to-Install.patch:41: trailing whitespace. warning: 1 line adds whitespace errors 2) It does unrelated and unnecessary changes to the main function: --- a/install/tools/ipa-server-install +++ b/install/tools/ipa-server-install @@ -560,10 +560,16 @@ def set_subject_in_config(realm_name, dm_password, suffix, subject_base): conn.disconnect() def main(): + """ + + + :return: + """ global ds global pw_name global uninstalling global installation_cleanup + ds = None safe_options, options = parse_options() 3) In the question, I would replace "bind" with "BIND" as this is how the project should be spelled. I see that few lines above we also refer to BIND with "bind" (it may have caused the confusion), I think this can be fixed too. Martin On 02/15/2013 03:07 AM, Brian Cook wrote: > Thanks Martin and Dmitri. I have attached a patch that I -think- is formatted > correctly... I removed the new variable and added check for --unattended. > > Thanks, > Brian > > > ------------------------------------------------------------------------------- > *From: *"Martin Kosek" > *To: *dpal at redhat.com > *Cc: *freeipa-devel at redhat.com > *Sent: *Wednesday, February 13, 2013 11:16:51 PM > *Subject: *Re: [Freeipa-devel] patch for trac 2575 > > On 02/14/2013 03:49 AM, Dmitri Pal wrote: >> On 02/13/2013 05:20 PM, Brian Cook wrote: >>> Please disregard the first patch as it still asked the user if they want to > install DNS even if --setup-dns was passed, this one is fixed. >>> >>> Brian >> >> Brian, >> >> Thanks for the patch. >> Can you please format it following these guidelines: >> https://fedorahosted.org/freeipa/wiki/PatchFormat >> >> Thanks >> Dmitri > > Hello Brian, > > Thanks for the patch! Also few technical notes: > > 1) There is no need to invent the new variable, you can ask and set > options.setup_dns to True. We already to this in other parts incode > > 2) This patch would --unattended mode when no --setup-dns is passed > > Martin > >>> >>> >>> >>> diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install >>> index 1559107..96ef802 100755 >>> --- a/install/tools/ipa-server-install >>> +++ b/install/tools/ipa-server-install >>> @@ -564,6 +564,7 @@ def main(): >>> global pw_name >>> global uninstalling >>> global installation_cleanup >>> + >>> ds = None >>> >>> safe_options, options = parse_options() >>> @@ -740,8 +741,18 @@ def main(): >>> admin_password = "" >>> reverse_zone = None >>> >>> - # check bind packages are installed >>> + # Setup a variable to use instead of options.setup_dns to enable > interactive DNS selection >>> + setup_dns=False >>> if options.setup_dns: >>> + setup_dns=True >>> + else: >>> + # Ask user if they want to install DNS >>> + if ipautil.user_input("Do you want to configure integrated DNS > (bind)?", False): >>> + setup_dns=True >>> + >>> + >>> + # check bind packages are installed >>> + if setup_dns: >>> if not bindinstance.check_inst(options.unattended): >>> sys.exit("Aborting installation") >>> >>> @@ -827,7 +838,7 @@ def main(): >>> else: >>> admin_password = options.admin_password >>> >>> - if options.setup_dns: >>> + if setup_dns: >>> if options.no_forwarders: >>> dns_forwarders = () >>> elif options.forwarders: >>> @@ -858,7 +869,7 @@ def main(): >>> print "Realm name: %s" % realm_name >>> print >>> >>> - if options.setup_dns: >>> + if setup_dns: >>> print "BIND DNS server will be configured to serve IPA domain with:" >>> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >>> else ", ".join([str(ip) for ip in dns_forwarders])) >>> @@ -1102,7 +1113,7 @@ def main(): >>> persistent_search=options.persistent_search, >>> serial_autoincrement=options.serial_autoincrement, >>> ca_configured=not options.selfsign) >>> - if options.setup_dns: >>> + if setup_dns: >>> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), > bind_pw=dm_password) >>> >>> bind.create_instance() >>> @@ -1147,11 +1158,11 @@ def main(): >>> print "\t\t * 80, 443: HTTP/HTTPS" >>> print "\t\t * 389, 636: LDAP/LDAPS" >>> print "\t\t * 88, 464: kerberos" >>> - if options.setup_dns: >>> + if setup_dns: >>> print "\t\t * 53: bind" >>> print "\t\tUDP Ports:" >>> print "\t\t * 88, 464: kerberos" >>> - if options.setup_dns: >>> + if setup_dns: >>> print "\t\t * 53: bind" >>> if options.conf_ntp: >>> print "\t\t * 123: ntp" >>> >>> >>> >>> >>>> Message: 8 >>>> Date: Wed, 13 Feb 2013 13:39:32 -0800 >>>> From: Brian Cook >>>> To: "freeipa-devel at redhat.com" >>>> Subject: [Freeipa-devel] patch for trac 2575 >>>> Message-ID: <9DD1D1BB-6B86-4EA1-B61B-B208E6BC7152 at redhat.com> >>>> Content-Type: text/plain; charset="windows-1252" >>>> >>>> This is a patch for ticket 2575 on trac: [RFE] Installer wizard should > prompt for DNS. This is my first time submitting a patch so I was looking for > something that seemed relatively easy? >>>> >>>> Thanks, >>>> Brian >>>> >>>> >>>> diff --git a/install/tools/ipa-server-install > b/install/tools/ipa-server-install >>>> index 1559107..d8c4ae5 100755 >>>> --- a/install/tools/ipa-server-install >>>> +++ b/install/tools/ipa-server-install >>>> @@ -564,6 +564,7 @@ def main(): >>>> global pw_name >>>> global uninstalling >>>> global installation_cleanup >>>> + >>>> ds = None >>>> >>>> safe_options, options = parse_options() >>>> @@ -740,8 +741,18 @@ def main(): >>>> admin_password = "" >>>> reverse_zone = None >>>> >>>> - # check bind packages are installed >>>> + # Setup a variable to use instead of options.setup_dns to enable > interactive DNS selection >>>> + setup_dns=False >>>> if options.setup_dns: >>>> + setup_dns=True >>>> + >>>> + # Ask user if they want to install DNS >>>> + if ipautil.user_input("Do you want to cnfigure integrated DNS > (bind)?", false): >>>> + setup_dns=True >>>> + >>>> + >>>> + # check bind packages are installed >>>> + if setup_dns: >>>> if not bindinstance.check_inst(options.unattended): >>>> sys.exit("Aborting installation") >>>> >>>> @@ -827,7 +838,7 @@ def main(): >>>> else: >>>> admin_password = options.admin_password >>>> >>>> - if options.setup_dns: >>>> + if setup_dns: >>>> if options.no_forwarders: >>>> dns_forwarders = () >>>> elif options.forwarders: >>>> @@ -858,7 +869,7 @@ def main(): >>>> print "Realm name: %s" % realm_name >>>> print >>>> >>>> - if options.setup_dns: >>>> + if setup_dns: >>>> print "BIND DNS server will be configured to serve IPA domain with:" >>>> print "Forwarders: %s" % ("No forwarders" if not dns_forwarders \ >>>> else ", ".join([str(ip) for ip in dns_forwarders])) >>>> @@ -1102,7 +1113,7 @@ def main(): >>>> persistent_search=options.persistent_search, >>>> serial_autoincrement=options.serial_autoincrement, >>>> ca_configured=not options.selfsign) >>>> - if options.setup_dns: >>>> + if setup_dns: >>>> api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), > bind_pw=dm_password) >>>> >>>> bind.create_instance() >>>> @@ -1147,11 +1158,11 @@ def main(): >>>> print "\t\t * 80, 443: HTTP/HTTPS" >>>> print "\t\t * 389, 636: LDAP/LDAPS" >>>> print "\t\t * 88, 464: kerberos" >>>> - if options.setup_dns: >>>> + if setup_dns: >>>> print "\t\t * 53: bind" >>>> print "\t\tUDP Ports:" >>>> print "\t\t * 88, 464: kerberos" >>>> - if options.setup_dns: >>>> + if setup_dns: >>>> print "\t\t * 53: bind" >>>> if options.conf_ntp: >>>> print "\t\t * 123: ntp" >>>> >>>> >>>> >>>> >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: > >>>> >>>> ------------------------------ >>>> >>>> _______________________________________________ >>>> Freeipa-devel mailing list >>>> Freeipa-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>>> >>>> End of Freeipa-devel Digest, Vol 69, Issue 49 >>>> ********************************************* >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> >> > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > From tbabej at redhat.com Thu Feb 21 13:58:19 2013 From: tbabej at redhat.com (Tomas Babej) Date: Thu, 21 Feb 2013 14:58:19 +0100 Subject: [Freeipa-devel] [PATCH 0035] Use default.conf as flag of IPA client being installed In-Reply-To: <51261820.10809@redhat.com> References: <512497ED.6080200@redhat.com> <5126094E.1040503@redhat.com> <5126133D.1080001@redhat.com> <51261820.10809@redhat.com> Message-ID: <512627FB.3020709@redhat.com> On 02/21/2013 01:50 PM, Martin Kosek wrote: > On 02/21/2013 01:29 PM, Tomas Babej wrote: >> On 02/21/2013 12:47 PM, Martin Kosek wrote: >>> On 02/20/2013 10:31 AM, Tomas Babej wrote: >>>> Hi, >>>> >>>> When installing / uninstalling IPA client, the checks that >>>> determine whether IPA client is installed now take the existence >>>> of /etc/ipa/default.conf into consideration. >>>> >>>> The client will not uninstall unless either something is backed >>>> up or /etc/ipa/default.conf file does exist. >>>> >>>> The client will not install if something is backed up or >>>> default.conf file does exist (unless it's installation on master). >>>> >>>> https://fedorahosted.org/freeipa/ticket/3331 >>>> >>>> Tomas >>>> >>>> >>> Can we create a function testing if ipa client is installed to avoid >>> duplication of the decision logic? Something like is_ipa_configured present in >>> ipaserver/install/installutils.py. >>> >>> Keep in mind that we cannot use ipaserver package as it may not be present on >>> client. >>> >>> Martin >> Moved to is_ipa_client_installed function to ipautils.py >> >> Updated patch attached. >> >> Tomas >> > You just created a nice import loop: > > ... I made the function part of ipa-client-install script then. We probably will not need to check whether client is installed anywhere else. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0035-3-Use-default.conf-as-flag-of-IPA-client-being-install.patch Type: text/x-patch Size: 2909 bytes Desc: not available URL: From pviktori at redhat.com Thu Feb 21 14:09:20 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 21 Feb 2013 15:09:20 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <51261BE9.4050107@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> <511B6874.5060106@redhat.com> <51235ED4.2000801@redhat.com> <5124F71D.701@redhat.com> <51260A09.3050001@redhat.com> <51261BE9.4050107@redhat.com> Message-ID: <51262A90.4070709@redhat.com> On 02/21/2013 02:06 PM, Martin Kosek wrote: > On 02/21/2013 12:50 PM, Petr Viktorin wrote: >> On 02/20/2013 05:17 PM, Martin Kosek wrote: >>> On 02/19/2013 12:15 PM, Petr Viktorin wrote: >>>> On 02/13/2013 11:18 AM, Petr Viktorin wrote: >>>>> On 01/29/2013 05:06 PM, Petr Viktorin wrote: >>>>>> On 01/04/2013 07:20 PM, Petr Viktorin wrote: >>>>>>> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>>>>>>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>>>>>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>>>>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>>>>>>> It's time to give this to another set of eyes :) >>>>>>>>>>> >>>>>>>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>>>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>>>>>>> >>>>>>>>>>> More info is in commit messages. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>>>>>>> change the >>>>>>>>>>> design document: when the client doesn't send the API version, it is >>>>>>>>>>> assumed >>>>>>>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>>>>>>> The client still gets a warning if the version is missing. Except >>>>>>>>>>> for >>>>>>>>>>> those >>>>>>>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>>>>>>> the >>>>>>>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>>>>>>> our client >>>>>>>>>>> is so far the only one that validates so strictly.) >>>>>>>>>> >>>>>>>>>> I did a basic test of this patch and also quickly read through the >>>>>>>>>> patches and >>>>>>>>>> besides nitpicks (like unused inspect module in >>>>>>>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>>>>>>> obvious >>>>>>>>>> errors in the Python code. >>>>>>>>> >>>>>>>>> Noted, will fix in future versions of the patch. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> However, this patch breaks WebUI badly, I did not even get to a >>>>>>>>>> log in >>>>>>>>>> screen. >>>>>>>>>> Cooperation with Petr Vobornik will be needed. In my case, I got >>>>>>>>>> blank >>>>>>>>>> screen >>>>>>>>>> and Javascript error: >>>>>>>>>> >>>>>>>>>> TypeError: IPA.messages.dialogs is undefined >>>>>>>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>>>>>>> Line 1460 >>>>>>>>>> >>>>>>>>>> I assume this is related to the Internal Error that was returned in >>>>>>>>>> the JSON call >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> "error": null, >>>>>>>>>> "id": null, >>>>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>>>> "result": { >>>>>>>>>> "count": 5, >>>>>>>>>> "results": [ >>>>>>>>>> { >>>>>>>>>> "error": "an internal error has occurred", >>>>>>>>>> "error_code": 903, >>>>>>>>>> "error_name": "InternalError" >>>>>>>>>> }, >>>>>>>>>> { >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> This can be reproduced with: >>>>>>>>>> >>>>>>>>>> # curl -v -H "Content-Type:application/json" -H >>>>>>>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>>>>>>> --negotiate -u : >>>>>>>>>> --cacert /etc/ipa/ca.crt -d >>>>>>>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>>>>>>> https://`hostname`/ipa/json >>>>>>>>> >>>>>>>>> Good catch! The i18n_messages plugin already defines a "messages" >>>>>>>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>>>>>>> check for clashes. >>>>>>>>> Since i18n_messages is an internal command only used by the Web UI, we >>>>>>>>> can rename its output to "texts" without breaking compatibility. >>>>>>>>> >>>>>>>>> I'm attaching a preliminary fix (for both backend and UI), but >>>>>>>>> hopefully >>>>>>>>> it won't be necessary, see below. >>>>>>>>> >>>>>>>>>> I am also not sure I like the requirement of a specific version >>>>>>>>>> option >>>>>>>>>> to be >>>>>>>>>> always passed. I would prefer that missing version option would mean >>>>>>>>>> "I use the >>>>>>>>>> most recent version of API" instead - it would make the custom >>>>>>>>>> JSONRPC/XMLRPC >>>>>>>>>> calls easier to use. >>>>>>>>>> >>>>>>>>>> But since the version option was not being sent for some commands, we >>>>>>>>>> may not >>>>>>>>>> have a choice anyway if we do not want to break old clients in >>>>>>>>>> case we >>>>>>>>>> add some >>>>>>>>>> capabilities to these commands. >>>>>>>>>> >>>>>>>>> >>>>>>>>> I see three other options, all worse: >>>>>>>>> - Do not use capabilities for the affected commands, meaning no new >>>>>>>>> functionality can be added to them (and by extension, no new >>>>>>>>> functionality common to all commands can be added). >>>>>>>>> - Treat a missing version as the current version >>>>>>>>> - Break backwards compatibility >>>>>>>>> >>>>>>>>> And one possibly better (thanks to Petr? and Martin for opening my >>>>>>>>> eyes >>>>>>>>> off-list!): >>>>>>>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>>>>>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>>>>>>> >>>>>>>>> This would also allow us to solve the above name-clashing problem >>>>>>>>> elegantly. Here is a reminder of what a JSON response looks like: >>>>>>>>> >>>>>>>>> { >>>>>>>>> "error": null, >>>>>>>>> "id": 0, >>>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>>> "result": { >>>>>>>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>>>>>>> 2.47" >>>>>>>>> }, >>>>>>>>> "version": "3.1.0GIT2e4bd02" >>>>>>>>> } >>>>>>>>> >>>>>>>>> A XML-RPC response only contains the "result" part of that. >>>>>>>>> So with JSON, we can put the messages in the top level, which is much >>>>>>>>> better design. >>>>>>> >>>>>>> Custom info in the "top level" seems to be a violation of the JSON-RPC >>>>>>> spec. I'd rather not do more of those, so I'm withdrawing this idea. >>>>>>> >>>>>>>>> >>>>>>>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>>>>>>> inability to attach extra info to errors (commit >>>>>>>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>>>>>>> PublicError). >>>>>>>>> >>>>>>>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>>>>>>> >>>>>>>> >>>>>>>> +1, XML-RPC sucks. This should have been done a long time ago. >>>>>>>> >>>>>>>> Honza >>>>>>>> >>>>>>> >>>>>>> Here are new patches. >>>>>>> >>>>>>> XML-RPC requests with missing version are assumed to be old (the version >>>>>>> before capabilities are introduced, 2.47). This takes care of backcompat >>>>>>> with clients with bug 3294. >>>>>>> JSON-RPC requests with missing version are assumed to be testing calls >>>>>>> (e.g. curl), they get behavior of the latest version but also a warning. >>>>>>> I've also added this info to the design doc. >>>>>>> >>>>>>> It turns out that these patches don't depend on whether our client uses >>>>>>> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >>>>>>> extra unit tests. >>>>>>> >>>>>> >>>>>> Patch 106 had a minor conflict with master, attaching fixed version. >>>>>> >>>>> >>>>> Patches 106 & 115 need an API version bump. >>>>> I also noticed that `makeapi --validate` wasn't checking capabilities >>>>> properly. Fixed. >>>>> >>>>> >>>> >>>> Rebasing patch 104 to current master. >>>> >>> >>> Patch 104 and 106 needs a rebase. >>> >>> Generally I think this patchset is OK and we will need it as a foundation for >>> other features. >>> >>> I may have done my custom rebasing wrong, but my WebUI stopped working with >>> these patches. I see this in error_log: >>> >>> [Wed Feb 20 10:38:22.351845 2013] [auth_kerb:error] [pid 22172] [client >>> 10.34.4.72:40777] gss_display_name() failed: A required input >>> parameter could not be read: An invalid name was supplied (, Unknown error), >>> referer: https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ >>> >>> >>> I also saw one failed test case: >>> ====================================================================== >>> FAIL: Try user_show with no version >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >>> self.test(*self.arg) >>> File "/root/freeipa-master/tests/test_xmlrpc/test_user_plugin.py", line 1760, >>> in test_user_show_without_version >>> assert res['messages'] == (expected_message.to_dict(), ) >>> AssertionError >>> >>> Martin >>> >> >> Here are the rebased patches. >> Please test from a clean tree, the UI stopped working here when I had some >> stale files around. >> > > Thanks, everything looks good except our first message (VersionMissing) which > is sent when I run commands via JSON-RPC which do not have any version > attached. Now, I get this error: > > "messages": [ > { > "code": 13001, > "message": "API Version number was not sent. Assuming client > version 2.47. Current API version is 2.53", > "name": "VersionMissing", > "type": "warning" > } > ], > > There are 2 issues with this message wording: > 1) 2.47 is deprecated, I assume you wanted VERSION_WITHOUT_CAPABILITIES > 2) We assume API_VERSION (current API version) and not VERSION_WITHOUT_CAPABILITIES > > Martin > Fixed, thanks for the catch. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0104-04-Add-the-version-option-to-all-Commands.patch Type: text/x-patch Size: 68325 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0105-04-Add-ipalib.messages.patch Type: text/x-patch Size: 17152 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0106-06-Add-client-capabilities-enable-messages.patch Type: text/x-patch Size: 20500 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0115-05-Rename-the-messages-Output-of-the-i18n_messages-comm.patch Type: text/x-patch Size: 3739 bytes Desc: not available URL: From pspacek at redhat.com Thu Feb 21 15:21:24 2013 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 21 Feb 2013 16:21:24 +0100 Subject: [Freeipa-devel] [PATCH 0115] Add support for DNAME substitution Message-ID: <51263B74.9080406@redhat.com> Hello, Add support for DNAME substitution. https://fedorahosted.org/bind-dyndb-ldap/ticket/63 -- Petr^2 Spacek From sbose at redhat.com Thu Feb 21 15:24:46 2013 From: sbose at redhat.com (Sumit Bose) Date: Thu, 21 Feb 2013 16:24:46 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type Message-ID: <20130221152446.GS27418@localhost.localdomain> Hi, this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 The related design page is http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . It was agreed while discussing the design page that the currently hardcoded value for the NFS service will be dropped completely, hence the first patch reverts to patch which added the hardcoded value. The second patch adds the needed attribute to compensate the now missing hardcoded value. In the following three patches the PAC type is read and used accordingly. The last patch adds a unit test for a new function. bye, Sumit -------------- next part -------------- From 8dfe114282730c2a3f02d3714d38c3ee51f332ff Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 09:59:00 +0100 Subject: [PATCH 94/99] Revert "MS-PAC: Special case NFS services" This reverts commit 5269458f552380759c86018cd1f30b64761be92e. With the implementation of https://fedorahosted.org/freeipa/ticket/2960 a special hardcoded handling of NFS service tickets is not needed anymore. --- daemons/ipa-kdb/ipa_kdb_mspac.c | 36 +----------------------------------- 1 file changed, 1 insertion(+), 35 deletions(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index 950000349702038e1d55bd257816f7f61e9557a5..f05c552fc18eb9b15ee3e39b513184589bc8c468 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -743,24 +743,6 @@ static bool is_cross_realm_krbtgt(krb5_const_principal princ) return true; } -static bool is_service_of_type(krb5_const_principal princ, const char *type) -{ - size_t len; - - if (princ->length < 2) { - return false; - } - - len = strlen(type); - - if ((princ->data[0].length == len) || - (strncasecmp(princ->data[0].data, type, len) == 0)) { - return true; - } - - return false; -} - static char *gen_sid_string(TALLOC_CTX *memctx, struct dom_sid *dom_sid, uint32_t rid) { @@ -1555,7 +1537,6 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, krb5_error_code kerr; krb5_pac pac = NULL; krb5_data pac_data; - bool is_nfs = false; /* When using s4u2proxy client_princ actually refers to the proxied user * while client->princ to the proxy service asking for the TGS on behalf @@ -1566,32 +1547,17 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, ks_client_princ = client->princ; } - /* NFS Server on Linux is limited and will choke on big tickets. - * So avoid attachnig the PAC to nfs/ tickets for now. - * FIXME: remove this when we have interface to support disabling - * PACs on arbitrary services */ - if (is_service_of_type(ks_client_princ, "nfs") || - is_service_of_type(server->princ, "nfs")) { - is_nfs = true; - } - is_as_req = ((flags & KRB5_KDB_FLAG_CLIENT_REFERRALS_ONLY) != 0); if (is_as_req && (flags & KRB5_KDB_FLAG_INCLUDE_PAC)) { - if (is_nfs) { - *signed_auth_data = NULL; - kerr = 0; - goto done; - } - kerr = ipadb_get_pac(context, client, &pac); if (kerr != 0 && kerr != ENOENT) { goto done; } } - if (!is_as_req & !is_nfs) { + if (!is_as_req) { /* find the existing PAC, if present */ kerr = krb5_find_authdata(context, tgt_auth_data, NULL, KRB5_AUTHDATA_WIN2K_PAC, &pac_auth_data); -- 1.8.1.2 -------------- next part -------------- From 2b9bd9e90cec36d5a4ff64ed0f9afcc464d55a33 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Thu, 21 Feb 2013 14:27:12 +0100 Subject: [PATCH 95/99] Add ipakrbauthzdata to NFS services during upgrades This update became necessary because instead of hardcoding a value of 'NONE' for ipakrbauthzdata for all NFS services ipakrbauthzdata can now be specified per service. To avoid issues during upgrades ipakrbauthzdata must be set to NONE for all NFS services where this attribute isn't set, because an unset attribute means using the default which might be different from 'NONE'. Another issue might arise on systems where this update is already run and for some NFS services the ipakrbauthzdata attribute is removed manually on purpose. Another update might set it back to 'NONE' which might be irritating. For this reason this plugin only update entries if none of the existing NFS service entries has ipakrbauthzdata set. --- ipaserver/install/plugins/update_services.py | 80 +++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 1 deletion(-) diff --git a/ipaserver/install/plugins/update_services.py b/ipaserver/install/plugins/update_services.py index c384af52fb91692c7f339c5fb2da92ae45984482..64610403cb592849c376fcbef6f14c05f0c0ad28 100644 --- a/ipaserver/install/plugins/update_services.py +++ b/ipaserver/install/plugins/update_services.py @@ -17,7 +17,7 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -from ipaserver.install.plugins import MIDDLE +from ipaserver.install.plugins import MIDDLE, LAST from ipaserver.install.plugins.baseupdate import PostUpdate from ipalib import api, errors from ipapython.dn import DN @@ -93,3 +93,81 @@ class update_service_principalalias(PostUpdate): return (False, False, []) api.register(update_service_principalalias) + +class update_service_nfs_ipakrbauthzdata(PostUpdate): + """ + Update all NFS services which do not have ipakrbauthzdata attribute set. + + Since this plugin uses ipakrbprincipalalias update_service_principalalias + must be run before, 'order' is set accordingly. + + This update became necessary because instead of hardcoding a value of + 'NONE' for ipakrbauthzdata for all NFS services ipakrbauthzdata can now be + specified per service. To avoid issues during upgrades ipakrbauthzdata + must be set to NONE for all NFS services where this attribute isn't set, + because an unset attribute means using the default which might be + different from 'NONE'. + + Another issue might arise on systems where this update is already run and + for some NFS services the ipakrbauthzdata attribute is removed manually on + purpose. Another update might it set back to 'NONE' which might be + irritating. For this reason this plugin only update entries if none of the + existing NFS service entries has ipakrbauthzdata set. + """ + order = LAST + + def execute(self, **options): + ldap = self.obj.backend + + base_dn = DN(api.env.container_service, api.env.basedn) + search_filter = ("(&(objectclass=ipakrbprincipal)" + "(objectclass=ipaservice)" + "(ipakrbprincipalalias=nfs*))") + root_logger.debug("update_service_nfs_ipakrbauthzdata: search for " + "affected services") + + while True: + # run the search in loop to avoid issues when LDAP limits are hit + # during update + try: + (entries, truncated) = ldap.find_entries(search_filter, + ['objectclass', 'ipakrbauthzdata'], base_dn, + time_limit=0, size_limit=0) + except errors.NotFound: + root_logger.debug("update_service_nfs_ipakrbauthzdata: " + "no NFS service to update found") + return (False, False, []) + except errors.ExecutionError, err: + root_logger.error("update_service_nfs_ipakrbauthzdata: cannot " + "retrieve list of NFS services: %s", err) + return (False, False, []) + if not entries: + # no entry was returned, rather break than continue cycling + root_logger.debug("update_service_nfs_ipakrbauthzdata: " + "no NFS service was returned") + return (False, False, []) + root_logger.debug("update_service_nfs_ipakrbauthzdata: found %d " + "NFS services, truncated: %s", + len(entries), truncated) + + for nfs_dn, entry in entries: + if entry.has_key('ipakrbauthzdata'): + root_logger.info("update_service_nfs_ipakrbauthzdata: " + "found a NFS service where " + "ipakrbauthzdata is already set, " + "nothing to do") + return (False, False, []) + + nfs_updates = {} + for nfs_dn, entry in entries: + nfs_updates[nfs_dn] = {'dn': nfs_dn, + 'updates': ['add:ipakrbauthzdata:NONE']} + + if not truncated: + root_logger.debug("update_service_nfs_ipakrbauthzdata: " + "all affected services updated") + return (False, True, [nfs_updates]) + + return (False, False, []) + +api.register(update_service_nfs_ipakrbauthzdata) -- 1.8.1.2 -------------- next part -------------- From c2cef2dac0c12174e10e5865162ffa5fea0464f9 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 11:01:11 +0100 Subject: [PATCH 96/99] ipa-kdb: Read global defaul ipaKrbAuthzData The ipaKrbAuthzData LDAP attribute is read from the ipaConfig object and the read value(s) are stored in the ipadb context. --- daemons/ipa-kdb/ipa_kdb.c | 27 ++++++++++++++++++++++++++- daemons/ipa-kdb/ipa_kdb.h | 3 +++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/daemons/ipa-kdb/ipa_kdb.c b/daemons/ipa-kdb/ipa_kdb.c index 2a344dc69158dbc3f6d0207ab0bb781676240ed9..e5c718ea9ffd38dbd49026e825d4a7f920638181 100644 --- a/daemons/ipa-kdb/ipa_kdb.c +++ b/daemons/ipa-kdb/ipa_kdb.c @@ -40,6 +40,8 @@ struct ipadb_context *ipadb_get_context(krb5_context kcontext) static void ipadb_context_free(krb5_context kcontext, struct ipadb_context **ctx) { + size_t c; + if (*ctx != NULL) { free((*ctx)->uri); free((*ctx)->base); @@ -51,6 +53,12 @@ static void ipadb_context_free(krb5_context kcontext, free((*ctx)->supp_encs); ipadb_mspac_struct_free(&(*ctx)->mspac); krb5_free_default_realm(kcontext, (*ctx)->realm); + + for (c = 0; (*ctx)->authz_data && (*ctx)->authz_data[c]; c++) { + free((*ctx)->authz_data[c]); + } + free((*ctx)->authz_data); + free(*ctx); *ctx = NULL; } @@ -167,13 +175,14 @@ done: int ipadb_get_global_configs(struct ipadb_context *ipactx) { - char *attrs[] = { "ipaConfigString", NULL }; + char *attrs[] = { "ipaConfigString", IPA_KRB_AUTHZ_DATA_ATTR, NULL }; struct berval **vals = NULL; LDAPMessage *res = NULL; LDAPMessage *first; char *base = NULL; int i; int ret; + char **authz_data_list; ret = asprintf(&base, "cn=ipaConfig,cn=etc,%s", ipactx->base); if (ret == -1) { @@ -215,6 +224,22 @@ int ipadb_get_global_configs(struct ipadb_context *ipactx) } } + ret = ipadb_ldap_attr_to_strlist(ipactx->lcontext, first, + IPA_KRB_AUTHZ_DATA_ATTR, &authz_data_list); + if (ret != 0 && ret != ENOENT) { + goto done; + } + if (ret == 0) { + if (ipactx->authz_data != NULL) { + for (i = 0; ipactx->authz_data[i]; i++) { + free(ipactx->authz_data[i]); + } + free(ipactx->authz_data); + } + + ipactx->authz_data = authz_data_list; + } + ret = 0; done: diff --git a/daemons/ipa-kdb/ipa_kdb.h b/daemons/ipa-kdb/ipa_kdb.h index f472f02458e040b921b9f3f85bcc36fa954785d5..7b1576124140ae53cf28a1ed47bfa1acf31cdd59 100644 --- a/daemons/ipa-kdb/ipa_kdb.h +++ b/daemons/ipa-kdb/ipa_kdb.h @@ -74,6 +74,8 @@ #define IPA_SETUP "ipa-setup-override-restrictions" +#define IPA_KRB_AUTHZ_DATA_ATTR "ipaKrbAuthzData" + struct ipadb_mspac; struct ipadb_context { @@ -89,6 +91,7 @@ struct ipadb_context { struct ipadb_mspac *mspac; bool disable_last_success; bool disable_lockout; + char **authz_data; }; #define IPA_E_DATA_MAGIC 0x0eda7a -- 1.8.1.2 -------------- next part -------------- From 8feec5cf1870d9a425ba3af5a4c817ac20ad7a41 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 09:44:32 +0100 Subject: [PATCH 97/99] ipa-kdb: Read ipaKrbAuthzData with other principal data The ipaKrbAuthzData LDAP attribute is read together with the other data of the requestedprincipal and the read value(s) are stored in the e-data of the entry for later use. --- daemons/ipa-kdb/ipa_kdb.h | 1 + daemons/ipa-kdb/ipa_kdb_principals.c | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/daemons/ipa-kdb/ipa_kdb.h b/daemons/ipa-kdb/ipa_kdb.h index 7b1576124140ae53cf28a1ed47bfa1acf31cdd59..9daaab80dc514dd1bb3e85775c1e284d19dac0cd 100644 --- a/daemons/ipa-kdb/ipa_kdb.h +++ b/daemons/ipa-kdb/ipa_kdb.h @@ -105,6 +105,7 @@ struct ipadb_e_data { char **pw_history; struct ipapwd_policy *pol; time_t last_admin_unlock; + char **authz_data; }; struct ipadb_context *ipadb_get_context(krb5_context kcontext); diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c index bb2074bf9c85b0bdcf228696cf8391c11d2384c4..46a921cb2f267b977df4a0042d81a49beccb5a91 100644 --- a/daemons/ipa-kdb/ipa_kdb_principals.c +++ b/daemons/ipa-kdb/ipa_kdb_principals.c @@ -63,6 +63,7 @@ static char *std_principal_attrs[] = { /* IPA SPECIFIC ATTRIBUTES */ "nsaccountlock", "passwordHistory", + IPA_KRB_AUTHZ_DATA_ATTR, "objectClass", NULL @@ -237,6 +238,7 @@ static krb5_error_code ipadb_parse_ldap_entry(krb5_context kcontext, krb5_kvno mkvno = 0; char **restrlist; char *restring; + char **authz_data_list; krb5_timestamp restime; bool resbool; int result; @@ -503,6 +505,17 @@ static krb5_error_code ipadb_parse_ldap_entry(krb5_context kcontext, ied->last_admin_unlock = restime; } + ret = ipadb_ldap_attr_to_strlist(lcontext, lentry, + IPA_KRB_AUTHZ_DATA_ATTR, &authz_data_list); + if (ret != 0 && ret != ENOENT) { + kerr = KRB5_KDB_INTERNAL_ERROR; + goto done; + } + if (ret == 0) { + ied->authz_data = authz_data_list; + } + + kerr = 0; done: @@ -831,6 +844,10 @@ void ipadb_free_principal(krb5_context kcontext, krb5_db_entry *entry) free(ied->pw_history[i]); } free(ied->pw_history); + for (i = 0; ied->authz_data && ied->authz_data[i]; i++) { + free(ied->authz_data[i]); + } + free(ied->authz_data); free(ied->pol); free(ied); } -- 1.8.1.2 -------------- next part -------------- From 4bc702e42960918e5e4fe69c22e84f3e0889dbda Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 14:02:27 +0100 Subject: [PATCH 98/99] ipa-kdb: add PAC only if requested Instead of always adding a PAC to the Kerberos ticket the global default for the authorization data and the authorization data of the service entry is evaluated and the PAC is added accordingly. --- daemons/ipa-kdb/ipa_kdb_mspac.c | 86 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 84 insertions(+), 2 deletions(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index f05c552fc18eb9b15ee3e39b513184589bc8c468..c06c57731799a38410ffee2e51714607b178bbe7 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -96,6 +96,10 @@ static char *memberof_pac_attrs[] = { #define SID_SUB_AUTHS 15 #define MAX(a,b) (((a)>(b))?(a):(b)) +#define AUTHZ_DATA_TYPE_PAC "MS-PAC" +#define AUTHZ_DATA_TYPE_PAD "PAD" +#define AUTHZ_DATA_TYPE_NONE "NONE" + static int string_to_sid(char *str, struct dom_sid *sid) { unsigned long val; @@ -1515,6 +1519,71 @@ done: return kerr; } +void get_authz_data_types(krb5_context context, krb5_db_entry *entry, + bool *_with_pac, bool *_with_pad) +{ + struct ipadb_e_data *ied = NULL; + struct ipadb_context *ipactx; + size_t c; + bool none_found = false; + char **authz_data_list; + bool with_pac = false; + bool with_pad = false; + + if (entry != NULL) { + ied = (struct ipadb_e_data *) entry->e_data; + } + + if (ied == NULL || ied->authz_data == NULL) { + if (context == NULL) { + krb5_klog_syslog(LOG_ERR, "Missing Kerberos context, no " \ + "authorization data will be added."); + goto done; + } + + ipactx = ipadb_get_context(context); + if (ipactx == NULL || ipactx->authz_data == NULL) { + krb5_klog_syslog(LOG_ERR, "No default authorization data types " \ + "available, no authorization data will " \ + "be added."); + goto done; + } + + authz_data_list = ipactx->authz_data; + } else { + authz_data_list = ied->authz_data; + } + + + for (c = 0; authz_data_list[c]; c++) { + if (strcmp(authz_data_list[c], AUTHZ_DATA_TYPE_PAC) == 0) { + with_pac = true; + } else if (strcmp(authz_data_list[c], AUTHZ_DATA_TYPE_PAD) == 0) { + with_pad = true; + } else if (strcmp(authz_data_list[c], AUTHZ_DATA_TYPE_NONE) == 0) { + none_found = true; + } else { + krb5_klog_syslog(LOG_ERR, "Ignoring unsupported " \ + "authorization data type [%s].", + authz_data_list[c]); + } + } + +done: + if (none_found) { + with_pac = false; + with_pad = false; + } + + if (_with_pac != NULL) { + *_with_pac = with_pac; + } + if (_with_pad != NULL) { + *_with_pad = with_pad; + } + +} + krb5_error_code ipadb_sign_authdata(krb5_context context, unsigned int flags, krb5_const_principal client_princ, @@ -1537,6 +1606,8 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, krb5_error_code kerr; krb5_pac pac = NULL; krb5_data pac_data; + bool with_pac; + bool with_pad; /* When using s4u2proxy client_princ actually refers to the proxied user * while client->princ to the proxy service asking for the TGS on behalf @@ -1547,9 +1618,20 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, ks_client_princ = client->princ; } + /* We only need to check the server entry here, because even if the client + * is a service with a valid authorization data it will result to NONE + * because ipadb_get_pac() can only generate a pac for 'real' IPA users. + * (I assume this will be the same for PAD.) */ + get_authz_data_types(context, server, &with_pac, &with_pad); + + if (with_pad) { + krb5_klog_syslog(LOG_ERR, "PAD authorization data is requested but " \ + "currently not supported."); + } + is_as_req = ((flags & KRB5_KDB_FLAG_CLIENT_REFERRALS_ONLY) != 0); - if (is_as_req && (flags & KRB5_KDB_FLAG_INCLUDE_PAC)) { + if (is_as_req && with_pac && (flags & KRB5_KDB_FLAG_INCLUDE_PAC)) { kerr = ipadb_get_pac(context, client, &pac); if (kerr != 0 && kerr != ENOENT) { @@ -1557,7 +1639,7 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, } } - if (!is_as_req) { + if (!is_as_req && with_pac) { /* find the existing PAC, if present */ kerr = krb5_find_authdata(context, tgt_auth_data, NULL, KRB5_AUTHDATA_WIN2K_PAC, &pac_auth_data); -- 1.8.1.2 -------------- next part -------------- From c9812e18c99fc1be7e5fce66f95318b49db6693e Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 19 Feb 2013 12:16:37 +0100 Subject: [PATCH 99/99] Add unit test for get_authz_data_types() --- daemons/ipa-kdb/Makefile.am | 29 +++++ daemons/ipa-kdb/tests/ipa_kdb_tests.c | 194 ++++++++++++++++++++++++++++++++++ 2 files changed, 223 insertions(+) create mode 100644 daemons/ipa-kdb/tests/ipa_kdb_tests.c diff --git a/daemons/ipa-kdb/Makefile.am b/daemons/ipa-kdb/Makefile.am index 5f4e6e2a6a940486a0c904f737f28c476df98773..23ba1cc05ec157a0f4d9b594350ebaf10b2098dc 100644 --- a/daemons/ipa-kdb/Makefile.am +++ b/daemons/ipa-kdb/Makefile.am @@ -52,6 +52,35 @@ ipadb_la_LIBADD = \ $(NDRPAC_LIBS) \ $(NULL) +if HAVE_CHECK +TESTS = ipa_kdb_tests +check_PROGRAMS = ipa_kdb_tests +endif + +ipa_kdb_tests_SOURCES = \ + tests/ipa_kdb_tests.c \ + ipa_kdb.c \ + ipa_kdb_common.c \ + ipa_kdb_mkey.c \ + ipa_kdb_passwords.c \ + ipa_kdb_principals.c \ + ipa_kdb_pwdpolicy.c \ + ipa_kdb_mspac.c \ + ipa_kdb_delegation.c \ + ipa_kdb_audit_as.c \ + $(KRB5_UTIL_SRCS) \ + $(NULL) +ipa_kdb_tests_CFLAGS = $(CHECK_CFLAGS) +ipa_kdb_tests_LDADD = \ + $(CHECK_LIBS) \ + $(KRB5_LIBS) \ + $(LDAP_LIBS) \ + $(NDRPAC_LIBS) \ + -lnss3 \ + -lkdb5 \ + -lsss_idmap \ + $(NULL) + dist_noinst_DATA = ipa_kdb.exports EXTRA_DIST = \ diff --git a/daemons/ipa-kdb/tests/ipa_kdb_tests.c b/daemons/ipa-kdb/tests/ipa_kdb_tests.c new file mode 100644 index 0000000000000000000000000000000000000000..144eddc67c9bad398618a85585bacac05d6322ea --- /dev/null +++ b/daemons/ipa-kdb/tests/ipa_kdb_tests.c @@ -0,0 +1,194 @@ +/** BEGIN COPYRIGHT BLOCK + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * Additional permission under GPLv3 section 7: + * + * In the following paragraph, "GPL" means the GNU General Public + * License, version 3 or any later version, and "Non-GPL Code" means + * code that is governed neither by the GPL nor a license + * compatible with the GPL. + * + * You may link the code of this Program with Non-GPL Code and convey + * linked combinations including the two, provided that such Non-GPL + * Code only links to the code of this Program through those well + * defined interfaces identified in the file named EXCEPTION found in + * the source code files (the "Approved Interfaces"). The files of + * Non-GPL Code may instantiate templates or use macros or inline + * functions from the Approved Interfaces without causing the resulting + * work to be covered by the GPL. Only the copyright holders of this + * Program may make changes or additions to the list of Approved + * Interfaces. + * + * Authors: + * Sumit Bose + * + * Copyright (C) 2013 Red Hat, Inc. + * All rights reserved. + * END COPYRIGHT BLOCK **/ + +#include +#include +#include +#include +#include +#include +#include + +#include "ipa-kdb/ipa_kdb.h" + +int krb5_klog_syslog(int l, const char *format, ...) +{ + va_list ap; + char *s = NULL; + int ret; + + va_start(ap, format); + + ret = vasprintf(&s, format, ap); + va_end(ap); + if (ret < 0) { + /* ENOMEM */ + return -1; + } + + fprintf(stderr, "%s\n", s); + free(s); + + return 0; +} + +extern void get_authz_data_types(krb5_context context, krb5_db_entry *entry, + bool *with_pac, bool *with_pad); + +START_TEST(test_get_authz_data_types) +{ + bool with_pac; + bool with_pad; + krb5_db_entry *entry; + struct ipadb_e_data *ied; + size_t c; + char *ad_none_only[] = {"NONE", NULL}; + char *ad_pad_only[] = {"PAD", NULL}; + char *ad_pac_only[] = {"MS-PAC", NULL}; + char *ad_illegal_only[] = {"abc", NULL}; + char *ad_pac_and_pad[] = {"MS-PAC", "PAD", NULL}; + char *ad_pac_and_none[] = {"MS-PAC", "NONE", NULL}; + char *ad_none_and_pad[] = {"NONE", "PAD", NULL}; + krb5_context krb5_ctx; + krb5_error_code kerr; + struct ipadb_context *ipa_ctx; + + struct test_set { + char **authz_data; + char **global_authz_data; + bool exp_with_pac; + bool exp_with_pad; + const char *err_msg; + } test_set[] = { + {ad_none_only, NULL, false, false, "with only NONE in entry"}, + {ad_pac_only, NULL, true, false, "with only MS-PAC in entry"}, + {ad_pad_only, NULL, false, true, "with only PAD in entry"}, + {ad_illegal_only, NULL, false, false, "with only an invalid value in entry"}, + {ad_pac_and_pad, NULL, true, true, "with MS-PAC and PAD in entry"}, + {ad_pac_and_none, NULL, false, false, "with MS-PAC and NONE in entry"}, + {ad_none_and_pad, NULL, false, false, "with NONE and PAD in entry"}, + {NULL, ad_none_only, false, false, "with only NONE in global config"}, + {NULL, ad_pac_only, true, false, "with only MS-PAC in global config"}, + {NULL, ad_pad_only, false, true, "with only PAD in global config"}, + {NULL, ad_illegal_only, false, false, "with only an invalid value in global config"}, + {NULL, ad_pac_and_pad, true, true, "with MS-PAC and PAD in global config"}, + {NULL, ad_pac_and_none, false, false, "with MS-PAC and NONE in global config"}, + {NULL, ad_none_and_pad, false, false, "with NONE and PAD in global entry"}, + {ad_none_only, ad_pac_only, false, false, "with NONE overriding PAC in global entry"}, + {ad_pad_only, ad_pac_only, false, true, "with PAC overriding PAC in global entry"}, + {ad_illegal_only, ad_pac_only, false, false, "with invalid value overriding PAC in global entry"}, + {ad_pac_and_pad, ad_pac_only, true, true, "with PAC and PAD overriding PAC in global entry"}, + {ad_none_and_pad, ad_pac_only, false, false, "with NONE and PAD overriding PAC in global entry"}, + {NULL, NULL, false, false, NULL} + }; + + get_authz_data_types(NULL, NULL, NULL, NULL); + + with_pad = true; + get_authz_data_types(NULL, NULL, NULL, &with_pad); + fail_unless(!with_pad, "with_pad not false with NULL inuput."); + + with_pac = true; + get_authz_data_types(NULL, NULL, &with_pac, NULL); + fail_unless(!with_pac, "with_pac not false with NULL inuput."); + + with_pad = true; + with_pac = true; + get_authz_data_types(NULL, NULL, &with_pac, &with_pad); + fail_unless(!with_pad, "with_pad not false with NULL inuput."); + fail_unless(!with_pac, "with_pac not false with NULL inuput."); + + entry = calloc(1, sizeof(krb5_db_entry)); + fail_unless(entry != NULL, "calloc krb5_db_entry failed."); + + ied = calloc(1, sizeof(struct ipadb_e_data)); + fail_unless(ied != NULL, "calloc struct ipadb_e_data failed."); + entry->e_data = (void *) ied; + + kerr = krb5_init_context(&krb5_ctx); + fail_unless(kerr == 0, "krb5_init_context failed."); + kerr = krb5_db_setup_lib_handle(krb5_ctx); + fail_unless(kerr == 0, "krb5_db_setup_lib_handle failed.\n"); + ipa_ctx = calloc(1, sizeof(struct ipadb_context)); + fail_unless(ipa_ctx != NULL, "calloc failed.\n"); + ipa_ctx->kcontext = krb5_ctx; + kerr = krb5_db_set_context(krb5_ctx, ipa_ctx); + fail_unless(kerr == 0, "krb5_db_set_context failed.\n"); + + for (c = 0; test_set[c].authz_data != NULL || + test_set[c].global_authz_data != NULL; c++) { + ied->authz_data = test_set[c].authz_data; + ipa_ctx->authz_data = test_set[c].global_authz_data; + get_authz_data_types(krb5_ctx, entry, &with_pac, &with_pad); + fail_unless(with_pad == test_set[c].exp_with_pad, "with_pad not %s %s.", + test_set[c].exp_with_pad ? "true" : "false", + test_set[c].err_msg); + fail_unless(with_pac == test_set[c].exp_with_pac, "with_pac not %s %s.", + test_set[c].exp_with_pac ? "true" : "false", + test_set[c].err_msg); + } + + krb5_db_fini(krb5_ctx); + krb5_free_context(krb5_ctx); +} +END_TEST + +Suite * ipa_kdb_suite(void) +{ + Suite *s = suite_create("IPA kdb"); + + TCase *tc_helper = tcase_create("Helper functions"); + tcase_add_test(tc_helper, test_get_authz_data_types); + suite_add_tcase(s, tc_helper); + + return s; +} + +int main(void) +{ + int number_failed; + + Suite *s = ipa_kdb_suite (); + SRunner *sr = srunner_create (s); + srunner_run_all (sr, CK_VERBOSE); + number_failed = srunner_ntests_failed (sr); + srunner_free (sr); + + return (number_failed == 0) ? EXIT_SUCCESS : EXIT_FAILURE; +} -- 1.8.1.2 From pspacek at redhat.com Thu Feb 21 15:27:03 2013 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 21 Feb 2013 16:27:03 +0100 Subject: [Freeipa-devel] [PATCH 0115] Add support for DNAME substitution In-Reply-To: <51263B74.9080406@redhat.com> References: <51263B74.9080406@redhat.com> Message-ID: <51263CC7.6080900@redhat.com> On 21.2.2013 16:21, Petr Spacek wrote: > Hello, > > Add support for DNAME substitution. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/63 > And now the patch :-) -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0115-Add-support-for-DNAME-substitution.patch Type: text/x-patch Size: 2768 bytes Desc: not available URL: From mkosek at redhat.com Thu Feb 21 15:58:02 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 21 Feb 2013 16:58:02 +0100 Subject: [Freeipa-devel] [PATCHES] 0104-0106 Provide means of displaying warning and informational messages on clients In-Reply-To: <51262A90.4070709@redhat.com> References: <50C9A709.8070806@redhat.com> <50C9F7A4.80909@redhat.com> <50CA0BDB.4000807@redhat.com> <50CADD74.1090802@redhat.com> <50E71D5F.4000304@redhat.com> <5107F377.5020007@redhat.com> <511B6874.5060106@redhat.com> <51235ED4.2000801@redhat.com> <5124F71D.701@redhat.com> <51260A09.3050001@redhat.com> <51261BE9.4050107@redhat.com> <51262A90.4070709@redhat.com> Message-ID: <5126440A.1010301@redhat.com> On 02/21/2013 03:09 PM, Petr Viktorin wrote: > On 02/21/2013 02:06 PM, Martin Kosek wrote: >> On 02/21/2013 12:50 PM, Petr Viktorin wrote: >>> On 02/20/2013 05:17 PM, Martin Kosek wrote: >>>> On 02/19/2013 12:15 PM, Petr Viktorin wrote: >>>>> On 02/13/2013 11:18 AM, Petr Viktorin wrote: >>>>>> On 01/29/2013 05:06 PM, Petr Viktorin wrote: >>>>>>> On 01/04/2013 07:20 PM, Petr Viktorin wrote: >>>>>>>> On 12/14/2012 09:04 AM, Jan Cholasta wrote: >>>>>>>>> On 13.12.2012 18:09, Petr Viktorin wrote: >>>>>>>>>> On 12/13/2012 04:43 PM, Martin Kosek wrote: >>>>>>>>>>> On 12/13/2012 10:59 AM, Petr Viktorin wrote: >>>>>>>>>>>> It's time to give this to another set of eyes :) >>>>>>>>>>>> >>>>>>>>>>>> Design document: http://freeipa.org/page/V3/Messages >>>>>>>>>>>> Ticket: https://fedorahosted.org/freeipa/ticket/2732 >>>>>>>>>>>> >>>>>>>>>>>> More info is in commit messages. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Because of https://fedorahosted.org/freeipa/ticket/3294, I needed to >>>>>>>>>>>> change the >>>>>>>>>>>> design document: when the client doesn't send the API version, it is >>>>>>>>>>>> assumed >>>>>>>>>>>> it's at a version before capabilities were introduced (i.e. 2.47). >>>>>>>>>>>> The client still gets a warning if the version is missing. Except >>>>>>>>>>>> for >>>>>>>>>>>> those >>>>>>>>>>>> commands where IPA didn't send a version -- ping, cert-show, etc. -- >>>>>>>>>>>> the >>>>>>>>>>>> warning wouldn't pass validation on old clients. (I'm assuming that >>>>>>>>>>>> our client >>>>>>>>>>>> is so far the only one that validates so strictly.) >>>>>>>>>>> >>>>>>>>>>> I did a basic test of this patch and also quickly read through the >>>>>>>>>>> patches and >>>>>>>>>>> besides nitpicks (like unused inspect module in >>>>>>>>>>> tests/test_ipalib/test_messages.py in patch 0105) I did not find any >>>>>>>>>>> obvious >>>>>>>>>>> errors in the Python code. >>>>>>>>>> >>>>>>>>>> Noted, will fix in future versions of the patch. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> However, this patch breaks WebUI badly, I did not even get to a >>>>>>>>>>> log in >>>>>>>>>>> screen. >>>>>>>>>>> Cooperation with Petr Vobornik will be needed. In my case, I got >>>>>>>>>>> blank >>>>>>>>>>> screen >>>>>>>>>>> and Javascript error: >>>>>>>>>>> >>>>>>>>>>> TypeError: IPA.messages.dialogs is undefined >>>>>>>>>>> https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ipa.js >>>>>>>>>>> Line 1460 >>>>>>>>>>> >>>>>>>>>>> I assume this is related to the Internal Error that was returned in >>>>>>>>>>> the JSON call >>>>>>>>>>> >>>>>>>>>>> { >>>>>>>>>>> "error": null, >>>>>>>>>>> "id": null, >>>>>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>>>>> "result": { >>>>>>>>>>> "count": 5, >>>>>>>>>>> "results": [ >>>>>>>>>>> { >>>>>>>>>>> "error": "an internal error has occurred", >>>>>>>>>>> "error_code": 903, >>>>>>>>>>> "error_name": "InternalError" >>>>>>>>>>> }, >>>>>>>>>>> { >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>>> This can be reproduced with: >>>>>>>>>>> >>>>>>>>>>> # curl -v -H "Content-Type:application/json" -H >>>>>>>>>>> "referer:https://`hostname`/ipa" -H "Accept:applicaton/json" >>>>>>>>>>> --negotiate -u : >>>>>>>>>>> --cacert /etc/ipa/ca.crt -d >>>>>>>>>>> '{"method":"i18n_messages","params":[[],{}],"id":0}' -X POST >>>>>>>>>>> https://`hostname`/ipa/json >>>>>>>>>> >>>>>>>>>> Good catch! The i18n_messages plugin already defines a "messages" >>>>>>>>>> output. When I renamed this from "warnings" to "messages" I forgot to >>>>>>>>>> check for clashes. >>>>>>>>>> Since i18n_messages is an internal command only used by the Web UI, we >>>>>>>>>> can rename its output to "texts" without breaking compatibility. >>>>>>>>>> >>>>>>>>>> I'm attaching a preliminary fix (for both backend and UI), but >>>>>>>>>> hopefully >>>>>>>>>> it won't be necessary, see below. >>>>>>>>>> >>>>>>>>>>> I am also not sure I like the requirement of a specific version >>>>>>>>>>> option >>>>>>>>>>> to be >>>>>>>>>>> always passed. I would prefer that missing version option would mean >>>>>>>>>>> "I use the >>>>>>>>>>> most recent version of API" instead - it would make the custom >>>>>>>>>>> JSONRPC/XMLRPC >>>>>>>>>>> calls easier to use. >>>>>>>>>>> >>>>>>>>>>> But since the version option was not being sent for some commands, we >>>>>>>>>>> may not >>>>>>>>>>> have a choice anyway if we do not want to break old clients in >>>>>>>>>>> case we >>>>>>>>>>> add some >>>>>>>>>>> capabilities to these commands. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I see three other options, all worse: >>>>>>>>>> - Do not use capabilities for the affected commands, meaning no new >>>>>>>>>> functionality can be added to them (and by extension, no new >>>>>>>>>> functionality common to all commands can be added). >>>>>>>>>> - Treat a missing version as the current version >>>>>>>>>> - Break backwards compatibility >>>>>>>>>> >>>>>>>>>> And one possibly better (thanks to Petr? and Martin for opening my >>>>>>>>>> eyes >>>>>>>>>> off-list!): >>>>>>>>>> - Deprecate XML-RPC. All XML-RPC requests would be pinned to current >>>>>>>>>> version (2.47). Capabilities/messages would only apply to JSON-RPC. >>>>>>>>>> >>>>>>>>>> This would also allow us to solve the above name-clashing problem >>>>>>>>>> elegantly. Here is a reminder of what a JSON response looks like: >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> "error": null, >>>>>>>>>> "id": 0, >>>>>>>>>> "principal": "admin at IDM.LAB.BOS.REDHAT.COM", >>>>>>>>>> "result": { >>>>>>>>>> "summary": "IPA server version 3.1.0GIT2e4bd02. API version >>>>>>>>>> 2.47" >>>>>>>>>> }, >>>>>>>>>> "version": "3.1.0GIT2e4bd02" >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> A XML-RPC response only contains the "result" part of that. >>>>>>>>>> So with JSON, we can put the messages in the top level, which is much >>>>>>>>>> better design. >>>>>>>> >>>>>>>> Custom info in the "top level" seems to be a violation of the JSON-RPC >>>>>>>> spec. I'd rather not do more of those, so I'm withdrawing this idea. >>>>>>>> >>>>>>>>>> >>>>>>>>>> XML-RPC sucks in other ways too. We already have a workaround for its >>>>>>>>>> inability to attach extra info to errors (commit >>>>>>>>>> 88262a75ffe7a25640333dcc4da2100830cae821, Add instructions support to >>>>>>>>>> PublicError). >>>>>>>>>> >>>>>>>>>> I've opened a RFC here: https://fedorahosted.org/freeipa/ticket/3299. >>>>>>>>>> >>>>>>>>> >>>>>>>>> +1, XML-RPC sucks. This should have been done a long time ago. >>>>>>>>> >>>>>>>>> Honza >>>>>>>>> >>>>>>>> >>>>>>>> Here are new patches. >>>>>>>> >>>>>>>> XML-RPC requests with missing version are assumed to be old (the version >>>>>>>> before capabilities are introduced, 2.47). This takes care of backcompat >>>>>>>> with clients with bug 3294. >>>>>>>> JSON-RPC requests with missing version are assumed to be testing calls >>>>>>>> (e.g. curl), they get behavior of the latest version but also a warning. >>>>>>>> I've also added this info to the design doc. >>>>>>>> >>>>>>>> It turns out that these patches don't depend on whether our client uses >>>>>>>> XML-RPC or JSON-RPC. If/when it supports both, I'll be able to add some >>>>>>>> extra unit tests. >>>>>>>> >>>>>>> >>>>>>> Patch 106 had a minor conflict with master, attaching fixed version. >>>>>>> >>>>>> >>>>>> Patches 106 & 115 need an API version bump. >>>>>> I also noticed that `makeapi --validate` wasn't checking capabilities >>>>>> properly. Fixed. >>>>>> >>>>>> >>>>> >>>>> Rebasing patch 104 to current master. >>>>> >>>> >>>> Patch 104 and 106 needs a rebase. >>>> >>>> Generally I think this patchset is OK and we will need it as a foundation for >>>> other features. >>>> >>>> I may have done my custom rebasing wrong, but my WebUI stopped working with >>>> these patches. I see this in error_log: >>>> >>>> [Wed Feb 20 10:38:22.351845 2013] [auth_kerb:error] [pid 22172] [client >>>> 10.34.4.72:40777] gss_display_name() failed: A required input >>>> parameter could not be read: An invalid name was supplied (, Unknown error), >>>> referer: https://vm-037.idm.lab.bos.redhat.com/ipa/ui/ >>>> >>>> >>>> I also saw one failed test case: >>>> ====================================================================== >>>> FAIL: Try user_show with no version >>>> ---------------------------------------------------------------------- >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >>>> self.test(*self.arg) >>>> File "/root/freeipa-master/tests/test_xmlrpc/test_user_plugin.py", line >>>> 1760, >>>> in test_user_show_without_version >>>> assert res['messages'] == (expected_message.to_dict(), ) >>>> AssertionError >>>> >>>> Martin >>>> >>> >>> Here are the rebased patches. >>> Please test from a clean tree, the UI stopped working here when I had some >>> stale files around. >>> >> >> Thanks, everything looks good except our first message (VersionMissing) which >> is sent when I run commands via JSON-RPC which do not have any version >> attached. Now, I get this error: >> >> "messages": [ >> { >> "code": 13001, >> "message": "API Version number was not sent. Assuming client >> version 2.47. Current API version is 2.53", >> "name": "VersionMissing", >> "type": "warning" >> } >> ], >> >> There are 2 issues with this message wording: >> 1) 2.47 is deprecated, I assume you wanted VERSION_WITHOUT_CAPABILITIES >> 2) We assume API_VERSION (current API version) and not >> VERSION_WITHOUT_CAPABILITIES >> >> Martin >> > > Fixed, thanks for the catch. > Thanks. Pushed all 4 patches to master, ipa-3-1. Martin From akrivoka at redhat.com Thu Feb 21 16:46:23 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Thu, 21 Feb 2013 17:46:23 +0100 Subject: [Freeipa-devel] [PATCH] 0002 Add missing error message when adding duplicate external member to group In-Reply-To: <5123E4B5.2060702@redhat.com> References: <50EEE184.6080804@redhat.com> <5123E4B5.2060702@redhat.com> Message-ID: <51264F5F.10209@redhat.com> On 02/19/2013 09:46 PM, Rob Crittenden wrote: > Ana Krivokapic wrote: >> When adding a duplicate member to a group, an error message is issued, >> informing the user that the entry is already a member of the group. This >> message was missing in case of an external member. >> >> Ticket: https://fedorahosted.org/freeipa/ticket/3254 > > This works ok but the sister command, group-remove-member, has the > same problem. Can you add a fix there as well? > > I don't know if there is a way to add a unit test for this since the > external member is validated meaning we'd need to set up trusts as > well. It might be nice to have an optional test that can be run when a > trust is configured to avoid regressions. > > rob > I fixed the group-remove-member command and added unit tests which can be run when the trust is established (they will be skipped when the trust is not established). I also noticed that, in contrast to group-add-member, group-remove-member did not allow the format 'AD\name' or 'name at ad.domain.com' for the --external option. I included this fix in the patch, so the two user friendly formats are now supported. Updated patch is attached. -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akrivoka-0002-02-Improve-error-messages-for-external-group-members.patch Type: text/x-patch Size: 11755 bytes Desc: not available URL: From pviktori at redhat.com Thu Feb 21 17:08:57 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 21 Feb 2013 18:08:57 +0100 Subject: [Freeipa-devel] [PATCHES] 0183-0185 Drop CSV support Message-ID: <512654A9.1060309@redhat.com> These patches remove CSV parsing from the client. Ticket: https://fedorahosted.org/freeipa/ticket/3352 Design: http://freeipa.org/page/V3/Drop_CSV The design page also talks about adding a warning for the user when they seem to use CSV, but this will need the JSON transport so it's not included in these patches. The "csv" flag is left on parameters so when that warning is added, we know for which params to show it. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0183-Remove-csv_separator-and-csv_skipspace-Param-argumen.patch Type: text/x-patch Size: 5138 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0184-Drop-support-for-CSV-in-the-CLI-client.patch Type: text/x-patch Size: 7864 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0185-Update-argument-docs-to-reflect-dropped-CSV-support.patch Type: text/x-patch Size: 12024 bytes Desc: not available URL: From dpal at redhat.com Thu Feb 21 17:25:19 2013 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 21 Feb 2013 12:25:19 -0500 Subject: [Freeipa-devel] [SSSD] F19 test day In-Reply-To: <20130221155031.GG17964@hendrix.brq.redhat.com> References: <20130221152323.GC17964@hendrix.brq.redhat.com> <51263EF1.9010508@redhat.com> <20130221155031.GG17964@hendrix.brq.redhat.com> Message-ID: <5126587F.4000505@redhat.com> On 02/21/2013 10:50 AM, Jakub Hrozek wrote: > On Thu, Feb 21, 2013 at 10:36:17AM -0500, Dmitri Pal wrote: >> On 02/21/2013 10:23 AM, Jakub Hrozek wrote: >>> Hi, >>> >>> Here is the schedule of the F19 test days: >>> https://fedoraproject.org/wiki/QA/Fedora_19_test_days >>> >>> I wanted to book one after mid-April but seems like the desktop team has >>> booked the whole April.. >>> >>> Then I'm going to propose we take 2013-04-11. I think we should rather >>> pick one before the Beta and if some features are not 100% there, then >>> test them out-of-band. The next vacant one is 2013-05-23 which I think >>> is too late. >>> >>> Comments? >> 4/11 seems reasonable. >> I think we can also overlap with desktop if we find it suitable for our >> needs. >> We plan 4/25 for IPA. > OK: > https://fedorahosted.org/fedora-qa/ticket/352 https://fedorahosted.org/fedora-qa/ticket/353 > _______________________________________________ > sssd-devel mailing list > sssd-devel at lists.fedorahosted.org > https://lists.fedorahosted.org/mailman/listinfo/sssd-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From mkosek at redhat.com Fri Feb 22 09:14:50 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 10:14:50 +0100 Subject: [Freeipa-devel] [PATCH 0035] Use default.conf as flag of IPA client being installed In-Reply-To: <512627FB.3020709@redhat.com> References: <512497ED.6080200@redhat.com> <5126094E.1040503@redhat.com> <5126133D.1080001@redhat.com> <51261820.10809@redhat.com> <512627FB.3020709@redhat.com> Message-ID: <5127370A.3040806@redhat.com> On 02/21/2013 02:58 PM, Tomas Babej wrote: > On 02/21/2013 01:50 PM, Martin Kosek wrote: >> On 02/21/2013 01:29 PM, Tomas Babej wrote: >>> On 02/21/2013 12:47 PM, Martin Kosek wrote: >>>> On 02/20/2013 10:31 AM, Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> When installing / uninstalling IPA client, the checks that >>>>> determine whether IPA client is installed now take the existence >>>>> of /etc/ipa/default.conf into consideration. >>>>> >>>>> The client will not uninstall unless either something is backed >>>>> up or /etc/ipa/default.conf file does exist. >>>>> >>>>> The client will not install if something is backed up or >>>>> default.conf file does exist (unless it's installation on master). >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/3331 >>>>> >>>>> Tomas >>>>> >>>>> >>>> Can we create a function testing if ipa client is installed to avoid >>>> duplication of the decision logic? Something like is_ipa_configured present in >>>> ipaserver/install/installutils.py. >>>> >>>> Keep in mind that we cannot use ipaserver package as it may not be present on >>>> client. >>>> >>>> Martin >>> Moved to is_ipa_client_installed function to ipautils.py >>> >>> Updated patch attached. >>> >>> Tomas >>> >> You just created a nice import loop: >> >> ... > I made the function part of ipa-client-install script then. > We probably will not need to check whether client is installed anywhere else. > > Tomas Thanks, this fixes the issue. ACK. Pushed to master, ipa-3-1. Martin From pspacek at redhat.com Fri Feb 22 09:19:12 2013 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 22 Feb 2013 10:19:12 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <51249F6E.7030005@redhat.com> References: <511E44A9.9090509@redhat.com> <51221027.7050803@redhat.com> <20130218114750.GC27418@localhost.localdomain> <512219D7.9000909@redhat.com> <51249F6E.7030005@redhat.com> Message-ID: <51273810.9060800@redhat.com> On 20.2.2013 11:03, Ana Krivokapic wrote: > On 02/18/2013 01:08 PM, Martin Kosek wrote: >> On 02/18/2013 12:47 PM, Sumit Bose wrote: >>> On Mon, Feb 18, 2013 at 12:27:35PM +0100, Petr Spacek wrote: >>>> On 15.2.2013 15:22, Ana Krivokapic wrote: >>>>> Hello, >>>>> >>>>> The .isalpha() check in validate_domain_name() was too strict, >>>>> causing some commands like ipa dnsrecord-add to fail. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/3385 >>>> I would add --force option rather than removing whole check, if it's possible. >>>> >>>> Would it be possible to mention RFC in the error message? Something >>>> like _('top level domain label must be alphabetic (RFC 1123 section >>>> 2.1)') >>>> ? >>>> >>>> IMHO it is handy, because it educates users. >>> The problem is that this check is always done on the last component of >>> the domain_name even if it is just a sub-domain of the FreeIPA domain, >>> where e.g. numbers are valid characters. >>> >>> At the beginning of validate_domain_name() a trailing '.' is stripped >>> away. iirc the trailing '.' is an indication for a complete, fully >>> qualified name. Would it work if the presence of the trailing '.' is >>> saved and the check is only done if there was a '.'? >>> >>> bye, >>> Sumit >>> >> Sure. Though I am now not 100% sure that some IPA functions do not use this >> validator with a fqdn hostname without trailing dot. If not, I am for fixing >> this function as Sumit and Petr suggested. >> >> Martin >> >> _______________________________________________ >> Freeipa-devel mailing list >> Freeipa-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > After some thought, I decided to change the approach. > > As pointed out by Sumit, the problem was that the validate_domain_name() > function did not distinguish between fqdn and non-fqdn domains > (subdomains of the IPA domain). The trailing dot is not a clear > indication either, because some IPA functions use this validator with an > fqdn without the trailing dot. > > To fix this, I introduced an additional parameter to this function - a > flag which indicates whether the domain name is an fqdn or not. The is > .isalpha() check is then performed only in the case of an fqdn. > > I also improved the error message to mention the relevant RFC, as > suggested by Petr. Please don't forget to add --force switch. It could be handy. -- Petr^2 Spacek From pviktori at redhat.com Fri Feb 22 10:16:09 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 22 Feb 2013 11:16:09 +0100 Subject: [Freeipa-devel] [PATCH] 0186 Change DNA magic value to -1 to make UID 999 usable Message-ID: <51274569.3000700@redhat.com> https://fedorahosted.org/freeipa/ticket/2886 This changes the DNA magic value to -1, and the corresponding IPA's parameters (gidnumber, uidnumber) to be optional (instead of autofill). Since the old clients still say "999" when they mean "pick one I don't care", we need to detect them and change 999 to -1. For that there's a new capability, optional_uid_params. Behavior summary: With --uid 999: old client, old server: sends 999, creates random UID old client, new server: sends 999, creates random UID new client, old server: incompatible new client, new server: sends 999, creates UID 999 Without --uid: old client, old server: sends 999, creates random UID old client, new server: sends 999, creates random UID new client, old server: incompatible new client, new server: doesn't send UID, creates random UID Upgrade should work fine. I didn't test winsync as I don't have a Windows machine. -- Petr? From pviktori at redhat.com Fri Feb 22 11:16:06 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 22 Feb 2013 12:16:06 +0100 Subject: [Freeipa-devel] [PATCH] 0186 Change DNA magic value to -1 to make UID 999 usable In-Reply-To: <51274569.3000700@redhat.com> References: <51274569.3000700@redhat.com> Message-ID: <51275376.8020509@redhat.com> On 02/22/2013 11:16 AM, Petr Viktorin wrote: > https://fedorahosted.org/freeipa/ticket/2886 > > This changes the DNA magic value to -1, and the corresponding IPA's > parameters (gidnumber, uidnumber) to be optional (instead of autofill). > > Since the old clients still say "999" when they mean "pick one I don't > care", we need to detect them and change 999 to -1. For that there's a > new capability, optional_uid_params. > > > Behavior summary: > > With --uid 999: > old client, old server: sends 999, creates random UID > old client, new server: sends 999, creates random UID > new client, old server: incompatible > new client, new server: sends 999, creates UID 999 > > Without --uid: > old client, old server: sends 999, creates random UID > old client, new server: sends 999, creates random UID > new client, old server: incompatible > new client, new server: doesn't send UID, creates random UID > > Upgrade should work fine. > > I didn't test winsync as I don't have a Windows machine. > The patch is here -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0186-Change-DNA-magic-value-to-1-to-make-UID-999-usable.patch Type: text/x-patch Size: 21619 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 22 12:36:58 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 13:36:58 +0100 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades In-Reply-To: <5122965B.3030909@redhat.com> References: <5122965B.3030909@redhat.com> Message-ID: <5127666A.3090406@redhat.com> On 02/18/2013 10:00 PM, Rob Crittenden wrote: > An objectclass and attribute are not being added on upgrades. Missing these > causes the UI to not work. > > I also noticed a typo in the ordering of a number of the trust attributes so > fix those as well. > > rob ACK, works for me. Pushed to master, ipa-3-1. Martin From pviktori at redhat.com Fri Feb 22 12:43:12 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 22 Feb 2013 13:43:12 +0100 Subject: [Freeipa-devel] [PATCH] 0187 CLI: Do interactive prompting after a context is created Message-ID: <512767E0.7020700@redhat.com> Hello, This fixes a regression introduced by one of my help patches (abe26d5). https://fedorahosted.org/freeipa/ticket/3453 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0187-cli-Do-interactive-prompting-after-a-context-is-crea.patch Type: text/x-patch Size: 2463 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 22 13:01:22 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 14:01:22 +0100 Subject: [Freeipa-devel] [PATCHES] 0183-0185 Drop CSV support In-Reply-To: <512654A9.1060309@redhat.com> References: <512654A9.1060309@redhat.com> Message-ID: <51276C22.5010702@redhat.com> On 02/21/2013 06:08 PM, Petr Viktorin wrote: > These patches remove CSV parsing from the client. > > Ticket: https://fedorahosted.org/freeipa/ticket/3352 > Design: http://freeipa.org/page/V3/Drop_CSV > > > The design page also talks about adding a warning for the user when they seem > to use CSV, but this will need the JSON transport so it's not included in these > patches. > > The "csv" flag is left on parameters so when that warning is added, we know for > which params to show it. > Generally, this works fine. This finally allows us to easily add records with quote or comma: # ipa dnsrecord-add example.com txt1 --txt-rec="foo, bar" Record name: txt1 TXT record: foo, bar # ipa dnsrecord-add example.com txt1 --txt-rec='"bar,bar"' Record name: txt1 TXT record: foo, bar, "bar,bar" # ipa dnsrecord-show example.com txt1 --all --raw dn: idnsname=txt1,idnsname=example.com,cn=dns,dc=example,dc=com idnsname: txt1 txtrecord: foo, bar txtrecord: "bar,bar" objectclass: top objectclass: idnsrecord Just couple minor remarks: 1) Can we then update the csv flag description? It is now a bit misleading given its new function - csv: this multivalue attribute is given in CSV format 2) We need to also update our help, for example selfservice online help presents a CSV formatted option use: ipa selfservice-add --permissions=write --attrs=street,postalCode,l,c,st "Users manage their own address" Btw if we suggest the curly braces shortcut, we may also want to write that it applies for BASH shell only... Few other examples I saw: ipa group-add-member --users=test1,test2 localadmins Martin From mkosek at redhat.com Fri Feb 22 12:49:33 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 13:49:33 +0100 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades In-Reply-To: <1361301797.12328.382.camel@willson.li.ssimo.org> References: <5122965B.3030909@redhat.com> <5123A7A7.1020100@redhat.com> <5123C53B.7030002@redhat.com> <1361301797.12328.382.camel@willson.li.ssimo.org> Message-ID: <5127695D.3050001@redhat.com> On 02/19/2013 08:23 PM, Simo Sorce wrote: > On Tue, 2013-02-19 at 13:32 -0500, Rob Crittenden wrote: >> Jan Cholasta wrote: >>> Hi, >>> >>> On 18.2.2013 22:00, Rob Crittenden wrote: >>>> An objectclass and attribute are not being added on upgrades. Missing >>>> these causes the UI to not work. >>>> >>>> I also noticed a typo in the ordering of a number of the trust >>>> attributes so fix those as well. >>>> >>>> rob >>>> >>> >>> The patch looks good, but I think errors like this will pop up from time >>> to time, because we have to maintain the same thing in two places - the >>> installation LDIFs and update files. Maybe we should start thinking >>> about merging these two somehow, e.g. using the LDIFs for both >>> installation and updates, with directives for the updater in specially >>> formatted comments. >>> >>> Honza >>> >> >> This idea came up long, long ago when we first added the updater very >> early in v2. The problem, as I recall, is that some schema is needed >> during the install so we need to ship it in ldif format, and the idea of >> splitting it didn't appeal to us. >> >> So perhaps what we should endeavor to do is add all new schema via >> updates and only update the schema files themselves if the schema is >> needed for a fresh install (since updates are done last). >> >> This also puts more schema into 99user.ldif which may or may not be >> desirable. > > Ron another option is to keep putting all updates only in schema files, > and then have the updater "validate" the schema files. > > Validation would be: > 1. Download schema from server (we already do this in the framework so > it comes for free) > 2. parse the schema files and check if each attribute and objectclass is > present and in the correct form. > 3. if any attribute is missing, we add it > 4. if any attribute has been changed, we change it > 5. same for object classes. > > This would allow us to keep everything just in schema files, and for now > only updates would end up in 99.ldif > > I know there is also work in 389ds to improve schema validation and > handling, so there is a chance in future we will have online interfaces > to put data in multiple files w/o lumping everything in 99.ldif > > So by keeping stuff in schema files rather than arbitrary update files > we are also sort of future proof. > > Finally keeping data in schema files instead of spreading it in updates > should make it easier to keep an eye on the whole schema. > > The main issue I see is that this approach needs new code to analyze and > compare schema files, however that shouldn't be overly hard. > > Simo. > I think this is a great idea. Having schema updates on 2 or more separate spaces is error prone. attributeTypes or objectClasses update files may be confusing as we often have 2 and more replace: directives when we update objectClasses or attributeTypes more that one time. As for the LDIF file parsing, we could also use python-ldap's convenience classes which will make the comparing easier. I created a ticket to address this effort: https://fedorahosted.org/freeipa/ticket/3454 Martin From mkosek at redhat.com Fri Feb 22 13:36:38 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 14:36:38 +0100 Subject: [Freeipa-devel] [PATCHES] 0183-0185 Drop CSV support In-Reply-To: <51276C22.5010702@redhat.com> References: <512654A9.1060309@redhat.com> <51276C22.5010702@redhat.com> Message-ID: <51277466.6070002@redhat.com> On 02/22/2013 02:01 PM, Martin Kosek wrote: > On 02/21/2013 06:08 PM, Petr Viktorin wrote: >> These patches remove CSV parsing from the client. >> >> Ticket: https://fedorahosted.org/freeipa/ticket/3352 >> Design: http://freeipa.org/page/V3/Drop_CSV >> >> >> The design page also talks about adding a warning for the user when they seem >> to use CSV, but this will need the JSON transport so it's not included in these >> patches. >> >> The "csv" flag is left on parameters so when that warning is added, we know for >> which params to show it. >> > > Generally, this works fine. This finally allows us to easily add records with > quote or comma: > > # ipa dnsrecord-add example.com txt1 --txt-rec="foo, bar" > Record name: txt1 > TXT record: foo, bar > # ipa dnsrecord-add example.com txt1 --txt-rec='"bar,bar"' > Record name: txt1 > TXT record: foo, bar, "bar,bar" > # ipa dnsrecord-show example.com txt1 --all --raw > dn: idnsname=txt1,idnsname=example.com,cn=dns,dc=example,dc=com > idnsname: txt1 > txtrecord: foo, bar > txtrecord: "bar,bar" > objectclass: top > objectclass: idnsrecord > > > > Just couple minor remarks: > > 1) Can we then update the csv flag description? It is now a bit misleading > given its new function > > - csv: this multivalue attribute is given in CSV format > > 2) We need to also update our help, for example selfservice online help > presents a CSV formatted option use: > > ipa selfservice-add --permissions=write --attrs=street,postalCode,l,c,st > "Users manage their own address" > > Btw if we suggest the curly braces shortcut, we may also want to write that it > applies for BASH shell only... > > Few other examples I saw: > ipa group-add-member --users=test1,test2 localadmins > > > Martin > Just discovered another issue... We now crash when adding permissions/acis with csv: # ipa permission-add --attrs=member --permissions=read,write --type=group foo2 ipa: ERROR: an internal error has occurred We should report some meaningful error, otherwise users used to add permissions with csv values would be confused. Martin From tbabej at redhat.com Fri Feb 22 14:01:37 2013 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 22 Feb 2013 15:01:37 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <51261FB3.8030005@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> <20130220132403.GN476@redhat.com> <5124DB7F.3020101@redhat.com> <51261FB3.8030005@redhat.com> Message-ID: <51277A41.4060902@redhat.com> On 02/21/2013 02:22 PM, Martin Kosek wrote: > On 02/20/2013 03:19 PM, Tomas Babej wrote: >> On Wed 20 Feb 2013 02:24:03 PM CET, Alexander Bokovoy wrote: >>> On Wed, 20 Feb 2013, Tomas Babej wrote: >>>> On 12/21/2012 12:15 PM, Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> Sending updated and rebased versions of patches 0024 and 0025. >>>>> >>>>> Tomas >>>>> >>>>> >>>> Sending rebased version, these got quite rotten. >>> Thanks for updating them. >>> >>>> @@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): >>>> 'not be found. Please specify the SID >>>> directly ' >>>> 'using dom-sid option.')) >>>> >>>> - try: >>>> - (old_dn, old_attrs) = ldap.get_entry(dn, >>>> - ['ipabaseid', >>>> - 'ipaidrangesize', >>>> - 'ipabaserid', >>>> - 'ipasecondarybaserid']) >>>> - except errors.NotFound: >>>> - self.obj.handle_not_found(*keys) >>>> + if in_updated_attrs('ipanttrusteddomainsid'): >>>> + if in_updated_attrs('ipasecondarybaserid'): >>>> + raise errors.ValidationError(name='ID Range setup', >>>> + error=_('Options dom_sid and secondary_rid_base >>>> cannot ' >>>> + 'be used together')) >>> Since we agreed to refer to options by their CLI name (--dom-sid and >>> --secondary-rid-base) in the other patch, it makes sense to use it >>> here too. >>> >>> >>>> - if is_set('ipanttrusteddomainsid'): >>>> - # Validate SID as the one of trusted domains >>>> - >>>> self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >>>> >>>> + if not in_updated_attrs('ipabaserid'): >>>> + raise errors.ValidationError(name='ID Range setup', >>>> + error=_('Options dom_sid and rid_base must ' >>>> + 'be used together')) >>> Same here. >>> >>>> + # secondary base rid must be set if and only if base rid >>>> is set >>>> + if in_updated_attrs('ipasecondarybaserid') !=\ >>>> + in_updated_attrs('ipabaserid'): >>>> + raise errors.ValidationError(name='ID Range setup', >>>> + error=_('Options secondary_rid_base and rid_base >>>> must ' >>>> + 'be used together')) >>> Same here. >>> >>>> + dict( >>>> + desc='Try to modify ID range %r so it has only primary >>>> rid range set' % (testrange8), >>>> + command=('idrange_mod', [testrange8], >>>> + dict(ipabaserid=testrange8_base_rid)), >>>> + expected=errors.ValidationError( >>>> + name='ID Range setup', error='Options >>>> secondary_rid_base and rid_base must be used together'), >>>> + ), >>> And synchronize error message here too. >>> >> Thanks! >> >> Sending the updated patch 0024. >> >> Tomas >> > In patch 0024 your intention is OK, but the checking functions are not: > > is_set = lambda x: (x in entry_attrs) and (x is not None) > + in_updated_attrs = lambda x: any((x in attrs and x is not None) > + for attrs in (entry_attrs, old_attrs)) > > They return True even when the attribute is None because they check if *x* is > None and not if *attrs[x]* is None. Example: > > # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 --secondary-rid-base=1000000 local_range > ---------------------------- > Added ID range "local_range" > ---------------------------- > Range name: local_range > First Posix ID of the range: 1200000 > Number of IDs in the range: 200000 > First RID of the corresponding RID range: 1000 > First RID of the secondary RID range: 1000000 > Range type: local domain range > > This command should be NOOP, but is not: > > # ipa idrange-mod local_range --dom-sid= > ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base > cannot be used together > > Martin Thanks for the catch, all checking functions fixed. Sending the complete patchset, up to date. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0025-6-Add-trusted-domain-range-objectclass-when-using-idra.patch Type: text/x-patch Size: 1325 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0024-6-Make-options-checks-in-idrange-add-mod-consistent.patch Type: text/x-patch Size: 9505 bytes Desc: not available URL: From pviktori at redhat.com Fri Feb 22 14:05:04 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 22 Feb 2013 15:05:04 +0100 Subject: [Freeipa-devel] [PATCHES] 0183-0185 Drop CSV support In-Reply-To: <51277466.6070002@redhat.com> References: <512654A9.1060309@redhat.com> <51276C22.5010702@redhat.com> <51277466.6070002@redhat.com> Message-ID: <51277B10.8000205@redhat.com> On 02/22/2013 02:36 PM, Martin Kosek wrote: > On 02/22/2013 02:01 PM, Martin Kosek wrote: >> On 02/21/2013 06:08 PM, Petr Viktorin wrote: >>> These patches remove CSV parsing from the client. >>> >>> Ticket: https://fedorahosted.org/freeipa/ticket/3352 >>> Design: http://freeipa.org/page/V3/Drop_CSV >>> >>> >>> The design page also talks about adding a warning for the user when they seem >>> to use CSV, but this will need the JSON transport so it's not included in these >>> patches. >>> >>> The "csv" flag is left on parameters so when that warning is added, we know for >>> which params to show it. >>> >> >> Generally, this works fine. This finally allows us to easily add records with >> quote or comma: >> >> # ipa dnsrecord-add example.com txt1 --txt-rec="foo, bar" >> Record name: txt1 >> TXT record: foo, bar >> # ipa dnsrecord-add example.com txt1 --txt-rec='"bar,bar"' >> Record name: txt1 >> TXT record: foo, bar, "bar,bar" >> # ipa dnsrecord-show example.com txt1 --all --raw >> dn: idnsname=txt1,idnsname=example.com,cn=dns,dc=example,dc=com >> idnsname: txt1 >> txtrecord: foo, bar >> txtrecord: "bar,bar" >> objectclass: top >> objectclass: idnsrecord >> >> >> >> Just couple minor remarks: >> >> 1) Can we then update the csv flag description? It is now a bit misleading >> given its new function >> >> - csv: this multivalue attribute is given in CSV format I have reworded that in patch 184: - - csv: this multivalue attribute is given in CSV format + - csv: this multivalue attribute used to be given in CSV format in CLI >> 2) We need to also update our help, for example selfservice online help >> presents a CSV formatted option use: >> >> ipa selfservice-add --permissions=write --attrs=street,postalCode,l,c,st >> "Users manage their own address" >> >> Btw if we suggest the curly braces shortcut, we may also want to write that it >> applies for BASH shell only... >> >> Few other examples I saw: >> ipa group-add-member --users=test1,test2 localadmins >> Attaching patch for that. I used the Bash shortcut when there were more than two items. I grepped through the sources to find examples that need changing, hopefully I've caught them all. > Just discovered another issue... We now crash when adding permissions/acis with > csv: > > # ipa permission-add --attrs=member --permissions=read,write --type=group foo2 > ipa: ERROR: an internal error has occurred > > We should report some meaningful error, otherwise users used to add permissions > with csv values would be confused. See ticket #3420, patch 0182. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0188-Update-plugin-docstrings-topic-help-to-reflect-dropp.patch Type: text/x-patch Size: 9879 bytes Desc: not available URL: From akrivoka at redhat.com Fri Feb 22 15:02:55 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Fri, 22 Feb 2013 16:02:55 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <51273810.9060800@redhat.com> References: <511E44A9.9090509@redhat.com> <51221027.7050803@redhat.com> <20130218114750.GC27418@localhost.localdomain> <512219D7.9000909@redhat.com> <51249F6E.7030005@redhat.com> <51273810.9060800@redhat.com> Message-ID: <5127889F.9020202@redhat.com> On 02/22/2013 10:19 AM, Petr Spacek wrote: > On 20.2.2013 11:03, Ana Krivokapic wrote: >> On 02/18/2013 01:08 PM, Martin Kosek wrote: >>> On 02/18/2013 12:47 PM, Sumit Bose wrote: >>>> On Mon, Feb 18, 2013 at 12:27:35PM +0100, Petr Spacek wrote: >>>>> On 15.2.2013 15:22, Ana Krivokapic wrote: >>>>>> Hello, >>>>>> >>>>>> The .isalpha() check in validate_domain_name() was too strict, >>>>>> causing some commands like ipa dnsrecord-add to fail. >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/3385 >>>>> I would add --force option rather than removing whole check, if >>>>> it's possible. >>>>> >>>>> Would it be possible to mention RFC in the error message? Something >>>>> like _('top level domain label must be alphabetic (RFC 1123 section >>>>> 2.1)') >>>>> ? >>>>> >>>>> IMHO it is handy, because it educates users. >>>> The problem is that this check is always done on the last component of >>>> the domain_name even if it is just a sub-domain of the FreeIPA domain, >>>> where e.g. numbers are valid characters. >>>> >>>> At the beginning of validate_domain_name() a trailing '.' is stripped >>>> away. iirc the trailing '.' is an indication for a complete, fully >>>> qualified name. Would it work if the presence of the trailing '.' is >>>> saved and the check is only done if there was a '.'? >>>> >>>> bye, >>>> Sumit >>>> >>> Sure. Though I am now not 100% sure that some IPA functions do not >>> use this >>> validator with a fqdn hostname without trailing dot. If not, I am >>> for fixing >>> this function as Sumit and Petr suggested. >>> >>> Martin >>> >>> _______________________________________________ >>> Freeipa-devel mailing list >>> Freeipa-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/freeipa-devel >> After some thought, I decided to change the approach. >> >> As pointed out by Sumit, the problem was that the validate_domain_name() >> function did not distinguish between fqdn and non-fqdn domains >> (subdomains of the IPA domain). The trailing dot is not a clear >> indication either, because some IPA functions use this validator with an >> fqdn without the trailing dot. >> >> To fix this, I introduced an additional parameter to this function - a >> flag which indicates whether the domain name is an fqdn or not. The is >> .isalpha() check is then performed only in the case of an fqdn. >> >> I also improved the error message to mention the relevant RFC, as >> suggested by Petr. > > Please don't forget to add --force switch. It could be handy. > I added the --force switch to ipa dnsrecord-add and opened a new ticket to handle the rest of the ipa commands that use domain name validation: https://fedorahosted.org/freeipa/ticket/3455 Updated patch is attached. -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akrivoka-0006-03-Improve-domain-name-validation.patch Type: text/x-patch Size: 2994 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 22 15:34:55 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 16:34:55 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <51277A41.4060902@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> <20130220132403.GN476@redhat.com> <5124DB7F.3020101@redhat.com> <51261FB3.8030005@redhat.com> <51277A41.4060902@redhat.com> Message-ID: <5127901F.4040408@redhat.com> On 02/22/2013 03:01 PM, Tomas Babej wrote: > On 02/21/2013 02:22 PM, Martin Kosek wrote: >> On 02/20/2013 03:19 PM, Tomas Babej wrote: >>> On Wed 20 Feb 2013 02:24:03 PM CET, Alexander Bokovoy wrote: >>>> On Wed, 20 Feb 2013, Tomas Babej wrote: >>>>> On 12/21/2012 12:15 PM, Tomas Babej wrote: >>>>>> Hi, >>>>>> >>>>>> Sending updated and rebased versions of patches 0024 and 0025. >>>>>> >>>>>> Tomas >>>>>> >>>>>> >>>>> Sending rebased version, these got quite rotten. >>>> Thanks for updating them. >>>> >>>>> @@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): >>>>> 'not be found. Please specify the SID >>>>> directly ' >>>>> 'using dom-sid option.')) >>>>> >>>>> - try: >>>>> - (old_dn, old_attrs) = ldap.get_entry(dn, >>>>> - ['ipabaseid', >>>>> - 'ipaidrangesize', >>>>> - 'ipabaserid', >>>>> - 'ipasecondarybaserid']) >>>>> - except errors.NotFound: >>>>> - self.obj.handle_not_found(*keys) >>>>> + if in_updated_attrs('ipanttrusteddomainsid'): >>>>> + if in_updated_attrs('ipasecondarybaserid'): >>>>> + raise errors.ValidationError(name='ID Range setup', >>>>> + error=_('Options dom_sid and secondary_rid_base >>>>> cannot ' >>>>> + 'be used together')) >>>> Since we agreed to refer to options by their CLI name (--dom-sid and >>>> --secondary-rid-base) in the other patch, it makes sense to use it >>>> here too. >>>> >>>> >>>>> - if is_set('ipanttrusteddomainsid'): >>>>> - # Validate SID as the one of trusted domains >>>>> - >>>>> self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >>>>> >>>>> + if not in_updated_attrs('ipabaserid'): >>>>> + raise errors.ValidationError(name='ID Range setup', >>>>> + error=_('Options dom_sid and rid_base must ' >>>>> + 'be used together')) >>>> Same here. >>>> >>>>> + # secondary base rid must be set if and only if base rid >>>>> is set >>>>> + if in_updated_attrs('ipasecondarybaserid') !=\ >>>>> + in_updated_attrs('ipabaserid'): >>>>> + raise errors.ValidationError(name='ID Range setup', >>>>> + error=_('Options secondary_rid_base and rid_base >>>>> must ' >>>>> + 'be used together')) >>>> Same here. >>>> >>>>> + dict( >>>>> + desc='Try to modify ID range %r so it has only primary >>>>> rid range set' % (testrange8), >>>>> + command=('idrange_mod', [testrange8], >>>>> + dict(ipabaserid=testrange8_base_rid)), >>>>> + expected=errors.ValidationError( >>>>> + name='ID Range setup', error='Options >>>>> secondary_rid_base and rid_base must be used together'), >>>>> + ), >>>> And synchronize error message here too. >>>> >>> Thanks! >>> >>> Sending the updated patch 0024. >>> >>> Tomas >>> >> In patch 0024 your intention is OK, but the checking functions are not: >> >> is_set = lambda x: (x in entry_attrs) and (x is not None) >> + in_updated_attrs = lambda x: any((x in attrs and x is not None) >> + for attrs in (entry_attrs, old_attrs)) >> >> They return True even when the attribute is None because they check if *x* is >> None and not if *attrs[x]* is None. Example: >> >> # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 >> --secondary-rid-base=1000000 local_range >> ---------------------------- >> Added ID range "local_range" >> ---------------------------- >> Range name: local_range >> First Posix ID of the range: 1200000 >> Number of IDs in the range: 200000 >> First RID of the corresponding RID range: 1000 >> First RID of the secondary RID range: 1000000 >> Range type: local domain range >> >> This command should be NOOP, but is not: >> >> # ipa idrange-mod local_range --dom-sid= >> ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base >> cannot be used together >> >> Martin > Thanks for the catch, all checking functions fixed. > > Sending the complete patchset, up to date. > > Tomas I think I am being a nitpicker now (sorry), but this condition also fails if the old_attrs has a setting, but I am removing it in a current -mod command: # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 --secondary-rid-base=1000000 local_range ---------------------------- Added ID range "local_range" ---------------------------- Range name: local_range First Posix ID of the range: 1200000 Number of IDs in the range: 200000 First RID of the corresponding RID range: 1000 First RID of the secondary RID range: 1000000 Range type: local domain range # ipa idrange-mod local_range --dom-sid S-1-2 --secondary-rid-base= ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base cannot be used together Martin From mkosek at redhat.com Fri Feb 22 16:26:21 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 17:26:21 +0100 Subject: [Freeipa-devel] [PATCH] 0182 Fix permission validation and normalization in aci.py In-Reply-To: <512619EC.3010601@redhat.com> References: <512619EC.3010601@redhat.com> Message-ID: <51279C2D.60905@redhat.com> On 02/21/2013 01:58 PM, Petr Viktorin wrote: > This patch fixes an internal error in the permission plugin that would become > more noticeable when CSV is dropped. > > https://fedorahosted.org/freeipa/ticket/3420 > ACK. Pushed to master, ipa-3-1. Martin From pvoborni at redhat.com Fri Feb 22 16:43:37 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 22 Feb 2013 17:43:37 +0100 Subject: [Freeipa-devel] [PATCH] 264-265 Web UI:Certificate pages Message-ID: <5127A039.6030902@redhat.com> Note: static json files for testing and such will be updated soon (there were several patch which changes API. I rather want to do one mass regeneration than several minor ones in a short period of time. 1) [PATCH] Web UI:Certificate pages Following pages were added to Web UI: * certificated details * certificate search Certificate is not regular object so it gets no metadata. Therefore artificial metadata were created for it to allow usage of search and details facet. Search and details facet were modified to allow removing of add/remove/update/reset buttons - certificates have no mod operation and they are not added by standard means. User can revoke and restore certificated in details facet. https://fedorahosted.org/freeipa/ticket/3419 2) [PATCH] Web UI:Choose different search option for cert-find This extends certificate search page by search option select. Therefore the search is not restricted to 'subject'. It should be replaced by https://fedorahosted.org/freeipa/ticket/191 in a future. https://fedorahosted.org/freeipa/ticket/3419 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0264-Web-UI-Certificate-pages.patch Type: text/x-patch Size: 22153 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0265-Web-UI-Choose-different-search-option-for-cert-find.patch Type: text/x-patch Size: 10431 bytes Desc: not available URL: From mkosek at redhat.com Fri Feb 22 16:33:45 2013 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 22 Feb 2013 17:33:45 +0100 Subject: [Freeipa-devel] [PATCHES] 0183-0185 Drop CSV support In-Reply-To: <51277B10.8000205@redhat.com> References: <512654A9.1060309@redhat.com> <51276C22.5010702@redhat.com> <51277466.6070002@redhat.com> <51277B10.8000205@redhat.com> Message-ID: <51279DE9.1030109@redhat.com> On 02/22/2013 03:05 PM, Petr Viktorin wrote: > On 02/22/2013 02:36 PM, Martin Kosek wrote: >> On 02/22/2013 02:01 PM, Martin Kosek wrote: >>> On 02/21/2013 06:08 PM, Petr Viktorin wrote: >>>> These patches remove CSV parsing from the client. >>>> >>>> Ticket: https://fedorahosted.org/freeipa/ticket/3352 >>>> Design: http://freeipa.org/page/V3/Drop_CSV >>>> >>>> >>>> The design page also talks about adding a warning for the user when they seem >>>> to use CSV, but this will need the JSON transport so it's not included in >>>> these >>>> patches. >>>> >>>> The "csv" flag is left on parameters so when that warning is added, we know >>>> for >>>> which params to show it. >>>> >>> >>> Generally, this works fine. This finally allows us to easily add records with >>> quote or comma: >>> >>> # ipa dnsrecord-add example.com txt1 --txt-rec="foo, bar" >>> Record name: txt1 >>> TXT record: foo, bar >>> # ipa dnsrecord-add example.com txt1 --txt-rec='"bar,bar"' >>> Record name: txt1 >>> TXT record: foo, bar, "bar,bar" >>> # ipa dnsrecord-show example.com txt1 --all --raw >>> dn: idnsname=txt1,idnsname=example.com,cn=dns,dc=example,dc=com >>> idnsname: txt1 >>> txtrecord: foo, bar >>> txtrecord: "bar,bar" >>> objectclass: top >>> objectclass: idnsrecord >>> >>> >>> >>> Just couple minor remarks: >>> >>> 1) Can we then update the csv flag description? It is now a bit misleading >>> given its new function >>> >>> - csv: this multivalue attribute is given in CSV format > > I have reworded that in patch 184: > > - - csv: this multivalue attribute is given in CSV format > + - csv: this multivalue attribute used to be given in CSV format in CLI > >>> 2) We need to also update our help, for example selfservice online help >>> presents a CSV formatted option use: >>> >>> ipa selfservice-add --permissions=write --attrs=street,postalCode,l,c,st >>> "Users manage their own address" >>> >>> Btw if we suggest the curly braces shortcut, we may also want to write that it >>> applies for BASH shell only... >>> >>> Few other examples I saw: >>> ipa group-add-member --users=test1,test2 localadmins >>> > > Attaching patch for that. > I used the Bash shortcut when there were more than two items. > I grepped through the sources to find examples that need changing, hopefully > I've caught them all. > >> Just discovered another issue... We now crash when adding permissions/acis with >> csv: >> >> # ipa permission-add --attrs=member --permissions=read,write --type=group foo2 >> ipa: ERROR: an internal error has occurred >> >> We should report some meaningful error, otherwise users used to add permissions >> with csv values would be confused. > > See ticket #3420, patch 0182. > Thanks for the fixes. ACK. Pushed to master. Martin From rcritten at redhat.com Fri Feb 22 20:19:43 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 22 Feb 2013 15:19:43 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges Message-ID: <5127D2DF.80809@redhat.com> Design to allow one to recover DNA ranges when deleting a replica or just for normal range management. http://freeipa.org/page/V3/Recover_DNA_Ranges Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 rob From tbabej at redhat.com Sat Feb 23 00:54:41 2013 From: tbabej at redhat.com (Tomas Babej) Date: Sat, 23 Feb 2013 01:54:41 +0100 Subject: [Freeipa-devel] [PATCH 0036] Make sure appropriate exit status is returned in make-test Message-ID: <51281351.6030405@redhat.com> Hi, The make-test script now exits with code 1 in case that any of the test cases that were run failed. Can we push this without a ticket under one-liner rule? Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0036-Make-sure-appropriate-exit-status-is-returned-in-mak.patch Type: text/x-patch Size: 692 bytes Desc: not available URL: From mkosek at redhat.com Mon Feb 25 12:42:26 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 25 Feb 2013 13:42:26 +0100 Subject: [Freeipa-devel] [PATCH] 373 Bump FreeIPA version for development branch Message-ID: <512B5C32.5040506@redhat.com> Current master branch represents future release of FreeIPA (3.2). Bump VERSION so that current development packages are not being updated with freeipa-3.1.x packages already released in downstream repositories. --- Pushed under the one-liner rule. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-373-bump-freeipa-version-for-development-branch.patch Type: text/x-patch Size: 940 bytes Desc: not available URL: From pviktori at redhat.com Mon Feb 25 12:44:11 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 25 Feb 2013 13:44:11 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <5127D2DF.80809@redhat.com> References: <5127D2DF.80809@redhat.com> Message-ID: <512B5C9B.8020805@redhat.com> On 02/22/2013 09:19 PM, Rob Crittenden wrote: > Design to allow one to recover DNA ranges when deleting a replica or > just for normal range management. > > http://freeipa.org/page/V3/Recover_DNA_Ranges > > Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 > > rob I wonder if it would be possible to have more on-deck ranges. Could dnaNextRange be multi-valued, and when the low-water mark is hit the plugin would pick one of them? -- Petr? From mkosek at redhat.com Mon Feb 25 13:09:15 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 25 Feb 2013 14:09:15 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B5C9B.8020805@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> Message-ID: <512B627B.8090403@redhat.com> On 02/25/2013 01:44 PM, Petr Viktorin wrote: > On 02/22/2013 09:19 PM, Rob Crittenden wrote: >> Design to allow one to recover DNA ranges when deleting a replica or >> just for normal range management. >> >> http://freeipa.org/page/V3/Recover_DNA_Ranges >> >> Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 >> >> rob > > I wonder if it would be possible to have more on-deck ranges. Could > dnaNextRange be multi-valued, and when the low-water mark is hit the plugin > would pick one of them? > Not at the moment, this is a single valued attribute type: attributetypes: ( 2.16.840.1.113730.3.1.2129 NAME 'dnaNextRange' DESC 'DNA ran ge of values to get from replica' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE -VALUE X-ORIGIN '389 Directory Server' ) But it is a good question for 389-ds guys, it would be a good extension to the DNA plugin and would prevent us from not-loosing the range when there is no master with empty dnaNextRange. But maybe there is a strong reason why this was made single value... As for the RFE, I have few comments/questions for Rob: 1) I would expand "Setting the on-deck range" section and add an information what should we do when the remote master is not accessible (this would result only in a warning probably). 2) We may want to make sure that the removed replica is readonly before we copy the range (just to be sure that we do not miss some value due to race condition. 3) In "Enhancing ipa-replica-manage": What does "ipa-replica-manage dnarange-set masterA.example.com 250-499" exactly do? I though that it would just overwrite active range, but based on the next "ipa-replica-manage dnanextrange-show" example, it moved the currently active range of masterA.example.com to the on-deck range. Do we want to do that? 4) What does "NOTE: We will need to be clear that this range has nothing to do with Trust ranges." actually mean? AFAIU, IPA should have all local ranges covered with a local "idrange" range(s). If it does not have it covered, it could happen that for example a new trust would overlap with this user-defined local range and we would have colliding POSIX IDs... IMO, dnarange-set and dnanextrange-set should at first check if the range is covered with some local idrange and only then allowed setting the new range. Martin From tbabej at redhat.com Mon Feb 25 13:16:28 2013 From: tbabej at redhat.com (Tomas Babej) Date: Mon, 25 Feb 2013 14:16:28 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <5127901F.4040408@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> <20130220132403.GN476@redhat.com> <5124DB7F.3020101@redhat.com> <51261FB3.8030005@redhat.com> <51277A41.4060902@redhat.com> <5127901F.4040408@redhat.com> Message-ID: <512B642C.3080609@redhat.com> On Fri 22 Feb 2013 04:34:55 PM CET, Martin Kosek wrote: > On 02/22/2013 03:01 PM, Tomas Babej wrote: >> On 02/21/2013 02:22 PM, Martin Kosek wrote: >>> On 02/20/2013 03:19 PM, Tomas Babej wrote: >>>> On Wed 20 Feb 2013 02:24:03 PM CET, Alexander Bokovoy wrote: >>>>> On Wed, 20 Feb 2013, Tomas Babej wrote: >>>>>> On 12/21/2012 12:15 PM, Tomas Babej wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Sending updated and rebased versions of patches 0024 and 0025. >>>>>>> >>>>>>> Tomas >>>>>>> >>>>>>> >>>>>> Sending rebased version, these got quite rotten. >>>>> Thanks for updating them. >>>>> >>>>>> @@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): >>>>>> 'not be found. Please specify the SID >>>>>> directly ' >>>>>> 'using dom-sid option.')) >>>>>> >>>>>> - try: >>>>>> - (old_dn, old_attrs) = ldap.get_entry(dn, >>>>>> - ['ipabaseid', >>>>>> - 'ipaidrangesize', >>>>>> - 'ipabaserid', >>>>>> - 'ipasecondarybaserid']) >>>>>> - except errors.NotFound: >>>>>> - self.obj.handle_not_found(*keys) >>>>>> + if in_updated_attrs('ipanttrusteddomainsid'): >>>>>> + if in_updated_attrs('ipasecondarybaserid'): >>>>>> + raise errors.ValidationError(name='ID Range setup', >>>>>> + error=_('Options dom_sid and secondary_rid_base >>>>>> cannot ' >>>>>> + 'be used together')) >>>>> Since we agreed to refer to options by their CLI name (--dom-sid and >>>>> --secondary-rid-base) in the other patch, it makes sense to use it >>>>> here too. >>>>> >>>>> >>>>>> - if is_set('ipanttrusteddomainsid'): >>>>>> - # Validate SID as the one of trusted domains >>>>>> - >>>>>> self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >>>>>> >>>>>> + if not in_updated_attrs('ipabaserid'): >>>>>> + raise errors.ValidationError(name='ID Range setup', >>>>>> + error=_('Options dom_sid and rid_base must ' >>>>>> + 'be used together')) >>>>> Same here. >>>>> >>>>>> + # secondary base rid must be set if and only if base rid >>>>>> is set >>>>>> + if in_updated_attrs('ipasecondarybaserid') !=\ >>>>>> + in_updated_attrs('ipabaserid'): >>>>>> + raise errors.ValidationError(name='ID Range setup', >>>>>> + error=_('Options secondary_rid_base and rid_base >>>>>> must ' >>>>>> + 'be used together')) >>>>> Same here. >>>>> >>>>>> + dict( >>>>>> + desc='Try to modify ID range %r so it has only primary >>>>>> rid range set' % (testrange8), >>>>>> + command=('idrange_mod', [testrange8], >>>>>> + dict(ipabaserid=testrange8_base_rid)), >>>>>> + expected=errors.ValidationError( >>>>>> + name='ID Range setup', error='Options >>>>>> secondary_rid_base and rid_base must be used together'), >>>>>> + ), >>>>> And synchronize error message here too. >>>>> >>>> Thanks! >>>> >>>> Sending the updated patch 0024. >>>> >>>> Tomas >>>> >>> In patch 0024 your intention is OK, but the checking functions are not: >>> >>> is_set = lambda x: (x in entry_attrs) and (x is not None) >>> + in_updated_attrs = lambda x: any((x in attrs and x is not None) >>> + for attrs in (entry_attrs, old_attrs)) >>> >>> They return True even when the attribute is None because they check if *x* is >>> None and not if *attrs[x]* is None. Example: >>> >>> # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 >>> --secondary-rid-base=1000000 local_range >>> ---------------------------- >>> Added ID range "local_range" >>> ---------------------------- >>> Range name: local_range >>> First Posix ID of the range: 1200000 >>> Number of IDs in the range: 200000 >>> First RID of the corresponding RID range: 1000 >>> First RID of the secondary RID range: 1000000 >>> Range type: local domain range >>> >>> This command should be NOOP, but is not: >>> >>> # ipa idrange-mod local_range --dom-sid= >>> ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base >>> cannot be used together >>> >>> Martin >> Thanks for the catch, all checking functions fixed. >> >> Sending the complete patchset, up to date. >> >> Tomas > > I think I am being a nitpicker now (sorry), but this condition also fails if > the old_attrs has a setting, but I am removing it in a current -mod command: > > # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 > --secondary-rid-base=1000000 local_range > ---------------------------- > Added ID range "local_range" > ---------------------------- > Range name: local_range > First Posix ID of the range: 1200000 > Number of IDs in the range: 200000 > First RID of the corresponding RID range: 1000 > First RID of the secondary RID range: 1000000 > Range type: local domain range > # ipa idrange-mod local_range --dom-sid S-1-2 --secondary-rid-base= > ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base > cannot be used together > > Martin Yep. Ok, the command should now handle this use case as well. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0025-7-Add-trusted-domain-range-objectclass-when-using-idra.patch Type: text/x-patch Size: 1325 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0024-7-Make-options-checks-in-idrange-add-mod-consistent.patch Type: text/x-patch Size: 9550 bytes Desc: not available URL: From mkosek at redhat.com Mon Feb 25 12:33:05 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 25 Feb 2013 13:33:05 +0100 Subject: [Freeipa-devel] [PATCH 0036] Make sure appropriate exit status is returned in make-test In-Reply-To: <51281351.6030405@redhat.com> References: <51281351.6030405@redhat.com> Message-ID: <512B5A01.7090201@redhat.com> On 02/23/2013 01:54 AM, Tomas Babej wrote: > Hi, > > The make-test script now exits with code 1 in case that > any of the test cases that were run failed. > > Can we push this without a ticket under one-liner rule? > > Tomas > I have no problem with this one, it will help with your unit test automation. ACK. Pushed to master, ipa-3-1. Martin From rcritten at redhat.com Mon Feb 25 14:09:27 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 25 Feb 2013 09:09:27 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B627B.8090403@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> Message-ID: <512B7097.3030809@redhat.com> Martin Kosek wrote: > On 02/25/2013 01:44 PM, Petr Viktorin wrote: >> On 02/22/2013 09:19 PM, Rob Crittenden wrote: >>> Design to allow one to recover DNA ranges when deleting a replica or >>> just for normal range management. >>> >>> http://freeipa.org/page/V3/Recover_DNA_Ranges >>> >>> Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 >>> >>> rob >> >> I wonder if it would be possible to have more on-deck ranges. Could >> dnaNextRange be multi-valued, and when the low-water mark is hit the plugin >> would pick one of them? >> > > Not at the moment, this is a single valued attribute type: > > attributetypes: ( 2.16.840.1.113730.3.1.2129 NAME 'dnaNextRange' DESC 'DNA ran > ge of values to get from replica' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE > -VALUE X-ORIGIN '389 Directory Server' ) > > But it is a good question for 389-ds guys, it would be a good extension to the > DNA plugin and would prevent us from not-loosing the range when there is no > master with empty dnaNextRange. But maybe there is a strong reason why this was > made single value... > > > As for the RFE, I have few comments/questions for Rob: > > 1) I would expand "Setting the on-deck range" section and add an information > what should we do when the remote master is not accessible (this would result > only in a warning probably). Ok. I'll try to clarify this. > > 2) We may want to make sure that the removed replica is readonly before we copy > the range (just to be sure that we do not miss some value due to race condition. That is attempted now as part of CLEANALLRUV. I'll mention it in the design so we time this properly. > > 3) In "Enhancing ipa-replica-manage": > > What does "ipa-replica-manage dnarange-set masterA.example.com 250-499" exactly > do? I though that it would just overwrite active range, but based on the next > "ipa-replica-manage dnanextrange-show" example, it moved the currently active > range of masterA.example.com to the on-deck range. Do we want to do that? I'm guessing I wasn't very precise with my example, focusing more on usage than output. These are all independent of one another. I'll make it more of a sequence. > > > 4) What does "NOTE: We will need to be clear that this range has nothing to do > with Trust ranges." actually mean? AFAIU, IPA should have all local ranges > covered with a local "idrange" range(s). IPA ranges is completely separate from DNA ranges. You can set/modify all the local ranges you want and it won't affect the UIDs getting assigned. > If it does not have it covered, it could happen that for example a new trust > would overlap with this user-defined local range and we would have colliding > POSIX IDs... Hmm, that's a good point. > IMO, dnarange-set and dnanextrange-set should at first check if the range is > covered with some local idrange and only then allowed setting the new range. I can do that as well, but again, the local ranges don't really affect the ids we hand out via DNA. rob From mkosek at redhat.com Mon Feb 25 14:12:19 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 25 Feb 2013 15:12:19 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B7097.3030809@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> Message-ID: <512B7143.6010204@redhat.com> On 02/25/2013 03:09 PM, Rob Crittenden wrote: > Martin Kosek wrote: ... >> 4) What does "NOTE: We will need to be clear that this range has nothing to do >> with Trust ranges." actually mean? AFAIU, IPA should have all local ranges >> covered with a local "idrange" range(s). > > IPA ranges is completely separate from DNA ranges. You can set/modify all the > local ranges you want and it won't affect the UIDs getting assigned. > >> If it does not have it covered, it could happen that for example a new trust >> would overlap with this user-defined local range and we would have colliding >> POSIX IDs... > > Hmm, that's a good point. > >> IMO, dnarange-set and dnanextrange-set should at first check if the range is >> covered with some local idrange and only then allowed setting the new range. > > I can do that as well, but again, the local ranges don't really affect the ids > we hand out via DNA. > > rob You are right, that DNA plugin is really not aware of the idranges we set in IPA. But the idrange is still a safeguard for our POSIX IDs to not overlap with trust ranges and I think we should respect that with ipa-replica-manage. I wonder if there was not even a plan to increase cooperation between our idranges and DNA plugin, maybe Sumit or Alexander knows more. Martin From pspacek at redhat.com Mon Feb 25 14:28:57 2013 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 25 Feb 2013 15:28:57 +0100 Subject: [Freeipa-devel] [PATCH 0116] Fix crash caused by invalid wildcard in update policy string Message-ID: <512B7529.5080506@redhat.com> Hello, Fix crash caused by invalid wildcard in update policy string. https://fedorahosted.org/bind-dyndb-ldap/ticket/108 Question: What we should do if update policy string contains an error? Should we disable all updates? Or let the old policy in place? I vote for disallowing all updates. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0116-Fix-crash-caused-by-invalid-wildcard-in-update-polic.patch Type: text/x-patch Size: 1283 bytes Desc: not available URL: From jcholast at redhat.com Mon Feb 25 15:10:26 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 25 Feb 2013 16:10:26 +0100 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc In-Reply-To: <511A6D60.6030300@redhat.com> References: <511A6D60.6030300@redhat.com> Message-ID: <512B7EE2.6050704@redhat.com> Hi, On 12.2.2013 17:27, Ana Krivokapic wrote: > Add new LDAP container to store the list of domains associated with IPA > realm. > Add two new ipa commands (ipa realmdomains-show and ipa > realmdomains-mod) to allow manipulation of the list of realm domains. > Unit test file covering these new commands was added. > > https://fedorahosted.org/freeipa/ticket/2945 > I have just noticed a failure in the test for this: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/home/jcholast/freeipa/tests/test_xmlrpc/test_realmdomains_plugin.py", line 35, in bad_domain = u'this-domain-does-not-exist-%s.com' % ''.join(random.choice(string.lowercase) for x in range(10)) UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 0: invalid continuation byte This is happening because string.lowercase is locale-dependent, so it might contain "bad" values, such as '\xed'. Please use string.ascii_lowercase instead to prevent this. Honza -- Jan Cholasta From rmeggins at redhat.com Mon Feb 25 15:12:25 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 25 Feb 2013 08:12:25 -0700 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B627B.8090403@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> Message-ID: <512B7F59.3040207@redhat.com> On 02/25/2013 06:09 AM, Martin Kosek wrote: > On 02/25/2013 01:44 PM, Petr Viktorin wrote: >> On 02/22/2013 09:19 PM, Rob Crittenden wrote: >>> Design to allow one to recover DNA ranges when deleting a replica or >>> just for normal range management. >>> >>> http://freeipa.org/page/V3/Recover_DNA_Ranges >>> >>> Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 >>> >>> rob >> I wonder if it would be possible to have more on-deck ranges. Could >> dnaNextRange be multi-valued, and when the low-water mark is hit the plugin >> would pick one of them? >> > Not at the moment, this is a single valued attribute type: > > attributetypes: ( 2.16.840.1.113730.3.1.2129 NAME 'dnaNextRange' DESC 'DNA ran > ge of values to get from replica' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE > -VALUE X-ORIGIN '389 Directory Server' ) > > But it is a good question for 389-ds guys, it would be a good extension to the > DNA plugin and would prevent us from not-loosing the range when there is no > master with empty dnaNextRange. But maybe there is a strong reason why this was > made single value... If you make it multi-valued, then you probably want to have some sort of ordering to the values . . . > > > As for the RFE, I have few comments/questions for Rob: > > 1) I would expand "Setting the on-deck range" section and add an information > what should we do when the remote master is not accessible (this would result > only in a warning probably). > > > 2) We may want to make sure that the removed replica is readonly before we copy > the range (just to be sure that we do not miss some value due to race condition. > > > 3) In "Enhancing ipa-replica-manage": > > What does "ipa-replica-manage dnarange-set masterA.example.com 250-499" exactly > do? I though that it would just overwrite active range, but based on the next > "ipa-replica-manage dnanextrange-show" example, it moved the currently active > range of masterA.example.com to the on-deck range. Do we want to do that? > > > 4) What does "NOTE: We will need to be clear that this range has nothing to do > with Trust ranges." actually mean? AFAIU, IPA should have all local ranges > covered with a local "idrange" range(s). > > If it does not have it covered, it could happen that for example a new trust > would overlap with this user-defined local range and we would have colliding > POSIX IDs... > > IMO, dnarange-set and dnanextrange-set should at first check if the range is > covered with some local idrange and only then allowed setting the new range. > > Martin From akrivoka at redhat.com Mon Feb 25 15:29:59 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Mon, 25 Feb 2013 16:29:59 +0100 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc In-Reply-To: <512B7EE2.6050704@redhat.com> References: <511A6D60.6030300@redhat.com> <512B7EE2.6050704@redhat.com> Message-ID: <512B8377.4050600@redhat.com> On 02/25/2013 04:10 PM, Jan Cholasta wrote: > Hi, > > On 12.2.2013 17:27, Ana Krivokapic wrote: >> Add new LDAP container to store the list of domains associated with IPA >> realm. >> Add two new ipa commands (ipa realmdomains-show and ipa >> realmdomains-mod) to allow manipulation of the list of realm domains. >> Unit test file covering these new commands was added. >> >> https://fedorahosted.org/freeipa/ticket/2945 >> > > I have just noticed a failure in the test for this: > > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/nose/loader.py", line 390, in > loadTestsFromName > addr.filename, addr.module) > File "/usr/lib/python2.7/site-packages/nose/importer.py", line 39, > in importFromPath > return self.importFromDir(dir_path, fqname) > File "/usr/lib/python2.7/site-packages/nose/importer.py", line 86, > in importFromDir > mod = load_module(part_fqname, fh, filename, desc) > File > "/home/jcholast/freeipa/tests/test_xmlrpc/test_realmdomains_plugin.py", line > 35, in > bad_domain = u'this-domain-does-not-exist-%s.com' % > ''.join(random.choice(string.lowercase) for x in range(10)) > UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 0: > invalid continuation byte > > This is happening because string.lowercase is locale-dependent, so it > might contain "bad" values, such as '\xed'. Please use > string.ascii_lowercase instead to prevent this. > > Honza > Thanks for the catch. I believe it is already fixed: https://www.redhat.com/archives/freeipa-devel/2013-February/msg00347.html -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. From jcholast at redhat.com Mon Feb 25 15:29:26 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 25 Feb 2013 16:29:26 +0100 Subject: [Freeipa-devel] [PATCH] 0005 Add list of domains associated to our realm to cn=etc In-Reply-To: <512B8377.4050600@redhat.com> References: <511A6D60.6030300@redhat.com> <512B7EE2.6050704@redhat.com> <512B8377.4050600@redhat.com> Message-ID: <512B8356.8090202@redhat.com> On 25.2.2013 16:29, Ana Krivokapic wrote: > On 02/25/2013 04:10 PM, Jan Cholasta wrote: >> Hi, >> >> On 12.2.2013 17:27, Ana Krivokapic wrote: >>> Add new LDAP container to store the list of domains associated with IPA >>> realm. >>> Add two new ipa commands (ipa realmdomains-show and ipa >>> realmdomains-mod) to allow manipulation of the list of realm domains. >>> Unit test file covering these new commands was added. >>> >>> https://fedorahosted.org/freeipa/ticket/2945 >>> >> >> I have just noticed a failure in the test for this: >> >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/nose/loader.py", line 390, in >> loadTestsFromName >> addr.filename, addr.module) >> File "/usr/lib/python2.7/site-packages/nose/importer.py", line 39, >> in importFromPath >> return self.importFromDir(dir_path, fqname) >> File "/usr/lib/python2.7/site-packages/nose/importer.py", line 86, >> in importFromDir >> mod = load_module(part_fqname, fh, filename, desc) >> File >> "/home/jcholast/freeipa/tests/test_xmlrpc/test_realmdomains_plugin.py", line >> 35, in >> bad_domain = u'this-domain-does-not-exist-%s.com' % >> ''.join(random.choice(string.lowercase) for x in range(10)) >> UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 0: >> invalid continuation byte >> >> This is happening because string.lowercase is locale-dependent, so it >> might contain "bad" values, such as '\xed'. Please use >> string.ascii_lowercase instead to prevent this. >> >> Honza >> > > Thanks for the catch. I believe it is already fixed: > https://www.redhat.com/archives/freeipa-devel/2013-February/msg00347.html > Oh, OK :) -- Jan Cholasta From mkosek at redhat.com Mon Feb 25 15:35:20 2013 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 25 Feb 2013 16:35:20 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <20130221152446.GS27418@localhost.localdomain> References: <20130221152446.GS27418@localhost.localdomain> Message-ID: <512B84B8.7080506@redhat.com> On 02/21/2013 04:24 PM, Sumit Bose wrote: > Hi, > > this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 > The related design page is > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > It was agreed while discussing the design page that the currently > hardcoded value for the NFS service will be dropped completely, hence > the first patch reverts to patch which added the hardcoded value. > > The second patch adds the needed attribute to compensate the now missing > hardcoded value. > > In the following three patches the PAC type is read and used > accordingly. The last patch adds a unit test for a new function. > > bye, > Sumit I did only sanity testing to the C part of the RFE so far, but by reading it it looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part is OK. I will comment on the Python parts: 1) Your update check testing if there is any NFS service with ipakrbauthzdata looks like a good heuristics to prevent admin frustration. Though an ideal solution would require https://fedorahosted.org/freeipa/ticket/2961 to prevent multiple updates when admin purposefully removes this attribute. We may want to reference the ticket in a comment in the update plugin... 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns truncated result. As you do not update entries directly but only return update instructions when you have no truncated results, you will loop. To simulate this, I just created 2 NFS principals and set size_limit=1 in your find_entries call. 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS service principals added by service-add? Shouldn't we set ipakrbauthzdata for new services too? As a default when no user ipakrhbauthzdata is set... Otherwise admins could be surprised why their new NFS services do not work while the others do. Also, I think we should have this NFS culprit documented somewhere (i.e. why we set ipakrbauthzdata to NONE by default). At least as a small section in the online help for services plugin... Martin From rcritten at redhat.com Mon Feb 25 16:23:10 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 25 Feb 2013 11:23:10 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B7F59.3040207@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7F59.3040207@redhat.com> Message-ID: <512B8FEE.50608@redhat.com> Rich Megginson wrote: > On 02/25/2013 06:09 AM, Martin Kosek wrote: >> On 02/25/2013 01:44 PM, Petr Viktorin wrote: >>> On 02/22/2013 09:19 PM, Rob Crittenden wrote: >>>> Design to allow one to recover DNA ranges when deleting a replica or >>>> just for normal range management. >>>> >>>> http://freeipa.org/page/V3/Recover_DNA_Ranges >>>> >>>> Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 >>>> >>>> rob >>> I wonder if it would be possible to have more on-deck ranges. Could >>> dnaNextRange be multi-valued, and when the low-water mark is hit the >>> plugin >>> would pick one of them? >>> >> Not at the moment, this is a single valued attribute type: >> >> attributetypes: ( 2.16.840.1.113730.3.1.2129 NAME 'dnaNextRange' DESC >> 'DNA ran >> ge of values to get from replica' SYNTAX >> 1.3.6.1.4.1.1466.115.121.1.15 SINGLE >> -VALUE X-ORIGIN '389 Directory Server' ) >> >> But it is a good question for 389-ds guys, it would be a good >> extension to the >> DNA plugin and would prevent us from not-loosing the range when there >> is no >> master with empty dnaNextRange. But maybe there is a strong reason why >> this was >> made single value... > > If you make it multi-valued, then you probably want to have some sort of > ordering to the values . . . I don't know. We don't have a whole lot of control of ordering when DNA gets a range, so I think that holes in the range happen now, so I wouldn't have a problem with lack of control. rob From rmeggins at redhat.com Mon Feb 25 16:22:33 2013 From: rmeggins at redhat.com (Rich Megginson) Date: Mon, 25 Feb 2013 09:22:33 -0700 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B8FEE.50608@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7F59.3040207@redhat.com> <512B8FEE.50608@redhat.com> Message-ID: <512B8FC9.6020309@redhat.com> On 02/25/2013 09:23 AM, Rob Crittenden wrote: > Rich Megginson wrote: >> On 02/25/2013 06:09 AM, Martin Kosek wrote: >>> On 02/25/2013 01:44 PM, Petr Viktorin wrote: >>>> On 02/22/2013 09:19 PM, Rob Crittenden wrote: >>>>> Design to allow one to recover DNA ranges when deleting a replica or >>>>> just for normal range management. >>>>> >>>>> http://freeipa.org/page/V3/Recover_DNA_Ranges >>>>> >>>>> Supporting ticket https://fedorahosted.org/freeipa/ticket/3321 >>>>> >>>>> rob >>>> I wonder if it would be possible to have more on-deck ranges. Could >>>> dnaNextRange be multi-valued, and when the low-water mark is hit the >>>> plugin >>>> would pick one of them? >>>> >>> Not at the moment, this is a single valued attribute type: >>> >>> attributetypes: ( 2.16.840.1.113730.3.1.2129 NAME 'dnaNextRange' DESC >>> 'DNA ran >>> ge of values to get from replica' SYNTAX >>> 1.3.6.1.4.1.1466.115.121.1.15 SINGLE >>> -VALUE X-ORIGIN '389 Directory Server' ) >>> >>> But it is a good question for 389-ds guys, it would be a good >>> extension to the >>> DNA plugin and would prevent us from not-loosing the range when there >>> is no >>> master with empty dnaNextRange. But maybe there is a strong reason why >>> this was >>> made single value... >> >> If you make it multi-valued, then you probably want to have some sort of >> ordering to the values . . . > > I don't know. We don't have a whole lot of control of ordering when > DNA gets a range, so I think that holes in the range happen now, so I > wouldn't have a problem with lack of control. Ok. Please file an RFE ticket. There are some code changes that we will need to make to DNA to make next range take multiple values. > > rob > From jcholast at redhat.com Mon Feb 25 16:39:02 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 25 Feb 2013 17:39:02 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <5124BB8F.80803@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> <512387D1.60603@redhat.com> <5124BB8F.80803@redhat.com> Message-ID: <512B93A6.4050602@redhat.com> On 20.2.2013 13:03, Petr Viktorin wrote: > On 02/19/2013 03:10 PM, Jan Cholasta wrote: >> On 1.2.2013 15:38, Petr Viktorin wrote: >>> Alright, I renamed get_single to single_value(). >>> >>> I also rebased to current master. >>> >> >> Patch 152: >> >> + def single_value(self, name, default=_missing): >> + values = self.get(name, [default]) >> + if len(values) != 1: >> + raise ValueError( >> + '%s has %s values, one expected' % (name, len(values))) >> + if values[0] is _missing: >> + raise KeyError(name) >> + return values[0] >> >> I would prefer if you used __getitem__() instead of get() and re-raised >> KeyError if default is _missing. Also, until we disallow non-lists, we >> need to check if values actually is list. I.e., do something like this: >> >> + def single_value(self, name, default=_missing): >> + try: >> + values = super(LDAPEntry, >> self).__getitem__(self._get_attr_name(name)) >> + except KeyError: >> + if default is _missing: >> + raise >> + return default >> + if not isinstance(values, list): >> + return values >> + if len(values) != 1: >> + raise ValueError( >> + '%s has %s values, one expected' % (name, len(values))) >> + return values[0] > > Updated, thanks. > > _get_attr_name is only added in your patch 99, so I used _attr_name here > and modified your patch. > > I wrote some tests for single_value, and while I was at it I added tests > for a few other LDAPEntry methods as well. I've also split things up > into more testcases. Including as patch 0181. Thanks. I think you should also add a tearDown method to test_LDAPEntry which disconnects self.conn if it is connected (the same thing test_ldap does). > >> Patch 159: >> >> Like I said in my review of your patch 143, I think we should use DNs >> instead of entries in delete_entry, so I think it would make sense to do >> delete_entry(entry.dn) in this patch. > > I think that for symmetry with add_entry and update_entry, delete_entry > should take entries too. If you already have the entry, having to type > the extra ".dn" is not very intuitive. > The proper thing to do would be a separate delete_by_dn(), but > delete_entry() taking either entries or DNs works fine IMO. OK, makes sense. > >> Patch 160: >> >> I think you should do this (replace % with ,): >> >> + root_logger.debug( >> + "failed to find mapping tree entry for %s", self.suffix) >> > > Fixed, thanks. > > I've also rebased 174 and your patch 104 to current master. > Honza -- Jan Cholasta From loris at lgs.com.ve Mon Feb 25 19:15:44 2013 From: loris at lgs.com.ve (Loris Santamaria) Date: Mon, 25 Feb 2013 14:45:44 -0430 Subject: [Freeipa-devel] What about desktop policies? Message-ID: <1361819744.2111.48.camel@toron.pzo.lgs.com.ve> Hi all, some customers of ours are interested in managing desktop policies for their linux workstations, really nothing fancy, corporate background and proxy settings are the most common requests. In the past I created Gnome desktop profiles using Sabayon, distributed them using puppet and associated them to user accounts with a Sabayon specific LDAP attribute, a process a bit convoluted, and no longer possible since sabayon is no longer developed. Also it was really buggy, and very gnome specific. I was thinking in how integrate desktop policies in freeIPA in a general manner and I wanted to share my ideas with you. Hopefully some of this may be incorporated in IPA at some point in the future. Properties of a "policy": * is a collection of "settings" * can be associated with users or groups (desktop policy) or with hosts or hostgroups (system policy) * is associated with a "consumer", the client software that interprets and applies the policy. This way one could define policies for dconf, policies for kde, policies for WBEM. Properties of a "setting" * is a key-value pair * must conform to a "schema" * may be mandatory The schema: * indicates which attributes a policy may consist of * indicates which kind of value may take an attribute. Bool, string, etc. * There may be more than one schema for a given "consumer". For example for dconf you may have an evolution schema, a gnome-games schema, etc. Sample policy creation process: 1. The admin creates a new schema in IPA, with a command like "ipa schema-add --consumer=dconf gnomeSettingsSchema" 2. The admin adds some definition to the schema: "ipa schema-add-setting gnomeSettingsSchema --name=/schemas/desktop/gnome/background/picture_filename --type=string --description='File to use for the background image.'" 3. He creates a new policy: "ipa policy-add corporateBackground --type=desktop --consumer=dconf 4. He adds a setting to the policy: "ipa policy-add-setting corporateBackground --name=/schemas/desktop/gnome/background/picture_filename --value=file:///san/wp/wallpaper.jpg --mandatory". Ipa would check that the key is defined in one of the dconf related schemas and the value is acceptable for that key. 5. He associates the policy with users: "ipa-policy-add-user corporateBackground --groups=ipausers" How should the policy be applied? On the workstation, on startup, an ipa related utility should check if there are any policies related to the workstation, if there are any it should call a helper capable of applying a specific type of policy. Then on user logon another ipa related utility should check if there are any policies associated with the user and call the appropriate helper, if available. For the policy created in the above example, on logon the ipa policy utility would find that there is a policy of type dconf associated with the user. It would check if there is a dconf policy helper installed and if positive it would call the helper passing it the parameters defined in the policy. Hope this is useful at least as a starting point in defining desktop policies in IPA. -- Loris Santamaria linux user #70506 xmpp:loris at lgs.com.ve Links Global Services, C.A. http://www.lgs.com.ve Tel: 0286 952.06.87 Cel: 0414 095.00.10 sip:103 at lgs.com.ve ------------------------------------------------------------ "If I'd asked my customers what they wanted, they'd have said a faster horse" - Henry Ford -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6173 bytes Desc: not available URL: From jcholast at redhat.com Tue Feb 26 07:55:02 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Tue, 26 Feb 2013 08:55:02 +0100 Subject: [Freeipa-devel] [PATCHES] 137-144 LDAP code refactoring (Part 3) In-Reply-To: <5123A0CA.5010206@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5107948C.40105@redhat.com> <51237B64.2040600@redhat.com> <5123A0CA.5010206@redhat.com> Message-ID: <512C6A56.6010202@redhat.com> On 19.2.2013 16:56, Petr Viktorin wrote: > On 02/19/2013 02:17 PM, Jan Cholasta wrote: >> On 29.1.2013 10:21, Jan Cholasta wrote: >>> A patch from this patchset (part 3) causes some of the dns plugin tests >>> to fail (idnsallowdynupdate is missing in dnszone_add output). >>> >>> Honza >>> >> >> Patch 143: >> >> + assert isinstance(entry_or_dn, DN) >> + if normalize is None or normalize: >> + entry_or_dn = self.normalize_dn(entry_or_dn) >> + entry_attrs = dict(entry_attrs) >> >> Can you please return LDAPEntry here as well, i.e. replace >> dict(entry_attrs) with self.make_entry(entry_or_dn, entry_attrs)? > > Sure. I tried to keep the old behavior with old usage, but you're right, > using LDAPEntry here will be better. > >> + def delete_entry(self, entry, normalize=None): >> + """Delete entry. >> + >> + The `normalize` argument does nothing when called with a >> LDAPEntry. >> + >> + The legacy variant is: >> + delete_entry(dn, normalize=True) >> + """ >> >> I don't think this is right. We don't need to know any of the attributes >> of an entry to delete it, just its DN. I think we should keep the DN >> variant of delete_entry as the primary one. >> > > Makes sense. I made them both primary. > I didn't bother documenting normalize since your patch will remove that. > > > Updated patch attached; I'll update my repo when I respond to your > comments on part 4. > ACK part 3. Honza -- Jan Cholasta From mkosek at redhat.com Tue Feb 26 08:44:09 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 26 Feb 2013 09:44:09 +0100 Subject: [Freeipa-devel] [PATCHES 0024-0025] Improvements to idrange.py In-Reply-To: <512B642C.3080609@redhat.com> References: <50D444E1.9050502@redhat.com> <5124A36C.70501@redhat.com> <20130220132403.GN476@redhat.com> <5124DB7F.3020101@redhat.com> <51261FB3.8030005@redhat.com> <51277A41.4060902@redhat.com> <5127901F.4040408@redhat.com> <512B642C.3080609@redhat.com> Message-ID: <512C75D9.3090907@redhat.com> On 02/25/2013 02:16 PM, Tomas Babej wrote: > On Fri 22 Feb 2013 04:34:55 PM CET, Martin Kosek wrote: >> On 02/22/2013 03:01 PM, Tomas Babej wrote: >>> On 02/21/2013 02:22 PM, Martin Kosek wrote: >>>> On 02/20/2013 03:19 PM, Tomas Babej wrote: >>>>> On Wed 20 Feb 2013 02:24:03 PM CET, Alexander Bokovoy wrote: >>>>>> On Wed, 20 Feb 2013, Tomas Babej wrote: >>>>>>> On 12/21/2012 12:15 PM, Tomas Babej wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Sending updated and rebased versions of patches 0024 and 0025. >>>>>>>> >>>>>>>> Tomas >>>>>>>> >>>>>>>> >>>>>>> Sending rebased version, these got quite rotten. >>>>>> Thanks for updating them. >>>>>> >>>>>>> @@ -504,25 +515,37 @@ class idrange_mod(LDAPUpdate): >>>>>>> 'not be found. Please specify the SID >>>>>>> directly ' >>>>>>> 'using dom-sid option.')) >>>>>>> >>>>>>> - try: >>>>>>> - (old_dn, old_attrs) = ldap.get_entry(dn, >>>>>>> - ['ipabaseid', >>>>>>> - 'ipaidrangesize', >>>>>>> - 'ipabaserid', >>>>>>> - 'ipasecondarybaserid']) >>>>>>> - except errors.NotFound: >>>>>>> - self.obj.handle_not_found(*keys) >>>>>>> + if in_updated_attrs('ipanttrusteddomainsid'): >>>>>>> + if in_updated_attrs('ipasecondarybaserid'): >>>>>>> + raise errors.ValidationError(name='ID Range setup', >>>>>>> + error=_('Options dom_sid and secondary_rid_base >>>>>>> cannot ' >>>>>>> + 'be used together')) >>>>>> Since we agreed to refer to options by their CLI name (--dom-sid and >>>>>> --secondary-rid-base) in the other patch, it makes sense to use it >>>>>> here too. >>>>>> >>>>>> >>>>>>> - if is_set('ipanttrusteddomainsid'): >>>>>>> - # Validate SID as the one of trusted domains >>>>>>> - >>>>>>> self.obj.validate_trusted_domain_sid(entry_attrs['ipanttrusteddomainsid']) >>>>>>> >>>>>>> + if not in_updated_attrs('ipabaserid'): >>>>>>> + raise errors.ValidationError(name='ID Range setup', >>>>>>> + error=_('Options dom_sid and rid_base must ' >>>>>>> + 'be used together')) >>>>>> Same here. >>>>>> >>>>>>> + # secondary base rid must be set if and only if base rid >>>>>>> is set >>>>>>> + if in_updated_attrs('ipasecondarybaserid') !=\ >>>>>>> + in_updated_attrs('ipabaserid'): >>>>>>> + raise errors.ValidationError(name='ID Range setup', >>>>>>> + error=_('Options secondary_rid_base and rid_base >>>>>>> must ' >>>>>>> + 'be used together')) >>>>>> Same here. >>>>>> >>>>>>> + dict( >>>>>>> + desc='Try to modify ID range %r so it has only primary >>>>>>> rid range set' % (testrange8), >>>>>>> + command=('idrange_mod', [testrange8], >>>>>>> + dict(ipabaserid=testrange8_base_rid)), >>>>>>> + expected=errors.ValidationError( >>>>>>> + name='ID Range setup', error='Options >>>>>>> secondary_rid_base and rid_base must be used together'), >>>>>>> + ), >>>>>> And synchronize error message here too. >>>>>> >>>>> Thanks! >>>>> >>>>> Sending the updated patch 0024. >>>>> >>>>> Tomas >>>>> >>>> In patch 0024 your intention is OK, but the checking functions are not: >>>> >>>> is_set = lambda x: (x in entry_attrs) and (x is not None) >>>> + in_updated_attrs = lambda x: any((x in attrs and x is not None) >>>> + for attrs in (entry_attrs, >>>> old_attrs)) >>>> >>>> They return True even when the attribute is None because they check if *x* is >>>> None and not if *attrs[x]* is None. Example: >>>> >>>> # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 >>>> --secondary-rid-base=1000000 local_range >>>> ---------------------------- >>>> Added ID range "local_range" >>>> ---------------------------- >>>> Range name: local_range >>>> First Posix ID of the range: 1200000 >>>> Number of IDs in the range: 200000 >>>> First RID of the corresponding RID range: 1000 >>>> First RID of the secondary RID range: 1000000 >>>> Range type: local domain range >>>> >>>> This command should be NOOP, but is not: >>>> >>>> # ipa idrange-mod local_range --dom-sid= >>>> ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base >>>> cannot be used together >>>> >>>> Martin >>> Thanks for the catch, all checking functions fixed. >>> >>> Sending the complete patchset, up to date. >>> >>> Tomas >> >> I think I am being a nitpicker now (sorry), but this condition also fails if >> the old_attrs has a setting, but I am removing it in a current -mod command: >> >> # ipa idrange-add --base-id=1200000 --range-size=200000 --rid-base=1000 >> --secondary-rid-base=1000000 local_range >> ---------------------------- >> Added ID range "local_range" >> ---------------------------- >> Range name: local_range >> First Posix ID of the range: 1200000 >> Number of IDs in the range: 200000 >> First RID of the corresponding RID range: 1000 >> First RID of the secondary RID range: 1000000 >> Range type: local domain range >> # ipa idrange-mod local_range --dom-sid S-1-2 --secondary-rid-base= >> ipa: ERROR: invalid 'ID Range setup': Options dom-sid and secondary-rid-base >> cannot be used together >> >> Martin > > Yep. Ok, the command should now handle this use case as well. > > Tomas Thanks, it is working fine now. ACK. Pushed to master, ipa-3-1. Martin From mkosek at redhat.com Tue Feb 26 08:54:10 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 26 Feb 2013 09:54:10 +0100 Subject: [Freeipa-devel] [PATCH] 1087 Some missing v3 schema on upgrades In-Reply-To: <5127666A.3090406@redhat.com> References: <5122965B.3030909@redhat.com> <5127666A.3090406@redhat.com> Message-ID: <512C7832.1060409@redhat.com> On 02/22/2013 01:36 PM, Martin Kosek wrote: > On 02/18/2013 10:00 PM, Rob Crittenden wrote: >> An objectclass and attribute are not being added on upgrades. Missing these >> causes the UI to not work. >> >> I also noticed a typo in the ordering of a number of the trust attributes so >> fix those as well. >> >> rob > > ACK, works for me. Pushed to master, ipa-3-1. > > Martin > I just noticed an issue with the ORDERING in dirsrv errors log: [26/Feb/2013:09:47:37 -0500] attr_syntax_create - Error: the ORDERING matching rule [caseIgnoreIA5OrderingMatch] is not compatible with the syntax [1.3.6.1.4.1.1466.115.121.1.26] for the attribute [ipaNTSecurityIdentifier] [26/Feb/2013:09:47:37 -0500] attr_syntax_create - Error: the ORDERING matching rule [caseIgnoreIA5OrderingMatch] is not compatible with the syntax [1.3.6.1.4.1.1466.115.121.1.26] for the attribute [ipaNTTrustedDomainSID] [26/Feb/2013:09:47:37 -0500] attr_syntax_create - Error: the ORDERING matching rule [caseIgnoreIA5OrderingMatch] is not compatible with the syntax [1.3.6.1.4.1.1466.115.121.1.26] for the attribute [ipaNTDomainGUID] Reopening the ticket. Martin From pviktori at redhat.com Tue Feb 26 10:03:05 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 26 Feb 2013 11:03:05 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <512B93A6.4050602@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> <512387D1.60603@redhat.com> <5124BB8F.80803@redhat.com> <512B93A6.4050602@redhat.com> Message-ID: <512C8859.8050001@redhat.com> On 02/25/2013 05:39 PM, Jan Cholasta wrote: > On 20.2.2013 13:03, Petr Viktorin wrote: >> On 02/19/2013 03:10 PM, Jan Cholasta wrote: >>> On 1.2.2013 15:38, Petr Viktorin wrote: >>>> Alright, I renamed get_single to single_value(). >>>> >>>> I also rebased to current master. >>>> >>> >>> Patch 152: >>> >>> + def single_value(self, name, default=_missing): >>> + values = self.get(name, [default]) >>> + if len(values) != 1: >>> + raise ValueError( >>> + '%s has %s values, one expected' % (name, len(values))) >>> + if values[0] is _missing: >>> + raise KeyError(name) >>> + return values[0] >>> >>> I would prefer if you used __getitem__() instead of get() and re-raised >>> KeyError if default is _missing. Also, until we disallow non-lists, we >>> need to check if values actually is list. I.e., do something like this: >>> >>> + def single_value(self, name, default=_missing): >>> + try: >>> + values = super(LDAPEntry, >>> self).__getitem__(self._get_attr_name(name)) >>> + except KeyError: >>> + if default is _missing: >>> + raise >>> + return default >>> + if not isinstance(values, list): >>> + return values >>> + if len(values) != 1: >>> + raise ValueError( >>> + '%s has %s values, one expected' % (name, len(values))) >>> + return values[0] >> >> Updated, thanks. >> >> _get_attr_name is only added in your patch 99, so I used _attr_name here >> and modified your patch. >> >> I wrote some tests for single_value, and while I was at it I added tests >> for a few other LDAPEntry methods as well. I've also split things up >> into more testcases. Including as patch 0181. > > Thanks. I think you should also add a tearDown method to test_LDAPEntry > which disconnects self.conn if it is connected (the same thing test_ldap > does). Thanks for the catch, added. >> >>> Patch 159: >>> >>> Like I said in my review of your patch 143, I think we should use DNs >>> instead of entries in delete_entry, so I think it would make sense to do >>> delete_entry(entry.dn) in this patch. >> >> I think that for symmetry with add_entry and update_entry, delete_entry >> should take entries too. If you already have the entry, having to type >> the extra ".dn" is not very intuitive. >> The proper thing to do would be a separate delete_by_dn(), but >> delete_entry() taking either entries or DNs works fine IMO. > > OK, makes sense. > >> >>> Patch 160: >>> >>> I think you should do this (replace % with ,): >>> >>> + root_logger.debug( >>> + "failed to find mapping tree entry for %s", >>> self.suffix) >>> >> >> Fixed, thanks. >> >> I've also rebased 174 and your patch 104 to current master. >> > > Honza > -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0181-02-Improve-LDAPEntry-tests.patch Type: text/x-patch Size: 5065 bytes Desc: not available URL: From sbose at redhat.com Tue Feb 26 13:15:00 2013 From: sbose at redhat.com (Sumit Bose) Date: Tue, 26 Feb 2013 14:15:00 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512B7143.6010204@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> Message-ID: <20130226131500.GB3633@localhost.localdomain> On Mon, Feb 25, 2013 at 03:12:19PM +0100, Martin Kosek wrote: > On 02/25/2013 03:09 PM, Rob Crittenden wrote: > > Martin Kosek wrote: > ... > >> 4) What does "NOTE: We will need to be clear that this range has nothing to do > >> with Trust ranges." actually mean? AFAIU, IPA should have all local ranges > >> covered with a local "idrange" range(s). > > > > IPA ranges is completely separate from DNA ranges. You can set/modify all the > > local ranges you want and it won't affect the UIDs getting assigned. > > > >> If it does not have it covered, it could happen that for example a new trust > >> would overlap with this user-defined local range and we would have colliding > >> POSIX IDs... > > > > Hmm, that's a good point. > > > >> IMO, dnarange-set and dnanextrange-set should at first check if the range is > >> covered with some local idrange and only then allowed setting the new range. > > > > I can do that as well, but again, the local ranges don't really affect the ids > > we hand out via DNA. > > > > rob > > You are right, that DNA plugin is really not aware of the idranges we set in > IPA. But the idrange is still a safeguard for our POSIX IDs to not overlap with > trust ranges and I think we should respect that with ipa-replica-manage. > > I wonder if there was not even a plan to increase cooperation between our > idranges and DNA plugin, maybe Sumit or Alexander knows more. If I understand the use case and design properly, it is about re-arranging the sub-ranges each replica can use from the original range, which was given/created at installation time and which is also stored as idrange in DOMAIN.NAME_id_range,cn=ranges,cn=etc,dc=domain,dc=name with objectclass=ipaDomainIDRange. If the re-arrangement does not result in IDs which are outside of this range give by the ipaDomainIDRange object, no conflicts with idranges used by trusted domain will occur, because it is one of the task of the idrange objects to avoid those conflicts. If the original given range is exhausted completely and a completely new DNA range must be created, it should be checked with ipa idrange-find that the new range is not used and a new ipaDomainIDRange object which reserves the local range should be added. There are https://fedorahosted.org/freeipa/ticket/591 which can be used to track the coordinated creation of DNA and id-range. bye, Sumit > > Martin From pviktori at redhat.com Tue Feb 26 14:50:49 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 26 Feb 2013 15:50:49 +0100 Subject: [Freeipa-devel] [PATCH] 0189 Remove option to use custom SSL certificates from ipa-server-install Message-ID: <512CCBC9.2070407@redhat.com> This removes the --{dirsrv,http,pkinit}-{pkcs12,pin} options. https://fedorahosted.org/freeipa/ticket/3151 The same options are in ipa-replica-prepare. I think we should leave those be, so people with existing servers with custom certs can install replicas. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0189-Remove-option-to-use-custom-SSL-certificates-from-ip.patch Type: text/x-patch Size: 9637 bytes Desc: not available URL: From mkosek at redhat.com Tue Feb 26 17:03:00 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 26 Feb 2013 18:03:00 +0100 Subject: [Freeipa-devel] [PATCH] 374 Remove ORDERING for IA5 attributeTypes Message-ID: <512CEAC4.5060403@redhat.com> IA5 string syntax does not have a compatible ORDERING matching rule. Simply use default ORDERING for these attributeTypes as we already do in other cases. https://fedorahosted.org/freeipa/ticket/3398 ----- This is a follow-up ticket for regression introduced by Rob's 1087. I did not add any update rules for IPA servers that were already wrongly updated as we did not release any IPA version with these bad attributeTypes and I want to keep .update files small. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-374-remove-ordering-for-ia5-attributetypes.patch Type: text/x-patch Size: 13174 bytes Desc: not available URL: From mkosek at redhat.com Tue Feb 26 17:20:34 2013 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 26 Feb 2013 18:20:34 +0100 Subject: [Freeipa-devel] [PATCH] 0187 CLI: Do interactive prompting after a context is created In-Reply-To: <512767E0.7020700@redhat.com> References: <512767E0.7020700@redhat.com> Message-ID: <512CEEE2.4000209@redhat.com> On 02/22/2013 01:43 PM, Petr Viktorin wrote: > Hello, > This fixes a regression introduced by one of my help patches (abe26d5). > > https://fedorahosted.org/freeipa/ticket/3453 > ACK. Pushed to master, ipa-3-1. Martin From dpal at redhat.com Tue Feb 26 20:11:24 2013 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 26 Feb 2013 15:11:24 -0500 Subject: [Freeipa-devel] What about desktop policies? In-Reply-To: <1361819744.2111.48.camel@toron.pzo.lgs.com.ve> References: <1361819744.2111.48.camel@toron.pzo.lgs.com.ve> Message-ID: <512D16EC.7090102@redhat.com> On 02/25/2013 02:15 PM, Loris Santamaria wrote: > Hi all, > > some customers of ours are interested in managing desktop policies for > their linux workstations, really nothing fancy, corporate background and > proxy settings are the most common requests. > > In the past I created Gnome desktop profiles using Sabayon, distributed > them using puppet and associated them to user accounts with a Sabayon > specific LDAP attribute, a process a bit convoluted, and no longer > possible since sabayon is no longer developed. Also it was really buggy, > and very gnome specific. > > I was thinking in how integrate desktop policies in freeIPA in a general > manner and I wanted to share my ideas with you. Hopefully some of this > may be incorporated in IPA at some point in the future. > > Properties of a "policy": > > * is a collection of "settings" > * can be associated with users or groups (desktop policy) or with > hosts or hostgroups (system policy) > * is associated with a "consumer", the client software that > interprets and applies the policy. This way one could define > policies for dconf, policies for kde, policies for WBEM. > > Properties of a "setting" > * is a key-value pair > * must conform to a "schema" > * may be mandatory > > The schema: > * indicates which attributes a policy may consist of > * indicates which kind of value may take an attribute. Bool, > string, etc. > * There may be more than one schema for a given "consumer". For > example for dconf you may have an evolution schema, a > gnome-games schema, etc. > > Sample policy creation process: > 1. The admin creates a new schema in IPA, with a command like "ipa > schema-add --consumer=dconf gnomeSettingsSchema" > 2. The admin adds some definition to the schema: "ipa > schema-add-setting gnomeSettingsSchema > --name=/schemas/desktop/gnome/background/picture_filename > --type=string --description='File to use for the background > image.'" > 3. He creates a new policy: "ipa policy-add corporateBackground > --type=desktop --consumer=dconf > 4. He adds a setting to the policy: "ipa policy-add-setting > corporateBackground > --name=/schemas/desktop/gnome/background/picture_filename > --value=file:///san/wp/wallpaper.jpg --mandatory". Ipa would > check that the key is defined in one of the dconf related > schemas and the value is acceptable for that key. > 5. He associates the policy with users: "ipa-policy-add-user > corporateBackground --groups=ipausers" > > How should the policy be applied? On the workstation, on startup, an ipa > related utility should check if there are any policies related to the > workstation, if there are any it should call a helper capable of > applying a specific type of policy. Then on user logon another ipa > related utility should check if there are any policies associated with > the user and call the appropriate helper, if available. > > For the policy created in the above example, on logon the ipa policy > utility would find that there is a policy of type dconf associated with > the user. It would check if there is a dconf policy helper installed and > if positive it would call the helper passing it the parameters defined > in the policy. > > Hope this is useful at least as a starting point in defining desktop > policies in IPA. This is great! Thank you for sharing some ideas. We sort of stayed away from centralized policy management for quite some time. Originally we thought that IPA will do policy management and did a lot of design around it. However at some point we realized that there is an overlap with the system management and content management for which things like puppet are more suitable. We said then that IdM would focus on managing identity related policies that are traditionally served via LDAP. The things that you are talking about resemble to some extent MSFT GPO and we felt that IdM might not be the right place for it. May be it is time to reassess it. I would however not go that route at least yet. If Desktop can read additional properties related to user (background, default language, etc.) from SSSD over a DBUS interface the SSSD should be able to pull this data from the IdM (eventually). On the IdM we probably can make these additional attributes available in the user entries using class of service like we do with password policies. We have plans for SSSD to handle more attributes than posix and integrate with Desktop. https://fedorahosted.org/sssd/wiki/DesignDocs/AccountsService IMO once this work is started we would be able to see how we can configure and serve more data from IPA for clients to consume. Meanwhile I suggest you create a ticket in IPA trac and put your ideas there. > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From edewata at redhat.com Wed Feb 27 00:50:13 2013 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 26 Feb 2013 19:50:13 -0500 (EST) Subject: [Freeipa-devel] [PATCH] 259 Combobox keyboard support In-Reply-To: <51225771.3040809@redhat.com> Message-ID: <940957870.1823903.1361926213062.JavaMail.root@redhat.com> Looks good. The patch already provides improvements, so feel free to push. > * when CB is non-editable, user can start typing, first character > will open list, second will be entered into search input. Note: I > wanted to copy the first char to the search box as well, but I did not > figure out reliable method for converting keycode to char for non ASCII > keyboard layouts Could we use keypress() for this instead of keydown()? See http://api.jquery.com/keypress/. The keypress() should return the actual character, whereas the keydown() would return the code of the keyboard key so it wouldn't be able to distinguish between 'a' and 'A', which is not what we want. Another minor thing, if the search box is in focus, you can use the Up/Down arrow to go to the list. However, from the list you cannot use the Up/Down arrow to go back to the search box, you'd have to use the Tab key. It's not really a problem, but it's not that consistent. -- Endi S. Dewata From mkosek at redhat.com Wed Feb 27 09:28:06 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 27 Feb 2013 10:28:06 +0100 Subject: [Freeipa-devel] [PATCHES 0031-0032] Improve HBAC rule handling in selinuxusermap-add/mod/find In-Reply-To: <5124B3FE.7000407@redhat.com> References: <511294EB.9050805@redhat.com> <5112A797.805@redhat.com> <511519C4.30907@redhat.com> <5123EFC7.9020708@redhat.com> <5124B3FE.7000407@redhat.com> Message-ID: <512DD1A6.3070504@redhat.com> On 02/20/2013 12:31 PM, Tomas Babej wrote: > On 02/19/2013 10:33 PM, Rob Crittenden wrote: >> Tomas Babej wrote: >>> On 02/06/2013 07:57 PM, Rob Crittenden wrote: >>>> Tomas Babej wrote: >>>>> Hi, >>>>> >>>>> this pair of patches improves HBAC rule handling in selinuxusermap >>>>> commands. >>>>> >>>>> Patch 0031 deals with: >>>>> https://fedorahosted.org/freeipa/ticket/3349 >>>>> >>>>> Patch 0032 takes care of: >>>>> https://fedorahosted.org/freeipa/ticket/3348 >>>>> >>>>> and is to be applied on top of Patch 0031. >>>>> >>>>> See commit messages for detailed info. >>>>> >>>>> Tomas >>>>> >>>> >>>> ACK for patch 0032. >>>> >>>> For patch 0031 we can't change the data type of an existing attribute. >>>> It will break backwards compatibility. Can you test with an older >>>> client to see if it cares (it may not care about the name of the >>>> type). If older clients will work then this is probably ok. >>>> >>>> I gather that seealso detected as a DN attribute and converted into a >>>> DN class and this is blowing up the Str validator? >>>> >>> Yes, that was exactly the case. >>>> rob >>> >>> I added a workaround for older client versions, tested it with >>> freeipa-client/admintools 2.2, works as expeceted. >>> However, this only should be issue if there is older admintools package >>> on the client than on the server. >>> >>> Outline is such as follows: I added a new flag for DNParam seelalso >>> attribute, called 'allow_malformed' that allows any string to be passed >>> to DNParam. Its value gets wrapped in 'malformed=yes,value='. >>> This allows to parse out the string in selinuxusermap-add/mod >>> pre_callback out of the DN and search for the rule with such name so >>> that it's DN gets in LDAP instead. >>> >>> Updated patch attached. >>> >>> Tomas >> >> I like where you're going with this, just a couple of comments: >> >> 1. Should we come up with a more universal name for allow_malformed? Is this >> something that we should allow at a higher level? I was thinking allow_raw, >> or allow_non_dn, or something like that. > > To me, allow_non_dn sounds is just as specific as allow_malformed, > they both refer to DN specifically. > > I'd go with allow_raw, if need for such pattern may eventually arise for > other parameter classes than DNParam. > > What do you mean by higher level, turning this hack into a feature > Param class? I don't see how this would work, each parameter > class that implements its own type validation as DNParam needs > to override _convert_scalar(). And in every such class we would need > to wrap our raw value so that it is represented in the type of this parameter, > as we do with DN(('malformed','yes'),('value',value)) now. > > Maybe we could skip type validation in _convert_scalar default > implementation or catch the error raised somehow, and let the type be > invalid, but I'm not aware of the consenquences. I would need to investigate. > Wouldn't it cause failure deeper in the framework? > > Or did you by higher level mean simply picking a more general name for the > flag so it can be reused in other parameter classes with the same name? >> >> 2. I think that if a bad dn is passed in as a Str the conversion into a DN >> won't be handled: >> >> + if 'allow_malformed' in self.flags: >> + dn = DN(('malformed','yes'),('value',value)) >> >> Should this be wrapped in a try/except to raise a ConversionError if it fails? > Yes, thanks for that catch. >> >> rob > Tomas Is it just me, or does the 0031 look overengineered? I think this is a general problem for each Str parameter which we then process/convert to DN in our pre_callbacks. selinuxusermap is one example where this does not work. This fix leaves other examples not working: # ipa trustconfig-mod --setattr "ipantfallbackprimarygroup=cn=Default SMB Group,cn=groups,cn=accounts,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com" ipa: ERROR: invalid 'ipantfallbackprimarygroup': must be Unicode text I would rather propose to not automatically encode DN of known attributes set by *attr: diff --git a/ipalib/plugins/baseldap.py b/ipalib/plugins/baseldap.py index 1ebbe7a..e4b9834 100644 --- a/ipalib/plugins/baseldap.py +++ b/ipalib/plugins/baseldap.py @@ -768,12 +768,6 @@ last, after all sets and adds."""), # None means "delete this attribute" value = None - if ldap.has_dn_syntax(attr): - try: - value = DN(value) - except ValueError: - raise errors.InvalidSyntax(attr=attr) - if attr in newdict: if type(value) in (tuple,): newdict[attr] += list(value) I think this conversion is just done too early as this Str param is processed and converted later in the pre_callback, when needed. The code above introduced inconsistent processing of IPA attributes with DN syntax coming from regular option and from *attr option - Str When I did this change, both selinuxusermap-mod and trustconfig-mod started working: # ipa selinuxusermap-mod foo --setattr=seealso=ipaUniqueID=70e42636-75db-11e2-9df6-001a4a104edc,cn=hbac,dc=rhel64,dc=ad,dc=test ------------------------------- Modified SELinux User Map "foo" ------------------------------- Rule name: foo SELinux User: unconfined_u:s0-s0:c0.c1023 HBAC Rule: allow_all Enabled: TRUE # ipa selinuxusermap-mod foo --setattr=seealso=allow_all ipa: ERROR: no modifications to be performed # ipa selinuxusermap-mod foo --hbacrule=allow_all ipa: ERROR: no modifications to be performed You would just need to investigate if this change would not have other consequences. Martin From mkosek at redhat.com Wed Feb 27 09:58:20 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 27 Feb 2013 10:58:20 +0100 Subject: [Freeipa-devel] [PATCH] 0006 Remove check for alphabetic only characters from domain name validation In-Reply-To: <5127889F.9020202@redhat.com> References: <511E44A9.9090509@redhat.com> <51221027.7050803@redhat.com> <20130218114750.GC27418@localhost.localdomain> <512219D7.9000909@redhat.com> <51249F6E.7030005@redhat.com> <51273810.9060800@redhat.com> <5127889F.9020202@redhat.com> Message-ID: <512DD8BC.2080904@redhat.com> On 02/22/2013 04:02 PM, Ana Krivokapic wrote: > On 02/22/2013 10:19 AM, Petr Spacek wrote: >> On 20.2.2013 11:03, Ana Krivokapic wrote: >>> On 02/18/2013 01:08 PM, Martin Kosek wrote: >>>> On 02/18/2013 12:47 PM, Sumit Bose wrote: >>>>> On Mon, Feb 18, 2013 at 12:27:35PM +0100, Petr Spacek wrote: >>>>>> On 15.2.2013 15:22, Ana Krivokapic wrote: >>>>>>> Hello, >>>>>>> >>>>>>> The .isalpha() check in validate_domain_name() was too strict, >>>>>>> causing some commands like ipa dnsrecord-add to fail. >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/3385 >>>>>> I would add --force option rather than removing whole check, if >>>>>> it's possible. >>>>>> >>>>>> Would it be possible to mention RFC in the error message? Something >>>>>> like _('top level domain label must be alphabetic (RFC 1123 section >>>>>> 2.1)') >>>>>> ? >>>>>> >>>>>> IMHO it is handy, because it educates users. >>>>> The problem is that this check is always done on the last component of >>>>> the domain_name even if it is just a sub-domain of the FreeIPA domain, >>>>> where e.g. numbers are valid characters. >>>>> >>>>> At the beginning of validate_domain_name() a trailing '.' is stripped >>>>> away. iirc the trailing '.' is an indication for a complete, fully >>>>> qualified name. Would it work if the presence of the trailing '.' is >>>>> saved and the check is only done if there was a '.'? >>>>> >>>>> bye, >>>>> Sumit >>>>> >>>> Sure. Though I am now not 100% sure that some IPA functions do not >>>> use this >>>> validator with a fqdn hostname without trailing dot. If not, I am >>>> for fixing >>>> this function as Sumit and Petr suggested. >>>> >>>> Martin >>>> >>>> _______________________________________________ >>>> Freeipa-devel mailing list >>>> Freeipa-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/freeipa-devel >>> After some thought, I decided to change the approach. >>> >>> As pointed out by Sumit, the problem was that the validate_domain_name() >>> function did not distinguish between fqdn and non-fqdn domains >>> (subdomains of the IPA domain). The trailing dot is not a clear >>> indication either, because some IPA functions use this validator with an >>> fqdn without the trailing dot. >>> >>> To fix this, I introduced an additional parameter to this function - a >>> flag which indicates whether the domain name is an fqdn or not. The is >>> .isalpha() check is then performed only in the case of an fqdn. >>> >>> I also improved the error message to mention the relevant RFC, as >>> suggested by Petr. >> >> Please don't forget to add --force switch. It could be handy. >> > I added the --force switch to ipa dnsrecord-add and opened a new ticket > to handle the rest of the ipa commands that use domain name validation: > https://fedorahosted.org/freeipa/ticket/3455 > > Updated patch is attached. > This patch fixed validation only partially. The --force flag you made available will not allow admin to for example add a zone "example.zone1" which technically will be resolvable, it is just not a good practice: # ipa dnszone-add example.zone1 --name-server `hostname`. --force ipa: ERROR: invalid 'name': top level domain label must be alphabetic (RFC 1123 section 2.1) To enable this, I think you would need to not postpone the validation to DNS zone pre_callback as you could not check --force flag presence right in the idnsname parameter validator. We may also want to change --force flag label, it now talks only about NS record validation, but we now expanded it a bit, so the label would need to be more general. Martin From pviktori at redhat.com Wed Feb 27 10:58:21 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 27 Feb 2013 11:58:21 +0100 Subject: [Freeipa-devel] [PATCH] 374 Remove ORDERING for IA5 attributeTypes In-Reply-To: <512CEAC4.5060403@redhat.com> References: <512CEAC4.5060403@redhat.com> Message-ID: <512DE6CD.8010901@redhat.com> On 02/26/2013 06:03 PM, Martin Kosek wrote: > IA5 string syntax does not have a compatible ORDERING matching rule. > Simply use default ORDERING for these attributeTypes as we already > do in other cases. > > https://fedorahosted.org/freeipa/ticket/3398 > > ----- > > This is a follow-up ticket for regression introduced by Rob's 1087. I did not > add any update rules for IPA servers that were already wrongly updated as we > did not release any IPA version with these bad attributeTypes and I want to > keep .update files small. > > Martin ACK -- Petr? From sbose at redhat.com Wed Feb 27 10:58:31 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 11:58:31 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <512B84B8.7080506@redhat.com> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> Message-ID: <20130227105831.GC3633@localhost.localdomain> On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: > On 02/21/2013 04:24 PM, Sumit Bose wrote: > > Hi, > > > > this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 > > The related design page is > > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > > > It was agreed while discussing the design page that the currently > > hardcoded value for the NFS service will be dropped completely, hence > > the first patch reverts to patch which added the hardcoded value. > > > > The second patch adds the needed attribute to compensate the now missing > > hardcoded value. > > > > In the following three patches the PAC type is read and used > > accordingly. The last patch adds a unit test for a new function. > > > > bye, > > Sumit > > I did only sanity testing to the C part of the RFE so far, but by reading it it > looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part > is OK. Alexander asked in irc to change strcmp() to strcasecmp() so that options like 'Ms-Pac' or 'none' are accepted as well. > > I will comment on the Python parts: > > 1) Your update check testing if there is any NFS service with ipakrbauthzdata > looks like a good heuristics to prevent admin frustration. Though an ideal > solution would require > https://fedorahosted.org/freeipa/ticket/2961 > to prevent multiple updates when admin purposefully removes this attribute. We > may want to reference the ticket in a comment in the update plugin... I added a comment in the code and in #2961. > > > 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns > truncated result. As you do not update entries directly but only return update > instructions when you have no truncated results, you will loop. > > To simulate this, I just created 2 NFS principals and set size_limit=1 in your > find_entries call. Thanks for catching this, I removed the truncated logic and added a messages if truncated was set. > > > 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS > service principals added by service-add? Shouldn't we set ipakrbauthzdata for > new services too? As a default when no user ipakrhbauthzdata is set... > Otherwise admins could be surprised why their new NFS services do not work > while the others do. > > Also, I think we should have this NFS culprit documented somewhere (i.e. why we > set ipakrbauthzdata to NONE by default). At least as a small section in the > online help for services plugin... I added comments to the help and the doc string in patch #100. If you think it is necessary to implicitly set PAC type to 'NONE' if NFS services are added and no type is given explicitly, I would prefer if you can open a new tickets so that it can be discussed independently. Thank you for the review. New versions attached. bye, Sumit > > Martin -------------- next part -------------- From 8eb34dd3190853bd3fa400434be4f7c720bf1354 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 09:59:00 +0100 Subject: [PATCH 094/100] Revert "MS-PAC: Special case NFS services" This reverts commit 5269458f552380759c86018cd1f30b64761be92e. With the implementation of https://fedorahosted.org/freeipa/ticket/2960 a special hardcoded handling of NFS service tickets is not needed anymore. --- daemons/ipa-kdb/ipa_kdb_mspac.c | 36 +----------------------------------- 1 file changed, 1 insertion(+), 35 deletions(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index 950000349702038e1d55bd257816f7f61e9557a5..f05c552fc18eb9b15ee3e39b513184589bc8c468 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -743,24 +743,6 @@ static bool is_cross_realm_krbtgt(krb5_const_principal princ) return true; } -static bool is_service_of_type(krb5_const_principal princ, const char *type) -{ - size_t len; - - if (princ->length < 2) { - return false; - } - - len = strlen(type); - - if ((princ->data[0].length == len) || - (strncasecmp(princ->data[0].data, type, len) == 0)) { - return true; - } - - return false; -} - static char *gen_sid_string(TALLOC_CTX *memctx, struct dom_sid *dom_sid, uint32_t rid) { @@ -1555,7 +1537,6 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, krb5_error_code kerr; krb5_pac pac = NULL; krb5_data pac_data; - bool is_nfs = false; /* When using s4u2proxy client_princ actually refers to the proxied user * while client->princ to the proxy service asking for the TGS on behalf @@ -1566,32 +1547,17 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, ks_client_princ = client->princ; } - /* NFS Server on Linux is limited and will choke on big tickets. - * So avoid attachnig the PAC to nfs/ tickets for now. - * FIXME: remove this when we have interface to support disabling - * PACs on arbitrary services */ - if (is_service_of_type(ks_client_princ, "nfs") || - is_service_of_type(server->princ, "nfs")) { - is_nfs = true; - } - is_as_req = ((flags & KRB5_KDB_FLAG_CLIENT_REFERRALS_ONLY) != 0); if (is_as_req && (flags & KRB5_KDB_FLAG_INCLUDE_PAC)) { - if (is_nfs) { - *signed_auth_data = NULL; - kerr = 0; - goto done; - } - kerr = ipadb_get_pac(context, client, &pac); if (kerr != 0 && kerr != ENOENT) { goto done; } } - if (!is_as_req & !is_nfs) { + if (!is_as_req) { /* find the existing PAC, if present */ kerr = krb5_find_authdata(context, tgt_auth_data, NULL, KRB5_AUTHDATA_WIN2K_PAC, &pac_auth_data); -- 1.8.1.2 -------------- next part -------------- From 6eaa83868c1c80c8eeac5029176cb6d1b3ce03a6 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Thu, 21 Feb 2013 14:27:12 +0100 Subject: [PATCH 095/100] Add ipakrbauthzdata to NFS services during upgrades This update became necessary because instead of hardcoding a value of 'NONE' for ipakrbauthzdata for all NFS services ipakrbauthzdata can now be specified per service. To avoid issues during upgrades ipakrbauthzdata must be set to NONE for all NFS services where this attribute isn't set, because an unset attribute means using the default which might be different from 'NONE'. Another issue might arise on systems where this update is already run and for some NFS services the ipakrbauthzdata attribute is removed manually on purpose. Another update might set it back to 'NONE' which might be irritating. For this reason this plugin only update entries if none of the existing NFS service entries has ipakrbauthzdata set. --- ipaserver/install/plugins/update_services.py | 82 +++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/ipaserver/install/plugins/update_services.py b/ipaserver/install/plugins/update_services.py index c384af52fb91692c7f339c5fb2da92ae45984482..22c847767983945354bee5dfa4f246c4ef720e4b 100644 --- a/ipaserver/install/plugins/update_services.py +++ b/ipaserver/install/plugins/update_services.py @@ -17,7 +17,7 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -from ipaserver.install.plugins import MIDDLE +from ipaserver.install.plugins import MIDDLE, LAST from ipaserver.install.plugins.baseupdate import PostUpdate from ipalib import api, errors from ipapython.dn import DN @@ -93,3 +93,83 @@ class update_service_principalalias(PostUpdate): return (False, False, []) api.register(update_service_principalalias) + +class update_service_nfs_ipakrbauthzdata(PostUpdate): + """ + Update all NFS services which do not have ipakrbauthzdata attribute set. + + Since this plugin uses ipakrbprincipalalias update_service_principalalias + must be run before, 'order' is set accordingly. + + This update became necessary because instead of hardcoding a value of + 'NONE' for ipakrbauthzdata for all NFS services ipakrbauthzdata can now be + specified per service. To avoid issues during upgrades ipakrbauthzdata + must be set to NONE for all NFS services where this attribute isn't set, + because an unset attribute means using the default which might be + different from 'NONE'. + + Another issue might arise on systems where this update is already run and + for some NFS services the ipakrbauthzdata attribute is removed manually on + purpose. Another update might it set back to 'NONE' which might be + irritating. For this reason this plugin only update entries if none of the + existing NFS service entries has ipakrbauthzdata set. If a solution for + https://fedorahosted.org/freeipa/ticket/2961 is implemented this heuristic + should be dropped. + """ + order = LAST + + def execute(self, **options): + ldap = self.obj.backend + + base_dn = DN(api.env.container_service, api.env.basedn) + search_filter = ("(&(objectclass=ipakrbprincipal)" + "(objectclass=ipaservice)" + "(ipakrbprincipalalias=nfs*))") + root_logger.debug("update_service_nfs_ipakrbauthzdata: search for " + "affected services") + + try: + (entries, truncated) = ldap.find_entries(search_filter, + ['objectclass', 'ipakrbauthzdata'], base_dn, + time_limit=0, size_limit=0) + except errors.NotFound: + root_logger.debug("update_service_nfs_ipakrbauthzdata: " + "no NFS service to update found") + return (False, False, []) + except errors.ExecutionError, err: + root_logger.error("update_service_nfs_ipakrbauthzdata: cannot " + "retrieve list of NFS services: %s", err) + return (False, False, []) + if not entries: + root_logger.debug("update_service_nfs_ipakrbauthzdata: " + "no NFS service was returned") + return (False, False, []) + root_logger.debug("update_service_nfs_ipakrbauthzdata: found %d " + "NFS services, truncated: %s", + len(entries), truncated) + + for nfs_dn, entry in entries: + if entry.has_key('ipakrbauthzdata'): + root_logger.info("update_service_nfs_ipakrbauthzdata: " + "found a NFS service where " + "ipakrbauthzdata is already set, " + "nothing to do") + return (False, False, []) + + nfs_updates = {} + for nfs_dn, entry in entries: + nfs_updates[nfs_dn] = {'dn': nfs_dn, + 'updates': ['add:ipakrbauthzdata:NONE']} + + if not truncated: + root_logger.debug("update_service_nfs_ipakrbauthzdata: " + "all affected services updated") + else: + root_logger.info("search results for NFS services have been " + "truncated by the server; please check manually " + "if the PAC type is set correctly for all NFS " + "services") + + return (False, True, [nfs_updates]) + +api.register(update_service_nfs_ipakrbauthzdata) -- 1.8.1.2 -------------- next part -------------- From 0b154a1f4929062ddec79ace9ca116ade54879d1 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 11:01:11 +0100 Subject: [PATCH 096/100] ipa-kdb: Read global defaul ipaKrbAuthzData The ipaKrbAuthzData LDAP attribute is read from the ipaConfig object and the read value(s) are stored in the ipadb context. --- daemons/ipa-kdb/ipa_kdb.c | 27 ++++++++++++++++++++++++++- daemons/ipa-kdb/ipa_kdb.h | 3 +++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/daemons/ipa-kdb/ipa_kdb.c b/daemons/ipa-kdb/ipa_kdb.c index 2a344dc69158dbc3f6d0207ab0bb781676240ed9..e5c718ea9ffd38dbd49026e825d4a7f920638181 100644 --- a/daemons/ipa-kdb/ipa_kdb.c +++ b/daemons/ipa-kdb/ipa_kdb.c @@ -40,6 +40,8 @@ struct ipadb_context *ipadb_get_context(krb5_context kcontext) static void ipadb_context_free(krb5_context kcontext, struct ipadb_context **ctx) { + size_t c; + if (*ctx != NULL) { free((*ctx)->uri); free((*ctx)->base); @@ -51,6 +53,12 @@ static void ipadb_context_free(krb5_context kcontext, free((*ctx)->supp_encs); ipadb_mspac_struct_free(&(*ctx)->mspac); krb5_free_default_realm(kcontext, (*ctx)->realm); + + for (c = 0; (*ctx)->authz_data && (*ctx)->authz_data[c]; c++) { + free((*ctx)->authz_data[c]); + } + free((*ctx)->authz_data); + free(*ctx); *ctx = NULL; } @@ -167,13 +175,14 @@ done: int ipadb_get_global_configs(struct ipadb_context *ipactx) { - char *attrs[] = { "ipaConfigString", NULL }; + char *attrs[] = { "ipaConfigString", IPA_KRB_AUTHZ_DATA_ATTR, NULL }; struct berval **vals = NULL; LDAPMessage *res = NULL; LDAPMessage *first; char *base = NULL; int i; int ret; + char **authz_data_list; ret = asprintf(&base, "cn=ipaConfig,cn=etc,%s", ipactx->base); if (ret == -1) { @@ -215,6 +224,22 @@ int ipadb_get_global_configs(struct ipadb_context *ipactx) } } + ret = ipadb_ldap_attr_to_strlist(ipactx->lcontext, first, + IPA_KRB_AUTHZ_DATA_ATTR, &authz_data_list); + if (ret != 0 && ret != ENOENT) { + goto done; + } + if (ret == 0) { + if (ipactx->authz_data != NULL) { + for (i = 0; ipactx->authz_data[i]; i++) { + free(ipactx->authz_data[i]); + } + free(ipactx->authz_data); + } + + ipactx->authz_data = authz_data_list; + } + ret = 0; done: diff --git a/daemons/ipa-kdb/ipa_kdb.h b/daemons/ipa-kdb/ipa_kdb.h index f472f02458e040b921b9f3f85bcc36fa954785d5..7b1576124140ae53cf28a1ed47bfa1acf31cdd59 100644 --- a/daemons/ipa-kdb/ipa_kdb.h +++ b/daemons/ipa-kdb/ipa_kdb.h @@ -74,6 +74,8 @@ #define IPA_SETUP "ipa-setup-override-restrictions" +#define IPA_KRB_AUTHZ_DATA_ATTR "ipaKrbAuthzData" + struct ipadb_mspac; struct ipadb_context { @@ -89,6 +91,7 @@ struct ipadb_context { struct ipadb_mspac *mspac; bool disable_last_success; bool disable_lockout; + char **authz_data; }; #define IPA_E_DATA_MAGIC 0x0eda7a -- 1.8.1.2 -------------- next part -------------- From 23d5892f70df57581157ba115016b6debfb0774f Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 09:44:32 +0100 Subject: [PATCH 097/100] ipa-kdb: Read ipaKrbAuthzData with other principal data The ipaKrbAuthzData LDAP attribute is read together with the other data of the requestedprincipal and the read value(s) are stored in the e-data of the entry for later use. --- daemons/ipa-kdb/ipa_kdb.h | 1 + daemons/ipa-kdb/ipa_kdb_principals.c | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/daemons/ipa-kdb/ipa_kdb.h b/daemons/ipa-kdb/ipa_kdb.h index 7b1576124140ae53cf28a1ed47bfa1acf31cdd59..9daaab80dc514dd1bb3e85775c1e284d19dac0cd 100644 --- a/daemons/ipa-kdb/ipa_kdb.h +++ b/daemons/ipa-kdb/ipa_kdb.h @@ -105,6 +105,7 @@ struct ipadb_e_data { char **pw_history; struct ipapwd_policy *pol; time_t last_admin_unlock; + char **authz_data; }; struct ipadb_context *ipadb_get_context(krb5_context kcontext); diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c index bb2074bf9c85b0bdcf228696cf8391c11d2384c4..46a921cb2f267b977df4a0042d81a49beccb5a91 100644 --- a/daemons/ipa-kdb/ipa_kdb_principals.c +++ b/daemons/ipa-kdb/ipa_kdb_principals.c @@ -63,6 +63,7 @@ static char *std_principal_attrs[] = { /* IPA SPECIFIC ATTRIBUTES */ "nsaccountlock", "passwordHistory", + IPA_KRB_AUTHZ_DATA_ATTR, "objectClass", NULL @@ -237,6 +238,7 @@ static krb5_error_code ipadb_parse_ldap_entry(krb5_context kcontext, krb5_kvno mkvno = 0; char **restrlist; char *restring; + char **authz_data_list; krb5_timestamp restime; bool resbool; int result; @@ -503,6 +505,17 @@ static krb5_error_code ipadb_parse_ldap_entry(krb5_context kcontext, ied->last_admin_unlock = restime; } + ret = ipadb_ldap_attr_to_strlist(lcontext, lentry, + IPA_KRB_AUTHZ_DATA_ATTR, &authz_data_list); + if (ret != 0 && ret != ENOENT) { + kerr = KRB5_KDB_INTERNAL_ERROR; + goto done; + } + if (ret == 0) { + ied->authz_data = authz_data_list; + } + + kerr = 0; done: @@ -831,6 +844,10 @@ void ipadb_free_principal(krb5_context kcontext, krb5_db_entry *entry) free(ied->pw_history[i]); } free(ied->pw_history); + for (i = 0; ied->authz_data && ied->authz_data[i]; i++) { + free(ied->authz_data[i]); + } + free(ied->authz_data); free(ied->pol); free(ied); } -- 1.8.1.2 -------------- next part -------------- From 157093b95b65e6e1bd1e906c4de12b6ff4e57d21 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 12 Feb 2013 14:02:27 +0100 Subject: [PATCH 098/100] ipa-kdb: add PAC only if requested Instead of always adding a PAC to the Kerberos ticket the global default for the authorization data and the authorization data of the service entry is evaluated and the PAC is added accordingly. --- daemons/ipa-kdb/ipa_kdb_mspac.c | 86 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 84 insertions(+), 2 deletions(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index f05c552fc18eb9b15ee3e39b513184589bc8c468..c72e762ca4c680def9ed97e492f6edfcc5e0677f 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -96,6 +96,10 @@ static char *memberof_pac_attrs[] = { #define SID_SUB_AUTHS 15 #define MAX(a,b) (((a)>(b))?(a):(b)) +#define AUTHZ_DATA_TYPE_PAC "MS-PAC" +#define AUTHZ_DATA_TYPE_PAD "PAD" +#define AUTHZ_DATA_TYPE_NONE "NONE" + static int string_to_sid(char *str, struct dom_sid *sid) { unsigned long val; @@ -1515,6 +1519,71 @@ done: return kerr; } +void get_authz_data_types(krb5_context context, krb5_db_entry *entry, + bool *_with_pac, bool *_with_pad) +{ + struct ipadb_e_data *ied = NULL; + struct ipadb_context *ipactx; + size_t c; + bool none_found = false; + char **authz_data_list; + bool with_pac = false; + bool with_pad = false; + + if (entry != NULL) { + ied = (struct ipadb_e_data *) entry->e_data; + } + + if (ied == NULL || ied->authz_data == NULL) { + if (context == NULL) { + krb5_klog_syslog(LOG_ERR, "Missing Kerberos context, no " \ + "authorization data will be added."); + goto done; + } + + ipactx = ipadb_get_context(context); + if (ipactx == NULL || ipactx->authz_data == NULL) { + krb5_klog_syslog(LOG_ERR, "No default authorization data types " \ + "available, no authorization data will " \ + "be added."); + goto done; + } + + authz_data_list = ipactx->authz_data; + } else { + authz_data_list = ied->authz_data; + } + + + for (c = 0; authz_data_list[c]; c++) { + if (strcasecmp(authz_data_list[c], AUTHZ_DATA_TYPE_PAC) == 0) { + with_pac = true; + } else if (strcasecmp(authz_data_list[c], AUTHZ_DATA_TYPE_PAD) == 0) { + with_pad = true; + } else if (strcasecmp(authz_data_list[c], AUTHZ_DATA_TYPE_NONE) == 0) { + none_found = true; + } else { + krb5_klog_syslog(LOG_ERR, "Ignoring unsupported " \ + "authorization data type [%s].", + authz_data_list[c]); + } + } + +done: + if (none_found) { + with_pac = false; + with_pad = false; + } + + if (_with_pac != NULL) { + *_with_pac = with_pac; + } + if (_with_pad != NULL) { + *_with_pad = with_pad; + } + +} + krb5_error_code ipadb_sign_authdata(krb5_context context, unsigned int flags, krb5_const_principal client_princ, @@ -1537,6 +1606,8 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, krb5_error_code kerr; krb5_pac pac = NULL; krb5_data pac_data; + bool with_pac; + bool with_pad; /* When using s4u2proxy client_princ actually refers to the proxied user * while client->princ to the proxy service asking for the TGS on behalf @@ -1547,9 +1618,20 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, ks_client_princ = client->princ; } + /* We only need to check the server entry here, because even if the client + * is a service with a valid authorization data it will result to NONE + * because ipadb_get_pac() can only generate a pac for 'real' IPA users. + * (I assume this will be the same for PAD.) */ + get_authz_data_types(context, server, &with_pac, &with_pad); + + if (with_pad) { + krb5_klog_syslog(LOG_ERR, "PAD authorization data is requested but " \ + "currently not supported."); + } + is_as_req = ((flags & KRB5_KDB_FLAG_CLIENT_REFERRALS_ONLY) != 0); - if (is_as_req && (flags & KRB5_KDB_FLAG_INCLUDE_PAC)) { + if (is_as_req && with_pac && (flags & KRB5_KDB_FLAG_INCLUDE_PAC)) { kerr = ipadb_get_pac(context, client, &pac); if (kerr != 0 && kerr != ENOENT) { @@ -1557,7 +1639,7 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, } } - if (!is_as_req) { + if (!is_as_req && with_pac) { /* find the existing PAC, if present */ kerr = krb5_find_authdata(context, tgt_auth_data, NULL, KRB5_AUTHDATA_WIN2K_PAC, &pac_auth_data); -- 1.8.1.2 -------------- next part -------------- From e263d27ab1363bd55ba7122daee95bf9cd32b4aa Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 19 Feb 2013 12:16:37 +0100 Subject: [PATCH 099/100] Add unit test for get_authz_data_types() --- daemons/ipa-kdb/Makefile.am | 29 +++++ daemons/ipa-kdb/tests/ipa_kdb_tests.c | 197 ++++++++++++++++++++++++++++++++++ 2 files changed, 226 insertions(+) create mode 100644 daemons/ipa-kdb/tests/ipa_kdb_tests.c diff --git a/daemons/ipa-kdb/Makefile.am b/daemons/ipa-kdb/Makefile.am index 5f4e6e2a6a940486a0c904f737f28c476df98773..23ba1cc05ec157a0f4d9b594350ebaf10b2098dc 100644 --- a/daemons/ipa-kdb/Makefile.am +++ b/daemons/ipa-kdb/Makefile.am @@ -52,6 +52,35 @@ ipadb_la_LIBADD = \ $(NDRPAC_LIBS) \ $(NULL) +if HAVE_CHECK +TESTS = ipa_kdb_tests +check_PROGRAMS = ipa_kdb_tests +endif + +ipa_kdb_tests_SOURCES = \ + tests/ipa_kdb_tests.c \ + ipa_kdb.c \ + ipa_kdb_common.c \ + ipa_kdb_mkey.c \ + ipa_kdb_passwords.c \ + ipa_kdb_principals.c \ + ipa_kdb_pwdpolicy.c \ + ipa_kdb_mspac.c \ + ipa_kdb_delegation.c \ + ipa_kdb_audit_as.c \ + $(KRB5_UTIL_SRCS) \ + $(NULL) +ipa_kdb_tests_CFLAGS = $(CHECK_CFLAGS) +ipa_kdb_tests_LDADD = \ + $(CHECK_LIBS) \ + $(KRB5_LIBS) \ + $(LDAP_LIBS) \ + $(NDRPAC_LIBS) \ + -lnss3 \ + -lkdb5 \ + -lsss_idmap \ + $(NULL) + dist_noinst_DATA = ipa_kdb.exports EXTRA_DIST = \ diff --git a/daemons/ipa-kdb/tests/ipa_kdb_tests.c b/daemons/ipa-kdb/tests/ipa_kdb_tests.c new file mode 100644 index 0000000000000000000000000000000000000000..77c5ed896bc1b94d9757007d5f0ee816ba5c6675 --- /dev/null +++ b/daemons/ipa-kdb/tests/ipa_kdb_tests.c @@ -0,0 +1,197 @@ +/** BEGIN COPYRIGHT BLOCK + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * Additional permission under GPLv3 section 7: + * + * In the following paragraph, "GPL" means the GNU General Public + * License, version 3 or any later version, and "Non-GPL Code" means + * code that is governed neither by the GPL nor a license + * compatible with the GPL. + * + * You may link the code of this Program with Non-GPL Code and convey + * linked combinations including the two, provided that such Non-GPL + * Code only links to the code of this Program through those well + * defined interfaces identified in the file named EXCEPTION found in + * the source code files (the "Approved Interfaces"). The files of + * Non-GPL Code may instantiate templates or use macros or inline + * functions from the Approved Interfaces without causing the resulting + * work to be covered by the GPL. Only the copyright holders of this + * Program may make changes or additions to the list of Approved + * Interfaces. + * + * Authors: + * Sumit Bose + * + * Copyright (C) 2013 Red Hat, Inc. + * All rights reserved. + * END COPYRIGHT BLOCK **/ + +#include +#include +#include +#include +#include +#include +#include + +#include "ipa-kdb/ipa_kdb.h" + +int krb5_klog_syslog(int l, const char *format, ...) +{ + va_list ap; + char *s = NULL; + int ret; + + va_start(ap, format); + + ret = vasprintf(&s, format, ap); + va_end(ap); + if (ret < 0) { + /* ENOMEM */ + return -1; + } + + fprintf(stderr, "%s\n", s); + free(s); + + return 0; +} + +extern void get_authz_data_types(krb5_context context, krb5_db_entry *entry, + bool *with_pac, bool *with_pad); + +START_TEST(test_get_authz_data_types) +{ + bool with_pac; + bool with_pad; + krb5_db_entry *entry; + struct ipadb_e_data *ied; + size_t c; + char *ad_none_only[] = {"NONE", NULL}; + char *ad_pad_only[] = {"PAD", NULL}; + char *ad_pac_only[] = {"MS-PAC", NULL}; + char *ad_illegal_only[] = {"abc", NULL}; + char *ad_pac_and_pad[] = {"MS-PAC", "PAD", NULL}; + char *ad_pac_and_none[] = {"MS-PAC", "NONE", NULL}; + char *ad_none_and_pad[] = {"NONE", "PAD", NULL}; + char *ad_pac_and_pad_case[] = {"Ms-Pac", "PaD", NULL}; + krb5_context krb5_ctx; + krb5_error_code kerr; + struct ipadb_context *ipa_ctx; + + struct test_set { + char **authz_data; + char **global_authz_data; + bool exp_with_pac; + bool exp_with_pad; + const char *err_msg; + } test_set[] = { + {ad_none_only, NULL, false, false, "with only NONE in entry"}, + {ad_pac_only, NULL, true, false, "with only MS-PAC in entry"}, + {ad_pad_only, NULL, false, true, "with only PAD in entry"}, + {ad_illegal_only, NULL, false, false, "with only an invalid value in entry"}, + {ad_pac_and_pad, NULL, true, true, "with MS-PAC and PAD in entry"}, + {ad_pac_and_none, NULL, false, false, "with MS-PAC and NONE in entry"}, + {ad_none_and_pad, NULL, false, false, "with NONE and PAD in entry"}, + {ad_pac_and_pad_case, NULL, true, true, "with MS-PAC and PAD in entry in different case"}, + {NULL, ad_none_only, false, false, "with only NONE in global config"}, + {NULL, ad_pac_only, true, false, "with only MS-PAC in global config"}, + {NULL, ad_pad_only, false, true, "with only PAD in global config"}, + {NULL, ad_illegal_only, false, false, "with only an invalid value in global config"}, + {NULL, ad_pac_and_pad, true, true, "with MS-PAC and PAD in global config"}, + {NULL, ad_pac_and_none, false, false, "with MS-PAC and NONE in global config"}, + {NULL, ad_none_and_pad, false, false, "with NONE and PAD in global entry"}, + {NULL, ad_pac_and_pad_case, true, true, "with MS-PAC and PAD in global entry in different case"}, + {ad_none_only, ad_pac_only, false, false, "with NONE overriding PAC in global entry"}, + {ad_pad_only, ad_pac_only, false, true, "with PAC overriding PAC in global entry"}, + {ad_illegal_only, ad_pac_only, false, false, "with invalid value overriding PAC in global entry"}, + {ad_pac_and_pad, ad_pac_only, true, true, "with PAC and PAD overriding PAC in global entry"}, + {ad_none_and_pad, ad_pac_only, false, false, "with NONE and PAD overriding PAC in global entry"}, + {NULL, NULL, false, false, NULL} + }; + + get_authz_data_types(NULL, NULL, NULL, NULL); + + with_pad = true; + get_authz_data_types(NULL, NULL, NULL, &with_pad); + fail_unless(!with_pad, "with_pad not false with NULL inuput."); + + with_pac = true; + get_authz_data_types(NULL, NULL, &with_pac, NULL); + fail_unless(!with_pac, "with_pac not false with NULL inuput."); + + with_pad = true; + with_pac = true; + get_authz_data_types(NULL, NULL, &with_pac, &with_pad); + fail_unless(!with_pad, "with_pad not false with NULL inuput."); + fail_unless(!with_pac, "with_pac not false with NULL inuput."); + + entry = calloc(1, sizeof(krb5_db_entry)); + fail_unless(entry != NULL, "calloc krb5_db_entry failed."); + + ied = calloc(1, sizeof(struct ipadb_e_data)); + fail_unless(ied != NULL, "calloc struct ipadb_e_data failed."); + entry->e_data = (void *) ied; + + kerr = krb5_init_context(&krb5_ctx); + fail_unless(kerr == 0, "krb5_init_context failed."); + kerr = krb5_db_setup_lib_handle(krb5_ctx); + fail_unless(kerr == 0, "krb5_db_setup_lib_handle failed.\n"); + ipa_ctx = calloc(1, sizeof(struct ipadb_context)); + fail_unless(ipa_ctx != NULL, "calloc failed.\n"); + ipa_ctx->kcontext = krb5_ctx; + kerr = krb5_db_set_context(krb5_ctx, ipa_ctx); + fail_unless(kerr == 0, "krb5_db_set_context failed.\n"); + + for (c = 0; test_set[c].authz_data != NULL || + test_set[c].global_authz_data != NULL; c++) { + ied->authz_data = test_set[c].authz_data; + ipa_ctx->authz_data = test_set[c].global_authz_data; + get_authz_data_types(krb5_ctx, entry, &with_pac, &with_pad); + fail_unless(with_pad == test_set[c].exp_with_pad, "with_pad not %s %s.", + test_set[c].exp_with_pad ? "true" : "false", + test_set[c].err_msg); + fail_unless(with_pac == test_set[c].exp_with_pac, "with_pac not %s %s.", + test_set[c].exp_with_pac ? "true" : "false", + test_set[c].err_msg); + } + + krb5_db_fini(krb5_ctx); + krb5_free_context(krb5_ctx); +} +END_TEST + +Suite * ipa_kdb_suite(void) +{ + Suite *s = suite_create("IPA kdb"); + + TCase *tc_helper = tcase_create("Helper functions"); + tcase_add_test(tc_helper, test_get_authz_data_types); + suite_add_tcase(s, tc_helper); + + return s; +} + +int main(void) +{ + int number_failed; + + Suite *s = ipa_kdb_suite (); + SRunner *sr = srunner_create (s); + srunner_run_all (sr, CK_VERBOSE); + number_failed = srunner_ntests_failed (sr); + srunner_free (sr); + + return (number_failed == 0) ? EXIT_SUCCESS : EXIT_FAILURE; +} -- 1.8.1.2 -------------- next part -------------- From 6bec0c6de14fbcaf731eed479a73c23073e7c018 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 27 Feb 2013 10:32:40 +0100 Subject: [PATCH 100/100] Mention PAC issue with NFS in service plugin doc --- ipalib/plugins/service.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/ipalib/plugins/service.py b/ipalib/plugins/service.py index a3d436e61eb7b7ef2958188a83055b53f52e2562..6b6634458a5980ed263e929365034a7d90bc5bb3 100644 --- a/ipalib/plugins/service.py +++ b/ipalib/plugins/service.py @@ -66,6 +66,11 @@ EXAMPLES: Override a default list of supported PAC types for the service: ipa service-mod HTTP/web.example.com --pac-type=MS-PAC + A typical use case where overriding the PAC type is needed is NFS. + Currently the related code in the Linux kernel can only handle Kerberos + tickets up to a maximal size. Since the PAC data can become quite large it + is recommended to set --pac-type=NONE for NFS services. + Delete an IPA service: ipa service-del HTTP/web.example.com @@ -258,7 +263,8 @@ class service(LDAPObject): cli_name='pac_type', label=_('PAC type'), doc=_("Override default list of supported PAC types." - " Use 'NONE' to disable PAC support for this service"), + " Use 'NONE' to disable PAC support for this service," + " e.g. this might be necessary for NFS services."), values=(u'MS-PAC', u'PAD', u'NONE'), csv=True, ), -- 1.8.1.2 From sbose at redhat.com Wed Feb 27 11:35:36 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 12:35:36 +0100 Subject: [Freeipa-devel] [PATCHES] 101-107 Fixes for various Coverity issues Message-ID: <20130227113536.GD3633@localhost.localdomain> Hi, the attached patches 102-107 fix issues found by Coverity which are tracked by tickets #3422-#3427 and remove an unused variable (patch 101). bye, Sumit -------------- next part -------------- From 97b3b7dedac28704d51e2fa4b4dc975a20d17ada Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 19 Feb 2013 12:48:58 +0100 Subject: [PATCH 101/107] ipa-kdb: remove unused variable --- daemons/ipa-kdb/ipa_kdb_principals.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemons/ipa-kdb/ipa_kdb_principals.c b/daemons/ipa-kdb/ipa_kdb_principals.c index 46a921cb2f267b977df4a0042d81a49beccb5a91..11c155e64e20d16da98158eb4d7b8034803bf1de 100644 --- a/daemons/ipa-kdb/ipa_kdb_principals.c +++ b/daemons/ipa-kdb/ipa_kdb_principals.c @@ -535,7 +535,7 @@ static krb5_error_code ipadb_fetch_principals(struct ipadb_context *ipactx, krb5_error_code kerr; char *src_filter = NULL; char *esc_original_princ = NULL; - int ret, i; + int ret; if (!ipactx->lcontext) { ret = ipadb_get_connection(ipactx); -- 1.8.1.2 -------------- next part -------------- From d5a64eced4800190221395d19341918ae417f118 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Tue, 19 Feb 2013 12:49:25 +0100 Subject: [PATCH 102/107] ipa-kdb: Uninitialized scalar variable in ipadb_reinit_mspac() There was a code path where ret was used instead of kerr to save a return value. Fixes https://fedorahosted.org/freeipa/ticket/3422 --- daemons/ipa-kdb/ipa_kdb_mspac.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index c72e762ca4c680def9ed97e492f6edfcc5e0677f..d022bd9a958f2ec591a6a338753b510492b2d400 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -1986,12 +1986,11 @@ krb5_error_code ipadb_reinit_mspac(struct ipadb_context *ipactx) if (ipactx->mspac && ipactx->mspac->num_trusts == 0) { /* Check if there is any trust configured. If not, just return * and do not re-initialize the MS-PAC structure. */ - ret = ipadb_mspac_check_trusted_domains(ipactx); - if (ret == KRB5_KDB_NOENTRY) { - ret = 0; + kerr = ipadb_mspac_check_trusted_domains(ipactx); + if (kerr == KRB5_KDB_NOENTRY) { + kerr = 0; goto done; - } else if (ret != 0) { - ret = EIO; + } else if (kerr != 0) { goto done; } } -- 1.8.1.2 -------------- next part -------------- From c36fc476b52315012d2d17994d0337761d25cb5e Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 27 Feb 2013 12:12:45 +0100 Subject: [PATCH 103/107] ipa-sam: Array compared against 0 in ipasam_set_trusted_domain() ipa_mspac_well_known_sids is a globally defined array so the check was always true. Fixes https://fedorahosted.org/freeipa/ticket/3423 --- daemons/ipa-sam/ipa_sam.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 0d4a27cf6def737412f6ab3ca48df8d1339c15b6..b9fc00c8d08821e09b3242a82b3673a88ace0a51 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -2293,7 +2293,7 @@ static NTSTATUS ipasam_set_trusted_domain(struct pdb_methods *methods, &td->trust_forest_trust_info); } - for (i = 0; ipa_mspac_well_known_sids && ipa_mspac_well_known_sids[i]; i++) { + for (i = 0; ipa_mspac_well_known_sids[i]; i++) { smbldap_make_mod(priv2ld(ldap_state), entry, &mods, LDAP_ATTRIBUTE_SID_BLACKLIST_INCOMING, ipa_mspac_well_known_sids[i]); -- 1.8.1.2 -------------- next part -------------- From bffd1ec51ea697887d012ec1eda4171f4affafaf Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 22 Feb 2013 13:30:30 +0100 Subject: [PATCH 104/107] ipa-kdb: Dereference after null check in ipa_kdb_mspac.c A wrong logic was used to check ipactx. Fixes https://fedorahosted.org/freeipa/ticket/3424 --- daemons/ipa-kdb/ipa_kdb_mspac.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index d022bd9a958f2ec591a6a338753b510492b2d400..2c07f4cffe38eb127a7b3f901dc7700e88d6af2a 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -1217,7 +1217,7 @@ static krb5_error_code filter_logon_info(krb5_context context, * */ if (info->info->info3.sidcount != 0) { ipactx = ipadb_get_context(context); - if (!ipactx && !ipactx->mspac) { + if (!ipactx || !ipactx->mspac) { return KRB5_KDB_DBNOTINITED; } count = info->info->info3.sidcount; -- 1.8.1.2 -------------- next part -------------- From e56a47f82f4600ebcb26c012c382c6686125c14a Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 22 Feb 2013 13:40:08 +0100 Subject: [PATCH 105/107] ipa-lockout: Wrong sizeof argument in ipa_lockout.c Fixes https://fedorahosted.org/freeipa/ticket/3425 --- daemons/ipa-slapi-plugins/ipa-lockout/ipa_lockout.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemons/ipa-slapi-plugins/ipa-lockout/ipa_lockout.c b/daemons/ipa-slapi-plugins/ipa-lockout/ipa_lockout.c index 351273d2af05437c52141164fe78cb97914fd1db..fea997d79825e05a6ffe4bf65e22821948794ff3 100644 --- a/daemons/ipa-slapi-plugins/ipa-lockout/ipa_lockout.c +++ b/daemons/ipa-slapi-plugins/ipa-lockout/ipa_lockout.c @@ -317,7 +317,7 @@ ipalockout_start(Slapi_PBlock * pb) g_plugin_started = 1; - global_ipactx = (struct ipa_context *)malloc(sizeof(global_ipactx)); + global_ipactx = (struct ipa_context *)malloc(sizeof(*global_ipactx)); global_ipactx->disable_last_success = false; global_ipactx->disable_lockout = false; ipalockout_get_global_config(global_ipactx); -- 1.8.1.2 -------------- next part -------------- From 03d7762db882b10a550ce0276fa42a21fe58f4aa Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 22 Feb 2013 13:45:58 +0100 Subject: [PATCH 106/107] ipa-extdom: Double-free in ipa_extdom_common.c Fixes https://fedorahosted.org/freeipa/ticket/3426 --- daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c | 1 - 1 file changed, 1 deletion(-) diff --git a/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c b/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c index 0c054fb84d375b4e429b35af8f2dc896a475cb2b..660ed04c2ced547027f79b9da01ede21775ede19 100644 --- a/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c +++ b/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_common.c @@ -297,7 +297,6 @@ static int get_domain_info(struct ipa_extdom_ctx *ctx, const char *domain_name, domain_info->sid, &range); } if (err != IDMAP_SUCCESS) { - free_domain_info(domain_info); ret = EFAULT; goto done; } -- 1.8.1.2 -------------- next part -------------- From ebe7b3fa98dcaa79fe9a72231399239c80b5447a Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 22 Feb 2013 13:56:06 +0100 Subject: [PATCH 107/107] ipa-pwd: Unchecked return value ipapwd_chpwop() Fixes https://fedorahosted.org/freeipa/ticket/3427 --- daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c index 7fcf647f783db48fe903aa11148df3dcd87e9404..974eda31889109de80a263d1715e3a194ae431ef 100644 --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipa_pwd_extop.c @@ -247,7 +247,11 @@ parse_req_done: slapi_sdn_free(&sdn); if (be) { chpwop_pb = slapi_pblock_new(); - slapi_pblock_set(chpwop_pb, SLAPI_BACKEND, be); + if (slapi_pblock_set(chpwop_pb, SLAPI_BACKEND, be)) { + LOG_FATAL("slapi_pblock_set failed!\n"); + rc = LDAP_OPERATIONS_ERROR; + goto free_and_return; + } rc = slapi_back_transaction_begin(chpwop_pb); if (rc) { LOG_FATAL("failed to start transaction\n"); -- 1.8.1.2 From pviktori at redhat.com Wed Feb 27 11:46:44 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 27 Feb 2013 12:46:44 +0100 Subject: [Freeipa-devel] Using the new LDAP code Message-ID: <512DF224.7000804@redhat.com> Hello, A big refactoring of our LDAP code should be merged soon-ish now. Here's a summary for developers. If you see these outside ipaldap.py, you're looking at deprecated API: - methods with camelCaseNames - methods with _s and _ext postfixes (modify_s, search_ext, ...) The exception is client code and places where we don't want to read the schema (migration, AD). These are still limited to raw python-ldap for now. The LDAPEntry class represents LDAP entries. It behaves like a dictionary of lists: entry.get(attrname) or entry[attrname] should always give you a list. LDAPEntry.dn will give you the entry's DN. Single-value attributes are represented as lists with a single value. (For now, some code still puts bare values instead of lists in entries, in that case you'll still get a bare value from get()/__getitem__. That should be fixed eventually.) The "single_value" method gets a single value, with checking. Always use `entry.single_value('abc')` instead of `entry.get('abc')[0]`. Also, single_value allows a default: `entry.single_value('abc', None)`. LDAPEntry is case-insensitive and handles attributes with multiple names: entry['cn'] and entry['CN'] and entry['CommonName'] are equivalent. IPA plugins traditionally use (dn, entry_attrs) pairs to represent entries. To make that work, iterating over an LDAPEntry will, for now, yield the DN and the entry itself. Always use keys() or values() when iterating over an entry. LDAPEntry objects are tied to a connection. Do not create them directly, use a connection method like make_entry() or get_entry(). Speaking of connections, there still are two classes for those: ldap2 and IPAdmin. ldap2 is an API plugin created using the IPA settings. It works in Apache (per-thread connections). It also applies the default IPA settings (time & records limit). Use IPAdmin if IPA is not installed yet (or when it's being uninstalled), or if you need to connect to a non-default server or bind as a user like the DM. Besides the connecting code, both of these (will ideally) have the same API, based on the old ldap2. A rough summary: - make_entry(dn, attrs) - get_entry(dn) - add_entry(entry) - update_entry(entry) - delete_entry(entry_or_dn) - get_entries(base_dn, [scope, [filter, [attrs_list]]]): simple search - find_entries: more powerful (and backwards-compatible) search - make_filter & friends, unchanged from ldap2 ldap2's DN normalization ? appending the suffix to DNs that don't end with it ?? is gone, you need to do that manually now. That should be it, if you don't intend to hack on ipaldap itself or the ldapupdater. If you have any questions, ask! (Or look at the code :) -- Petr? From mkosek at redhat.com Wed Feb 27 11:53:06 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 27 Feb 2013 12:53:06 +0100 Subject: [Freeipa-devel] [PATCH] 374 Remove ORDERING for IA5 attributeTypes In-Reply-To: <512DE6CD.8010901@redhat.com> References: <512CEAC4.5060403@redhat.com> <512DE6CD.8010901@redhat.com> Message-ID: <512DF3A2.5040204@redhat.com> On 02/27/2013 11:58 AM, Petr Viktorin wrote: > On 02/26/2013 06:03 PM, Martin Kosek wrote: >> IA5 string syntax does not have a compatible ORDERING matching rule. >> Simply use default ORDERING for these attributeTypes as we already >> do in other cases. >> >> https://fedorahosted.org/freeipa/ticket/3398 >> >> ----- >> >> This is a follow-up ticket for regression introduced by Rob's 1087. I did not >> add any update rules for IPA servers that were already wrongly updated as we >> did not release any IPA version with these bad attributeTypes and I want to >> keep .update files small. >> >> Martin > > ACK > Pushed to master, ipa-3-1, ipa-3-0. Martin From mkosek at redhat.com Wed Feb 27 12:39:31 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 27 Feb 2013 13:39:31 +0100 Subject: [Freeipa-devel] [PATCHES] 101-107 Fixes for various Coverity issues In-Reply-To: <20130227113536.GD3633@localhost.localdomain> References: <20130227113536.GD3633@localhost.localdomain> Message-ID: <512DFE83.6060009@redhat.com> On 02/27/2013 12:35 PM, Sumit Bose wrote: > Hi, > > the attached patches 102-107 fix issues found by Coverity which are > tracked by tickets #3422-#3427 and remove an unused variable > (patch 101). > > bye, > Sumit > I see just one issue. In patch 0105: - global_ipactx = (struct ipa_context *)malloc(sizeof(global_ipactx)); + global_ipactx = (struct ipa_context *)malloc(sizeof(*global_ipactx)); I do not think this will work right. *global_ipactx will just de-reference global_ipactxt and run sizeof on the result, right? I would prefer this change: global_ipactx = (struct ipa_context *)malloc(sizeof(struct ipa_context *)); Martin From pvoborni at redhat.com Wed Feb 27 12:54:55 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 27 Feb 2013 13:54:55 +0100 Subject: [Freeipa-devel] [PATCH] 259 Combobox keyboard support In-Reply-To: <940957870.1823903.1361926213062.JavaMail.root@redhat.com> References: <940957870.1823903.1361926213062.JavaMail.root@redhat.com> Message-ID: <512E021F.5030004@redhat.com> On 02/27/2013 01:50 AM, Endi Sukma Dewata wrote: > Looks good. The patch already provides improvements, so feel free to push. > >> * when CB is non-editable, user can start typing, first character >> will open list, second will be entered into search input. Note: I >> wanted to copy the first char to the search box as well, but I did not >> figure out reliable method for converting keycode to char for non ASCII >> keyboard layouts > > Could we use keypress() for this instead of keydown()? See http://api.jquery.com/keypress/. The keypress() should return the actual character, whereas the keydown() would return the code of the keyboard key so it wouldn't be able to distinguish between 'a' and 'A', which is not what we want. Weird, I could bet that I tried it and it failed. Probably I did some stupid mistake. Anyway there are still limitations: 1) Firefox behaves differently than Chrome: * when keydown is prevented. FF raises keypress, Chrome doesn't * FF doesn't raise keypress for '?' (Czech character), Chrome does. (the input method is pressing 'shift', '?' and 's'. 2) what to do on paste? We can't use input nor change event for evaluating it - default is prevented. I'm inclined to leave it as is. Mentioned cases can be tested in following fiddle: http://jsfiddle.net/vopet/aMynp/ > > Another minor thing, if the search box is in focus, you can use the Up/Down arrow to go to the list. However, from the list you cannot use the Up/Down arrow to go back to the search box, you'd have to use the Tab key. It's not really a problem, but it's not that consistent. > Up/Down keys are natural way of changing value of a list - this part seems OK to me. So Up/Down arrow in search box might the problem, but I consider it a feature - saves some strokes. Also, User can always use tab to switch between search box and list. > -- > Endi S. Dewata > -- Petr Vobornik From mkosek at redhat.com Wed Feb 27 12:55:27 2013 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 27 Feb 2013 13:55:27 +0100 Subject: [Freeipa-devel] [PATCHES] 101-107 Fixes for various Coverity issues In-Reply-To: <512DFE83.6060009@redhat.com> References: <20130227113536.GD3633@localhost.localdomain> <512DFE83.6060009@redhat.com> Message-ID: <512E023F.9000106@redhat.com> On 02/27/2013 01:39 PM, Martin Kosek wrote: > On 02/27/2013 12:35 PM, Sumit Bose wrote: >> Hi, >> >> the attached patches 102-107 fix issues found by Coverity which are >> tracked by tickets #3422-#3427 and remove an unused variable >> (patch 101). >> >> bye, >> Sumit >> > > I see just one issue. In patch 0105: > > - global_ipactx = (struct ipa_context *)malloc(sizeof(global_ipactx)); > + global_ipactx = (struct ipa_context *)malloc(sizeof(*global_ipactx)); > > I do not think this will work right. *global_ipactx will just de-reference > global_ipactxt and run sizeof on the result, right? > > I would prefer this change: > > global_ipactx = (struct ipa_context *)malloc(sizeof(struct ipa_context *)); > > Martin > We just discussed this in Brno - in fact your change should be right as you allocate memory for the whole structure and not just a pointer to the structure (though I personally would prefer sizeof(struct ipa_context) as it is clearer). Hmrph, C language 101 revisited :-) Martin From pspacek at redhat.com Wed Feb 27 12:55:47 2013 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 27 Feb 2013 13:55:47 +0100 Subject: [Freeipa-devel] CA name constrains Message-ID: <512E0253.7090405@redhat.com> Hello list, during our last meeting with Simo we discussed support for name constraint extension in CA certificates and clients. The Name Constraints Extensions is defined here: http://tools.ietf.org/html/rfc5280#section-4.2.1.10 Following article could be interesting for you if you like longer stories: "Mozilla changes policy to limit risk of subordinate CA certificate abuse" Author: Lucian Constantin 19.02.2013 kl 21:50 http://news.idg.no/cw/art.cfm?id=8C9E7CFA-0E65-24B0-1539C891C8F4C09B If I remember correctly, questions were mainly about support on client side and about implications for older clients. -- Petr^2 Spacek From abokovoy at redhat.com Wed Feb 27 12:55:42 2013 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 27 Feb 2013 14:55:42 +0200 Subject: [Freeipa-devel] [PATCHES] 101-107 Fixes for various Coverity issues In-Reply-To: <512DFE83.6060009@redhat.com> References: <20130227113536.GD3633@localhost.localdomain> <512DFE83.6060009@redhat.com> Message-ID: <20130227125542.GB7238@redhat.com> On Wed, 27 Feb 2013, Martin Kosek wrote: >On 02/27/2013 12:35 PM, Sumit Bose wrote: >> Hi, >> >> the attached patches 102-107 fix issues found by Coverity which are >> tracked by tickets #3422-#3427 and remove an unused variable >> (patch 101). >> >> bye, >> Sumit >> > >I see just one issue. In patch 0105: > >- global_ipactx = (struct ipa_context *)malloc(sizeof(global_ipactx)); >+ global_ipactx = (struct ipa_context *)malloc(sizeof(*global_ipactx)); > >I do not think this will work right. *global_ipactx will just de-reference >global_ipactxt and run sizeof on the result, right? No, sizeof() is compile-time operator. Compiler will do proper type inferring. See C11 6.5.3.4 for details. It is a common idiom to do foo = (foo_t*) malloc(sizeof(*foo)); -- / Alexander Bokovoy From jcholast at redhat.com Wed Feb 27 12:59:52 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 13:59:52 +0100 Subject: [Freeipa-devel] [PATCHES] 146-164 LDAP code refactoring (Part 4) In-Reply-To: <512C8859.8050001@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510AB17B.80909@redhat.com> <510B8116.9060409@redhat.com> <510B89BA.2040903@redhat.com> <510BD364.7050809@redhat.com> <512387D1.60603@redhat.com> <5124BB8F.80803@redhat.com> <512B93A6.4050602@redhat.com> <512C8859.8050001@redhat.com> Message-ID: <512E0348.2040407@redhat.com> On 26.2.2013 11:03, Petr Viktorin wrote: >> Thanks. I think you should also add a tearDown method to test_LDAPEntry >> which disconnects self.conn if it is connected (the same thing test_ldap >> does). > > Thanks for the catch, added. > ACK. -- Jan Cholasta From loris at lgs.com.ve Wed Feb 27 13:16:41 2013 From: loris at lgs.com.ve (Loris Santamaria) Date: Wed, 27 Feb 2013 08:46:41 -0430 Subject: [Freeipa-devel] What about desktop policies? In-Reply-To: <512D16EC.7090102@redhat.com> References: <1361819744.2111.48.camel@toron.pzo.lgs.com.ve> <512D16EC.7090102@redhat.com> Message-ID: <1361971001.25048.15.camel@toron.pzo.lgs.com.ve> El mar, 26-02-2013 a las 15:11 -0500, Dmitri Pal escribi?: > On 02/25/2013 02:15 PM, Loris Santamaria wrote: > > Hi all, > > > > some customers of ours are interested in managing desktop policies for > > their linux workstations, really nothing fancy, corporate background and > > proxy settings are the most common requests. > > > > In the past I created Gnome desktop profiles using Sabayon, distributed > > them using puppet and associated them to user accounts with a Sabayon > > specific LDAP attribute, a process a bit convoluted, and no longer > > possible since sabayon is no longer developed. Also it was really buggy, > > and very gnome specific. > > > > I was thinking in how integrate desktop policies in freeIPA in a general > > manner and I wanted to share my ideas with you. Hopefully some of this > > may be incorporated in IPA at some point in the future. > > > > Properties of a "policy": > > > > * is a collection of "settings" > > * can be associated with users or groups (desktop policy) or with > > hosts or hostgroups (system policy) > > * is associated with a "consumer", the client software that > > interprets and applies the policy. This way one could define > > policies for dconf, policies for kde, policies for WBEM. > > > > Properties of a "setting" > > * is a key-value pair > > * must conform to a "schema" > > * may be mandatory > > > > The schema: > > * indicates which attributes a policy may consist of > > * indicates which kind of value may take an attribute. Bool, > > string, etc. > > * There may be more than one schema for a given "consumer". For > > example for dconf you may have an evolution schema, a > > gnome-games schema, etc. > > > > Sample policy creation process: > > 1. The admin creates a new schema in IPA, with a command like "ipa > > schema-add --consumer=dconf gnomeSettingsSchema" > > 2. The admin adds some definition to the schema: "ipa > > schema-add-setting gnomeSettingsSchema > > --name=/schemas/desktop/gnome/background/picture_filename > > --type=string --description='File to use for the background > > image.'" > > 3. He creates a new policy: "ipa policy-add corporateBackground > > --type=desktop --consumer=dconf > > 4. He adds a setting to the policy: "ipa policy-add-setting > > corporateBackground > > --name=/schemas/desktop/gnome/background/picture_filename > > --value=file:///san/wp/wallpaper.jpg --mandatory". Ipa would > > check that the key is defined in one of the dconf related > > schemas and the value is acceptable for that key. > > 5. He associates the policy with users: "ipa-policy-add-user > > corporateBackground --groups=ipausers" > > > > How should the policy be applied? On the workstation, on startup, an ipa > > related utility should check if there are any policies related to the > > workstation, if there are any it should call a helper capable of > > applying a specific type of policy. Then on user logon another ipa > > related utility should check if there are any policies associated with > > the user and call the appropriate helper, if available. > > > > For the policy created in the above example, on logon the ipa policy > > utility would find that there is a policy of type dconf associated with > > the user. It would check if there is a dconf policy helper installed and > > if positive it would call the helper passing it the parameters defined > > in the policy. > > > > Hope this is useful at least as a starting point in defining desktop > > policies in IPA. > > This is great! > Thank you for sharing some ideas. > We sort of stayed away from centralized policy management for quite > some time. > Originally we thought that IPA will do policy management and did a lot > of design around it. > However at some point we realized that there is an overlap with the > system management and content management for which things like puppet > are more suitable. We said then that IdM would focus on managing > identity related policies that are traditionally served via LDAP. > The things that you are talking about resemble to some extent MSFT GPO > and we felt that IdM might not be the right place for it. May be it is > time to reassess it. > I would however not go that route at least yet. Hey puppet is great and we use it a lot to install packages and to distribute configuration files, yet it is not so great to set these key=value kind of settings of which more and more "modern" software consists of. When you take into consideration gconf settings for gnome 2.x, dconf settings for gnome 3.x, mozilla settings, thunderbird settings it quickly becomes a PITA to manage them distributing around files with puppet. Having those key=value pairs in an ldap would allow a helper on the client to pull only the keys it understands and to merge them in the configuration database in the appropriate way. Of course i took inspiration from AD GPOs, yet I think that IPA should manage these key=value kind of policies in a more general way, for one because nobody in the linux world controls all of the desktop stack and because the end user experience is changing so fast that we can't really know how the user will access IT resources ten year from now. > > If Desktop can read additional properties related to user (background, > default language, etc.) from SSSD over a DBUS interface the SSSD > should be able to pull this data from the IdM (eventually). On the IdM > we probably can make these additional attributes available in the user > entries using class of service like we do with password policies. > > We have plans for SSSD to handle more attributes than posix and > integrate with Desktop. > https://fedorahosted.org/sssd/wiki/DesignDocs/AccountsService > > IMO once this work is started we would be able to see how we can > configure and serve more data from IPA for clients to consume. > > Meanwhile I suggest you create a ticket in IPA trac and put your ideas > there. Ok I'll file the RFE. Thanks! -- Loris Santamaria linux user #70506 xmpp:loris at lgs.com.ve Links Global Services, C.A. http://www.lgs.com.ve Tel: 0286 952.06.87 Cel: 0414 095.00.10 sip:103 at lgs.com.ve ------------------------------------------------------------ "If I'd asked my customers what they wanted, they'd have said a faster horse" - Henry Ford -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6173 bytes Desc: not available URL: From simo at redhat.com Wed Feb 27 13:16:49 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 27 Feb 2013 08:16:49 -0500 Subject: [Freeipa-devel] CA name constrains In-Reply-To: <512E0253.7090405@redhat.com> References: <512E0253.7090405@redhat.com> Message-ID: <1361971009.2962.41.camel@willson.li.ssimo.org> On Wed, 2013-02-27 at 13:55 +0100, Petr Spacek wrote: > Hello list, > > during our last meeting with Simo we discussed support for name constraint > extension in CA certificates and clients. > > The Name Constraints Extensions is defined here: > http://tools.ietf.org/html/rfc5280#section-4.2.1.10 > > Following article could be interesting for you if you like longer stories: > "Mozilla changes policy to limit risk of subordinate CA certificate abuse" > Author: Lucian Constantin 19.02.2013 kl 21:50 > http://news.idg.no/cw/art.cfm?id=8C9E7CFA-0E65-24B0-1539C891C8F4C09B > > If I remember correctly, questions were mainly about support on client side > and about implications for older clients. I had a chat with Kai Engert (in CC) at DevConf.cz about this, we'll try to work on this as time permits. NSS seem to support this extension but so far we do not have tests covering it apparently. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 27 13:37:18 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 27 Feb 2013 08:37:18 -0500 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <20130227105831.GC3633@localhost.localdomain> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> Message-ID: <1361972238.2962.54.camel@willson.li.ssimo.org> On Wed, 2013-02-27 at 11:58 +0100, Sumit Bose wrote: > On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: > > On 02/21/2013 04:24 PM, Sumit Bose wrote: > > > Hi, > > > > > > this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 > > > The related design page is > > > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > > > > > It was agreed while discussing the design page that the currently > > > hardcoded value for the NFS service will be dropped completely, hence > > > the first patch reverts to patch which added the hardcoded value. > > > > > > The second patch adds the needed attribute to compensate the now missing > > > hardcoded value. > > > > > > In the following three patches the PAC type is read and used > > > accordingly. The last patch adds a unit test for a new function. > > > > > > bye, > > > Sumit > > > > I did only sanity testing to the C part of the RFE so far, but by reading it it > > looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part > > is OK. > > Alexander asked in irc to change strcmp() to strcasecmp() so that > options like 'Ms-Pac' or 'none' are accepted as well. Is there a good reason to allow insensitive matching ? in LDAP you can't use true or false either for booleans, you have to use TRUE and FALSE, so I am not so thrilled to allow insensitive matching for this option as well. > > I will comment on the Python parts: > > > > 1) Your update check testing if there is any NFS service with ipakrbauthzdata > > looks like a good heuristics to prevent admin frustration. Though an ideal > > solution would require > > https://fedorahosted.org/freeipa/ticket/2961 > > to prevent multiple updates when admin purposefully removes this attribute. We > > may want to reference the ticket in a comment in the update plugin... > > I added a comment in the code and in #2961. > > > > > > > 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns > > truncated result. As you do not update entries directly but only return update > > instructions when you have no truncated results, you will loop. > > > > To simulate this, I just created 2 NFS principals and set size_limit=1 in your > > find_entries call. > > Thanks for catching this, I removed the truncated logic and added a > messages if truncated was set. > > > > > > > 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS > > service principals added by service-add? Shouldn't we set ipakrbauthzdata for > > new services too? As a default when no user ipakrhbauthzdata is set... > > Otherwise admins could be surprised why their new NFS services do not work > > while the others do. > > > > Also, I think we should have this NFS culprit documented somewhere (i.e. why we > > set ipakrbauthzdata to NONE by default). At least as a small section in the > > online help for services plugin... > > I added comments to the help and the doc string in patch #100. If you > think it is necessary to implicitly set PAC type to 'NONE' if NFS > services are added and no type is given explicitly, I would prefer if > you can open a new tickets so that it can be discussed independently. > > Thank you for the review. New versions attached. I do not think we should set the policy to NONE by default, OTOH I see the problem with upgrades. Maybe we should have a way to express additional defaults, per service type. Ie add additional values like: ipakrbAuthsData: nfs:NONE This would be a default but only for services that have the form nfs/*@* This would allow us to avoid magical special casing in the code and allow admins to change the overall default for specific services as needed. Maybe we should add a new ticket for this ? Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Wed Feb 27 13:44:44 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 27 Feb 2013 08:44:44 -0500 Subject: [Freeipa-devel] CA name constrains In-Reply-To: <1361971009.2962.41.camel@willson.li.ssimo.org> References: <512E0253.7090405@redhat.com> <1361971009.2962.41.camel@willson.li.ssimo.org> Message-ID: <1361972684.2962.55.camel@willson.li.ssimo.org> On Wed, 2013-02-27 at 08:16 -0500, Simo Sorce wrote: > On Wed, 2013-02-27 at 13:55 +0100, Petr Spacek wrote: > > Hello list, > > > > during our last meeting with Simo we discussed support for name constraint > > extension in CA certificates and clients. > > > > The Name Constraints Extensions is defined here: > > http://tools.ietf.org/html/rfc5280#section-4.2.1.10 > > > > Following article could be interesting for you if you like longer stories: > > "Mozilla changes policy to limit risk of subordinate CA certificate abuse" > > Author: Lucian Constantin 19.02.2013 kl 21:50 > > http://news.idg.no/cw/art.cfm?id=8C9E7CFA-0E65-24B0-1539C891C8F4C09B > > > > If I remember correctly, questions were mainly about support on client side > > and about implications for older clients. > > I had a chat with Kai Engert (in CC) at DevConf.cz about this, we'll try > to work on this as time permits. > NSS seem to support this extension but so far we do not have tests > covering it apparently. > > Simo. > Btw I opened ticket https://fedorahosted.org/freeipa/ticket/3466 to track this. Simo. -- Simo Sorce * Red Hat, Inc * New York From jcholast at redhat.com Wed Feb 27 13:55:36 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 14:55:36 +0100 Subject: [Freeipa-devel] [PATCH] 105 Fix remove while iterating in suppress_netgroup_memberof Message-ID: <512E1058.1080308@redhat.com> Hi, this patch fixes . Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-105-Fix-remove-while-iterating-in-suppress_netgroup_memb.patch Type: text/x-patch Size: 2086 bytes Desc: not available URL: From rcritten at redhat.com Wed Feb 27 14:50:21 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 27 Feb 2013 09:50:21 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <20130226131500.GB3633@localhost.localdomain> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> Message-ID: <512E1D2D.1090501@redhat.com> Sumit Bose wrote: > On Mon, Feb 25, 2013 at 03:12:19PM +0100, Martin Kosek wrote: >> On 02/25/2013 03:09 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >> ... >>>> 4) What does "NOTE: We will need to be clear that this range has nothing to do >>>> with Trust ranges." actually mean? AFAIU, IPA should have all local ranges >>>> covered with a local "idrange" range(s). >>> >>> IPA ranges is completely separate from DNA ranges. You can set/modify all the >>> local ranges you want and it won't affect the UIDs getting assigned. >>> >>>> If it does not have it covered, it could happen that for example a new trust >>>> would overlap with this user-defined local range and we would have colliding >>>> POSIX IDs... >>> >>> Hmm, that's a good point. >>> >>>> IMO, dnarange-set and dnanextrange-set should at first check if the range is >>>> covered with some local idrange and only then allowed setting the new range. >>> >>> I can do that as well, but again, the local ranges don't really affect the ids >>> we hand out via DNA. >>> >>> rob >> >> You are right, that DNA plugin is really not aware of the idranges we set in >> IPA. But the idrange is still a safeguard for our POSIX IDs to not overlap with >> trust ranges and I think we should respect that with ipa-replica-manage. >> >> I wonder if there was not even a plan to increase cooperation between our >> idranges and DNA plugin, maybe Sumit or Alexander knows more. > > If I understand the use case and design properly, it is about > re-arranging the sub-ranges each replica can use from the original > range, which was given/created at installation time and which is also > stored as idrange in > DOMAIN.NAME_id_range,cn=ranges,cn=etc,dc=domain,dc=name with > objectclass=ipaDomainIDRange. > > If the re-arrangement does not result in IDs which are outside of this > range give by the ipaDomainIDRange object, no conflicts with idranges > used by trusted domain will occur, because it is one of the task of the > idrange objects to avoid those conflicts. > > If the original given range is exhausted completely and a completely new > DNA range must be created, it should be checked with ipa idrange-find > that the new range is not used and a new ipaDomainIDRange object which > reserves the local range should be added. > > There are https://fedorahosted.org/freeipa/ticket/591 which can be used > to track the coordinated creation of DNA and id-range. > Honestly, I was going to skip the range stuff altogether and just focus on the DNA ranges. I'm primarily interested in saving any range from a server that is going to be deleted. I can save that in the on-base attribute of any existing servers. Where I run into trouble is if all the on-base are populated, then we'd lose range. So I need to alert the user somehow. This leads to wanting to be able to add that back in so I came up with the idea of letting users manage the DNA range directly. It may be possible to check the DOMAIN ranges as well but that bumps the complexity up considerably. rob From jcholast at redhat.com Wed Feb 27 15:00:22 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 16:00:22 +0100 Subject: [Freeipa-devel] 0165-0174 LDAP code refactoring (Part 5) In-Reply-To: <510A4167.5030001@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510A4167.5030001@redhat.com> Message-ID: <512E1F86.1090606@redhat.com> On 31.1.2013 11:03, Petr Viktorin wrote: > And hee is another batch of patches. This one is about converting the > legacy IPAdmin and raw python-ldap calls to the new wrappers. > Patch 165: I have noticed two things that are not really related to your work, but here they are nonetheless: + if self.admin_conn.get_entries( + DN(api.env.container_ranges, self.suffix), + ldap.SCOPE_ONELEVEL, + "objectclass=ipaDomainIDRange"): Is that a valid filter? + if self.admin_conn.get_entries("cn=accounts," + self.suffix, + ldap.SCOPE_SUBTREE, id_filter): This doesn't seem right as well, why is the DN class not used here? Patch 167: - conn.sasl_interactive_bind_s(None, sasl_auth) + conn.do_sasl_gssapi_bind() sasl_auth is unused after this change, can you please remove it as well (and cb_info too)? - self.sasl_interactive_bind_s, timeout, None, SASL_AUTH) + self.conn.sasl_interactive_bind_s, timeout, None, SASL_AUTH) Again, this is not related to your work, but can we please rename SASL_AUTH to SASL_GSSAPI? Patch 173: - res = con.search_st(str(base), - ldap.SCOPE_SUBTREE, - filterstr=srcfilter, - attrlist=attrs, - timeout=10) + res = con.get_entries(base, con.SCOPE_SUBTREE, srcfilter, attrs) I assume the timeout is there for a reason, can you please keep it? Patch 174: - conn.modify_s( - def_dn, - [(ldap.MOD_REPLACE, - 'originfilter', - disable_attr)] - ) + entry['originfilter'] = [disable_attr] I think you forgot to call update_entry here. Honza -- Jan Cholasta From akrivoka at redhat.com Wed Feb 27 15:20:38 2013 From: akrivoka at redhat.com (Ana Krivokapic) Date: Wed, 27 Feb 2013 16:20:38 +0100 Subject: [Freeipa-devel] [PATCH] 0007 Web UI: Realm Domains page Message-ID: <512E2446.5060802@redhat.com> Add support for Realm Domains to web UI. https://fedorahosted.org/freeipa/ticket/3407 -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-akrivoka-0007-Realm-Domains-page.patch Type: text/x-patch Size: 8319 bytes Desc: not available URL: From pspacek at redhat.com Wed Feb 27 15:21:16 2013 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 27 Feb 2013 16:21:16 +0100 Subject: [Freeipa-devel] [PATCH 0112] Make log messages related to Kerberos more verbose In-Reply-To: <511A3C84.4020205@redhat.com> References: <511A3C84.4020205@redhat.com> Message-ID: <512E246C.7070702@redhat.com> On 12.2.2013 13:58, Petr Spacek wrote: > Hello, > > Make log messages related to Kerberos more verbose. > > This change should help people supporting bind-dyndb-ldap to figure out what > is happening under covers. Added explanatory error message for case where Kerberos context initialization failed. -- Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0112-2-Make-log-messages-related-to-Kerberos-more-verbose.patch Type: text/x-patch Size: 4730 bytes Desc: not available URL: From jcholast at redhat.com Wed Feb 27 15:31:45 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 16:31:45 +0100 Subject: [Freeipa-devel] [PATCHES] 106-113 Access raw LDAP values directly from LDAPEntry Message-ID: <512E26E1.5010302@redhat.com> Hi, these patches add the ability to access and manipulate raw attribute values as they are returned from python-ldap to the LDAPEntry class. This is useful for comparing entries, computing modlists for the modify operation, deleting values that don't match the syntax of an attribute, etc., because we don't need to care what syntax does a particular attribute use, or what Python type is used for it in the framework (raw values are always list of str). It should also make interaction with non-389 DS LDAP servers easier in the framework. (It might be too late for this kind of changes to get into 3.2 now, I'm posting these patches mainly so that you are aware that they exist.) Honza -- Jan Cholasta -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-106-Make-LDAPEntry-a-wrapper-around-dict-rather-than-a-d.patch Type: text/x-patch Size: 6998 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-107-Introduce-IPASimpleLDAPObject.decode-method-for-deco.patch Type: text/x-patch Size: 3057 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-108-Always-use-lists-for-values-in-LDAPEntry-internally.patch Type: text/x-patch Size: 3524 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-109-Store-both-encoded-and-decoded-values-in-LDAPEntry.patch Type: text/x-patch Size: 13414 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-110-Make-sure-attributeTypes-updates-are-done-before-obj.patch Type: text/x-patch Size: 1039 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-111-Remove-legacy-toDict-and-origDataDict-methods-of-LDA.patch Type: text/x-patch Size: 4780 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-112-Save-raw-attribute-values-of-search-results-in-IPASi.patch Type: text/x-patch Size: 830 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-jcholast-113-Use-raw-attribute-values-when-generating-modlists-in.patch Type: text/x-patch Size: 3124 bytes Desc: not available URL: From jdennis at redhat.com Wed Feb 27 16:09:07 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 27 Feb 2013 11:09:07 -0500 Subject: [Freeipa-devel] Using the new LDAP code In-Reply-To: <512DF224.7000804@redhat.com> References: <512DF224.7000804@redhat.com> Message-ID: <512E2FA3.9050601@redhat.com> On 02/27/2013 06:46 AM, Petr Viktorin wrote: > Hello, > A big refactoring of our LDAP code should be merged soon-ish now. Here's > a summary for developers. Great, that's fabulous news and thanks for the good work! > IPA plugins traditionally use (dn, entry_attrs) pairs to represent > entries. To make that work, iterating over an LDAPEntry will, for now, > yield the DN and the entry itself. Always use keys() or values() when > iterating over an entry. I'm trying parse what the above means, it seems to be one of the two: 1) There is still a bunch of code that continues to use 2-tuples in the plugins and additional work remains to converge on a single API. 2) All the code that used 2-tuples has been cleaned up and expunged, however if the old 2-tuple methodology was used it would still work. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pvoborni at redhat.com Wed Feb 27 16:10:24 2013 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 27 Feb 2013 17:10:24 +0100 Subject: [Freeipa-devel] [PATCH] 0007 Web UI: Realm Domains page In-Reply-To: <512E2446.5060802@redhat.com> References: <512E2446.5060802@redhat.com> Message-ID: <512E2FF0.9040609@redhat.com> On 02/27/2013 04:20 PM, Ana Krivokapic wrote: > Add support for Realm Domains to web UI. > > https://fedorahosted.org/freeipa/ticket/3407 The patch looks good, but there is a issue we don't have a precedence for. The mod command is doing dns check for new domains. Currently we can't specify --force option to bypass the check. I see two possible implementations: 1) On update, when user adds or modifies the values, a dialog would pop up and ask user whether he wants to force it. 2) Another option is to disable edit on the list(deletion would be still allowed) and move the add operation to separate action in action list. I prefer the former. Latter might have issues with two modifications (delete and add) at the same time at two different places (facet and add dialog). -- Petr Vobornik From pviktori at redhat.com Wed Feb 27 16:21:55 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 27 Feb 2013 17:21:55 +0100 Subject: [Freeipa-devel] Using the new LDAP code In-Reply-To: <512E2FA3.9050601@redhat.com> References: <512DF224.7000804@redhat.com> <512E2FA3.9050601@redhat.com> Message-ID: <512E32A3.2060100@redhat.com> On 02/27/2013 05:09 PM, John Dennis wrote: > On 02/27/2013 06:46 AM, Petr Viktorin wrote: >> Hello, >> A big refactoring of our LDAP code should be merged soon-ish now. Here's >> a summary for developers. > > Great, that's fabulous news and thanks for the good work! > >> IPA plugins traditionally use (dn, entry_attrs) pairs to represent >> entries. To make that work, iterating over an LDAPEntry will, for now, >> yield the DN and the entry itself. Always use keys() or values() when >> iterating over an entry. > > I'm trying parse what the above means, it seems to be one of the two: > > 1) There is still a bunch of code that continues to use 2-tuples in the > plugins and additional work remains to converge on a single API. > > 2) All the code that used 2-tuples has been cleaned up and expunged, > however if the old 2-tuple methodology was used it would still work. The first. There are minimal changes to the plugins (most changes there are related to the removal of DN normalization). -- Petr? From jcholast at redhat.com Wed Feb 27 16:23:56 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 17:23:56 +0100 Subject: [Freeipa-devel] Using the new LDAP code In-Reply-To: <512E2FA3.9050601@redhat.com> References: <512DF224.7000804@redhat.com> <512E2FA3.9050601@redhat.com> Message-ID: <512E331C.2050106@redhat.com> Hi, On 27.2.2013 17:09, John Dennis wrote: >> IPA plugins traditionally use (dn, entry_attrs) pairs to represent >> entries. To make that work, iterating over an LDAPEntry will, for now, >> yield the DN and the entry itself. Always use keys() or values() when >> iterating over an entry. > > I'm trying parse what the above means, it seems to be one of the two: > > 1) There is still a bunch of code that continues to use 2-tuples in the > plugins and additional work remains to converge on a single API. > > 2) All the code that used 2-tuples has been cleaned up and expunged, > however if the old 2-tuple methodology was used it would still work. > it's the former, there is still code that uses 2-tuples. Honza -- Jan Cholasta From jcholast at redhat.com Wed Feb 27 17:01:57 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 18:01:57 +0100 Subject: [Freeipa-devel] 0165-0174 LDAP code refactoring (Part 5) In-Reply-To: <512E3863.4090201@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510A4167.5030001@redhat.com> <512E1F86.1090606@redhat.com> <512E3863.4090201@redhat.com> Message-ID: <512E3C05.3060709@redhat.com> On 27.2.2013 17:46, Petr Viktorin wrote: > On 02/27/2013 04:00 PM, Jan Cholasta wrote: >> Patch 165: >> >> I have noticed two things that are not really related to your work, but >> here they are nonetheless: >> >> + if self.admin_conn.get_entries( >> + DN(api.env.container_ranges, self.suffix), >> + ldap.SCOPE_ONELEVEL, >> + "objectclass=ipaDomainIDRange"): >> >> Is that a valid filter? > > I don't think it is, fixed. > >> + if self.admin_conn.get_entries("cn=accounts," + self.suffix, >> + ldap.SCOPE_SUBTREE, id_filter): >> >> This doesn't seem right as well, why is the DN class not used here? > > I made it a DN. > >> Patch 167: >> >> - conn.sasl_interactive_bind_s(None, sasl_auth) >> + conn.do_sasl_gssapi_bind() >> >> sasl_auth is unused after this change, can you please remove it as well >> (and cb_info too)? > > Removed. > >> - self.sasl_interactive_bind_s, timeout, None, SASL_AUTH) >> + self.conn.sasl_interactive_bind_s, timeout, None, SASL_AUTH) >> >> Again, this is not related to your work, but can we please rename >> SASL_AUTH to SASL_GSSAPI? > > OK, makes sense > >> Patch 173: >> >> - res = con.search_st(str(base), >> - ldap.SCOPE_SUBTREE, >> - filterstr=srcfilter, >> - attrlist=attrs, >> - timeout=10) >> + res = con.get_entries(base, con.SCOPE_SUBTREE, srcfilter, attrs) >> >> I assume the timeout is there for a reason, can you please keep it? > > Fixed, thanks > >> Patch 174: >> >> - conn.modify_s( >> - def_dn, >> - [(ldap.MOD_REPLACE, >> - 'originfilter', >> - disable_attr)] >> - ) >> + entry['originfilter'] = [disable_attr] >> >> I think you forgot to call update_entry here. > > Fixed, thanks again. > > Thanks. ACK. -- Jan Cholasta From jdennis at redhat.com Wed Feb 27 17:14:14 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 27 Feb 2013 12:14:14 -0500 Subject: [Freeipa-devel] Using the new LDAP code In-Reply-To: <512E331C.2050106@redhat.com> References: <512DF224.7000804@redhat.com> <512E2FA3.9050601@redhat.com> <512E331C.2050106@redhat.com> Message-ID: <512E3EE6.8080707@redhat.com> On 02/27/2013 11:23 AM, Jan Cholasta wrote: > Hi, > > On 27.2.2013 17:09, John Dennis wrote: >>> IPA plugins traditionally use (dn, entry_attrs) pairs to represent >>> entries. To make that work, iterating over an LDAPEntry will, for now, >>> yield the DN and the entry itself. Always use keys() or values() when >>> iterating over an entry. >> >> I'm trying parse what the above means, it seems to be one of the two: >> >> 1) There is still a bunch of code that continues to use 2-tuples in the >> plugins and additional work remains to converge on a single API. >> >> 2) All the code that used 2-tuples has been cleaned up and expunged, >> however if the old 2-tuple methodology was used it would still work. >> > > it's the former, there is still code that uses 2-tuples. O.K. so we're still a ways away from completing the task. Would this be a correct summary? Phase 1 is completed, a consistent API has been defined and implemented. Phase 2 is converting all the code to use the API defined in Phase 1, a task yet to be done. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jcholast at redhat.com Wed Feb 27 17:22:28 2013 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 27 Feb 2013 18:22:28 +0100 Subject: [Freeipa-devel] Using the new LDAP code In-Reply-To: <512E3EE6.8080707@redhat.com> References: <512DF224.7000804@redhat.com> <512E2FA3.9050601@redhat.com> <512E331C.2050106@redhat.com> <512E3EE6.8080707@redhat.com> Message-ID: <512E40D4.7030306@redhat.com> On 27.2.2013 18:14, John Dennis wrote: > On 02/27/2013 11:23 AM, Jan Cholasta wrote: >> Hi, >> >> On 27.2.2013 17:09, John Dennis wrote: >>>> IPA plugins traditionally use (dn, entry_attrs) pairs to represent >>>> entries. To make that work, iterating over an LDAPEntry will, for now, >>>> yield the DN and the entry itself. Always use keys() or values() when >>>> iterating over an entry. >>> >>> I'm trying parse what the above means, it seems to be one of the two: >>> >>> 1) There is still a bunch of code that continues to use 2-tuples in the >>> plugins and additional work remains to converge on a single API. >>> >>> 2) All the code that used 2-tuples has been cleaned up and expunged, >>> however if the old 2-tuple methodology was used it would still work. >>> >> >> it's the former, there is still code that uses 2-tuples. > > O.K. so we're still a ways away from completing the task. Would this be > a correct summary? > > Phase 1 is completed, a consistent API has been defined and implemented. > Phase 2 is converting all the code to use the API defined in Phase 1, a > task yet to be done. > Correct. But I don't think converting 2-tuples to the new API will be a huge task. Honza -- Jan Cholasta From jdennis at redhat.com Wed Feb 27 17:22:16 2013 From: jdennis at redhat.com (John Dennis) Date: Wed, 27 Feb 2013 12:22:16 -0500 Subject: [Freeipa-devel] Using the new LDAP code In-Reply-To: <512E40D4.7030306@redhat.com> References: <512DF224.7000804@redhat.com> <512E2FA3.9050601@redhat.com> <512E331C.2050106@redhat.com> <512E3EE6.8080707@redhat.com> <512E40D4.7030306@redhat.com> Message-ID: <512E40C8.2030809@redhat.com> On 02/27/2013 12:22 PM, Jan Cholasta wrote: > On 27.2.2013 18:14, John Dennis wrote: >> On 02/27/2013 11:23 AM, Jan Cholasta wrote: >>> Hi, >>> >>> On 27.2.2013 17:09, John Dennis wrote: >>>>> IPA plugins traditionally use (dn, entry_attrs) pairs to represent >>>>> entries. To make that work, iterating over an LDAPEntry will, for now, >>>>> yield the DN and the entry itself. Always use keys() or values() when >>>>> iterating over an entry. >>>> >>>> I'm trying parse what the above means, it seems to be one of the two: >>>> >>>> 1) There is still a bunch of code that continues to use 2-tuples in the >>>> plugins and additional work remains to converge on a single API. >>>> >>>> 2) All the code that used 2-tuples has been cleaned up and expunged, >>>> however if the old 2-tuple methodology was used it would still work. >>>> >>> >>> it's the former, there is still code that uses 2-tuples. >> >> O.K. so we're still a ways away from completing the task. Would this be >> a correct summary? >> >> Phase 1 is completed, a consistent API has been defined and implemented. >> Phase 2 is converting all the code to use the API defined in Phase 1, a >> task yet to be done. >> > > Correct. But I don't think converting 2-tuples to the new API will be a > huge task. Cool! -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pviktori at redhat.com Wed Feb 27 16:46:27 2013 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 27 Feb 2013 17:46:27 +0100 Subject: [Freeipa-devel] 0165-0174 LDAP code refactoring (Part 5) In-Reply-To: <512E1F86.1090606@redhat.com> References: <50F83498.9050104@redhat.com> <50FD7CF8.2070401@redhat.com> <50FEAA66.4010704@redhat.com> <51010252.1030200@redhat.com> <51013FD5.8030004@redhat.com> <51028E91.4070601@redhat.com> <51063804.4060507@redhat.com> <5106949D.408@redhat.com> <5107ED49.4000502@redhat.com> <5108EDAB.9040406@redhat.com> <510A40C0.5060406@redhat.com> <510A4167.5030001@redhat.com> <512E1F86.1090606@redhat.com> Message-ID: <512E3863.4090201@redhat.com> On 02/27/2013 04:00 PM, Jan Cholasta wrote: > On 31.1.2013 11:03, Petr Viktorin wrote: >> And hee is another batch of patches. This one is about converting the >> legacy IPAdmin and raw python-ldap calls to the new wrappers. >> > > Patch 165: > > I have noticed two things that are not really related to your work, but > here they are nonetheless: > > + if self.admin_conn.get_entries( > + DN(api.env.container_ranges, self.suffix), > + ldap.SCOPE_ONELEVEL, > + "objectclass=ipaDomainIDRange"): > > Is that a valid filter? I don't think it is, fixed. > + if self.admin_conn.get_entries("cn=accounts," + self.suffix, > + ldap.SCOPE_SUBTREE, id_filter): > > This doesn't seem right as well, why is the DN class not used here? I made it a DN. > Patch 167: > > - conn.sasl_interactive_bind_s(None, sasl_auth) > + conn.do_sasl_gssapi_bind() > > sasl_auth is unused after this change, can you please remove it as well > (and cb_info too)? Removed. > - self.sasl_interactive_bind_s, timeout, None, SASL_AUTH) > + self.conn.sasl_interactive_bind_s, timeout, None, SASL_AUTH) > > Again, this is not related to your work, but can we please rename > SASL_AUTH to SASL_GSSAPI? OK, makes sense > Patch 173: > > - res = con.search_st(str(base), > - ldap.SCOPE_SUBTREE, > - filterstr=srcfilter, > - attrlist=attrs, > - timeout=10) > + res = con.get_entries(base, con.SCOPE_SUBTREE, srcfilter, attrs) > > I assume the timeout is there for a reason, can you please keep it? Fixed, thanks > Patch 174: > > - conn.modify_s( > - def_dn, > - [(ldap.MOD_REPLACE, > - 'originfilter', > - disable_attr)] > - ) > + entry['originfilter'] = [disable_attr] > > I think you forgot to call update_entry here. Fixed, thanks again. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0165-02-Remove-search_s-and-search_ext_s-from-IPAdmin.patch Type: text/x-patch Size: 8992 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0167-03-Remove-IPAdmin.sasl_interactive_bind_s.patch Type: text/x-patch Size: 4354 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0173-03-Use-IPAdmin-rather-than-raw-python-ldap-in-ipactl.patch Type: text/x-patch Size: 5612 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0174-04-Remove-some-uses-of-raw-python-ldap.patch Type: text/x-patch Size: 37190 bytes Desc: not available URL: From sbose at redhat.com Wed Feb 27 17:39:44 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 18:39:44 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512E1D2D.1090501@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> Message-ID: <20130227173944.GE3633@localhost.localdomain> On Wed, Feb 27, 2013 at 09:50:21AM -0500, Rob Crittenden wrote: > Sumit Bose wrote: > >On Mon, Feb 25, 2013 at 03:12:19PM +0100, Martin Kosek wrote: > >>On 02/25/2013 03:09 PM, Rob Crittenden wrote: > >>>Martin Kosek wrote: > >>... > >>>>4) What does "NOTE: We will need to be clear that this range has nothing to do > >>>>with Trust ranges." actually mean? AFAIU, IPA should have all local ranges > >>>>covered with a local "idrange" range(s). > >>> > >>>IPA ranges is completely separate from DNA ranges. You can set/modify all the > >>>local ranges you want and it won't affect the UIDs getting assigned. > >>> > >>>>If it does not have it covered, it could happen that for example a new trust > >>>>would overlap with this user-defined local range and we would have colliding > >>>>POSIX IDs... > >>> > >>>Hmm, that's a good point. > >>> > >>>>IMO, dnarange-set and dnanextrange-set should at first check if the range is > >>>>covered with some local idrange and only then allowed setting the new range. > >>> > >>>I can do that as well, but again, the local ranges don't really affect the ids > >>>we hand out via DNA. > >>> > >>>rob > >> > >>You are right, that DNA plugin is really not aware of the idranges we set in > >>IPA. But the idrange is still a safeguard for our POSIX IDs to not overlap with > >>trust ranges and I think we should respect that with ipa-replica-manage. > >> > >>I wonder if there was not even a plan to increase cooperation between our > >>idranges and DNA plugin, maybe Sumit or Alexander knows more. > > > >If I understand the use case and design properly, it is about > >re-arranging the sub-ranges each replica can use from the original > >range, which was given/created at installation time and which is also > >stored as idrange in > >DOMAIN.NAME_id_range,cn=ranges,cn=etc,dc=domain,dc=name with > >objectclass=ipaDomainIDRange. > > > >If the re-arrangement does not result in IDs which are outside of this > >range give by the ipaDomainIDRange object, no conflicts with idranges > >used by trusted domain will occur, because it is one of the task of the > >idrange objects to avoid those conflicts. > > > >If the original given range is exhausted completely and a completely new > >DNA range must be created, it should be checked with ipa idrange-find > >that the new range is not used and a new ipaDomainIDRange object which > >reserves the local range should be added. > > > >There are https://fedorahosted.org/freeipa/ticket/591 which can be used > >to track the coordinated creation of DNA and id-range. > > > > Honestly, I was going to skip the range stuff altogether and just > focus on the DNA ranges. > > I'm primarily interested in saving any range from a server that is > going to be deleted. I can save that in the on-base attribute of any > existing servers. Where I run into trouble is if all the on-base are > populated, then we'd lose range. So I need to alert the user > somehow. If it is only about moving existing DNA range it is fine, because this means that DNA will not give ID which are outside of the idrange of the IPA domain which is created at installation or during upgrade to v3. > > This leads to wanting to be able to add that back in so I came up > with the idea of letting users manage the DNA range directly. It may > be possible to check the DOMAIN ranges as well but that bumps the > complexity up considerably. But it looks like dnarange-set and dnanextrange-set can also be used to not only move existing DNA ranges but to create new DNA ranges which will lead to ID which are not in the idrange of the IPA domain but might be in an idrange which is used by a trusted domain. So I think a check and a warning are desirable. bye, Sumit > > rob From sbose at redhat.com Wed Feb 27 17:48:27 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 18:48:27 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <1361972238.2962.54.camel@willson.li.ssimo.org> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> <1361972238.2962.54.camel@willson.li.ssimo.org> Message-ID: <20130227174826.GF3633@localhost.localdomain> On Wed, Feb 27, 2013 at 08:37:18AM -0500, Simo Sorce wrote: > On Wed, 2013-02-27 at 11:58 +0100, Sumit Bose wrote: > > On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: > > > On 02/21/2013 04:24 PM, Sumit Bose wrote: > > > > Hi, > > > > > > > > this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 > > > > The related design page is > > > > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > > > > > > > It was agreed while discussing the design page that the currently > > > > hardcoded value for the NFS service will be dropped completely, hence > > > > the first patch reverts to patch which added the hardcoded value. > > > > > > > > The second patch adds the needed attribute to compensate the now missing > > > > hardcoded value. > > > > > > > > In the following three patches the PAC type is read and used > > > > accordingly. The last patch adds a unit test for a new function. > > > > > > > > bye, > > > > Sumit > > > > > > I did only sanity testing to the C part of the RFE so far, but by reading it it > > > looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part > > > is OK. > > > > Alexander asked in irc to change strcmp() to strcasecmp() so that > > options like 'Ms-Pac' or 'none' are accepted as well. > > Is there a good reason to allow insensitive matching ? > > in LDAP you can't use true or false either for booleans, you have to use > TRUE and FALSE, so I am not so thrilled to allow insensitive matching > for this option as well. I'm fine with this. Alexander, any comments. > > > > I will comment on the Python parts: > > > > > > 1) Your update check testing if there is any NFS service with ipakrbauthzdata > > > looks like a good heuristics to prevent admin frustration. Though an ideal > > > solution would require > > > https://fedorahosted.org/freeipa/ticket/2961 > > > to prevent multiple updates when admin purposefully removes this attribute. We > > > may want to reference the ticket in a comment in the update plugin... > > > > I added a comment in the code and in #2961. > > > > > > > > > > > 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns > > > truncated result. As you do not update entries directly but only return update > > > instructions when you have no truncated results, you will loop. > > > > > > To simulate this, I just created 2 NFS principals and set size_limit=1 in your > > > find_entries call. > > > > Thanks for catching this, I removed the truncated logic and added a > > messages if truncated was set. > > > > > > > > > > > 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS > > > service principals added by service-add? Shouldn't we set ipakrbauthzdata for > > > new services too? As a default when no user ipakrhbauthzdata is set... > > > Otherwise admins could be surprised why their new NFS services do not work > > > while the others do. > > > > > > Also, I think we should have this NFS culprit documented somewhere (i.e. why we > > > set ipakrbauthzdata to NONE by default). At least as a small section in the > > > online help for services plugin... > > > > I added comments to the help and the doc string in patch #100. If you > > think it is necessary to implicitly set PAC type to 'NONE' if NFS > > services are added and no type is given explicitly, I would prefer if > > you can open a new tickets so that it can be discussed independently. > > > > Thank you for the review. New versions attached. > > I do not think we should set the policy to NONE by default, OTOH I see > the problem with upgrades. Maybe we should have a way to express > additional defaults, per service type. > > Ie add additional values like: > > ipakrbAuthsData: nfs:NONE > > This would be a default but only for services that have the form nfs/*@* > > This would allow us to avoid magical special casing in the code and > allow admins to change the overall default for specific services as > needed. > > Maybe we should add a new ticket for this ? This sounds like a good compromise and will make updating much easier. If we agree to do it this way I can add ipakrbAuthsData: nfs:NONE and fix the code with this ticket. But we would need tickets for the CLI and WebUI to handle this new case. For the WebUI there already is https://fedorahosted.org/freeipa/ticket/3404 , maybe it can be extended to handle this new case as well? As for design documents I think I can added the needed information to http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . bye, Sumit > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York > From rcritten at redhat.com Wed Feb 27 19:03:24 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 27 Feb 2013 14:03:24 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <20130227173944.GE3633@localhost.localdomain> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> <20130227173944.GE3633@localhost.localdomain> Message-ID: <512E587C.7040709@redhat.com> Sumit Bose wrote: > On Wed, Feb 27, 2013 at 09:50:21AM -0500, Rob Crittenden wrote: >> Sumit Bose wrote: >>> On Mon, Feb 25, 2013 at 03:12:19PM +0100, Martin Kosek wrote: >>>> On 02/25/2013 03:09 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>> ... >>>>>> 4) What does "NOTE: We will need to be clear that this range has nothing to do >>>>>> with Trust ranges." actually mean? AFAIU, IPA should have all local ranges >>>>>> covered with a local "idrange" range(s). >>>>> >>>>> IPA ranges is completely separate from DNA ranges. You can set/modify all the >>>>> local ranges you want and it won't affect the UIDs getting assigned. >>>>> >>>>>> If it does not have it covered, it could happen that for example a new trust >>>>>> would overlap with this user-defined local range and we would have colliding >>>>>> POSIX IDs... >>>>> >>>>> Hmm, that's a good point. >>>>> >>>>>> IMO, dnarange-set and dnanextrange-set should at first check if the range is >>>>>> covered with some local idrange and only then allowed setting the new range. >>>>> >>>>> I can do that as well, but again, the local ranges don't really affect the ids >>>>> we hand out via DNA. >>>>> >>>>> rob >>>> >>>> You are right, that DNA plugin is really not aware of the idranges we set in >>>> IPA. But the idrange is still a safeguard for our POSIX IDs to not overlap with >>>> trust ranges and I think we should respect that with ipa-replica-manage. >>>> >>>> I wonder if there was not even a plan to increase cooperation between our >>>> idranges and DNA plugin, maybe Sumit or Alexander knows more. >>> >>> If I understand the use case and design properly, it is about >>> re-arranging the sub-ranges each replica can use from the original >>> range, which was given/created at installation time and which is also >>> stored as idrange in >>> DOMAIN.NAME_id_range,cn=ranges,cn=etc,dc=domain,dc=name with >>> objectclass=ipaDomainIDRange. >>> >>> If the re-arrangement does not result in IDs which are outside of this >>> range give by the ipaDomainIDRange object, no conflicts with idranges >>> used by trusted domain will occur, because it is one of the task of the >>> idrange objects to avoid those conflicts. >>> >>> If the original given range is exhausted completely and a completely new >>> DNA range must be created, it should be checked with ipa idrange-find >>> that the new range is not used and a new ipaDomainIDRange object which >>> reserves the local range should be added. >>> >>> There are https://fedorahosted.org/freeipa/ticket/591 which can be used >>> to track the coordinated creation of DNA and id-range. >>> >> >> Honestly, I was going to skip the range stuff altogether and just >> focus on the DNA ranges. >> >> I'm primarily interested in saving any range from a server that is >> going to be deleted. I can save that in the on-base attribute of any >> existing servers. Where I run into trouble is if all the on-base are >> populated, then we'd lose range. So I need to alert the user >> somehow. > > If it is only about moving existing DNA range it is fine, because this > means that DNA will not give ID which are outside of the idrange of the > IPA domain which is created at installation or during upgrade to v3. >> >> This leads to wanting to be able to add that back in so I came up >> with the idea of letting users manage the DNA range directly. It may >> be possible to check the DOMAIN ranges as well but that bumps the >> complexity up considerably. > > But it looks like dnarange-set and dnanextrange-set can also be used to > not only move existing DNA ranges but to create new DNA ranges which > will lead to ID which are not in the idrange of the IPA domain but might > be in an idrange which is used by a trusted domain. So I think a check > and a warning are desirable. That is an excellent point. I've updated the design. rob From sbose at redhat.com Wed Feb 27 19:42:23 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 20:42:23 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <20130227174826.GF3633@localhost.localdomain> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> <1361972238.2962.54.camel@willson.li.ssimo.org> <20130227174826.GF3633@localhost.localdomain> Message-ID: <20130227194223.GG3633@localhost.localdomain> On Wed, Feb 27, 2013 at 06:48:27PM +0100, Sumit Bose wrote: > On Wed, Feb 27, 2013 at 08:37:18AM -0500, Simo Sorce wrote: > > On Wed, 2013-02-27 at 11:58 +0100, Sumit Bose wrote: > > > On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: > > > > On 02/21/2013 04:24 PM, Sumit Bose wrote: > > > > > Hi, > > > > > > > > > > this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 > > > > > The related design page is > > > > > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > > > > > > > > > It was agreed while discussing the design page that the currently > > > > > hardcoded value for the NFS service will be dropped completely, hence > > > > > the first patch reverts to patch which added the hardcoded value. > > > > > > > > > > The second patch adds the needed attribute to compensate the now missing > > > > > hardcoded value. > > > > > > > > > > In the following three patches the PAC type is read and used > > > > > accordingly. The last patch adds a unit test for a new function. > > > > > > > > > > bye, > > > > > Sumit > > > > > > > > I did only sanity testing to the C part of the RFE so far, but by reading it it > > > > looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part > > > > is OK. > > > > > > Alexander asked in irc to change strcmp() to strcasecmp() so that > > > options like 'Ms-Pac' or 'none' are accepted as well. > > > > Is there a good reason to allow insensitive matching ? > > > > in LDAP you can't use true or false either for booleans, you have to use > > TRUE and FALSE, so I am not so thrilled to allow insensitive matching > > for this option as well. > > I'm fine with this. Alexander, any comments. Alexander said on irc that he is fine with the strict case-sensitive handling of the values, as long as the user input is automatically upper-cased by the framework as it is currently done for booleans. Since the WebUI does not allow user entered values this will only affect the CLI. bye, Sumit > > > > > > > I will comment on the Python parts: > > > > > > > > 1) Your update check testing if there is any NFS service with ipakrbauthzdata > > > > looks like a good heuristics to prevent admin frustration. Though an ideal > > > > solution would require > > > > https://fedorahosted.org/freeipa/ticket/2961 > > > > to prevent multiple updates when admin purposefully removes this attribute. We > > > > may want to reference the ticket in a comment in the update plugin... > > > > > > I added a comment in the code and in #2961. > > > > > > > > > > > > > > > 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns > > > > truncated result. As you do not update entries directly but only return update > > > > instructions when you have no truncated results, you will loop. > > > > > > > > To simulate this, I just created 2 NFS principals and set size_limit=1 in your > > > > find_entries call. > > > > > > Thanks for catching this, I removed the truncated logic and added a > > > messages if truncated was set. > > > > > > > > > > > > > > > 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS > > > > service principals added by service-add? Shouldn't we set ipakrbauthzdata for > > > > new services too? As a default when no user ipakrhbauthzdata is set... > > > > Otherwise admins could be surprised why their new NFS services do not work > > > > while the others do. > > > > > > > > Also, I think we should have this NFS culprit documented somewhere (i.e. why we > > > > set ipakrbauthzdata to NONE by default). At least as a small section in the > > > > online help for services plugin... > > > > > > I added comments to the help and the doc string in patch #100. If you > > > think it is necessary to implicitly set PAC type to 'NONE' if NFS > > > services are added and no type is given explicitly, I would prefer if > > > you can open a new tickets so that it can be discussed independently. > > > > > > Thank you for the review. New versions attached. > > > > I do not think we should set the policy to NONE by default, OTOH I see > > the problem with upgrades. Maybe we should have a way to express > > additional defaults, per service type. > > > > Ie add additional values like: > > > > ipakrbAuthsData: nfs:NONE > > > > This would be a default but only for services that have the form nfs/*@* > > > > This would allow us to avoid magical special casing in the code and > > allow admins to change the overall default for specific services as > > needed. > > > > Maybe we should add a new ticket for this ? > > This sounds like a good compromise and will make updating much easier. > If we agree to do it this way I can add ipakrbAuthsData: nfs:NONE and > fix the code with this ticket. But we would need tickets for the CLI and > WebUI to handle this new case. > > For the WebUI there already is > https://fedorahosted.org/freeipa/ticket/3404 , maybe it can be extended > to handle this new case as well? > > As for design documents I think I can added the needed information to > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > bye, > Sumit > > > > Simo. > > > > -- > > Simo Sorce * Red Hat, Inc * New York > > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From sbose at redhat.com Wed Feb 27 19:55:06 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 20:55:06 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512E587C.7040709@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> <20130227173944.GE3633@localhost.localdomain> <512E587C.7040709@redhat.com> Message-ID: <20130227195506.GH3633@localhost.localdomain> On Wed, Feb 27, 2013 at 02:03:24PM -0500, Rob Crittenden wrote: > Sumit Bose wrote: > >On Wed, Feb 27, 2013 at 09:50:21AM -0500, Rob Crittenden wrote: > >>Sumit Bose wrote: > >>>On Mon, Feb 25, 2013 at 03:12:19PM +0100, Martin Kosek wrote: > >>>>On 02/25/2013 03:09 PM, Rob Crittenden wrote: > >>>>>Martin Kosek wrote: > >>>>... > >>>>>>4) What does "NOTE: We will need to be clear that this range has nothing to do > >>>>>>with Trust ranges." actually mean? AFAIU, IPA should have all local ranges > >>>>>>covered with a local "idrange" range(s). > >>>>> > >>>>>IPA ranges is completely separate from DNA ranges. You can set/modify all the > >>>>>local ranges you want and it won't affect the UIDs getting assigned. > >>>>> > >>>>>>If it does not have it covered, it could happen that for example a new trust > >>>>>>would overlap with this user-defined local range and we would have colliding > >>>>>>POSIX IDs... > >>>>> > >>>>>Hmm, that's a good point. > >>>>> > >>>>>>IMO, dnarange-set and dnanextrange-set should at first check if the range is > >>>>>>covered with some local idrange and only then allowed setting the new range. > >>>>> > >>>>>I can do that as well, but again, the local ranges don't really affect the ids > >>>>>we hand out via DNA. > >>>>> > >>>>>rob > >>>> > >>>>You are right, that DNA plugin is really not aware of the idranges we set in > >>>>IPA. But the idrange is still a safeguard for our POSIX IDs to not overlap with > >>>>trust ranges and I think we should respect that with ipa-replica-manage. > >>>> > >>>>I wonder if there was not even a plan to increase cooperation between our > >>>>idranges and DNA plugin, maybe Sumit or Alexander knows more. > >>> > >>>If I understand the use case and design properly, it is about > >>>re-arranging the sub-ranges each replica can use from the original > >>>range, which was given/created at installation time and which is also > >>>stored as idrange in > >>>DOMAIN.NAME_id_range,cn=ranges,cn=etc,dc=domain,dc=name with > >>>objectclass=ipaDomainIDRange. > >>> > >>>If the re-arrangement does not result in IDs which are outside of this > >>>range give by the ipaDomainIDRange object, no conflicts with idranges > >>>used by trusted domain will occur, because it is one of the task of the > >>>idrange objects to avoid those conflicts. > >>> > >>>If the original given range is exhausted completely and a completely new > >>>DNA range must be created, it should be checked with ipa idrange-find > >>>that the new range is not used and a new ipaDomainIDRange object which > >>>reserves the local range should be added. > >>> > >>>There are https://fedorahosted.org/freeipa/ticket/591 which can be used > >>>to track the coordinated creation of DNA and id-range. > >>> > >> > >>Honestly, I was going to skip the range stuff altogether and just > >>focus on the DNA ranges. > >> > >>I'm primarily interested in saving any range from a server that is > >>going to be deleted. I can save that in the on-base attribute of any > >>existing servers. Where I run into trouble is if all the on-base are > >>populated, then we'd lose range. So I need to alert the user > >>somehow. > > > >If it is only about moving existing DNA range it is fine, because this > >means that DNA will not give ID which are outside of the idrange of the > >IPA domain which is created at installation or during upgrade to v3. > >> > >>This leads to wanting to be able to add that back in so I came up > >>with the idea of letting users manage the DNA range directly. It may > >>be possible to check the DOMAIN ranges as well but that bumps the > >>complexity up considerably. > > > >But it looks like dnarange-set and dnanextrange-set can also be used to > >not only move existing DNA ranges but to create new DNA ranges which > >will lead to ID which are not in the idrange of the IPA domain but might > >be in an idrange which is used by a trusted domain. So I think a check > >and a warning are desirable. > > That is an excellent point. I've updated the design. Thank you. I would like to suggest a slight edit. Instead of "That doesn't remove our responsibility to not test for overlaps in the idranges though. We will need to verify that the manual configuration changes do not overlap with all ranges in cn=ranges,cn=etc,$SUFFIX." I would say "That doesn't remove our responsibility to not test for overlaps in the idranges though. We will need to verify that the manual configuration changes do * not overlap with ranges from trusted domains (objectclass=ipatrustedaddomainrange) in cn=ranges,cn=etc,$SUFFIX, or reject the change otherwise. * overlap completely with ranges from the local IPA domain (objectclass=ipaDomainIDRange) in cn=ranges,cn=etc,$SUFFIX, or give a warning otherwise which asks the user to add a new suitable idrange." Codewise the logic could be: - check if the new range is a subset of a local idrange, if yes, all is fine - if not, check if it overlaps with an idrange of a trusted domain, if yes, reject - if not, give a warning and ask to add a new idrange for the local domain bye, Sumit > > rob > From rcritten at redhat.com Wed Feb 27 20:00:10 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 27 Feb 2013 15:00:10 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <20130227195506.GH3633@localhost.localdomain> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> <20130227173944.GE3633@localhost.localdomain> <512E587C.7040709@redhat.com> <20130227195506.GH3633@localhost.localdomain> Message-ID: <512E65CA.6000800@redhat.com> Sumit Bose wrote: > On Wed, Feb 27, 2013 at 02:03:24PM -0500, Rob Crittenden wrote: >> Sumit Bose wrote: >>> But it looks like dnarange-set and dnanextrange-set can also be used to >>> not only move existing DNA ranges but to create new DNA ranges which >>> will lead to ID which are not in the idrange of the IPA domain but might >>> be in an idrange which is used by a trusted domain. So I think a check >>> and a warning are desirable. >> >> That is an excellent point. I've updated the design. > > Thank you. I would like to suggest a slight edit. Instead of > > "That doesn't remove our responsibility to not test for overlaps in the > idranges though. We will need to verify that the manual configuration > changes do not overlap with all ranges in cn=ranges,cn=etc,$SUFFIX." > > I would say > > "That doesn't remove our responsibility to not test for overlaps in the > idranges though. We will need to verify that the manual configuration > changes do > * not overlap with ranges from trusted domains > (objectclass=ipatrustedaddomainrange) in cn=ranges,cn=etc,$SUFFIX, or > reject the change otherwise. > * overlap completely with ranges from the local IPA domain > (objectclass=ipaDomainIDRange) in cn=ranges,cn=etc,$SUFFIX, or give a > warning otherwise which asks the user to add a new suitable idrange." > > Codewise the logic could be: > - check if the new range is a subset of a local idrange, if yes, all is > fine > - if not, check if it overlaps with an idrange of a trusted domain, if > yes, reject > - if not, give a warning and ask to add a new idrange for the local > domain I took this almost verbatim but I'm more harsh. If the range is outside the local range and not overlapping anything I will still reject it. If we're going to go through the trouble of keeping them in sync we should be consistent. thanks rob From simo at redhat.com Wed Feb 27 20:14:04 2013 From: simo at redhat.com (Simo Sorce) Date: Wed, 27 Feb 2013 15:14:04 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512E65CA.6000800@redhat.com> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> <20130227173944.GE3633@localhost.localdomain> <512E587C.7040709@redhat.com> <20130227195506.GH3633@localhost.localdomain> <512E65CA.6000800@redhat.com> Message-ID: <1361996044.2962.84.camel@willson.li.ssimo.org> On Wed, 2013-02-27 at 15:00 -0500, Rob Crittenden wrote: > Sumit Bose wrote: > > On Wed, Feb 27, 2013 at 02:03:24PM -0500, Rob Crittenden wrote: > >> Sumit Bose wrote: > >>> But it looks like dnarange-set and dnanextrange-set can also be used to > >>> not only move existing DNA ranges but to create new DNA ranges which > >>> will lead to ID which are not in the idrange of the IPA domain but might > >>> be in an idrange which is used by a trusted domain. So I think a check > >>> and a warning are desirable. > >> > >> That is an excellent point. I've updated the design. > > > > Thank you. I would like to suggest a slight edit. Instead of > > > > "That doesn't remove our responsibility to not test for overlaps in the > > idranges though. We will need to verify that the manual configuration > > changes do not overlap with all ranges in cn=ranges,cn=etc,$SUFFIX." > > > > I would say > > > > "That doesn't remove our responsibility to not test for overlaps in the > > idranges though. We will need to verify that the manual configuration > > changes do > > * not overlap with ranges from trusted domains > > (objectclass=ipatrustedaddomainrange) in cn=ranges,cn=etc,$SUFFIX, or > > reject the change otherwise. > > * overlap completely with ranges from the local IPA domain > > (objectclass=ipaDomainIDRange) in cn=ranges,cn=etc,$SUFFIX, or give a > > warning otherwise which asks the user to add a new suitable idrange." > > > > Codewise the logic could be: > > - check if the new range is a subset of a local idrange, if yes, all is > > fine > > - if not, check if it overlaps with an idrange of a trusted domain, if > > yes, reject > > - if not, give a warning and ask to add a new idrange for the local > > domain > > I took this almost verbatim but I'm more harsh. If the range is outside > the local range and not overlapping anything I will still reject it. If the local ranges ^^ the admin may have already added additional ones, so it needs to be plural. > we're going to go through the trouble of keeping them in sync we should > be consistent. Otherwise good plan. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Feb 27 20:17:31 2013 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 27 Feb 2013 15:17:31 -0500 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <1361996044.2962.84.camel@willson.li.ssimo.org> References: <5127D2DF.80809@redhat.com> <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> <20130227173944.GE3633@localhost.localdomain> <512E587C.7040709@redhat.com> <20130227195506.GH3633@localhost.localdomain> <512E65CA.6000800@redhat.com> <1361996044.2962.84.camel@willson.li.ssimo.org> Message-ID: <512E69DB.9010308@redhat.com> Simo Sorce wrote: > On Wed, 2013-02-27 at 15:00 -0500, Rob Crittenden wrote: >> Sumit Bose wrote: >>> On Wed, Feb 27, 2013 at 02:03:24PM -0500, Rob Crittenden wrote: >>>> Sumit Bose wrote: >>>>> But it looks like dnarange-set and dnanextrange-set can also be used to >>>>> not only move existing DNA ranges but to create new DNA ranges which >>>>> will lead to ID which are not in the idrange of the IPA domain but might >>>>> be in an idrange which is used by a trusted domain. So I think a check >>>>> and a warning are desirable. >>>> >>>> That is an excellent point. I've updated the design. >>> >>> Thank you. I would like to suggest a slight edit. Instead of >>> >>> "That doesn't remove our responsibility to not test for overlaps in the >>> idranges though. We will need to verify that the manual configuration >>> changes do not overlap with all ranges in cn=ranges,cn=etc,$SUFFIX." >>> >>> I would say >>> >>> "That doesn't remove our responsibility to not test for overlaps in the >>> idranges though. We will need to verify that the manual configuration >>> changes do >>> * not overlap with ranges from trusted domains >>> (objectclass=ipatrustedaddomainrange) in cn=ranges,cn=etc,$SUFFIX, or >>> reject the change otherwise. >>> * overlap completely with ranges from the local IPA domain >>> (objectclass=ipaDomainIDRange) in cn=ranges,cn=etc,$SUFFIX, or give a >>> warning otherwise which asks the user to add a new suitable idrange." >>> >>> Codewise the logic could be: >>> - check if the new range is a subset of a local idrange, if yes, all is >>> fine >>> - if not, check if it overlaps with an idrange of a trusted domain, if >>> yes, reject >>> - if not, give a warning and ask to add a new idrange for the local >>> domain >> >> I took this almost verbatim but I'm more harsh. If the range is outside >> the local range and not overlapping anything I will still reject it. If > > the local ranges ^^ the admin may have already added additional ones, so > it needs to be plural. > >> we're going to go through the trouble of keeping them in sync we should >> be consistent. > > Otherwise good plan. It is worded better in the wiki to not imply this covers only a single local range. rob From sbose at redhat.com Wed Feb 27 20:17:49 2013 From: sbose at redhat.com (Sumit Bose) Date: Wed, 27 Feb 2013 21:17:49 +0100 Subject: [Freeipa-devel] DESIGN: Recover DNA Ranges In-Reply-To: <512E65CA.6000800@redhat.com> References: <512B5C9B.8020805@redhat.com> <512B627B.8090403@redhat.com> <512B7097.3030809@redhat.com> <512B7143.6010204@redhat.com> <20130226131500.GB3633@localhost.localdomain> <512E1D2D.1090501@redhat.com> <20130227173944.GE3633@localhost.localdomain> <512E587C.7040709@redhat.com> <20130227195506.GH3633@localhost.localdomain> <512E65CA.6000800@redhat.com> Message-ID: <20130227201749.GI3633@localhost.localdomain> On Wed, Feb 27, 2013 at 03:00:10PM -0500, Rob Crittenden wrote: > Sumit Bose wrote: > >On Wed, Feb 27, 2013 at 02:03:24PM -0500, Rob Crittenden wrote: > >>Sumit Bose wrote: > >>>But it looks like dnarange-set and dnanextrange-set can also be used to > >>>not only move existing DNA ranges but to create new DNA ranges which > >>>will lead to ID which are not in the idrange of the IPA domain but might > >>>be in an idrange which is used by a trusted domain. So I think a check > >>>and a warning are desirable. > >> > >>That is an excellent point. I've updated the design. > > > >Thank you. I would like to suggest a slight edit. Instead of > > > >"That doesn't remove our responsibility to not test for overlaps in the > >idranges though. We will need to verify that the manual configuration > >changes do not overlap with all ranges in cn=ranges,cn=etc,$SUFFIX." > > > >I would say > > > >"That doesn't remove our responsibility to not test for overlaps in the > >idranges though. We will need to verify that the manual configuration > >changes do > >* not overlap with ranges from trusted domains > > (objectclass=ipatrustedaddomainrange) in cn=ranges,cn=etc,$SUFFIX, or > > reject the change otherwise. > >* overlap completely with ranges from the local IPA domain > > (objectclass=ipaDomainIDRange) in cn=ranges,cn=etc,$SUFFIX, or give a > > warning otherwise which asks the user to add a new suitable idrange." > > > >Codewise the logic could be: > >- check if the new range is a subset of a local idrange, if yes, all is > > fine > >- if not, check if it overlaps with an idrange of a trusted domain, if > > yes, reject > >- if not, give a warning and ask to add a new idrange for the local > > domain > > I took this almost verbatim but I'm more harsh. If the range is > outside the local range and not overlapping anything I will still > reject it. If we're going to go through the trouble of keeping them > in sync we should be consistent. even better, thank you. bye, Sumit > > thanks > > rob > From mkosek at redhat.com Thu Feb 28 07:44:35 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 28 Feb 2013 08:44:35 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <20130227174826.GF3633@localhost.localdomain> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> <1361972238.2962.54.camel@willson.li.ssimo.org> <20130227174826.GF3633@localhost.localdomain> Message-ID: <512F0AE3.8080000@redhat.com> On 02/27/2013 06:48 PM, Sumit Bose wrote: > On Wed, Feb 27, 2013 at 08:37:18AM -0500, Simo Sorce wrote: >> On Wed, 2013-02-27 at 11:58 +0100, Sumit Bose wrote: >>> On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: >>>> On 02/21/2013 04:24 PM, Sumit Bose wrote: >>>>> Hi, >>>>> >>>>> this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 >>>>> The related design page is >>>>> http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . >>>>> >>>>> It was agreed while discussing the design page that the currently >>>>> hardcoded value for the NFS service will be dropped completely, hence >>>>> the first patch reverts to patch which added the hardcoded value. >>>>> >>>>> The second patch adds the needed attribute to compensate the now missing >>>>> hardcoded value. >>>>> >>>>> In the following three patches the PAC type is read and used >>>>> accordingly. The last patch adds a unit test for a new function. >>>>> >>>>> bye, >>>>> Sumit >>>> >>>> I did only sanity testing to the C part of the RFE so far, but by reading it it >>>> looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part >>>> is OK. >>> >>> Alexander asked in irc to change strcmp() to strcasecmp() so that >>> options like 'Ms-Pac' or 'none' are accepted as well. >> >> Is there a good reason to allow insensitive matching ? >> >> in LDAP you can't use true or false either for booleans, you have to use >> TRUE and FALSE, so I am not so thrilled to allow insensitive matching >> for this option as well. > > I'm fine with this. Alexander, any comments. > >> >>>> I will comment on the Python parts: >>>> >>>> 1) Your update check testing if there is any NFS service with ipakrbauthzdata >>>> looks like a good heuristics to prevent admin frustration. Though an ideal >>>> solution would require >>>> https://fedorahosted.org/freeipa/ticket/2961 >>>> to prevent multiple updates when admin purposefully removes this attribute. We >>>> may want to reference the ticket in a comment in the update plugin... >>> >>> I added a comment in the code and in #2961. >>> >>>> >>>> >>>> 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns >>>> truncated result. As you do not update entries directly but only return update >>>> instructions when you have no truncated results, you will loop. >>>> >>>> To simulate this, I just created 2 NFS principals and set size_limit=1 in your >>>> find_entries call. >>> >>> Thanks for catching this, I removed the truncated logic and added a >>> messages if truncated was set. >>> >>>> >>>> >>>> 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS >>>> service principals added by service-add? Shouldn't we set ipakrbauthzdata for >>>> new services too? As a default when no user ipakrhbauthzdata is set... >>>> Otherwise admins could be surprised why their new NFS services do not work >>>> while the others do. >>>> >>>> Also, I think we should have this NFS culprit documented somewhere (i.e. why we >>>> set ipakrbauthzdata to NONE by default). At least as a small section in the >>>> online help for services plugin... >>> >>> I added comments to the help and the doc string in patch #100. If you >>> think it is necessary to implicitly set PAC type to 'NONE' if NFS >>> services are added and no type is given explicitly, I would prefer if >>> you can open a new tickets so that it can be discussed independently. >>> >>> Thank you for the review. New versions attached. >> >> I do not think we should set the policy to NONE by default, OTOH I see >> the problem with upgrades. Maybe we should have a way to express >> additional defaults, per service type. >> >> Ie add additional values like: >> >> ipakrbAuthsData: nfs:NONE >> >> This would be a default but only for services that have the form nfs/*@* >> >> This would allow us to avoid magical special casing in the code and >> allow admins to change the overall default for specific services as >> needed. >> >> Maybe we should add a new ticket for this ? > > This sounds like a good compromise and will make updating much easier. > If we agree to do it this way I can add ipakrbAuthsData: nfs:NONE and > fix the code with this ticket. But we would need tickets for the CLI and > WebUI to handle this new case. > > For the WebUI there already is > https://fedorahosted.org/freeipa/ticket/3404 , maybe it can be extended > to handle this new case as well? > > As for design documents I think I can added the needed information to > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > bye, > Sumit Hi Sumit, This looks like a good idea and would prevent the magic default PAC type, yes. Though I would not add this service-specific setting to global IPA config object. I would rather like to see that in the service tree, for example as a configuration option of the service root which could be controlled with serviceconfig-* commands (we already have dnsconfig, trustconfig), e.g: # ipa serviceconfig-add-pacmap --service=nfs --pac-type=NONE # ipa serviceconfig-add-pacmap --service=cifs --pac-type=PAD # ipa serviceconfig-show Default PAC Map: nfs:NONE, cifs:PAD Martin From mkosek at redhat.com Thu Feb 28 11:35:17 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 28 Feb 2013 12:35:17 +0100 Subject: [Freeipa-devel] [PATCHES] 101-107 Fixes for various Coverity issues In-Reply-To: <512E023F.9000106@redhat.com> References: <20130227113536.GD3633@localhost.localdomain> <512DFE83.6060009@redhat.com> <512E023F.9000106@redhat.com> Message-ID: <512F40F5.2090507@redhat.com> On 02/27/2013 01:55 PM, Martin Kosek wrote: > On 02/27/2013 01:39 PM, Martin Kosek wrote: >> On 02/27/2013 12:35 PM, Sumit Bose wrote: >>> Hi, >>> >>> the attached patches 102-107 fix issues found by Coverity which are >>> tracked by tickets #3422-#3427 and remove an unused variable >>> (patch 101). >>> >>> bye, >>> Sumit >>> >> >> I see just one issue. In patch 0105: >> >> - global_ipactx = (struct ipa_context *)malloc(sizeof(global_ipactx)); >> + global_ipactx = (struct ipa_context *)malloc(sizeof(*global_ipactx)); >> >> I do not think this will work right. *global_ipactx will just de-reference >> global_ipactxt and run sizeof on the result, right? >> >> I would prefer this change: >> >> global_ipactx = (struct ipa_context *)malloc(sizeof(struct ipa_context *)); >> >> Martin >> > > We just discussed this in Brno - in fact your change should be right as you > allocate memory for the whole structure and not just a pointer to the structure > (though I personally would prefer sizeof(struct ipa_context) as it is clearer). > > Hmrph, C language 101 revisited :-) > > Martin > Good, my tests were OK with your patchset. ACK for all patches, pushed to master, ipa-3-1. Martin From sbose at redhat.com Thu Feb 28 11:42:05 2013 From: sbose at redhat.com (Sumit Bose) Date: Thu, 28 Feb 2013 12:42:05 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <512F0AE3.8080000@redhat.com> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> <1361972238.2962.54.camel@willson.li.ssimo.org> <20130227174826.GF3633@localhost.localdomain> <512F0AE3.8080000@redhat.com> Message-ID: <20130228114205.GL3633@localhost.localdomain> On Thu, Feb 28, 2013 at 08:44:35AM +0100, Martin Kosek wrote: > On 02/27/2013 06:48 PM, Sumit Bose wrote: > > On Wed, Feb 27, 2013 at 08:37:18AM -0500, Simo Sorce wrote: > >> On Wed, 2013-02-27 at 11:58 +0100, Sumit Bose wrote: > >>> On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: > >>>> On 02/21/2013 04:24 PM, Sumit Bose wrote: > >>>>> Hi, > >>>>> > >>>>> this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 > >>>>> The related design page is > >>>>> http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > >>>>> > >>>>> It was agreed while discussing the design page that the currently > >>>>> hardcoded value for the NFS service will be dropped completely, hence > >>>>> the first patch reverts to patch which added the hardcoded value. > >>>>> > >>>>> The second patch adds the needed attribute to compensate the now missing > >>>>> hardcoded value. > >>>>> > >>>>> In the following three patches the PAC type is read and used > >>>>> accordingly. The last patch adds a unit test for a new function. > >>>>> > >>>>> bye, > >>>>> Sumit > >>>> > >>>> I did only sanity testing to the C part of the RFE so far, but by reading it it > >>>> looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part > >>>> is OK. > >>> > >>> Alexander asked in irc to change strcmp() to strcasecmp() so that > >>> options like 'Ms-Pac' or 'none' are accepted as well. > >> > >> Is there a good reason to allow insensitive matching ? > >> > >> in LDAP you can't use true or false either for booleans, you have to use > >> TRUE and FALSE, so I am not so thrilled to allow insensitive matching > >> for this option as well. > > > > I'm fine with this. Alexander, any comments. > > > >> > >>>> I will comment on the Python parts: > >>>> > >>>> 1) Your update check testing if there is any NFS service with ipakrbauthzdata > >>>> looks like a good heuristics to prevent admin frustration. Though an ideal > >>>> solution would require > >>>> https://fedorahosted.org/freeipa/ticket/2961 > >>>> to prevent multiple updates when admin purposefully removes this attribute. We > >>>> may want to reference the ticket in a comment in the update plugin... > >>> > >>> I added a comment in the code and in #2961. > >>> > >>>> > >>>> > >>>> 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns > >>>> truncated result. As you do not update entries directly but only return update > >>>> instructions when you have no truncated results, you will loop. > >>>> > >>>> To simulate this, I just created 2 NFS principals and set size_limit=1 in your > >>>> find_entries call. > >>> > >>> Thanks for catching this, I removed the truncated logic and added a > >>> messages if truncated was set. > >>> > >>>> > >>>> > >>>> 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS > >>>> service principals added by service-add? Shouldn't we set ipakrbauthzdata for > >>>> new services too? As a default when no user ipakrhbauthzdata is set... > >>>> Otherwise admins could be surprised why their new NFS services do not work > >>>> while the others do. > >>>> > >>>> Also, I think we should have this NFS culprit documented somewhere (i.e. why we > >>>> set ipakrbauthzdata to NONE by default). At least as a small section in the > >>>> online help for services plugin... > >>> > >>> I added comments to the help and the doc string in patch #100. If you > >>> think it is necessary to implicitly set PAC type to 'NONE' if NFS > >>> services are added and no type is given explicitly, I would prefer if > >>> you can open a new tickets so that it can be discussed independently. > >>> > >>> Thank you for the review. New versions attached. > >> > >> I do not think we should set the policy to NONE by default, OTOH I see > >> the problem with upgrades. Maybe we should have a way to express > >> additional defaults, per service type. > >> > >> Ie add additional values like: > >> > >> ipakrbAuthsData: nfs:NONE > >> > >> This would be a default but only for services that have the form nfs/*@* > >> > >> This would allow us to avoid magical special casing in the code and > >> allow admins to change the overall default for specific services as > >> needed. > >> > >> Maybe we should add a new ticket for this ? > > > > This sounds like a good compromise and will make updating much easier. > > If we agree to do it this way I can add ipakrbAuthsData: nfs:NONE and > > fix the code with this ticket. But we would need tickets for the CLI and > > WebUI to handle this new case. > > > > For the WebUI there already is > > https://fedorahosted.org/freeipa/ticket/3404 , maybe it can be extended > > to handle this new case as well? > > > > As for design documents I think I can added the needed information to > > http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . > > > > bye, > > Sumit > > Hi Sumit, > > This looks like a good idea and would prevent the magic default PAC type, yes. > Though I would not add this service-specific setting to global IPA config object. > > I would rather like to see that in the service tree, for example as a > configuration option of the service root which could be controlled with > serviceconfig-* commands (we already have dnsconfig, trustconfig), e.g: > > # ipa serviceconfig-add-pacmap --service=nfs --pac-type=NONE > # ipa serviceconfig-add-pacmap --service=cifs --pac-type=PAD > # ipa serviceconfig-show > Default PAC Map: nfs:NONE, cifs:PAD Are you thinking of having this in addition to the for-all-services default values in cn=ipaConfig,cn=etc or shall those be dropped? I don't like the first case because then three different objects needs to be consulted to find out which is the right type. This wouldn't be an issue for the plugin, but I think it is hard for the user/admin to follow. If the current defaults shall be dropped I think this is a major change because it will require changes in the current CLI and WebUI which will be visible to the users. I'm not against this change, I'm just wondering if it is worth the effort for the next release? Maybe an argument to keep this is in global default is that the settings are used for the host/*.* services as well which are in a different sub-tree of the cn=accounts container. Additionally in future we might want apply those setting to the user TGTs as well? bye, Sumit > > Martin From mkosek at redhat.com Thu Feb 28 12:02:54 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 28 Feb 2013 13:02:54 +0100 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <20130228114205.GL3633@localhost.localdomain> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> <1361972238.2962.54.camel@willson.li.ssimo.org> <20130227174826.GF3633@localhost.localdomain> <512F0AE3.8080000@redhat.com> <20130228114205.GL3633@localhost.localdomain> Message-ID: <512F476E.9050501@redhat.com> On 02/28/2013 12:42 PM, Sumit Bose wrote: > On Thu, Feb 28, 2013 at 08:44:35AM +0100, Martin Kosek wrote: >> On 02/27/2013 06:48 PM, Sumit Bose wrote: >>> On Wed, Feb 27, 2013 at 08:37:18AM -0500, Simo Sorce wrote: >>>> On Wed, 2013-02-27 at 11:58 +0100, Sumit Bose wrote: >>>>> On Mon, Feb 25, 2013 at 04:35:20PM +0100, Martin Kosek wrote: >>>>>> On 02/21/2013 04:24 PM, Sumit Bose wrote: >>>>>>> Hi, >>>>>>> >>>>>>> this series of patches fix https://fedorahosted.org/freeipa/ticket/2960 >>>>>>> The related design page is >>>>>>> http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . >>>>>>> >>>>>>> It was agreed while discussing the design page that the currently >>>>>>> hardcoded value for the NFS service will be dropped completely, hence >>>>>>> the first patch reverts to patch which added the hardcoded value. >>>>>>> >>>>>>> The second patch adds the needed attribute to compensate the now missing >>>>>>> hardcoded value. >>>>>>> >>>>>>> In the following three patches the PAC type is read and used >>>>>>> accordingly. The last patch adds a unit test for a new function. >>>>>>> >>>>>>> bye, >>>>>>> Sumit >>>>>> >>>>>> I did only sanity testing to the C part of the RFE so far, but by reading it it >>>>>> looks ok. It may be beneficial if Simo or Alexander checked if the ipa-kdb part >>>>>> is OK. >>>>> >>>>> Alexander asked in irc to change strcmp() to strcasecmp() so that >>>>> options like 'Ms-Pac' or 'none' are accepted as well. >>>> >>>> Is there a good reason to allow insensitive matching ? >>>> >>>> in LDAP you can't use true or false either for booleans, you have to use >>>> TRUE and FALSE, so I am not so thrilled to allow insensitive matching >>>> for this option as well. >>> >>> I'm fine with this. Alexander, any comments. >>> >>>> >>>>>> I will comment on the Python parts: >>>>>> >>>>>> 1) Your update check testing if there is any NFS service with ipakrbauthzdata >>>>>> looks like a good heuristics to prevent admin frustration. Though an ideal >>>>>> solution would require >>>>>> https://fedorahosted.org/freeipa/ticket/2961 >>>>>> to prevent multiple updates when admin purposefully removes this attribute. We >>>>>> may want to reference the ticket in a comment in the update plugin... >>>>> >>>>> I added a comment in the code and in #2961. >>>>> >>>>>> >>>>>> >>>>>> 2) The upgrade plugin may get into infinite loop if ldap.find_entries returns >>>>>> truncated result. As you do not update entries directly but only return update >>>>>> instructions when you have no truncated results, you will loop. >>>>>> >>>>>> To simulate this, I just created 2 NFS principals and set size_limit=1 in your >>>>>> find_entries call. >>>>> >>>>> Thanks for catching this, I removed the truncated logic and added a >>>>> messages if truncated was set. >>>>> >>>>>> >>>>>> >>>>>> 3) Ok, we set ipakrbauthzdata to NONE on upgrades. But what about new NFS >>>>>> service principals added by service-add? Shouldn't we set ipakrbauthzdata for >>>>>> new services too? As a default when no user ipakrhbauthzdata is set... >>>>>> Otherwise admins could be surprised why their new NFS services do not work >>>>>> while the others do. >>>>>> >>>>>> Also, I think we should have this NFS culprit documented somewhere (i.e. why we >>>>>> set ipakrbauthzdata to NONE by default). At least as a small section in the >>>>>> online help for services plugin... >>>>> >>>>> I added comments to the help and the doc string in patch #100. If you >>>>> think it is necessary to implicitly set PAC type to 'NONE' if NFS >>>>> services are added and no type is given explicitly, I would prefer if >>>>> you can open a new tickets so that it can be discussed independently. >>>>> >>>>> Thank you for the review. New versions attached. >>>> >>>> I do not think we should set the policy to NONE by default, OTOH I see >>>> the problem with upgrades. Maybe we should have a way to express >>>> additional defaults, per service type. >>>> >>>> Ie add additional values like: >>>> >>>> ipakrbAuthsData: nfs:NONE >>>> >>>> This would be a default but only for services that have the form nfs/*@* >>>> >>>> This would allow us to avoid magical special casing in the code and >>>> allow admins to change the overall default for specific services as >>>> needed. >>>> >>>> Maybe we should add a new ticket for this ? >>> >>> This sounds like a good compromise and will make updating much easier. >>> If we agree to do it this way I can add ipakrbAuthsData: nfs:NONE and >>> fix the code with this ticket. But we would need tickets for the CLI and >>> WebUI to handle this new case. >>> >>> For the WebUI there already is >>> https://fedorahosted.org/freeipa/ticket/3404 , maybe it can be extended >>> to handle this new case as well? >>> >>> As for design documents I think I can added the needed information to >>> http://freeipa.org/page/V3/Read_and_use_per_service_pac_type . >>> >>> bye, >>> Sumit >> >> Hi Sumit, >> >> This looks like a good idea and would prevent the magic default PAC type, yes. >> Though I would not add this service-specific setting to global IPA config object. >> >> I would rather like to see that in the service tree, for example as a >> configuration option of the service root which could be controlled with >> serviceconfig-* commands (we already have dnsconfig, trustconfig), e.g: >> >> # ipa serviceconfig-add-pacmap --service=nfs --pac-type=NONE >> # ipa serviceconfig-add-pacmap --service=cifs --pac-type=PAD >> # ipa serviceconfig-show >> Default PAC Map: nfs:NONE, cifs:PAD > > Are you thinking of having this in addition to the for-all-services > default values in cn=ipaConfig,cn=etc or shall those be dropped? I don't > like the first case because then three different objects needs to be > consulted to find out which is the right type. This wouldn't be an issue > for the plugin, but I think it is hard for the user/admin to follow. Hm, you are right. > > If the current defaults shall be dropped I think this is a major change > because it will require changes in the current CLI and WebUI which will > be visible to the users. I'm not against this change, I'm just wondering > if it is worth the effort for the next release? > > Maybe an argument to keep this is in global default is that the settings > are used for the host/*.* services as well which are in a different > sub-tree of the cn=accounts container. Additionally in future we might > want apply those setting to the user TGTs as well? Yeah, that was actually my point. That we are mixing service-specific PAC "rules" to the global setting. Which may be shared with host/*.* principals and user principals. This automatic PAC rules may require some designing so that is is generally usable. As for your current patch set, I just finished my tests (Kerberos secured NFS share, accessed from Linux/AD with Trust) and it worked fine... Though in my case I did not have to disable PAC for the NFS service principal - its size probably did not exceed the kernel ticket size limit. I am now just concerned to this error message: + root_logger.info("search results for NFS services have been " + "truncated by the server; please check manually " + "if the PAC type is set correctly for all NFS " + "services") I am thinking we may want to increase the error message to ERROR so that it is visible during rpm update. Otherwise the error message could get lost easily. Martin From mkosek at redhat.com Thu Feb 28 14:27:00 2013 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 28 Feb 2013 15:27:00 +0100 Subject: [Freeipa-devel] [PATCH] 375 Fix includedir directive in krb5.conf template Message-ID: <512F6934.1040702@redhat.com> We did not have the includedir directory with a trailing slash which made rpm update add a redundant line. https://fedorahosted.org/freeipa/ticket/3132 --- Pushed as a one-liner (with Rob's IRC blessing) to master, ipa-3-1, ipa-3-0. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-375-fix-includedir-directive-in-krb5.conf-template.patch Type: text/x-patch Size: 892 bytes Desc: not available URL: From simo at redhat.com Thu Feb 28 14:28:26 2013 From: simo at redhat.com (Simo Sorce) Date: Thu, 28 Feb 2013 09:28:26 -0500 Subject: [Freeipa-devel] [PATCHES] 94-99 Read and use per-service PAC type In-Reply-To: <512F476E.9050501@redhat.com> References: <20130221152446.GS27418@localhost.localdomain> <512B84B8.7080506@redhat.com> <20130227105831.GC3633@localhost.localdomain> <1361972238.2962.54.camel@willson.li.ssimo.org> <20130227174826.GF3633@localhost.localdomain> <512F0AE3.8080000@redhat.com> <20130228114205.GL3633@localhost.localdomain> <512F476E.9050501@redhat.com> Message-ID: <1362061706.2962.93.camel@willson.li.ssimo.org> On Thu, 2013-02-28 at 13:02 +0100, Martin Kosek wrote: > On 02/28/2013 12:42 PM, Sumit Bose wrote: > > On Thu, Feb 28, 2013 at 08:44:35AM +0100, Martin Kosek wrote: > >> On 02/27/2013 06:48 PM, Sumit Bose wrote: > >> Hi Sumit, > >> > >> This looks like a good idea and would prevent the magic default PAC type, yes. > >> Though I would not add this service-specific setting to global IPA config object. > >> > >> I would rather like to see that in the service tree, for example as a > >> configuration option of the service root which could be controlled with > >> serviceconfig-* commands (we already have dnsconfig, trustconfig), e.g: > >> > >> # ipa serviceconfig-add-pacmap --service=nfs --pac-type=NONE > >> # ipa serviceconfig-add-pacmap --service=cifs --pac-type=PAD > >> # ipa serviceconfig-show > >> Default PAC Map: nfs:NONE, cifs:PAD > > > > Are you thinking of having this in addition to the for-all-services > > default values in cn=ipaConfig,cn=etc or shall those be dropped? I don't > > like the first case because then three different objects needs to be > > consulted to find out which is the right type. This wouldn't be an issue > > for the plugin, but I think it is hard for the user/admin to follow. > > Hm, you are right. > > > > > If the current defaults shall be dropped I think this is a major change > > because it will require changes in the current CLI and WebUI which will > > be visible to the users. I'm not against this change, I'm just wondering > > if it is worth the effort for the next release? > > > > Maybe an argument to keep this is in global default is that the settings > > are used for the host/*.* services as well which are in a different > > sub-tree of the cn=accounts container. Additionally in future we might > > want apply those setting to the user TGTs as well? > > Yeah, that was actually my point. That we are mixing service-specific PAC > "rules" to the global setting. Which may be shared with host/*.* principals and > user principals. This automatic PAC rules may require some designing so that is > is generally usable. I think putting everything in the general config is more understandable and discoverable. These per-service defaults are basically exceptions to the general rule so it make sense to keep everything together. Simo. -- Simo Sorce * Red Hat, Inc * New York