From mkosek at redhat.com Mon Jul 2 06:16:09 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 08:16:09 +0200 Subject: [Freeipa-devel] [PATCH] 1030 Fedora 18 compatibility In-Reply-To: <4FEDC4A0.3070800@redhat.com> References: <4FEB4711.9010603@redhat.com> <4FEC1764.2020809@redhat.com> <4FEDC4A0.3070800@redhat.com> Message-ID: <4FF13CA9.6050602@redhat.com> On 06/29/2012 05:07 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 06/27/2012 07:46 PM, Rob Crittenden wrote: >>> I found a few minor issues when building and installing the master branch on >>> Fedora 18. This patch should address it. >>> >>> rob >>> >> >> 1) This will fail for on F17->F18 upgrades, we need to bump VERSION in >> ipa-rewrite.conf. >> >> Besides that, ipa-upgradeconfig needs to be fixed, otherwise it will crash >> during ipa-rewrite.conf upgrade - ${AUTOREDIR} variable is not set. >> >> However, this variable will need to be figured out from current >> ipa-rewrite.conf contents as it depends on whether the IPA server was installed >> with --no-ui-redirect or not. >> >> 2) Shouldn't we bump tomcat6 version as well since we depend on the tomcat6 >> fixed in BZ 831464? >> >> 3) %changelog entry is missing >> >> Martin >> > > This should do it > > rob This looks as a way to go, but this one won't fly yet - the server FQDN is hard-coded to the find_autoredirect function. Martin From mkosek at redhat.com Mon Jul 2 06:39:28 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 08:39:28 +0200 Subject: [Freeipa-devel] [PATCH] 0067 Explicitly filter options that permission-{add, mod} passes to aci-{add, mod} In-Reply-To: <20120629125724.GJ6687@redhat.com> References: <4FED958A.2010504@redhat.com> <20120629125724.GJ6687@redhat.com> Message-ID: <4FF14220.8030904@redhat.com> On 06/29/2012 02:57 PM, Alexander Bokovoy wrote: > On Fri, 29 Jun 2012, Petr Viktorin wrote: >> The permission commands were not filtering their options properly before >> passing them to the underlying ACI commands. This upset the new input >> validation when --addattr/--setattr was used. >> >> This patch adds a filter that only lets options listed in aci_attributes >> through to the ACI commands. >> >> https://fedorahosted.org/freeipa/ticket/2885 > ACK. > Pushed to master. Martin From mkosek at redhat.com Mon Jul 2 06:45:11 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 08:45:11 +0200 Subject: [Freeipa-devel] [PATCH] [WIP] 281 Enable SOA serial autoincrement In-Reply-To: <4FEDFB47.40504@redhat.com> References: <4FEC7314.9030508@redhat.com> <4FEDFB47.40504@redhat.com> Message-ID: <4FF14377.80609@redhat.com> On 06/29/2012 09:00 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> This patch enables currently developed SOA serial autoincrement feature in >> bind-dyndb-ldap. The patch may be updated if any assumptions about this feature >> are changed (or somebody finds a bug). >> >> --- >> >> SOA serial autoincrement is a requirement for major DNS features, >> e.g. zone transfers or DNSSEC. Enable it by default in named.conf >> both for new and upgraded installations. Name of the bind-dyndb-ldap >> option is "serial_autoincrement". >> >>> From now on, idnsSOAserial attribute also has to be put to >> replication agreement exclude list as serial will be incremented >> on each DNS server separately and won't be shared. Exclude list >> has to be updated both for new replication agreements and the >> current ones. >> >> https://fedorahosted.org/freeipa/ticket/2554 > > What version of bind/bind-dyndb-ldap is needed for serial_autoincrement? > > rob Such version is not ready yet, there is only a semi-working patch from Petr Spacek on freeipa-devel list. When a working version of bind-dyndb-ldap package with working serial_autoincrement feature, it should be enough to simply bump package version in bind-dyndb-ldap (that's why I tagged this patch as [WIP]). But otherwise, this patch is reviewable, it should prepare our install tools for the new feature, turn it on in named.conf on upgrades and also update replication agreements to not replicate SOA serial from now on. Martin From pvoborni at redhat.com Mon Jul 2 07:55:04 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 02 Jul 2012 09:55:04 +0200 Subject: [Freeipa-devel] [PATCH] 165 Display loginas information only after login In-Reply-To: <4FECC2DF.4030200@redhat.com> References: <4FEC6506.7050205@redhat.com> <4FECC2DF.4030200@redhat.com> Message-ID: <4FF153D8.70206@redhat.com> On 06/28/2012 10:47 PM, Endi Sukma Dewata wrote: > On 6/28/2012 9:07 AM, Petr Vobornik wrote: >> Message 'Logged in as: user at FREEIPA.ORG' was displayed before user was >> logged in. It was wrong. >> >> Now 'Logged in as: XXX' is displayed only when user XXX is logged in. So >> no more user at FREEIPA.ORG :) . >> >> https://fedorahosted.org/freeipa/ticket/2882 > > It might be better to use visibility instead of display to reserve the > space. Right now the password expiration warning will initially appear > on the right, then shift to the left when the "Logged in as" appears. > Seems like better approach. Updated patch attached. Another improvement might be: display password expiration warning at the same time as login information. What do you think? Does it matter? -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0165-1-Display-loginas-information-only-after-login.patch Type: text/x-patch Size: 2035 bytes Desc: not available URL: From mkosek at redhat.com Mon Jul 2 07:55:36 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 09:55:36 +0200 Subject: [Freeipa-devel] [PATCHES] 22-24 Add initial support for ID ranges In-Reply-To: <4FEE25CF.5070302@redhat.com> References: <20120613191726.GI20850@localhost.localdomain> <1339634303.8230.634.camel@willson.li.ssimo.org> <20120614103538.GJ20850@localhost.localdomain> <1339674880.8230.674.camel@willson.li.ssimo.org> <20120614122501.GK20850@localhost.localdomain> <20120617194720.GB29454@localhost.localdomain> <20120626103014.GC29454@localhost.localdomain> <20120627191935.GL29454@localhost.localdomain> <20120629121731.GC922@localhost.localdomain> <20120629125215.GI6687@redhat.com> <4FEDFF17.6030300@redhat.com> <4FEE124D.4090401@redhat.com> <4FEE25CF.5070302@redhat.com> Message-ID: <4FF153F8.3000709@redhat.com> On 06/30/2012 12:01 AM, Rob Crittenden wrote: > Rob Crittenden wrote: >> Rob Crittenden wrote: >>> Alexander Bokovoy wrote: >>>> On Fri, 29 Jun 2012, Sumit Bose wrote: >>>>> On Wed, Jun 27, 2012 at 09:19:36PM +0200, Sumit Bose wrote: >>>>>> On Tue, Jun 26, 2012 at 12:30:14PM +0200, Sumit Bose wrote: >>>>>> > On Sun, Jun 17, 2012 at 09:47:20PM +0200, Sumit Bose wrote: >>>>>> > > On Thu, Jun 14, 2012 at 02:25:01PM +0200, Sumit Bose wrote: >>>>>> > > > On Thu, Jun 14, 2012 at 07:54:40AM -0400, Simo Sorce wrote: >>>>>> > > > > On Thu, 2012-06-14 at 12:35 +0200, Sumit Bose wrote: >>>>>> > > > > > On Wed, Jun 13, 2012 at 08:38:23PM -0400, Simo Sorce wrote: >>>>>> > > > > > > On Wed, 2012-06-13 at 21:17 +0200, Sumit Bose wrote: >>>>>> > > > > > > > >>>>>> > > > > > > > to keep track of the different ranges we use for >>>>>> UIDs/GIDs for local >>>>>> > > > > > > > users/groups and users from trusted domains new range >>>>>> objects are >>>>>> > > > > > > > introduced which are stored below >>>>>> cn=range,cn=etc,$SUFFIX. >>>>>> > > > > > > > >>>>>> > > > > > > > 0022: LDAP schema update >>>>>> > > > > > > >>>>>> > > > > > > ack >>>>>> > > > > > > >>>>>> > > > > > > > 0023: Create a range object during installation fir the >>>>>> local ID range >>>>>> > > > > > > >>>>>> > > > > > > nack, I think we need to find a way to handle adding at >>>>>> least the base >>>>>> > > > > > > range on update. Otherwise an updated server won't be >>>>>> able to have IDs >>>>>> > > > > > > for most of its users. >>>>>> > > > > > >>>>>> > > > > > I fully agree, but since we said that we concentrate on >>>>>> update issues in >>>>>> > > > > > beta2 I wanted to send the version for the fresh install >>>>>> first to allow >>>>>> > > > > > testing. >>>>>> > > > > >>>>>> > > > > The reason I'd like updates is that this patchset can be >>>>>> installed on >>>>>> > > > > top of existing servers for testing w/o having to reinstall >>>>>> from scratch >>>>>> > > > > or manually creating the ipaDomainIDRange object :):) >>>>>> > > > >>>>>> > > > ok, will do. >>>>>> > > > >>>>>> > > > Do you otherwise agree with the patches or is there something I >>>>>> should >>>>>> > > > change while adding the updates? >>>>>> > > > >>>>>> > > > bye, >>>>>> > > > Sumit >>>>>> > > > >>>>>> > > > > >>>>>> > > > > > > >>>>>> > > > > > > > 0024: add primary and secondary RID base to the local >>>>>> range object >>>>>> > > > > > > > during ipa-adtrust-install >>>>>> > > > > > > >>>>>> > > > > > > Not sure if setting the range belongs in the previous >>>>>> patch or this one. >>>>>> > > > > > >>>>>> > > > > > I think it is right here, because a plain IPA server does >>>>>> not need the >>>>>> > > > > > RID related attributes. >>>>>> > > > > > >>>>>> > > > > > > We might decide to ask questions during >>>>>> ipa-adtrust-install if the range >>>>>> > > > > > > is not available, maybe presenting a set of pre-canned >>>>>> choices if we can >>>>>> > > > > > > detect them. >>>>>> > > > > > >>>>>> > > > > > I agree here, too. But as above I would like to handle >>>>>> update issues >>>>>> > > > > > in a second round. >>>>>> > > > > > >>>>>> > > > > > > >>>>>> > > > > > > Finally I think we need to do a search with uid/gidNmber >>>>>> < base and >>>>>> > > > > > > uid/gidNumber > max and prompt/warn the user if we detect >>>>>> any ID the >>>>>> > > > > > > falls outside the configured range (either because we >>>>>> failed to detect >>>>>> > > > > > > ranges on upgrade and the user botched the question or >>>>>> because the admin >>>>>> > > > > > > added arbitrary IDs. >>>>>> > > > > > > If a warning we should warn that missing a range that >>>>>> suitably covers >>>>>> > > > > > > these IDs, those users/groups will not be available for >>>>>> the trust. >>>>>> > > > > > > >>>>>> > > > > > > Maybe we should also have a simple ipa command that can >>>>>> list all >>>>>> > > > > > > users/groups that fall outside the ranges as well. >>>>>> > > > > > >>>>>> > > > > > I'm working on the ranges cli plugin to allow 'ipa >>>>>> range-add', 'ipa >>>>>> > > > > > range-find' etc. I can add it there. >>>>>> > > > > > >>>>>> > > >>>>>> > > Hi, >>>>>> > > >>>>>> > > this new series of patches add the cli plugin to create the ID >>>>>> ranges >>>>>> > > manually. I'm still working on a detection of the locally used id >>>>>> range >>>>>> > > of an upgrade domain in ipa-adtrust-install and an plugin which >>>>>> rejects >>>>>> > > new ranges which overlaps with existing ones. >>>>>> > > >>>>>> > > bye, >>>>>> > > Sumit >>>>>> > >>>>>> > the attached patch adds a preop plugin which checks for overlaps >>>>>> with >>>>>> > existing ranges. >>>>>> > >>>>>> > bye, >>>>>> > Sumit >>>>>> >>>>>> Finally I added a method to guess and create the initial ID range, >>>>>> if no >>>>>> one is preset, e.g. when updating from an older version of freeIPA. A >>>>>> full series of patches is attached. >>>>>> >>>>>> bye, >>>>>> Sumit >>>>> >>>>> This version of patches fixes review comments by Alexander and also >>>>> adds >>>>> some test for the range CLI plugin which were kindly provided by >>>>> Alexander. >>>> ACK >>>> >>> >>> These patches aren't applying for me. >>> >>> rob >> >> Hmm. Pulled a fresh tree and they imported fine. >> >> pushed to master >> >> rob > > I had only pushed 22-24 before, pushed 25 and 29 as well. > > rob > I examined the latest changes and found several rather serious issues which will break this functionality on upgraded servers: https://fedorahosted.org/freeipa/ticket/2891 Martin From KECKEL at de.ibm.com Mon Jul 2 10:16:31 2012 From: KECKEL at de.ibm.com (Klaus Eckel) Date: Mon, 2 Jul 2012 12:16:31 +0200 Subject: [Freeipa-devel] [PATCHES] 22-24 Add initial support for ID ranges In-Reply-To: <4FF153F8.3000709@redhat.com> References: <20120613191726.GI20850@localhost.localdomain> <1339634303.8230.634.camel@willson.li.ssimo.org> <20120614103538.GJ20850@localhost.localdomain> <1339674880.8230.674.camel@willson.li.ssimo.org> <20120614122501.GK20850@localhost.localdomain> <20120617194720.GB29454@localhost.localdomain> <20120626103014.GC29454@localhost.localdomain> <20120627191935.GL29454@localhost.localdomain> <20120629121731.GC922@localhost.localdomain> <20120629125215.GI6687@redhat.com> <4FEDFF17.6030300@redhat.com> <4FEE124D.4090401@redhat.com> <4FEE25CF.5070302@redhat.com> <4FF153F8.3000709@redhat.com> Message-ID: hi all, when I tried to install FreeIPA 2.99.0 on Fedora 17 I got the following error: [root at linux yum.repos.d]# cat ipa-devel.repo [ipa-devel] name=IPA development $releasever - $basearch baseurl=http://jdennis.fedorapeople.org/ipa-devel/fedora/$releasever/$basearch/os/ enabled=1 gpgcheck=0 new yum update .. [root at linux yum.repos.d]# uname -a Linux linux.fritz.box 3.4.4-3.fc17.x86_64 #1 SMP Tue Jun 26 20:54:56 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux freeipa-server-2.99.0-0.20120630T2358Zgit50ebd1a.fc17.x86_64.. ipa-server-install -a ###t --hostname=linux.fritz.box -r fritz.box -p ###### -n fritz.box -U [21/36]: adding default layout Unexpected error - see /var/log/ipaserver-install.log for details: KeyError: 'REALM_id_range' log .. 2012-07-02T10:07:32Z DEBUG [21/36]: adding default layout 2012-07-02T10:07:32Z INFO File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 696, in run_script return_value = main_function() File "/sbin/ipa-server-install", line 958, in main hbac_allow=not options.hbac_allow) File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 249, in create_instance self.start_creation("Configuring directory server", 60) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 259, in start_creation method() File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 569, in __add_default_layout self._ldap_mod("bootstrap-template.ldif", self.sub_dict) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 98, in _ldap_mod txt = ipautil.template_file(path, sub_dict) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 218, in template_file return template_str(txt, vars) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 206, in template_str val = string.Template(txt).substitute(vars) File "/usr/lib64/python2.7/string.py", line 172, in substitute return self.pattern.sub(convert, self.template) File "/usr/lib64/python2.7/string.py", line 162, in convert val = mapping[named] 2012-07-02T10:07:32Z INFO The ipa-server-install command failed, exception: KeyError: 'REALM_id_range' thx klaus Best Regards, Klaus Eckel, UNIX Consultant HPC (AIX,Linux) GPFS, BIA, SAP ITS/STG (SSIS) Server, Storage & Data Infrastructure Services IBM Deutschland GmbH Laatzener str, 1 30539 Hannover Germany Email: keckel at de.ibm.com Phone: +49-(0)52319489906 Handy: +49 (0)170 6323416 Visit the IBM Deutschland ITS Pages. IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Erich Clementi Gesch?ftsf?hrung: Martin Jetter (Vorsitzender), Reinhard Reschke, Dieter Scholz, Klaus Lintelmann, Michael Diemer, Martina Koederitz Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940 freeipa-devel-bounces at redhat.com wrote on 07/02/2012 09:55:36 AM: > From: > > Martin Kosek > > To: > > Rob Crittenden , > > Cc: > > freeipa-devel at redhat.com > > Date: > > 07/02/2012 09:57 AM > > Subject: > > Re: [Freeipa-devel] [PATCHES] 22-24 Add initial support for ID ranges > > Sent by: > > freeipa-devel-bounces at redhat.com > > On 06/30/2012 12:01 AM, Rob Crittenden wrote: > > Rob Crittenden wrote: > >> Rob Crittenden wrote: > >>> Alexander Bokovoy wrote: > >>>> On Fri, 29 Jun 2012, Sumit Bose wrote: > >>>>> On Wed, Jun 27, 2012 at 09:19:36PM +0200, Sumit Bose wrote: > >>>>>> On Tue, Jun 26, 2012 at 12:30:14PM +0200, Sumit Bose wrote: > >>>>>> > On Sun, Jun 17, 2012 at 09:47:20PM +0200, Sumit Bose wrote: > >>>>>> > > On Thu, Jun 14, 2012 at 02:25:01PM +0200, Sumit Bose wrote: > >>>>>> > > > On Thu, Jun 14, 2012 at 07:54:40AM -0400, Simo Sorce wrote: > >>>>>> > > > > On Thu, 2012-06-14 at 12:35 +0200, Sumit Bose wrote: > >>>>>> > > > > > On Wed, Jun 13, 2012 at 08:38:23PM -0400, Simo Sorce wrote: > >>>>>> > > > > > > On Wed, 2012-06-13 at 21:17 +0200, Sumit Bose wrote: > >>>>>> > > > > > > > > >>>>>> > > > > > > > to keep track of the different ranges we use for > >>>>>> UIDs/GIDs for local > >>>>>> > > > > > > > users/groups and users from trusted domains new range > >>>>>> objects are > >>>>>> > > > > > > > introduced which are stored below > >>>>>> cn=range,cn=etc,$SUFFIX. > >>>>>> > > > > > > > > >>>>>> > > > > > > > 0022: LDAP schema update > >>>>>> > > > > > > > >>>>>> > > > > > > ack > >>>>>> > > > > > > > >>>>>> > > > > > > > 0023: Create a range object during installation fir the > >>>>>> local ID range > >>>>>> > > > > > > > >>>>>> > > > > > > nack, I think we need to find a way to handle adding at > >>>>>> least the base > >>>>>> > > > > > > range on update. Otherwise an updated server won't be > >>>>>> able to have IDs > >>>>>> > > > > > > for most of its users. > >>>>>> > > > > > > >>>>>> > > > > > I fully agree, but since we said that we concentrate on > >>>>>> update issues in > >>>>>> > > > > > beta2 I wanted to send the version for the fresh install > >>>>>> first to allow > >>>>>> > > > > > testing. > >>>>>> > > > > > >>>>>> > > > > The reason I'd like updates is that this patchset can be > >>>>>> installed on > >>>>>> > > > > top of existing servers for testing w/o having to reinstall > >>>>>> from scratch > >>>>>> > > > > or manually creating the ipaDomainIDRange object :):) > >>>>>> > > > > >>>>>> > > > ok, will do. > >>>>>> > > > > >>>>>> > > > Do you otherwise agree with the patches or is there something I > >>>>>> should > >>>>>> > > > change while adding the updates? > >>>>>> > > > > >>>>>> > > > bye, > >>>>>> > > > Sumit > >>>>>> > > > > >>>>>> > > > > > >>>>>> > > > > > > > >>>>>> > > > > > > > 0024: add primary and secondary RID base to the local > >>>>>> range object > >>>>>> > > > > > > > during ipa-adtrust-install > >>>>>> > > > > > > > >>>>>> > > > > > > Not sure if setting the range belongs in the previous > >>>>>> patch or this one. > >>>>>> > > > > > > >>>>>> > > > > > I think it is right here, because a plain IPA server does > >>>>>> not need the > >>>>>> > > > > > RID related attributes. > >>>>>> > > > > > > >>>>>> > > > > > > We might decide to ask questions during > >>>>>> ipa-adtrust-install if the range > >>>>>> > > > > > > is not available, maybe presenting a set of pre-canned > >>>>>> choices if we can > >>>>>> > > > > > > detect them. > >>>>>> > > > > > > >>>>>> > > > > > I agree here, too. But as above I would like to handle > >>>>>> update issues > >>>>>> > > > > > in a second round. > >>>>>> > > > > > > >>>>>> > > > > > > > >>>>>> > > > > > > Finally I think we need to do a search with uid/gidNmber > >>>>>> < base and > >>>>>> > > > > > > uid/gidNumber > max and prompt/warn the user if we detect > >>>>>> any ID the > >>>>>> > > > > > > falls outside the configured range (either because we > >>>>>> failed to detect > >>>>>> > > > > > > ranges on upgrade and the user botched the question or > >>>>>> because the admin > >>>>>> > > > > > > added arbitrary IDs. > >>>>>> > > > > > > If a warning we should warn that missing a range that > >>>>>> suitably covers > >>>>>> > > > > > > these IDs, those users/groups will not be available for > >>>>>> the trust. > >>>>>> > > > > > > > >>>>>> > > > > > > Maybe we should also have a simple ipa command that can > >>>>>> list all > >>>>>> > > > > > > users/groups that fall outside the ranges as well. > >>>>>> > > > > > > >>>>>> > > > > > I'm working on the ranges cli plugin to allow 'ipa > >>>>>> range-add', 'ipa > >>>>>> > > > > > range-find' etc. I can add it there. > >>>>>> > > > > > > >>>>>> > > > >>>>>> > > Hi, > >>>>>> > > > >>>>>> > > this new series of patches add the cli plugin to create the ID > >>>>>> ranges > >>>>>> > > manually. I'm still working on a detection of the locally used id > >>>>>> range > >>>>>> > > of an upgrade domain in ipa-adtrust-install and an plugin which > >>>>>> rejects > >>>>>> > > new ranges which overlaps with existing ones. > >>>>>> > > > >>>>>> > > bye, > >>>>>> > > Sumit > >>>>>> > > >>>>>> > the attached patch adds a preop plugin which checks for overlaps > >>>>>> with > >>>>>> > existing ranges. > >>>>>> > > >>>>>> > bye, > >>>>>> > Sumit > >>>>>> > >>>>>> Finally I added a method to guess and create the initial ID range, > >>>>>> if no > >>>>>> one is preset, e.g. when updating from an older version of freeIPA. A > >>>>>> full series of patches is attached. > >>>>>> > >>>>>> bye, > >>>>>> Sumit > >>>>> > >>>>> This version of patches fixes review comments by Alexander and also > >>>>> adds > >>>>> some test for the range CLI plugin which were kindly provided by > >>>>> Alexander. > >>>> ACK > >>>> > >>> > >>> These patches aren't applying for me. > >>> > >>> rob > >> > >> Hmm. Pulled a fresh tree and they imported fine. > >> > >> pushed to master > >> > >> rob > > > > I had only pushed 22-24 before, pushed 25 and 29 as well. > > > > rob > > > > I examined the latest changes and found several rather serious issues which > will break this functionality on upgraded servers: > > https://fedorahosted.org/freeipa/ticket/2891 > > Martin > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkosek at redhat.com Mon Jul 2 10:34:00 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 12:34:00 +0200 Subject: [Freeipa-devel] [PATCHES] 22-24 Add initial support for ID ranges In-Reply-To: References: <20120613191726.GI20850@localhost.localdomain> <1339634303.8230.634.camel@willson.li.ssimo.org> <20120614103538.GJ20850@localhost.localdomain> <1339674880.8230.674.camel@willson.li.ssimo.org> <20120614122501.GK20850@localhost.localdomain> <20120617194720.GB29454@localhost.localdomain> <20120626103014.GC29454@localhost.localdomain> <20120627191935.GL29454@localhost.localdomain> <20120629121731.GC922@localhost.localdomain> <20120629125215.GI6687@redhat.com> <4FEDFF17.6030300@redhat.com> <4FEE124D.4090401@redhat.com> <4FEE25CF.5070302@redhat.com> <4FF153F8.3000709@redhat.com> Message-ID: <4FF17918.6080905@redhat.com> On 07/02/2012 12:16 PM, Klaus Eckel wrote: > hi all, > when I tried to install FreeIPA 2.99.0 on Fedora 17 I got the following error: > > [root at linux yum.repos.d]# cat ipa-devel.repo > [ipa-devel] > name=IPA development $releasever - $basearch > baseurl=http://jdennis.fedorapeople.org/ipa-devel/fedora/$releasever/$basearch/os/ > > enabled=1 > gpgcheck=0 > > new yum update .. > > [root at linux yum.repos.d]# uname -a > Linux linux.fritz.box 3.4.4-3.fc17.x86_64 #1 SMP Tue Jun 26 20:54:56 UTC 2012 > x86_64 x86_64 x86_64 GNU/Linux > > freeipa-server-2.99.0-0.20120630T2358Zgit50ebd1a.fc17.x86_64.. > > ipa-server-install -a ###t --hostname=linux.fritz.box -r fritz.box -p ###### > -n fritz.box -U > > [21/36]: adding default layout > Unexpected error - see /var/log/ipaserver-install.log for details: > KeyError: 'REALM_id_range' > > log .. > > 2012-07-02T10:07:32Z DEBUG [21/36]: adding default layout > 2012-07-02T10:07:32Z INFO File > "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 696, > in run_script > return_value = main_function() > > File "/sbin/ipa-server-install", line 958, in main > hbac_allow=not options.hbac_allow) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line > 249, in create_instance > self.start_creation("Configuring directory server", 60) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line > 259, in start_creation > method() > > File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line > 569, in __add_default_layout > self._ldap_mod("bootstrap-template.ldif", self.sub_dict) > > File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line > 98, in _ldap_mod > txt = ipautil.template_file(path, sub_dict) > > File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 218, in > template_file > return template_str(txt, vars) > > File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 206, in > template_str > val = string.Template(txt).substitute(vars) > > File "/usr/lib64/python2.7/string.py", line 172, in substitute > return self.pattern.sub(convert, self.template) > > File "/usr/lib64/python2.7/string.py", line 162, in convert > val = mapping[named] > > 2012-07-02T10:07:32Z INFO The ipa-server-install command failed, exception: > KeyError: 'REALM_id_range' > > thx klaus > > Best Regards, > Klaus Eckel > Klaus>, UNIX > Consultant HPC (AIX,Linux) GPFS, BIA, SAP > ITS/STG (SSIS) > Server, Storage & Data Infrastructure Services IBM Deutschland GmbH > > Laatzener str, 1 > 30539 Hannover > Germany Email: keckel at de.ibm.com > Phone: +49-(0)52319489906 > Handy: +49 (0)170 6323416 > > > Visit the IBM Deutschland ITS Pages. > > > IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Erich Clementi > Gesch?ftsf?hrung: Martin Jetter (Vorsitzender), Reinhard Reschke, > Dieter Scholz, Klaus Lintelmann, Michael Diemer, Martina Koederitz Sitz der > Gesellschaft: > Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE > 99369940 > > freeipa-devel-bounces at redhat.com wrote on 07/02/2012 09:55:36 AM: > >> From: >> >> Martin Kosek >> >> To: >> >> Rob Crittenden , >> >> Cc: >> >> freeipa-devel at redhat.com >> >> Date: >> >> 07/02/2012 09:57 AM >> >> Subject: >> >> Re: [Freeipa-devel] [PATCHES] 22-24 Add initial support for ID ranges >> >> Sent by: >> >> freeipa-devel-bounces at redhat.com >> >> On 06/30/2012 12:01 AM, Rob Crittenden wrote: >> > Rob Crittenden wrote: >> >> Rob Crittenden wrote: >> >>> Alexander Bokovoy wrote: >> >>>> On Fri, 29 Jun 2012, Sumit Bose wrote: >> >>>>> On Wed, Jun 27, 2012 at 09:19:36PM +0200, Sumit Bose wrote: >> >>>>>> On Tue, Jun 26, 2012 at 12:30:14PM +0200, Sumit Bose wrote: >> >>>>>> > On Sun, Jun 17, 2012 at 09:47:20PM +0200, Sumit Bose wrote: >> >>>>>> > > On Thu, Jun 14, 2012 at 02:25:01PM +0200, Sumit Bose wrote: >> >>>>>> > > > On Thu, Jun 14, 2012 at 07:54:40AM -0400, Simo Sorce wrote: >> >>>>>> > > > > On Thu, 2012-06-14 at 12:35 +0200, Sumit Bose wrote: >> >>>>>> > > > > > On Wed, Jun 13, 2012 at 08:38:23PM -0400, Simo Sorce wrote: >> >>>>>> > > > > > > On Wed, 2012-06-13 at 21:17 +0200, Sumit Bose wrote: >> >>>>>> > > > > > > > >> >>>>>> > > > > > > > to keep track of the different ranges we use for >> >>>>>> UIDs/GIDs for local >> >>>>>> > > > > > > > users/groups and users from trusted domains new range >> >>>>>> objects are >> >>>>>> > > > > > > > introduced which are stored below >> >>>>>> cn=range,cn=etc,$SUFFIX. >> >>>>>> > > > > > > > >> >>>>>> > > > > > > > 0022: LDAP schema update >> >>>>>> > > > > > > >> >>>>>> > > > > > > ack >> >>>>>> > > > > > > >> >>>>>> > > > > > > > 0023: Create a range object during installation fir the >> >>>>>> local ID range >> >>>>>> > > > > > > >> >>>>>> > > > > > > nack, I think we need to find a way to handle adding at >> >>>>>> least the base >> >>>>>> > > > > > > range on update. Otherwise an updated server won't be >> >>>>>> able to have IDs >> >>>>>> > > > > > > for most of its users. >> >>>>>> > > > > > >> >>>>>> > > > > > I fully agree, but since we said that we concentrate on >> >>>>>> update issues in >> >>>>>> > > > > > beta2 I wanted to send the version for the fresh install >> >>>>>> first to allow >> >>>>>> > > > > > testing. >> >>>>>> > > > > >> >>>>>> > > > > The reason I'd like updates is that this patchset can be >> >>>>>> installed on >> >>>>>> > > > > top of existing servers for testing w/o having to reinstall >> >>>>>> from scratch >> >>>>>> > > > > or manually creating the ipaDomainIDRange object :):) >> >>>>>> > > > >> >>>>>> > > > ok, will do. >> >>>>>> > > > >> >>>>>> > > > Do you otherwise agree with the patches or is there something I >> >>>>>> should >> >>>>>> > > > change while adding the updates? >> >>>>>> > > > >> >>>>>> > > > bye, >> >>>>>> > > > Sumit >> >>>>>> > > > >> >>>>>> > > > > >> >>>>>> > > > > > > >> >>>>>> > > > > > > > 0024: add primary and secondary RID base to the local >> >>>>>> range object >> >>>>>> > > > > > > > during ipa-adtrust-install >> >>>>>> > > > > > > >> >>>>>> > > > > > > Not sure if setting the range belongs in the previous >> >>>>>> patch or this one. >> >>>>>> > > > > > >> >>>>>> > > > > > I think it is right here, because a plain IPA server does >> >>>>>> not need the >> >>>>>> > > > > > RID related attributes. >> >>>>>> > > > > > >> >>>>>> > > > > > > We might decide to ask questions during >> >>>>>> ipa-adtrust-install if the range >> >>>>>> > > > > > > is not available, maybe presenting a set of pre-canned >> >>>>>> choices if we can >> >>>>>> > > > > > > detect them. >> >>>>>> > > > > > >> >>>>>> > > > > > I agree here, too. But as above I would like to handle >> >>>>>> update issues >> >>>>>> > > > > > in a second round. >> >>>>>> > > > > > >> >>>>>> > > > > > > >> >>>>>> > > > > > > Finally I think we need to do a search with uid/gidNmber >> >>>>>> < base and >> >>>>>> > > > > > > uid/gidNumber > max and prompt/warn the user if we detect >> >>>>>> any ID the >> >>>>>> > > > > > > falls outside the configured range (either because we >> >>>>>> failed to detect >> >>>>>> > > > > > > ranges on upgrade and the user botched the question or >> >>>>>> because the admin >> >>>>>> > > > > > > added arbitrary IDs. >> >>>>>> > > > > > > If a warning we should warn that missing a range that >> >>>>>> suitably covers >> >>>>>> > > > > > > these IDs, those users/groups will not be available for >> >>>>>> the trust. >> >>>>>> > > > > > > >> >>>>>> > > > > > > Maybe we should also have a simple ipa command that can >> >>>>>> list all >> >>>>>> > > > > > > users/groups that fall outside the ranges as well. >> >>>>>> > > > > > >> >>>>>> > > > > > I'm working on the ranges cli plugin to allow 'ipa >> >>>>>> range-add', 'ipa >> >>>>>> > > > > > range-find' etc. I can add it there. >> >>>>>> > > > > > >> >>>>>> > > >> >>>>>> > > Hi, >> >>>>>> > > >> >>>>>> > > this new series of patches add the cli plugin to create the ID >> >>>>>> ranges >> >>>>>> > > manually. I'm still working on a detection of the locally used id >> >>>>>> range >> >>>>>> > > of an upgrade domain in ipa-adtrust-install and an plugin which >> >>>>>> rejects >> >>>>>> > > new ranges which overlaps with existing ones. >> >>>>>> > > >> >>>>>> > > bye, >> >>>>>> > > Sumit >> >>>>>> > >> >>>>>> > the attached patch adds a preop plugin which checks for overlaps >> >>>>>> with >> >>>>>> > existing ranges. >> >>>>>> > >> >>>>>> > bye, >> >>>>>> > Sumit >> >>>>>> >> >>>>>> Finally I added a method to guess and create the initial ID range, >> >>>>>> if no >> >>>>>> one is preset, e.g. when updating from an older version of freeIPA. A >> >>>>>> full series of patches is attached. >> >>>>>> >> >>>>>> bye, >> >>>>>> Sumit >> >>>>> >> >>>>> This version of patches fixes review comments by Alexander and also >> >>>>> adds >> >>>>> some test for the range CLI plugin which were kindly provided by >> >>>>> Alexander. >> >>>> ACK >> >>>> >> >>> >> >>> These patches aren't applying for me. >> >>> >> >>> rob >> >> >> >> Hmm. Pulled a fresh tree and they imported fine. >> >> >> >> pushed to master >> >> >> >> rob >> > >> > I had only pushed 22-24 before, pushed 25 and 29 as well. >> > >> > rob >> > >> >> I examined the latest changes and found several rather serious issues which >> will break this functionality on upgraded servers: >> >> https://fedorahosted.org/freeipa/ticket/2891 >> >> Martin >> Hello Klaus, Thanks for reporting this. We already know about this issue and it will be fixed soon in a scope of ticket 2891 I filed and which I am working on right now. Martin From jcholast at redhat.com Mon Jul 2 11:21:16 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 02 Jul 2012 13:21:16 +0200 Subject: [Freeipa-devel] [PATCH][WIP] LDAP encoding redone In-Reply-To: <4FEDFCD7.7030507@redhat.com> References: <4FEB2E71.6040105@redhat.com> <4FEDFCD7.7030507@redhat.com> Message-ID: <4FF1842C.2000600@redhat.com> Dne 29.6.2012 21:07, Rob Crittenden napsal(a): > Jan Cholasta wrote: >> Hi, >> >> this is the next patch in the input validation & handling series >> . It changes the way >> entries are encoded and decoded in the LDAP backend. >> >> The patch consists of several changes: >> >> * Refactored the Encoder class to be more universally usable. It uses >> a polymorphic interface, which hopefully makes the encoding code more >> readable. >> >> * Attribute values now use Python data types matching the syntax of >> the attribute. This removes the need to decode/encode the values from/to >> raw LDAP values in the CallbackInterface callbacks as well as other >> parts of IPA. >> >> * On command output, attribute values are converted to strings so >> that the resulting entry is the same as it is without the patch. I don't >> like this code and I'd like to get rid of at least some parts of it, but >> I'm not sure how that would affect API compatibility. Removing the >> special case for boolean values would fix >> . >> >> * Entries are more strictly checked when they are encoded and >> decoded. Values of multi-value attributes must be lists (not tuples!) of >> objects of the appropriate python type, values of single-value >> attributes must be objects of the appropriate python type. This helps >> detecting data type errors that would previously go unnoticed. >> >> * Some parameters use data type that doesn't match the syntax of the >> according attribute, or are single-value even when the according >> attribute is multi-value. Values of such parameters wouldn't pass the >> new strict checking if they were used in attributes without >> modifications. To remedy this, added a new parameter option >> attr_convertor, which allows specifying a custom function for converting >> parameter values to attribute values. >> >> Note that this is work in progress, some things may be (and certainly >> are) broken, there is some low-quality code and docstrings, comments and >> tests are TBD. >> >> Suggestions and comments are welcome. >> >> Honza > > I haven't tried this yet, but this change jumped out at me: > > if attr not in ('aciname', 'group', 'memberof', 'nsaccountlock', > 'subtree', 'targetgroup', 'type') and self.obj is not None and attr in > self.obj.params and 'virtual_attribute' not in self.obj.params[attr].flags: > > Why exclude this subset of attributes? Generally, attribute are returned in the form returned by python-ldap, that is lists of strings. This rule does not apply to the attributes in the subset (most of them are single strings, nsaccountlock is a single boolean), so they must be excluded from the conversion. > > Is the big block of code adding to __call__ meant to maintain backwards > compatibility? Yes. Like I said above, I'm not sure how much of this is really necessary and what can be safely removed. We can skip the whole thing for new clients and return entries without converting them first, but that would require modifications on the client side as well. If the excluded attribute subset stays, it would be better to introduce a new parameter flag that inhibits the conversion, instead of having a hardcoded list of attributes. > > This seems to make lists out of a lot of things that weren't previously > lists. Is that to satisfy the schema? Exactly. I am concerned about how much we can depend on the schema to be what we expect it to be. I know that an attribute value might not match the syntax of that attribute because of replication (). It is probably also possible for a single-value attribute to have more than one value this way. But what if someone changes the schema so that an attribute that was previously single-value becomes multi-value? Do we allow that? The code in my patch wouldn't cope with such a situation very well. Maybe it would be better to require all (including single-value) attributes to be represented by lists in IPA...? > > rob Honza -- Jan Cholasta From mkosek at redhat.com Mon Jul 2 12:26:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 14:26:44 +0200 Subject: [Freeipa-devel] [PATCH] 282 Create default range entry after upgrade Message-ID: <4FF19384.8090403@redhat.com> Create default range both on new install and on upgrades. Also make sure that all range object classes are present for upgraded machines. Default range LDIF entry for new install was fixed so that new installation does not crash. https://fedorahosted.org/freeipa/ticket/2891 -- Martin Kosek Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-282-create-default-range-entry-after-upgrade.patch Type: text/x-patch Size: 6998 bytes Desc: not available URL: From sbose at redhat.com Mon Jul 2 12:46:13 2012 From: sbose at redhat.com (Sumit Bose) Date: Mon, 2 Jul 2012 14:46:13 +0200 Subject: [Freeipa-devel] [PATCH] 31 Use DN objects instead of strings in adtrustinstance Message-ID: <20120702124613.GI922@localhost.localdomain> Hi, as pointed out by John adtrustinstance.py does not use the DN objects but strings to define LDAP DNs. This patch fixes it. bye, Sumit -------------- next part -------------- From e91540c323791f06791c973754e7773eaccaf08e Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 2 Jul 2012 12:20:23 +0200 Subject: [PATCH] Use DN objects instead of strings in adtrustinstance --- ipaserver/install/adtrustinstance.py | 41 +++++++++++++++++++++------------- 1 Datei ge?ndert, 25 Zeilen hinzugef?gt(+), 16 Zeilen entfernt(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 9646f7e7b1dc7e9954b681550d3ffa7a54a6f139..20feec4df309b5793aa1c29fdf18bc5bfe180943 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -29,6 +29,7 @@ from ipaserver.install.dsinstance import realm_to_serverid from ipaserver.install.bindinstance import get_rr, add_rr, del_rr, \ dns_zone_exists from ipalib import errors, api +from ipalib.dn import DN from ipapython import sysrestore from ipapython import ipautil from ipapython.ipa_log_manager import * @@ -129,8 +130,10 @@ class ADTRUSTInstance(service.Service): return "S-1-5-21-%d-%d-%d" % (sub_ids[0], sub_ids[1], sub_ids[2]) def __add_admin_sids(self): - admin_dn = "uid=admin,cn=users,cn=accounts,%s" % self.suffix - admin_group_dn = "cn=admins,cn=groups,cn=accounts,%s" % self.suffix + admin_dn = str(DN(('uid', 'admin'), api.env.container_user, + self.suffix)) + admin_group_dn = str(DN(('cn', 'admins'), api.env.container_group, + self.suffix)) try: dom_entry = self.admin_conn.getEntry(self.smb_dom_dn, \ @@ -184,7 +187,8 @@ class ADTRUSTInstance(service.Service): """ try: - res = self.admin_conn.search_s("cn=ranges,cn=etc,"+self.suffix, + res = self.admin_conn.search_s(str(DN(api.env.container_ranges, + self.suffix)), ldap.SCOPE_ONELEVEL, "(objectclass=ipaDomainIDRange)") if len(res) != 1: @@ -227,8 +231,8 @@ class ADTRUSTInstance(service.Service): pass for new_dn in (self.trust_dn, \ - "cn=ad,"+self.trust_dn, \ - "cn=ad,cn=etc,"+self.suffix): + str(DN(('cn', 'ad'), self.trust_dn)), \ + str(DN(api.env.container_cifsdomains, self.suffix))): try: self.admin_conn.getEntry(new_dn, ldap.SCOPE_BASE) except errors.NotFound: @@ -469,14 +473,16 @@ class ADTRUSTInstance(service.Service): self.smb_conf = "/etc/samba/smb.conf" - self.smb_dn = "cn=adtrust agents,cn=sysaccounts,cn=etc,%s" % self.suffix + self.smb_dn = str(DN(('cn', 'adtrust agents'), ('cn', 'sysaccounts'), + ('cn', 'etc'), self.suffix)) - self.trust_dn = "cn=trusts,%s" % self.suffix - self.smb_dom_dn = "cn=%s,cn=ad,cn=etc,%s" % (self.domain_name, \ - self.suffix) + self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) + self.smb_dom_dn = str(DN(('cn', self.domain_name), + api.env.container_cifsdomains, self.suffix)) self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name - self.cifs_agent = "krbprincipalname=%s,cn=services,cn=accounts,%s" % \ - (self.cifs_principal.lower(), self.suffix) + self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), + api.env.container_service, + self.suffix)) self.selinux_booleans = ["samba_portmapper"] self.__setup_sub_dict() @@ -484,14 +490,16 @@ class ADTRUSTInstance(service.Service): def find_local_id_range(self): self.ldap_connect() - if self.admin_conn.search_s("cn=ranges,cn=etc," + self.suffix, + if self.admin_conn.search_s(str(DN(api.env.container_ranges, + self.suffix)), ldap.SCOPE_ONELEVEL, "objectclass=ipaDomainIDRange"): return try: - entry = self.admin_conn.getEntry("cn=admins,cn=groups,cn=accounts," \ - + self.suffix, + entry = self.admin_conn.getEntry(str(DN(('cn', 'admins'), + api.env.container_group, + self.suffix)), ldap.SCOPE_BASE) except errors.NotFound: raise ValueError("No local ID range and no admins group found.\n" \ @@ -514,8 +522,9 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry("cn=%s_id_range,cn=ranges,cn=etc,%s" % \ - (self.realm_name, self.suffix)) + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + api.env.container_ranges, + self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') entry.setValue('cn', ('%s_id_range' % self.realm_name)) entry.setValue('ipaBaseID', str(base_id)) -- 1.7.10.2 From rcritten at redhat.com Mon Jul 2 12:47:43 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 02 Jul 2012 08:47:43 -0400 Subject: [Freeipa-devel] [PATCH] 1030 Fedora 18 compatibility In-Reply-To: <4FF13CA9.6050602@redhat.com> References: <4FEB4711.9010603@redhat.com> <4FEC1764.2020809@redhat.com> <4FEDC4A0.3070800@redhat.com> <4FF13CA9.6050602@redhat.com> Message-ID: <4FF1986F.8090500@redhat.com> Martin Kosek wrote: > On 06/29/2012 05:07 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 06/27/2012 07:46 PM, Rob Crittenden wrote: >>>> I found a few minor issues when building and installing the master branch on >>>> Fedora 18. This patch should address it. >>>> >>>> rob >>>> >>> >>> 1) This will fail for on F17->F18 upgrades, we need to bump VERSION in >>> ipa-rewrite.conf. >>> >>> Besides that, ipa-upgradeconfig needs to be fixed, otherwise it will crash >>> during ipa-rewrite.conf upgrade - ${AUTOREDIR} variable is not set. >>> >>> However, this variable will need to be figured out from current >>> ipa-rewrite.conf contents as it depends on whether the IPA server was installed >>> with --no-ui-redirect or not. >>> >>> 2) Shouldn't we bump tomcat6 version as well since we depend on the tomcat6 >>> fixed in BZ 831464? >>> >>> 3) %changelog entry is missing >>> >>> Martin >>> >> >> This should do it >> >> rob > > This looks as a way to go, but this one won't fly yet - the server FQDN is > hard-coded to the find_autoredirect function. > > Martin > Updated. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1030-3-f18.patch Type: text/x-diff Size: 4225 bytes Desc: not available URL: From mkosek at redhat.com Mon Jul 2 13:21:00 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 15:21:00 +0200 Subject: [Freeipa-devel] [PATCH] 1030 Fedora 18 compatibility In-Reply-To: <4FF1986F.8090500@redhat.com> References: <4FEB4711.9010603@redhat.com> <4FEC1764.2020809@redhat.com> <4FEDC4A0.3070800@redhat.com> <4FF13CA9.6050602@redhat.com> <4FF1986F.8090500@redhat.com> Message-ID: <4FF1A03C.4070709@redhat.com> On 07/02/2012 02:47 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 06/29/2012 05:07 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 06/27/2012 07:46 PM, Rob Crittenden wrote: >>>>> I found a few minor issues when building and installing the master branch on >>>>> Fedora 18. This patch should address it. >>>>> >>>>> rob >>>>> >>>> >>>> 1) This will fail for on F17->F18 upgrades, we need to bump VERSION in >>>> ipa-rewrite.conf. >>>> >>>> Besides that, ipa-upgradeconfig needs to be fixed, otherwise it will crash >>>> during ipa-rewrite.conf upgrade - ${AUTOREDIR} variable is not set. >>>> >>>> However, this variable will need to be figured out from current >>>> ipa-rewrite.conf contents as it depends on whether the IPA server was >>>> installed >>>> with --no-ui-redirect or not. >>>> >>>> 2) Shouldn't we bump tomcat6 version as well since we depend on the tomcat6 >>>> fixed in BZ 831464? >>>> >>>> 3) %changelog entry is missing >>>> >>>> Martin >>>> >>> >>> This should do it >>> >>> rob >> >> This looks as a way to go, but this one won't fly yet - the server FQDN is >> hard-coded to the find_autoredirect function. >> >> Martin >> > > Updated. > > rob > ACK. Pushed to master. Martin From mkosek at redhat.com Mon Jul 2 13:54:35 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 15:54:35 +0200 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FEDEB77.2010307@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> <4FBE560E.1080108@redhat.com> <1338293075.30643.52.camel@balmora.brq.redhat.com> <4FC4DDBB.7070902@redhat.com> <4FEB0B0D.1000206@redhat.com> <4FEDEB77.2010307@redhat.com> Message-ID: <4FF1A81B.7020704@redhat.com> On 06/29/2012 07:52 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 05/29/2012 04:31 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On Thu, 2012-05-24 at 11:38 -0400, Rob Crittenden wrote: >>>>> Petr Viktorin wrote: >>>>>> On 05/18/2012 10:03 PM, Rob Crittenden wrote: >>>>>>> Rob Crittenden wrote: >>>>>>>> A hardcoded timeout was used in ipactl for service restarts, set rather >>>>>>>> low. A separate timeout was hardcoded into the installer. >>>>>>>> >>>>>>>> I centralized them into a single timeout, configurable in the standard >>>>>>>> way in /etc/ipa/*.conf. >>>>>>>> >>>>>>>> On install it will always default to 120 seconds and remain there unless >>>>>>>> changed in default.conf (not replicated either). >>>>>>>> >>>>>>>> I tested this on systemd systems and sysV systems and it works ok for >>>>>>>> me. You'll also want to double-check that this works when other 389-ds >>>>>>>> instances are installed. >>>>>>>> >>>>>>>> Getting the naming of instances right was a bit tricky. >>>>>>> >>>>>>> Noticed a problem on upgrades and fixed that. Updated patch attached. >>>>>>> >>>>>>> rob >>>>>>> >>>>>>> >>>>>> >>>>>> Please rebase the patch onto current master. >>>>>> >>>>>> >>>>> >>>>> Done >>>> >>>> This is a good start. I just found few places where I found that the >>>> remaining wait function calls are redundant: >>>> >>>> 1) install/tools/ipactl: >>>> >>>> if lurl.urlscheme == 'ldapi': >>>> - wait_for_open_socket(lurl.hostport, timeout=6) >>>> + wait_for_open_socket(lurl.hostport, >>>> timeout=api.env.startup_timeout) >>>> else: >>>> (host,port) = lurl.hostport.split(':') >>>> - wait_for_open_ports(host, [int(port)], timeout=6) >>>> + wait_for_open_ports(host, [int(port)], >>>> timeout=api.env.startup_timeout) >>>> >>>> Aren't these calls redundant? We already wait for ports when dirsrv is >>>> started (dirsrv.start()) or restarted (dirsrv.restart()). >>> >>> It is redundant in some cases but there are some calls we make where this is >>> used to determine the availability of the service. This call is needed. >>> >>>> 2) ipaserver/install/replication.py: >>>> - installutils.wait_for_open_ports('localhost', [389, 636], 300) >>>> + ipautil.wait_for_open_ports('localhost', [389, 636], 300) >>>> >>>> Isn't this now redundant? Port check should be done in service restart. >>> >>> Yes, looks like this call can go. >>> >>>> 3) ipaserver/install/plugins/updateclient.py: >>>> >>>> - installutils.wait_for_open_socket(socket_name) >>>> + wait_for_open_socket(socket_name) >>>> >>>> Also seems redundant, dirsrv should be already up as it was restarted >>>> via our Service framework. Though we only check for ports in the Service >>>> framework, I wonder if this is enough and we can be sure that when ports >>>> are up, the LDAPI socket is also up. >>> >>> No, sockets and ports are separate, particularly when updating. In fact, we >>> disable the ports so a wait_for_port() will always fail which is why I added >>> the wait flag. This may be a case I missed with upgrades. Let me test upgrades >>> again... >>> >>> rob >> >> I think we want to either send a revised patch to this ticket to get it to Beta >> 1 or to defer it to some future version... >> >> Martin >> > > Here is a rebased patch. > > rob This rather looks as a SELinux user map test patch... Martin From rcritten at redhat.com Mon Jul 2 13:57:54 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 02 Jul 2012 09:57:54 -0400 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FF1A81B.7020704@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> <4FBE560E.1080108@redhat.com> <1338293075.30643.52.camel@balmora.brq.redhat.com> <4FC4DDBB.7070902@redhat.com> <4FEB0B0D.1000206@redhat.com> <4FEDEB77.2010307@redhat.com> <4FF1A81B.7020704@redhat.com> Message-ID: <4FF1A8E2.9040603@redhat.com> Martin Kosek wrote: > On 06/29/2012 07:52 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 05/29/2012 04:31 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On Thu, 2012-05-24 at 11:38 -0400, Rob Crittenden wrote: >>>>>> Petr Viktorin wrote: >>>>>>> On 05/18/2012 10:03 PM, Rob Crittenden wrote: >>>>>>>> Rob Crittenden wrote: >>>>>>>>> A hardcoded timeout was used in ipactl for service restarts, set rather >>>>>>>>> low. A separate timeout was hardcoded into the installer. >>>>>>>>> >>>>>>>>> I centralized them into a single timeout, configurable in the standard >>>>>>>>> way in /etc/ipa/*.conf. >>>>>>>>> >>>>>>>>> On install it will always default to 120 seconds and remain there unless >>>>>>>>> changed in default.conf (not replicated either). >>>>>>>>> >>>>>>>>> I tested this on systemd systems and sysV systems and it works ok for >>>>>>>>> me. You'll also want to double-check that this works when other 389-ds >>>>>>>>> instances are installed. >>>>>>>>> >>>>>>>>> Getting the naming of instances right was a bit tricky. >>>>>>>> >>>>>>>> Noticed a problem on upgrades and fixed that. Updated patch attached. >>>>>>>> >>>>>>>> rob >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Please rebase the patch onto current master. >>>>>>> >>>>>>> >>>>>> >>>>>> Done >>>>> >>>>> This is a good start. I just found few places where I found that the >>>>> remaining wait function calls are redundant: >>>>> >>>>> 1) install/tools/ipactl: >>>>> >>>>> if lurl.urlscheme == 'ldapi': >>>>> - wait_for_open_socket(lurl.hostport, timeout=6) >>>>> + wait_for_open_socket(lurl.hostport, >>>>> timeout=api.env.startup_timeout) >>>>> else: >>>>> (host,port) = lurl.hostport.split(':') >>>>> - wait_for_open_ports(host, [int(port)], timeout=6) >>>>> + wait_for_open_ports(host, [int(port)], >>>>> timeout=api.env.startup_timeout) >>>>> >>>>> Aren't these calls redundant? We already wait for ports when dirsrv is >>>>> started (dirsrv.start()) or restarted (dirsrv.restart()). >>>> >>>> It is redundant in some cases but there are some calls we make where this is >>>> used to determine the availability of the service. This call is needed. >>>> >>>>> 2) ipaserver/install/replication.py: >>>>> - installutils.wait_for_open_ports('localhost', [389, 636], 300) >>>>> + ipautil.wait_for_open_ports('localhost', [389, 636], 300) >>>>> >>>>> Isn't this now redundant? Port check should be done in service restart. >>>> >>>> Yes, looks like this call can go. >>>> >>>>> 3) ipaserver/install/plugins/updateclient.py: >>>>> >>>>> - installutils.wait_for_open_socket(socket_name) >>>>> + wait_for_open_socket(socket_name) >>>>> >>>>> Also seems redundant, dirsrv should be already up as it was restarted >>>>> via our Service framework. Though we only check for ports in the Service >>>>> framework, I wonder if this is enough and we can be sure that when ports >>>>> are up, the LDAPI socket is also up. >>>> >>>> No, sockets and ports are separate, particularly when updating. In fact, we >>>> disable the ports so a wait_for_port() will always fail which is why I added >>>> the wait flag. This may be a case I missed with upgrades. Let me test upgrades >>>> again... >>>> >>>> rob >>> >>> I think we want to either send a revised patch to this ticket to get it to Beta >>> 1 or to defer it to some future version... >>> >>> Martin >>> >> >> Here is a rebased patch. >> >> rob > > > This rather looks as a SELinux user map test patch... > > Martin Ouch, off-by-one error. This is the right one. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1014-5-timeout.patch Type: text/x-diff Size: 30155 bytes Desc: not available URL: From rcritten at redhat.com Mon Jul 2 14:20:04 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 02 Jul 2012 10:20:04 -0400 Subject: [Freeipa-devel] [PATCH] 282 Create default range entry after upgrade In-Reply-To: <4FF19384.8090403@redhat.com> References: <4FF19384.8090403@redhat.com> Message-ID: <4FF1AE14.3070705@redhat.com> Martin Kosek wrote: > Create default range both on new install and on upgrades. Also make > sure that all range object classes are present for upgraded machines. > > Default range LDIF entry for new install was fixed so that new > installation does not crash. > > https://fedorahosted.org/freeipa/ticket/2891 ACK From mkosek at redhat.com Mon Jul 2 14:28:51 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 02 Jul 2012 16:28:51 +0200 Subject: [Freeipa-devel] [PATCH] 282 Create default range entry after upgrade In-Reply-To: <4FF1AE14.3070705@redhat.com> References: <4FF19384.8090403@redhat.com> <4FF1AE14.3070705@redhat.com> Message-ID: <4FF1B023.5080609@redhat.com> On 07/02/2012 04:20 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> Create default range both on new install and on upgrades. Also make >> sure that all range object classes are present for upgraded machines. >> >> Default range LDIF entry for new install was fixed so that new >> installation does not crash. >> >> https://fedorahosted.org/freeipa/ticket/2891 > > ACK Pushed to master. Martin From rcritten at redhat.com Mon Jul 2 14:59:07 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 02 Jul 2012 10:59:07 -0400 Subject: [Freeipa-devel] [PATCH] 31 Use DN objects instead of strings in adtrustinstance In-Reply-To: <20120702124613.GI922@localhost.localdomain> References: <20120702124613.GI922@localhost.localdomain> Message-ID: <4FF1B73B.8050206@redhat.com> Sumit Bose wrote: > Hi, > > as pointed out by John adtrustinstance.py does not use the DN objects > but strings to define LDAP DNs. This patch fixes it. > > bye, > Sumit ACK, pushed to master rob From edewata at redhat.com Mon Jul 2 15:49:50 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 02 Jul 2012 10:49:50 -0500 Subject: [Freeipa-devel] [PATCH] 165 Display loginas information only after login In-Reply-To: <4FF153D8.70206@redhat.com> References: <4FEC6506.7050205@redhat.com> <4FECC2DF.4030200@redhat.com> <4FF153D8.70206@redhat.com> Message-ID: <4FF1C31E.7010205@redhat.com> ACK. Some more comments below. Feel free to fix before push or later separately. On 7/2/2012 2:55 AM, Petr Vobornik wrote: > On 06/28/2012 10:47 PM, Endi Sukma Dewata wrote: >> On 6/28/2012 9:07 AM, Petr Vobornik wrote: >>> Message 'Logged in as: user at FREEIPA.ORG' was displayed before user was >>> logged in. It was wrong. >>> >>> Now 'Logged in as: XXX' is displayed only when user XXX is logged in. So >>> no more user at FREEIPA.ORG :) . >> >> It might be better to use visibility instead of display to reserve the >> space. Right now the password expiration warning will initially appear >> on the right, then shift to the left when the "Logged in as" appears. >> > Seems like better approach. Updated patch attached. The message still shifts, but this time from left to right, probably because the "loggedinas" element doesn't have a fixed width. > Another improvement might be: display password expiration warning at the > same time as login information. What do you think? Does it matter? Yes, I was thinking about that too. It doesn't really matter much but I agree it would look better if they appear at the same time. The "user at FREEIPA.ORG" in the HTML code is never visible anymore, so feel free to remove it. You can also replace the with a then define the style in CSS. A separate issue, under IPA Server tab, the Trusts menu comes after Configuration. Would it make more sense to show Configuration last because Configuration is really like "Other Settings"? -- Endi S. Dewata From rcritten at redhat.com Mon Jul 2 21:28:30 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 02 Jul 2012 17:28:30 -0400 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 1 Release Message-ID: <4FF2127E.6080206@redhat.com> The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 1. It can be downloaded from http://www.freeipa.org/page/Downloads. A build is available in the Fedora rawhide repositories or for Fedora 17 via the freeipa-devel repo on www.freeipa.org: http://freeipa.org/downloads/freeipa-devel.repo . To install in Fedora 17 the updates-repo repository needs to be enabled as well. For additional information see the AD Trust design page http://freeipa.org/page/IPAv3_AD_trust and the AD Trust testing page http://freeipa.org/page/IPAv3_testing_AD_trust. == Highlights in 3.0.0 == * Support for AD Trust * Per-domain DNS permissions * DNS persistent search enabled by default, new zones are seen immediately * New DNS resolver library * Migration improvements * The last administrator cannot be removed * Forms-based password reset * Redesigned action panels in UI * Sessions for command-line users * Tool to configure automount client, ipa-client-automount == Upgrading == An IPA server can be upgraded simply by installing updated rpms. The server does not need to be shut down in advance. If you have multiple servers you may upgrade them one at a time. It is expected that all servers will be upgraded in a relatively short period (days or weeks not months). They should be able to co-exist peacefully but new features will not be available on old servers and enrolling a new client against an old server will result in the SSH keys not being uploaded. Downgrading a server once upgraded is not supported. Upgrading from 2.2.0 should work but has not been fully tested. Proceed with caution. An enrolled client does not need the new packages installed unless you want to re-enroll it. SSH keys for already installed clients are not uploaded, you will have to re-enroll the client or manually upload the keys. == Feedback == Please provide comments, bugs and other feedback via the freeipa-devel mailing list: http://www.redhat.com/mailman/listinfo/freeipa-devel == Detailed changelog including 2.2.0 == The development of 3.0 occurred simultaneously with 2.2.0 so there is some overlap. Adam Young (10): * enable proxy for dogtag * split metadata call * Make mod_nss renegotiation configuration a public function * Execute pki proxy setup when server is upgraded if needed * Force the upgrade of pki-setup when upgrading the RPMS * Fix dynamic display of UI tabs based on rights * remove enrolled column * Add priority to pwpolicy list * Remove delegation from browser config * ignore generated services file. Alexander Bokovoy (61): * Propagate environment when it is required. * Incorrect name in examples of ipa help hbactest * Unroll groups when testing HBAC rules * Convert server install code to platform-independent access to system services * Convert client-side tools to platform-independent access to system services * Convert installation tools to platform-independent access to system services * Cleanup whitespace * Introduce platform-specific adaptation for services used by FreeIPA. * When external host is specified in HBAC rule, allow its use in simulation * Unroll StrEnum values when displaying help * Configure pam_krb5 on the client only if sssd is not configured * Setup and restore ntp configuration on the client side properly * Fix 'referenced before assignment' warning * Before kinit, try to sync time with the NTP servers of the domain we are joining * Increase number of 'getent passwd attempts' to 10 * Force kerberos realm to be a string * Include indirect membership and canonicalize hosts during HBAC rules testing * Refactor backup_and_replace_hostname() into a flexible config modification tool * Write KRB5REALM to /etc/sysconfig/krb5kdc and make use of common backup_config_and_replace_variables() tool * Refactor authconfig use in ipa-client-install * Document --preserve-sssd option of ipa-client-install * Use set class instead of dictview class as set is wider supported * hbactest fails while you have svcgroup in hbacrule * Add support for systemd environments and use it to support Fedora 16 * Spin for connection success also when socket is not (yet) available * Update spec file to use systemd on Fedora 16 and above * Quote multiple workers option * Check for Python.h during build of py_default_encoding extension * Add configure check for libintl.h * Create directories for client install * Add "Extending FreeIPA" developer guide * Small fix to the guide CSS: enable vertical scroll bar * Rename included snippets to avoid problems with pylint * Fix dependency for samba4-devel package * Merge branch 'master' of git+ssh://git.fedorahosted.org/git/freeipa * Check through all LDAP servers in the domain during IPA discovery * Validate sudo RunAsUser/RunAsGroup arguments * Allow hbactest to work with HBAC rules exceeding default IPA limits * Add management of inifiles to allow manipulation of systemd units * Handle upgrade issues with systemd in Fedora 16 and above * Adopt to python-ldap 2.4.6 by removing unused references which are not available in python-ldap anymore * When changing multiple booleans with setsebool, pass each of them separately. * Add separate attribute to store trusted domain SID * Use dedicated keytab for Samba * Add trust management for Active Directory trusts * Use fully qualified PDC name when contacting for extended DN information * Perform case-insensitive searches for principals on TGS requests * Properly handle multiple IP addresses per host when installing trust support * Restart KDC after installing trust support to allow MS PAC generation * Add trust-related ACIs * get_fqdn() moved to ipaserver.installutils * ipa-sam: update sid_to_id() interface to follow passdb API changes in Samba * Add python-crypto to build requires for AD server-side code * Move AD trust support code to freeipa-server-trust-ad subpackage * restart dirsrv as part of ipa-adtrust-install * Re-format ipa-adtrust-install final message to be within 80 characters wide * Use correct SID attribute for trusted domains * Rename 'ipa trust-add-ad' to 'ipa trust-add --type=ad' * Support requests for DOMAIN$ account for trusted domains in ipasam module * Add error condition handling to the SASL bind callback in ipasam * Add support for external group members Endi S. Dewata (105): * Fixed browser configuration pages * Hide activation/deactivation link from regular users. * Fixed problem selecting value from combobox * Fixed inconsistent layout for password reset dialog. * Removed 'Hide already enrolled' checkbox. * Replaced page dirty dialog title. * Updated add and delete association dialog titles. * Removed unnecessary HBAC/sudo rule category modification. * Fixed command partial failure handling. * Fixed default map type in automount map adder dialog. * Fixed host OTP status. * Fixed host keytab status after setting OTP. * Fixed host adder dialog to show default DNS zone. * Fixed hard-coded UI messages. * Fixed problem adding hostgroup into netgroup. * Fixed problem with combobox. * Fixed hard-coded UI message in entity.js. * Fixed missing permission filter field. * Fixed problem with combobox using Sahi * Fixed unit test for entity select widget. * Fixed layout problem in permission adder dialog. * Fixed sudo rule association dialogs. * Fixed missing optional field. * Fixed labels for run-as users and groups. * Fixed problem opening host adder dialog. * Removed entitlement menu. * Fixed posix group checkbox. * Fixed columns in HBAC/sudo rules list pages. * Removed HBAC rule type. * Fixed missing cancel button in unprovisioning dialog. * Fixed problem enabling/disabling DNS zone. * Fixed problem enrolling member with the same name. * Modified dialog to use sections. * Removed undo flags from dialog field specs. * Fixed problem on combobox with search limit. * Fixed problem displaying special characters. * Updated DNS zone details page. * Replaced description text fields with text areas. * Fixed add/delete arrows position. * Fixed duplicate entries in enrollment dialog. * Updated color scheme. * Fixed tab and dialog widths. * Use editable combobox for service type. * Disable enroll button if nothing selected. * Fixed missing default shell field. * I18n clean-up. * Disable sudo options Delete button if nothing selected. * Added confirmation when adding multiple entries. * Added selectable labels for radio buttons. * Fixed dependency problem in UI test. * Fixed inconsistent required/optional attributes. * Removed HBAC deny rule warning. * Fixed host Enrolled column. * Fixed problem clearing validation error on checkboxes. * Fixed "enroll" labels. * Merged widget's metadata and param_info. * Refactored validation code. * Fixed inconsistent image names. * Fixed inconsistent details facet validation. * Added password field in user adder dialog. * Fixed blank krbtpolicy and config pages. * Moved facet code into facet.js. * Added extensible UI framework. * Added current password field. * Fixed problem changing page in association facet. * Updated sample data. * Added paging on search facet. * Refactored permission target section. * Removed develop.js. * Added commands into metadata. * Refactored entity object resolution. * Fixed ipa.js for sessions. * Fixed entity definition in test cases. * Added support for radio buttons in table widget. * Fixed entity metadata resolution. * Refactored facet.load(). * Added HBAC Test page. * Fixed navigation buttons for HBAC Test. * Fixed search filter in HBAC Test. * Added external fields for HBAC Test. * Fixed CSS for HBAC Test * Fixed I18n labels for HBAC Test * Fixed matched/unmatched checkboxes in HBAC Test * Added HBAC Test input validation. * Fixed problem loading DNS records. * Fixed unmatched checkbox name. * Fixed combobox icon position. * Fixed combobox search icon position. * Reload UI when the user changes. * Reload UI on server upgrade. * Added account status into user search facet. * Added policies into user details page. * Load user data and policies in a single batch. * Added instructions to generate CSR. * Fixed problem removing automount keys and DNS records. * Enabled paging on self-service permissions and delegations. * Enabled paging on automount keys. * Show disabled entries in gray. * Fixed inconsistent status labels. * Fixed host managed-by adder dialog. * Added icons for status column. * Hide Add/Delete buttons in self-service mode. * Use fixed font when displaying certificate. * Show password expiration date. * Fixed boot.ldif permission. JR Aquino (5): * Create Tool for Enabling/Disabling Managed Entry Plugins * Replication: Adjust replica installation to omit processing memberof computations * Improve sudorule documentation * Create FreeIPA CLI Plugin for the 389 Auto Membership plugin * Move Managed Entries into their own container in the replicated space. Jan Cholasta (42): * Make sure messagebus is running prior to starting certmonger. * Verify that passwords specified through command line options of ipa-server-install meet the length requirement. * Add option to install without the automatic redirect to the Web UI. * Search for users in all the naming contexts present on the directory server. * Add subscription-manager dependency for RHEL. * Verify that the external CA certificate files are correct. * Check that install hostname matches the server hostname. * Fix client install on IPv6 machines. * Fix ipa-replica-prepare always warning the user about not using the system hostname. * Validate name_from_ip parameter of dnszone. * Add a function for formatting network locations of the form host:port for use in URLs. * Work around pkisilent bugs. * Disallow deletion of global password policy. * Don't leak passwords through kdb5_ldap_util command line arguments. * Remove more redundant configuration values from krb5.conf. * Finalize plugin initialization on demand. * Parse comma-separated lists of values in all parameter types. This can be enabled for a specific parameter by setting the "csv" option to True. * Fix make-lint crash under certain circumstances. * Fix attempted write to attribute of read-only object. * Add LDAP schema for SSH public keys. * Add LDAP ACIs for SSH public key schema. * Add support for SSH public keys to user and host objects. * Add API initialization to ipa-client-install. * Move the nsupdate functionality to separate function in ipa-client-install. * Update host SSH public keys on the server during client install. * Configure ssh and sshd during ipa-client-install. * Base64-decode unicode values in Bytes parameters. * Add SSH service to platform-specific services. * Move the compat module from ipalib to ipapython. * Configure SSH features of SSSD in ipa-client-install. * Wait for child process to terminate after receiving SIGINT in ipautil.run. * Parse zone indices in IPv6 addresses in CheckedIPAddress. * Fix uses of O=REALM instead of the configured certificate subject base. * Fix the procedure for getting default values of command parameters. * Change parameters to use only default_from for dynamic default values. * Check whether the default user group is POSIX when adding new user with --noprivate. * Check configured maximum user login length on user rename. * Fix internal error when renaming user with an empty string. * Refactor exc_callback invocation. * Set the "KerberosAuthentication" option in sshd_config to "no" instead of "yes". * Redo boolean value encoding. * SSH configuration fixes. John Dennis (38): * DN objects should support the insert method * Test DN object non-latin Unicode support * convert unittests to use DN objects * invalid i18n string in dns.py * update LINGUAS file, add missing po files * Update all po files * compute accurate translation statistics * add documentation validation to makeapi tool * internationalize help topics * internationalize cli help framework * improve i18n docstring extraction * Fix Spanish po translation file * Unable to Download Certificate with Browser * Add log manager module * modify codebase to utilize IPALogManager, obsoletes logging * IPAdmin undefined anonymous parameter lists * subclass SimpleLDAPObject * Restore default log level in server to INFO * If "make rpms" fails so will the next make * Remove old RPMROOT contents before it is used for rpmbuild * update i18n pot file for branch master * Add ipa_memcached service * add session manager and cache krb auth * Update pot file and list of explicit Python files needing translation * pulled new po files from Transifex * update translation pot file * Tweak the session auth to reflect developer consensus. * Implement session activity timeout * Implement password based session login * Log a message when returning non-success HTTP result * Replace broken i18n shell test with Python test * improve handling of ds instances during uninstall * Use indexed format specifiers in i18n strings * text unit test should validate using installed mo file * Validate DN & RDN parameters for migrate command * don't append basedn to container if it is included * Fix name error in hbactest * validate i18n strings when running "make lint" Lars Sjostrom (1): * Add disovery domain if client domain is different from server domain Marko Myllynen (2): * include for uintptr_t * Don't remove /tmp when removing temp cert dir Martin Kosek (171): * Add missing attribute labels for sudorule * Fix automountkey-mod * Fix automountlocation-import conflicts * ipa-client-install breaks network configuration * Fix sudo help and summaries * Let Bind track data changes * Improve man pages structure * Improve ipa-join man page * Fix permissions in installers * Fix configure.jar permissions * Set bind and bind-dyndb-ldap min nvr * Fix pylint false positive in hbactest module * ipactl does not stop dirsrv * dirsrv is not stopped correctly in the fallback * Remove checks for ds-replication plugin * Fix /usr/bin/ipa dupled server list * Revert "Always require SSL in the Kerberos authorization block." * Fix error messages in hbacrule * Fix LDAPCreate search failure * Fix HBAC tests hostnames * ipa-client assumes a single namingcontext * migrate process cannot handle multivalued pkey attribute * Be more clear about selfsign option * Install tools crash when password prompt is interrupted * Improve ipa-replica-prepare DNS check * Prevent collisions of hostgroup and netgroup * Make sure ipa-client-install returns correct error code * Improve default user/group object class validation * Fix i18n in config plugin * Fix dnszone-add name_from_ip server validation * Improve handling of GIDs when migrating groups * ipa-client-install hangs if the discovered server is unresponsive * Optimize member/memberof searches in LDAP * Make IPv4 address parsing more strict * Check hostname resolution sanity * Hostname used by IPA must be a system hostname * Check /etc/hosts file in ipa-server-install * Fix ipa-client-install -U option alignment * Improve hostgroup/netgroup collision checks * Fix client krb5 domain mapping and DNS * Add --zonemgr/--admin-mail validator * Fix ipa-managed-entries password option long form * Create pkey-only option for find commands * Fix ipa-server-install answer cache * Fix ipa-replica-conncheck port labels * Allow custom server backend encoding * Fix DNS zone --allow-dynupdate option behavior * Improve DNS record data validation * Polish ipa config help * Hosts file not updated when IP is passed as option * Fix API.txt * Fix LDAP object parameter encoding * Remove redundant information from API.txt * Fix ipa-managed-entries bind procedure * Let PublicError accept Gettext objects * Fix coverity issues in client CLI tools * Enable automember for upgraded servers * Make ipa-server-install clean after itself * Add --delattr option to complement --setattr/--addattr * Revert "Add DNS service records for Windows" * Improve zonemgr validator and normalizer * Change default DNS zone manager to hostmaster * Fix config migration option * Ask for user confirmation in ipa-server-install * Add connection failure recovery to IPAdmin * Add DNS check to conncheck port probe * Refactor dnsrecord processing * Fix Parameter csv parsing * Improve CLI output for complex commands * Create per-type DNS API * Fix maxvalue in DNS plugin * Fix LDAP add calls in replication module * Prevent service restart failures in ipa-replica-install * Fix LDAP updates in ipa-replica-install * Let replicas install without DNS * Restore ACI when aci_mod fails * Add missing --pkey-only option for selfservice and delegation * Replace float with Decimal * Improve host-add error message * Fix ipa-server-install for dual NICs * Fix selfservice-find crashes * Mark optional DNS record parts * Fix ldap2 combine_filters for ldap2.MATCH_NONE * Add missing managing hosts filtering options * Improve netgroup-add error messages * Fix TXT record parsing * Fix NSEC record conversion * Add SRV record target validator * Add data field for A6 record * Improve dnszone-add error message * Improve migration help * Fix raw format for ACI commands * Improve password change error message * Remove debug messages * Add argument help to CLI * Return proper DN in netgroup-add * Remove unused options from ipa-managed-entries * Add Petr Viktor?n to Contributors.txt * Ease zonemgr restrictions * Update schema for bind-dyndb-ldap * Global DNS options * Query and transfer ACLs for DNS zones * Add DNS conditional forwarding * Add API for PTR sync control * Add gidnumber minvalue * Add reverse DNS record when forward is created * Sanitize UDP checks in conncheck * Add client hostname requirements to man * Add SSHFP update policy for existing zones * Improve dns error message * Improve dnsrecord-add interactive mode * Improve hostname and domain name validation * Improve FQDN handling in DNS and host plugins * Improve hostname verification in install tools * Fix typos in ipa-replica-manage man page * Remove memberPrincipal for deleted replicas * Fix encoding for setattr/addattr/delattr * Add help for new structured DNS framework * Improve dnsrecord interactive help * Ignore case in yes/no prompts * Refresh resolvers after DNS install * Fix migration plugin compat check * Fix ipa-replica-manage TLS connection error * Treat UPGs correctly in winsync replication * Allow port numbers for idnsForwarders * Add missing global options in dnsconfig * Fix precallback validators in DNS plugin * Harden raw record processing in DNS plugin * Fix LDAP effective rights control with python-ldap 2.4.x * Avoid deleting DNS zone when a context is reused * Fix default SOA serial format * Amend permissions for new DNS attributes * Improve user awareness about dnsconfig * Fix dnsrecord-del interactive mode * Tolerate UDP port failures in conncheck * Improve automount indirect map error message * Forbid public access to DNS tree * Configure SELinux for httpd during upgrades * Fix installation when server hostname is not in a default domain * Return correct record name in DNS plugin * Fix dnsrecord_add interactive mode * Fix DNS and permissions unit tests * Raise proper exception when LDAP limits are exceeded * Do not fail migration because of duplicate groups * Fix help of --hostname option in ipa-client-install * Sort password policies properly with --pkey-only * Improve error message in zonemgr validator * Make ipa 2.2 client capable of joining an older server * Fix python Requires in Fedora 17 build * Remove ipa-server-install LDAP update errors * Remove LDAP limits from DNS service * Replace DNS client based on acutil with python-dns * Fix default_server configuration in ipapython.config * Reset krbtpolicy when a unit test is finished * Add rename option for DNS records * permission-find missed some results with --pkey-only option * Allow relative DNS name in NS validator * Fill new DNS zone update policy by default * Improve migration NotFound error * Fix dnszone-mod --forwader option help string * Add sysupgrade state file * Enable persistent search by default * Enable psearch on upgrades * Only set sebools when necessary * Password change capability for form-based auth * Remove trust work unit test failures * Decimal parameter conversion and normalization * Remove ipaNTHash from global allow ACI * Add missing libsss_idmap Requires on freeipa-server-trust-ad * Per-domain DNS record permissions * Create default range entry after upgrade Nalin Dahyabhai (5): * list users from nested groups, too * note that PKCS#12 files also contain private keys, and that the "pkinit" options refer to the KDC's credentials * index the fqdn and macAddress attributes for the sake of the compat plugin * create a "cn=computers" compat area populated with ieee802Device entries corresponding to computers with fqdn and macAddress attributes * add a pair of ethers maps for computers with hardware addresses on file Ondrej Hamada (26): * Misleading Keytab field * Client install root privileges check * Sort password policy by priority * Client install checks for nss_ldap * User-add random password support * HBAC test optional sourcehost option * localhost.localdomain clients refused to join * Leave nsds5replicaupdateschedule parameter unset * Fix 'no-reverse' option description * Memberof attribute control and update * Validate attributes in permission-add * Migration warning when compat enabled * ipa-client-install not calling authconfig * More exception handlers in ipa-client-install * Search allowed attributes in superior objectclasses * Typos in FreeIPA messages * Netgroup nisdomain and hosts validation * Confusing default user groups * Unable to rename permission object * Fix empty external member processing * Allow one letter net/hostgroups names * permission-mod prompts for all parameters * ipa-server-install reword message * Always set ipa_hostname for sssd.conf * Case sensitive renaming of objects * Change random passwords behaviour Petr Viktorin (60): * Switch --group and --membergroup in example for delegation * Fix/add options in ipa-managed-entries man page * Honor default home directory and login shell in user_add * Clean up i18n strings * Internationalization for HBAC and ipalib.output * Make ipausers a non-posix group on new installs * Add extra checking function to XMLRPC test framework * Add common helper for interactive prompts * Make sure the nolog argument to ipautil.run is not a bare string * Use stricter semantics when checking IP address for DNS records * Use reboot from /sbin * Allow removing sudo commands with special characters from command groups * Enforce that required attributes can't be set to None in CRUD Update * Mark most config options as required * Don't crash when searching with empty relationship options * Remove ipausers' gidnumber from tests * Use nose tools to check for exceptions * Only split CSV in the client, quote instead of escaping * Add missing BuildRequires * Use valid argument names in tests * Add CLI parsing tests * Allow multi-line CSV parameters * Move test skipping to class setup * Fix little test errors * Test the batch plugin * Defer conversion and validation until after --{add,del,set}attr are handled * Limit permission and selfservice names to alphanumerics, -, _, space * Convert --setattr values for attributes marked no_update * Fix expected error messages in tests * Remove pattern_errmsg from API.txt * Pass make-test arguments through to Nose * Document the 'nonempty' flag * Additional tests for pwpolicy * Update hostname validator error messages in tests * Do not use extra command options in the automount plugin * Do not crash on empty reverse member options * Do not crash on empty --setattr, --getattr, --addattr * Don't fail when adding default objectclasses using config-mod * Remove duplicate and unused utility code * Validate externalhost (when added by --addattr/--setattr) * Do not use extra command options in ACI, permission, selfservice * Check for empty/single value parameters before calling callbacks * Disallow '<' and non-ASCII characters in the DM password * Fix the pwpolicy_find post_callback * Disallow setattr on no_update/no_create params * Provide a better error message when deleting nonexistent attributes * Move install script error handling to a common function * Add more automount tests * Add samba4-python to BuildRequires * Prevent deletion of the last admin * Only allow root to run update plugins * Clean keytabs before installing new keys into them * Fix update plugin order * Rework the CallbackInterface * Improve ipa-client-install debug output * Improve autodiscovery logging * Fail on unknown Command options * Typo fixes * Improve output validation * Explicitly filter options that permission-{add,mod} passes to aci-{add,mod} Petr Vobornik (158): * error dialog for batch command * Uncheck checkboxes in association after deletion * Show error in adding associations * Validation of details facet before update https://fedorahosted.org/freeipa/ticket/1676 The ticket is a duplicate of server error, but it revealed few UI errors. * Modify serial associator to use batch * Modifying sudo options refreshes the whole page * Enable update and reset button only if dirty * Attributes table not scrollable * Fixed: JavaScript type error in entitlement page * Fixed inconsistency in enabling delete buttons * Code cleanup: widget creation * Fixed: Column header for attributes table should be full width * Fixed: Enrolment dialog offers to add entity to reflexive association. * Fixed: Some widgets do not have space for validation error message * Disables gid field if not posix group in group adder dialog * Fixed links to images in config and migration pages * Split Web UI initialization to several smaller calls #2 * Split Web UI initialization to several smaller calls * Added missing fields to password policy page * Fixed: Unable to add external user for RunAs User for Sudo rules * Circular entity dependency * Fixed: Duplicate CSS definitions * Fixing infinite loop in UI navigation unit test. * Minor visual enhancement of required indicator * Page is cleared before it is visible * Field for DNS SOA class changed to combobox with options * Extending facet's mechanism of gathering changes * Added cross browser support of Array.indexOf method * Splitting widget into widget and field * Splitting basic widgets into visual widgets and fields * Improved fields dirty status detection logic * Builders and collections for fields and widgets * Removing sections as special type of object * Added possibility to define facet/dialog specific policies * Modifying users to work with new concept * Modifying hosts to work with new concept * Modifying dns to work with new concept * Modifying services to work with new concept * Separation of writable update from field load method * Modifying ACI to work with new concept * Modifying groups to work with new concept * Code cleanup of HBAC, Sudo rules * Changing definition of basic fields in section from factory to type * Modifying automount to work with new concept * Fixed unit tests after widget refactoring * Removed usage of bitwise assignment operators in logical operations * Search facets show translated boolean values * Better displaying of long names in tables and facet headers * Additional better displaying of long names * Reordered facets in ACI * Association facets are read only in self service * Added facet tabs coloring * Fixed displaying of external records in rule association widgets * Distinguishing of external values in association tables * Better table column width computing * Fixed labels in Sudo, HBAC rules * Parsing of IPv4 and IPv6 addresses * Added support of custom field validators * Added validation logic to multivalued text field * Added client-side validation of A and AAAA DNS records * Fixed IPv6 validation special case: single colon * Added support for memberof attribute in permission * Added IP address validator to Host and DNS record adder dialog * Fixed entity link disabling * Fixed content type check in login_password * Improved usability of login dialog * Removed CSV creation from UI * Fixed mask validation in network_validator * Fixed checkbox value in table without pkey * Certificate serial number in hex format - ui testing data * Fixed evaluating checkbox dirty status * Better hbactest validation message * Content is no more overwritten by error message * Show_content on refresh success * Fixed rpm build warning - extension.js listed twice * Add support of new options in dnsconfig * DNS forwarder validator * Added mac address to host page * Facet expiration flag * Inter-facet expiration * Reworked netgroup Web UI to allow setting user/host category * Fixed: permission attrs table didn't update its available options on load * Added attrs field to permission for target=subtree * DNS forward policy: checkboxes changed to radio buttons * Removed mutex option from checkboxes * Removal of memberofindirect_permissons from privileges * User is notified that password needs to be reset in forms-based login * Added permission field to delegation * Paging disable for password policies * General builder support * Action lists * Control buttons * Redefined details control buttons * Redefined search control buttons * Hide search facet add/delete buttons in self-service * Batch action for search page control buttons * General details facet actions * Consistent change of entry status. * Instructions to generate cert use certutil instead of openssl * Host page fixed to work with disabled DNS support * Improved calculation of max pkey length in facet header * Correction of nested search facets tab labels * Refactored action list and control buttons to use shared list of actions * Refactored entities to use changed actions concept * Action panel * User password widget modified. * Action panel for user * Added missing i18n in action list and action panel * Add shadow to dialog * Enable reset password action according to attribute perrmission * Added cancel button to service unprovision dialog * Removal of illegal options in JSON-RPC calls * Added links to netgroup member tables * Text widget's dirty state is changed on various input methods * Change json serialization to serialize useful data * Removal of illegal options in association dialog * Update of serverconfig ipaconfigstring options * Action panel for host enrollment * Action panel for service provisioning * Separate reset password page * Added password reset capabilities to unauthorized dialog * Set network.http.sendRefererHeader to 2 on browser config * Custom Web UI error message for IPA error 911 * Trust Web UI * Same password validator * Action panel for certificates * Web UI password is going to expire in n days notification * Refactored associatin facet to use facet buttons with actions * Continuation of removing of not supported command options from Web UI * UI for SELinux user mapping * Added refresh button for UI * Modifying DNS UI to benefit from new DNS API * Added paging to DNS record search facet * Navigation and redirection to various facets * Automember UI * Automember UI - default groups * Automember UI - Fixed I18n labels * Removed question marks from field labels * UI support for ssh keys * Redirection to PTR records from A,AAAA records * Fixed problem when attributes_widget was displaying empty option * Added missing configuration options * Static metadata update - new DNS options * New checkboxes option: Mutual exclusive * DNS Zone UI: added new attributes * DNS UI: added A,AAAA create reverse options to adder dialog * Fixed displaying of A6 Record * New UI for DNS global configuration * Moved is_empty method from field to IPA object * Making validators to return true result if empty * Fixed DNS record add handling of 4304 error * Added unsupported_validator * Fixed redirection in Add and edit in automember hostgroup. * Fixed selection of single value in combobox * Multiple fields for one attribute * Added attrs to permission when target is group or filter * Added logout button * Forms based authentication UI Rob Crittenden (191): * Add information on setting api.env.host in the ipactl.8 man page * Log each command in a batch separately. * Do batch logging on successful commands too, not just failures. * Fix wording in examples of delegation plugin. * Suppress 389-ds debug output when starting services * Fix thread deadlock by using pthreads library instead of NSPR. * Change the way has_keytab is determined, also check for password. * Add additional pam ftp services to HBAC, and a ftp HBAC service group * Add label for HBAC services to show as members * Add option to only prompt once for passwords, use in entitle_register * Retrieve password/keytab state when modifying a host. * Disable reverse lookups in ipa-join and ipa-getkeytab * Remove more 389-ds files/directories on uninstallation. * Remove 389-ds upgrade state during uninstall * Set min nvr of pki-ca to 9.0.12 for fix in BZ 700505 * Add common is_installed() fn, better uninstall logging, check for errors. * Add external source hosts to HBAC. * Roll back changes if client installation fails. * Add netgroup as possible memberOf for hostgroups * Sort lists so order is predictable and tests pass as expected. * Suppress managed netgroups from showing as memberof hostgroups. * Use the IPA server cert profile in the installer. * Set min nvr of 389-ds-base to 1.2.9.7-1 for BZ 728605 * Don't allow a OTP to be set on an enrolled host * Remove normalizer that made role, privilege and permission names lower-case * Improved handling for ipa-pki-proxy.conf * The precendence on the modrdn plugin was set in the wrong location. * Update ipa-ldap-updater man page saying it is not an end-user utility * Skip the cert validator if the csr we are passed in is a valid filename * Change the Requires for the server and server-selinux for proper order * Suppress managed netgroups as indirect members of hosts. * The return value of restorecon is not reliable, ignore it. * Normalize uid in user principal to lower-case and do validation * Shut down duplicated file handle when HTTP response code is not 200. * Don't log one-time password in logs when configuring client. * Always require SSL in the Kerberos authorization block. * Include failed service and service groups in hbac rule management * Add regular expression pattern to host names. * Detect CA installation type in ipa-replica-prepare and ipa-ca-install. * Require current password when using passwd to change your own password. * Migration: don't assume there is only one naming context, add logging. * When calculating indirect membership don't test nesting on users and hosts. * Fix DNS permissions and membership in privileges * Fix upgrades of selfsign server * Make ipa-join work against an LDAP server that disallows anon binds * Fix has_upg() to work with relocated managed entries configuration. * Work around limits not being updatable in 389-ds. * Save the value of hostname even if it doesn't appear in /etc/sysconfig/network * Add explicit instructions to ipa-replica-manage for winsync replication * Set min nvr of 389-ds-base to 1.2.10-0.4.a4 for limits fixes (740942, 742324) * Handle an empty value in a name/value pair in config_replace_variables() * Update all LDAP configuration files that we can. * If our domain is already configured in sssd.conf start with a new config. * Fix typo in invalid PTR record error message * Fix problems in help system * Fix nis netgroup config entry so users appear in netgroup triple. * Don't allow default objectclass list to be empty. * Remove calls to has_managed_entries() * Fix copy/paste error in parameter description. * Add Ondrej Hamada to Contributors.txt * Don't check for 389-instances. * Clarify usage of --posix argument in group plugin. * Add plugin framework to LDAP updates. * Fix some issues introduced when rebasing update patch * Remove extraneous trailing single quote in nis.uldif * Mark some attributes required to match the schema. * Use absolute paths when trying to find certmonger request id. * Reorder privileges so that memberof for permissions are generated properly * Add SELinux user mapping framework. * Require an HTTP Referer header in the server. Send one in ipa tools. * Display the value of memberOf ACIs in permission plugin. * Fix two typos in role help. * Configure s4u2proxy during installation. * Document the ping plugin. * Catch exception when trying to list missing managed entries definitions * Fix some typos in automember help and paramters. * Add labels so HBAC and Sudo rules show under hosts/hostgroups. * Use correct template variable for hosts, FQDN. * In sudo when the category is all do not allow members, and vice versa. * Update and package ipa-upgradeconfig man page. * Fix deletion of HBAC Rules when there are SELinux user maps defined * Add support for storing MAC address in host entries. * Don't try to bind on TLS failure * Check for the existence of a replication agreement before deleting it. * %ghost the UI files that we install/create on the fly * Make submount automount maps work. * Require minimum SSF 56, confidentially. Also ensure minssf <= maxssf. * Consolidate external member code into two functions in baseldap.py * Make ipaconfigstring modifiable by users. * Don't use sets when calculating the modlist so order is preserved. * Add update files for SELinuxUserMap * Add update file for new schema in v2.2/3.0 * Stop and uninstall ipa_kpasswd on upgrade, fix dbmodules in krb5.conf * Don't set delegation flag in client, we're using S4U2Proxy now * Update S4U2proxy delegation list when creating replicas * Correct update syntax in 30-s4u2proxy.update * Remove Apache ccache on upgrade. * Add S4U2Proxy delegation permissions on upgrades * Disable false pylint error in freeipa-systemd-upgrade * Enable ipa_memcached when upgrading * Configure ipa_memcached when a replica is installed. * Use FQDN in place of FQHN for consistency in sub_dict. * Set min for 389-ds-base to 1.2.10.1-1 to fix install segfault, schema replication. * Limit the change password permission so it can't change admin passwords * Don't allow "Modify Group membership" permission to manage admins * Add the -v option to sslget to provide more verbose errors * Make sure memberof is in replication attribute exclusion list. * Don't check for schema uniqueness when comparing in ldapupdate. * Add Conflicts on mod_ssl because it interferes with mod_proxy and dogtag * Don't allow IPA master hosts or important services be deleted. * Catch public exceptions when creating the LDAP context in WSGI. * Don't consider virtual attributes when validating custom objectclasses * Add Requires to ipa-client on oddjob-mkhomedir * Fix managing winsync replication agreements with ipa-replica-manage * Check for duplicate winsync agreement before trying to set one up. * Remove unused kpasswd.keytab and ldappwd files if they exist. * Make sure 389-ds is running when adding memcache service in upgrade. * Don't run restorecon if SELinux is disabled or not present. * Limit allowed characters in a netgroup name to alpha, digit, -, _ and . * Don't call memberof task when re-initializing a replica. * Fix bad merge of not calling memberof task when re-initializing a replica * Add support defaultNamingContext and add --basedn to migrate-ds * Fix nested netgroups in NIS. * Warn that deleting replica is irreversible, try to detect reconnection. * Don't set migrated user's GID to that of default users group. * Don't delete system users that are added during installation. * Only apply validation rules when adding and updating. * subclass HTTP_Status from plugable.Plugin, fix not_found tests * Make hostnames adhere to new standards in HBAC tests * Fix WSGI error handling * Add status command to retrieve user lockout status * Add support for sudoOrder * Make hostnames adhere to new standards in hbactest plugin tests * Fix API.txt and VERSION to reflect new sudoOrder option. * Add --noac option to ipa-client-install man page * Do kinit in client before connecting to backend * Only warn if ipa-getkeytab doesn't get all requested enctypes. * Fix NSS no_init in the NSSHTTPS class * Set minimum version of selinux-policy to pick up memcached fix * Fix nsslapd-anonlimitsdn dn in cn=config * Set SELinux boolean httpd_manage_ipa so ipa_memcached will work. * Don't set dbdir in the connection until after the connection is created. * Display serial number as HEX (DECIMAL) when showing certificates. * Add subject key identifier to the dogtag server cert profile. * Configure a basic ldap.conf for OpenLDAP in /etc/openldap/ldap.conf * Import the ipaserver plugins based on context, not env.in_server. * Don't allow hosts and services of IPA masters to be disabled. * Use a consistent parameter name in errors, defaulting to cli_name. * No longer shell escape the DM password when calling pkisilent. * Fix test failure testing rename with an invalid hostname. * Fix attributes that contain DNs when migrating. * Normalize the primary key value to lowercase during migration. * Fix unit tests to work with new comma-support, validation requirements * Set minimum version of 389-ds-base to 1.2.10.4-2 to fix upgrade issue * Set nsslapd-minssf-exclude-rootdse to on so the DSE is always available. * Add requires on python-krbV to client subpackage * Fix failure count interval attribute name in query for password policy. * Handle updating replication agreements that lack nsDS5ReplicatedAttributeList * Don't create private groups for migrated users, check for valid gidnumber * Add updated Output format for batch to API.txt * Make revocation_reason required when revoking a certificate. * Add missing comma to list of services that cannot be disabled. * Return consistent value when hostcat and usercat is all. * Dereference pointer when comparing password history in qsort compare. * Configure certmonger to execute restart scripts on renewal. * Remove the running state when uninstalling DS instances. * Return consistent expiration message for forms-based login * Use mixed-case for Read DNS Entries permission * Update docs for user-status, always show disabled, time for each server. * Revert "Search allowed attributes in superior objectclasses" * Revert "Validate attributes in permission-add" * Return LDAP_SUCCESS on mods on a referral entry. * Fix overlapping cn param/option issue, pass cn as aciname in find * Implement permission/aci find by subtree * Include more information when IP address is not local during installation. * Validate on the user-provided domain name in the installer. * During replication installation see if an agreement already exists. * Check for locked-out user before incrementing lastfail. * Retry retrieving ldap principals when setting up replication. * Normalize uid to lower case in winsync. * Enforce sizelimit in permission-find, post_callback returns truncated * If SELinux is enabled ensure we also have restorecon. * Store session cookie in ccache for cli users * Add flag to ipa-client-install to managed order of ipa_server in sssd * Increase LimitRequestFieldSize in Apache config to support a 64KiB PAC * Add logging to ipa-upgradeconfig * Configure automount using autofs or sssd. * Defer adding ipa-cifs-delegation-targets until the Updates phase. * Add missing option to range_add in API.txt * Fix compatibility with Fedora 18. * Become IPA v3 beta 1 (3.0.0.pre1) Simo Sorce (104): * Set VERSION to 2.99.0 on the 3.0 development branch * Fix build warnings * ipa-pwd_extop: use endian.h instead of nih function * krbinstance: use helper function to get realm suffix * ipa-pwd-extop: Remove unused variables and code to set them * ipa-pwd-extop: do not append mkvno to krbExtraData * ipa-pwd-extop: Use the proper mkvno number in keys * ipa-pwd-extop: re-indent code using old style * ipa-pwd-extop: Use common krb5 structs from kdb.h * ipa-pwd-extop: Move encryption of keys in common * ipa-pwd-extop: Move encoding in common too * ipa-pwd-extop: make encsalt parsing function common * ipa-kdb: Initial plugin skeleton * ipa-kdb: add exports file * ipa-kdb: initialize module functions * ipa-kdb: implement get_time function * ipa-kdb: add common utility ldap wrapper functions * ipa-kdb: functions to get principal * ipa-kdb: add function to free principals * ipa-kdb: add functions to delete principals * ipa-kdb: add function to iterate over principals * ipa-kdb: add functions to change principals * ipa-kdb: Get/Store Master Key directly from LDAP * ipa-kdb: implement function to retrieve password policies * ipa-kdb: implement change_pwd function * util: add password policy manipulation functions * ipa-pwd-extop: Use common password policy code * ipa-kdb: add password policy support * ipa-pwd-extop: Allow kadmin to set krb keys * ipa-kdb: Change install to use the new ipa-kdb kdc backend * install: Remove uid=kdc user * ipa-kdb: Be flexible * install: Use proper case for boolean values * daemons: Remove ipa_kpasswd * schema: Split ipadns definitions from basev2 ones * v3-schema: Add new ipaExternalGroup objectclass * install: We do not need a ldap password anymore * install: We do not need a kpasswd keytab anymore * conncheck: Fix List of ports to check * ipa-kdb: Properly set password expiration time. * schema: Add new attributes and objectclasses for AD Trusts * conncheck: Additional check to verify the admin password is ok * ipa-pwd-extop: Fix segfault in password change. * ipa-pwd-extop: Enforce old password checks * ipa-kdb: Fix expiration time calculation * ipa-client-install: Fix joining when LDAP access is restricted * replica-prepare: anonymous binds may be disallowed * ipa-kdb: Fix legacy password hashes generation * updates: Change default limits on ldap searches * ipa-kdb: Fix memory leak * Modify random salt creation for interoperability * Amend #2038 fix * Fix CID 10742: Unchecked return value * Fix CID 10743: Unchecked return value * Fix CID 10745: Unchecked return value * Fix CID 11019: Resource leak * Fix CID 11020: Resource leak * Fix CID 11021: Resource leak * Fix CID 11022: Resource leak * Fix CID 11023: Resource leak * Fix CID 11024: Resource leak * Fix CID 11025: Resource leak * Fix CID 11026: Resource leak * Fix CID 11027: Wrong sizeof argument * Add support for generating PAC for AS requests for user principals * MS-PAC: Add support for verifying PAC in TGS requests * Add missing copyright header * Add NT domain GUID attribute. * Create skeleton CLDAP server as a DS plugin * ipa-cldap: Implement worker thread. * ipa-cldap: Decode CLDAP request. * ipa-cldap: Create netlogon blob * ipa-cldap: send cldap reply * ipa-kdb: Support re-signing PAC with different checksum * spec: We do not need krb5-server-ldap anymore * ipa-kdb: fix free() of uninitialized var * ipa-kdb: Remove unused CFLAGS/LIBS from Makefiles * ipa-kdb: fix memleaks in ipa_kdb_mspac.c * ipa-kdb: Fix copy and paste typo * ipa-kdb: Delegation ACL schema * ipa-kdb: enhance deref searches * ipa-kdb: Add delgation access control support * ipa-kdb: return properly when no PAC is available * ipa-cldap: Support clients asking for default domain * ipa-kdb: Verify the correct checksum in PAC validation * ipa-kdb: Create PAC's KDC checksum with right key * Fix replication setup * slapi-plugins: use thread-safe ldap library * ipa-kdb: add AS auditing support * ipa-kdb: Avoid lookup on modify if possible * ipa-kdb: set krblastpwdchange only when keys have been effectively changed * Remove compat defines * Require krb5 1.10 * ipa-kdb: Fix ACL evaluator * policy: add function to check lockout policy * ipa-kdb: fix delegation acl check * Fix ticket checks when using either s4u2proxy or a delegated krbtgt * Fix memleak and silence Coverity defects * Fix MS-PAC checks when using s4u2proxy * Fix theoretical leak discovered by coverity * Fix migration code password setting. * Fix setting domain_sid * ipa-kdb: Add MS-PAC on constrained delegation. * Add support for disabling KDC writes Sumit Bose (32): * Call standard_logging_setup() before any logging is done * Add ipa-adtrust-install utility * Fix ACIs in ipa-adtrust-install * Update samba LDAP schema * Fix typo in v3 base schema * Add admin SIDs * ipa-pwd-extop: allow password change on all connections with SSF>1 * Add DNS service records for Windows * Add DNS service records for Windows * Move our own domain info into cn=etc * Add trust objectclass and attributes to v3 schema * Use new objectclasses and attributes for trust * Fix some pylint warnings * Add ipasam samba passdb backend * activate CLDAP * Make pwd-extop aware of new ipaNTHash attribute * Add a second module init call for newer samba versions * Use exop instead of kadmin.local * ipasam: remove unused struct elements * Move some krb5 keys related functions from ipa-client to util * Add sidgen postop and task * Filter groups in the PAC * Add configure check for C Unit-Test framework check * Add external domain extop DS plugin * Use lower case names in LDAP to meet freeIPA convention * Extend LDAP schema * Add objects for initial ID range * Set RID bases for local domain during ipa-adtrust-install * Add CLI for ID ranges * Add range check preop plugin * Use DN objects instead of strings in adtrustinstance * Set samba_portmapper SELinux boolean during ipa-adtrust-install Yuri Chornoivan (1): * Fix typos From jfenal at gmail.com Mon Jul 2 22:01:09 2012 From: jfenal at gmail.com (=?UTF-8?B?SsOpcsO0bWUgRmVuYWw=?=) Date: Tue, 3 Jul 2012 00:01:09 +0200 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 1 Release In-Reply-To: <4FF2127E.6080206@redhat.com> References: <4FF2127E.6080206@redhat.com> Message-ID: 2012/7/2 Rob Crittenden > The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 1. > > It can be downloaded from http://www.freeipa.org/page/**Downloads > . > > A build is available in the Fedora rawhide repositories or for Fedora 17 > via the freeipa-devel repo on www.freeipa.org: > http://freeipa.org/downloads/**freeipa-devel.repo. To install in Fedora 17 the updates-repo repository needs to be enabled > as well. > > For additional information see the AD Trust design page > http://freeipa.org/page/IPAv3_**AD_trustand the AD Trust testing page > http://freeipa.org/page/IPAv3_**testing_AD_trust > . > > Wow! Dmitri told me last week in Boston that something was cooking, but I'm impressed at the changelog. Congrats to the team! Did you update transifex with the new strings for 3.0 for localization? Regards, J. -- J?r?me Fenal - jfenal AT gmail.com - http://fenal.org/ Paris.pm - http://paris.mongueurs.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pviktori at redhat.com Tue Jul 3 07:52:27 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 03 Jul 2012 09:52:27 +0200 Subject: [Freeipa-devel] Announcing FreeIPA v3.0.0 beta 1 Release In-Reply-To: References: <4FF2127E.6080206@redhat.com> Message-ID: <4FF2A4BB.6060603@redhat.com> On 07/03/2012 12:01 AM, J?r?me Fenal wrote: > 2012/7/2 Rob Crittenden > > > The FreeIPA team is proud to announce version FreeIPA v3.0.0 beta 1. > > It can be downloaded from http://www.freeipa.org/page/__Downloads > . > > A build is available in the Fedora rawhide repositories or for > Fedora 17 via the freeipa-devel repo on www.freeipa.org > : > http://freeipa.org/downloads/__freeipa-devel.repo > . To install in > Fedora 17 the updates-repo repository needs to be enabled as well. > > For additional information see the AD Trust design page > http://freeipa.org/page/IPAv3___AD_trust > and the AD Trust testing > page http://freeipa.org/page/IPAv3___testing_AD_trust > . > > > Wow! > > Dmitri told me last week in Boston that something was cooking, but I'm > impressed at the changelog. > Congrats to the team! > > Did you update transifex with the new strings for 3.0 for localization? > No, not yet. We didn't even update the repository with translations from Transifex. There is an i18n ticket scheduled for beta 2: https://fedorahosted.org/freeipa/ticket/2435. For this ticket I plan to change the way we store translations in Git (a patch is already on the list), and after that to sync with Transifex. AFAIK, Transifex doesn't store any history. When it is updated with new strings, all translations for the old strings are lost. That's why I would like to pull the current state into Git before updating Transifex. -- Petr? From pviktori at redhat.com Tue Jul 3 08:33:48 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 03 Jul 2012 10:33:48 +0200 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <4FEE1E0B.8030804@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> Message-ID: <4FF2AE6C.6010800@redhat.com> On 06/29/2012 11:28 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 06/25/2012 03:00 PM, Petr Viktorin wrote: >>> On 06/20/2012 06:15 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> On 06/04/2012 04:56 PM, Petr Viktorin wrote: >>>>>> Currently, FreeIPA's install/admin scripts are long pieces of code >>>>>> that aren't very reusable, importable, or testable. >>>>>> They have been extended over time with features such as logging and >>>>>> error handling, but since each tool was extended individually, there >>>>>> is much inconsistency and code duplication. >>>>>> This patch starts a framework which the admin tools can use, and >>>>>> converts ipa-ldap-updater to use the framework. >>>>>> >>>>>> In an earlier patch I found that improving a particular >>>>>> functionality in >>>>>> all the commands is not workable, so I want to tackle this one tool >>>>>> at a >>>>>> time. >>>>>> I'm starting with ipa-ldap-updater, because it's pretty small, >>>>>> doesn't >>>>>> use DNs (I don't want conflicts with John's work), and has the >>>>>> interesting --upgrade option. >>>>>> >>>>>> >>>>>> The framework does these tasks: >>>>>> - Parse options >>>>>> - Select tool to run (see below) >>>>>> - Validate options >>>>>> - Set up logging >>>>>> - Run the tool code >>>>>> - Handle any errors >>>>>> - Log success/failure >>>>>> >>>>>> The base class has some defaults for these that the tools can >>>>>> extend/override. >>>>>> >>>>>> >>>>>> To handle the case where one script does two different things >>>>>> (ipa-ldap-updater with/without --upgrade, or ipa-server-install >>>>>> with/without --uninstall), I want to split the tool in two classes >>>>>> rather than have repeated ifs in the code. >>>>>> This meant that option parsing (and initializing the parser) has >>>>>> to be >>>>>> done before creating an instance of the tool. I use a factory >>>>>> classmethod. >>>>>> >>>>>> >>>>>> I put the admintool base class in ipapython/ as it should be useful >>>>>> for >>>>>> ipa-client-install as well. >>>>>> >>>>>> >>>>>> >>>>>> First part of the work for: >>>>>> https://fedorahosted.org/freeipa/ticket/2652 >>>>>> >>>>>> >>>>> >>>>> Attaching rebased patch. >>>> >>>> I gather you want people to be calling run_cli() in their admin tools. >>>> Should main() be made private then? I could see someone getting >>>> confused >>>> and using main instead, which would work, but then the return value >>>> might not do the right thing. >>>> >>>> Or maybe just drop run_cli and have main exit with sys.exit()? >>> >>> I don't see why running a command as a Python function should be >>> discouraged. In fact it could even help -- for example logging could >>> only be set up once, so if we call, say, ipa-ldap-updater from >>> ipa-server-install, all related logs would go to a single file. >>> A C-style main (taking a list of arguments and returning the exit >>> status) is a good thing for modularity and testability. >>> The `run_cli` method is just a convenient shortcut for the usual usage, >>> so the calling modules can be as small as possible. >>> >>> If people get confused and call main instead of run_cli, they need to >>> manually pass in sys.argv. I think this is enough of a warning that >>> their assumptions aren't right. >>> To make it even clearer I've removed the possibility to pass None as >>> argv to main() and have it auto-filled. >>> >>> Some relevant reading: >>> http://www.artima.com/weblogs/viewpost.jsp?thread=4829 (old but still >>> valid) >>> http://en.wikipedia.org/wiki/Main_function#Python >>> >>>> It isn't correctly handling the case of an update not found: >>>> >>>> ipa : INFO Parsing file ad >>>> [Errno 2] No such file or directory: 'ad' >>>> ipa : INFO File >>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 151, in >>>> execute >>>> self.run() >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ldap_updater.py", >>>> >>>> >>>> line >>>> 180, in run >>>> modified = ld.update(self.files) >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", >>>> line 828, in update >>>> sys.exit(1) >>>> >>>> ipa : INFO The ipa-ldap-updater command failed, exception: >>>> SystemExit: 1 >>> >>> I've added validation for missing files, and improved the error message >>> ldapupdate raises (for cases the validation doesn't catch, like passing >>> directories or unreadable files). >>> Ideally ldapupdate would not try to handle the error itself, but that >>> code is used in more places that I don't want to break, so I'm leaving >>> the extraneous print in. >>> >>>> Running in test mode with the attached update doesn't seem to work >>>> either. There is nothing special about this file, just something I had >>>> lying around: >>>> >>>> ipa : INFO File >>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 151, in >>>> execute >>>> self.run() >>>> File >>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ldap_updater.py", >>>> >>>> >>>> line >>>> 184, in run >>>> 'Update complete, changes to be made, test mode', 2) >>>> >>>> ipa : INFO The ipa-ldap-updater command failed, exception: ScriptError: >>>> Update complete, changes to be made, test mode >>>> ipa : ERROR Update complete, changes to be made, test mode >>>> >>>> ipa : ERROR None >>> >>> Fixed. >>> >>>> The unit tests still pass which is good. >>>> >>>> With ipa-ldap-updater the return value is a bit strange. All the >>>> updates >>>> themselves can fail for one reason or another and the command can still >>>> consider this a success (it may fail because a feature is not enabled, >>>> for example). Still, the success message displayed at the end is a bit >>>> jarring when the updates themselves aren't applied. Here is a snippet >>>> when running ad.update live: >>>> >>>> ipa : INFO New entry: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>> ipa : DEBUG --------------------------------------------- >>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>> ipa : DEBUG add: 'account' to objectClass, current value [] >>>> ipa : DEBUG add: updated value [u'account'] >>>> ipa : DEBUG --------------------------------------------- >>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>> ipa : DEBUG objectClass: >>>> ipa : DEBUG account >>>> ipa : DEBUG add: 'adtrust' to uid, current value [] >>>> ipa : DEBUG add: updated value [u'adtrust'] >>>> ipa : DEBUG --------------------------------------------- >>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>> ipa : DEBUG objectClass: >>>> ipa : DEBUG account >>>> ipa : DEBUG uid: >>>> ipa : DEBUG adtrust >>>> ipa : DEBUG --------------------------------------------- >>>> ipa : DEBUG Final value >>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>> ipa : DEBUG objectClass: >>>> ipa : DEBUG account >>>> ipa : DEBUG uid: >>>> ipa : DEBUG adtrust >>>> ipa : INFO Parent DN of >>>> uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>> may not exist, cannot create the entry >>>> ipa : INFO The ipa-ldap-updater command was successful >>>> [root at pinto freeipa]# echo $? >>>> 0 >>>> >>>> This may be contrasting just because it is a contrived case. The >>>> command >>>> rval is separate from whether the updates all applied, so maybe this >>>> is ok. >>> >>> The current ipa-ldap-updater also works this way, so this should go in a >>> separate ticket. >>> I worry that changing the return value could make installations fail, >>> for example. >>> >>>> rob >>> >>> >>> Thanks for the review! >>> >> >> Once again, this time with the patch. > > Almost there. When running in test mode and an update that would be > applied should return 2. > > rob > > Fixed -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0056-04-Framework-for-admin-install-tools-with-ipa-ldap-upda.patch Type: text/x-patch Size: 30872 bytes Desc: not available URL: From mkosek at redhat.com Tue Jul 3 08:35:55 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 03 Jul 2012 10:35:55 +0200 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FF1A8E2.9040603@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> <4FBE560E.1080108@redhat.com> <1338293075.30643.52.camel@balmora.brq.redhat.com> <4FC4DDBB.7070902@redhat.com> <4FEB0B0D.1000206@redhat.com> <4FEDEB77.2010307@redhat.com> <4FF1A81B.7020704@redhat.com> <4FF1A8E2.9040603@redhat.com> Message-ID: <4FF2AEEB.5090100@redhat.com> On 07/02/2012 03:57 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 06/29/2012 07:52 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> On 05/29/2012 04:31 PM, Rob Crittenden wrote: >>>>> Martin Kosek wrote: >>>>>> On Thu, 2012-05-24 at 11:38 -0400, Rob Crittenden wrote: >>>>>>> Petr Viktorin wrote: >>>>>>>> On 05/18/2012 10:03 PM, Rob Crittenden wrote: >>>>>>>>> Rob Crittenden wrote: >>>>>>>>>> A hardcoded timeout was used in ipactl for service restarts, set rather >>>>>>>>>> low. A separate timeout was hardcoded into the installer. >>>>>>>>>> >>>>>>>>>> I centralized them into a single timeout, configurable in the standard >>>>>>>>>> way in /etc/ipa/*.conf. >>>>>>>>>> >>>>>>>>>> On install it will always default to 120 seconds and remain there unless >>>>>>>>>> changed in default.conf (not replicated either). >>>>>>>>>> >>>>>>>>>> I tested this on systemd systems and sysV systems and it works ok for >>>>>>>>>> me. You'll also want to double-check that this works when other 389-ds >>>>>>>>>> instances are installed. >>>>>>>>>> >>>>>>>>>> Getting the naming of instances right was a bit tricky. >>>>>>>>> >>>>>>>>> Noticed a problem on upgrades and fixed that. Updated patch attached. >>>>>>>>> >>>>>>>>> rob >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Please rebase the patch onto current master. >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Done >>>>>> >>>>>> This is a good start. I just found few places where I found that the >>>>>> remaining wait function calls are redundant: >>>>>> >>>>>> 1) install/tools/ipactl: >>>>>> >>>>>> if lurl.urlscheme == 'ldapi': >>>>>> - wait_for_open_socket(lurl.hostport, timeout=6) >>>>>> + wait_for_open_socket(lurl.hostport, >>>>>> timeout=api.env.startup_timeout) >>>>>> else: >>>>>> (host,port) = lurl.hostport.split(':') >>>>>> - wait_for_open_ports(host, [int(port)], timeout=6) >>>>>> + wait_for_open_ports(host, [int(port)], >>>>>> timeout=api.env.startup_timeout) >>>>>> >>>>>> Aren't these calls redundant? We already wait for ports when dirsrv is >>>>>> started (dirsrv.start()) or restarted (dirsrv.restart()). >>>>> >>>>> It is redundant in some cases but there are some calls we make where this is >>>>> used to determine the availability of the service. This call is needed. >>>>> >>>>>> 2) ipaserver/install/replication.py: >>>>>> - installutils.wait_for_open_ports('localhost', [389, 636], 300) >>>>>> + ipautil.wait_for_open_ports('localhost', [389, 636], 300) >>>>>> >>>>>> Isn't this now redundant? Port check should be done in service restart. >>>>> >>>>> Yes, looks like this call can go. >>>>> >>>>>> 3) ipaserver/install/plugins/updateclient.py: >>>>>> >>>>>> - installutils.wait_for_open_socket(socket_name) >>>>>> + wait_for_open_socket(socket_name) >>>>>> >>>>>> Also seems redundant, dirsrv should be already up as it was restarted >>>>>> via our Service framework. Though we only check for ports in the Service >>>>>> framework, I wonder if this is enough and we can be sure that when ports >>>>>> are up, the LDAPI socket is also up. >>>>> >>>>> No, sockets and ports are separate, particularly when updating. In fact, we >>>>> disable the ports so a wait_for_port() will always fail which is why I added >>>>> the wait flag. This may be a case I missed with upgrades. Let me test >>>>> upgrades >>>>> again... >>>>> >>>>> rob >>>> >>>> I think we want to either send a revised patch to this ticket to get it to >>>> Beta >>>> 1 or to defer it to some future version... >>>> >>>> Martin >>>> >>> >>> Here is a rebased patch. >>> >>> rob >> >> >> This rather looks as a SELinux user map test patch... >> >> Martin > > Ouch, off-by-one error. This is the right one. > > rob I did not find any further issue, so ACK. Martin From rcritten at redhat.com Tue Jul 3 14:36:24 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 03 Jul 2012 10:36:24 -0400 Subject: [Freeipa-devel] [PATCH] 1014 configurable service timeout In-Reply-To: <4FF2AEEB.5090100@redhat.com> References: <4FB3F267.2070104@redhat.com> <4FB6AAF6.9080800@redhat.com> <4FBE2654.4030207@redhat.com> <4FBE560E.1080108@redhat.com> <1338293075.30643.52.camel@balmora.brq.redhat.com> <4FC4DDBB.7070902@redhat.com> <4FEB0B0D.1000206@redhat.com> <4FEDEB77.2010307@redhat.com> <4FF1A81B.7020704@redhat.com> <4FF1A8E2.9040603@redhat.com> <4FF2AEEB.5090100@redhat.com> Message-ID: <4FF30368.9090502@redhat.com> Martin Kosek wrote: > On 07/02/2012 03:57 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> On 06/29/2012 07:52 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> On 05/29/2012 04:31 PM, Rob Crittenden wrote: >>>>>> Martin Kosek wrote: >>>>>>> On Thu, 2012-05-24 at 11:38 -0400, Rob Crittenden wrote: >>>>>>>> Petr Viktorin wrote: >>>>>>>>> On 05/18/2012 10:03 PM, Rob Crittenden wrote: >>>>>>>>>> Rob Crittenden wrote: >>>>>>>>>>> A hardcoded timeout was used in ipactl for service restarts, set rather >>>>>>>>>>> low. A separate timeout was hardcoded into the installer. >>>>>>>>>>> >>>>>>>>>>> I centralized them into a single timeout, configurable in the standard >>>>>>>>>>> way in /etc/ipa/*.conf. >>>>>>>>>>> >>>>>>>>>>> On install it will always default to 120 seconds and remain there unless >>>>>>>>>>> changed in default.conf (not replicated either). >>>>>>>>>>> >>>>>>>>>>> I tested this on systemd systems and sysV systems and it works ok for >>>>>>>>>>> me. You'll also want to double-check that this works when other 389-ds >>>>>>>>>>> instances are installed. >>>>>>>>>>> >>>>>>>>>>> Getting the naming of instances right was a bit tricky. >>>>>>>>>> >>>>>>>>>> Noticed a problem on upgrades and fixed that. Updated patch attached. >>>>>>>>>> >>>>>>>>>> rob >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> Please rebase the patch onto current master. >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Done >>>>>>> >>>>>>> This is a good start. I just found few places where I found that the >>>>>>> remaining wait function calls are redundant: >>>>>>> >>>>>>> 1) install/tools/ipactl: >>>>>>> >>>>>>> if lurl.urlscheme == 'ldapi': >>>>>>> - wait_for_open_socket(lurl.hostport, timeout=6) >>>>>>> + wait_for_open_socket(lurl.hostport, >>>>>>> timeout=api.env.startup_timeout) >>>>>>> else: >>>>>>> (host,port) = lurl.hostport.split(':') >>>>>>> - wait_for_open_ports(host, [int(port)], timeout=6) >>>>>>> + wait_for_open_ports(host, [int(port)], >>>>>>> timeout=api.env.startup_timeout) >>>>>>> >>>>>>> Aren't these calls redundant? We already wait for ports when dirsrv is >>>>>>> started (dirsrv.start()) or restarted (dirsrv.restart()). >>>>>> >>>>>> It is redundant in some cases but there are some calls we make where this is >>>>>> used to determine the availability of the service. This call is needed. >>>>>> >>>>>>> 2) ipaserver/install/replication.py: >>>>>>> - installutils.wait_for_open_ports('localhost', [389, 636], 300) >>>>>>> + ipautil.wait_for_open_ports('localhost', [389, 636], 300) >>>>>>> >>>>>>> Isn't this now redundant? Port check should be done in service restart. >>>>>> >>>>>> Yes, looks like this call can go. >>>>>> >>>>>>> 3) ipaserver/install/plugins/updateclient.py: >>>>>>> >>>>>>> - installutils.wait_for_open_socket(socket_name) >>>>>>> + wait_for_open_socket(socket_name) >>>>>>> >>>>>>> Also seems redundant, dirsrv should be already up as it was restarted >>>>>>> via our Service framework. Though we only check for ports in the Service >>>>>>> framework, I wonder if this is enough and we can be sure that when ports >>>>>>> are up, the LDAPI socket is also up. >>>>>> >>>>>> No, sockets and ports are separate, particularly when updating. In fact, we >>>>>> disable the ports so a wait_for_port() will always fail which is why I added >>>>>> the wait flag. This may be a case I missed with upgrades. Let me test >>>>>> upgrades >>>>>> again... >>>>>> >>>>>> rob >>>>> >>>>> I think we want to either send a revised patch to this ticket to get it to >>>>> Beta >>>>> 1 or to defer it to some future version... >>>>> >>>>> Martin >>>>> >>>> >>>> Here is a rebased patch. >>>> >>>> rob >>> >>> >>> This rather looks as a SELinux user map test patch... >>> >>> Martin >> >> Ouch, off-by-one error. This is the right one. >> >> rob > > I did not find any further issue, so ACK. > > Martin > pushed to master From rcritten at redhat.com Tue Jul 3 14:41:00 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 03 Jul 2012 10:41:00 -0400 Subject: [Freeipa-devel] [PATCH] 1031 run cleanallruv task Message-ID: <4FF3047C.9020202@redhat.com> Deleting a replica can leave a replication vector (RUV) on the other servers. This can confuse things if the replica is re-added, and it also causes the server to calculate changes against a server that may no longer exist. 389-ds-base provides a new task that self-propogates itself to all available replicas to clean this RUV data. This patch will create this task at deletion time to hopefully clean things up. It isn't perfect. If any replica is down or unavailable at the time the cleanruv task fires, and then comes back up, the old RUV data may be re-propogated around. To make things easier in this case I've added two new commands to ipa-replica-manage. The first lists the replication ids of all the servers we have a RUV for. Using this you can call clean_ruv with the replication id of a server that no longer exists to try the cleanallruv step again. This is quite dangerous though. If you run cleanruv against a replica id that does exist it can cause a loss of data. I believe I've put in enough scary warnings about this. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1031-cleanruv.patch Type: text/x-diff Size: 9600 bytes Desc: not available URL: From dpal at redhat.com Tue Jul 3 17:52:56 2012 From: dpal at redhat.com (Dmitri Pal) Date: Tue, 03 Jul 2012 13:52:56 -0400 Subject: [Freeipa-devel] freeIPA as a samba backend In-Reply-To: <1340743448.11173.28.camel@toron.pzo.lgs.com.ve> References: <1340672533.4964.17.camel@toron.pzo.lgs.com.ve> <4FE9C8A6.5040400@redhat.com> <1340723461.8801.9.camel@toron.pzo.lgs.com.ve> <4FE9EDAE.2090203@redhat.com> <4FE9F14B.9050702@redhat.com> <4FE9F3DB.2090403@redhat.com> <1340743448.11173.28.camel@toron.pzo.lgs.com.ve> Message-ID: <4FF33178.4090705@redhat.com> On 06/26/2012 04:44 PM, Loris Santamaria wrote: > El mar, 26-06-2012 a las 13:39 -0400, Dmitri Pal escribi?: >> On 06/26/2012 01:28 PM, Rich Megginson wrote: >>> On 06/26/2012 11:13 AM, Dmitri Pal wrote: >>>> On 06/26/2012 11:11 AM, Loris Santamaria wrote: >>>>> El mar, 26-06-2012 a las 10:35 -0400, Dmitri Pal escribi?: >>>>>> On 06/25/2012 09:02 PM, Loris Santamaria wrote: >>>>>>> Hi, >>>>>>> >>>>>>> while using freeIPA as a user database for a samba installation I found >>>>>>> a problem in the enforcement of password policies. FreeIPA password >>>>>>> policies are more detailed than samba's, in freeIPA one may enforce >>>>>>> password history and the number of character classes in a password, but >>>>>>> normally samba connects to freeIPA with the "Directory Manager" so those >>>>>>> policies are not enforced. >>>>>>> >>>>>>> Reading the source of ipa_pwd_extop I see there are three possibilities >>>>>>> when changing passwords: >>>>>>> >>>>>>> * Password change by the user, with full enforcement of policies >>>>>>> * Password change by an admin, with no enforcement of policies and >>>>>>> the new password is set as expired so the user has to change it >>>>>>> on next logon >>>>>>> * Password change by Directory Manager, with no enforcement of >>>>>>> policies and the password is not set as expired. >>>>>>> >>>>>>> None of the aforementioned possibilities are ideal for samba, samba >>>>>>> should connect to freeIPA with a user privileged enough to change >>>>>>> password for all users but with fully enforced policies. >>>>>>> >>>>>>> What do you think about this? Would you consider adding such feature? >>>>>>> Would you accept patches? >>>>>>> >>>>>> Can you please explain why samba needs to connect to IPA and change >>>>>> the passwords? >>>>>> In what role you use samba? As a file server or as something else? >>>>>> I am not sure I follow why you need the password change functionality. >>>>>> There is a way to setup Samba FS with IPA without trying to make IPA a >>>>>> back end for Samba. >>>>>> I can try to dig some writeups on the matter if you are interested. >>>>> Samba 3 when used as a PDC/BDC can use a LDAP server as its user/group >>>>> database. To do that samba connects with a privileged user to the LDAP >>>>> directory and manages some attributes of users and groups in the >>>>> directory, adding the sambaSAMAccount objectclass and the sambaSID >>>>> attribute to users, groups and machines of the domain. >>>>> >>>>> When users of Windows workstations in a samba domain change their >>>>> passwords samba updates the sambaNTPassword, userPassword, >>>>> sambaLastPwdChange, sambaPwdMustChange attributes of the corresponding >>>>> ldap user. >>>>> >>>>> Using freeIPA as ldap user backend for samba works quite well, except >>>>> for the password policy problem mentioned in last mail and that it is >>>>> hard to mantain in sync the enabled/disabled status of an account. >>>> What is the value of using FreeIPA as a Samba back end in >>>> comparison to other variants? >>>> Why IPA is more interesting than say 389-DS or OpenLDAP or native >>>> Samba? >>> IPA will keep all of your passwords in sync - userPassword, >>> sambaNTPassword, sambaLMPassword, and your kerberos passwords. 389 >>> cannot do this - the functionality that does this is provided by an >>> IPA password plugin. Openldap has a similar plugin, but I think it >>> is "contrib" and not "officially supported". >>> >> >> I know that Endi did the work to make 389 be a viable back end for >> Samba and it passed all the Samba torture tests so I am not sure I >> agree with you. Samba does the kerberos operations itself and uses >> LDAP as a storage only. This is why I am struggling to understand the >> use case. It seems that Loris has a different configuration that I do >> not quite understand, thus questions. >> >>>> What other features of IPA are used in such setup? >>>> >>>> Answering these (and may be other) questions would help us to >>>> understand how common is the use case that you brought up. > First of all, the use case is that of using Samba 3 as a Domain > Controller. Here in Venezuela the government itself promotes the use of > free software so most government agencies and industries won't install > Active Directory to administer windows desktops. There are some medium > to large deployments of Samba 3 as a domain controller here, and there > are a number of Linux desktops deployed in the same networks. > > When you use Samba 3 as a Domain Controller with a largish number of > users and machines is mandatory to use a Ldap server as a backend, and > for that you have basically two choices which are of course OpenLdap and > 389-DS, but those servers have to be combined with some administration > tools to be really useful. > > The ideal choice for us is 389-DS with freeIPA as an administration > framework because of: > > * Really easy to use to administer users and groups. Those users > and groups are visible from the samba domain and from linux > machines in the IPA realm or from legacy unix and linux machines > configured as ldap clients > * ipa_pwd_extop keeps ldap, samba and kerberos passwords in sync. > Well, this could be better > * freeIPA sets up 389-ds very well, with sane indexes and > permissions. For good performance with samba you just have to > add indexes for some samba attributes > > So you set up Samba 3 with freeIPA, and use the ipa tools to administer > users and groups, and those users and group can login in windows and > linux workstations. > > To set up Samba 3 with freeIPA 2.x you have to: > > * extend the ipa user and group objects to have them add the > sambaSAMAccount and SambaGroupType objectclasses > * add a Distributed Numeric Assignment configuration to have > 389-DS generate the sambaSID for objects with the above > objectClasses > * add a script used by samba 3 when a windows machine tries to > join the domain, so that ipa creates a new host if it doesn't > exist > * add indexes for common samba attributes. > > The alternative to all this, using free software, would be using samba 4 > and have Linux workstations join the Samba 4 domain, or use samba 4 for > the windows domain with a trust relationship to ipa 3 when it is ready. > > Thank you, > Loris Santamaria > Thank you for the detailed explanation. It makes perfect sense now. Since the LDAP back ends are not supported by Samba we either have to support them ourselves or make an effort to allow joining the Windows workstations into the IPA domain. IMO the latter is a more promising avenue in a long run as it would address more use cases than trying to solve it using old Samba3 DC. Would be interesting to really have a project that would build an IdM client for Windows using Kerberos for Windows. > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From abokovoy at redhat.com Tue Jul 3 19:08:47 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 3 Jul 2012 22:08:47 +0300 Subject: [Freeipa-devel] freeIPA as a samba backend In-Reply-To: <4FF33178.4090705@redhat.com> References: <1340672533.4964.17.camel@toron.pzo.lgs.com.ve> <4FE9C8A6.5040400@redhat.com> <1340723461.8801.9.camel@toron.pzo.lgs.com.ve> <4FE9EDAE.2090203@redhat.com> <4FE9F14B.9050702@redhat.com> <4FE9F3DB.2090403@redhat.com> <1340743448.11173.28.camel@toron.pzo.lgs.com.ve> <4FF33178.4090705@redhat.com> Message-ID: <20120703190847.GC16309@redhat.com> On Tue, 03 Jul 2012, Dmitri Pal wrote: >On 06/26/2012 04:44 PM, Loris Santamaria wrote: >> El mar, 26-06-2012 a las 13:39 -0400, Dmitri Pal escribi?: >>> On 06/26/2012 01:28 PM, Rich Megginson wrote: >>>> On 06/26/2012 11:13 AM, Dmitri Pal wrote: >>>>> On 06/26/2012 11:11 AM, Loris Santamaria wrote: >>>>>> El mar, 26-06-2012 a las 10:35 -0400, Dmitri Pal escribi?: >>>>>>> On 06/25/2012 09:02 PM, Loris Santamaria wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> while using freeIPA as a user database for a samba installation I found >>>>>>>> a problem in the enforcement of password policies. FreeIPA password >>>>>>>> policies are more detailed than samba's, in freeIPA one may enforce >>>>>>>> password history and the number of character classes in a password, but >>>>>>>> normally samba connects to freeIPA with the "Directory Manager" so those >>>>>>>> policies are not enforced. >>>>>>>> >>>>>>>> Reading the source of ipa_pwd_extop I see there are three possibilities >>>>>>>> when changing passwords: >>>>>>>> >>>>>>>> * Password change by the user, with full enforcement of policies >>>>>>>> * Password change by an admin, with no enforcement of policies and >>>>>>>> the new password is set as expired so the user has to change it >>>>>>>> on next logon >>>>>>>> * Password change by Directory Manager, with no enforcement of >>>>>>>> policies and the password is not set as expired. >>>>>>>> >>>>>>>> None of the aforementioned possibilities are ideal for samba, samba >>>>>>>> should connect to freeIPA with a user privileged enough to change >>>>>>>> password for all users but with fully enforced policies. >>>>>>>> >>>>>>>> What do you think about this? Would you consider adding such feature? >>>>>>>> Would you accept patches? >>>>>>>> >>>>>>> Can you please explain why samba needs to connect to IPA and change >>>>>>> the passwords? >>>>>>> In what role you use samba? As a file server or as something else? >>>>>>> I am not sure I follow why you need the password change functionality. >>>>>>> There is a way to setup Samba FS with IPA without trying to make IPA a >>>>>>> back end for Samba. >>>>>>> I can try to dig some writeups on the matter if you are interested. >>>>>> Samba 3 when used as a PDC/BDC can use a LDAP server as its user/group >>>>>> database. To do that samba connects with a privileged user to the LDAP >>>>>> directory and manages some attributes of users and groups in the >>>>>> directory, adding the sambaSAMAccount objectclass and the sambaSID >>>>>> attribute to users, groups and machines of the domain. >>>>>> >>>>>> When users of Windows workstations in a samba domain change their >>>>>> passwords samba updates the sambaNTPassword, userPassword, >>>>>> sambaLastPwdChange, sambaPwdMustChange attributes of the corresponding >>>>>> ldap user. >>>>>> >>>>>> Using freeIPA as ldap user backend for samba works quite well, except >>>>>> for the password policy problem mentioned in last mail and that it is >>>>>> hard to mantain in sync the enabled/disabled status of an account. >>>>> What is the value of using FreeIPA as a Samba back end in >>>>> comparison to other variants? >>>>> Why IPA is more interesting than say 389-DS or OpenLDAP or native >>>>> Samba? >>>> IPA will keep all of your passwords in sync - userPassword, >>>> sambaNTPassword, sambaLMPassword, and your kerberos passwords. 389 >>>> cannot do this - the functionality that does this is provided by an >>>> IPA password plugin. Openldap has a similar plugin, but I think it >>>> is "contrib" and not "officially supported". >>>> >>> >>> I know that Endi did the work to make 389 be a viable back end for >>> Samba and it passed all the Samba torture tests so I am not sure I >>> agree with you. Samba does the kerberos operations itself and uses >>> LDAP as a storage only. This is why I am struggling to understand the >>> use case. It seems that Loris has a different configuration that I do >>> not quite understand, thus questions. >>> >>>>> What other features of IPA are used in such setup? >>>>> >>>>> Answering these (and may be other) questions would help us to >>>>> understand how common is the use case that you brought up. >> First of all, the use case is that of using Samba 3 as a Domain >> Controller. Here in Venezuela the government itself promotes the use of >> free software so most government agencies and industries won't install >> Active Directory to administer windows desktops. There are some medium >> to large deployments of Samba 3 as a domain controller here, and there >> are a number of Linux desktops deployed in the same networks. >> >> When you use Samba 3 as a Domain Controller with a largish number of >> users and machines is mandatory to use a Ldap server as a backend, and >> for that you have basically two choices which are of course OpenLdap and >> 389-DS, but those servers have to be combined with some administration >> tools to be really useful. >> >> The ideal choice for us is 389-DS with freeIPA as an administration >> framework because of: >> >> * Really easy to use to administer users and groups. Those users >> and groups are visible from the samba domain and from linux >> machines in the IPA realm or from legacy unix and linux machines >> configured as ldap clients >> * ipa_pwd_extop keeps ldap, samba and kerberos passwords in sync. >> Well, this could be better >> * freeIPA sets up 389-ds very well, with sane indexes and >> permissions. For good performance with samba you just have to >> add indexes for some samba attributes >> >> So you set up Samba 3 with freeIPA, and use the ipa tools to administer >> users and groups, and those users and group can login in windows and >> linux workstations. >> >> To set up Samba 3 with freeIPA 2.x you have to: >> >> * extend the ipa user and group objects to have them add the >> sambaSAMAccount and SambaGroupType objectclasses >> * add a Distributed Numeric Assignment configuration to have >> 389-DS generate the sambaSID for objects with the above >> objectClasses >> * add a script used by samba 3 when a windows machine tries to >> join the domain, so that ipa creates a new host if it doesn't >> exist >> * add indexes for common samba attributes. >> >> The alternative to all this, using free software, would be using samba 4 >> and have Linux workstations join the Samba 4 domain, or use samba 4 for >> the windows domain with a trust relationship to ipa 3 when it is ready. >> >> Thank you, >> Loris Santamaria >> > >Thank you for the detailed explanation. >It makes perfect sense now. > >Since the LDAP back ends are not supported by Samba we either have to >support them ourselves or make an effort to allow joining the Windows >workstations into the IPA domain. IMO the latter is a more promising smbd from Samba 4 supports LDAP backends. Samba 4 AD DC does not support a separate LDAP backend, it uses its integrated LDAP server implementation. So there are two different cases: using smbd or using Samba AD DC. We can try to cover first one with FreeIPA v3 as we use smbd there already with LDAP backend but it would still require some work. >avenue in a long run as it would address more use cases than trying to >solve it using old Samba3 DC. >Would be interesting to really have a project that would build an IdM >client for Windows using Kerberos for Windows. There is a project Opendirectory (in Russian: http://relay.logotek.ru/~viking/opendirectory/) which attempts to build its own way to implement this. Instead of using domain infra it uses KDC+389-ds on Linux side and home-brew Windows 'client' that imports users and groups from LDAP into local authority storage on each Windows machine and uses Kerberos (MIT-compatible) to auth them. Source code of Windows side is available, written in C#. The issues with such approach are related to local storage sync, SID inconsistencies, and questionable scalability of the client side. In addition, there are many challenges ahead in Opendirectory project that we already solved in FreeIPA in past years, including various 389-ds plugins. Simo and I were looking into what it will take to join Windows client to FreeIPA v3 smbd without introducing Windows-side changes. On Windows side: - NT style join DomainCompatibilityMode=1 causes Windows7 to avoid contacting both Kerberos and LDAP but fail on creating machine account -- IPA side issue, can be fixed but we prefer to fix it by creating hosts in IPA during join. This will not get Kerberos auth working on Windows side but will still allow to use Windows machines. The key point here is that single sign-on will work against smbd and across Windows machines and unified SID management will be applied to all machines. This is shortest way to get Windows machines on FreeIPA smbd domain. - AD style join DomainCompatibilityMode=0 treats Windows7 into following: - discover IPA via SRV records - auth via IPA KDC (successful) - attempt to connect to IPA LDAP with SASL GSSAPI - the connection then abruptly stops after LDAP server replies SASL bind is in progress, no packets are sent from both sides 389-ds advertises DIGEST-MD5 SASL if cyrus-sasl-digest-md5 package is installed even if there is no configuration done properly to support DIGEST-MD5. As result, Windows7 attempts to use DIGEST-MD5 to SASL auth and LDAP server fails. We have filed ticket for 389-ds to solve this problem, will be attacked in 1.3 cycle: https://fedorahosted.org/389/ticket/395 Once we'll solve the 'connection drops' issue, we can see what Windows wants from our LDAP and how to avert it from contacting. No need to repeat full AD path here as it simply what Samba 4 AD DC already gives, so we want to understand if it is possible to convince Windows that we are NT-style domain yet with Kerberos. -- / Alexander Bokovoy From rcritten at redhat.com Tue Jul 3 22:12:08 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 03 Jul 2012 18:12:08 -0400 Subject: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ Message-ID: <4FF36E38.2040600@redhat.com> If you pass in --server and --fixed-primary then don't add _srv_ to ipa_server in sssd.conf. This necessitates the desire to be able to provide multiple servers so make --server accept multiple values. This represents the bulk of the code changes. In every case we only use the additional values in sssd.conf. I also made some minor tweaks to discovery. There were cases where DNS discovery wasn't successful but we set dnsok anyway which could cause some cascading issues. There are a ton of possible corner cases with this so please, be brutal. I tested the following against a DNS server that had SRV records and against one that did not. - ipa-client-install - ipa-client-install --server=ipa.example.com --domain=example.com - ipa-client-install --server=ipa.example.com --server=ipa1.example.com --domain-example.com - ipa-client-install -server=ipa.example.com --server=ipa1.example.com --domain-example.com --fixed-primary - ipa-client-install -server=ipa.example.com --server=ipa1.example.com --domain-example.com --fixed-primary --no-sssd - ipa-client-install -server=ipa.example.com --server=ipa1.example.com --domain-example.com --no-sssd rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1032-client.patch Type: text/x-diff Size: 16668 bytes Desc: not available URL: From mkosek at redhat.com Wed Jul 4 07:13:08 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 04 Jul 2012 09:13:08 +0200 Subject: [Freeipa-devel] [PATCH] 283 Improve address family handling in sockets Message-ID: <4FF3ED04.9020002@redhat.com> I did various tests with IPv4 and IPv6 and everything worked for me. I also tried a mixed IPv4+IPv6 and IPv6-only environment and I was able to install an IPv6-only replica without issues. --- Many functions use low-level socket interface for connection or various checks. However, most of the time we don't respect automatic address family detection but rather try to force our values. This may cause either redundat connection tries when an address family is disabled on system tries or even crashes when socket exceptions are not properly caught. Instead of forcing address families to socket, rather use getaddrinfo interface to automatically retrieve a list of all relevant address families and other connection settings when connecting to remote/local machine or binding to a local port. Now, we will also fill correctly all connection parameters like flowinfo and scopeid for IPv6 connections which will for example prevent issues with scoped IPv6 addresses. bind_port_responder function was changed to at first try to bind to IPv6 wildcard address before IPv4 as IPv6 socket is able to accept both IPv4 and IPv6 connections (unlike IPv4 socket). nsslib connection was refactored to use nss.io.AddrInfo class to get all the available connections. Socket is now not created by default in NSSConnection class initializer, but rather when the actual connection is being made, becase we do not an address family where connection is successful. https://fedorahosted.org/freeipa/ticket/2695 -- Martin Kosek Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-283-improve-address-family-handling-in-sockets.patch Type: text/x-patch Size: 20508 bytes Desc: not available URL: From mkosek at redhat.com Wed Jul 4 10:35:59 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 04 Jul 2012 12:35:59 +0200 Subject: [Freeipa-devel] [PATCH] 1031 run cleanallruv task In-Reply-To: <4FF3047C.9020202@redhat.com> References: <4FF3047C.9020202@redhat.com> Message-ID: <4FF41C8F.2020003@redhat.com> On 07/03/2012 04:41 PM, Rob Crittenden wrote: > Deleting a replica can leave a replication vector (RUV) on the other servers. > This can confuse things if the replica is re-added, and it also causes the > server to calculate changes against a server that may no longer exist. > > 389-ds-base provides a new task that self-propogates itself to all available > replicas to clean this RUV data. > > This patch will create this task at deletion time to hopefully clean things up. > > It isn't perfect. If any replica is down or unavailable at the time the > cleanruv task fires, and then comes back up, the old RUV data may be > re-propogated around. > > To make things easier in this case I've added two new commands to > ipa-replica-manage. The first lists the replication ids of all the servers we > have a RUV for. Using this you can call clean_ruv with the replication id of a > server that no longer exists to try the cleanallruv step again. > > This is quite dangerous though. If you run cleanruv against a replica id that > does exist it can cause a loss of data. I believe I've put in enough scary > warnings about this. > > rob > Good work there, this should make cleaning RUVs much easier than with the previous version. This is what I found during review: 1) list_ruv and clean_ruv command help in man is quite lost. I think it would help if we for example have all info for commands indented. This way user could simply over-look the new commands in the man page. 2) I would rename new commands to clean-ruv and list-ruv to make them consistent with the rest of the commands (re-initialize, force-sync). 3) It would be nice to be able to run clean_ruv command in an unattended way (for better testing), i.e. respect --force option as we already do for ipa-replica-manage del. This fix would aid test automation in the future. 4) (minor) The new question (and the del too) does not react too well for CTRL+D: # ipa-replica-manage clean_ruv 3 --force Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 Cleaning the wrong replica ID will cause that server to no longer replicate so it may miss updates while the process is running. It would need to be re-initialized to maintain consistency. Be very careful. Continue to clean? [no]: unexpected error: 5) Help for clean_ruv command without a required parameter is quite confusing as it reports that command is wrong and not the parameter: # ipa-replica-manage clean_ruv Usage: ipa-replica-manage [options] ipa-replica-manage: error: must provide a command [clean_ruv | force-sync | disconnect | connect | del | re-initialize | list | list_ruv] It seems you just forgot to specify the error message in the command definition 6) When the remote replica is down, the clean_ruv command fails with an unexpected error: [root at vm-086 ~]# ipa-replica-manage clean_ruv 5 Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 Cleaning the wrong replica ID will cause that server to no longer replicate so it may miss updates while the process is running. It would need to be re-initialized to maintain consistency. Be very careful. Continue to clean? [no]: y unexpected error: {'desc': 'Operations error'} /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors: [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: failed to connect to repl agreement connection (cn=meTovm-055.idm.lab.bos.redhat.com,cn=replica, cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping tree,cn=config), error 105 [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: replica (cn=meTovm-055.idm.lab. bos.redhat.com,cn=replica,cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping tree, cn=config) has not been cleaned. You will need to rerun the CLEANALLRUV task on this replica. [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: Task failed (1) In this case I think we should inform user that the command failed, possibly because of disconnected replicas and that they could enable the replicas and try again. 7) (minor) "pass" is now redundant in replication.py: + except ldap.INSUFFICIENT_ACCESS: + # We can't make the server we're removing read-only but + # this isn't a show-stopper + root_logger.debug("No permission to switch replica to read-only, continuing anyway") + pass Martin From pvoborni at redhat.com Wed Jul 4 13:08:57 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 04 Jul 2012 15:08:57 +0200 Subject: [Freeipa-devel] [PATCH] 165 Display loginas information only after login In-Reply-To: <4FF1C31E.7010205@redhat.com> References: <4FEC6506.7050205@redhat.com> <4FECC2DF.4030200@redhat.com> <4FF153D8.70206@redhat.com> <4FF1C31E.7010205@redhat.com> Message-ID: <4FF44069.1020409@redhat.com> On 07/02/2012 05:49 PM, Endi Sukma Dewata wrote: > ACK. Some more comments below. Feel free to fix before push or later > separately. Implemented most of the issues, look bellow. Update patch attached. > > On 7/2/2012 2:55 AM, Petr Vobornik wrote: >> On 06/28/2012 10:47 PM, Endi Sukma Dewata wrote: >>> On 6/28/2012 9:07 AM, Petr Vobornik wrote: >>>> Message 'Logged in as: user at FREEIPA.ORG' was displayed before user was >>>> logged in. It was wrong. >>>> >>>> Now 'Logged in as: XXX' is displayed only when user XXX is logged >>>> in. So >>>> no more user at FREEIPA.ORG :) . >>> >>> It might be better to use visibility instead of display to reserve the >>> space. Right now the password expiration warning will initially appear >>> on the right, then shift to the left when the "Logged in as" appears. >>> >> Seems like better approach. Updated patch attached. > > The message still shifts, but this time from left to right, probably > because the "loggedinas" element doesn't have a fixed width. I don't like fixed width for loggedinas - some names can be long and so we don't know how much space should be reserved. To avoid it, warning and login info are now displayed at the same time. > >> Another improvement might be: display password expiration warning at the >> same time as login information. What do you think? Does it matter? > > Yes, I was thinking about that too. It doesn't really matter much but I > agree it would look better if they appear at the same time. > > The "user at FREEIPA.ORG" in the HTML code is never visible anymore, so > feel free to remove it. You can also replace the with a > then define the style in CSS. Removed and replaced. > > A separate issue, under IPA Server tab, the Trusts menu comes after > Configuration. Would it make more sense to show Configuration last > because Configuration is really like "Other Settings"? > Agree, https://fedorahosted.org/freeipa/ticket/2900 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0165-2-Display-loginas-information-only-after-login.patch Type: text/x-patch Size: 4046 bytes Desc: not available URL: From pvoborni at redhat.com Wed Jul 4 13:20:22 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 04 Jul 2012 15:20:22 +0200 Subject: [Freeipa-devel] [PATCH] 166 Moved configuration to last position in navigation Message-ID: <4FF44316.5010703@redhat.com> Configuration was the last navigation item in IPA server tab. Trusts changed it. It was wrong because configuration is like 'other settings' and so it should be last. This patch moves configuration navigation item to the last position again. https://fedorahosted.org/freeipa/ticket/2900 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0166-Moved-configuration-to-last-position-in-navigation.patch Type: text/x-patch Size: 1145 bytes Desc: not available URL: From mkosek at redhat.com Wed Jul 4 14:59:20 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 04 Jul 2012 16:59:20 +0200 Subject: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ In-Reply-To: <4FF36E38.2040600@redhat.com> References: <4FF36E38.2040600@redhat.com> Message-ID: <4FF45A48.1050801@redhat.com> On 07/04/2012 12:12 AM, Rob Crittenden wrote: > If you pass in --server and --fixed-primary then don't add _srv_ to ipa_server > in sssd.conf. > > This necessitates the desire to be able to provide multiple servers so make > --server accept multiple values. This represents the bulk of the code changes. > In every case we only use the additional values in sssd.conf. > > I also made some minor tweaks to discovery. There were cases where DNS > discovery wasn't successful but we set dnsok anyway which could cause some > cascading issues. > > There are a ton of possible corner cases with this so please, be brutal. > > I tested the following against a DNS server that had SRV records and against > one that did not. > > - ipa-client-install > - ipa-client-install --server=ipa.example.com --domain=example.com > - ipa-client-install --server=ipa.example.com --server=ipa1.example.com > --domain-example.com > - ipa-client-install -server=ipa.example.com --server=ipa1.example.com > --domain-example.com --fixed-primary > - ipa-client-install -server=ipa.example.com --server=ipa1.example.com > --domain-example.com --fixed-primary --no-sssd > - ipa-client-install -server=ipa.example.com --server=ipa1.example.com > --domain-example.com --no-sssd > > rob I did various checks, generally the patch behaves ok, I did not find any major bug. I have just 2 questions/suggestions: 1) Since we allow more fixed servers to be passed as --server parameter, we could name them all in /etc/krb5.conf in "kdc" and "admin_server" options when DNS is not OK instead of writing just the first one in the list. Kerberos tools then should be able to fall-back when some of them is not available. 2) What DNS discovery is not OK, we still add _srv_ to ipa_server option in sssd.conf. Is it intentional? Martin From abokovoy at redhat.com Wed Jul 4 17:57:44 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 4 Jul 2012 20:57:44 +0300 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes Message-ID: <20120704175744.GE16309@redhat.com> Hi, when chasing what looked like ccache corruption with Sumit, I've found yet another issue: use of local stack variable in long-time living code. This local stack use was absent in the original patch version and was proposed by Sumit on one of reviews. It worked for us by luck, it should not have. Hence, the patch that fixes the issue by moving service principal to a longer term storage (ipasam private struct). In order to avoid ccache corruption we also need to move back to in-memory ccache. When multiple LSASD and smbd processes try to auth against LDAP in ipasam, they may write to the same ccache (common ccache) when another process reads from it. This is not what we need. And writing to a persistent on-disk ccache is not needed anyway, as smbldap connections re-authenticate themselves (smbldap connections expire in few minutes). The patch also removes kerberos operations that are not needed when using memory ccache. -- / Alexander Bokovoy -------------- next part -------------- >From 9fc235938045c4d98ca8c6f62fc56da4d7b2280f Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 4 Jul 2012 20:47:03 +0300 Subject: [PATCH 4/4] ipasam: improve SASL bind callback SASL bind callback due to refactoring was referencing local variable which didn't exist all the time. Fix that by including a copy of service principal into ipasam long term private struct. Also remove unneeded default ccache management and use in-memory ccache to avoid stepping over multiple LSASD processes. --- daemons/ipa-sam/ipa_sam.c | 58 +++++++++++++++++++++------------------------ 1 file changed, 27 insertions(+), 31 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 9baac1b2d35a640c36fe7f95a6154ec8582649d8..63f2506a3acdbb739a7cb227bfb6f0ffa723a0ab 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -177,6 +177,7 @@ struct ipasam_privates { char *trust_dn; char *flat_name; char *fallback_primary_group; + char *service_principal; }; static LDAP *priv2ld(struct ldapsam_privates *priv) @@ -3125,12 +3126,20 @@ static int ldap_sasl_interact(LDAP *ld, unsigned flags, void *priv_data, void *s return ret; } -static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data) +static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data, krb5_error_code rc) { + const char *errstring = NULL; + if (!data->context) { return; } + if (rc) { + errstring = krb5_get_error_message(data->context, rc); + DEBUG(0,("kerberos error: code=%d, message=%s\n", rc, errstring)); + krb5_free_error_message(data->context, errstring); + } + krb5_free_cred_contents(data->context, &data->creds); if (data->options) { @@ -3182,59 +3191,38 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo rc = krb5_parse_name(data.context, data.name, &data.principal); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_cc_default(data.context, &data.ccache); + rc = krb5_cc_new_unique(data.context, "MEMORY", NULL, &data.ccache); if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_initialize(data.context, data.ccache, data.principal); - if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_get_full_name(data.context, data.ccache, &ccache_name); - if (rc) { - if (ccache_name) { - krb5_free_string(data.context, ccache_name); - } - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_set_default_name(data.context, ccache_name); - if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_kt_resolve(data.context, "FILE:/etc/samba/samba.keytab", &data.keytab); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_get_init_creds_opt_set_out_ccache(data.context, data.options, data.ccache); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_get_init_creds_keytab(data.context, &data.creds, data.principal, data.keytab, 0, NULL, data.options); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } @@ -3247,7 +3235,7 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo DEBUG(0, ("bind_callback: cannot perform interactive SASL bind with GSSAPI\n")); } - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, 0); return ret; } @@ -3304,13 +3292,21 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, uri, false, bind_dn, bind_secret, &ldap_state->smbldap_state); } else { + ldap_state->ipasam_privates->service_principal = talloc_strdup( + ldap_state->ipasam_privates, + service_principal); + if (ldap_state->ipasam_privates->service_principal == NULL) { + DEBUG(0, ("Failed to create service principal.\n")); + return NT_STATUS_NO_MEMORY; + } /* We authenticate via GSSAPI and thus will use kerberos principal to bind our access */ status = smbldap_init(*pdb_method, pdb_get_tevent_context(), uri, false, NULL, NULL, &ldap_state->smbldap_state); if (NT_STATUS_IS_OK(status)) { ldap_state->smbldap_state->bind_callback = bind_callback; - ldap_state->smbldap_state->bind_callback_data = service_principal; + ldap_state->smbldap_state->bind_callback_data = + ldap_state->ipasam_privates->service_principal; } } -- 1.7.10.4 From sbose at redhat.com Wed Jul 4 18:22:11 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 4 Jul 2012 20:22:11 +0200 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <20120704175744.GE16309@redhat.com> References: <20120704175744.GE16309@redhat.com> Message-ID: <20120704182210.GL922@localhost.localdomain> On Wed, Jul 04, 2012 at 08:57:44PM +0300, Alexander Bokovoy wrote: > Hi, > > when chasing what looked like ccache corruption with Sumit, I've found > yet another issue: use of local stack variable in long-time living code. > > This local stack use was absent in the original patch version and was > proposed by Sumit on one of reviews. It worked for us by luck, it should > not have. > > Hence, the patch that fixes the issue by moving service principal to a > longer term storage (ipasam private struct). > > In order to avoid ccache corruption we also need to move back to > in-memory ccache. When multiple LSASD and smbd processes try to auth > against LDAP in ipasam, they may write to the same ccache (common > ccache) when another process reads from it. This is not what we need. > > And writing to a persistent on-disk ccache is not needed anyway, as > smbldap connections re-authenticate themselves (smbldap connections > expire in few minutes). > > The patch also removes kerberos operations that are not needed when > using memory ccache. > > -- > / Alexander Bokovoy This patch works for me and I have no objections using the in-memory ccache, so ACK from my side, but please wait for Simo's answer before pushing. Simo, you were in favour of using the default cache, do you agree as well? Before pushing please fix: ipa_sam.c:3164:8: warning: unused variable 'ccache_name' [-Wunused-variable] bye, Sumit From sbose at redhat.com Wed Jul 4 19:02:32 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 4 Jul 2012 21:02:32 +0200 Subject: [Freeipa-devel] [PATCH] 32 Only check local ID range during ipa-adtrust-install Message-ID: <20120704190232.GM922@localhost.localdomain> Hi, since the default range for the local domain is now created during updates it is not necessary anymore to have duplicated code in adtrustinstance.py. Instead of trying to create a new range the code now only makes some sanity checks. bye, Sumit -------------- next part -------------- From 27616076953028a6ad11977c68e86344d7bca665 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Mon, 2 Jul 2012 18:19:38 +0200 Subject: [PATCH] Only check local ID range during ipa-adtrust-install Since the local ID range it now added during the update process it does not have to be created during ipa-adtrust-install. To be on the safe side we keep some checks for consistency. --- install/tools/ipa-adtrust-install | 2 +- ipaserver/install/adtrustinstance.py | 74 +++++++++++++++++----------------- 2 Dateien ge?ndert, 39 Zeilen hinzugef?gt(+), 37 Zeilen entfernt(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..d34f0ebde54ed3bbcfed73a65e99bae832f2b0bc 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -215,7 +215,7 @@ def main(): smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, options.no_msdcs) - smb.find_local_id_range() + smb.check_local_id_range() smb.create_instance() print """ diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..889e5e2e295eacde962ec95d1ec4ee9557d19265 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -487,49 +487,51 @@ class ADTRUSTInstance(service.Service): self.__setup_sub_dict() - def find_local_id_range(self): + def check_local_id_range(self): + """ + Look for a ID range of the local domain and check if all used IDs are + coming from this range. If more than one range is found we assume that + the configuration is valid, because there are already manually added + ranges. Additionally we assume the range is valid if the RID + attributes are set. + """ self.ldap_connect() - if self.admin_conn.search_s(str(DN(api.env.container_ranges, - self.suffix)), - ldap.SCOPE_ONELEVEL, - "objectclass=ipaDomainIDRange"): - return - try: - entry = self.admin_conn.getEntry(str(DN(('cn', 'admins'), - api.env.container_group, - self.suffix)), - ldap.SCOPE_BASE) - except errors.NotFound: - raise ValueError("No local ID range and no admins group found.\n" \ + entries = self.admin_conn.search_s(str(DN(api.env.container_ranges, + self.suffix)), + ldap.SCOPE_ONELEVEL, + "objectclass=ipadomainidrange") + except ldap.NO_SUCH_OBJECT: + raise ValueError("Ranges container does not exists.\n" \ + "Please upgrade to IPAv3 or later and try again!") + + if not entries: + raise ValueError("No local ID range found.\n" \ "Add local ID range manually and try again!") - base_id = int(entry.getValue('gidNumber')) - id_range_size = 200000 + if len(entries) == 1: + if (entries[0].getValue('ipaBaseRID') or + entries[0].getValue('ipaSecondaryBaseRID')): + return - id_filter = "(&" \ - "(|(objectclass=posixAccount)" \ - "(objectclass=posixGroup)" \ - "(objectclass=ipaIDObject))" \ - "(|(uidNumber<=%d)(uidNumber>=%d)" \ - "(gidNumber<=%d)(gidNumner>=%d)))" % \ - ((base_id - 1), (base_id + id_range_size), - (base_id - 1), (base_id + id_range_size)) - if self.admin_conn.search_s("cn=accounts," + self.suffix, - ldap.SCOPE_SUBTREE, id_filter): - raise ValueError("There are objects with IDs out of the expected" \ - "range.\nAdd local ID range manually and try " \ - "again!") + base_id = int(entries[0].getValue('ipabaseid')) + id_range_size = int(entries[0].getValue('ipaidrangesize')) - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), - api.env.container_ranges, - self.suffix))) - entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) - entry.setValue('ipaBaseID', str(base_id)) - entry.setValue('ipaIDRangeSize', str(id_range_size)) - self.admin_conn.addEntry(entry) + id_filter = "(&" \ + "(|(objectclass=posixAccount)" \ + "(objectclass=posixGroup)" \ + "(objectclass=ipaIDObject))" \ + "(|(uidNumber<=%d)(uidNumber>=%d)" \ + "(gidNumber<=%d)(gidNumner>=%d)))" % \ + ((base_id - 1), (base_id + id_range_size), + (base_id - 1), (base_id + id_range_size)) + if self.admin_conn.search_s(str(DN(api.env.container_accounts, + self.suffix)), + ldap.SCOPE_SUBTREE, id_filter): + raise ValueError("There are objects with IDs out of the " \ + "expected range.\nAdd local ID range " \ + "manually and try again!") def create_instance(self): -- 1.7.10.2 From sbose at redhat.com Wed Jul 4 19:06:17 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 4 Jul 2012 21:06:17 +0200 Subject: [Freeipa-devel] [PATCH] 33 Allow silent build if available Message-ID: <20120704190617.GN922@localhost.localdomain> Hi, this patch allows to use 'make V=0' to get a less verbose output during the compilation of the freeipa C-code. Since AM_SILENT_RULES is only available with automake 1.11 or later, I used m4_ifdef to check if ti is present. bye, Sumit -------------- next part -------------- From 950ff4855415d72154ec7b3386b67a37d4ce3164 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 4 Jul 2012 12:15:05 +0200 Subject: [PATCH] Allow silent build if available --- daemons/configure.ac | 1 + 1 Datei ge?ndert, 1 Zeile hinzugef?gt(+) diff --git a/daemons/configure.ac b/daemons/configure.ac index b94673026a2c6b71670a67b1f629d9960d8fad31..ebf625ebffd8a92e0a3b050955b9376e002ed6c9 100644 --- a/daemons/configure.ac +++ b/daemons/configure.ac @@ -7,6 +7,7 @@ AC_INIT([ipa-server], AC_CONFIG_HEADERS([config.h]) AM_INIT_AUTOMAKE([foreign]) +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES]) AM_MAINTAINER_MODE AC_PROG_CC -- 1.7.10.2 From sbose at redhat.com Wed Jul 4 19:10:17 2012 From: sbose at redhat.com (Sumit Bose) Date: Wed, 4 Jul 2012 21:10:17 +0200 Subject: [Freeipa-devel] [PATCH] 34-35 ipasam fixes Message-ID: <20120704191017.GO922@localhost.localdomain> Hi, the following two patches contain fixes for ipa_sam.c. The first fixes several issues which were found by clang and the second removes some testing stuff I forgot to change. bye, Sumit -------------- next part -------------- From 116631a3fd2a50e3c2b5a44ed4cff44fe4f0ab99 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 4 Jul 2012 16:18:21 +0200 Subject: [PATCH] ipasam: fixes for clang warnings --- daemons/ipa-sam/ipa_sam.c | 48 +++++++++++++++++++-------------------------- 1 Datei ge?ndert, 20 Zeilen hinzugef?gt(+), 28 Zeilen entfernt(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 63f2506a3acdbb739a7cb227bfb6f0ffa723a0ab..d102b4f0c163c4ae084804f9df672cce568af842 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -100,19 +100,6 @@ bool sid_peek_check_rid(const struct dom_sid *exp_dom_sid, const struct dom_sid char *escape_ldap_string(TALLOC_CTX *mem_ctx, const char *s); /* available in libsmbconf.so */ extern const struct dom_sid global_sid_Builtin; /* available in libsecurity.so */ bool secrets_store(const char *key, const void *data, size_t size); /* available in libpdb.so */ -/* from smb_macros.h */ -#define SMB_REALLOC_ARRAY(p,type,count) (type *)realloc_array((p),sizeof(type),(count),true) /* Always frees p on error or s == 0 */ -#define ADD_TO_ARRAY(mem_ctx, type, elem, array, num) \ -do { \ - *(array) = ((mem_ctx) != NULL) ? \ - talloc_realloc(mem_ctx, (*(array)), type, (*(num))+1) : \ - SMB_REALLOC_ARRAY((*(array)), type, (*(num))+1); \ - SMB_ASSERT((*(array)) != NULL); \ - (*(array))[*(num)] = (elem); \ - (*(num)) += 1; \ -} while (0) - - #define LDAP_SUFFIX "dc=ipa,dc=devel" /* FIXME !!! */ #define LDAP_PAGE_SIZE 1024 #define LDAP_OBJ_SAMBASAMACCOUNT "ipaNTUserAttrs" @@ -1216,8 +1203,6 @@ static bool ldapsam_search_grouptype(struct pdb_methods *methods, return false; } - state->connection = ldap_state->smbldap_state; - state->base = talloc_strdup(search, LDAP_SUFFIX); state->connection = ldap_state->smbldap_state; state->scope = LDAP_SCOPE_SUBTREE; @@ -1402,7 +1387,9 @@ static int set_cross_realm_pw(struct ldapsam_privates *ldap_state, goto done; } - ret = create_keys(krbctx, service_princ, discard_const(pwd), NULL, &keys, &err_msg); + ret = create_keys(krbctx, service_princ, discard_const(pwd), NULL, + &keys, &err_msg); + krb5_free_principal(krbctx, service_princ); if (!ret) { if (err_msg != NULL) { DEBUG(1, ("create_keys returned [%s]\n", err_msg)); @@ -1437,7 +1424,6 @@ done: ber_bvfree(reqdata); } free_keys_contents(krbctx, &keys); - krb5_free_principal(krbctx, service_princ); krb5_free_context(krbctx); return ret; @@ -2245,6 +2231,7 @@ static NTSTATUS ipasam_enum_trusted_domains(struct pdb_methods *methods, int scope = LDAP_SCOPE_SUBTREE; LDAPMessage *result = NULL; LDAPMessage *entry = NULL; + struct pdb_trusted_domain **tmp; filter = talloc_asprintf(mem_ctx, "(objectClass=%s)", LDAP_OBJ_TRUSTED_DOMAIN); @@ -2285,16 +2272,20 @@ static NTSTATUS ipasam_enum_trusted_domains(struct pdb_methods *methods, if (!fill_pdb_trusted_domain(*domains, ldap_state, entry, &dom_info)) { + talloc_free(*domains); return NT_STATUS_UNSUCCESSFUL; } - ADD_TO_ARRAY(*domains, struct pdb_trusted_domain *, dom_info, - domains, num_domains); - - if (*domains == NULL) { - DEBUG(1, ("talloc failed\n")); + tmp = talloc_realloc(*domains, *domains, + struct pdb_trusted_domain *, + (*(num_domains))+1); + if (tmp == NULL) { + talloc_free(*domains); return NT_STATUS_NO_MEMORY; } + *domains = tmp; + (*(domains))[*(num_domains)] = dom_info; + (*(num_domains)) += 1; } DEBUG(5, ("ipasam_enum_trusted_domains: got %d domains\n", *num_domains)); @@ -2699,14 +2690,15 @@ static bool ipasam_get_trusteddom_pw(struct pdb_methods *methods, goto done; } + status = get_trust_pwd(tmp_ctx, &td->trust_auth_incoming, + &trustpw, &last_update); + if (!NT_STATUS_IS_OK(status)) { + ret = false; + goto done; + } + /* trusteddom_pw routines do not use talloc yet... */ if (pwd != NULL) { - status = get_trust_pwd(tmp_ctx, &td->trust_auth_incoming, - &trustpw, &last_update); - if (!NT_STATUS_IS_OK(status)) { - ret = false; - goto done; - } *pwd = strdup(trustpw); memset(trustpw, 0, strlen(trustpw)); talloc_free(trustpw); -- 1.7.10.2 -------------- next part -------------- From 3656f29bf55918b0070bcb42e64eee7d0e7ba6bf Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Wed, 4 Jul 2012 16:19:03 +0200 Subject: [PATCH] ipasam: replace testing code --- daemons/ipa-sam/ipa_sam.c | 10 +++++----- 1 Datei ge?ndert, 5 Zeilen hinzugef?gt(+), 5 Zeilen entfernt(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index d102b4f0c163c4ae084804f9df672cce568af842..9e553551514da0e07babf8a06e59c17cba51ebaf 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -100,7 +100,7 @@ bool sid_peek_check_rid(const struct dom_sid *exp_dom_sid, const struct dom_sid char *escape_ldap_string(TALLOC_CTX *mem_ctx, const char *s); /* available in libsmbconf.so */ extern const struct dom_sid global_sid_Builtin; /* available in libsecurity.so */ bool secrets_store(const char *key, const void *data, size_t size); /* available in libpdb.so */ -#define LDAP_SUFFIX "dc=ipa,dc=devel" /* FIXME !!! */ + #define LDAP_PAGE_SIZE 1024 #define LDAP_OBJ_SAMBASAMACCOUNT "ipaNTUserAttrs" #define LDAP_OBJ_TRUSTED_DOMAIN "ipaNTTrustedDomain" @@ -1044,12 +1044,12 @@ static bool ldapsam_search_users(struct pdb_methods *methods, state->connection = ldap_state->smbldap_state; if ((acct_flags != 0) && ((acct_flags & ACB_NORMAL) != 0)) - state->base = LDAP_SUFFIX; + state->base = ldap_state->ipasam_privates->base_dn; else if ((acct_flags != 0) && ((acct_flags & (ACB_WSTRUST|ACB_SVRTRUST|ACB_DOMTRUST)) != 0)) - state->base = LDAP_SUFFIX; + state->base = ldap_state->ipasam_privates->base_dn; else - state->base = LDAP_SUFFIX; + state->base = ldap_state->ipasam_privates->base_dn; state->acct_flags = acct_flags; state->base = talloc_strdup(search, state->base); @@ -1203,7 +1203,7 @@ static bool ldapsam_search_grouptype(struct pdb_methods *methods, return false; } - state->base = talloc_strdup(search, LDAP_SUFFIX); + state->base = talloc_strdup(search, ldap_state->ipasam_privates->base_dn); state->connection = ldap_state->smbldap_state; state->scope = LDAP_SCOPE_SUBTREE; state->filter = talloc_asprintf(search, "(&(objectclass=%s)" -- 1.7.10.2 From abokovoy at redhat.com Wed Jul 4 20:10:25 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 4 Jul 2012 23:10:25 +0300 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <20120704182210.GL922@localhost.localdomain> References: <20120704175744.GE16309@redhat.com> <20120704182210.GL922@localhost.localdomain> Message-ID: <20120704201025.GF16309@redhat.com> On Wed, 04 Jul 2012, Sumit Bose wrote: >On Wed, Jul 04, 2012 at 08:57:44PM +0300, Alexander Bokovoy wrote: >> Hi, >> >> when chasing what looked like ccache corruption with Sumit, I've found >> yet another issue: use of local stack variable in long-time living code. >> >> This local stack use was absent in the original patch version and was >> proposed by Sumit on one of reviews. It worked for us by luck, it should >> not have. >> >> Hence, the patch that fixes the issue by moving service principal to a >> longer term storage (ipasam private struct). >> >> In order to avoid ccache corruption we also need to move back to >> in-memory ccache. When multiple LSASD and smbd processes try to auth >> against LDAP in ipasam, they may write to the same ccache (common >> ccache) when another process reads from it. This is not what we need. >> >> And writing to a persistent on-disk ccache is not needed anyway, as >> smbldap connections re-authenticate themselves (smbldap connections >> expire in few minutes). >> >> The patch also removes kerberos operations that are not needed when >> using memory ccache. >> >> -- >> / Alexander Bokovoy > >This patch works for me and I have no objections using the in-memory >ccache, so ACK from my side, but please wait for Simo's answer before >pushing. > >Simo, you were in favour of using the default cache, do you >agree as well? > >Before pushing please fix: > >ipa_sam.c:3164:8: warning: unused variable 'ccache_name' [-Wunused-variable] After talking with Sumit and researching I've changed back to use on-disk ccache and avoid re-initing on every callback call. Attached patch should be good. The only trouble is that you'll need to re-run 'ipa-adtrust-install' as smb.conf's parameter we use has been changed to a different one. That or you can manually run net conf setparm 'global' 'ipasam:principal template' '$FQDN@$REALM' where $FQDN is the fully-qualified domain name of the IPA server where smbd will run and $REALM is IPA realm name. Note that I had to do this change to avoid manipulating by ipasam:principal name as I need both client and server principals to use krb5_get_credentials(). With this change ipasam will build proper principal names by itself. Note that we cannot use use ipasam_privates.realm value because it is not available at the point when callback is called. It is fetched from ldap later, after callback has been successfully used. -- / Alexander Bokovoy -------------- next part -------------- >From de956384778bc7ebae8eec80c8e2127ed25f87df Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 4 Jul 2012 20:47:03 +0300 Subject: [PATCH 2/2] ipasam: improve SASL bind callback SASL bind callback due to refactoring was referencing local variable which didn't exist all the time. Fix that by including a copy of service principal into ipasam long term private struct. Rework ccache handling to avoid re-initing every time callback is called Please note that you need to re-run ipa-adtrust-install as smb.conf option and its meaning has been changed (ipasam:principal vs ipasam:principal template) --- daemons/ipa-sam/ipa_sam.c | 117 +++++++++++++++++++++++++-------------- install/share/smb.conf.template | 2 +- 2 files changed, 76 insertions(+), 43 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 9baac1b2d35a640c36fe7f95a6154ec8582649d8..4666e0f02c19c124f858b0c491af27f42e0a1519 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -177,6 +177,8 @@ struct ipasam_privates { char *trust_dn; char *flat_name; char *fallback_primary_group; + char *server_princ; + char *client_princ; }; static LDAP *priv2ld(struct ldapsam_privates *priv) @@ -3125,12 +3127,20 @@ static int ldap_sasl_interact(LDAP *ld, unsigned flags, void *priv_data, void *s return ret; } -static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data) +static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data, krb5_error_code rc) { + const char *errstring = NULL; + if (!data->context) { return; } + if (rc) { + errstring = krb5_get_error_message(data->context, rc); + DEBUG(0,("kerberos error: code=%d, message=%s\n", rc, errstring)); + krb5_free_error_message(data->context, errstring); + } + krb5_free_cred_contents(data->context, &data->creds); if (data->options) { @@ -3158,21 +3168,27 @@ static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data) } extern const char *lp_parm_const_string(int snum, const char *type, const char *option, const char *def); -static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_principal) +static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_priv) { - char *ccache_name = NULL; krb5_error_code rc; + krb5_creds *out_creds = NULL; + krb5_creds in_creds; struct ipasam_sasl_interact_priv data; + struct ipasam_privates *ipasam_private = NULL; int ret; memset(&data, 0, sizeof(struct ipasam_sasl_interact_priv)); - data.name = (const char*)ipasam_principal; - if (data.name == NULL) { - DEBUG(0, ("bind_callback: ipasam:principal is not set, cannot use GSSAPI bind\n")); + memset(&in_creds, 0, sizeof(krb5_creds)); + + ipasam_private = (struct ipasam_privates*)ipasam_priv; + + if ((ipasam_private->client_princ == NULL) && (ipasam_private->server_princ == NULL)) { + DEBUG(0, ("bind_callback: 'ipasam:principal template' is not set, cannot use GSSAPI bind\n")); return LDAP_LOCAL_ERROR; } + data.name = ipasam_private->client_princ; data.name_len = strlen(data.name); rc = krb5_init_context(&data.context); @@ -3182,60 +3198,60 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo rc = krb5_parse_name(data.context, data.name, &data.principal); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_cc_default(data.context, &data.ccache); - if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_initialize(data.context, data.ccache, data.principal); - if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_get_full_name(data.context, data.ccache, &ccache_name); - if (rc) { - if (ccache_name) { - krb5_free_string(data.context, ccache_name); - } - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - rc = krb5_cc_set_default_name(data.context, ccache_name); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_kt_resolve(data.context, "FILE:/etc/samba/samba.keytab", &data.keytab); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); + rc = krb5_parse_name(data.context, ipasam_private->client_princ, &in_creds.client); if (rc) { - bind_callback_cleanup(&data); + krb5_free_principal(data.context, data.creds.client); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_get_init_creds_opt_set_out_ccache(data.context, data.options, data.ccache); + rc = krb5_parse_name(data.context, ipasam_private->server_princ, &in_creds.server); if (rc) { - bind_callback_cleanup(&data); + krb5_free_principal(data.context, in_creds.server); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_get_init_creds_keytab(data.context, &data.creds, data.principal, data.keytab, - 0, NULL, data.options); + rc = krb5_get_credentials(data.context, KRB5_GC_CACHED, data.ccache, &in_creds, &out_creds); + krb5_free_principal(data.context, in_creds.server); + krb5_free_principal(data.context, in_creds.client); + if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; + rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } + + rc = krb5_get_init_creds_opt_set_out_ccache(data.context, data.options, data.ccache); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } + + rc = krb5_get_init_creds_keytab(data.context, &data.creds, data.principal, data.keytab, + 0, NULL, data.options); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } } ret = ldap_sasl_interactive_bind_s(ldap_struct, @@ -3247,7 +3263,10 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo DEBUG(0, ("bind_callback: cannot perform interactive SASL bind with GSSAPI\n")); } - bind_callback_cleanup(&data); + if (out_creds) { + krb5_free_creds(data.context, out_creds); + } + bind_callback_cleanup(&data, 0); return ret; } @@ -3263,7 +3282,7 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, struct dom_sid ldap_domain_sid; char *bind_dn = NULL; char *bind_secret = NULL; - const char *service_principal = NULL; + const char *princ_template = NULL; LDAPMessage *result = NULL; LDAPMessage *entry = NULL; @@ -3293,9 +3312,9 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, } trim_char( uri, '\"', '\"' ); - service_principal = lp_parm_const_string(-1, "ipasam", "principal", NULL); + princ_template = lp_parm_const_string(-1, "ipasam", "principal template", NULL); - if (service_principal == NULL) { + if (princ_template == NULL) { if (!fetch_ldap_pw(&bind_dn, &bind_secret)) { DEBUG(0, ("pdb_init_ipasam: Failed to retrieve LDAP password from secrets.tdb\n")); return NT_STATUS_NO_MEMORY; @@ -3304,13 +3323,27 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, uri, false, bind_dn, bind_secret, &ldap_state->smbldap_state); } else { + ldap_state->ipasam_privates->client_princ = talloc_asprintf(ldap_state->ipasam_privates, + "cifs/%s", + princ_template); + if (ldap_state->ipasam_privates->client_princ == NULL) { + DEBUG(0, ("Failed to create ipasam client principal out of the template.\n")); + return NT_STATUS_NO_MEMORY; + } + ldap_state->ipasam_privates->server_princ = talloc_asprintf(ldap_state->ipasam_privates, + "ldap/%s", + princ_template); + if (ldap_state->ipasam_privates->client_princ == NULL) { + DEBUG(0, ("Failed to create ipasam server principal out of the template.\n")); + return NT_STATUS_NO_MEMORY; + } /* We authenticate via GSSAPI and thus will use kerberos principal to bind our access */ status = smbldap_init(*pdb_method, pdb_get_tevent_context(), uri, false, NULL, NULL, &ldap_state->smbldap_state); if (NT_STATUS_IS_OK(status)) { ldap_state->smbldap_state->bind_callback = bind_callback; - ldap_state->smbldap_state->bind_callback_data = service_principal; + ldap_state->smbldap_state->bind_callback_data = ldap_state->ipasam_privates; } } diff --git a/install/share/smb.conf.template b/install/share/smb.conf.template index 3107350aa6e94514354b73f0152846e1d01e1e68..e0954e1661542eb43ee78adb4675cc2e18250dcc 100644 --- a/install/share/smb.conf.template +++ b/install/share/smb.conf.template @@ -18,7 +18,7 @@ ldap suffix = $SUFFIX ldap user suffix = cn=users,cn=accounts ldap group suffix = cn=groups,cn=accounts ldap machine suffix = cn=computers,cn=accounts -ipasam:principal = cifs/$FQDN@$REALM +ipasam:principal template = $FQDN@$REALM rpc_server:epmapper = external rpc_server:lsarpc = external rpc_server:lsass = external -- 1.7.10.4 From simo at redhat.com Thu Jul 5 12:29:18 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 05 Jul 2012 08:29:18 -0400 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <20120704201025.GF16309@redhat.com> References: <20120704175744.GE16309@redhat.com> <20120704182210.GL922@localhost.localdomain> <20120704201025.GF16309@redhat.com> Message-ID: <1341491358.14199.158.camel@willson.li.ssimo.org> On Wed, 2012-07-04 at 23:10 +0300, Alexander Bokovoy wrote: > On Wed, 04 Jul 2012, Sumit Bose wrote: > >On Wed, Jul 04, 2012 at 08:57:44PM +0300, Alexander Bokovoy wrote: > >> Hi, > >> > >> when chasing what looked like ccache corruption with Sumit, I've found > >> yet another issue: use of local stack variable in long-time living code. > >> > >> This local stack use was absent in the original patch version and was > >> proposed by Sumit on one of reviews. It worked for us by luck, it should > >> not have. > >> > >> Hence, the patch that fixes the issue by moving service principal to a > >> longer term storage (ipasam private struct). > >> > >> In order to avoid ccache corruption we also need to move back to > >> in-memory ccache. When multiple LSASD and smbd processes try to auth > >> against LDAP in ipasam, they may write to the same ccache (common > >> ccache) when another process reads from it. This is not what we need. > >> > >> And writing to a persistent on-disk ccache is not needed anyway, as > >> smbldap connections re-authenticate themselves (smbldap connections > >> expire in few minutes). > >> > >> The patch also removes kerberos operations that are not needed when > >> using memory ccache. > >> > >> -- > >> / Alexander Bokovoy > > > >This patch works for me and I have no objections using the in-memory > >ccache, so ACK from my side, but please wait for Simo's answer before > >pushing. > > > >Simo, you were in favour of using the default cache, do you > >agree as well? > > > >Before pushing please fix: > > > >ipa_sam.c:3164:8: warning: unused variable 'ccache_name' [-Wunused-variable] > > After talking with Sumit and researching I've changed back to use > on-disk ccache and avoid re-initing on every callback call. Thank, this is what I would have asked anyway. > Attached patch should be good. The only trouble is that you'll need to > re-run 'ipa-adtrust-install' as smb.conf's parameter we use has been > changed to a different one. That or you can manually run > > net conf setparm 'global' 'ipasam:principal template' '$FQDN@$REALM' > > where $FQDN is the fully-qualified domain name of the IPA server where > smbd will run and $REALM is IPA realm name. We do not need to store this information in the configuration explicitly, we have all that info already. So NACK, please remove this configuration option completely and fetch fqdn from the system (gethostname) and the realm from the krb5 context. > Note that I had to do this change to avoid manipulating by ipasam:principal > name as I need both client and server principals to use krb5_get_credentials(). > With this change ipasam will build proper principal names by itself. > > Note that we cannot use use ipasam_privates.realm value because it is > not available at the point when callback is called. It is fetched from > ldap later, after callback has been successfully used. See above, we have all the information we need via simple API calls to glibc or krb5 libs, we do not need to retrieve that information by other means. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Jul 5 12:33:03 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 05 Jul 2012 08:33:03 -0400 Subject: [Freeipa-devel] [PATCH] 32 Only check local ID range during ipa-adtrust-install In-Reply-To: <20120704190232.GM922@localhost.localdomain> References: <20120704190232.GM922@localhost.localdomain> Message-ID: <1341491583.14199.161.camel@willson.li.ssimo.org> On Wed, 2012-07-04 at 21:02 +0200, Sumit Bose wrote: > Hi, > > since the default range for the local domain is now created during > updates it is not necessary anymore to have duplicated code in > adtrustinstance.py. Instead of trying to create a new range the code now > only makes some sanity checks. NACK, I think the last check is too strong. Some people may need to add special users (legacy applications, migrations, etc.. with specific IDs but they do not care at all for those to be able to get a SID/PAC. I think we should not fail but only warn that some users seem to fall out of the range. Ideally in interactive mode, we ask if the admin wants to see the list of objects falling off the ranges. And in any case ask if the user wants to proceed anyway, with a warning that those users/groups will not be usable for the cross-realm trust unless they are part of a range of IDs comprising their specific ID. Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Thu Jul 5 12:41:38 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 05 Jul 2012 08:41:38 -0400 Subject: [Freeipa-devel] [PATCH] 33 Allow silent build if available In-Reply-To: <20120704190617.GN922@localhost.localdomain> References: <20120704190617.GN922@localhost.localdomain> Message-ID: <1341492098.14199.162.camel@willson.li.ssimo.org> On Wed, 2012-07-04 at 21:06 +0200, Sumit Bose wrote: > Hi, > > this patch allows to use 'make V=0' to get a less verbose output during > the compilation of the freeipa C-code. Since AM_SILENT_RULES is only > available with automake 1.11 or later, I used m4_ifdef to check if ti is > present. ACK Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Thu Jul 5 13:27:01 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 5 Jul 2012 16:27:01 +0300 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <1341491358.14199.158.camel@willson.li.ssimo.org> References: <20120704175744.GE16309@redhat.com> <20120704182210.GL922@localhost.localdomain> <20120704201025.GF16309@redhat.com> <1341491358.14199.158.camel@willson.li.ssimo.org> Message-ID: <20120705132701.GH16309@redhat.com> On Thu, 05 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-04 at 23:10 +0300, Alexander Bokovoy wrote: >> On Wed, 04 Jul 2012, Sumit Bose wrote: >> >On Wed, Jul 04, 2012 at 08:57:44PM +0300, Alexander Bokovoy wrote: >> >> Hi, >> >> >> >> when chasing what looked like ccache corruption with Sumit, I've found >> >> yet another issue: use of local stack variable in long-time living code. >> >> >> >> This local stack use was absent in the original patch version and was >> >> proposed by Sumit on one of reviews. It worked for us by luck, it should >> >> not have. >> >> >> >> Hence, the patch that fixes the issue by moving service principal to a >> >> longer term storage (ipasam private struct). >> >> >> >> In order to avoid ccache corruption we also need to move back to >> >> in-memory ccache. When multiple LSASD and smbd processes try to auth >> >> against LDAP in ipasam, they may write to the same ccache (common >> >> ccache) when another process reads from it. This is not what we need. >> >> >> >> And writing to a persistent on-disk ccache is not needed anyway, as >> >> smbldap connections re-authenticate themselves (smbldap connections >> >> expire in few minutes). >> >> >> >> The patch also removes kerberos operations that are not needed when >> >> using memory ccache. >> >> >> >> -- >> >> / Alexander Bokovoy >> > >> >This patch works for me and I have no objections using the in-memory >> >ccache, so ACK from my side, but please wait for Simo's answer before >> >pushing. >> > >> >Simo, you were in favour of using the default cache, do you >> >agree as well? >> > >> >Before pushing please fix: >> > >> >ipa_sam.c:3164:8: warning: unused variable 'ccache_name' [-Wunused-variable] >> >> After talking with Sumit and researching I've changed back to use >> on-disk ccache and avoid re-initing on every callback call. > >Thank, this is what I would have asked anyway. > >> Attached patch should be good. The only trouble is that you'll need to >> re-run 'ipa-adtrust-install' as smb.conf's parameter we use has been >> changed to a different one. That or you can manually run >> >> net conf setparm 'global' 'ipasam:principal template' '$FQDN@$REALM' >> >> where $FQDN is the fully-qualified domain name of the IPA server where >> smbd will run and $REALM is IPA realm name. > >We do not need to store this information in the configuration >explicitly, we have all that info already. > >So NACK, please remove this configuration option completely and fetch >fqdn from the system (gethostname) and the realm from the krb5 context. > >> Note that I had to do this change to avoid manipulating by ipasam:principal >> name as I need both client and server principals to use krb5_get_credentials(). >> With this change ipasam will build proper principal names by itself. >> >> Note that we cannot use use ipasam_privates.realm value because it is >> not available at the point when callback is called. It is fetched from >> ldap later, after callback has been successfully used. > >See above, we have all the information we need via simple API calls to >glibc or krb5 libs, we do not need to retrieve that information by other >means. New patch is attached. Works for me without any smb.conf parameters. -- / Alexander Bokovoy -------------- next part -------------- >From a10a96ce0c1d024fa4c25ade692d8e2abff1ba44 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 4 Jul 2012 20:47:03 +0300 Subject: [PATCH 2/2] ipasam: improve SASL bind callback SASL bind callback due to refactoring was referencing local variable which didn't exist all the time. Fix that by including a copy of service principals into ipasam long term private struct. Rework ccache handling to avoid re-initing every time callback is called --- daemons/ipa-sam/ipa_sam.c | 180 +++++++++++++++++++++++++++++---------- install/share/smb.conf.template | 1 - 2 files changed, 137 insertions(+), 44 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 9baac1b2d35a640c36fe7f95a6154ec8582649d8..ab766dce87563ed7f5dba1fc08f3eea1d5cea727 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -177,6 +177,8 @@ struct ipasam_privates { char *trust_dn; char *flat_name; char *fallback_primary_group; + char *server_princ; + char *client_princ; }; static LDAP *priv2ld(struct ldapsam_privates *priv) @@ -3125,12 +3127,20 @@ static int ldap_sasl_interact(LDAP *ld, unsigned flags, void *priv_data, void *s return ret; } -static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data) +static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data, krb5_error_code rc) { + const char *errstring = NULL; + if (!data->context) { return; } + if (rc) { + errstring = krb5_get_error_message(data->context, rc); + DEBUG(0,("kerberos error: code=%d, message=%s\n", rc, errstring)); + krb5_free_error_message(data->context, errstring); + } + krb5_free_cred_contents(data->context, &data->creds); if (data->options) { @@ -3157,22 +3167,27 @@ static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data) data->context = NULL; } -extern const char *lp_parm_const_string(int snum, const char *type, const char *option, const char *def); -static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_principal) +static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_priv) { - char *ccache_name = NULL; krb5_error_code rc; + krb5_creds *out_creds = NULL; + krb5_creds in_creds; struct ipasam_sasl_interact_priv data; + struct ipasam_privates *ipasam_private = NULL; int ret; memset(&data, 0, sizeof(struct ipasam_sasl_interact_priv)); - data.name = (const char*)ipasam_principal; - if (data.name == NULL) { - DEBUG(0, ("bind_callback: ipasam:principal is not set, cannot use GSSAPI bind\n")); + memset(&in_creds, 0, sizeof(krb5_creds)); + + ipasam_private = (struct ipasam_privates*)ipasam_priv; + + if ((ipasam_private->client_princ == NULL) && (ipasam_private->server_princ == NULL)) { + DEBUG(0, ("bind_callback: 'ipasam:principal template' is not set, cannot use GSSAPI bind\n")); return LDAP_LOCAL_ERROR; } + data.name = ipasam_private->client_princ; data.name_len = strlen(data.name); rc = krb5_init_context(&data.context); @@ -3182,60 +3197,60 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo rc = krb5_parse_name(data.context, data.name, &data.principal); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_cc_default(data.context, &data.ccache); - if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_initialize(data.context, data.ccache, data.principal); - if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - - rc = krb5_cc_get_full_name(data.context, data.ccache, &ccache_name); - if (rc) { - if (ccache_name) { - krb5_free_string(data.context, ccache_name); - } - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; - } - rc = krb5_cc_set_default_name(data.context, ccache_name); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } rc = krb5_kt_resolve(data.context, "FILE:/etc/samba/samba.keytab", &data.keytab); if (rc) { - bind_callback_cleanup(&data); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); + rc = krb5_parse_name(data.context, ipasam_private->client_princ, &in_creds.client); if (rc) { - bind_callback_cleanup(&data); + krb5_free_principal(data.context, data.creds.client); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_get_init_creds_opt_set_out_ccache(data.context, data.options, data.ccache); + rc = krb5_parse_name(data.context, ipasam_private->server_princ, &in_creds.server); if (rc) { - bind_callback_cleanup(&data); + krb5_free_principal(data.context, in_creds.server); + bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; } - rc = krb5_get_init_creds_keytab(data.context, &data.creds, data.principal, data.keytab, - 0, NULL, data.options); + rc = krb5_get_credentials(data.context, KRB5_GC_CACHED, data.ccache, &in_creds, &out_creds); + krb5_free_principal(data.context, in_creds.server); + krb5_free_principal(data.context, in_creds.client); + if (rc) { - bind_callback_cleanup(&data); - return LDAP_LOCAL_ERROR; + rc = krb5_get_init_creds_opt_alloc(data.context, &data.options); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } + + rc = krb5_get_init_creds_opt_set_out_ccache(data.context, data.options, data.ccache); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } + + rc = krb5_get_init_creds_keytab(data.context, &data.creds, data.principal, data.keytab, + 0, NULL, data.options); + if (rc) { + bind_callback_cleanup(&data, rc); + return LDAP_LOCAL_ERROR; + } } ret = ldap_sasl_interactive_bind_s(ldap_struct, @@ -3247,10 +3262,90 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo DEBUG(0, ("bind_callback: cannot perform interactive SASL bind with GSSAPI\n")); } - bind_callback_cleanup(&data); + if (out_creds) { + krb5_free_creds(data.context, out_creds); + } + bind_callback_cleanup(&data, 0); return ret; } +static NTSTATUS ipasam_generate_principals(struct ipasam_privates *privates) { + + krb5_error_code rc; + int ret; + krb5_context context; + NTSTATUS status = NT_STATUS_UNSUCCESSFUL; + char hostname[255]; + char *default_realm = NULL; + + if (!privates) { + return status; + } + + rc = krb5_init_context(&context); + if (rc) { + return status; + } + + ret = gethostname(hostname, sizeof(hostname)); + if (ret == -1) { + DEBUG(1, ("gethostname failed.\n")); + goto done; + } + hostname[sizeof(hostname)-1] = '\0'; + + rc = krb5_get_default_realm(context, &default_realm); + if (rc) { + goto done; + }; + + if (privates->client_princ) { + talloc_free(privates->client_princ); + privates->client_princ = NULL; + } + + privates->client_princ = talloc_asprintf(privates, + "cifs/%s@%s", + hostname, + default_realm); + + if (privates->client_princ == NULL) { + DEBUG(0, ("Failed to create ipasam client principal.\n")); + status = NT_STATUS_NO_MEMORY; + goto done; + } + + if (privates->server_princ) { + talloc_free(privates->server_princ); + privates->server_princ = NULL; + } + + privates->server_princ = talloc_asprintf(privates, + "ldap/%s@%s", + hostname, + default_realm); + + if (privates->server_princ == NULL) { + DEBUG(0, ("Failed to create ipasam server principal.\n")); + status = NT_STATUS_NO_MEMORY; + goto done; + } + + status = NT_STATUS_OK; + +done: + + if (default_realm) { + krb5_free_default_realm(context, default_realm); + } + + if (context) { + krb5_free_context(context); + } + return status; +} + + static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, const char *location) { @@ -3263,7 +3358,6 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, struct dom_sid ldap_domain_sid; char *bind_dn = NULL; char *bind_secret = NULL; - const char *service_principal = NULL; LDAPMessage *result = NULL; LDAPMessage *entry = NULL; @@ -3293,9 +3387,9 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, } trim_char( uri, '\"', '\"' ); - service_principal = lp_parm_const_string(-1, "ipasam", "principal", NULL); + status = ipasam_generate_principals(ldap_state->ipasam_privates); - if (service_principal == NULL) { + if (!NT_STATUS_IS_OK(status)) { if (!fetch_ldap_pw(&bind_dn, &bind_secret)) { DEBUG(0, ("pdb_init_ipasam: Failed to retrieve LDAP password from secrets.tdb\n")); return NT_STATUS_NO_MEMORY; @@ -3310,7 +3404,7 @@ static NTSTATUS pdb_init_ipasam(struct pdb_methods **pdb_method, &ldap_state->smbldap_state); if (NT_STATUS_IS_OK(status)) { ldap_state->smbldap_state->bind_callback = bind_callback; - ldap_state->smbldap_state->bind_callback_data = service_principal; + ldap_state->smbldap_state->bind_callback_data = ldap_state->ipasam_privates; } } diff --git a/install/share/smb.conf.template b/install/share/smb.conf.template index 3107350aa6e94514354b73f0152846e1d01e1e68..086b0fcfe5cff2bc3582f2a89962a99c9095b4bb 100644 --- a/install/share/smb.conf.template +++ b/install/share/smb.conf.template @@ -18,7 +18,6 @@ ldap suffix = $SUFFIX ldap user suffix = cn=users,cn=accounts ldap group suffix = cn=groups,cn=accounts ldap machine suffix = cn=computers,cn=accounts -ipasam:principal = cifs/$FQDN@$REALM rpc_server:epmapper = external rpc_server:lsarpc = external rpc_server:lsass = external -- 1.7.10.4 From dpal at redhat.com Thu Jul 5 16:09:14 2012 From: dpal at redhat.com (Dmitri Pal) Date: Thu, 05 Jul 2012 12:09:14 -0400 Subject: [Freeipa-devel] [PATCH] Add sidgen postop and task In-Reply-To: <20120627162702.GH16889@redhat.com> References: <20120625095310.GU29454@localhost.localdomain> <20120627162702.GH16889@redhat.com> Message-ID: <4FF5BC2A.6020007@redhat.com> On 06/27/2012 12:27 PM, Alexander Bokovoy wrote: > On Mon, 25 Jun 2012, Sumit Bose wrote: >> Hi, >> >> this patch added support to automatically create SIDs for local objects >> as described in ticket https://fedorahosted.org/freeipa/ticket/2825. >> >> The post-operation plugin adds the SID and if necessary the needed >> objectclass for a newly created object. > ACK. > > Works for me in tests. > >> The directory server task can you used to set SID to existing objects in >> one run. Since there were concerns about the amount of replication >> traffic this task accepts a parameter 'delay' to let the task pause for >> the given number of micro-seconds after an object was changed. I also do >> not start the task during ipa-adtrust-install to allow to run the task >> at a more appropriate time. I wonder if it is ok to just have an ldif >> file as example and explain in the docs how to start the task with >> ldapmodify or if a tighter integration is needed. Typically this task >> should be called only once after ipa-adtrust-install. > We probably would need to make something like 'ipa-task-manage' that > would allow listing, enabling, scheduling, and disabling all supported > tasks. > > Something to work on once we have refactored installer/tools > infrastructure in 3.1? > Do we need a ticket for that? -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From simo at redhat.com Thu Jul 5 18:10:47 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 05 Jul 2012 14:10:47 -0400 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <20120705132701.GH16309@redhat.com> References: <20120704175744.GE16309@redhat.com> <20120704182210.GL922@localhost.localdomain> <20120704201025.GF16309@redhat.com> <1341491358.14199.158.camel@willson.li.ssimo.org> <20120705132701.GH16309@redhat.com> Message-ID: <1341511847.14199.181.camel@willson.li.ssimo.org> On Thu, 2012-07-05 at 16:27 +0300, Alexander Bokovoy wrote: > > > >See above, we have all the information we need via simple API calls > to > >glibc or krb5 libs, we do not need to retrieve that information by > other > >means. > New patch is attached. Works for me without any smb.conf parameters. > Ack Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Thu Jul 5 18:39:53 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 05 Jul 2012 14:39:53 -0400 Subject: [Freeipa-devel] [PATCH] 1031 run cleanallruv task In-Reply-To: <4FF41C8F.2020003@redhat.com> References: <4FF3047C.9020202@redhat.com> <4FF41C8F.2020003@redhat.com> Message-ID: <4FF5DF79.6090806@redhat.com> Martin Kosek wrote: > On 07/03/2012 04:41 PM, Rob Crittenden wrote: >> Deleting a replica can leave a replication vector (RUV) on the other servers. >> This can confuse things if the replica is re-added, and it also causes the >> server to calculate changes against a server that may no longer exist. >> >> 389-ds-base provides a new task that self-propogates itself to all available >> replicas to clean this RUV data. >> >> This patch will create this task at deletion time to hopefully clean things up. >> >> It isn't perfect. If any replica is down or unavailable at the time the >> cleanruv task fires, and then comes back up, the old RUV data may be >> re-propogated around. >> >> To make things easier in this case I've added two new commands to >> ipa-replica-manage. The first lists the replication ids of all the servers we >> have a RUV for. Using this you can call clean_ruv with the replication id of a >> server that no longer exists to try the cleanallruv step again. >> >> This is quite dangerous though. If you run cleanruv against a replica id that >> does exist it can cause a loss of data. I believe I've put in enough scary >> warnings about this. >> >> rob >> > > Good work there, this should make cleaning RUVs much easier than with the > previous version. > > This is what I found during review: > > 1) list_ruv and clean_ruv command help in man is quite lost. I think it would > help if we for example have all info for commands indented. This way user could > simply over-look the new commands in the man page. > > > 2) I would rename new commands to clean-ruv and list-ruv to make them > consistent with the rest of the commands (re-initialize, force-sync). > > > 3) It would be nice to be able to run clean_ruv command in an unattended way > (for better testing), i.e. respect --force option as we already do for > ipa-replica-manage del. This fix would aid test automation in the future. > > > 4) (minor) The new question (and the del too) does not react too well for CTRL+D: > > # ipa-replica-manage clean_ruv 3 --force > Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 > > Cleaning the wrong replica ID will cause that server to no > longer replicate so it may miss updates while the process > is running. It would need to be re-initialized to maintain > consistency. Be very careful. > Continue to clean? [no]: unexpected error: > > > 5) Help for clean_ruv command without a required parameter is quite confusing > as it reports that command is wrong and not the parameter: > > # ipa-replica-manage clean_ruv > Usage: ipa-replica-manage [options] > > ipa-replica-manage: error: must provide a command [clean_ruv | force-sync | > disconnect | connect | del | re-initialize | list | list_ruv] > > It seems you just forgot to specify the error message in the command definition > > > 6) When the remote replica is down, the clean_ruv command fails with an > unexpected error: > > [root at vm-086 ~]# ipa-replica-manage clean_ruv 5 > Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 > > Cleaning the wrong replica ID will cause that server to no > longer replicate so it may miss updates while the process > is running. It would need to be re-initialized to maintain > consistency. Be very careful. > Continue to clean? [no]: y > unexpected error: {'desc': 'Operations error'} > > > /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors: > [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: failed > to connect to repl agreement connection > (cn=meTovm-055.idm.lab.bos.redhat.com,cn=replica, > cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping > tree,cn=config), error 105 > [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: replica > (cn=meTovm-055.idm.lab. > bos.redhat.com,cn=replica,cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping > tree, cn=config) has not been cleaned. You will need to rerun the > CLEANALLRUV task on this replica. > [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: Task > failed (1) > > In this case I think we should inform user that the command failed, possibly > because of disconnected replicas and that they could enable the replicas and > try again. > > > 7) (minor) "pass" is now redundant in replication.py: > + except ldap.INSUFFICIENT_ACCESS: > + # We can't make the server we're removing read-only but > + # this isn't a show-stopper > + root_logger.debug("No permission to switch replica to read-only, > continuing anyway") > + pass > I think this addresses everything. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1031-2-cleanruv.patch Type: text/x-diff Size: 11300 bytes Desc: not available URL: From rcritten at redhat.com Thu Jul 5 19:18:15 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 05 Jul 2012 15:18:15 -0400 Subject: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ In-Reply-To: <4FF45A48.1050801@redhat.com> References: <4FF36E38.2040600@redhat.com> <4FF45A48.1050801@redhat.com> Message-ID: <4FF5E877.2090108@redhat.com> Martin Kosek wrote: > On 07/04/2012 12:12 AM, Rob Crittenden wrote: >> If you pass in --server and --fixed-primary then don't add _srv_ to ipa_server >> in sssd.conf. >> >> This necessitates the desire to be able to provide multiple servers so make >> --server accept multiple values. This represents the bulk of the code changes. >> In every case we only use the additional values in sssd.conf. >> >> I also made some minor tweaks to discovery. There were cases where DNS >> discovery wasn't successful but we set dnsok anyway which could cause some >> cascading issues. >> >> There are a ton of possible corner cases with this so please, be brutal. >> >> I tested the following against a DNS server that had SRV records and against >> one that did not. >> >> - ipa-client-install >> - ipa-client-install --server=ipa.example.com --domain=example.com >> - ipa-client-install --server=ipa.example.com --server=ipa1.example.com >> --domain-example.com >> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >> --domain-example.com --fixed-primary >> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >> --domain-example.com --fixed-primary --no-sssd >> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >> --domain-example.com --no-sssd >> >> rob > > I did various checks, generally the patch behaves ok, I did not find any major > bug. I have just 2 questions/suggestions: > > 1) Since we allow more fixed servers to be passed as --server parameter, we > could name them all in /etc/krb5.conf in "kdc" and "admin_server" options when > DNS is not OK instead of writing just the first one in the list. Kerberos tools > then should be able to fall-back when some of them is not available. Sure, that makes sense. Done. > 2) What DNS discovery is not OK, we still add _srv_ to ipa_server option in > sssd.conf. Is it intentional? Yes, it was sort of a future-proofing if SRV records are ever made available. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1032-2-client.patch Type: text/x-diff Size: 17034 bytes Desc: not available URL: From abokovoy at redhat.com Thu Jul 5 19:27:22 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 5 Jul 2012 22:27:22 +0300 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <1341511847.14199.181.camel@willson.li.ssimo.org> References: <20120704175744.GE16309@redhat.com> <20120704182210.GL922@localhost.localdomain> <20120704201025.GF16309@redhat.com> <1341491358.14199.158.camel@willson.li.ssimo.org> <20120705132701.GH16309@redhat.com> <1341511847.14199.181.camel@willson.li.ssimo.org> Message-ID: <20120705192722.GI16309@redhat.com> On Thu, 05 Jul 2012, Simo Sorce wrote: >On Thu, 2012-07-05 at 16:27 +0300, Alexander Bokovoy wrote: >> > >> >See above, we have all the information we need via simple API calls >> to >> >glibc or krb5 libs, we do not need to retrieve that information by >> other >> >means. >> New patch is attached. Works for me without any smb.conf parameters. >> >Ack Thanks. I'll fix DEBUG() statement leftover for ipasam:principal template and will push it. -- / Alexander Bokovoy From riffraff169 at yahoo.com Thu Jul 5 19:49:01 2012 From: riffraff169 at yahoo.com (Lance Dillon) Date: Thu, 5 Jul 2012 12:49:01 -0700 (PDT) Subject: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ In-Reply-To: <4FF5E877.2090108@redhat.com> References: <4FF36E38.2040600@redhat.com> <4FF45A48.1050801@redhat.com> <4FF5E877.2090108@redhat.com> Message-ID: <1341517741.96273.YahooMailNeo@web83802.mail.sp1.yahoo.com> >________________________________ > From: Rob Crittenden >To: Martin Kosek >Cc: freeipa-devel >Sent: Thursday, July 5, 2012 3:18 PM >Subject: Re: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ > >Martin Kosek wrote: >> On 07/04/2012 12:12 AM, Rob Crittenden wrote: >>> If you pass in --server and --fixed-primary then don't add _srv_ to ipa_server >>> in sssd.conf. >>> >>> This necessitates the desire to be able to provide multiple servers? so make >>> --server accept multiple values. This represents the bulk of the code changes. >>> In every case we only use the additional values in sssd.conf. >>> >>> I also made some minor tweaks to discovery. There were cases where DNS >>> discovery wasn't successful but we set dnsok anyway which could cause some >>> cascading issues. >>> >>> There are a ton of possible corner cases with this so please, be brutal. >>> >>> I tested the following against a DNS server that had SRV records and against >>> one that did not. >>> >>> - ipa-client-install >>> - ipa-client-install --server=ipa.example.com --domain=example.com >>> - ipa-client-install --server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com >>> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com --fixed-primary >>> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com --fixed-primary --no-sssd >>> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com --no-sssd >>> >>> rob >> >> I did various checks, generally the patch behaves ok, I did not find any major >> bug. I have just 2 questions/suggestions: >> >> 1) Since we allow more fixed servers to be passed as --server parameter, we >> could name them all in /etc/krb5.conf in "kdc" and "admin_server" options when >> DNS is not OK instead of writing just the first one in the list. Kerberos tools >> then should be able to fall-back when some of them is not available. > >Sure, that makes sense. Done. > >> 2) What DNS discovery is not OK, we still add _srv_ to ipa_server option in >> sssd.conf. Is it intentional? > >Yes, it was sort of a future-proofing if SRV records are ever made >available. > >rob > > Could I request an option to not add _srv_ at all, like a --no-dns-discovery option.? This way those of us who unfortunately are in situations where we can't create SRV records at all can have it designated at install time?? Otherwise I have to edit the config files afterwards anyway to get rid of it. It could be made default false, of course, but if set the _srv_ entry would not be added. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcritten at redhat.com Thu Jul 5 20:25:25 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 05 Jul 2012 16:25:25 -0400 Subject: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ In-Reply-To: <1341517741.96273.YahooMailNeo@web83802.mail.sp1.yahoo.com> References: <4FF36E38.2040600@redhat.com> <4FF45A48.1050801@redhat.com> <4FF5E877.2090108@redhat.com> <1341517741.96273.YahooMailNeo@web83802.mail.sp1.yahoo.com> Message-ID: <4FF5F835.6060509@redhat.com> Lance Dillon wrote: > > > ------------------------------------------------------------------------ > *From:* Rob Crittenden > *To:* Martin Kosek > *Cc:* freeipa-devel > *Sent:* Thursday, July 5, 2012 3:18 PM > *Subject:* Re: [Freeipa-devel] [PATCH] 1032 allow multiple --server > in client install, don't always set _srv_ > > Martin Kosek wrote: > > On 07/04/2012 12:12 AM, Rob Crittenden wrote: > >> If you pass in --server and --fixed-primary then don't add _srv_ > to ipa_server > >> in sssd.conf. > >> > >> This necessitates the desire to be able to provide multiple > servers so make > >> --server accept multiple values. This represents the bulk of the > code changes. > >> In every case we only use the additional values in sssd.conf. > >> > >> I also made some minor tweaks to discovery. There were cases > where DNS > >> discovery wasn't successful but we set dnsok anyway which could > cause some > >> cascading issues. > >> > >> There are a ton of possible corner cases with this so please, be > brutal. > >> > >> I tested the following against a DNS server that had SRV records > and against > >> one that did not. > >> > >> - ipa-client-install > >> - ipa-client-install --server=ipa.example.com --domain=example.com > >> - ipa-client-install --server=ipa.example.com > --server=ipa1.example.com > >> --domain-example.com > >> - ipa-client-install -server=ipa.example.com > --server=ipa1.example.com > >> --domain-example.com --fixed-primary > >> - ipa-client-install -server=ipa.example.com > --server=ipa1.example.com > >> --domain-example.com --fixed-primary --no-sssd > >> - ipa-client-install -server=ipa.example.com > --server=ipa1.example.com > >> --domain-example.com --no-sssd > >> > >> rob > > > > I did various checks, generally the patch behaves ok, I did not > find any major > > bug. I have just 2 questions/suggestions: > > > > 1) Since we allow more fixed servers to be passed as --server > parameter, we > > could name them all in /etc/krb5.conf in "kdc" and "admin_server" > options when > > DNS is not OK instead of writing just the first one in the list. > Kerberos tools > > then should be able to fall-back when some of them is not available. > > Sure, that makes sense. Done. > > > 2) What DNS discovery is not OK, we still add _srv_ to ipa_server > option in > > sssd.conf. Is it intentional? > > Yes, it was sort of a future-proofing if SRV records are ever made > available. > > rob > > Could I request an option to not add _srv_ at all, like a > --no-dns-discovery option. This way those of us who unfortunately are > in situations where we can't create SRV records at all can have it > designated at install time? Otherwise I have to edit the config files > afterwards anyway to get rid of it. > > It could be made default false, of course, but if set the _srv_ entry > would not be added. You'll be able to do that by specifying --server and --fixed-primary. rob From abokovoy at redhat.com Fri Jul 6 09:29:53 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 12:29:53 +0300 Subject: [Freeipa-devel] [PATCH] 34-35 ipasam fixes In-Reply-To: <20120704191017.GO922@localhost.localdomain> References: <20120704191017.GO922@localhost.localdomain> Message-ID: <20120706092953.GJ16309@redhat.com> On Wed, 04 Jul 2012, Sumit Bose wrote: >Hi, > >the following two patches contain fixes for ipa_sam.c. The first fixes >several issues which were found by clang and the second removes some >testing stuff I forgot to change. ACK. I'll commit them after SASL bind callback as they touch the same file. The second patch is exactly the same as I have in my tree :) -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 09:47:12 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 12:47:12 +0300 Subject: [Freeipa-devel] [PATCH] use 'dedicated keytab file' parameter value instead of hard-coded string Message-ID: <20120706094712.GK16309@redhat.com> Hi, another small two-line cleanup. We already set 'dedicated keytab file' in smb.conf when installing trusts via ipa-adtrust-install. -- / Alexander Bokovoy -------------- next part -------------- >From 48340d9c7dcdd10fa03ee8c4f4894a077babd42e Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 6 Jul 2012 12:43:50 +0300 Subject: [PATCH 5/5] Use smb.conf 'dedicated keytab file' parameter instead of hard-coded value --- daemons/ipa-sam/ipa_sam.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 153733dbfea35cf1426f73827fb83753c259491b..29fc95e457179716c1c70c6f061b1cde9e3f472b 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -3159,6 +3159,7 @@ static void bind_callback_cleanup(struct ipasam_sasl_interact_priv *data, krb5_e data->context = NULL; } +extern const char * lp_dedicated_keytab_file(void); static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, void* ipasam_priv) { krb5_error_code rc; @@ -3200,7 +3201,7 @@ static int bind_callback(LDAP *ldap_struct, struct smbldap_state *ldap_state, vo return LDAP_LOCAL_ERROR; } - rc = krb5_kt_resolve(data.context, "FILE:/etc/samba/samba.keytab", &data.keytab); + rc = krb5_kt_resolve(data.context, lp_dedicated_keytab_file(), &data.keytab); if (rc) { bind_callback_cleanup(&data, rc); return LDAP_LOCAL_ERROR; -- 1.7.10.4 From abokovoy at redhat.com Fri Jul 6 09:51:28 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 12:51:28 +0300 Subject: [Freeipa-devel] [PATCH] reduce redundant checks in ldapsam_search_users() Message-ID: <20120706095128.GL16309@redhat.com> Hi, Obvious clean up in ldapsam_search_users(): every branch is setting the same base dn and nothing else. -- / Alexander Bokovoy -------------- next part -------------- >From 73250245710e26fecf8034667cb5c2358a592fad Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 6 Jul 2012 12:48:27 +0300 Subject: [PATCH 6/6] reduce redundant checks in ldapsam_search_users() to a single statement --- daemons/ipa-sam/ipa_sam.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 29fc95e457179716c1c70c6f061b1cde9e3f472b..c972930b1056e626159f7c6f044076fc88d827ef 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -1044,13 +1044,7 @@ static bool ldapsam_search_users(struct pdb_methods *methods, state->connection = ldap_state->smbldap_state; - if ((acct_flags != 0) && ((acct_flags & ACB_NORMAL) != 0)) - state->base = ldap_state->ipasam_privates->base_dn; - else if ((acct_flags != 0) && - ((acct_flags & (ACB_WSTRUST|ACB_SVRTRUST|ACB_DOMTRUST)) != 0)) - state->base = ldap_state->ipasam_privates->base_dn; - else - state->base = ldap_state->ipasam_privates->base_dn; + state->base = ldap_state->ipasam_privates->base_dn; state->acct_flags = acct_flags; state->base = talloc_strdup(search, state->base); -- 1.7.10.4 From abokovoy at redhat.com Fri Jul 6 10:18:28 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:18:28 +0300 Subject: [Freeipa-devel] [PATCH] reduce redundant checks in ldapsam_search_users() In-Reply-To: <20120706095128.GL16309@redhat.com> References: <20120706095128.GL16309@redhat.com> Message-ID: <20120706101828.GM16309@redhat.com> On Fri, 06 Jul 2012, Alexander Bokovoy wrote: > Hi, > > Obvious clean up in ldapsam_search_users(): every branch is setting the > same base dn and nothing else. Merged the line with talloc_strdup() call few lines after that. -- / Alexander Bokovoy -------------- next part -------------- >From bf06e39a143967d3d99dcfe3fcd4d7f2a5f0142c Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 6 Jul 2012 12:48:27 +0300 Subject: [PATCH 2/2] reduce redundant checks in ldapsam_search_users() to a single statement --- daemons/ipa-sam/ipa_sam.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 29fc95e457179716c1c70c6f061b1cde9e3f472b..86ed3fbd3e6d1894fd398c3c1c94d34c2b7ec273 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -1044,16 +1044,9 @@ static bool ldapsam_search_users(struct pdb_methods *methods, state->connection = ldap_state->smbldap_state; - if ((acct_flags != 0) && ((acct_flags & ACB_NORMAL) != 0)) - state->base = ldap_state->ipasam_privates->base_dn; - else if ((acct_flags != 0) && - ((acct_flags & (ACB_WSTRUST|ACB_SVRTRUST|ACB_DOMTRUST)) != 0)) - state->base = ldap_state->ipasam_privates->base_dn; - else - state->base = ldap_state->ipasam_privates->base_dn; + state->base = talloc_strdup(search, ldap_state->ipasam_privates->base_dn); state->acct_flags = acct_flags; - state->base = talloc_strdup(search, state->base); state->scope = LDAP_SCOPE_SUBTREE; state->filter = get_ldap_filter(search, "*"); state->attrs = talloc_attrs(search, "uid", LDAP_ATTRIBUTE_SID, -- 1.7.10.4 From sbose at redhat.com Fri Jul 6 10:25:55 2012 From: sbose at redhat.com (Sumit Bose) Date: Fri, 6 Jul 2012 12:25:55 +0200 Subject: [Freeipa-devel] [PATCH] use 'dedicated keytab file' parameter value instead of hard-coded string In-Reply-To: <20120706094712.GK16309@redhat.com> References: <20120706094712.GK16309@redhat.com> Message-ID: <20120706102555.GW922@localhost.localdomain> On Fri, Jul 06, 2012 at 12:47:12PM +0300, Alexander Bokovoy wrote: > Hi, > > another small two-line cleanup. We already set 'dedicated keytab file' > in smb.conf when installing trusts via ipa-adtrust-install. ACK bye, Sumit > > -- > / Alexander Bokovoy From sbose at redhat.com Fri Jul 6 10:26:38 2012 From: sbose at redhat.com (Sumit Bose) Date: Fri, 6 Jul 2012 12:26:38 +0200 Subject: [Freeipa-devel] [PATCH] reduce redundant checks in ldapsam_search_users() In-Reply-To: <20120706101828.GM16309@redhat.com> References: <20120706095128.GL16309@redhat.com> <20120706101828.GM16309@redhat.com> Message-ID: <20120706102638.GX922@localhost.localdomain> On Fri, Jul 06, 2012 at 01:18:28PM +0300, Alexander Bokovoy wrote: > On Fri, 06 Jul 2012, Alexander Bokovoy wrote: > >Hi, > > > >Obvious clean up in ldapsam_search_users(): every branch is setting the > >same base dn and nothing else. > Merged the line with talloc_strdup() call few lines after that. > ACK bye, Sumit > -- > / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 10:41:45 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:41:45 +0300 Subject: [Freeipa-devel] [PATCH] use 'dedicated keytab file' parameter value instead of hard-coded string In-Reply-To: <20120706102555.GW922@localhost.localdomain> References: <20120706094712.GK16309@redhat.com> <20120706102555.GW922@localhost.localdomain> Message-ID: <20120706104145.GN16309@redhat.com> On Fri, 06 Jul 2012, Sumit Bose wrote: >On Fri, Jul 06, 2012 at 12:47:12PM +0300, Alexander Bokovoy wrote: >> Hi, >> >> another small two-line cleanup. We already set 'dedicated keytab file' >> in smb.conf when installing trusts via ipa-adtrust-install. > >ACK Thanks. Pushed to master. -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 10:42:04 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:42:04 +0300 Subject: [Freeipa-devel] [PATCH] reduce redundant checks in ldapsam_search_users() In-Reply-To: <20120706102638.GX922@localhost.localdomain> References: <20120706095128.GL16309@redhat.com> <20120706101828.GM16309@redhat.com> <20120706102638.GX922@localhost.localdomain> Message-ID: <20120706104204.GO16309@redhat.com> On Fri, 06 Jul 2012, Sumit Bose wrote: >On Fri, Jul 06, 2012 at 01:18:28PM +0300, Alexander Bokovoy wrote: >> On Fri, 06 Jul 2012, Alexander Bokovoy wrote: >> >Hi, >> > >> >Obvious clean up in ldapsam_search_users(): every branch is setting the >> >same base dn and nothing else. >> Merged the line with talloc_strdup() call few lines after that. >> > >ACK Pushed to master. -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 10:42:29 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:42:29 +0300 Subject: [Freeipa-devel] [PATCH] 33 Allow silent build if available In-Reply-To: <1341492098.14199.162.camel@willson.li.ssimo.org> References: <20120704190617.GN922@localhost.localdomain> <1341492098.14199.162.camel@willson.li.ssimo.org> Message-ID: <20120706104229.GP16309@redhat.com> On Thu, 05 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-04 at 21:06 +0200, Sumit Bose wrote: >> Hi, >> >> this patch allows to use 'make V=0' to get a less verbose output during >> the compilation of the freeipa C-code. Since AM_SILENT_RULES is only >> available with automake 1.11 or later, I used m4_ifdef to check if ti is >> present. > >ACK Pushed to master -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 10:43:35 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:43:35 +0300 Subject: [Freeipa-devel] [PATCH] 34-35 ipasam fixes In-Reply-To: <20120706092953.GJ16309@redhat.com> References: <20120704191017.GO922@localhost.localdomain> <20120706092953.GJ16309@redhat.com> Message-ID: <20120706104335.GQ16309@redhat.com> On Fri, 06 Jul 2012, Alexander Bokovoy wrote: >On Wed, 04 Jul 2012, Sumit Bose wrote: >>Hi, >> >>the following two patches contain fixes for ipa_sam.c. The first fixes >>several issues which were found by clang and the second removes some >>testing stuff I forgot to change. >ACK. I'll commit them after SASL bind callback as they touch the same >file. > >The second patch is exactly the same as I have in my tree :) Both patches pushed to master. -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 10:44:14 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:44:14 +0300 Subject: [Freeipa-devel] [PATCH] ipasam SASL bind callback fixes In-Reply-To: <20120705192722.GI16309@redhat.com> References: <20120704175744.GE16309@redhat.com> <20120704182210.GL922@localhost.localdomain> <20120704201025.GF16309@redhat.com> <1341491358.14199.158.camel@willson.li.ssimo.org> <20120705132701.GH16309@redhat.com> <1341511847.14199.181.camel@willson.li.ssimo.org> <20120705192722.GI16309@redhat.com> Message-ID: <20120706104414.GR16309@redhat.com> On Thu, 05 Jul 2012, Alexander Bokovoy wrote: >On Thu, 05 Jul 2012, Simo Sorce wrote: >>On Thu, 2012-07-05 at 16:27 +0300, Alexander Bokovoy wrote: >>>> >>>>See above, we have all the information we need via simple API calls >>>to >>>>glibc or krb5 libs, we do not need to retrieve that information by >>>other >>>>means. >>>New patch is attached. Works for me without any smb.conf parameters. >>> >>Ack >Thanks. > >I'll fix DEBUG() statement leftover for ipasam:principal template and >will push it. Done and pushed to master. -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 6 10:47:19 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 6 Jul 2012 13:47:19 +0300 Subject: [Freeipa-devel] [PATCH] 166 Moved configuration to last position in navigation In-Reply-To: <4FF44316.5010703@redhat.com> References: <4FF44316.5010703@redhat.com> Message-ID: <20120706104719.GS16309@redhat.com> On Wed, 04 Jul 2012, Petr Vobornik wrote: >Configuration was the last navigation item in IPA server tab. Trusts >changed it. It was wrong because configuration is like 'other >settings' and so it should be last. > >This patch moves configuration navigation item to the last position again. > >https://fedorahosted.org/freeipa/ticket/2900 ACK. Pushed to master. -- / Alexander Bokovoy From simo at redhat.com Fri Jul 6 11:34:04 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 06 Jul 2012 07:34:04 -0400 Subject: [Freeipa-devel] [PATCH] use 'dedicated keytab file' parameter value instead of hard-coded string In-Reply-To: <20120706094712.GK16309@redhat.com> References: <20120706094712.GK16309@redhat.com> Message-ID: <1341574444.14199.189.camel@willson.li.ssimo.org> On Fri, 2012-07-06 at 12:47 +0300, Alexander Bokovoy wrote: > Hi, > > another small two-line cleanup. We already set 'dedicated keytab file' > in smb.conf when installing trusts via ipa-adtrust-install. > ACK, Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Fri Jul 6 11:34:36 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 06 Jul 2012 07:34:36 -0400 Subject: [Freeipa-devel] [PATCH] reduce redundant checks in ldapsam_search_users() In-Reply-To: <20120706095128.GL16309@redhat.com> References: <20120706095128.GL16309@redhat.com> Message-ID: <1341574476.14199.190.camel@willson.li.ssimo.org> On Fri, 2012-07-06 at 12:51 +0300, Alexander Bokovoy wrote: > Hi, > > Obvious clean up in ldapsam_search_users(): every branch is setting > the > same base dn and nothing else. > ACK, Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Fri Jul 6 14:18:34 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 06 Jul 2012 10:18:34 -0400 Subject: [Freeipa-devel] [PATCH] 1028 service pac types In-Reply-To: <1340661350.32038.574.camel@willson.li.ssimo.org> References: <5251d02c-d545-4dcc-a1aa-61934d312bb9@zmail17.collab.prod.int.phx2.redhat.com> <4FE8C8C4.8040600@redhat.com> <1340659322.32038.560.camel@willson.li.ssimo.org> <4FE8DA50.9050403@redhat.com> <1340661350.32038.574.camel@willson.li.ssimo.org> Message-ID: <4FF6F3BA.6030702@redhat.com> Simo Sorce wrote: > On Mon, 2012-06-25 at 17:38 -0400, Rob Crittenden wrote: >> Simo Sorce wrote: >>> On Mon, 2012-06-25 at 16:23 -0400, Rob Crittenden wrote: >>>> Simo Sorce wrote: >>>>> ----- Original Message ----- >>>>>> This patch is more a WIP than anything. I want to see if I'm on the >>>>>> right track. >>>>> >>>>> Hi Rob, >>>>> I don't think we need ipaDefaultKrbAuthzData, we can use the same attribute both in ipaGuiConfig and ipaService, where it is placed makes the difference. >>>>> >>>>> You haven't changed ipaService in the base ldif. >>>> >>>> On new installs the updates are still applied, gets added. >>> >>> Sure it 'works' but the ldif files are now incomplete and slightly >>> misleading, is there a good reason to not update them ? >> >> It is because it is in a file 60basev2.ldif. This is a v3 schema >> addition. It is one confusing element over another. > > My concern is that if you pick the ipa schema files to install somewhere > else you will not have the full schema. > > If we do not provide the full schema in our installable ldif files then > we also need to publish a separate set of documents with the official > schema. > > If that's what we decide to do, then please open a ticket to address > publication of this separate set of ldif file, although it will become > yet another thing to maintain and make sure it doesn't get > de-synchronized with the actual data in the git tree. Ok, moved some things around. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1028-3-pac.patch Type: text/x-diff Size: 25161 bytes Desc: not available URL: From jdennis at redhat.com Sat Jul 7 18:45:51 2012 From: jdennis at redhat.com (John Dennis) Date: Sat, 07 Jul 2012 14:45:51 -0400 Subject: [Freeipa-devel] DN patch and documentation Message-ID: <4FF883DF.4020502@redhat.com> The DN work I was doing on master is ready for review and testing. It's been a long haul and I've been working relentlessly to get this work completed. I am on PTO for a week starting today (I know bad timing) but I spent yesterday and my first day of PTO today writing up extensive documentation for the work so others can start taking a look at it while I'm gone. The documentation as well as where to find the code can be found here: http://jdennis.fedorapeople.org/dn_summary.html The document is long but I felt it was better to provide explanations for as much as possible. I may check in during the week but I'm going to try and discipline myself not to and take an actual much needed break. John -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From simo at redhat.com Sat Jul 7 19:46:22 2012 From: simo at redhat.com (Simo Sorce) Date: Sat, 07 Jul 2012 15:46:22 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <1341690382.2599.3.camel@willson.li.ssimo.org> On Sat, 2012-07-07 at 14:45 -0400, John Dennis wrote: > The DN work I was doing on master is ready for review and testing. It's > been a long haul and I've been working relentlessly to get this work > completed. I am on PTO for a week starting today (I know bad timing) but > I spent yesterday and my first day of PTO today writing up extensive > documentation for the work so others can start taking a look at it while > I'm gone. The documentation as well as where to find the code can be > found here: > > http://jdennis.fedorapeople.org/dn_summary.html > > The document is long but I felt it was better to provide explanations > for as much as possible. > > I may check in during the week but I'm going to try and discipline > myself not to and take an actual much needed break. John, I've read the doc. and everything in there sounds agreeable to me, including delaying mutable vs immutable conversions. However it would be *really* useful if you split the code in a set of patches instead of a humongous patch. At the very least I would like to see it split into a patch that addresses the creation of the IPASimpleLDAPObject (btw why not just ipaLDAPObject which would be shorter ?), one patch that changes the core stuff DN wise, one patch for the tests, one patch for all the actual binaries. Please do not do this on your time off, I am sure it can be handled once you are back :-) Simo. -- Simo Sorce * Red Hat, Inc * New York From simo at redhat.com Sat Jul 7 20:31:55 2012 From: simo at redhat.com (Simo Sorce) Date: Sat, 07 Jul 2012 16:31:55 -0400 Subject: [Freeipa-devel] [PATCH] Fix bogus check Message-ID: <1341693115.2599.4.camel@willson.li.ssimo.org> I pushed the attached patch to master under the one-line rule. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-wrong-check-after-allocation.patch Type: text/x-patch Size: 1021 bytes Desc: not available URL: From simo at redhat.com Sat Jul 7 20:45:12 2012 From: simo at redhat.com (Simo Sorce) Date: Sat, 07 Jul 2012 16:45:12 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user Message-ID: <1341693912.2599.15.camel@willson.li.ssimo.org> When installing the adtrust code we need to be able to get the ipaNTHash populated as in some cases we may need it to authenticate connections over SMB w/o using kerberos during the trust setup phase. The NT hash is really just the same thing as the rc4-hmac key we already have by default in the Kerberos Keys. This patch-set implements a check in the password plugin for the pre-mod operation to catch the attempt to replace the attribute with the value 'MagicRegen' in the ipaNThash attribute. If no previous ipaNTHash value is present, and the kerberos keys are available, then we attempt to find the rc4-hmac key and if we find it we store it in the ipaNTHash. This will allow us to give the admin user (and potentially any other user) the NT hash samba requires without forcing them to reset their password, assuming the rc4-hmac key is present (currently it is by default). I marked this patch-set as RFC as I want opinions on the method (LDAP modify with replace operation) I utilized to perform the extraction. If it bode well with everybody we can consider the patch-set for inclusion. I tested it and extracting the hash works fine and it works later on using smbclient to access a share. This patchset implements task #2867: https://fedorahosted.org/freeipa/ticket/2867 Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-494-0001-1-Move-code-into-common-krb5-utils.patch Type: text/x-patch Size: 11561 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-494-0002-1-Improve-loops-around-slapi-mods.patch Type: text/x-patch Size: 10982 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-494-0003-1-Add-special-modify-op-to-regen-ipaNTHash.patch Type: text/x-patch Size: 7199 bytes Desc: not available URL: From pvoborni at redhat.com Mon Jul 9 11:55:05 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 09 Jul 2012 13:55:05 +0200 Subject: [Freeipa-devel] [PATCH] 167 Add and remove dns per-domain permission in Web UI Message-ID: <4FFAC699.4070308@redhat.com> Patch functionality depends on not yet posted pviktori's patch which adds error_code (in case of command error) to batch response. Patch description: This patch adds support for new per-domain permissions to Web UI. User with assigned permission (through role,priviledge) can edit DNS zone. These permissions can be added/remove by ipa dnszone-{add/remove}permission $dnszone command. For adding/removing of this permission in Web UI new actions in DNS zone action list were created. DNS zone object doesn't contain information about existance of related permission. Such information is required for enabling/disabling of new actions. Web UI has to search for the permission to get it. DNS zone facet was modified to use batch command, in a same way as user facet, for loading dnszone and the permission at the same time - on load. Batch command has a feature to report all errors. Such behavior is unwanted because we expect that permission-show command will fail when the permission doesn't exist. Batch command was therefore modified to not report commands which has retry attribute set to false. This attr was chosen because it has similar purpose in single command execution. New actions should be enabled only for users with appropriate rights. It is not possible to obtain rights for certain action in advance so an approximation is used: write right for dns zones' managedby attribute. https://fedorahosted.org/freeipa/ticket/2851 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0167-Add-and-remove-dns-per-domain-permission-in-Web-UI.patch Type: text/x-patch Size: 16495 bytes Desc: not available URL: From sbose at redhat.com Mon Jul 9 12:20:50 2012 From: sbose at redhat.com (Sumit Bose) Date: Mon, 9 Jul 2012 14:20:50 +0200 Subject: [Freeipa-devel] [PATCH] Fix typo In-Reply-To: <20120626082900.GB29454@localhost.localdomain> References: <20120626082900.GB29454@localhost.localdomain> Message-ID: <20120709122050.GY922@localhost.localdomain> On Tue, Jun 26, 2012 at 10:29:00AM +0200, Sumit Bose wrote: > Hi, > > this patch fixes a small typo and silences a compiler warning. I think > it is right to use authdata instead of &authdata here, but I have to > admit that I cannot say why we have not seen any issues before. > > bye, > Sumit I think I have an idea why both versions are working. I would expect that in the 'authdata' case gcc does not pass the krb5_authdata array by value but by reference. In the '&authdata' case gcc passes the pointer to the krb5_authdata array by value. As a result in both case the same address is on the stack and krb5_encode_authdata_container() can work as expected. I hope this makes sense. Nevertheless I still think the 'authdata' version is the correct one. bye, Sumit > From 94ee2395539bad666f0ffea4ccb688d4a5330582 Mon Sep 17 00:00:00 2001 > From: Sumit Bose > Date: Tue, 26 Jun 2012 09:58:01 +0200 > Subject: [PATCH] Fix typo > > --- > daemons/ipa-kdb/ipa_kdb_mspac.c | 2 +- > 1 Datei ge?ndert, 1 Zeile hinzugef?gt(+), 1 Zeile entfernt(-) > > diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c > index f640b545a636a2c58e3eb31951de142e5b0ffbe2..1c7487c3c8f75d02466a2e0746fbef5d36e3d995 100644 > --- a/daemons/ipa-kdb/ipa_kdb_mspac.c > +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c > @@ -1267,7 +1267,7 @@ krb5_error_code ipadb_sign_authdata(krb5_context context, > > kerr = krb5_encode_authdata_container(context, > KRB5_AUTHDATA_IF_RELEVANT, > - &authdata, > + authdata, > signed_auth_data); > if (kerr != 0) { > goto done; > -- > 1.7.10.2 > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel From simo at redhat.com Mon Jul 9 12:36:32 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 09 Jul 2012 08:36:32 -0400 Subject: [Freeipa-devel] [PATCH] Fix typo In-Reply-To: <20120709122050.GY922@localhost.localdomain> References: <20120626082900.GB29454@localhost.localdomain> <20120709122050.GY922@localhost.localdomain> Message-ID: <1341837392.2599.34.camel@willson.li.ssimo.org> On Mon, 2012-07-09 at 14:20 +0200, Sumit Bose wrote: > On Tue, Jun 26, 2012 at 10:29:00AM +0200, Sumit Bose wrote: > > Hi, > > > > this patch fixes a small typo and silences a compiler warning. I think > > it is right to use authdata instead of &authdata here, but I have to > > admit that I cannot say why we have not seen any issues before. > > > > bye, > > Sumit > > I think I have an idea why both versions are working. I would expect > that in the 'authdata' case gcc does not pass the krb5_authdata array > by value but by reference. In the '&authdata' case gcc passes the > pointer to the krb5_authdata array by value. As a result in both case > the same address is on the stack and krb5_encode_authdata_container() > can work as expected. I hope this makes sense. Yes, by sheer luck authdata == &authdata when it is an array. > Nevertheless I still think the 'authdata' version is the correct one. Indeed it is. Pushed to master Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Mon Jul 9 12:53:29 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 09 Jul 2012 14:53:29 +0200 Subject: [Freeipa-devel] [PATCH] 284 Do not change LDAPObject objectclass list Message-ID: <4FFAD449.5060901@redhat.com> This one caused dnszone_find command in the Web UI to sometimes return an empty list. Pushed to master as a one-liner patch. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-284-do-not-change-ldapobject-objectclass-list.patch Type: text/x-patch Size: 1444 bytes Desc: not available URL: From pvoborni at redhat.com Mon Jul 9 15:21:14 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 09 Jul 2012 17:21:14 +0200 Subject: [Freeipa-devel] [PATCH] 168 Password policy measurement units Message-ID: <4FFAF6EA.9030509@redhat.com> Note: I think we should improve handling of measurement units in server plugins. Label and measurement unit should be separated and send in metadata. Client - Web UI or CLI would then decide when and where to display the units. I will create an enhancement ticket for it. Patch description: When filling password policy it may be unclear what value to enter because user may not remember field's measurement unit. This patch adds support for declaring measurement units. It's done in field's/widget's spec by entering key for unit's string (which is in IPA.messages.measurement_units[key]). Measurement units in table layout are displayed in parenthesis after label. It is to be consistent with some fields which have measurement unit integrated in label. This patch defines measurement units for password policy's 'History size', 'Failure reset interval' and 'Lockout duration' fields. https://fedorahosted.org/freeipa/ticket/2437 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0168-Password-policy-measurement-units.patch Type: text/x-patch Size: 7827 bytes Desc: not available URL: From pvoborni at redhat.com Mon Jul 9 15:32:00 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 09 Jul 2012 17:32:00 +0200 Subject: [Freeipa-devel] [PATCH] 169 Web UI: kerberos ticket policy measurement units Message-ID: <4FFAF970.1010905@redhat.com> Added measurement units for kerberos ticket policy. https://fedorahosted.org/freeipa/ticket/2444 Note: patch depends on pvoborni-0168. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0169-Web-UI-kerberos-ticket-policy-measurement-units.patch Type: text/x-patch Size: 2316 bytes Desc: not available URL: From edewata at redhat.com Tue Jul 10 05:40:02 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 10 Jul 2012 00:40:02 -0500 Subject: [Freeipa-devel] [PATCH] 165 Display loginas information only after login In-Reply-To: <4FF44069.1020409@redhat.com> References: <4FEC6506.7050205@redhat.com> <4FECC2DF.4030200@redhat.com> <4FF153D8.70206@redhat.com> <4FF1C31E.7010205@redhat.com> <4FF44069.1020409@redhat.com> Message-ID: <4FFBC032.3080105@redhat.com> On 7/4/2012 8:08 AM, Petr Vobornik wrote: > On 07/02/2012 05:49 PM, Endi Sukma Dewata wrote: >> ACK. Some more comments below. Feel free to fix before push or later >> separately. > > Implemented most of the issues, look bellow. Update patch attached. ACK. -- Endi S. Dewata From edewata at redhat.com Tue Jul 10 05:40:58 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 10 Jul 2012 00:40:58 -0500 Subject: [Freeipa-devel] [PATCH] 167 Add and remove dns per-domain permission in Web UI In-Reply-To: <4FFAC699.4070308@redhat.com> References: <4FFAC699.4070308@redhat.com> Message-ID: <4FFBC06A.2040307@redhat.com> On 7/9/2012 6:55 AM, Petr Vobornik wrote: > Patch functionality depends on not yet posted pviktori's patch which > adds error_code (in case of command error) to batch response. > > Patch description: > > This patch adds support for new per-domain permissions to Web UI. > > User with assigned permission (through role,priviledge) can edit DNS > zone. These permissions can be added/remove by ipa > dnszone-{add/remove}permission $dnszone command. > > For adding/removing of this permission in Web UI new actions in DNS zone > action list were created. DNS zone object doesn't contain information > about existance of related permission. Such information is required for > enabling/disabling of new actions. Web UI has to search for the > permission to get it. DNS zone facet was modified to use batch command, > in a same way as user facet, for loading dnszone and the permission at > the same time - on load. > > Batch command has a feature to report all errors. Such behavior is > unwanted because we expect that permission-show command will fail when > the permission doesn't exist. Batch command was therefore modified to > not report commands which has retry attribute set to false. This attr > was chosen because it has similar purpose in single command execution. > > New actions should be enabled only for users with appropriate rights. It > is not possible to obtain rights for certain action in advance so an > approximation is used: write right for dns zones' managedby attribute. > > https://fedorahosted.org/freeipa/ticket/2851 ACK. The code works, feel free to push when the other patch is ready. But if time allows it might be better to use a checkbox or radio buttons inside the details page. That way you can see whether the per-domain permission is enabled without having to click the drop-down list and infer from the enabled/disabled actions. -- Endi S. Dewata From edewata at redhat.com Tue Jul 10 05:41:49 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 10 Jul 2012 00:41:49 -0500 Subject: [Freeipa-devel] [PATCH] 168 Password policy measurement units In-Reply-To: <4FFAF6EA.9030509@redhat.com> References: <4FFAF6EA.9030509@redhat.com> Message-ID: <4FFBC09D.5050104@redhat.com> On 7/9/2012 10:21 AM, Petr Vobornik wrote: > Note: I think we should improve handling of measurement units in server > plugins. Label and measurement unit should be separated and send in > metadata. Client - Web UI or CLI would then decide when and where to > display the units. I will create an enhancement ticket for it. > > Patch description: > > When filling password policy it may be unclear what value to enter > because user may not remember field's measurement unit. > > This patch adds support for declaring measurement units. It's done in > field's/widget's spec by entering key for unit's string (which is in > IPA.messages.measurement_units[key]). > > Measurement units in table layout are displayed in parenthesis after > label. It is to be consistent with some fields which have measurement > unit integrated in label. > > This patch defines measurement units for password policy's 'History > size', 'Failure reset interval' and 'Lockout duration' fields. > > https://fedorahosted.org/freeipa/ticket/2437 ACK. The elements of the label and the input field have tooltip which will show the field label. I'm not sure if it's necessary to show the unit in the tooltip too. Some parameters have a doc attribute, maybe we should show the doc in the tooltip instead? -- Endi S. Dewata From edewata at redhat.com Tue Jul 10 05:41:59 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 10 Jul 2012 00:41:59 -0500 Subject: [Freeipa-devel] [PATCH] 169 Web UI: kerberos ticket policy measurement units In-Reply-To: <4FFAF970.1010905@redhat.com> References: <4FFAF970.1010905@redhat.com> Message-ID: <4FFBC0A7.1050208@redhat.com> On 7/9/2012 10:32 AM, Petr Vobornik wrote: > Added measurement units for kerberos ticket policy. > > https://fedorahosted.org/freeipa/ticket/2444 > > Note: patch depends on pvoborni-0168. ACK. -- Endi S. Dewata From mkosek at redhat.com Tue Jul 10 06:31:20 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 10 Jul 2012 08:31:20 +0200 Subject: [Freeipa-devel] [PATCH] 1031 run cleanallruv task In-Reply-To: <4FF5DF79.6090806@redhat.com> References: <4FF3047C.9020202@redhat.com> <4FF41C8F.2020003@redhat.com> <4FF5DF79.6090806@redhat.com> Message-ID: <4FFBCC38.7080105@redhat.com> On 07/05/2012 08:39 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 07/03/2012 04:41 PM, Rob Crittenden wrote: >>> Deleting a replica can leave a replication vector (RUV) on the other servers. >>> This can confuse things if the replica is re-added, and it also causes the >>> server to calculate changes against a server that may no longer exist. >>> >>> 389-ds-base provides a new task that self-propogates itself to all available >>> replicas to clean this RUV data. >>> >>> This patch will create this task at deletion time to hopefully clean things up. >>> >>> It isn't perfect. If any replica is down or unavailable at the time the >>> cleanruv task fires, and then comes back up, the old RUV data may be >>> re-propogated around. >>> >>> To make things easier in this case I've added two new commands to >>> ipa-replica-manage. The first lists the replication ids of all the servers we >>> have a RUV for. Using this you can call clean_ruv with the replication id of a >>> server that no longer exists to try the cleanallruv step again. >>> >>> This is quite dangerous though. If you run cleanruv against a replica id that >>> does exist it can cause a loss of data. I believe I've put in enough scary >>> warnings about this. >>> >>> rob >>> >> >> Good work there, this should make cleaning RUVs much easier than with the >> previous version. >> >> This is what I found during review: >> >> 1) list_ruv and clean_ruv command help in man is quite lost. I think it would >> help if we for example have all info for commands indented. This way user could >> simply over-look the new commands in the man page. >> >> >> 2) I would rename new commands to clean-ruv and list-ruv to make them >> consistent with the rest of the commands (re-initialize, force-sync). >> >> >> 3) It would be nice to be able to run clean_ruv command in an unattended way >> (for better testing), i.e. respect --force option as we already do for >> ipa-replica-manage del. This fix would aid test automation in the future. >> >> >> 4) (minor) The new question (and the del too) does not react too well for >> CTRL+D: >> >> # ipa-replica-manage clean_ruv 3 --force >> Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 >> >> Cleaning the wrong replica ID will cause that server to no >> longer replicate so it may miss updates while the process >> is running. It would need to be re-initialized to maintain >> consistency. Be very careful. >> Continue to clean? [no]: unexpected error: >> >> >> 5) Help for clean_ruv command without a required parameter is quite confusing >> as it reports that command is wrong and not the parameter: >> >> # ipa-replica-manage clean_ruv >> Usage: ipa-replica-manage [options] >> >> ipa-replica-manage: error: must provide a command [clean_ruv | force-sync | >> disconnect | connect | del | re-initialize | list | list_ruv] >> >> It seems you just forgot to specify the error message in the command definition >> >> >> 6) When the remote replica is down, the clean_ruv command fails with an >> unexpected error: >> >> [root at vm-086 ~]# ipa-replica-manage clean_ruv 5 >> Clean the Replication Update Vector for vm-055.idm.lab.bos.redhat.com:389 >> >> Cleaning the wrong replica ID will cause that server to no >> longer replicate so it may miss updates while the process >> is running. It would need to be re-initialized to maintain >> consistency. Be very careful. >> Continue to clean? [no]: y >> unexpected error: {'desc': 'Operations error'} >> >> >> /var/log/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/errors: >> [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: failed >> to connect to repl agreement connection >> (cn=meTovm-055.idm.lab.bos.redhat.com,cn=replica, >> cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping >> tree,cn=config), error 105 >> [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: replica >> (cn=meTovm-055.idm.lab. >> bos.redhat.com,cn=replica,cn=dc\3Didm\2Cdc\3Dlab\2Cdc\3Dbos\2Cdc\3Dredhat\2Cdc\3Dcom,cn=mapping >> >> tree, cn=config) has not been cleaned. You will need to rerun the >> CLEANALLRUV task on this replica. >> [04/Jul/2012:06:28:16 -0400] NSMMReplicationPlugin - cleanAllRUV_task: Task >> failed (1) >> >> In this case I think we should inform user that the command failed, possibly >> because of disconnected replicas and that they could enable the replicas and >> try again. >> >> >> 7) (minor) "pass" is now redundant in replication.py: >> + except ldap.INSUFFICIENT_ACCESS: >> + # We can't make the server we're removing read-only but >> + # this isn't a show-stopper >> + root_logger.debug("No permission to switch replica to read-only, >> continuing anyway") >> + pass >> > > I think this addresses everything. > > rob Thanks, almost there! I just found one more issue which needs to be fixed before we push: # ipa-replica-manage del vm-055.idm.lab.bos.redhat.com --force Directory Manager password: Unable to connect to replica vm-055.idm.lab.bos.redhat.com, forcing removal Failed to get data from 'vm-055.idm.lab.bos.redhat.com': {'desc': "Can't contact LDAP server"} Forcing removal on 'vm-086.idm.lab.bos.redhat.com' There were issues removing a connection: %d format: a number is required, not str Failed to get data from 'vm-055.idm.lab.bos.redhat.com': {'desc': "Can't contact LDAP server"} This is a traceback I retrieved: Traceback (most recent call last): File "/sbin/ipa-replica-manage", line 425, in del_master del_link(realm, r, hostname, options.dirman_passwd, force=True) File "/sbin/ipa-replica-manage", line 271, in del_link repl1.cleanallruv(replica_id) File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 1094, in cleanallruv root_logger.debug("Creating CLEANALLRUV task for replica id %d" % replicaId) The problem here is that you don't convert replica_id to int in this part: + replica_id = None + if repl2: + replica_id = repl2._get_replica_id(repl2.conn, None) + else: + servers = get_ruv(realm, replica1, dirman_passwd) + for (netloc, rid) in servers: + if netloc.startswith(replica2): + replica_id = rid + break Martin From pviktori at redhat.com Tue Jul 10 08:23:10 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 10 Jul 2012 10:23:10 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <4FFBE66E.9040802@redhat.com> On 07/07/2012 08:45 PM, John Dennis wrote: > The DN work I was doing on master is ready for review and testing. It's > been a long haul and I've been working relentlessly to get this work > completed. I am on PTO for a week starting today (I know bad timing) but > I spent yesterday and my first day of PTO today writing up extensive > documentation for the work so others can start taking a look at it while > I'm gone. The documentation as well as where to find the code can be > found here: > > http://jdennis.fedorapeople.org/dn_summary.html > > The document is long but I felt it was better to provide explanations > for as much as possible. > > I may check in during the week but I'm going to try and discipline > myself not to and take an actual much needed break. > > John > I've read your summary (which you should summarize into a commit message before this is pushed), and gone through the patch. Here is what I found doing that; I didn't get to actual testing yet. I also didn't do a detailed review of ldap2.py changes yet. I agree with Simo that it would have been better to put at least your automated changes in a separate patch. Without knowing what was automated and what wasn't, I have to eyeball each change at least to figure that out. Not complaining, it's a reviewer's work after all, and with an intra-line diff tool it's not that hard, but I can't really honor your suggestion for a faster review :) ==== Blockers ==== Mutable objects that support __hash__ are a *very* bad thing to do. It violates a fundamental assumption in Python, as you outlined. Mutable objects *must not* be allowed to be dictionary keys. I understand the patch is not perfect, but this is a big issue that invites very hard-to-debug errors. You say you're already working on a patch to fix this. Please post that ASAP; I can't ack the DN work without it. In the long run, it would be good to use immutable DNs only: I think modeling them after Python strings in this way would be beneficial. You seem to forged locked=True for some global DNs, e.g. ipalib/plugins/pwpolicy.py:172: global_policy_dn ipalib/plugins/migration.py:117: _compat_dn Locked by default would prevent these omissions. Please fix a lint failure; ipaserver.ipautil moved to ipapython.ipautil. ==== Design decision criticism ==== > Originally I was against automatic type promotion on the belief if you were comparing a string to a DN it was a failure of program logic, but there were some circumstances where it was evil necessity. Would you care to elaborate? I also think that's a logic failure; such circumstances should at least be marked by a TODO. > The find() and rfind() methods (modeled after their string cousins) to locate a RDN or DN in an existing DN. I would much rather see the index() and rindex() methods, which do the Pythonic thing of raising an exception rather than silently returning -1 when the thing isn't found. Compare: something[something.find(thing)] => something[-1] if not found something[something.index(thing)] => raises exception if not found ipapython/dn.py:448: Each argument must be scalar convertable to unicode or a callable object whose result is convertable to unicode. Is it really necessary to allow callables here? If the user has a callable, he/she should call it before making an AVA out of it. Same with converting to Unicode. As you point out in dn_summary, automatic coercion is evil. ==== Major issues ==== ipalib/plugins/aci.py:340: + groupdn = DN(groupdn) + try: + if groupdn[0].attr == 'cn': + dn = DN() + entry_attrs = {} + try: + (dn, entry_attrs) = ldap.get_entry(groupdn, ['cn']) + except errors.NotFound, e: + # FIXME, use real name here + if test: + dn = DN(('cn', 'test'), api.env.container_permission) + entry_attrs = {'cn': [u'test']} + if api.env.container_permission in dn: + kw['permission'] = entry_attrs['cn'][0] + else: + if 'cn' in entry_attrs: + kw['group'] = entry_attrs['cn'][0] + except IndexError: + pass Why is any IndexError ignored here? That try block is much too long, it could catch unintended errors. Please only `try` the smallest piece of code that could raise the exception you want to catch. There are enough `DN(xyz.replace('ldap:///',''))` calls in aci.py to warrant a strip_ldap_prefix helper. (It could also only remove the prefix, not all the occurrences -- unlikely to matter but it's better to do things correctly.) ipalib/plugins/baseldap.py:1028: + try: + if dn[0].attr != self.obj.primary_key.name: + self.obj.handle_duplicate_entry(*keys) + except (IndexError, KeyError): + pass Again, there's too much in the try block. You don't want to catch errors from handle_duplicate_entry. Do this instead: try: dn_attr = dn[0].attr except (IndexError, KeyError): pass else: if dn_attr != self.obj.primary_key.name: self.obj.handle_duplicate_entry(*keys) ==== Minor issues ==== In ipalib/plugins/aci.py: @@ -343,10 +341,12 @@ def _aci_to_kw(ldap, a, test=False, pkey_only=False): else: # See if the target is a group. If so we set the # targetgroup attr, otherwise we consider it a subtree - if api.env.container_group in target: - targetdn = unicode(target.replace('ldap:///','')) - target = DN(targetdn) - kw['targetgroup'] = target['cn'] + targetdn = DN(target.replace('ldap:///','')) + if api.env.container_group in targetdn: + try: + kw['targetgroup'] = targetdn[0]['cn'] + except (IndexError, KeyError): + pass else: kw['subtree'] = unicode(target) Why is (IndexError, KeyError) ignored? Wouldn't be more correct to use endswith(DN(container_group, suffix)) instead of the `in` operator here, and in other places like this. ipapython/dn.py:26 LOCKED_ERROR_MSG = 'Object is locked, it cannot be modified: %s' and then: raise TypeError(LOCKED_ERROR_MSG % (repr(self))) You should define a TypeError subclass for this. Using a string instead of a specialized object is exactly what this patch tries so hard to get rid of... :) ipapython/entity.py:49: elif isinstance(entrydata, DN): self.dn = entrydata self.data = ipautil.CIDict() elif isinstance(entrydata, basestring): self.dn = DN(entrydata) self.data = ipautil.CIDict() Should we not use DNs exclusively here? At least a TODO is warranted. ipaserver/ipaldap.py:109: elif isinstance(entrydata,DN): self.dn = entrydata self.data = ipautil.CIDict() elif isinstance(entrydata,str) or isinstance(entrydata,unicode): self.dn = DN(entrydata) self.data = ipautil.CIDict() Again, why not use DNs exclusively? At least put in a TODO. For now at least do isinstance(entrydata, basestring) instead of the two isinstances. In several places: + # Use a property for to assure it's always a DN type + def _set_(self, value): + self._ = DN(value) + + def _get_(self): + assert isinstance(getattr(self, '_'), DN) + return self._ + + ldap_suffix = property(_get_, _set_) Shouldn't the assert be in the setter? Do we not want people to only put DNs in these properties, instead of silently converting? This appears enough times to justify using a helper to construct these. But if you don't want to generalize, the getattr call is unnecessary. tests/util.py:307: if isinstance(expected, DN): if isinstance(got, basestring): got = DN(got) This may be overly broad, the tests should be able to check if they got a DN or a string. What about something like: if isinstance(expected, DNString): assert isinstance(got, basestring) assert DN(got) == expected.dn with a suitable DNString class? ipapython/dn.py:541: def _set_locked(self, value): There should be only one way to do things, either the lock/unlock methods, or setting an attribute. More ways to do one same thing lead to subtle errors in one of the ways. The locking code is duplicated in AVA, RDN and DN; it would be good to make a Lockable mixin to avoid that. A similar case is with __repr__. Anyway unlocking is bad (see above) so these are mostly moot points. ipaserver/plugins/ldap2.py:134: class ServerSchema(object): Please don't use nested classes without reason. ==== Nitpicks/opinions ==== In ipa-replica-manage:369 > sys.exit("winsync agreement already exists on subtree %s" % please remove the trailing space ipapython/dn.py: in docstring: DN(arg0, ..., locked=False, first_key_match=True) followed by: def __init__(self, *args, **kwds): and: kwds.get('first_key_match', True) I don't see the reason for this. Just write `def __init__(self, *args, locked=False, first_key_match=True)` and put a proper summary in the first line of the docstring. Same in AVA & RDN. ipapython/ipautil.py:201: - """Convert a kerberos realm into the IPA suffix.""" Why are you removing a docstring? ipaserver/plugins/ldap2.py:79: _debug_log_ldap = True if _debug_log_ldap: import pprint Just import it unconditionally if you need it. ipalib/plugins/baseldap.py:1874: - sortfn=lambda x,y: cmp(x[1][self.obj.primary_key.name][0].lower(), y[1][self.obj.primary_key.name][0].lower()) + sortfn=lambda x,y: cmp(x[1][self.obj.primary_key.name][0], + y[1][self.obj.primary_key.name][0]) entries.sort(sortfn) Since you're touching this: it's better (easier, faster, more readable) to use a key function. (Also, defining a named function is better with `def foo` than `foo=lambda...`): def sortkey(x): return x[1][self.obj.primary_key.name][0] entries.sort(key=sortkey) ipapython/entity.py:95: # Python "internal" class attributes are prefixed and suffixed with double underscores. # Make sure we continue to return things like __str__ from this class. I think a better solution would be making Entity a new-style class. Or even better, killing Entity off altogether. Which is the plan, so this is a moot point. Same with Entry in ipaserver/ipaldap.py. ipaserver/install/ldapupdate.py:132: + assert isinstance(suffix, (None.__class__, DN)) type(x) is preferable to x.__class__ (the same way e.g. len(x) is preferable to x.__len__). However, I think it would be more readable to use: if suffix is not None: assert isinstance(suffix, DN) ipaserver/plugins/ldap2.py:109: def value_to_utf8(val): ''' If val is not a string we need to convert it to a string (specifically a unicode string). Naively we might think we need to call str(val) to convert to a string. This is incorrect because if val is already a unicode object then str() will call encode(default_encoding) returning a str object encoded with default_encoding. But we don't want to apply the default_encoding! Rather we want to guarantee the val object has been converted to a unicode string because from a unicode string we want to explicitly encode to a str using our desired encoding (utf-8 in this case). Note: calling unicode on a unicode object simply returns the exact same object (with it's ref count incremented). This means calling unicode on a unicode object is effectively a no-op, thus it's not inefficient. ''' That is not an explanation of *what* the function does, but *how* it does it. Thus it should be in a comment, not in a docstring. Also, if something needs such an explanation, it also need unit tests to ensure it actually works correctly. IPASimpleLDAPObject has the same issue -- lengthy descriptions of design decisions should go in a comment (if not in a commit message or on the mailing list). > Require that every place a dn is used it must be a DN object. This > translates into lot of `assert isinstance(dn, DN)` sprinkled through > out the code. That seems like a crutch to get your DN work done. We should remove these once the dust settles. > Python provides the __all__ module export list, we really should be > making better use of that. I recommend importing names explicitly, or importing a module and using qualified names, over __all__ and import *. Star imports make it unnecessarily hard to see where names come from. (You do use explicit imports in the code, my gripe is just about recommending * and __all__). > I equate Python decorators with C preprocessor hell when CPP macros > are abused. I do not share that view. It is possible to abuse decorators, but not all of their use is necessarily evil. It's true that encode.py wasn't a good use of them. > Removed all usage of get_schema() which was being called prior to > accessing the .schema attribute of an object. If a class is using > internal lazy loading as an optimization it's not right to require > users of the interface to be aware of internal optimization's. > Instead the class should handle it's lazy loading completely > internally by loading the data on first access. > This is very easy to do with class properties. schema is now a > property of the class instead of an attribute. When the schema > property is accessed it actually calls a private internal method to > perform the lazy loading. In other words, you're reinventing the reify decorator: https://github.com/Pylons/pyramid/blob/master/pyramid/decorator.py#L1 Being a grammar nazi, I must point out the it's/its confusion and plurals formed with apostrophes. ==== Thanks for the work! This will really make IPA better when it's polished a bit. Hopefully all the criticism doesn't sound too negative :) -- Petr? From mkosek at redhat.com Tue Jul 10 08:57:14 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 10 Jul 2012 10:57:14 +0200 Subject: [Freeipa-devel] [PATCH] 1032 allow multiple --server in client install, don't always set _srv_ In-Reply-To: <4FF5E877.2090108@redhat.com> References: <4FF36E38.2040600@redhat.com> <4FF45A48.1050801@redhat.com> <4FF5E877.2090108@redhat.com> Message-ID: <4FFBEE6A.6070505@redhat.com> On 07/05/2012 09:18 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 07/04/2012 12:12 AM, Rob Crittenden wrote: >>> If you pass in --server and --fixed-primary then don't add _srv_ to ipa_server >>> in sssd.conf. >>> >>> This necessitates the desire to be able to provide multiple servers so make >>> --server accept multiple values. This represents the bulk of the code changes. >>> In every case we only use the additional values in sssd.conf. >>> >>> I also made some minor tweaks to discovery. There were cases where DNS >>> discovery wasn't successful but we set dnsok anyway which could cause some >>> cascading issues. >>> >>> There are a ton of possible corner cases with this so please, be brutal. >>> >>> I tested the following against a DNS server that had SRV records and against >>> one that did not. >>> >>> - ipa-client-install >>> - ipa-client-install --server=ipa.example.com --domain=example.com >>> - ipa-client-install --server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com >>> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com --fixed-primary >>> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com --fixed-primary --no-sssd >>> - ipa-client-install -server=ipa.example.com --server=ipa1.example.com >>> --domain-example.com --no-sssd >>> >>> rob >> >> I did various checks, generally the patch behaves ok, I did not find any major >> bug. I have just 2 questions/suggestions: >> >> 1) Since we allow more fixed servers to be passed as --server parameter, we >> could name them all in /etc/krb5.conf in "kdc" and "admin_server" options when >> DNS is not OK instead of writing just the first one in the list. Kerberos tools >> then should be able to fall-back when some of them is not available. > > Sure, that makes sense. Done. > >> 2) What DNS discovery is not OK, we still add _srv_ to ipa_server option in >> sssd.conf. Is it intentional? > > Yes, it was sort of a future-proofing if SRV records are ever made available. > > rob I did not find any other patch-related issue, I just hit an issue with SELinux which prevents IPA clients with --no-sssd from working: https://bugzilla.redhat.com/show_bug.cgi?id=838822 But since this issue is not related to your patch, it is good to go. ACK. Pushed to master. Martin From pvoborni at redhat.com Tue Jul 10 11:43:53 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 10 Jul 2012 13:43:53 +0200 Subject: [Freeipa-devel] [PATCH] 165 Display loginas information only after login In-Reply-To: <4FFBC032.3080105@redhat.com> References: <4FEC6506.7050205@redhat.com> <4FECC2DF.4030200@redhat.com> <4FF153D8.70206@redhat.com> <4FF1C31E.7010205@redhat.com> <4FF44069.1020409@redhat.com> <4FFBC032.3080105@redhat.com> Message-ID: <4FFC1579.5010603@redhat.com> On 07/10/2012 07:40 AM, Endi Sukma Dewata wrote: > On 7/4/2012 8:08 AM, Petr Vobornik wrote: >> On 07/02/2012 05:49 PM, Endi Sukma Dewata wrote: >>> ACK. Some more comments below. Feel free to fix before push or later >>> separately. >> >> Implemented most of the issues, look bellow. Update patch attached. > > ACK. > Pushed to master -- Petr Vobornik From pvoborni at redhat.com Tue Jul 10 11:52:06 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 10 Jul 2012 13:52:06 +0200 Subject: [Freeipa-devel] [PATCH] 168 Password policy measurement units In-Reply-To: <4FFBC09D.5050104@redhat.com> References: <4FFAF6EA.9030509@redhat.com> <4FFBC09D.5050104@redhat.com> Message-ID: <4FFC1766.7000502@redhat.com> On 07/10/2012 07:41 AM, Endi Sukma Dewata wrote: > On 7/9/2012 10:21 AM, Petr Vobornik wrote: >> Note: I think we should improve handling of measurement units in server >> plugins. Label and measurement unit should be separated and send in >> metadata. Client - Web UI or CLI would then decide when and where to >> display the units. I will create an enhancement ticket for it. >> >> Patch description: >> >> When filling password policy it may be unclear what value to enter >> because user may not remember field's measurement unit. >> >> This patch adds support for declaring measurement units. It's done in >> field's/widget's spec by entering key for unit's string (which is in >> IPA.messages.measurement_units[key]). >> >> Measurement units in table layout are displayed in parenthesis after >> label. It is to be consistent with some fields which have measurement >> unit integrated in label. >> >> This patch defines measurement units for password policy's 'History >> size', 'Failure reset interval' and 'Lockout duration' fields. >> >> https://fedorahosted.org/freeipa/ticket/2437 > > ACK. Pushed to master. > > The elements of the label and the input field have tooltip which > will show the field label. I'm not sure if it's necessary to show the > unit in the tooltip too. Some parameters have a doc attribute, maybe we > should show the doc in the tooltip instead? > https://fedorahosted.org/freeipa/ticket/2912 -- Petr Vobornik From pvoborni at redhat.com Tue Jul 10 11:52:35 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 10 Jul 2012 13:52:35 +0200 Subject: [Freeipa-devel] [PATCH] 169 Web UI: kerberos ticket policy measurement units In-Reply-To: <4FFBC0A7.1050208@redhat.com> References: <4FFAF970.1010905@redhat.com> <4FFBC0A7.1050208@redhat.com> Message-ID: <4FFC1783.9010308@redhat.com> On 07/10/2012 07:41 AM, Endi Sukma Dewata wrote: > On 7/9/2012 10:32 AM, Petr Vobornik wrote: >> Added measurement units for kerberos ticket policy. >> >> https://fedorahosted.org/freeipa/ticket/2444 >> >> Note: patch depends on pvoborni-0168. > > ACK. > Pushed to master. -- Petr Vobornik From pspacek at redhat.com Tue Jul 10 13:15:03 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 10 Jul 2012 15:15:03 +0200 Subject: [Freeipa-devel] [PATCH 0024] Add debug message to ldap_cache_addrdatalist() Message-ID: <4FFC2AD7.4050303@redhat.com> Hello, this patch adds an debug message to ldap_cache_addrdatalist(). It is very useful for persistent search debugging. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0024-Add-debug-message-to-ldap_cache_addrdatalist.patch Type: text/x-patch Size: 1124 bytes Desc: not available URL: From pspacek at redhat.com Tue Jul 10 13:57:24 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 10 Jul 2012 15:57:24 +0200 Subject: [Freeipa-devel] [PATCH] 0025-0028 Implement SOA serial number increments for external changes Message-ID: <4FFC34C4.6050902@redhat.com> Hello, these patches provides SOA serial auto-increment feature for external changes. Related ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/67 It is necessary to set "psearch" AND "serial_autoincrement" to "yes" in /etc/named.conf to enable this feature. In replicated environment idnsSOAserial attribute has to be declared as non-replicated. It is done by mkosek's patch 281 for 389 DS & FreeIPA. For testing purposes it is enough to add "idnsSOAserial" to end of exclude list in nsDS5ReplicatedAttributeList attribute for each replication agreement located in cn=mapping tree,cn=config subtree. My patch 28 contains "trick" necessary for replicated environments with 389 DS. 389 sends entry change notification (ECN) in cases when non-replicated attribute idnsSOAserial was changed on *other side*. In that case no change is visible in DNS attributes, but ECN is sent by 389. (Attribute modifyTimestamp is changed also.) Patch 28 computes digest/hash from all resource records in idnsZone object and compares old and new digest after each received ECN. This approach eliminates "false changes". Each patch depends on all preceding patches, but each patch implements visible (and testable) part of functionality. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0025-Increment-SOA-serial-for-each-ordinary-record-receiv.patch Type: text/x-patch Size: 5825 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0026-Do-not-bump-serial-for-each-record-during-initial-da.patch Type: text/x-patch Size: 2885 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0027-Maintain-SOA-serial-for-zone-record-changes-also.-Bu.patch Type: text/x-patch Size: 8655 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0028-Add-support-for-replicated-environments-to-SOA-seria.patch Type: text/x-patch Size: 14918 bytes Desc: not available URL: From pviktori at redhat.com Tue Jul 10 15:58:05 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 10 Jul 2012 17:58:05 +0200 Subject: [Freeipa-devel] [PATCH] 0089 Fix batch command error reporting Message-ID: <4FFC510D.6000409@redhat.com> There are a few problems with Batch plugin error reporting: - It reports the text of all errors, not only PublicError. In the normal (non-batch) RPC interface, we hide non-public errors under a generic "internal error" message. - Errors are not localized properly https://fedorahosted.org/freeipa/ticket/2874 - Unlike in the non-batch interface, the batch plugin only provides the error text, not the code and name. https://fedorahosted.org/freeipa/ticket/2901 Here is a simple fix; a more proper way to do it will be to unify this with the RPC code. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0068-Fix-batch-command-error-reporting.patch Type: text/x-patch Size: 5883 bytes Desc: not available URL: From mkosek at redhat.com Tue Jul 10 16:28:49 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 10 Jul 2012 18:28:49 +0200 Subject: [Freeipa-devel] [PATCH] 285 Add automount map/key update permissions Message-ID: <4FFC5841.8000704@redhat.com> Add missing permissions that can be used to delegate write access to existing automount maps or keys. Since automount key RDN has been changed in the past from "automountkey" to "description" and there can be LDAP entries with both RDNs, structure of relevant ACI need to be changed to different scheme. Now, it rather targets a DN of parent automount map object and uses targetfilter to limit the target to automount key objects only. https://fedorahosted.org/freeipa/ticket/2687 -- Martin Kosek Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-285-add-automount-map-key-update-permissions.patch Type: text/x-patch Size: 7056 bytes Desc: not available URL: From sbose at redhat.com Tue Jul 10 21:04:20 2012 From: sbose at redhat.com (Sumit Bose) Date: Tue, 10 Jul 2012 23:04:20 +0200 Subject: [Freeipa-devel] [PATCH] Improve performance of get_group_sids() Message-ID: <20120710210420.GD922@localhost.localdomain> Hi, the following two patches are the first step to fix https://fedorahosted.org/freeipa/ticket/2881. Unit tests with time measurements are added and the performance of the get_group_sids() function is improved by an order of magnitude. The caching of the LDAP results is still missing. I will finish this after my PTO. But I haven't started to work on this. So if you think it should be fixed earlier feel free to take the ticket. bye, Sumit -------------- next part -------------- From a70dd5049943ae88aba46ef3e95b06a944efcf60 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 6 Jul 2012 12:24:01 +0200 Subject: [PATCH] Add unit tests for ipa-kdb To improve the performance of some functions used for group filtering we need a way to see if changes really help or not. This patch adds unit tests for some of these functions and measures the execution time. --- daemons/ipa-kdb/Makefile.am | 29 +++ daemons/ipa-kdb/ipa_kdb_mspac.c | 12 +- daemons/ipa-kdb/ipa_kdb_tests.c | 548 +++++++++++++++++++++++++++++++++++++++ util/ipa_pwd.c | 2 + 4 Dateien ge?ndert, 587 Zeilen hinzugef?gt(+), 4 Zeilen entfernt(-) create mode 100644 daemons/ipa-kdb/ipa_kdb_tests.c diff --git a/daemons/ipa-kdb/Makefile.am b/daemons/ipa-kdb/Makefile.am index 17c090418ec5a0e2a39d948dc385d509c5d05321..34b7d93c4912f988130ae7dd143e054574605071 100644 --- a/daemons/ipa-kdb/Makefile.am +++ b/daemons/ipa-kdb/Makefile.am @@ -50,6 +50,35 @@ ipadb_la_LIBADD = \ $(NDRPAC_LIBS) \ $(NULL) +if HAVE_CHECK +TESTS = ipa_kdb_tests +check_PROGRAMS = ipa_kdb_tests +endif + +ipa_kdb_tests_SOURCES = \ + ipa_kdb_tests.c \ + ipa_kdb.c \ + ipa_kdb_common.c \ + ipa_kdb_mkey.c \ + ipa_kdb_passwords.c \ + ipa_kdb_principals.c \ + ipa_kdb_pwdpolicy.c \ + ipa_kdb_mspac.c \ + ipa_kdb_delegation.c \ + ipa_kdb_audit_as.c \ + $(KRB5_UTIL_SRCS) \ + $(NULL) +ipa_kdb_tests_CFLAGS = $(CHECK_CFLAGS) +ipa_kdb_tests_LDADD = \ + $(CHECK_LIBS) \ + $(KRB5_LIBS) \ + $(LDAP_LIBS) \ + $(NDRPAC_LIBS) \ + -lnss3 \ + -lkdb5 \ + -lsss_idmap \ + $(NULL) + dist_noinst_DATA = ipa_kdb.exports EXTRA_DIST = \ diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index 1c7487c3c8f75d02466a2e0746fbef5d36e3d995..ef9aa96943a7e93c21c665e3292a9843fdd8e03d 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -643,9 +643,9 @@ static char *gen_sid_string(TALLOC_CTX *memctx, struct dom_sid *dom_sid, return str; } -static int get_group_sids(TALLOC_CTX *memctx, - struct PAC_LOGON_INFO_CTR *logon_info, - char ***_group_sids) +int get_group_sids(TALLOC_CTX *memctx, + struct PAC_LOGON_INFO_CTR *logon_info, + char ***_group_sids) { int ret; size_t c; @@ -653,6 +653,10 @@ static int get_group_sids(TALLOC_CTX *memctx, struct dom_sid *domain_sid = NULL; char **group_sids = NULL; + if (logon_info == NULL || logon_info->info == NULL) { + return EINVAL; + } + domain_sid = dom_sid_dup(memctx, logon_info->info->info3.base.domain_sid); if (domain_sid == NULL) { krb5_klog_syslog(LOG_ERR, "dom_sid_dup failed"); @@ -714,7 +718,7 @@ done: return ret; } -static int add_groups(TALLOC_CTX *memctx, +int add_groups(TALLOC_CTX *memctx, struct PAC_LOGON_INFO_CTR *logon_info, size_t ipa_group_sids_count, struct dom_sid2 *ipa_group_sids) diff --git a/daemons/ipa-kdb/ipa_kdb_tests.c b/daemons/ipa-kdb/ipa_kdb_tests.c new file mode 100644 index 0000000000000000000000000000000000000000..d3f6c2f38176e2fe3eb081b050c974f351f221ac --- /dev/null +++ b/daemons/ipa-kdb/ipa_kdb_tests.c @@ -0,0 +1,548 @@ +/** BEGIN COPYRIGHT BLOCK + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * Additional permission under GPLv3 section 7: + * + * In the following paragraph, "GPL" means the GNU General Public + * License, version 3 or any later version, and "Non-GPL Code" means + * code that is governed neither by the GPL nor a license + * compatible with the GPL. + * + * You may link the code of this Program with Non-GPL Code and convey + * linked combinations including the two, provided that such Non-GPL + * Code only links to the code of this Program through those well + * defined interfaces identified in the file named EXCEPTION found in + * the source code files (the "Approved Interfaces"). The files of + * Non-GPL Code may instantiate templates or use macros or inline + * functions from the Approved Interfaces without causing the resulting + * work to be covered by the GPL. Only the copyright holders of this + * Program may make changes or additions to the list of Approved + * Interfaces. + * + * Authors: + * Sumit Bose + * + * Copyright (C) 2012 Red Hat, Inc. + * All rights reserved. + * END COPYRIGHT BLOCK **/ + +#include +#include +#include +#include +#include +#include + +#include "gen_ndr/ndr_krb5pac.h" + +#include "ipa_kdb.h" + +TALLOC_CTX *global_talloc_context = NULL; +struct sss_idmap_ctx *global_idmap_ctx = NULL; + +#define DOM_SID "S-1-5-21-1234-5678-9012" +#define OTHER_DOM_SID "S-1-5-21-1111-2222-3333" +#define PRIMARY_GID 1234 +#define RID_BASE 10000 + +/* talloc leak checks written by Martin Nagy for sssd */ +#define DLIST_ADD(list, p) \ +do { \ + if (!(list)) { \ + (list) = (p); \ + (p)->next = (p)->prev = NULL; \ + } else { \ + (list)->prev = (p); \ + (p)->next = (list); \ + (p)->prev = NULL; \ + (list) = (p); \ + }\ +} while (0) + +#define DLIST_REMOVE(list, p) \ +do { \ + if ((p) == (list)) { \ + (list) = (p)->next; \ + if (list) (list)->prev = NULL; \ + } else { \ + if ((p)->prev) (p)->prev->next = (p)->next; \ + if ((p)->next) (p)->next->prev = (p)->prev; \ + } \ + if ((p) != (list)) (p)->next = (p)->prev = NULL; \ +} while (0) + +#define check_leaks(ctx, bytes) _check_leaks((ctx), (bytes), __location__) +void _check_leaks(TALLOC_CTX *ctx, + size_t bytes, + const char *location); + +void check_leaks_push(TALLOC_CTX *ctx); + +#define check_leaks_pop(ctx) _check_leaks_pop((ctx), __location__) +void _check_leaks_pop(TALLOC_CTX *ctx, const char *location); +struct size_snapshot { + struct size_snapshot *prev; + struct size_snapshot *next; + + TALLOC_CTX *ctx; + size_t bytes_allocated; +}; + +static struct size_snapshot *snapshot_stack; + +void +_check_leaks(TALLOC_CTX *ctx, size_t bytes, const char *location) +{ + size_t bytes_allocated; + + bytes_allocated = talloc_total_size(ctx); + if (bytes_allocated != bytes) { + fprintf(stderr, "Leak report for %s:\n", location); + talloc_report_full(ctx, stderr); + fail("%s: memory leaks detected, %d bytes still allocated", + location, bytes_allocated - bytes); + } +} + +void +check_leaks_push(TALLOC_CTX *ctx) +{ + struct size_snapshot *snapshot; + + snapshot = talloc(NULL, struct size_snapshot); + snapshot->ctx = ctx; + snapshot->bytes_allocated = talloc_total_size(ctx); + DLIST_ADD(snapshot_stack, snapshot); +} + +void +_check_leaks_pop(TALLOC_CTX *ctx, const char *location) +{ + struct size_snapshot *snapshot; + TALLOC_CTX *old_ctx; + size_t bytes_allocated; + + if (snapshot_stack == NULL) { + fail("%s: trying to pop an empty stack"); + } + + snapshot = snapshot_stack; + DLIST_REMOVE(snapshot_stack, snapshot); + + old_ctx = snapshot->ctx; + bytes_allocated = snapshot->bytes_allocated; + + fail_if(old_ctx != ctx, "Bad push/pop order"); + + talloc_free(snapshot); + snapshot = NULL; + _check_leaks(old_ctx, bytes_allocated, location); +} + +static void leak_check_setup(void) +{ + talloc_enable_null_tracking(); + global_talloc_context = talloc_new(NULL); + fail_unless(global_talloc_context != NULL, "talloc_new failed"); + check_leaks_push(global_talloc_context); +} + +static void leak_check_teardown(void) +{ + check_leaks_pop(global_talloc_context); + if (snapshot_stack != NULL) { + fail("Exiting with a non-empty stack"); + } + check_leaks(global_talloc_context, 0); +} + +int krb5_klog_syslog(int l, const char *format, ...) +{ + va_list ap; + char *s = NULL; + int ret; + + va_start(ap, format); + + ret = vasprintf(&s, format, ap); + va_end(ap); + if (ret < 0) { + /* ENOMEM */ + return -1; + } + + fprintf(stderr, "%s\n", s); + free(s); + + return 0; +} + +int get_group_sids(TALLOC_CTX *memctx, + struct PAC_LOGON_INFO_CTR *logon_info, + char ***_group_sids); + +int add_groups(TALLOC_CTX *memctx, + struct PAC_LOGON_INFO_CTR *logon_info, + size_t ipa_group_sids_count, + struct dom_sid2 *ipa_group_sids); + +static void *idmap_talloc(size_t size, void *pvt) +{ + return talloc_size(pvt, size); +} + +static void idmap_free(void *ptr, void *pvt) +{ + talloc_free(ptr); +} + +static int make_dom_sid_array(TALLOC_CTX *mem_ctx, size_t count, + struct dom_sid **_sids) +{ + size_t c; + char *sid_str; + enum idmap_error_code err; + struct dom_sid *sids; + struct dom_sid *sid; + + sids = talloc_array(mem_ctx, struct dom_sid, count); + if (sids == NULL) { + return ENOMEM; + } + + for (c = 0; c < count; c++) { + sid_str = talloc_asprintf(mem_ctx, "%s-%lu", OTHER_DOM_SID, + (unsigned long) RID_BASE + c); + if (sid_str == NULL) { + return ENOMEM; + } + err = sss_idmap_sid_to_smb_sid(global_idmap_ctx, sid_str, &sid); + talloc_free(sid_str); + if (err != IDMAP_SUCCESS) { + return EINVAL; + } + memcpy(&sids[c], sid, sizeof(struct dom_sid)); + talloc_free(sid); + } + + *_sids = sids; + return 0; +} + +static int make_login_info(TALLOC_CTX *mem_ctx, size_t d_group_count, + size_t e_group_count, + struct PAC_LOGON_INFO_CTR **_logon_info) +{ + struct PAC_LOGON_INFO_CTR *logon_info; + enum idmap_error_code err; + size_t c; + char *sid_str; + struct dom_sid *sid; + + logon_info = talloc_zero(mem_ctx, struct PAC_LOGON_INFO_CTR); + if (logon_info == NULL) { + return ENOMEM; + } + + logon_info->info = talloc_zero(logon_info, struct PAC_LOGON_INFO); + if (logon_info->info == NULL) { + return ENOMEM; + } + + err = sss_idmap_sid_to_smb_sid(global_idmap_ctx, DOM_SID, &sid); + if (err != IDMAP_SUCCESS) { + return EINVAL; + } + logon_info->info->info3.base.domain_sid = talloc_steal(logon_info->info, sid); + + logon_info->info->info3.base.primary_gid = PRIMARY_GID; + + if (d_group_count > 0) { + logon_info->info->info3.base.groups.count = d_group_count; + logon_info->info->info3.base.groups.rids = talloc_array(logon_info, + struct samr_RidWithAttribute, + d_group_count); + if (logon_info->info->info3.base.groups.rids == NULL) { + return ENOMEM; + } + + for (c = 0; c < d_group_count; c++) { + logon_info->info->info3.base.groups.rids[c].rid = RID_BASE + c; + logon_info->info->info3.base.groups.rids[c].attributes = 0; + } + } + + if (e_group_count > 0) { + logon_info->info->info3.sidcount = e_group_count; + logon_info->info->info3.sids = talloc_array(logon_info, + struct netr_SidAttr, + e_group_count); + if (logon_info->info->info3.sids == NULL) { + return ENOMEM; + } + + for (c = 0; c < e_group_count; c++) { + sid_str = talloc_asprintf(mem_ctx, "%s-%lu", OTHER_DOM_SID, + (unsigned long) RID_BASE + c); + if (sid_str == NULL) { + return ENOMEM; + } + err = sss_idmap_sid_to_smb_sid(global_idmap_ctx, sid_str, &sid); + talloc_free(sid_str); + if (err != IDMAP_SUCCESS) { + return EINVAL; + } + logon_info->info->info3.sids[c].sid = talloc_steal(logon_info->info, sid); + logon_info->info->info3.sids[c].attributes = 0; + } + } + + *_logon_info = logon_info; + return 0; +} + +static int check_groups(size_t d_group_count, size_t e_group_count, + char **group_sids) +{ + int ret; + TALLOC_CTX *tmp_ctx; + char *tmp_str = NULL; + size_t c; + + tmp_ctx = talloc_new(NULL); + if (tmp_ctx == NULL) { + ret = ENOMEM; + goto done; + } + + tmp_str = talloc_asprintf(tmp_ctx, "%s-%d", DOM_SID, PRIMARY_GID); + if (tmp_str == NULL) { + ret = ENOMEM; + goto done; + } + + if (strcmp(group_sids[0], tmp_str) != 0) { + ret = EINVAL; + goto done; + } + + for (c = 0; c < d_group_count; c++) { + tmp_str = talloc_asprintf(tmp_ctx, "%s-%lu", DOM_SID, + (unsigned long) RID_BASE + c); + if (tmp_str == NULL) { + ret = ENOMEM; + goto done; + } + + if (strcmp(group_sids[1 + c], tmp_str) != 0) { + fprintf(stderr, "[%s][%s]\n", group_sids[1 + c], tmp_str); + ret = EINVAL; + goto done; + } + } + + for (c = 0; c < e_group_count; c++) { + tmp_str = talloc_asprintf(tmp_ctx, "%s-%lu", OTHER_DOM_SID, + (unsigned long) RID_BASE + c); + if (tmp_str == NULL) { + ret = ENOMEM; + goto done; + } + + if (strcmp(group_sids[1 + d_group_count + c], tmp_str) != 0) { + fprintf(stderr, "[%s][%s]\n", group_sids[1 + d_group_count + c], + tmp_str); + ret = EINVAL; + goto done; + } + } + + ret = 0; + +done: + talloc_free(tmp_ctx); + return ret; +} + +struct get_groups_test_params { + size_t dom_group_count; + size_t external_group_count; +}; + +START_TEST(test_get_groups) +{ + struct PAC_LOGON_INFO_CTR *logon_info = NULL; + char **group_sids = NULL; + int ret; + size_t c; + struct timeval start_time; + struct timeval end_time; + struct timeval diff_time; + struct get_groups_test_params params[] = { + {0, 0}, + {1000, 0}, + {10000, 0}, + {100000, 0}, + {1000000, 0}, + {0, 1000}, + {0, 10000}, + {0, 100000}, + {0, 1000000}, + {1000, 1000}, + {10000, 10000}, + {100000, 100000}, + {1000000, 1000000}, + {(size_t) -1, (size_t) -1} + }; + + for (c = 0; params[c].dom_group_count != (size_t) - 1; c++) { + ret = make_login_info(global_talloc_context, params[c].dom_group_count, + params[c].external_group_count, &logon_info); + fail_unless(ret == 0, "make_login_info failed."); + + ret = gettimeofday(&start_time, NULL); + fail_unless(ret == 0, "gettimeofday failed."); + + ret = get_group_sids(global_talloc_context, logon_info, &group_sids); + fail_unless(ret == 0, "get_group_sids failed [%d][%s].", + ret, strerror(ret)); + + ret = gettimeofday(&end_time, NULL); + fail_unless(ret == 0, "gettimeofday failed."); + + talloc_free(logon_info); + logon_info = NULL; + + fail_unless(check_groups(params[c].dom_group_count, + params[c].external_group_count, + group_sids) == 0, + "Unexpected SIDs returned."); + + talloc_free(group_sids); + group_sids = NULL; + + timersub(&end_time, &start_time, &diff_time); + fprintf(stderr, "get_groups: domain groups [%d] external group [%d] duration [%ds %dus]\n", + params[c].dom_group_count, + params[c].external_group_count, + (int) diff_time.tv_sec, + (int) diff_time.tv_usec); + + } + +} +END_TEST + +struct add_groups_test_params { + size_t dom_group_count; +}; + +START_TEST(test_add_groups) +{ + struct PAC_LOGON_INFO_CTR *logon_info = NULL; + struct dom_sid *group_sids = NULL; + int ret; + size_t c; + struct timeval start_time; + struct timeval end_time; + struct timeval diff_time; + struct add_groups_test_params params[] = { + {0}, + {1000}, + {10000}, + {100000}, + {1000000}, + {2000000}, + {(size_t) -1} + }; + + for (c = 0; params[c].dom_group_count != (size_t) - 1; c++) { + ret = make_login_info(global_talloc_context, 10, 10, &logon_info); + fail_unless(ret == 0, "make_login_info failed."); + + ret = make_dom_sid_array(global_talloc_context, + params[c].dom_group_count, &group_sids); + fail_unless(ret == 0, "make_dom_sid_array failed."); + + ret = gettimeofday(&start_time, NULL); + fail_unless(ret == 0, "gettimeofday failed."); + + ret = add_groups(global_talloc_context, logon_info, + params[c].dom_group_count, group_sids); + fail_unless(ret == 0, "add_groups failed."); + + ret = gettimeofday(&end_time, NULL); + fail_unless(ret == 0, "gettimeofday failed."); + + talloc_free(group_sids); + group_sids = NULL; + talloc_free(logon_info); + logon_info = NULL; + + timersub(&end_time, &start_time, &diff_time); + fprintf(stderr, "add_groups: domain groups [%d] duration [%ds %dus]\n", + params[c].dom_group_count, + (int) diff_time.tv_sec, + (int) diff_time.tv_usec); + } + +} +END_TEST + +static void idmap_setup(void) +{ + enum idmap_error_code err; + + err = sss_idmap_init(idmap_talloc, global_talloc_context, idmap_free, + &global_idmap_ctx); + fail_unless(err == IDMAP_SUCCESS, "sss_idmap_init failed."); +} + +static void idmap_teardown(void) +{ + talloc_free(global_idmap_ctx); + global_idmap_ctx = NULL; +} + +Suite * ipa_kdb_suite(void) +{ + Suite *s = suite_create("IPA kdb"); + + TCase *tc_group = tcase_create("Group filtering"); + tcase_add_checked_fixture(tc_group, + leak_check_setup, + leak_check_teardown); + tcase_add_checked_fixture(tc_group, + idmap_setup, + idmap_teardown); + tcase_add_test(tc_group, test_get_groups); + tcase_add_test(tc_group, test_add_groups); + suite_add_tcase(s, tc_group); + + tcase_set_timeout(tc_group, 300); + return s; +} + +int main(void) +{ + int number_failed; + + Suite *s = ipa_kdb_suite (); + SRunner *sr = srunner_create (s); + srunner_run_all (sr, CK_VERBOSE); + number_failed = srunner_ntests_failed (sr); + srunner_free (sr); + + return (number_failed == 0) ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/util/ipa_pwd.c b/util/ipa_pwd.c index 92fb3b0298418592881d100fb7a9ccfac99fd665..761d1efb8cbcb303d4ec4edd49254b433b048b31 100644 --- a/util/ipa_pwd.c +++ b/util/ipa_pwd.c @@ -20,7 +20,9 @@ * along with this program. If not, see . */ +#ifndef _GNU_SOURCE #define _GNU_SOURCE +#endif #include #include #include -- 1.7.10.2 -------------- next part -------------- From 6d0decc2fb812583c5e1e210cb63cca047e60378 Mon Sep 17 00:00:00 2001 From: Sumit Bose Date: Fri, 6 Jul 2012 21:40:35 +0200 Subject: [PATCH] Improve performance of get_group_sids() This patch tries to improve the performance of get_group_sids() by reducing the number of memory allocations. --- daemons/ipa-kdb/ipa_kdb_mspac.c | 110 +++++++++++++-------------------------- 1 Datei ge?ndert, 36 Zeilen hinzugef?gt(+), 74 Zeilen entfernt(-) diff --git a/daemons/ipa-kdb/ipa_kdb_mspac.c b/daemons/ipa-kdb/ipa_kdb_mspac.c index ef9aa96943a7e93c21c665e3292a9843fdd8e03d..e1144a1d3e1aa559a47a305c1aa14c4462c7899a 100644 --- a/daemons/ipa-kdb/ipa_kdb_mspac.c +++ b/daemons/ipa-kdb/ipa_kdb_mspac.c @@ -169,43 +169,6 @@ static char *dom_sid_string(TALLOC_CTX *memctx, const struct dom_sid *dom_sid) return buf; } -static struct dom_sid *dom_sid_dup(TALLOC_CTX *memctx, - const struct dom_sid *dom_sid) -{ - struct dom_sid *new_sid; - size_t c; - - if (dom_sid == NULL) { - return NULL; - } - - new_sid = talloc(memctx, struct dom_sid); - if (new_sid == NULL) { - return NULL; - } - - new_sid->sid_rev_num = dom_sid->sid_rev_num; - for (c = 0; c < SID_ID_AUTHS; c++) { - new_sid->id_auth[c] = dom_sid->id_auth[c]; - } - new_sid->num_auths = dom_sid->num_auths; - for (c = 0; c < SID_SUB_AUTHS; c++) { - new_sid->sub_auths[c] = dom_sid->sub_auths[c]; - } - - return new_sid; -} - -static int sid_append_rid(struct dom_sid *sid, uint32_t rid) -{ - if (sid->num_auths >= SID_SUB_AUTHS) { - return EINVAL; - } - - sid->sub_auths[sid->num_auths++] = rid; - return 0; -} - /** * @brief Takes a user sid and removes the rid. * The sid is changed by this function, @@ -620,29 +583,6 @@ static bool is_cross_realm_krbtgt(krb5_const_principal princ) return true; } -static char *gen_sid_string(TALLOC_CTX *memctx, struct dom_sid *dom_sid, - uint32_t rid) -{ - char *str = NULL; - int ret; - - ret = sid_append_rid(dom_sid, rid); - if (ret != 0) { - krb5_klog_syslog(LOG_ERR, "sid_append_rid failed"); - return NULL; - } - - str = dom_sid_string(memctx, dom_sid); - ret = sid_split_rid(dom_sid, NULL); - if (ret != 0) { - krb5_klog_syslog(LOG_ERR, "sid_split_rid failed"); - talloc_free(str); - return NULL; - } - - return str; -} - int get_group_sids(TALLOC_CTX *memctx, struct PAC_LOGON_INFO_CTR *logon_info, char ***_group_sids) @@ -650,19 +590,27 @@ int get_group_sids(TALLOC_CTX *memctx, int ret; size_t c; size_t p = 0; - struct dom_sid *domain_sid = NULL; + char *dom_sid_str = NULL; char **group_sids = NULL; + char *group_sids_buf = NULL; + size_t dom_sid_str_len; + size_t sid_len; + char *idx; + if (logon_info == NULL || logon_info->info == NULL) { return EINVAL; } - domain_sid = dom_sid_dup(memctx, logon_info->info->info3.base.domain_sid); - if (domain_sid == NULL) { - krb5_klog_syslog(LOG_ERR, "dom_sid_dup failed"); + dom_sid_str = dom_sid_string(memctx, + logon_info->info->info3.base.domain_sid); + if (dom_sid_str == NULL) { + krb5_klog_syslog(LOG_ERR, "dom_sid_string failed."); ret = ENOMEM; goto done; } + dom_sid_str_len = strlen(dom_sid_str); + sid_len = dom_sid_str_len + 12; group_sids = talloc_array(memctx, char *, 2 + @@ -674,27 +622,41 @@ int get_group_sids(TALLOC_CTX *memctx, goto done; } - group_sids[p] = gen_sid_string(memctx, domain_sid, - logon_info->info->info3.base.primary_gid); - if (group_sids[p] == NULL) { - krb5_klog_syslog(LOG_ERR, "gen_sid_string failed"); + group_sids_buf = talloc_zero_size(group_sids, + sid_len * (1 + logon_info->info->info3.base.groups.count)); + if (group_sids_buf == NULL) { + krb5_klog_syslog(LOG_ERR, "talloc_zero failed"); + ret = ENOMEM; + goto done; + } + + idx = group_sids_buf+(p*sid_len); + memcpy(idx, dom_sid_str, dom_sid_str_len); + ret = snprintf(idx+dom_sid_str_len, 12, "-%lu", + (unsigned long) logon_info->info->info3.base.primary_gid); + if (ret < 0 || ret >= 12) { + krb5_klog_syslog(LOG_ERR, "snprintf failed"); ret = EINVAL; goto done; } + group_sids[p] = idx; p++; for (c = 0; c < logon_info->info->info3.base.groups.count; c++) { - group_sids[p] = gen_sid_string(memctx, domain_sid, - logon_info->info->info3.base.groups.rids[c].rid); - if (group_sids[p] == NULL) { - krb5_klog_syslog(LOG_ERR, "gen_sid_string 2 failed"); + idx = group_sids_buf+(p*sid_len); + memcpy(idx, dom_sid_str, dom_sid_str_len); + ret = snprintf(idx+dom_sid_str_len, 12, "-%lu", + (unsigned long) logon_info->info->info3.base.groups.rids[c].rid); + if (ret < 0 || ret >= 12) { + krb5_klog_syslog(LOG_ERR, "snprintf failed"); ret = EINVAL; goto done; } + group_sids[p] = idx; p++; } for (c = 0; c < logon_info->info->info3.sidcount; c++) { - group_sids[p] = dom_sid_string(memctx, + group_sids[p] = dom_sid_string(group_sids, logon_info->info->info3.sids[c].sid); if (group_sids[p] == NULL) { krb5_klog_syslog(LOG_ERR, "dom_sid_string failed"); @@ -710,7 +672,7 @@ int get_group_sids(TALLOC_CTX *memctx, ret = 0; done: - talloc_free(domain_sid); + talloc_free(dom_sid_str); if (ret != 0) { talloc_free(group_sids); } -- 1.7.10.2 From pvoborni at redhat.com Wed Jul 11 07:19:13 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 11 Jul 2012 09:19:13 +0200 Subject: [Freeipa-devel] [PATCH] [solarus] 001 Indirect roles in WebUI Message-ID: <4FFD28F1.1030408@redhat.com> We have a Web UI patch in track submitted by David Sp?ngberg (solarus). I'm sending it here to keep order. https://fedorahosted.org/freeipa/ticket/2899 I ACKed and pushed it to master. Problem description: We use roles to determine if to show self-service or admin interface for not-admin users. We only checked for memberof_role and not memberofindirect_role. This patch adds a check for memberofindirect_role. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-solarus-0001-Indirect-roles-in-WebUI.patch Type: text/x-patch Size: 1087 bytes Desc: not available URL: From pviktori at redhat.com Wed Jul 11 07:59:51 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 11 Jul 2012 09:59:51 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <4FFD3277.9070607@redhat.com> On 07/07/2012 08:45 PM, John Dennis wrote: > The DN work I was doing on master is ready for review and testing. It's > been a long haul and I've been working relentlessly to get this work > completed. I am on PTO for a week starting today (I know bad timing) but > I spent yesterday and my first day of PTO today writing up extensive > documentation for the work so others can start taking a look at it while > I'm gone. The documentation as well as where to find the code can be > found here: > > http://jdennis.fedorapeople.org/dn_summary.html > > The document is long but I felt it was better to provide explanations > for as much as possible. > > I may check in during the week but I'm going to try and discipline > myself not to and take an actual much needed break. > > John > I went over the changes to ipaserver/plugins/ldap2.py; now I can start the testing. I must ask about the wrapped _ext methods. I assume python-ldap exposes them only to mimic the C interface, which has them because C doesn't have default arguments. Would we lose anything if the class only defined the _ext methods, and aliased the non-ext variants to them? e.g. def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): assert isinstance(dn, DN) dn = str(dn) modlist = self.encode(modlist) return self.conn.add_ext(dn, modlist, serverctrls, clientctrls) add = add_ext The non-_ext variants are deprecated in the underlying C library, anyway. Why are some of the wrapper methods left out? (compare, delete_ext, modify, etc.) Line 1519: checkmembers = copy.deepcopy(members) for member_dn in checkmembers: ... if m not in checkmembers: checkmembers.append(m) # FIXME jrd; can't modify list during traversal NACK. You really can't modify lists during traversal, a FIXME won't cut it. A smaller issue is that the list `in` operator does alinear scan, so calling it in a loop can very slow as the list grows. Sets are much better for membership tests. So I believe you want something like: checkmembers = set(DN(m, locked=True) for m in members) checked = set() while checkmembers: member_dn = checkmembers.pop() checked.add(member_dn) ... if m not in checked: checkmembers.add(m) Lines 203, 825: os.environ['KRB5CCNAME'] = ccache_file Should KRB5CCNAME be unset/restored once the code's done with it. Line 228: except IndexError: The try clause is too long for such a general exception, this can hide real errors. ==== And as usual I have a lot of nitpicks. Sometimes I just want to share my ideas about how good code should look like, don't take them too seriously if you think otherwise :) Line 107 & 127: return utf8_codec.decode(val)[0] return utf8_codec.encode(unicode(val))[0] I believe the following would be more readable: return val.decode('utf-8') return unicode(val).encode('utf-8') Line 134: class ServerSchema(object): Don't use nested classes without reason Line 148: def get_schema(self, url, conn=None, force_update=False): Is the force_update & flush() useful? None of the code seems to use it. Line 354: _SCHEMA_OVERRIDE = { We have a case-insensitive dict class, why not use it here? Lines 388, 750: def is_dn_syntax(self, attr): """ Check the schema to see if the attribute uses DN syntax. uses_dn_syntax or has_dn_syntax would be a better name. Line 395: if syntax is None: return False return syntax == DN_SYNTAX_OID The if is redundant. Line 406: if isinstance(val, bool): if val: val = 'TRUE' else: val = 'FALSE' return self.encode(val) Encoding an ASCII string should be a no-op, why bother doing it? Line 421: dct = dict() for (k, v) in val.iteritems(): dct[self.encode(k)] = self.encode(v) return dct Consider: dct = dict((self.encode(k), self.encode(v)) for k, v in val.iteritems()) Line 437: if type(target_type) is type and isinstance(original_value, target_type): Don't use `type(x) is expected_type` for type checking! Use isinstance(target_type, type) so you also catch type's subclasses. Or better yet, handle the TypeError that isinstance will raise if you don't give it a class. To be honest, I don't think it's worth it to do all this convoluted checking just to make sure you're not copying unnecessarily. Do you have a reason I can't see? Line 594, 604, 613: ipa_result = self.convert_result(ldap_result) return ipa_result Just return the result directly, no need for the extra variable. Line 876, 894, 1041: parent_dn -- DN of the parent entry (default '') The default is None. In any case it's redundant to specify default values in docstrings, as you can just look at the function itself. Line 1422: self.handle_errors(e, **{}) Wouldn't the following work better? self.handle_errors(e, *[{}]*0) Line 1518: checkmembers = copy.deepcopy(members) deepcopy() is rarely what you want; it either copies too much or too little. The Python developers made a mistake by assigning a short name to such an ill-specified operation. It's better to do this explicitly. checkmembers = [DN(m) for m in members] Or to combine with my previous suggestion, checkmembers = set(DN(m, locked=True) for m in members) Line 1609: indirect = memberof[:] # make a copy of the list I believe the list() is more readable ?? doesn't need a comment: indirect = list(memberof) -- Petr? From mkosek at redhat.com Wed Jul 11 08:34:19 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 11 Jul 2012 10:34:19 +0200 Subject: [Freeipa-devel] [PATCH] 283 Improve address family handling in sockets In-Reply-To: <4FF3ED04.9020002@redhat.com> References: <4FF3ED04.9020002@redhat.com> Message-ID: <4FFD3A8B.1040708@redhat.com> On 07/04/2012 09:13 AM, Martin Kosek wrote: > I did various tests with IPv4 and IPv6 and everything worked for me. I also > tried a mixed IPv4+IPv6 and IPv6-only environment and I was able to install an > IPv6-only replica without issues. > > --- > > Many functions use low-level socket interface for connection or > various checks. However, most of the time we don't respect > automatic address family detection but rather try to force our > values. This may cause either redundat connection tries when an > address family is disabled on system tries or even crashes > when socket exceptions are not properly caught. > > Instead of forcing address families to socket, rather use > getaddrinfo interface to automatically retrieve a list of all > relevant address families and other connection settings when > connecting to remote/local machine or binding to a local port. > Now, we will also fill correctly all connection parameters like > flowinfo and scopeid for IPv6 connections which will for example > prevent issues with scoped IPv6 addresses. > > bind_port_responder function was changed to at first try to bind > to IPv6 wildcard address before IPv4 as IPv6 socket is able to > accept both IPv4 and IPv6 connections (unlike IPv4 socket). > > nsslib connection was refactored to use nss.io.AddrInfo class to > get all the available connections. Socket is now not created by > default in NSSConnection class initializer, but rather when the > actual connection is being made, becase we do not an address family > where connection is successful. > > https://fedorahosted.org/freeipa/ticket/2695 > Attaching a rebased patch with updated comment - the patch also fix issues in ticket 2913. I just found an easy way to reproduce an issue caused by incorrect address family handling that can be tried during review: 1) Turn of IPv6 in your (Fedora) OS: - add "ipv6.disable=1" as kernel parameter in your kernel line in your bootloader conf - add "NETWORKING_IPV6=no" to your /etc/sysconfig/network 2) Run "ipa-replica-conncheck -m " where is a fqdn of some of your running IPA servers. Current IPA version will produce bunch of tracebacks, patched IPA should work without any issue Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-283-2-improve-address-family-handling-in-sockets.patch Type: text/x-patch Size: 20553 bytes Desc: not available URL: From mkosek at redhat.com Wed Jul 11 08:50:46 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 11 Jul 2012 10:50:46 +0200 Subject: [Freeipa-devel] [PATCH] 0089 Fix batch command error reporting In-Reply-To: <4FFC510D.6000409@redhat.com> References: <4FFC510D.6000409@redhat.com> Message-ID: <4FFD3E66.9090104@redhat.com> On 07/10/2012 05:58 PM, Petr Viktorin wrote: > There are a few problems with Batch plugin error reporting: > > - It reports the text of all errors, not only PublicError. In the normal > (non-batch) RPC interface, we hide non-public errors under a generic "internal > error" message. > - Errors are not localized properly > https://fedorahosted.org/freeipa/ticket/2874 > - Unlike in the non-batch interface, the batch plugin only provides the error > text, not the code and name. > https://fedorahosted.org/freeipa/ticket/2901 > > Here is a simple fix; a more proper way to do it will be to unify this with the > RPC code. > The patch was already ACKed by Rob on different channel. Giving ACK #2. Pushed to master. Martin From abokovoy at redhat.com Wed Jul 11 11:55:21 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 11 Jul 2012 14:55:21 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1341693912.2599.15.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> Message-ID: <20120711115521.GA9427@redhat.com> On Sat, 07 Jul 2012, Simo Sorce wrote: >When installing the adtrust code we need to be able to get the ipaNTHash >populated as in some cases we may need it to authenticate connections >over SMB w/o using kerberos during the trust setup phase. > >The NT hash is really just the same thing as the rc4-hmac key we already >have by default in the Kerberos Keys. > >This patch-set implements a check in the password plugin for the pre-mod >operation to catch the attempt to replace the attribute with the value >'MagicRegen' in the ipaNThash attribute. > >If no previous ipaNTHash value is present, and the kerberos keys are >available, then we attempt to find the rc4-hmac key and if we find it we >store it in the ipaNTHash. > >This will allow us to give the admin user (and potentially any other >user) the NT hash samba requires without forcing them to reset their >password, assuming the rc4-hmac key is present (currently it is by >default). > >I marked this patch-set as RFC as I want opinions on the method (LDAP >modify with replace operation) I utilized to perform the extraction. > >If it bode well with everybody we can consider the patch-set for >inclusion. > >I tested it and extracting the hash works fine and it works later on >using smbclient to access a share. > >This patchset implements task #2867: >https://fedorahosted.org/freeipa/ticket/2867 I've read through the code and I think it is a sensible approach. However, let's get through its possible use first. If I understood correctly, ipasam is supposed to do following: 1. Upon user lookup ldapsam_getsampwnam() in ipa_sam.c will be called 2. It will call init_sam_from_ldap() 3. init_sam_from_ldap() will attempt to read ipaNTHash 4. If (3) failed, we don't call pdb_set_nt_passwd() to set NT hash in internal pdb structure that is later used by smbd to authenticate the user. Now, with this RFC in action, we'll have: 3a. read ipaNTHash 3b. if failed, perform mod/replace ipaNTHash value with MagicRegen 3c. read ipaNTHash Besides two reads, we may have possible race condition where attribute value is not yet written back when (3c) occurs. Can we replace this sequence with a single extended operation? 3. Perform extended operation, say, IPA_OBTAIN_NTHASH, get ipaNTHash value back for a specified DN The operation internally would handle both cases when ipaNTHash is available and when it is not. We also can simply use ipaNTHash attribute length as differentiating factor for cases when ipaNTHash generation should be prevented. I.e. absent or empty ipaNTHash would cause generating the hash from kerberos keys if possible, ipaNTHash set to size less than 16 would prevent generating the hash. -- / Alexander Bokovoy From simo at redhat.com Wed Jul 11 12:06:49 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 11 Jul 2012 08:06:49 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <20120711115521.GA9427@redhat.com> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> Message-ID: <1342008409.2599.62.camel@willson.li.ssimo.org> On Wed, 2012-07-11 at 14:55 +0300, Alexander Bokovoy wrote: > On Sat, 07 Jul 2012, Simo Sorce wrote: > >When installing the adtrust code we need to be able to get the ipaNTHash > >populated as in some cases we may need it to authenticate connections > >over SMB w/o using kerberos during the trust setup phase. > > > >The NT hash is really just the same thing as the rc4-hmac key we already > >have by default in the Kerberos Keys. > > > >This patch-set implements a check in the password plugin for the pre-mod > >operation to catch the attempt to replace the attribute with the value > >'MagicRegen' in the ipaNThash attribute. > > > >If no previous ipaNTHash value is present, and the kerberos keys are > >available, then we attempt to find the rc4-hmac key and if we find it we > >store it in the ipaNTHash. > > > >This will allow us to give the admin user (and potentially any other > >user) the NT hash samba requires without forcing them to reset their > >password, assuming the rc4-hmac key is present (currently it is by > >default). > > > >I marked this patch-set as RFC as I want opinions on the method (LDAP > >modify with replace operation) I utilized to perform the extraction. > > > >If it bode well with everybody we can consider the patch-set for > >inclusion. > > > >I tested it and extracting the hash works fine and it works later on > >using smbclient to access a share. > > > >This patchset implements task #2867: > >https://fedorahosted.org/freeipa/ticket/2867 > I've read through the code and I think it is a sensible approach. However, > let's get through its possible use first. If I understood correctly, > ipasam is supposed to do following: > > 1. Upon user lookup ldapsam_getsampwnam() in ipa_sam.c will be called > 2. It will call init_sam_from_ldap() > 3. init_sam_from_ldap() will attempt to read ipaNTHash > 4. If (3) failed, we don't call pdb_set_nt_passwd() to set NT hash in > internal pdb structure that is later used by smbd to authenticate the > user. > > Now, with this RFC in action, we'll have: > 3a. read ipaNTHash > 3b. if failed, perform mod/replace ipaNTHash value with MagicRegen > 3c. read ipaNTHash > > Besides two reads, we may have possible race condition where attribute value > is not yet written back when (3c) occurs. My idea ws that you run the MagicRegen operation out of band, and not in ipasam. The reason is that for users without an RC4-HMAC key you'd keep doing it over and over again. My idea was that when the ipa trust-add operation is run we execute a magicregen op for the user that run it. Then we can run a process that adds ipaNThash via magicregen for all users we want it to. This is something you really need to do only once as after you have added the appropriate objectclass any password change will keep things in sync. > Can we replace this sequence with a single extended operation? I prefer avoiding whole extended operations for trivial stuff that needs to be run once only. > 3. Perform extended operation, say, IPA_OBTAIN_NTHASH, get ipaNTHash > value back for a specified DN > > The operation internally would handle both cases when ipaNTHash is > available and when it is not. > > We also can simply use ipaNTHash attribute length as differentiating > factor for cases when ipaNTHash generation should be prevented. I.e. > absent or empty ipaNTHash would cause generating the hash from kerberos > keys if possible, ipaNTHash set to size less than 16 would prevent > generating the hash. We could do that but I think it would be a lot of work for very little gain. Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Wed Jul 11 12:41:21 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 11 Jul 2012 15:41:21 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1342008409.2599.62.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> Message-ID: <20120711124121.GB9427@redhat.com> On Wed, 11 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-11 at 14:55 +0300, Alexander Bokovoy wrote: >> On Sat, 07 Jul 2012, Simo Sorce wrote: >> >When installing the adtrust code we need to be able to get the ipaNTHash >> >populated as in some cases we may need it to authenticate connections >> >over SMB w/o using kerberos during the trust setup phase. >> > >> >The NT hash is really just the same thing as the rc4-hmac key we already >> >have by default in the Kerberos Keys. >> > >> >This patch-set implements a check in the password plugin for the pre-mod >> >operation to catch the attempt to replace the attribute with the value >> >'MagicRegen' in the ipaNThash attribute. >> > >> >If no previous ipaNTHash value is present, and the kerberos keys are >> >available, then we attempt to find the rc4-hmac key and if we find it we >> >store it in the ipaNTHash. >> > >> >This will allow us to give the admin user (and potentially any other >> >user) the NT hash samba requires without forcing them to reset their >> >password, assuming the rc4-hmac key is present (currently it is by >> >default). >> > >> >I marked this patch-set as RFC as I want opinions on the method (LDAP >> >modify with replace operation) I utilized to perform the extraction. >> > >> >If it bode well with everybody we can consider the patch-set for >> >inclusion. >> > >> >I tested it and extracting the hash works fine and it works later on >> >using smbclient to access a share. >> > >> >This patchset implements task #2867: >> >https://fedorahosted.org/freeipa/ticket/2867 >> I've read through the code and I think it is a sensible approach. However, >> let's get through its possible use first. If I understood correctly, >> ipasam is supposed to do following: >> >> 1. Upon user lookup ldapsam_getsampwnam() in ipa_sam.c will be called >> 2. It will call init_sam_from_ldap() >> 3. init_sam_from_ldap() will attempt to read ipaNTHash >> 4. If (3) failed, we don't call pdb_set_nt_passwd() to set NT hash in >> internal pdb structure that is later used by smbd to authenticate the >> user. >> >> Now, with this RFC in action, we'll have: >> 3a. read ipaNTHash >> 3b. if failed, perform mod/replace ipaNTHash value with MagicRegen >> 3c. read ipaNTHash >> >> Besides two reads, we may have possible race condition where attribute value >> is not yet written back when (3c) occurs. > >My idea ws that you run the MagicRegen operation out of band, and not in >ipasam. The reason is that for users without an RC4-HMAC key you'd keep >doing it over and over again. If users don't have RC4-HMAC key and don't have ipaNTHash set, they can't log in into smbd anyway until they change their password. >My idea was that when the ipa trust-add operation is run we execute a >magicregen op for the user that run it. Then we can run a process that >adds ipaNThash via magicregen for all users we want it to. So we get to the same issue of a task run against potentially unbound number of users, including replication interaction. Instead, a scheme with ipasam-based generator would mean we: 1. Fetch the user attributes from LDAP 2. Notice ipaNTHash is missing and not disabled 3. Issue ipaNTHash update request if (2) is true. Maybe we can turn off ipaNTHash from your pre-mod code if there is no RC4-HMAC key and ipaNTHash wasn't set? Password change op will get that overriden, of course. Then we can rely on it in (2) above. >This is something you really need to do only once as after you have >added the appropriate objectclass any password change will keep things >in sync. > >> Can we replace this sequence with a single extended operation? > >I prefer avoiding whole extended operations for trivial stuff that needs >to be run once only. If we decide to use it in ipasam, extended operation will be simpliest thing -- contrary to other approaches which would require two LDAP requests. It also allows to return the key in the same go. >> 3. Perform extended operation, say, IPA_OBTAIN_NTHASH, get ipaNTHash >> value back for a specified DN >> >> The operation internally would handle both cases when ipaNTHash is >> available and when it is not. >> >> We also can simply use ipaNTHash attribute length as differentiating >> factor for cases when ipaNTHash generation should be prevented. I.e. >> absent or empty ipaNTHash would cause generating the hash from kerberos >> keys if possible, ipaNTHash set to size less than 16 would prevent >> generating the hash. > >We could do that but I think it would be a lot of work for very little >gain. See above. Being able to differentiate cases - RC4-HMAC is missing, no automatic ipaNTHash generation - RC4-HMAC is available, generate ipaNTHash - ipaNTHash is generated via password change is worth it because it allows to avoid additional roundtrips from ipasam and still be able to avoid generator tasks. -- / Alexander Bokovoy From pspacek at redhat.com Wed Jul 11 13:10:11 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 11 Jul 2012 15:10:11 +0200 Subject: [Freeipa-devel] [PATCH 0029] Add documention for serial_autoincrement feature Message-ID: <4FFD7B33.1000800@redhat.com> Hello, this patch adds documention for serial_autoincrement feature to README. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0029-Add-documention-for-serial_autoincrement-feature.patch Type: text/x-patch Size: 1293 bytes Desc: not available URL: From simo at redhat.com Wed Jul 11 13:20:41 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 11 Jul 2012 09:20:41 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <20120711124121.GB9427@redhat.com> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> Message-ID: <1342012841.2599.75.camel@willson.li.ssimo.org> On Wed, 2012-07-11 at 15:41 +0300, Alexander Bokovoy wrote: > If users don't have RC4-HMAC key and don't have ipaNTHash set, they > can't log in into smbd anyway until they change their password. Yes the point is that you may have users you do not want to give a password to. No need to keep retrying to generate a hash. > >My idea was that when the ipa trust-add operation is run we execute a > >magicregen op for the user that run it. Then we can run a process that > >adds ipaNThash via magicregen for all users we want it to. > So we get to the same issue of a task run against potentially unbound > number of users, including replication interaction. > > Instead, a scheme with ipasam-based generator would mean we: > 1. Fetch the user attributes from LDAP > 2. Notice ipaNTHash is missing and not disabled > 3. Issue ipaNTHash update request if (2) is true. > > Maybe we can turn off ipaNTHash from your pre-mod code if there is no > RC4-HMAC key and ipaNTHash wasn't set? Password change op will get that > overriden, of course. Then we can rely on it in (2) above. Not sure what you mean by 'turn off ipaNTHash from your pre-mod code'. > If we decide to use it in ipasam, extended operation will be simpliest > thing -- contrary to other approaches which would require two LDAP > requests. It also allows to return the key in the same go. True, but it is still required only once per user, in normal course of action you should always get the ipaNTHash back. Even in the race condition case the worst that can happen is that you fail auth once. Given it is not that critical as it can happen only once per user I am not sure it is worth optimizing for this case and create a whole new extended operation for it. > See above. Being able to differentiate cases > - RC4-HMAC is missing, no automatic ipaNTHash generation > - RC4-HMAC is available, generate ipaNTHash > - ipaNTHash is generated via password change > > is worth it because it allows to avoid additional roundtrips from > ipasam and still be able to avoid generator tasks. It still doesn't give you much, there are 2 cases: 1) For users that are supposed to have the ipaNTHash, you will go through this operation *once* in the lifetime of a pre-existing user (new users get ipaNTHash immediately). 2) For users that will never get the ipaNTHash will simply never have it, you only keep repeating this operation and then fail authentication as you won't get back a valid hash, I do not think optimizing this failure case is worth a full extop. The generator task is not strictly necessary, it's an option, you can make the 2 roundtrips in ipasam (I think I will have to relax permissions to allow you to do that as a cifs/fqdn principal) once per user, doesn't look like a big deal. Simo. -- Simo Sorce * Red Hat, Inc * New York From pviktori at redhat.com Wed Jul 11 13:31:18 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 11 Jul 2012 15:31:18 +0200 Subject: [Freeipa-devel] [PATCH] [one-liner] 0069 Fix wrong option name in ipa-managed-entries man page Message-ID: <4FFD8026.8030207@redhat.com> The page said `-y` but the actual option is `-p`. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0069-Fix-wrong-option-name-in-ipa-managed-entries-man-pag.patch Type: text/x-patch Size: 1207 bytes Desc: not available URL: From mkosek at redhat.com Wed Jul 11 13:33:32 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 11 Jul 2012 15:33:32 +0200 Subject: [Freeipa-devel] [PATCH] [one-liner] 0069 Fix wrong option name in ipa-managed-entries man page In-Reply-To: <4FFD8026.8030207@redhat.com> References: <4FFD8026.8030207@redhat.com> Message-ID: <4FFD80AC.1090906@redhat.com> On 07/11/2012 03:31 PM, Petr Viktorin wrote: > The page said `-y` but the actual option is `-p`. > ACK. Pushed to master. Martin From abokovoy at redhat.com Wed Jul 11 13:40:37 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 11 Jul 2012 16:40:37 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1342012841.2599.75.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> Message-ID: <20120711134037.GC9427@redhat.com> On Wed, 11 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-11 at 15:41 +0300, Alexander Bokovoy wrote: >> If users don't have RC4-HMAC key and don't have ipaNTHash set, they >> can't log in into smbd anyway until they change their password. > >Yes the point is that you may have users you do not want to give a >password to. No need to keep retrying to generate a hash. > >> >My idea was that when the ipa trust-add operation is run we execute a >> >magicregen op for the user that run it. Then we can run a process that >> >adds ipaNThash via magicregen for all users we want it to. >> So we get to the same issue of a task run against potentially unbound >> number of users, including replication interaction. >> >> Instead, a scheme with ipasam-based generator would mean we: >> 1. Fetch the user attributes from LDAP >> 2. Notice ipaNTHash is missing and not disabled >> 3. Issue ipaNTHash update request if (2) is true. >> >> Maybe we can turn off ipaNTHash from your pre-mod code if there is no >> RC4-HMAC key and ipaNTHash wasn't set? Password change op will get that >> overriden, of course. Then we can rely on it in (2) above. > >Not sure what you mean by 'turn off ipaNTHash from your pre-mod code'. Set ipaNTHash value to '0', for example. I.e. not 16 bytes and not missing. >> If we decide to use it in ipasam, extended operation will be simpliest >> thing -- contrary to other approaches which would require two LDAP >> requests. It also allows to return the key in the same go. > >True, but it is still required only once per user, in normal course of >action you should always get the ipaNTHash back. Even in the race >condition case the worst that can happen is that you fail auth once. >Given it is not that critical as it can happen only once per user I am >not sure it is worth optimizing for this case and create a whole new >extended operation for it. As per discussion with Simo on IRC, NACK for current approach with LDAP_MOD_REPLACE, NACK for extended operation as well. Please replace LDAP_MOD_REPLACE with LDAP_MOD_ADD detection. smbldap code in smbd uses LDAP_MOD_DELETE/LDAP_MOD_ADD combination as replacement of LDAP_MOD_REPLACE to avoid some nasty bugs with Novell Directory so we have to live with this approach. >> See above. Being able to differentiate cases >> - RC4-HMAC is missing, no automatic ipaNTHash generation >> - RC4-HMAC is available, generate ipaNTHash >> - ipaNTHash is generated via password change >> >> is worth it because it allows to avoid additional roundtrips from >> ipasam and still be able to avoid generator tasks. > >It still doesn't give you much, there are 2 cases: > >1) For users that are supposed to have the ipaNTHash, you will go >through this operation *once* in the lifetime of a pre-existing user >(new users get ipaNTHash immediately). > >2) For users that will never get the ipaNTHash will simply never have >it, you only keep repeating this operation and then fail authentication >as you won't get back a valid hash, I do not think optimizing this >failure case is worth a full extop. My point was to get pre-mod code to set ipaNTHash to invalid (non-16 byte) value to signify that they are 'disabled' for NTLM operations. This way I can get ipaNTHash on user fetch but can locally detect that the user is without password and therefore avoid the whole process. -- / Alexander Bokovoy From pspacek at redhat.com Wed Jul 11 13:54:07 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 11 Jul 2012 15:54:07 +0200 Subject: [Freeipa-devel] [PATCH 0030] Prevent doubled LDAP queries during nonexistent DNS name lookup Message-ID: <4FFD857F.6070708@redhat.com> Hello, this patch fixes bug introduced by CVE-2012-2134 fix (commit cd33194c5a61e98cba53212458cce02b849077ba). From cd33194c5a61e98cba53212458cce02b849077ba up to now each query for nonexistent DNS name results to two (exactly same) LDAP queries. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0030-Prevent-doubled-LDAP-queries-during-nonexistent-DNS-.patch Type: text/x-patch Size: 1330 bytes Desc: not available URL: From pviktori at redhat.com Wed Jul 11 14:45:26 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 11 Jul 2012 16:45:26 +0200 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <4FE848D7.1010004@redhat.com> References: <4FE848D7.1010004@redhat.com> Message-ID: <4FFD9186.6090502@redhat.com> On 06/25/2012 01:17 PM, Petr Viktorin wrote: > The translation files we currently store in Git are full of redundant > information: source strings for untranslated messages, and file locations. > The first causes unnecessarily huge files. The second makes diffs > unreadable: when code is edited and line numbers change, metadata for > all messages shows up as changed. This makes reviewing translation > patches, and merging possible conflicts, hard -- it requires specialized > tools. > > This patch changes the Makefile to strip the unneeded data from .po files. > > Translators using Git must now run msgmerge (or, `make merge-po`) to get > .po files they can work with. Transifex users are unaffected, as the > source .pot file is not changed. > > The i18n tests use file locations for producing nice error reports?. > To make this work as before, the .pot is merged in before validation to > restore comments. > Currently this takes a noticeable amount of time, because polib uses a > particularly na?ve algorithm for merging. I've sent a patch to polib to > resolve this; once that makes it downstream merging will be fast again. > > Updating the translations with the new Makefile will cause a >5MB patch. > I don't want to pollute the mailing list with it, at least until the > Makefile patch is reviewed. It's available > https://github.com/encukou/freeipa/commit/65e2e4.patch > > > https://fedorahosted.org/freeipa/ticket/2435 > Could someone (John?) take some time to look at the patch? I'll be away from office, returning on Tuesday 17th before the beta. It would be nice to have a review when I return. -- Petr? From pviktori at redhat.com Wed Jul 11 14:46:10 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 11 Jul 2012 16:46:10 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <4FFD91B2.5030007@redhat.com> On 07/07/2012 08:45 PM, John Dennis wrote: > The DN work I was doing on master is ready for review and testing. It's > been a long haul and I've been working relentlessly to get this work > completed. I am on PTO for a week starting today (I know bad timing) but > I spent yesterday and my first day of PTO today writing up extensive > documentation for the work so others can start taking a look at it while > I'm gone. The documentation as well as where to find the code can be > found here: > > http://jdennis.fedorapeople.org/dn_summary.html > > The document is long but I felt it was better to provide explanations > for as much as possible. > > I may check in during the week but I'm going to try and discipline > myself not to and take an actual much needed break. > > John > Two more code review points: ipa-adtrust-install uses DN without importing it, that'll fail You've changed API.txt, be sure to also bump IPA_API_VERSION_MINOR in VERSION. And now for the functional testing. I ran through the unit tests, and tested the command-line utilities. I did not test replica stuff (replica-prepare doesn't work, see below) and AD integration (I'd like to ask someone else to do the tests here). I rebased the patch to master, so some of the problems I found may be new regressions. I'm attaching an additional patch I've tested with, which solves some errors I've encountered: ? The lint error mentioned earlier ? ipa-client-install passing a DN object to ipautil.run $ sudo ipa-client-install Discovery was successful! Hostname: vm-149.idm.lab.bos.redhat.com Realm: IDM.LAB.BOS.REDHAT.COM DNS Domain: idm.lab.bos.redhat.com IPA Server: vm-044.idm.lab.bos.redhat.com BaseDN: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com Continue to configure the system with these values? [no]: y User authorized to enroll computers: admin Synchronizing time with KDC... Password for admin at IDM.LAB.BOS.REDHAT.COM: Traceback (most recent call last): File "/sbin/ipa-client-install", line 1763, in sys.exit(main()) File "/sbin/ipa-client-install", line 1749, in main rval = install(options, env, fstore, statestore) File "/sbin/ipa-client-install", line 1473, in install (stdout, stderr, returncode) = run(join_args, raiseonerr=False, env=env, nolog=nolog) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 285, in run close_fds=True, env=env) File "/usr/lib64/python2.7/subprocess.py", line 679, in __init__ errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1249, in _execute_child raise child_exception TypeError: coercing to Unicode: need string or buffer, DN found I also ran into: ? ipa-replica-setup uses removed a LDAPEntry method that got removed when LDAPEntry became a namedtuple $ sudo ipa-replica-prepare vm-$REPLICANUM.idm.lab.bos.redhat.com -p 12345678 --ip-address 10.16.78.28 Preparing replica for vm-028.idm.lab.bos.redhat.com from vm-044.idm.lab.bos.redhat.com preparation of replica failed: 'LDAPEntry' object has no attribute 'getValue' 'LDAPEntry' object has no attribute 'getValue' File "/sbin/ipa-replica-prepare", line 461, in main() File "/sbin/ipa-replica-prepare", line 309, in main dirman_password) File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 89, in enable_replication_version_checking if entry[0].getValue('nsslapd-pluginenabled') == 'off': ? dnsrecord_{del,mod} AAAA unit tests fail, e.g. ipa: ERROR: non-public: AssertionError: Traceback (most recent call last): File "/home/pviktori/freeipa/ipaserver/rpcserver.py", line 332, in wsgi_execute result = self.Command[name](*args, **options) File "/home/pviktori/freeipa/ipalib/frontend.py", line 435, in __call__ ret = self.run(*args, **options) File "/home/pviktori/freeipa/ipalib/frontend.py", line 747, in run return self.execute(*args, **options) File "/home/pviktori/freeipa/ipalib/plugins/dns.py", line 2601, in execute result = super(dnsrecord_del, self).execute(*keys, **options) File "/home/pviktori/freeipa/ipalib/plugins/baseldap.py", line 1350, in execute assert isinstance(dn, DN) AssertionError ipa: INFO: admin at IDM.LAB.BOS.REDHAT.COM: dnsrecord_del(u'dnszone.test', u'testdnsres', arecord=(u'127.0.0.1',), del_all=False, struct ? ipa-compliance still uses strings for DNs (see lines 119, 139). It fails with an AssertionError (which may not be apparent at first because the tool isn't very good at error reporting). Traceback (most recent call last): File "/sbin/ipa-compliance", line 179, in main check_compliance(tmpdir, options.debug) File "/sbin/ipa-compliance", line 121, in check_compliance size_limit = -1) File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 1050, in find_entries assert isinstance(base_dn, DN) AssertionError ? ipa-ldap-updater fails when running plugins. The offending code around updateclient.py:134 is wrong. $ sudo ipa-ldap-updater Directory Manager password: ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO PRE_UPDATE ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO Parsing update file /usr/share/ipa/updates/10-60basev2.update ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO Parsing update file /usr/share/ipa/updates/10-60basev3.update [...] ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO Done ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO Updating existing entry: cn=UPG Definition,cn=Definitions,cn=Managed Entries,cn=etc,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO Done ipa.ipaserver.install.ldapupdate.LDAPUpdate: INFO POST_UPDATE Traceback (most recent call last): File "/sbin/ipa-ldap-updater", line 163, in sys.exit(main()) File "/sbin/ipa-ldap-updater", line 144, in main modified = ld.update(files) File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 879, in update updates = api.Backend.updateclient.update(POST_UPDATE, self.dm_password, self.ldapi, self.live_run) File "/usr/lib/python2.7/site-packages/ipaserver/install/plugins/updateclient.py", line 134, in update if dn not in rdn_count_list[rdn_count]: IndexError: list index out of range ? ipa-nis-manage uses unlocked global DNs. But it works! ? ipa-managed-entries still uses strings for DNs (see line 97), so it can't find the entries it manages (again due to AssertionError). $ sudo ipa-managed-entries -l Directory Manager password: Unable to find managed entries at cn=Definitions,cn=Managed Entries,cn=etc,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: fixup-dn-conversion.patch Type: text/x-patch Size: 1738 bytes Desc: not available URL: From pvoborni at redhat.com Wed Jul 11 15:19:05 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 11 Jul 2012 17:19:05 +0200 Subject: [Freeipa-devel] [PATCH] 167 Add and remove dns per-domain permission in Web UI In-Reply-To: <4FFBC06A.2040307@redhat.com> References: <4FFAC699.4070308@redhat.com> <4FFBC06A.2040307@redhat.com> Message-ID: <4FFD9969.8070202@redhat.com> On 07/10/2012 07:40 AM, Endi Sukma Dewata wrote: > On 7/9/2012 6:55 AM, Petr Vobornik wrote: >> Patch functionality depends on not yet posted pviktori's patch which >> adds error_code (in case of command error) to batch response. >> >> Patch description: >> >> This patch adds support for new per-domain permissions to Web UI. >> >> User with assigned permission (through role,priviledge) can edit DNS >> zone. These permissions can be added/remove by ipa >> dnszone-{add/remove}permission $dnszone command. >> >> For adding/removing of this permission in Web UI new actions in DNS zone >> action list were created. DNS zone object doesn't contain information >> about existance of related permission. Such information is required for >> enabling/disabling of new actions. Web UI has to search for the >> permission to get it. DNS zone facet was modified to use batch command, >> in a same way as user facet, for loading dnszone and the permission at >> the same time - on load. >> >> Batch command has a feature to report all errors. Such behavior is >> unwanted because we expect that permission-show command will fail when >> the permission doesn't exist. Batch command was therefore modified to >> not report commands which has retry attribute set to false. This attr >> was chosen because it has similar purpose in single command execution. >> >> New actions should be enabled only for users with appropriate rights. It >> is not possible to obtain rights for certain action in advance so an >> approximation is used: write right for dns zones' managedby attribute. >> >> https://fedorahosted.org/freeipa/ticket/2851 > > ACK. Pushed to master. (dependency was pushed) > > The code works, feel free to push when the other patch is ready. But if > time allows it might be better to use a checkbox or radio buttons inside > the details page. That way you can see whether the per-domain permission > is enabled without having to click the drop-down list and infer from the > enabled/disabled actions. > Yes we might want to display the state but I would probably just show it as text, maybe with icon - no interactive control. We should also add something similar to user, sudo rules... (you have pointed out this before). -- Petr Vobornik From mkosek at redhat.com Wed Jul 11 15:20:22 2012 From: mkosek at redhat.com (Martin Kosek) Date: Wed, 11 Jul 2012 17:20:22 +0200 Subject: [Freeipa-devel] [PATCH] 286-288 Warn when ID range with incorrect size was created Message-ID: <4FFD99B6.8030800@redhat.com> IPA 3.0 introduced range ID objects in replicated space which specify a range of IDs assigned via DNA plugin. ipa-ldap-updater generates the default ID range which should correspond with IDs assigned to IPA users. However, since correct range size is not known, we should at least warn that a range with invalid size was created so that user can amend it. I created 2 new tickets to add further improve this area: 1) #2918: [doc] Upgrade procedure section should mention ipa-ldap-updater 2) #2919: Improve safety checks in range command To test this patch, you can: 1) Install unpatched IPA server (and you may install replicas too) with custom --idstart and --idmax options where difference is greater then 200000 2) Remove default range with range-del command (will be restored during upgrade) 3) Run RPM upgrade with RPMs built from patched sources - ERROR should now be printed during update stating that a new range was created but its size is not right Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-286-add-range-mod-command.patch Type: text/x-patch Size: 8298 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-287-id-range-warning.patch Type: text/x-patch Size: 5670 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-288-ipa-ldap-updater-errors.patch Type: text/x-patch Size: 1955 bytes Desc: not available URL: From abokovoy at redhat.com Wed Jul 11 15:24:54 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 11 Jul 2012 18:24:54 +0300 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FFD91B2.5030007@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFD91B2.5030007@redhat.com> Message-ID: <20120711152454.GD9427@redhat.com> On Wed, 11 Jul 2012, Petr Viktorin wrote: >On 07/07/2012 08:45 PM, John Dennis wrote: >>The DN work I was doing on master is ready for review and testing. It's >>been a long haul and I've been working relentlessly to get this work >>completed. I am on PTO for a week starting today (I know bad timing) but >>I spent yesterday and my first day of PTO today writing up extensive >>documentation for the work so others can start taking a look at it while >>I'm gone. The documentation as well as where to find the code can be >>found here: >> >>http://jdennis.fedorapeople.org/dn_summary.html >> >>The document is long but I felt it was better to provide explanations >>for as much as possible. >> >>I may check in during the week but I'm going to try and discipline >>myself not to and take an actual much needed break. >> >>John >> > >Two more code review points: >ipa-adtrust-install uses DN without importing it, that'll fail Where? $ git grep DN ipaserver/install/adtrustinstance.py|grep import ipaserver/install/adtrustinstance.py:from ipalib.dn import DN This is in master. -- / Alexander Bokovoy From simo at redhat.com Wed Jul 11 18:11:25 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 11 Jul 2012 14:11:25 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <20120711134037.GC9427@redhat.com> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> Message-ID: <1342030285.2599.82.camel@willson.li.ssimo.org> On Wed, 2012-07-11 at 16:40 +0300, Alexander Bokovoy wrote: > On Wed, 11 Jul 2012, Simo Sorce wrote: > >On Wed, 2012-07-11 at 15:41 +0300, Alexander Bokovoy wrote: > >> If users don't have RC4-HMAC key and don't have ipaNTHash set, they > >> can't log in into smbd anyway until they change their password. > > > >Yes the point is that you may have users you do not want to give a > >password to. No need to keep retrying to generate a hash. > > > >> >My idea was that when the ipa trust-add operation is run we execute a > >> >magicregen op for the user that run it. Then we can run a process that > >> >adds ipaNThash via magicregen for all users we want it to. > >> So we get to the same issue of a task run against potentially unbound > >> number of users, including replication interaction. > >> > >> Instead, a scheme with ipasam-based generator would mean we: > >> 1. Fetch the user attributes from LDAP > >> 2. Notice ipaNTHash is missing and not disabled > >> 3. Issue ipaNTHash update request if (2) is true. > >> > >> Maybe we can turn off ipaNTHash from your pre-mod code if there is no > >> RC4-HMAC key and ipaNTHash wasn't set? Password change op will get that > >> overriden, of course. Then we can rely on it in (2) above. > > > >Not sure what you mean by 'turn off ipaNTHash from your pre-mod code'. > Set ipaNTHash value to '0', for example. I.e. not 16 bytes and not > missing. > > > >> If we decide to use it in ipasam, extended operation will be simpliest > >> thing -- contrary to other approaches which would require two LDAP > >> requests. It also allows to return the key in the same go. > > > >True, but it is still required only once per user, in normal course of > >action you should always get the ipaNTHash back. Even in the race > >condition case the worst that can happen is that you fail auth once. > >Given it is not that critical as it can happen only once per user I am > >not sure it is worth optimizing for this case and create a whole new > >extended operation for it. > As per discussion with Simo on IRC, NACK for current approach with > LDAP_MOD_REPLACE, NACK for extended operation as well. > > Please replace LDAP_MOD_REPLACE with LDAP_MOD_ADD detection. smbldap > code in smbd uses LDAP_MOD_DELETE/LDAP_MOD_ADD combination as > replacement of LDAP_MOD_REPLACE to avoid some nasty bugs with Novell > Directory so we have to live with this approach. Attached patch that changes REPLACE -> ADD > >It still doesn't give you much, there are 2 cases: > > > >1) For users that are supposed to have the ipaNTHash, you will go > >through this operation *once* in the lifetime of a pre-existing user > >(new users get ipaNTHash immediately). > > > >2) For users that will never get the ipaNTHash will simply never have > >it, you only keep repeating this operation and then fail authentication > >as you won't get back a valid hash, I do not think optimizing this > >failure case is worth a full extop. > My point was to get pre-mod code to set ipaNTHash to invalid (non-16 > byte) value to signify that they are 'disabled' for NTLM operations. > This way I can get ipaNTHash on user fetch but can locally detect that > the user is without password and therefore avoid the whole process. Do you still want to do this ? We could store the value 'DISABLED' instead of the hash, but then I'd have to change the password plugin to respect it. If you want that I think we need to open a new bug and treat it as a separate feature. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-494-0003-2-Add-special-modify-op-to-regen-ipaNTHash.patch Type: text/x-patch Size: 7195 bytes Desc: not available URL: From rcritten at redhat.com Wed Jul 11 18:12:55 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 11 Jul 2012 14:12:55 -0400 Subject: [Freeipa-devel] [PATCH] 285 Add automount map/key update permissions In-Reply-To: <4FFC5841.8000704@redhat.com> References: <4FFC5841.8000704@redhat.com> Message-ID: <4FFDC227.7000605@redhat.com> Martin Kosek wrote: > Add missing permissions that can be used to delegate write access > to existing automount maps or keys. > > Since automount key RDN has been changed in the past from "automountkey" > to "description" and there can be LDAP entries with both RDNs, > structure of relevant ACI need to be changed to different scheme. Now, > it rather targets a DN of parent automount map object and uses > targetfilter to limit the target to automount key objects only. > > https://fedorahosted.org/freeipa/ticket/2687 > > ACK, pushed to master From rcritten at redhat.com Wed Jul 11 19:27:39 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 11 Jul 2012 15:27:39 -0400 Subject: [Freeipa-devel] [PATCH] 286-288 Warn when ID range with incorrect size was created In-Reply-To: <4FFD99B6.8030800@redhat.com> References: <4FFD99B6.8030800@redhat.com> Message-ID: <4FFDD3AB.6080903@redhat.com> Martin Kosek wrote: > IPA 3.0 introduced range ID objects in replicated space which specify > a range of IDs assigned via DNA plugin. ipa-ldap-updater generates the > default ID range which should correspond with IDs assigned to IPA > users. > > However, since correct range size is not known, we should at least > warn that a range with invalid size was created so that user can > amend it. > > > I created 2 new tickets to add further improve this area: > > 1) #2918: [doc] Upgrade procedure section should mention ipa-ldap-updater > 2) #2919: Improve safety checks in range command > > > To test this patch, you can: > 1) Install unpatched IPA server (and you may install replicas too) with custom > --idstart and --idmax options where difference is greater then 200000 > 2) Remove default range with range-del command (will be restored during upgrade) > 3) Run RPM upgrade with RPMs built from patched sources - ERROR should now be > printed during update stating that a new range was created but its size is not > right I don't understand step 2, why would someone remove their range before upgrading? I installed with a 50k range, didn't remove it, then upgraded with no warning. I deleted the range and re-installed the packages again, still no warning but a new 200k range was created for me. rob From mkosek at redhat.com Thu Jul 12 05:46:04 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 12 Jul 2012 07:46:04 +0200 Subject: [Freeipa-devel] [PATCH] 286-288 Warn when ID range with incorrect size was created In-Reply-To: <4FFDD3AB.6080903@redhat.com> References: <4FFD99B6.8030800@redhat.com> <4FFDD3AB.6080903@redhat.com> Message-ID: <4FFE649C.3040908@redhat.com> On 07/11/2012 09:27 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> IPA 3.0 introduced range ID objects in replicated space which specify >> a range of IDs assigned via DNA plugin. ipa-ldap-updater generates the >> default ID range which should correspond with IDs assigned to IPA >> users. >> >> However, since correct range size is not known, we should at least >> warn that a range with invalid size was created so that user can >> amend it. >> >> >> I created 2 new tickets to add further improve this area: >> >> 1) #2918: [doc] Upgrade procedure section should mention ipa-ldap-updater >> 2) #2919: Improve safety checks in range command >> >> >> To test this patch, you can: >> 1) Install unpatched IPA server (and you may install replicas too) with custom >> --idstart and --idmax options where difference is greater then 200000 >> 2) Remove default range with range-del command (will be restored during upgrade) >> 3) Run RPM upgrade with RPMs built from patched sources - ERROR should now be >> printed during update stating that a new range was created but its size is not >> right > > I don't understand step 2, why would someone remove their range before upgrading? > > I installed with a 50k range, didn't remove it, then upgraded with no warning. > I deleted the range and re-installed the packages again, still no warning but a > new 200k range was created for me. > > rob The step 2 is artificial and is only done to force the default_range update plugin to create/restore the default IPA range. The plugin would just be skipped otherwise. We can only detect ranges larger than 200k - judging just from the number of free IDs. Thus, 50k range will pass without any warning or error. If you create a bigger range (this can be detected unless you deplete all IDs below 200k mark), you will receive the warning. All this procedure will not handle all situations ATM, its just heuristics to cover most cases... Martin From abokovoy at redhat.com Thu Jul 12 07:46:32 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 12 Jul 2012 10:46:32 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1342030285.2599.82.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> Message-ID: <20120712074632.GE9427@redhat.com> On Wed, 11 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-11 at 16:40 +0300, Alexander Bokovoy wrote: >> On Wed, 11 Jul 2012, Simo Sorce wrote: >> >On Wed, 2012-07-11 at 15:41 +0300, Alexander Bokovoy wrote: >> >> If users don't have RC4-HMAC key and don't have ipaNTHash set, they >> >> can't log in into smbd anyway until they change their password. >> > >> >Yes the point is that you may have users you do not want to give a >> >password to. No need to keep retrying to generate a hash. >> > >> >> >My idea was that when the ipa trust-add operation is run we execute a >> >> >magicregen op for the user that run it. Then we can run a process that >> >> >adds ipaNThash via magicregen for all users we want it to. >> >> So we get to the same issue of a task run against potentially unbound >> >> number of users, including replication interaction. >> >> >> >> Instead, a scheme with ipasam-based generator would mean we: >> >> 1. Fetch the user attributes from LDAP >> >> 2. Notice ipaNTHash is missing and not disabled >> >> 3. Issue ipaNTHash update request if (2) is true. >> >> >> >> Maybe we can turn off ipaNTHash from your pre-mod code if there is no >> >> RC4-HMAC key and ipaNTHash wasn't set? Password change op will get that >> >> overriden, of course. Then we can rely on it in (2) above. >> > >> >Not sure what you mean by 'turn off ipaNTHash from your pre-mod code'. >> Set ipaNTHash value to '0', for example. I.e. not 16 bytes and not >> missing. >> >> >> >> If we decide to use it in ipasam, extended operation will be simpliest >> >> thing -- contrary to other approaches which would require two LDAP >> >> requests. It also allows to return the key in the same go. >> > >> >True, but it is still required only once per user, in normal course of >> >action you should always get the ipaNTHash back. Even in the race >> >condition case the worst that can happen is that you fail auth once. >> >Given it is not that critical as it can happen only once per user I am >> >not sure it is worth optimizing for this case and create a whole new >> >extended operation for it. >> As per discussion with Simo on IRC, NACK for current approach with >> LDAP_MOD_REPLACE, NACK for extended operation as well. >> >> Please replace LDAP_MOD_REPLACE with LDAP_MOD_ADD detection. smbldap >> code in smbd uses LDAP_MOD_DELETE/LDAP_MOD_ADD combination as >> replacement of LDAP_MOD_REPLACE to avoid some nasty bugs with Novell >> Directory so we have to live with this approach. > >Attached patch that changes REPLACE -> ADD > >> >It still doesn't give you much, there are 2 cases: >> > >> >1) For users that are supposed to have the ipaNTHash, you will go >> >through this operation *once* in the lifetime of a pre-existing user >> >(new users get ipaNTHash immediately). >> > >> >2) For users that will never get the ipaNTHash will simply never have >> >it, you only keep repeating this operation and then fail authentication >> >as you won't get back a valid hash, I do not think optimizing this >> >failure case is worth a full extop. >> My point was to get pre-mod code to set ipaNTHash to invalid (non-16 >> byte) value to signify that they are 'disabled' for NTLM operations. >> This way I can get ipaNTHash on user fetch but can locally detect that >> the user is without password and therefore avoid the whole process. > >Do you still want to do this ? >We could store the value 'DISABLED' instead of the hash, but then I'd >have to change the password plugin to respect it. If you want that I >think we need to open a new bug and treat it as a separate feature. Yes, I think it could be good optimization. I've made following ticket: https://fedorahosted.org/freeipa/ticket/2921 -- / Alexander Bokovoy From abokovoy at redhat.com Thu Jul 12 07:48:40 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 12 Jul 2012 10:48:40 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1342030285.2599.82.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> Message-ID: <20120712074840.GF9427@redhat.com> On Wed, 11 Jul 2012, Simo Sorce wrote: >From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001 >From: Simo Sorce >Date: Fri, 6 Jul 2012 16:18:29 -0400 >Subject: [PATCH] Add special modify op to regen ipaNTHash > >The NT Hash is the same thing as the RC4-HMAC key, so we add a function to >extract it from krb5 keys if they are available to avoid forcing a password >change when configuring trust relationships. >--- > .../ipa-pwd-extop/ipapwd_prepost.c | 147 +++++++++++++++++++- > 1 file changed, 144 insertions(+), 3 deletions(-) > >diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >index deae6477772f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d 100644 >--- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >+++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >@@ -41,7 +41,12 @@ > # include > #endif > >-#define _XOPEN_SOURCE /* strptime needs this */ >+/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD >+ * _GNU_SOURCE imply both, and we use it elsewhere, so use this */ >+#ifndef _GNU_SOURCE >+#define _GNU_SOURCE 1 >+#endif >+ > #include > #include > #include >@@ -53,6 +58,7 @@ > #include > #include > #include >+#include > > #include "ipapwd.h" > #include "util.h" >@@ -379,6 +385,12 @@ done: > return 0; > } > >+#define NTHASH_REGEN_VAL "MagicRegen" >+#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL) >+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, >+ char *dn, struct slapi_entry *entry, >+ struct ipapwd_krbcfg *krbcfg); >+ > /* PRE MOD Operation: > * Gets the clean text password (fail the operation if the password came > * pre-hashed, unless this is a replicated operation). >@@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > int has_krb_keys = 0; > int has_history = 0; > int gen_krb_keys = 0; >+ int is_magic_regen = 0; > int ret, rc; > > LOG_TRACE( "=>\n"); >@@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > default: > break; > } >+ } else if (slapi_attr_types_equivalent(lmod->mod_type, "ipaNTHash")) { >+ /* check op filtering out LDAP_MOD_BVALUES */ >+ switch (lmod->mod_op & 0x0f) { >+ case LDAP_MOD_REPLACE: This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD. >+ if (!lmod->mod_bvalues || >+ !lmod->mod_bvalues[0]) { >+ rc = LDAP_OPERATIONS_ERROR; >+ goto done; >+ } >+ bv = lmod->mod_bvalues[0]; >+ if ((bv->bv_len >= NTHASH_REGEN_LEN -1) && >+ (bv->bv_len <= NTHASH_REGEN_LEN) && >+ (strncmp(NTHASH_REGEN_VAL, >+ bv->bv_val, bv->bv_len) == 0)) { >+ is_magic_regen = 1; >+ /* make sure the database will later ignore this mod */ >+ slapi_mods_remove(smods); >+ } >+ default: >+ break; >+ } > } else if (slapi_attr_types_equivalent(lmod->mod_type, > "unhashed#user#password")) { > /* we check for unahsehd password here so that we are sure to >@@ -472,8 +506,9 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > lmod = slapi_mods_get_next_mod(smods); > } > >- /* If userPassword is not modified we are done here */ >- if (! is_pwd_op) { >+ /* If userPassword is not modified check if this is a request to generate >+ * NT hashes otherwise we are done here */ >+ if (!is_pwd_op && !is_magic_regen) { > rc = LDAP_SUCCESS; > goto done; > } >@@ -522,6 +557,22 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > goto done; > } > >+ if (!is_pwd_op) { >+ /* This may be a magic op to ask us to generate the NT hashes */ >+ if (is_magic_regen) { >+ /* Make sense to call only if this entry has krb keys to source >+ * the nthash from */ >+ if (is_krb) { >+ rc = ipapwd_regen_nthash(pb, smods, dn, e, krbcfg); >+ } else { >+ rc = LDAP_UNWILLING_TO_PERFORM; >+ } >+ } else { >+ rc = LDAP_OPERATIONS_ERROR; >+ } >+ goto done; >+ } >+ > /* run through the mods again and adjust flags if operations affect them */ > lmod = slapi_mods_get_first_mod(smods); > while (lmod) { >@@ -831,6 +882,96 @@ done: > return 0; > } > >+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, >+ char *dn, struct slapi_entry *entry, >+ struct ipapwd_krbcfg *krbcfg) >+{ >+ Slapi_Attr *attr; >+ Slapi_Value *value; >+ const struct berval *val; >+ struct berval *ntvals[2] = { NULL, NULL }; >+ struct berval bval; >+ krb5_key_data *keys; >+ int num_keys; >+ int mkvno; >+ int ret; >+ int i; >+ >+ ret = slapi_entry_attr_find(entry, "ipaNTHash", &attr); >+ if (ret == 0) { >+ /* We refuse to regen if there is already a value */ >+ return LDAP_CONSTRAINT_VIOLATION; >+ } >+ >+ /* ok let's see if we can find the RC4 hash in the keys */ >+ ret = slapi_entry_attr_find(entry, "krbPrincipalKey", &attr); >+ if (ret) { >+ return LDAP_UNWILLING_TO_PERFORM; >+ } >+ >+ ret = slapi_attr_first_value(attr, &value); >+ if (ret) { >+ return LDAP_OPERATIONS_ERROR; >+ } >+ >+ val = slapi_value_get_berval(value); >+ if (!val) { >+ return LDAP_OPERATIONS_ERROR; >+ } >+ >+ ret = ber_decode_krb5_key_data((struct berval *)val, >+ &mkvno, &num_keys, &keys); >+ if (ret) { >+ return LDAP_OPERATIONS_ERROR; >+ } >+ >+ ret = LDAP_UNWILLING_TO_PERFORM; >+ >+ for (i = 0; i < num_keys; i++) { >+ char nthash[16]; >+ krb5_enc_data cipher; >+ krb5_data plain; >+ krb5_int16 t; >+ >+ if (keys[i].key_data_type[0] != ENCTYPE_ARCFOUR_HMAC) { >+ continue; >+ } >+ >+ memcpy(&t, keys[i].key_data_contents[0], 2); >+ plain.length = le16toh(t); >+ if (plain.length != 16) { >+ continue; >+ } >+ plain.data = nthash; >+ >+ memset(&cipher, 0, sizeof(krb5_enc_data)); >+ cipher.enctype = krbcfg->kmkey->enctype; >+ cipher.ciphertext.length = keys[i].key_data_length[0] - 2; >+ cipher.ciphertext.data = ((char *)keys[i].key_data_contents[0]) + 2; >+ >+ ret = krb5_c_decrypt(krbcfg->krbctx, krbcfg->kmkey, >+ 0, NULL, &cipher, &plain); >+ if (ret) { >+ ret = LDAP_OPERATIONS_ERROR; >+ break; >+ } >+ >+ bval.bv_val = nthash; >+ bval.bv_len = 16; >+ ntvals[0] = &bval; >+ >+ slapi_mods_add_modbvps(smods, LDAP_MOD_REPLACE, >+ "ipaNTHash", ntvals); >+ >+ ret = LDAP_SUCCESS; >+ break; >+ } >+ >+ ipa_krb5_free_key_data(keys, num_keys); >+ >+ return ret; >+} >+ > static int ipapwd_post_op(Slapi_PBlock *pb) > { > void *op; >-- >1.7.10.4 > -- / Alexander Bokovoy From mkosek at redhat.com Thu Jul 12 09:45:36 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 12 Jul 2012 11:45:36 +0200 Subject: [Freeipa-devel] [PATCH] Adding exit status 3 & 4 to ipa-client-install man page Message-ID: <4FFE9CC0.3000902@redhat.com> ACK for shank's patch for ipa-client-install man page (attached). Pushed to master. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-shanks-0001-client-man.patch Type: text/x-patch Size: 834 bytes Desc: not available URL: From pspacek at redhat.com Thu Jul 12 09:58:47 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 12 Jul 2012 11:58:47 +0200 Subject: [Freeipa-devel] [PATCH 0020] Separate LDAP result from LDAP connection, fix deadlock. In-Reply-To: <20120515123139.GA2688@redhat.com> References: <4FA7C4C3.7020202@redhat.com> <20120511102654.GA2785@redhat.com> <4FB11A5A.8000804@redhat.com> <20120515123139.GA2688@redhat.com> Message-ID: <4FFE9FD7.9010209@redhat.com> On 05/15/2012 02:32 PM, Adam Tkac wrote: > On Mon, May 14, 2012 at 04:44:42PM +0200, Petr Spacek wrote: >> On 05/11/2012 12:26 PM, Adam Tkac wrote: >>> On Mon, May 07, 2012 at 02:49:07PM +0200, Petr Spacek wrote: >>>> Hello, >>>> >>>> this patch fixes https://fedorahosted.org/bind-dyndb-ldap/ticket/66: >>>> Plugin deadlocks during new zone load when connections == 1. >>>> >>>> It fixes structural problem, when LDAP query result was tied with >>>> LDAP connection up. It wasn't possible to release connection and >>>> work with query result after that. >>>> Described deadlock is consequence of this problematic design. >>>> >>>> Now LDAP connection is separated from LDAP result. Next planed patch >>>> will avoid "manual" connection management, so possibility of >>>> deadlock should be next to zero. >>>> >>>> Petr^2 Spacek >>> >>> Hello Peter, >>> >>> good work, please check my comments below. >>> >>> Regards, Adam >>> >>>> From 8ee1fd607531ef71369e99c9228456baea45b65d Mon Sep 17 00:00:00 2001 >>>> From: Petr Spacek >>>> Date: Mon, 7 May 2012 12:51:09 +0200 >>>> Subject: [PATCH] Separate LDAP result from LDAP connection, fix deadlock. >>>> https://fedorahosted.org/bind-dyndb-ldap/ticket/66 >>>> Signed-off-by: Petr Spacek >> >> Hello Adam, >> >> thanks for ideas/improvements! >> >> Reworked patch is attached. I did all proposed changes except this one: >> >> @ ldap_psearch_watcher: >>>> restart: >> (... snip ...) >>>> soft_err: >>>> - >>>> - ldap_msgfree(conn->result); >>>> - ldap_entrylist_destroy(conn->mctx, >>>> - &conn->ldap_entries); >>>> + ; >>> >>> Empty label "soft_err:" is useless, please remove it and use "continue;" on >>> appropriate places; >> >> I think "continue" in this place can lead to memory leak, so I >> removed soft_err by other way. > > Thanks for the patch, now it looks fine to me, except that it doesn't apply on > the current master: > > [atkac at drtic bind-dyndb-ldap]$ git am ../bind-dyndb-ldap-pspacek-0020-2-Separate-LDAP-result-from-LDAP-connection-fix-deadlo.patch > Applying: Separate LDAP result from LDAP connection, fix deadlock. https://fedorahosted.org/bind-dyndb-ldap/ticket/66 Signed-off-by: Petr Spacek > error: patch failed: src/ldap_helper.c:271 > error: src/ldap_helper.c: patch does not apply > Patch failed at 0001 Separate LDAP result from LDAP connection, fix deadlock. https://fedorahosted.org/bind-dyndb-ldap/ticket/66 Signed-off-by: Petr Spacek > When you have resolved this problem run "git am --resolved". > If you would prefer to skip this patch, instead run "git am --skip". > To restore the original branch and stop patching run "git am --abort". > > Please rebase the patch and then push it, you don't have to resend it here. > > Regards, Adam Finally, I rebased the patch and pushed it to the master. Sorry for delay, I forgot to this ticket completely. Rebased version is attached. https://fedorahosted.org/bind-dyndb-ldap/changeset/88dcade344af6e71503b85c4d2630343dbf7d7c0 Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0020-3-Separate-LDAP-result-from-LDAP-connection-fix-deadlo.patch Type: text/x-patch Size: 20351 bytes Desc: not available URL: From mkosek at redhat.com Thu Jul 12 10:01:48 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 12 Jul 2012 12:01:48 +0200 Subject: [Freeipa-devel] [PATCH] 286-288 Warn when ID range with incorrect size was created In-Reply-To: <4FFE649C.3040908@redhat.com> References: <4FFD99B6.8030800@redhat.com> <4FFDD3AB.6080903@redhat.com> <4FFE649C.3040908@redhat.com> Message-ID: <4FFEA08C.3000404@redhat.com> On 07/12/2012 07:46 AM, Martin Kosek wrote: > On 07/11/2012 09:27 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> IPA 3.0 introduced range ID objects in replicated space which specify >>> a range of IDs assigned via DNA plugin. ipa-ldap-updater generates the >>> default ID range which should correspond with IDs assigned to IPA >>> users. >>> >>> However, since correct range size is not known, we should at least >>> warn that a range with invalid size was created so that user can >>> amend it. >>> >>> >>> I created 2 new tickets to add further improve this area: >>> >>> 1) #2918: [doc] Upgrade procedure section should mention ipa-ldap-updater >>> 2) #2919: Improve safety checks in range command >>> >>> >>> To test this patch, you can: >>> 1) Install unpatched IPA server (and you may install replicas too) with custom >>> --idstart and --idmax options where difference is greater then 200000 >>> 2) Remove default range with range-del command (will be restored during upgrade) >>> 3) Run RPM upgrade with RPMs built from patched sources - ERROR should now be >>> printed during update stating that a new range was created but its size is not >>> right >> >> I don't understand step 2, why would someone remove their range before upgrading? >> >> I installed with a 50k range, didn't remove it, then upgraded with no warning. >> I deleted the range and re-installed the packages again, still no warning but a >> new 200k range was created for me. >> >> rob > > The step 2 is artificial and is only done to force the default_range update > plugin to create/restore the default IPA range. The plugin would just be > skipped otherwise. > > We can only detect ranges larger than 200k - judging just from the number of > free IDs. Thus, 50k range will pass without any warning or error. If you create > a bigger range (this can be detected unless you deplete all IDs below 200k > mark), you will receive the warning. All this procedure will not handle all > situations ATM, its just heuristics to cover most cases... > > Martin Sending an updated patch with 2 small changes: 1) Console error formatting was changed similar to ipa-client-install 2) ipa-ldap-updater does not print information message when IPA is not configured to stderr so that rpm update output stays clean when updating rpms in machine without IPA installed This is the output of RPM with the new patch set: # ipa range-del IDM.LAB.BOS.REDHAT.COM_id_range -------------------------------------------------- Deleted ID range "IDM.LAB.BOS.REDHAT.COM_id_range" -------------------------------------------------- # rpm -Uvh --force freeipa-* Preparing... ########################################### [100%] 1:freeipa-python ########################################### [ 14%] 2:freeipa-client ########################################### [ 29%] 3:freeipa-admintools ########################################### [ 43%] 4:freeipa-server ########################################### [ 57%] 5:freeipa-server-selinux ########################################### [ 71%] 6:freeipa-server-trust-ad########################################### [ 86%] 7:freeipa-debuginfo ########################################### [100%] ERROR: default_range: could not verify default ID range size Please use the following command to set correct ID range size $ ipa range-mod IDM.LAB.BOS.REDHAT.COM_id_range --range-size=RANGE_SIZE RANGE_SIZE may be computed from --idstart and --idmax options used during IPA server installation: RANGE_SIZE = (--idmax) - (--idstart) + 1 Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-286-2-add-range-mod-command.patch Type: text/x-patch Size: 8298 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-287-2-id-range-warning.patch Type: text/x-patch Size: 5669 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-288-2-ipa-ldap-updater-errors.patch Type: text/x-patch Size: 3065 bytes Desc: not available URL: From simo at redhat.com Thu Jul 12 12:08:02 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 12 Jul 2012 08:08:02 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <20120712074840.GF9427@redhat.com> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> <20120712074840.GF9427@redhat.com> Message-ID: <1342094882.2599.973.camel@willson.li.ssimo.org> On Thu, 2012-07-12 at 10:48 +0300, Alexander Bokovoy wrote: > On Wed, 11 Jul 2012, Simo Sorce wrote: > >From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001 > >From: Simo Sorce > >Date: Fri, 6 Jul 2012 16:18:29 -0400 > >Subject: [PATCH] Add special modify op to regen ipaNTHash > > > >The NT Hash is the same thing as the RC4-HMAC key, so we add a function to > >extract it from krb5 keys if they are available to avoid forcing a password > >change when configuring trust relationships. > >--- > > .../ipa-pwd-extop/ipapwd_prepost.c | 147 +++++++++++++++++++- > > 1 file changed, 144 insertions(+), 3 deletions(-) > > > >diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > >index deae6477772f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d 100644 > >--- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > >+++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > >@@ -41,7 +41,12 @@ > > # include > > #endif > > > >-#define _XOPEN_SOURCE /* strptime needs this */ > >+/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD > >+ * _GNU_SOURCE imply both, and we use it elsewhere, so use this */ > >+#ifndef _GNU_SOURCE > >+#define _GNU_SOURCE 1 > >+#endif > >+ > > #include > > #include > > #include > >@@ -53,6 +58,7 @@ > > #include > > #include > > #include > >+#include > > > > #include "ipapwd.h" > > #include "util.h" > >@@ -379,6 +385,12 @@ done: > > return 0; > > } > > > >+#define NTHASH_REGEN_VAL "MagicRegen" > >+#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL) > >+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, > >+ char *dn, struct slapi_entry *entry, > >+ struct ipapwd_krbcfg *krbcfg); > >+ > > /* PRE MOD Operation: > > * Gets the clean text password (fail the operation if the password came > > * pre-hashed, unless this is a replicated operation). > >@@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > > int has_krb_keys = 0; > > int has_history = 0; > > int gen_krb_keys = 0; > >+ int is_magic_regen = 0; > > int ret, rc; > > > > LOG_TRACE( "=>\n"); > >@@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > > default: > > break; > > } > >+ } else if (slapi_attr_types_equivalent(lmod->mod_type, "ipaNTHash")) { > >+ /* check op filtering out LDAP_MOD_BVALUES */ > >+ switch (lmod->mod_op & 0x0f) { > >+ case LDAP_MOD_REPLACE: > This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD. This is because I resent the old patch :( Hopefully the correct patch is now attached. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-494-0003-3-Add-special-modify-op-to-regen-ipaNTHash.patch Type: text/x-patch Size: 7152 bytes Desc: not available URL: From abokovoy at redhat.com Thu Jul 12 12:42:23 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 12 Jul 2012 15:42:23 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1342094882.2599.973.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> <20120712074840.GF9427@redhat.com> <1342094882.2599.973.camel@willson.li.ssimo.org> Message-ID: <20120712124223.GH9427@redhat.com> On Thu, 12 Jul 2012, Simo Sorce wrote: >On Thu, 2012-07-12 at 10:48 +0300, Alexander Bokovoy wrote: >> On Wed, 11 Jul 2012, Simo Sorce wrote: >> >From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001 >> >From: Simo Sorce >> >Date: Fri, 6 Jul 2012 16:18:29 -0400 >> >Subject: [PATCH] Add special modify op to regen ipaNTHash >> > >> >The NT Hash is the same thing as the RC4-HMAC key, so we add a function to >> >extract it from krb5 keys if they are available to avoid forcing a password >> >change when configuring trust relationships. >> >--- >> > .../ipa-pwd-extop/ipapwd_prepost.c | 147 +++++++++++++++++++- >> > 1 file changed, 144 insertions(+), 3 deletions(-) >> > >> >diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >> >index deae6477772f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d 100644 >> >--- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >> >+++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >> >@@ -41,7 +41,12 @@ >> > # include >> > #endif >> > >> >-#define _XOPEN_SOURCE /* strptime needs this */ >> >+/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD >> >+ * _GNU_SOURCE imply both, and we use it elsewhere, so use this */ >> >+#ifndef _GNU_SOURCE >> >+#define _GNU_SOURCE 1 >> >+#endif >> >+ >> > #include >> > #include >> > #include >> >@@ -53,6 +58,7 @@ >> > #include >> > #include >> > #include >> >+#include >> > >> > #include "ipapwd.h" >> > #include "util.h" >> >@@ -379,6 +385,12 @@ done: >> > return 0; >> > } >> > >> >+#define NTHASH_REGEN_VAL "MagicRegen" >> >+#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL) >> >+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, >> >+ char *dn, struct slapi_entry *entry, >> >+ struct ipapwd_krbcfg *krbcfg); >> >+ >> > /* PRE MOD Operation: >> > * Gets the clean text password (fail the operation if the password came >> > * pre-hashed, unless this is a replicated operation). >> >@@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) >> > int has_krb_keys = 0; >> > int has_history = 0; >> > int gen_krb_keys = 0; >> >+ int is_magic_regen = 0; >> > int ret, rc; >> > >> > LOG_TRACE( "=>\n"); >> >@@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) >> > default: >> > break; >> > } >> >+ } else if (slapi_attr_types_equivalent(lmod->mod_type, "ipaNTHash")) { >> >+ /* check op filtering out LDAP_MOD_BVALUES */ >> >+ switch (lmod->mod_op & 0x0f) { >> >+ case LDAP_MOD_REPLACE: >> This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD. > >This is because I resent the old patch :( > >Hopefully the correct patch is now attached. Yes, now it is updated, thanks. I'm going to experiment a bit with these patches, adding ipasam responder to test them. -- / Alexander Bokovoy From pvoborni at redhat.com Thu Jul 12 13:10:26 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 12 Jul 2012 15:10:26 +0200 Subject: [Freeipa-devel] [PATCH] 170 Differentiation of widget type and text_widget input type Message-ID: <4FFECCC2.7090201@redhat.com> There was a clash of 'type' attribute in widget's spec. Usually 'type' is used for telling a builder which field and widget to build. Text widget used this attribute also for definion of html input type. It was problematic for some special widgets, which defined own field and used text_widget, like service_type or dnszone_name. In those and possibly other cases it used widget type for specifying input type which lead to execution error in Internet Explorer. Firefox and Chrome took it. This patch is changing text_widget's 'type' to 'input_type' which removes the collision and hence fixes the problem. https://fedorahosted.org/freeipa/ticket/2806 and half of: https://fedorahosted.org/freeipa/ticket/2834 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0170-Differentiation-of-widget-type-and-text_widget-input.patch Type: text/x-patch Size: 2105 bytes Desc: not available URL: From mkosek at redhat.com Thu Jul 12 13:12:09 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 12 Jul 2012 15:12:09 +0200 Subject: [Freeipa-devel] [PATCH] 289 Fix ipa-managed-entries man page typo Message-ID: <4FFECD29.7020906@redhat.com> Extra new line in .TH section of the man page caused invalid wrapping. Pushed to master as one-liner. -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-289-fix-ipa-managed-entries-man-page-typo.patch Type: text/x-patch Size: 1016 bytes Desc: not available URL: From mkosek at redhat.com Thu Jul 12 14:25:17 2012 From: mkosek at redhat.com (Martin Kosek) Date: Thu, 12 Jul 2012 16:25:17 +0200 Subject: [Freeipa-devel] [PATCH] 281 Enable SOA serial autoincrement In-Reply-To: <4FF14377.80609@redhat.com> References: <4FEC7314.9030508@redhat.com> <4FEDFB47.40504@redhat.com> <4FF14377.80609@redhat.com> Message-ID: <4FFEDE4D.5020007@redhat.com> On 07/02/2012 08:45 AM, Martin Kosek wrote: > On 06/29/2012 09:00 PM, Rob Crittenden wrote: >> Martin Kosek wrote: >>> This patch enables currently developed SOA serial autoincrement feature in >>> bind-dyndb-ldap. The patch may be updated if any assumptions about this feature >>> are changed (or somebody finds a bug). >>> >>> --- >>> >>> SOA serial autoincrement is a requirement for major DNS features, >>> e.g. zone transfers or DNSSEC. Enable it by default in named.conf >>> both for new and upgraded installations. Name of the bind-dyndb-ldap >>> option is "serial_autoincrement". >>> >>>> From now on, idnsSOAserial attribute also has to be put to >>> replication agreement exclude list as serial will be incremented >>> on each DNS server separately and won't be shared. Exclude list >>> has to be updated both for new replication agreements and the >>> current ones. >>> >>> https://fedorahosted.org/freeipa/ticket/2554 >> >> What version of bind/bind-dyndb-ldap is needed for serial_autoincrement? >> >> rob > > Such version is not ready yet, there is only a semi-working patch from Petr > Spacek on freeipa-devel list. > > When a working version of bind-dyndb-ldap package with working > serial_autoincrement feature, it should be enough to simply bump package > version in bind-dyndb-ldap (that's why I tagged this patch as [WIP]). > > But otherwise, this patch is reviewable, it should prepare our install tools > for the new feature, turn it on in named.conf on upgrades and also update > replication agreements to not replicate SOA serial from now on. > > Martin Sending a rebased and updated patch with few more fixes: 1) Minimum number of connections has been rised to 4 to cover the most recent requirements for bind-dyndb-ldap's serial_automember feature 2) ipa-upgradeconfig named.conf has been fixed to not crash when the updated options are not in the file I think that we can choose to push this patch earlier before bind-dyndb-ldap with serial_automember released. We just need to make sure this patch sets serial_automember option in named.conf correctly + does the right thing with replication agreement exclude list update. Later on, we would just need to bump bind-dyndb-ldap version in our spec file when that's released. Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-281-2-enable-soa-serial-autoincrement.patch Type: text/x-patch Size: 22965 bytes Desc: not available URL: From pspacek at redhat.com Thu Jul 12 15:18:35 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 12 Jul 2012 17:18:35 +0200 Subject: [Freeipa-devel] [PATCH 0031] Prevent crashes in ldap_pool_*() function family Message-ID: <4FFEEACB.5010603@redhat.com> Hello, this patch fixes occasional crashes caused by incorrect error handling in ldap_pool_*() functions. https://fedorahosted.org/bind-dyndb-ldap/ticket/84 It can be caused by memory allocation error OR timeout during connection establishing phase. To trigger this problem first connection has to be established properly and some other connection has to fail. It is not enough to timeout at first connection/try, that case was handled properly. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0031-Prevent-crashes-in-ldap_pool_-function-family.patch Type: text/x-patch Size: 2351 bytes Desc: not available URL: From simo at redhat.com Thu Jul 12 20:30:09 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 12 Jul 2012 16:30:09 -0400 Subject: [Freeipa-devel] [PATCHES] 495 Fix ipa-replica-manage issues Message-ID: <1342125009.2718.26.camel@willson.li.ssimo.org> These 2 patches fix issues found with ipa-replica-manage and connect/disconnect commands. Fixes ticket #2925 Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-495-0001-1-Fix-safety-checks-to-prevent-orphaning-replicas.patch Type: text/x-patch Size: 1360 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-495-0002-1-Fix-detection-of-deleted-masters.patch Type: text/x-patch Size: 3191 bytes Desc: not available URL: From mkosek at redhat.com Fri Jul 13 10:40:58 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 13 Jul 2012 12:40:58 +0200 Subject: [Freeipa-devel] [PATCH] 290 Enforce CNAME constrains for DNS commands Message-ID: <4FFFFB3A.2010806@redhat.com> RFC 1912 states that no record (besides PTR) is allowed to coexist with any other record type. When BIND detects this situation, it refuses to load such records. Enforce the constrain for dnsrecord-mod and dnsrecord-add commands. https://fedorahosted.org/freeipa/ticket/2601 -- Martin Kosek Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-290-enforce-cname-constrains-for-dns-commands.patch Type: text/x-patch Size: 11613 bytes Desc: not available URL: From KECKEL at de.ibm.com Fri Jul 13 10:45:24 2012 From: KECKEL at de.ibm.com (Klaus Eckel) Date: Fri, 13 Jul 2012 12:45:24 +0200 Subject: [Freeipa-devel] Multitenancy in FreeIPA 3.0x Message-ID: Hey All , where can to try an Multitenancy IPA 3.0 system or change , config my hope .. one ldap-system which sub system my hope .. for only Tenant one KDC over other port .. ( same linux system ) Can i try that ! Klaus Best Regards, Klaus Eckel, UNIX Consultant HPC (AIX,Linux) GPFS, BIA, SAP ITS/STG (SSIS) Server, Storage & Data Infrastructure Services IBM Deutschland GmbH Laatzener str, 1 30539 Hannover Germany Email: keckel at de.ibm.com Phone: +49-(0)52319489906 Handy: +49 (0)170 6323416 Visit the IBM Deutschland ITS Pages. IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Erich Clementi Gesch?ftsf?hrung: Martin Jetter (Vorsitzender), Reinhard Reschke, Dieter Scholz, Klaus Lintelmann, Michael Diemer, Martina Koederitz Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvoborni at redhat.com Fri Jul 13 11:09:00 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 13 Jul 2012 13:09:00 +0200 Subject: [Freeipa-devel] DHCP support - Request for review In-Reply-To: <4FEB0B64.7020604@firstyear.id.au> References: <4FEB0B64.7020604@firstyear.id.au> Message-ID: <500001CC.1030307@redhat.com> On 06/27/2012 03:32 PM, William Brown wrote: > Hi, > > I have been working on adding support for FreeIPA to support > configuration storage for ISC-DHCP 4.X servers. I have added the schema > which is included at installation, added the template / empty files that > will be filled in and used for the installation and created the ipalib > plugin for this. At this stage, this feature is still far from done. I > would appreciate a review of the work I have done to ensure I am on the > right track. > > I would like to know if there is a better way to add ACLs than by > manually updating ldap by hand (IE, using the ACL libraries) (See > /install/share/dhcpd.ldif). > > I just looked on the plugin part from Web UI (API) perspective. Proper plugin review should do somebody else. What seems bad: 1) all entity params are required. When I'm looking at ldif, only couple of attributes are MUST. Param is made optional by adding '?' after its name. Example from user.py: Str('displayname?', label=_('Display name'), default_from=lambda givenname, sn: '%s %s' % (givenname, sn), autofill=True, ), However you can probably enforce some params to be required if it's required by DHCP server and not to blindly copy the LDAP schema. 2) You don't have to specify 'primary_key=False', it's default. 3) Don't use prefix for cli_name like 'dhcp_server_implementation' cli_name is used by IPA Client as an option name. Same for labels. For example for adding dhcp server proper command would probably be: ipa dhcpserver-add $NAME --service-dn=$SERVICE_DN etc not: ipa dhcpserver-add $NAME --dhcp-server-service-dn=$SERVICE_DN etc so the param def should be: Str('dhcpservicedn?', cli_name='service_dn', label=_('Service DN'), ), 5) All params are STR. I think some might be INT or something else. 6) You can fill 'default_attributes' list with more param names. What is mentioned in default attributes is displayed by `XXX-show $CN` command. What is not have to be displayed by `XXX-show $CN --all` command. 7) In labels you use 'Dhcp'. IMO it should be all uppercase. Regards -- Petr Vobornik From atkac at redhat.com Fri Jul 13 11:17:34 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 13 Jul 2012 13:17:34 +0200 Subject: [Freeipa-devel] [PATCH 0024] Add debug message to ldap_cache_addrdatalist() In-Reply-To: <4FFC2AD7.4050303@redhat.com> References: <4FFC2AD7.4050303@redhat.com> Message-ID: <20120713111734.GA1184@redhat.com> On Tue, Jul 10, 2012 at 03:15:03PM +0200, Petr Spacek wrote: > Hello, > > this patch adds an debug message to ldap_cache_addrdatalist(). > > It is very useful for persistent search debugging. Hi, although idea of the patch is fine, I don't think that statements which allocate memory should be in the "cleanup" path. What about this (generally used in BIND): ldap_cache_getrdatalist() { ... char namebuf[DNS_NAME_FORMATSIZE]; ... cleanup: dns_name_format(name, namebuf, sizeof(namebuf)); log_debug(20, "%s", namebuf); } Or you can put this entire code into "if (isc_log_debuglevel() >= 20)" statement to save some CPU cycles... Regards, Adam > From 29a95bb7480802bfd9f10ccdffca6158eedf4581 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Thu, 28 Jun 2012 13:52:38 +0200 > Subject: [PATCH] Add debug message to ldap_cache_addrdatalist() > > --- > src/cache.c | 14 ++++++++++++++ > 1 files changed, 14 insertions(+), 0 deletions(-) > > diff --git a/src/cache.c b/src/cache.c > index c8afb99babd9de5263f0926ffd509e52fb87d8cd..2125d97bec2429307d3e0d16c31f9a2cc4bc1d00 100644 > --- a/src/cache.c > +++ b/src/cache.c > @@ -26,6 +26,7 @@ > > #include > #include > +#include > > #include > > @@ -207,6 +208,19 @@ ldap_cache_getrdatalist(isc_mem_t *mctx, ldap_cache_t *cache, > > cleanup: > UNLOCK(&cache->mutex); > + > + if (isc_log_getdebuglevel(dns_lctx) >= 20) { > + char *dns_str = NULL; > + if (dns_name_tostring(name, &dns_str, mctx) == ISC_R_SUCCESS) { > + log_debug(20, "cache search for '%s': %s", dns_str, > + isc_result_totext(result)); > + isc_mem_free(mctx, dns_str); > + } else { > + log_debug(20, "cache search for '': %s", > + isc_result_totext(result)); > + } > + } > + > return result; > } > > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From simo at redhat.com Fri Jul 13 11:45:27 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 13 Jul 2012 07:45:27 -0400 Subject: [Freeipa-devel] [PATCH] (master) Support case-insensitive searches for principals during TGS request processing In-Reply-To: <20120402155010.GD2377@localhost.localdomain> References: <20120329133035.GH3500@redhat.com> <1333054951.22628.169.camel@willson.li.ssimo.org> <20120402155010.GD2377@localhost.localdomain> Message-ID: <1342179927.2718.37.camel@willson.li.ssimo.org> On Mon, 2012-04-02 at 17:50 +0200, Sumit Bose wrote: > On Thu, Mar 29, 2012 at 05:02:31PM -0400, Simo Sorce wrote: > > On Thu, 2012-03-29 at 16:30 +0300, Alexander Bokovoy wrote: > > > This is due to some krbtgt/realm at REALM searches performed in KDC > > > without > > > allowing for principal aliases and therefore no chance to our > > > case-insensitive searches to kick in. Additional discussion is needed, > > > I > > > think, if we want to support case-insensitive realms. > > > > > I do not think we want to support case-insensitive realm lookups just > > yet. So what you have now seem sufficient to me. > > Patch looks good and is passing my testing. > > ACK This patch has long since been pushed. (Closing loose ends.) Simo. -- Simo Sorce * Red Hat, Inc * New York From pspacek at redhat.com Fri Jul 13 12:16:30 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 13 Jul 2012 14:16:30 +0200 Subject: [Freeipa-devel] [PATCH 0024] Add debug message to ldap_cache_addrdatalist() In-Reply-To: <20120713111734.GA1184@redhat.com> References: <4FFC2AD7.4050303@redhat.com> <20120713111734.GA1184@redhat.com> Message-ID: <5000119E.70501@redhat.com> On 07/13/2012 01:17 PM, Adam Tkac wrote: > On Tue, Jul 10, 2012 at 03:15:03PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch adds an debug message to ldap_cache_addrdatalist(). >> >> It is very useful for persistent search debugging. > > Hi, > > although idea of the patch is fine, I don't think that statements which allocate > memory should be in the "cleanup" path. > > What about this (generally used in BIND): > > ldap_cache_getrdatalist() { > ... > char namebuf[DNS_NAME_FORMATSIZE]; > ... > cleanup: > dns_name_format(name, namebuf, sizeof(namebuf)); > log_debug(20, "%s", namebuf); > } > > Or you can put this entire code into "if (isc_log_debuglevel() >= 20)" statement > to save some CPU cycles... > > Regards, Adam It is definitely good idea. I forgot to upper-limit for DNS names. Revised patch was pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/3c382dd0296f6fe2931ddb0d18de220e6740011c Petr^2 Spacek From jcholast at redhat.com Fri Jul 13 12:20:34 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 13 Jul 2012 14:20:34 +0200 Subject: [Freeipa-devel] [PATCH] 283 Improve address family handling in sockets In-Reply-To: <4FFD3A8B.1040708@redhat.com> References: <4FF3ED04.9020002@redhat.com> <4FFD3A8B.1040708@redhat.com> Message-ID: <50001292.50008@redhat.com> Dne 11.7.2012 10:34, Martin Kosek napsal(a): > On 07/04/2012 09:13 AM, Martin Kosek wrote: >> I did various tests with IPv4 and IPv6 and everything worked for me. I also >> tried a mixed IPv4+IPv6 and IPv6-only environment and I was able to install an >> IPv6-only replica without issues. >> >> --- >> >> Many functions use low-level socket interface for connection or >> various checks. However, most of the time we don't respect >> automatic address family detection but rather try to force our >> values. This may cause either redundat connection tries when an >> address family is disabled on system tries or even crashes >> when socket exceptions are not properly caught. >> >> Instead of forcing address families to socket, rather use >> getaddrinfo interface to automatically retrieve a list of all >> relevant address families and other connection settings when >> connecting to remote/local machine or binding to a local port. >> Now, we will also fill correctly all connection parameters like >> flowinfo and scopeid for IPv6 connections which will for example >> prevent issues with scoped IPv6 addresses. >> >> bind_port_responder function was changed to at first try to bind >> to IPv6 wildcard address before IPv4 as IPv6 socket is able to >> accept both IPv4 and IPv6 connections (unlike IPv4 socket). >> >> nsslib connection was refactored to use nss.io.AddrInfo class to >> get all the available connections. Socket is now not created by >> default in NSSConnection class initializer, but rather when the >> actual connection is being made, becase we do not an address family >> where connection is successful. >> >> https://fedorahosted.org/freeipa/ticket/2695 >> > > Attaching a rebased patch with updated comment - the patch also fix issues in > ticket 2913. > > I just found an easy way to reproduce an issue caused by incorrect address > family handling that can be tried during review: > > 1) Turn of IPv6 in your (Fedora) OS: > - add "ipv6.disable=1" as kernel parameter in your kernel line in your > bootloader conf > - add "NETWORKING_IPV6=no" to your /etc/sysconfig/network > > 2) Run "ipa-replica-conncheck -m " where is a fqdn of some of > your running IPA servers. Current IPA version will produce bunch of tracebacks, > patched IPA should work without any issue > > Martin > ACK, both IPv4-only and IPv6-only installs work fine. Honza -- Jan Cholasta From mkosek at redhat.com Fri Jul 13 12:37:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 13 Jul 2012 14:37:44 +0200 Subject: [Freeipa-devel] [PATCH] 283 Improve address family handling in sockets In-Reply-To: <50001292.50008@redhat.com> References: <4FF3ED04.9020002@redhat.com> <4FFD3A8B.1040708@redhat.com> <50001292.50008@redhat.com> Message-ID: <50001698.7020605@redhat.com> On 07/13/2012 02:20 PM, Jan Cholasta wrote: > Dne 11.7.2012 10:34, Martin Kosek napsal(a): >> On 07/04/2012 09:13 AM, Martin Kosek wrote: >>> I did various tests with IPv4 and IPv6 and everything worked for me. I also >>> tried a mixed IPv4+IPv6 and IPv6-only environment and I was able to install an >>> IPv6-only replica without issues. >>> >>> --- >>> >>> Many functions use low-level socket interface for connection or >>> various checks. However, most of the time we don't respect >>> automatic address family detection but rather try to force our >>> values. This may cause either redundat connection tries when an >>> address family is disabled on system tries or even crashes >>> when socket exceptions are not properly caught. >>> >>> Instead of forcing address families to socket, rather use >>> getaddrinfo interface to automatically retrieve a list of all >>> relevant address families and other connection settings when >>> connecting to remote/local machine or binding to a local port. >>> Now, we will also fill correctly all connection parameters like >>> flowinfo and scopeid for IPv6 connections which will for example >>> prevent issues with scoped IPv6 addresses. >>> >>> bind_port_responder function was changed to at first try to bind >>> to IPv6 wildcard address before IPv4 as IPv6 socket is able to >>> accept both IPv4 and IPv6 connections (unlike IPv4 socket). >>> >>> nsslib connection was refactored to use nss.io.AddrInfo class to >>> get all the available connections. Socket is now not created by >>> default in NSSConnection class initializer, but rather when the >>> actual connection is being made, becase we do not an address family >>> where connection is successful. >>> >>> https://fedorahosted.org/freeipa/ticket/2695 >>> >> >> Attaching a rebased patch with updated comment - the patch also fix issues in >> ticket 2913. >> >> I just found an easy way to reproduce an issue caused by incorrect address >> family handling that can be tried during review: >> >> 1) Turn of IPv6 in your (Fedora) OS: >> - add "ipv6.disable=1" as kernel parameter in your kernel line in your >> bootloader conf >> - add "NETWORKING_IPV6=no" to your /etc/sysconfig/network >> >> 2) Run "ipa-replica-conncheck -m " where is a fqdn of some of >> your running IPA servers. Current IPA version will produce bunch of tracebacks, >> patched IPA should work without any issue >> >> Martin >> > > ACK, both IPv4-only and IPv6-only installs work fine. > > Honza > Thanks for thorough review. Pushed to master. Martin From simo at redhat.com Fri Jul 13 13:16:25 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 13 Jul 2012 09:16:25 -0400 Subject: [Freeipa-devel] [PATCH] Improve performance of get_group_sids() In-Reply-To: <20120710210420.GD922@localhost.localdomain> References: <20120710210420.GD922@localhost.localdomain> Message-ID: <1342185385.2718.55.camel@willson.li.ssimo.org> On Tue, 2012-07-10 at 23:04 +0200, Sumit Bose wrote: > Hi, > > the following two patches are the first step to fix > https://fedorahosted.org/freeipa/ticket/2881. Unit tests with time > measurements are added and the performance of the get_group_sids() > function is improved by an order of magnitude. > > The caching of the LDAP results is still missing. I will finish this > after my PTO. But I haven't started to work on this. So if you think it > should be fixed earlier feel free to take the ticket. Nack, for minor issues. Patch1: - Please create a tests/ directory in daemons/ipa-kdb for the tests - Please fit the whole file within 79 columns, there are a number of lines towards the end of the tests that go beyond that. - There is a spurious comment at the top: +/* talloc leak checks written by Martin Nagy for sssd */ - please add a dlinkslist.h file to /util instead of copying macros in the tests file Patch2: - Needs spacing in arithmetic operations. This line: + idx = group_sids_buf+(p*sid_len); should be: + idx = group_sids_buf + (p * sid_len); and other similar cases. - Should we align the sids in the big memory allocation you make ? + dom_sid_str_len = strlen(dom_sid_str); + sid_len = dom_sid_str_len + 12; this can result in any alignment, should we align32/64 sid_len ? It would waste a bit of space, but may be beneficial on some archs. - Please fit the whole file within 79 columns, there are a number of lines that go beyond that. - A Rid is a 32 bit unsigned, no need to cast to (unsigned long), just declare the format string as %u (unsigned) - Not introduced with this patch, but can you remove most of the debug stuff going to syslog ? I do not think we want to spam syslog with this low level stuff, we must not confuse debugging with logging like it happens in samba-land. If you really want to retain that stuff, wrap it around a macro that is enabled only with a debugging option passed to configure or make. Simo. -- Simo Sorce * Red Hat, Inc * New York From atkac at redhat.com Fri Jul 13 13:42:46 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 13 Jul 2012 15:42:46 +0200 Subject: [Freeipa-devel] [PATCH] 0025-0028 Implement SOA serial number increments for external changes In-Reply-To: <4FFC34C4.6050902@redhat.com> References: <4FFC34C4.6050902@redhat.com> Message-ID: <20120713134245.GA3004@redhat.com> On Tue, Jul 10, 2012 at 03:57:24PM +0200, Petr Spacek wrote: > Hello, > > these patches provides SOA serial auto-increment feature for external changes. > Related ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/67 > > It is necessary to set "psearch" AND "serial_autoincrement" to "yes" > in /etc/named.conf to enable this feature. > > In replicated environment idnsSOAserial attribute has to be declared > as non-replicated. It is done by mkosek's patch 281 for 389 DS & > FreeIPA. > > For testing purposes it is enough to add "idnsSOAserial" to end of > exclude list in nsDS5ReplicatedAttributeList attribute for each > replication agreement located in cn=mapping tree,cn=config subtree. > > > My patch 28 contains "trick" necessary for replicated environments > with 389 DS. 389 sends entry change notification (ECN) in cases when > non-replicated attribute idnsSOAserial was changed on *other side*. > In that case no change is visible in DNS attributes, but ECN is sent > by 389. (Attribute modifyTimestamp is changed also.) > > Patch 28 computes digest/hash from all resource records in idnsZone > object and compares old and new digest after each received ECN. This > approach eliminates "false changes". > > Each patch depends on all preceding patches, but each patch > implements visible (and testable) part of functionality. Hello Peter, please check my comments below. Regards, Adam > From 0ed1d3dd4910e1c94617b0209420b1c6598de68e Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 27 Jun 2012 10:36:26 +0200 > Subject: [PATCH] Increment SOA serial for each ordinary record received > through psearch > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++- > 1 files changed, 97 insertions(+), 1 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 7f0a6f4b37171a6fa4db79cd32fdd8bc62288e0f..5c509443dfb176607d8ee4714d196efaa4afa66b 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -34,6 +34,8 @@ > #include > #include > #include > +#include > +#include > > #include > #include > @@ -172,6 +174,7 @@ struct ldap_instance { > isc_boolean_t exiting; > isc_boolean_t sync_ptr; > isc_boolean_t dyn_update; > + isc_boolean_t serial_autoincrement; > }; > > struct ldap_pool { > @@ -343,6 +346,7 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > { "ldap_hostname", default_string("") }, > { "sync_ptr", default_boolean(ISC_FALSE) }, > { "dyn_update", default_boolean(ISC_FALSE) }, > + { "serial_autoincrement", default_boolean(ISC_FALSE) }, > end_of_settings > }; > > @@ -401,6 +405,7 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > ldap_settings[i++].target = ldap_inst->ldap_hostname; > ldap_settings[i++].target = &ldap_inst->sync_ptr; > ldap_settings[i++].target = &ldap_inst->dyn_update; > + ldap_settings[i++].target = &ldap_inst->serial_autoincrement; > CHECK(set_settings(ldap_settings, argv)); > > /* Set timer for deadlock detection inside semaphore_wait_timed . */ > @@ -463,6 +468,13 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > "increasing limit"); > ldap_inst->connections = 3; > } > + if (ldap_inst->serial_autoincrement == ISC_TRUE > + && ldap_inst->psearch != ISC_TRUE) { > + log_error("SOA serial number auto-increment feature requires " > + "persistent search"); > + result = ISC_R_FAILURE; > + goto cleanup; > + } > > CHECK(new_ldap_cache(mctx, argv, &ldap_inst->cache, ldap_inst->psearch)); > CHECK(ldap_pool_create(mctx, ldap_inst->connections, &ldap_inst->pool)); > @@ -2741,6 +2753,86 @@ ldap_pscontrol_destroy(isc_mem_t *mctx, LDAPControl **ctrlp) > *ctrlp = NULL; > } > > +isc_result_t Better is "static inline isc_result_t" > +soa_serial_get(ldap_instance_t *inst, dns_name_t *zone_name, > + isc_uint32_t *serial) { > + isc_result_t result; > + dns_zone_t *zone = NULL; > + > + REQUIRE(inst != NULL); > + REQUIRE(zone_name != NULL); > + REQUIRE(serial != NULL); All those REQUIREs are redundant, please remove them. The "inst" parameter is checked before function call in soa_serial_increment and last two params are checked by functions below. > + > + CHECK(zr_get_zone_ptr(inst->zone_register, zone_name, &zone)); > + CHECK(dns_zone_getserial2(zone, serial)); > + dns_zone_detach(&zone); > + > +cleanup: > + return result; > +} > + > +isc_result_t > +soa_serial_increment(isc_mem_t *mctx, ldap_instance_t *inst, > + dns_name_t *zone_name) { > + isc_result_t result = ISC_R_FAILURE; > + ldap_connection_t * conn = NULL; > + ld_string_t *zone_dn = NULL; > + ldapdb_rdatalist_t rdatalist; > + dns_rdatalist_t *rdlist = NULL; > + dns_rdata_t *soa_rdata = NULL; > + isc_uint32_t old_serial; > + isc_uint32_t new_serial; > + isc_time_t curr_time; > + > + REQUIRE(inst != NULL); > + REQUIRE(zone_name != NULL); > + > + CHECK(str_new(mctx, &zone_dn)); > + CHECK(dnsname_to_dn(inst->zone_register, zone_name, zone_dn)); > + log_debug(5, "incrementing SOA serial number in zone '%s'", > + str_buf(zone_dn)); > + > + /* get original SOA rdata and serial value */ > + INIT_LIST(rdatalist); rdatalist variable should be initialized before the first CHECK call, othewise we will crash in the "cleanup" path in ldapdb_rdatalist_destroy when earlier CHECK() fails. > + CHECK(ldapdb_rdatalist_get(mctx, inst, zone_name, zone_name, &rdatalist)); > + CHECK(ldapdb_rdatalist_findrdatatype(&rdatalist, dns_rdatatype_soa, &rdlist)); > + soa_rdata = ISC_LIST_HEAD(rdlist->rdata); > + CHECK(soa_serial_get(inst, zone_name, &old_serial)); > + > + /* Compute the new SOA serial - use actual timestamp. > + * If timestamp <= oldSOAserial then increment old serial by one. */ > + isc_time_now(&curr_time); > + new_serial = isc_time_seconds(&curr_time) & 0xFFFFFFFF; > + if (!isc_serial_gt(new_serial, old_serial)) { > + /* increment by one, RFC1982, from bind-9.8.2/bin/named/update.c */ > + new_serial = (old_serial + 1) & 0xFFFFFFFF; > + } > + if (new_serial == 0) > + new_serial = 1; > + log_debug(5,"zone '%s': old serial %u, new serial %u", > + str_buf(zone_dn), old_serial, new_serial); > + dns_soa_setserial(new_serial, soa_rdata); > + > + /* write the new serial back to DB */ > + CHECK(ldap_pool_getconnection(inst->pool, &conn)); > + CHECK(modify_soa_record(conn, str_buf(zone_dn), soa_rdata)); > + CHECK(discard_from_cache(ldap_instance_getcache(inst), zone_name)); > + > + /* put the new SOA to inst->cache and compare old and new serials */ > + CHECK(soa_serial_get(inst, zone_name, &new_serial)); > + INSIST(isc_serial_gt(new_serial, old_serial) == ISC_TRUE); > + > +cleanup: > + if (result != ISC_R_SUCCESS) > + log_error("SOA serial number incrementation failed in zone '%s'", > + str_buf(zone_dn)); > + > + ldap_pool_putconnection(inst->pool, &conn); > + str_destroy(&zone_dn); > + ldapdb_rdatalist_destroy(mctx, &rdatalist); > + return result; > +} > + > /* > * update_action routine is processed asynchronously so it cannot assume > * anything about state of ldap_inst from where it was sent. The ldap_inst > @@ -2892,7 +2984,7 @@ update_record(isc_task_t *task, isc_event_t *event) > if (PSEARCH_DEL(pevent->chgtype)) { > log_debug(5, "psearch_update: Removing item from cache (%s)", > pevent->dn); > - } > + } > > /* Get cache instance & clean old record */ > cache = ldap_instance_getcache(inst); > @@ -2916,6 +3008,10 @@ update_record(isc_task_t *task, isc_event_t *event) > /* Destroy rdatalist, it is now in the cache. */ > ldapdb_rdatalist_destroy(mctx, &rdatalist); > } > + > + if (inst->serial_autoincrement) { > + CHECK(soa_serial_increment(mctx, inst, &origin)); > + } > cleanup: > if (result != ISC_R_SUCCESS) > log_error("update_record (psearch) failed for %s. " > -- > 1.7.7.6 > > From b82c951ad09c72e524be93ba06b0f46897c85f71 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 2 Jul 2012 11:01:58 +0200 > Subject: [PATCH] Do not bump serial for each record during initial database > dump. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 18 ++++++++++++++---- > 1 files changed, 14 insertions(+), 4 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 5c509443dfb176607d8ee4714d196efaa4afa66b..d744642364ba70919f52abd5ea31998f8e91ccd4 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -2677,6 +2677,7 @@ cleanup: > #define LDAP_CONTROL_PERSISTENTSEARCH "2.16.840.1.113730.3.4.3" > #define LDAP_CONTROL_ENTRYCHANGE "2.16.840.1.113730.3.4.7" > > +#define LDAP_ENTRYCHANGE_NONE 0 /* entry change control is not present */ > #define LDAP_ENTRYCHANGE_ADD 1 > #define LDAP_ENTRYCHANGE_DEL 2 > #define LDAP_ENTRYCHANGE_MOD 4 > @@ -2687,6 +2688,7 @@ cleanup: > #define PSEARCH_DEL(chgtype) ((chgtype & LDAP_ENTRYCHANGE_DEL) != 0) > #define PSEARCH_MOD(chgtype) ((chgtype & LDAP_ENTRYCHANGE_MOD) != 0) > #define PSEARCH_MODDN(chgtype) ((chgtype & LDAP_ENTRYCHANGE_MODDN) != 0) > +#define PSEARCH_ANY(chgtype) ((chgtype & LDAP_ENTRYCHANGE_ALL) != 0) > /* > * Creates persistent search (aka psearch, > * http://tools.ietf.org/id/draft-ietf-ldapext-psearch-03.txt) control. > @@ -2990,9 +2992,11 @@ update_record(isc_task_t *task, isc_event_t *event) > cache = ldap_instance_getcache(inst); > CHECK(discard_from_cache(cache, &name)); > > - if (PSEARCH_ADD(pevent->chgtype) || PSEARCH_MOD(pevent->chgtype)) { > + if (PSEARCH_ADD(pevent->chgtype) || PSEARCH_MOD(pevent->chgtype) || > + !PSEARCH_ANY(pevent->chgtype)) { > /* > - * Find new data in LDAP. > + * Find new data in LDAP. !PSEARCH_ANY indicates unchanged entry > + * found during initial lookup (i.e. database dump). > * > * @todo Change this to convert ldap_entry_t to ldapdb_rdatalist_t. > */ > @@ -3009,7 +3013,13 @@ update_record(isc_task_t *task, isc_event_t *event) > ldapdb_rdatalist_destroy(mctx, &rdatalist); > } > > - if (inst->serial_autoincrement) { > + log_debug(20,"psearch change type: none%d, add%d, del%d, mod%d, moddn%d", > + !PSEARCH_ANY(pevent->chgtype), PSEARCH_ADD(pevent->chgtype), > + PSEARCH_DEL(pevent->chgtype), PSEARCH_MOD(pevent->chgtype), > + PSEARCH_MODDN(pevent->chgtype)); > + > + /* Do not bump serial during initial database dump. */ > + if (inst->serial_autoincrement && PSEARCH_ANY(pevent->chgtype)) { > CHECK(soa_serial_increment(mctx, inst, &origin)); > } > cleanup: > @@ -3092,7 +3102,7 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > ldap_entryclass_t class; > isc_result_t result = ISC_R_SUCCESS; > ldap_psearchevent_t *pevent = NULL; > - int chgtype = LDAP_ENTRYCHANGE_ADD; > + int chgtype = LDAP_ENTRYCHANGE_NONE; > char *moddn = NULL; > char *dn = NULL; > char *dbname = NULL; > -- > 1.7.7.6 > > From 9e85fd45baff2dd6fb696ce416af8ae8a8ecccbe Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Mon, 2 Jul 2012 16:40:23 +0200 > Subject: [PATCH] Maintain SOA serial for zone record changes also. Bump > serial after each BIND startup. Manual changes to zone > serial are allowed. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 61 ++++++++++++++++++++++++++++++++++++--------------- > src/ldap_helper.h | 6 +++++ > src/zone_register.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++ > src/zone_register.h | 6 +++++ > 4 files changed, 114 insertions(+), 18 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index d744642364ba70919f52abd5ea31998f8e91ccd4..da6f8562ef59dadc8882743ee87bf84305a1de82 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -1000,8 +1000,9 @@ ldap_parse_zoneentry(ldap_entry_t *entry, ldap_instance_t *inst) > isc_result_t result; > isc_boolean_t unlock = ISC_FALSE; > isc_boolean_t publish = ISC_FALSE; > - isc_boolean_t load = ISC_FALSE; > isc_task_t *task = inst->task; > + isc_uint32_t ldap_serial; > + isc_uint32_t zr_serial; > > dns_name_init(&name, NULL); > > @@ -1020,11 +1021,11 @@ ldap_parse_zoneentry(ldap_entry_t *entry, ldap_instance_t *inst) > * Fetch forwarders. > * Forwarding has top priority hence when the forwarders are properly > * set up all others attributes are ignored. > - */ > + */ > result = ldap_entry_getvalues(entry, "idnsForwarders", &values); > if (result == ISC_R_SUCCESS) { > log_debug(2, "Setting forwarders for %s", dn); > - CHECK(configure_zone_forwarders(entry, inst, &name, &values)); > + CHECK(configure_zone_forwarders(entry, inst, &name, &values)); > /* DO NOT CHANGE ANYTHING ELSE after forwarders are set up! */ > goto cleanup; > } else { > @@ -1080,25 +1081,49 @@ ldap_parse_zoneentry(ldap_entry_t *entry, ldap_instance_t *inst) > /* Everything is set correctly, publish zone */ > CHECK(publish_zone(inst, zone)); > } > - load = ISC_TRUE; > + > + /* > + * Don't bother if load fails, server will return > + * SERVFAIL for queries beneath this zone. This is > + * admin's problem. > + */ > + result = dns_zone_load(zone); > + if (result != ISC_R_SUCCESS && result != DNS_R_UPTODATE > + && result != DNS_R_DYNAMIC && result != DNS_R_CONTINUE) > + goto cleanup; > + > + result = ISC_R_SUCCESS; > + > + /* initialize serial in zone register and always increment serial > + * for a new zone (typically after BIND start) > + * - the zone was possibly changed in meanwhile */ > + if (publish) { > + CHECK(ldap_get_zone_serial(inst, &name, &ldap_serial)); > + CHECK(zr_set_zone_serial(inst->zone_register, &name, ldap_serial)); > + } > + > + /* SOA serial autoincrement feature for SOA record: > + * 1) remember old SOA serial from zone register > + * 2) compare old and new SOA serial from LDAP DB > + * 3) if (old == new) then do autoincrement, remember new serial otherwise */ > + if (inst->serial_autoincrement) { > + CHECK(ldap_get_zone_serial(inst, &name, &ldap_serial)); > + CHECK(zr_get_zone_serial(inst->zone_register, &name, &zr_serial)); > + if (ldap_serial == zr_serial) > + CHECK(soa_serial_increment(inst->mctx, inst, &name)); > + else > + CHECK(zr_set_zone_serial(inst->zone_register, &name, ldap_serial)); > + } > > cleanup: > - if (load) { > - /* > - * Don't bother if load fails, server will return > - * SERVFAIL for queries beneath this zone. This is > - * admin's problem. > - */ > - (void) dns_zone_load(zone); > - } > if (unlock) > isc_task_endexclusive(task); > if (dns_name_dynamic(&name)) > dns_name_free(&name, inst->mctx); > if (zone != NULL) > dns_zone_detach(&zone); > > - return ISC_R_SUCCESS; > + return result; > } > > /* > @@ -2756,7 +2781,7 @@ ldap_pscontrol_destroy(isc_mem_t *mctx, LDAPControl **ctrlp) > } > > isc_result_t > -soa_serial_get(ldap_instance_t *inst, dns_name_t *zone_name, > +ldap_get_zone_serial(ldap_instance_t *inst, dns_name_t *zone_name, > isc_uint32_t *serial) { > isc_result_t result; > dns_zone_t *zone = NULL; > @@ -2799,7 +2824,7 @@ soa_serial_increment(isc_mem_t *mctx, ldap_instance_t *inst, > CHECK(ldapdb_rdatalist_get(mctx, inst, zone_name, zone_name, &rdatalist)); > CHECK(ldapdb_rdatalist_findrdatatype(&rdatalist, dns_rdatatype_soa, &rdlist)); > soa_rdata = ISC_LIST_HEAD(rdlist->rdata); > - CHECK(soa_serial_get(inst, zone_name, &old_serial)); > + CHECK(ldap_get_zone_serial(inst, zone_name, &old_serial)); > > /* Compute the new SOA serial - use actual timestamp. > * If timestamp <= oldSOAserial then increment old serial by one. */ > @@ -2821,7 +2846,7 @@ soa_serial_increment(isc_mem_t *mctx, ldap_instance_t *inst, > CHECK(discard_from_cache(ldap_instance_getcache(inst), zone_name)); > > /* put the new SOA to inst->cache and compare old and new serials */ > - CHECK(soa_serial_get(inst, zone_name, &new_serial)); > + CHECK(ldap_get_zone_serial(inst, zone_name, &new_serial)); > INSIST(isc_serial_gt(new_serial, old_serial) == ISC_TRUE); > > cleanup: > @@ -2887,9 +2912,9 @@ update_action(isc_task_t *task, isc_event_t *event) > > cleanup: > if (result != ISC_R_SUCCESS) > - log_error("update_action (psearch) failed for %s. " > + log_error("update_action (psearch) failed for '%s': %s. " > "Zones can be outdated, run `rndc reload`", > - pevent->dn); > + pevent->dn, isc_result_totext(result)); > > ldap_pool_putconnection(inst->pool, &conn); > isc_mem_free(mctx, pevent->dbname); > diff --git a/src/ldap_helper.h b/src/ldap_helper.h > index bc784106abefe15787841578687469539c8052c3..b6e46e307fa7e2f0af1b71600fcabdb0c13c5649 100644 > --- a/src/ldap_helper.h > +++ b/src/ldap_helper.h > @@ -93,4 +93,10 @@ isc_result_t remove_from_ldap(dns_name_t *owner, ldap_instance_t *ldap_inst, > /* Get cache associated with ldap_inst */ > ldap_cache_t *ldap_instance_getcache(ldap_instance_t *ldap_inst); > > +isc_result_t ldap_get_zone_serial(ldap_instance_t *inst, dns_name_t *zone_name, > + isc_uint32_t *serial); > + > +isc_result_t soa_serial_increment(isc_mem_t *mctx, ldap_instance_t *inst, > + dns_name_t *zone_name); > + Why do you need to export those functions out of the ldap_helper.c? I prefer to mark them as static and don't export them. > #endif /* !_LD_LDAP_HELPER_H_ */ > diff --git a/src/zone_register.c b/src/zone_register.c > index 81d208fc6e3c0dba6efb72ae10db54a79c336eca..1de6bf1ac121dc02c54f187cd36f8b588ee14d18 100644 > --- a/src/zone_register.c > +++ b/src/zone_register.c > @@ -50,6 +50,7 @@ struct zone_register { > typedef struct { > dns_zone_t *zone; > char *dn; > + isc_uint32_t serial; /* last value processed by plugin (!= value in DB) */ > } zone_info_t; > > /* Callback for dns_rbt_create(). */ > @@ -136,6 +137,7 @@ create_zone_info(isc_mem_t *mctx, dns_zone_t *zone, const char *dn, > > CHECKED_MEM_GET_PTR(mctx, zinfo); > CHECKED_MEM_STRDUP(mctx, dn, zinfo->dn); > + zinfo->serial = 0; > zinfo->zone = NULL; > dns_zone_attach(zone, &zinfo->zone); > > @@ -312,3 +314,60 @@ zr_get_zone_ptr(zone_register_t *zr, dns_name_t *name, dns_zone_t **zonep) > > return result; > } > + > +/** > + * Return last SOA serial value processed by autoincrement feature. > + */ > +isc_result_t > +zr_get_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t *serialp) > +{ > + isc_result_t result; > + void *zinfo = NULL; > + > + REQUIRE(zr != NULL); > + REQUIRE(name != NULL); ^^^ This REQUIRE is redundant. > + REQUIRE(serialp != NULL); > + > + if (!dns_name_isabsolute(name)) { > + log_bug("trying to find zone with a relative name"); > + return ISC_R_FAILURE; > + } > + > + RWLOCK(&zr->rwlock, isc_rwlocktype_read); > + > + result = dns_rbt_findname(zr->rbt, name, 0, NULL, &zinfo); > + if (result == ISC_R_SUCCESS) > + *serialp = ((zone_info_t *)zinfo)->serial; > + > + RWUNLOCK(&zr->rwlock, isc_rwlocktype_read); > + > + return result; > +} > + > +/** > + * Set last SOA serial value processed by autoincrement feature. > + */ > +isc_result_t > +zr_set_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t serial) > +{ > + isc_result_t result; > + void *zinfo = NULL; > + > + REQUIRE(zr != NULL); > + REQUIRE(name != NULL); > + > + if (!dns_name_isabsolute(name)) { > + log_bug("trying to find zone with a relative name"); > + return ISC_R_FAILURE; > + } > + > + RWLOCK(&zr->rwlock, isc_rwlocktype_read); > + > + result = dns_rbt_findname(zr->rbt, name, 0, NULL, &zinfo); > + if (result == ISC_R_SUCCESS) > + ((zone_info_t *)zinfo)->serial = serial; > + > + RWUNLOCK(&zr->rwlock, isc_rwlocktype_read); > + > + return result; > +} > diff --git a/src/zone_register.h b/src/zone_register.h > index fa8ef255ef9cf212bca04aaafba0e6450d40a5c6..6ac3a92fbe755fcff2d9af3e6e26b32ae83d4133 100644 > --- a/src/zone_register.h > +++ b/src/zone_register.h > @@ -48,4 +48,10 @@ zr_get_rbt(zone_register_t *zr); > isc_mem_t * > zr_get_mctx(zone_register_t *zr); > > +isc_result_t > +zr_set_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t serial); > + > +isc_result_t > +zr_get_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t *serialp); > + > #endif /* !_LD_ZONE_REGISTER_H_ */ > -- > 1.7.7.6 > > From 6918535ee7abd3121db7f31c0c7ecba6a5e847cf Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Tue, 10 Jul 2012 14:23:46 +0200 > Subject: [PATCH] Add support for replicated environments to SOA serial > autoincrement feature. 389 DS sends entry change > notification even if modifyTimestamp was modified because > of replication from another DS. This code computes digest > from resource records in zone object and compares these > digests for each received entry change notification. False > entry change notifications are revealed this way and no > incrementation is done in that case. > http://fedorahosted.org/bind-dyndb-ldap/ticket/67 > > Signed-off-by: Petr Spacek > --- > src/ldap_driver.c | 16 +------ > src/ldap_helper.c | 47 +++++++++++++++---- > src/rdlist.c | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++- > src/rdlist.h | 13 +++++ > src/zone_register.c | 35 +++++++++----- > src/zone_register.h | 6 ++- > 6 files changed, 204 insertions(+), 40 deletions(-) > > diff --git a/src/ldap_driver.c b/src/ldap_driver.c > index a87db180b0aa72c97aa51dd178ecdab7b90a0afd..cae45d4f6cc1f201c40ca3413d1f626e03a0318e 100644 > --- a/src/ldap_driver.c > +++ b/src/ldap_driver.c > @@ -110,7 +110,6 @@ static void *ldapdb_version = &dummy; > > static void free_ldapdb(ldapdb_t *ldapdb); > static void detachnode(dns_db_t *db, dns_dbnode_t **targetp); > -static unsigned int rdatalist_length(const dns_rdatalist_t *rdlist); > static isc_result_t clone_rdatalist_to_rdataset(isc_mem_t *mctx, > dns_rdatalist_t *rdlist, > dns_rdataset_t *rdataset); > @@ -545,6 +544,7 @@ found: > goto skipfind; > > result = ldapdb_rdatalist_findrdatatype(&rdatalist, type, &rdlist); > + > if (result != ISC_R_SUCCESS) { > /* No exact rdtype match. Check CNAME */ > > @@ -1006,20 +1006,6 @@ cleanup: > return result; > } > > -static unsigned int > -rdatalist_length(const dns_rdatalist_t *rdlist) > -{ > - dns_rdata_t *ptr = HEAD(rdlist->rdata); > - unsigned int length = 0; > - > - while (ptr != NULL) { > - length++; > - ptr = NEXT(ptr, link); > - } > - > - return length; > -} > - > static isc_result_t > subtractrdataset(dns_db_t *db, dns_dbnode_t *node, dns_dbversion_t *version, > dns_rdataset_t *rdataset, unsigned int options, > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index da6f8562ef59dadc8882743ee87bf84305a1de82..323ee81e70bd3e4e0ffd59a024f047bbcbde9d89 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -71,6 +71,7 @@ > #include "ldap_entry.h" > #include "ldap_helper.h" > #include "log.h" > +#include "rdlist.h" > #include "semaphore.h" > #include "settings.h" > #include "str.h" > @@ -1002,9 +1003,16 @@ ldap_parse_zoneentry(ldap_entry_t *entry, ldap_instance_t *inst) > isc_boolean_t publish = ISC_FALSE; > isc_task_t *task = inst->task; > isc_uint32_t ldap_serial; > - isc_uint32_t zr_serial; > + isc_uint32_t zr_serial; /* SOA serial value from in-memory zone register */ > + unsigned char ldap_digest[RDLIST_DIGESTLENGTH]; > + unsigned char *zr_digest = NULL; > + ldapdb_rdatalist_t rdatalist; > + > + REQUIRE(entry != NULL); > + REQUIRE(inst != NULL); > > dns_name_init(&name, NULL); > + INIT_LIST(rdatalist); > > /* Derive the dns name of the zone from the DN. */ > dn = entry->dn; > @@ -1098,30 +1106,49 @@ ldap_parse_zoneentry(ldap_entry_t *entry, ldap_instance_t *inst) > * for a new zone (typically after BIND start) > * - the zone was possibly changed in meanwhile */ > if (publish) { > + memset(ldap_digest, 0, RDLIST_DIGESTLENGTH); > CHECK(ldap_get_zone_serial(inst, &name, &ldap_serial)); > - CHECK(zr_set_zone_serial(inst->zone_register, &name, ldap_serial)); > + CHECK(zr_set_zone_serial_digest(inst->zone_register, &name, ldap_serial, > + ldap_digest)); > } > > /* SOA serial autoincrement feature for SOA record: > - * 1) remember old SOA serial from zone register > - * 2) compare old and new SOA serial from LDAP DB > - * 3) if (old == new) then do autoincrement, remember new serial otherwise */ > + * 1) Remember old (already processed) SOA serial and digest computed from > + * zone root records in zone register. > + * 2) After each change notification compare the old and new SOA serials > + * and recomputed digests. If: > + * 3a) Nothing was changed (false change notification received) - do nothing > + * 3b) Serial was changed - remember the new serial and recompute digest, > + * do not autoincrement (respect external change). > + * 3c) The old and new serials are same: autoincrement only if something > + * else was changed. > + */ > if (inst->serial_autoincrement) { > CHECK(ldap_get_zone_serial(inst, &name, &ldap_serial)); > - CHECK(zr_get_zone_serial(inst->zone_register, &name, &zr_serial)); > - if (ldap_serial == zr_serial) > - CHECK(soa_serial_increment(inst->mctx, inst, &name)); > - else > - CHECK(zr_set_zone_serial(inst->zone_register, &name, ldap_serial)); > + CHECK(ldapdb_rdatalist_get(inst->mctx, inst, &name, > + &name, &rdatalist)); > + CHECK(rdatalist_digest(inst->mctx, &rdatalist, ldap_digest)); > + > + CHECK(zr_get_zone_serial_digest(inst->zone_register, &name, &zr_serial, > + &zr_digest)); > + if (ldap_serial == zr_serial) { > + /* serials are same - increment only if something was changed */ > + if (memcmp(zr_digest, ldap_digest, RDLIST_DIGESTLENGTH) != 0) > + CHECK(soa_serial_increment(inst->mctx, inst, &name)); > + } else { /* serial in LDAP was changed - update zone register */ > + CHECK(zr_set_zone_serial_digest(inst->zone_register, &name, > + ldap_serial, ldap_digest)); > + } > } > > cleanup: > if (unlock) > isc_task_endexclusive(task); > if (dns_name_dynamic(&name)) > dns_name_free(&name, inst->mctx); > if (zone != NULL) > dns_zone_detach(&zone); > + ldapdb_rdatalist_destroy(inst->mctx, &rdatalist); > > return result; > } > diff --git a/src/rdlist.c b/src/rdlist.c > index 903c948f401a9bd82cbc0eb06ac55aa05452c976..a16de45bcc49d00a1eacf42b23c87f24be8d3b2f 100644 > --- a/src/rdlist.c > +++ b/src/rdlist.c > @@ -2,7 +2,7 @@ > * Authors: Adam Tkac > * Martin Nagy > * > - * Copyright (C) 2009 Red Hat > + * Copyright (C) 2009-2012 Red Hat > * see file 'COPYING' for use and warranty information > * > * This program is free software; you can redistribute it and/or > @@ -22,17 +22,27 @@ > #include > #include > #include > +#include > +#include > > #include > #include > > #include > +#include > > #include "ldap_helper.h" /* TODO: Move things from ldap_helper here? */ > #include "rdlist.h" > #include "util.h" > > > +/* useful only for RR sorting purposes */ > +typedef struct rr_sort rr_sort_t; > +struct rr_sort { > + dns_rdatalist_t *rdatalist; /* contains RR class, type, TTL */ > + isc_region_t rdatareg; /* handle to binary area with RR data */ > +}; > + > static isc_result_t > rdata_clone(isc_mem_t *mctx, dns_rdata_t *source, dns_rdata_t **targetp) > { > @@ -106,6 +116,121 @@ cleanup: > return result; > } > > +unsigned int > +rdatalist_length(const dns_rdatalist_t *rdlist) There is no reason to have this function exported, please mark it as static (and probably also as inline). > +{ > + dns_rdata_t *ptr = HEAD(rdlist->rdata); > + unsigned int length = 0; > + > + while (ptr != NULL) { > + length++; > + ptr = NEXT(ptr, link); > + } > + > + return length; > +} > + > +static int > +rr_sort_compare(const void *rdl1, const void *rdl2) { > + const rr_sort_t *r1 = rdl1; > + const rr_sort_t *r2 = rdl2; > + int res; > + > + res = r1->rdatalist->rdclass - r2->rdatalist->rdclass; > + if (res != 0) > + return res; > + > + res = r1->rdatalist->type - r2->rdatalist->type; > + if (res != 0) > + return res; > + > + res = r1->rdatalist->ttl - r2->rdatalist->ttl; > + if (res != 0) > + return res; > + > + res = isc_region_compare((isc_region_t *)&r1->rdatareg, > + (isc_region_t *)&r2->rdatareg); > + > + return res; > +} > + > +/** > + * Compute MD5 digest from all resource records in input rrdatalist. > + * All RRs are sorted by class, type, ttl and data respectively. For this reason > + * digest should be unambigous. > + * > + * @param rdlist[in] List of RRsets. Each RRset contains a list of individual RR > + * @param digest[out] Pointer to unsigned char[RDLIST_DIGESTLENGTH] array > + * @return ISC_R_SUCCESS and MD5 digest in unsigned char array "digest" > + * In case of any error the array will stay untouched. > + */ > +isc_result_t > +rdatalist_digest(isc_mem_t *mctx, ldapdb_rdatalist_t *rdlist, > + unsigned char *digest) { > + isc_result_t result; > + isc_buffer_t *rrs = NULL; /* array of all resource records from input rdlist */ > + unsigned int rrs_len = 0; > + isc_md5_t md5ctx; > + > + REQUIRE(rdlist != NULL); > + REQUIRE(digest != NULL); > + > + /* Compute count of RRs to avoid dynamic reallocations. > + * The count is expected to be small number (< 20). */ > + for (dns_rdatalist_t *rrset = HEAD(*rdlist); > + rrset != NULL; > + rrset = NEXT(rrset, link)) { > + > + rrs_len += rdatalist_length(rrset); > + } > + CHECK(isc_buffer_allocate(mctx, &rrs, rrs_len*sizeof(rr_sort_t))); > + > + /* Fill each rr_sort structure in array rrs with pointer to RRset > + * and coresponding data region from each RR. rrs array will be sorted. */ > + for (dns_rdatalist_t *rrset = HEAD(*rdlist); > + rrset != NULL; > + rrset = NEXT(rrset, link)) { > + > + for (dns_rdata_t *rr = HEAD(rrset->rdata); > + rr != NULL; > + rr = NEXT(rr, link)) { > + > + rr_sort_t rr_sort_rec; > + rr_sort_rec.rdatalist = rrset; > + dns_rdata_toregion(rr, &rr_sort_rec.rdatareg); > + > + isc_buffer_putmem(rrs, (const unsigned char *)(&rr_sort_rec), > + sizeof(rr_sort_t)); > + } > + } > + qsort(isc_buffer_base(rrs), rrs_len, sizeof(rr_sort_t), rr_sort_compare); > + > + isc_md5_init(&md5ctx); > + for (unsigned int i = 0; i < rrs_len; i++ ) { > + rr_sort_t *rr_rec = (rr_sort_t *)isc_buffer_base(rrs) + i; > + isc_md5_update(&md5ctx, > + (const unsigned char *)&rr_rec->rdatalist->rdclass, > + sizeof(rr_rec->rdatalist->rdclass)); > + isc_md5_update(&md5ctx, > + (const unsigned char *)&rr_rec->rdatalist->type, > + sizeof(rr_rec->rdatalist->type)); > + isc_md5_update(&md5ctx, > + (const unsigned char *)&rr_rec->rdatalist->ttl, > + sizeof(rr_rec->rdatalist->ttl)); > + isc_md5_update(&md5ctx, > + (const unsigned char *)(rr_rec->rdatareg.base), > + rr_rec->rdatareg.length); > + } > + isc_md5_final(&md5ctx, digest); > + isc_md5_invalidate(&md5ctx); > + > +cleanup: > + if (rrs != NULL) > + isc_buffer_free(&rrs); > + > + return result; > +} > + > isc_result_t > ldap_rdatalist_copy(isc_mem_t *mctx, ldapdb_rdatalist_t source, > ldapdb_rdatalist_t *target) > diff --git a/src/rdlist.h b/src/rdlist.h > index 04b991508e47ae320df32a60adeb5384e68951ae..465197a63933f2ec8fa1835bf0b34eeabf108d1b 100644 > --- a/src/rdlist.h > +++ b/src/rdlist.h > @@ -22,12 +22,25 @@ > #ifndef _LD_RDLIST_H_ > #define _LD_RDLIST_H_ > > +#include > + > +#include "types.h" > + > +#define RDLIST_DIGESTLENGTH ISC_MD5_DIGESTLENGTH > + > isc_result_t > rdatalist_clone(isc_mem_t *mctx, dns_rdatalist_t *source, > dns_rdatalist_t **targetp); > > isc_result_t > ldap_rdatalist_copy(isc_mem_t *mctx, ldapdb_rdatalist_t source, > ldapdb_rdatalist_t *target); > > +unsigned int > +rdatalist_length(const dns_rdatalist_t *rdlist); > + > +isc_result_t > +rdatalist_digest(isc_mem_t *mctx, ldapdb_rdatalist_t *rdlist, > + unsigned char *digest); > + > #endif /* !_LD_RDLIST_H_ */ > diff --git a/src/zone_register.c b/src/zone_register.c > index 1de6bf1ac121dc02c54f187cd36f8b588ee14d18..b2b938f3336b23a40d43c85062c9389a2190f3cb 100644 > --- a/src/zone_register.c > +++ b/src/zone_register.c > @@ -21,6 +21,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -31,6 +32,7 @@ > #include "log.h" > #include "util.h" > #include "zone_register.h" > +#include "rdlist.h" > > /* > * The zone register is a red-black tree that maps a dns name of a zone to the > @@ -51,6 +53,7 @@ typedef struct { > dns_zone_t *zone; > char *dn; > isc_uint32_t serial; /* last value processed by plugin (!= value in DB) */ > + unsigned char digest[RDLIST_DIGESTLENGTH]; /* MD5 digest from all RRs in zone record */ > } zone_info_t; > > /* Callback for dns_rbt_create(). */ > @@ -316,56 +319,64 @@ zr_get_zone_ptr(zone_register_t *zr, dns_name_t *name, dns_zone_t **zonep) > } > > /** > - * Return last SOA serial value processed by autoincrement feature. > + * Return last values processed by autoincrement feature. > */ > isc_result_t > -zr_get_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t *serialp) > +zr_get_zone_serial_digest(zone_register_t *zr, dns_name_t *name, > + isc_uint32_t *serialp, unsigned char ** digestp) > { > isc_result_t result; > - void *zinfo = NULL; > + zone_info_t *zinfo = NULL; > > REQUIRE(zr != NULL); > REQUIRE(name != NULL); > REQUIRE(serialp != NULL); > + REQUIRE(digestp != NULL && *digestp == NULL); > > if (!dns_name_isabsolute(name)) { > log_bug("trying to find zone with a relative name"); > return ISC_R_FAILURE; > } > > RWLOCK(&zr->rwlock, isc_rwlocktype_read); > > - result = dns_rbt_findname(zr->rbt, name, 0, NULL, &zinfo); > - if (result == ISC_R_SUCCESS) > - *serialp = ((zone_info_t *)zinfo)->serial; > + result = dns_rbt_findname(zr->rbt, name, 0, NULL, (void *)&zinfo); > + if (result == ISC_R_SUCCESS) { > + *serialp = zinfo->serial; > + *digestp = zinfo->digest; > + } > > RWUNLOCK(&zr->rwlock, isc_rwlocktype_read); > > return result; > } > > /** > - * Set last SOA serial value processed by autoincrement feature. > + * Set last SOA serial and digest from RRs processed by autoincrement feature. > */ > isc_result_t > -zr_set_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t serial) > +zr_set_zone_serial_digest(zone_register_t *zr, dns_name_t *name, > + isc_uint32_t serial, unsigned char *digest) > { > isc_result_t result; > - void *zinfo = NULL; > + zone_info_t *zinfo = NULL; > > REQUIRE(zr != NULL); > REQUIRE(name != NULL); > + REQUIRE(digest != NULL); > > if (!dns_name_isabsolute(name)) { > log_bug("trying to find zone with a relative name"); > return ISC_R_FAILURE; > } > > RWLOCK(&zr->rwlock, isc_rwlocktype_read); > > - result = dns_rbt_findname(zr->rbt, name, 0, NULL, &zinfo); > - if (result == ISC_R_SUCCESS) > - ((zone_info_t *)zinfo)->serial = serial; > + result = dns_rbt_findname(zr->rbt, name, 0, NULL, (void *)&zinfo); > + if (result == ISC_R_SUCCESS) { > + zinfo->serial = serial; > + memcpy(zinfo->digest, digest, RDLIST_DIGESTLENGTH); > + } > > RWUNLOCK(&zr->rwlock, isc_rwlocktype_read); > > diff --git a/src/zone_register.h b/src/zone_register.h > index 6ac3a92fbe755fcff2d9af3e6e26b32ae83d4133..dea2c9dce054daf1764ba31154627419acada27d 100644 > --- a/src/zone_register.h > +++ b/src/zone_register.h > @@ -49,9 +49,11 @@ isc_mem_t * > zr_get_mctx(zone_register_t *zr); > > isc_result_t > -zr_set_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t serial); > +zr_set_zone_serial_digest(zone_register_t *zr, dns_name_t *name, > + isc_uint32_t serial, unsigned char *digest); > > isc_result_t > -zr_get_zone_serial(zone_register_t *zr, dns_name_t *name, isc_uint32_t *serialp); > +zr_get_zone_serial_digest(zone_register_t *zr, dns_name_t *name, > + isc_uint32_t *serialp, unsigned char ** digestp); > > #endif /* !_LD_ZONE_REGISTER_H_ */ > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Fri Jul 13 13:44:31 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 13 Jul 2012 15:44:31 +0200 Subject: [Freeipa-devel] [PATCH 0029] Add documention for serial_autoincrement feature In-Reply-To: <4FFD7B33.1000800@redhat.com> References: <4FFD7B33.1000800@redhat.com> Message-ID: <20120713134431.GB3004@redhat.com> On Wed, Jul 11, 2012 at 03:10:11PM +0200, Petr Spacek wrote: > Hello, > > this patch adds documention for serial_autoincrement feature to README. Please add note about slave servers. Slave servers should be configured to use only one IPA master when serial_autoincrement is enabled, otherwise things can badly break... Regards, Adam > From 6abf6d54ca1b61e699118813aa24808edbcede0c Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 11 Jul 2012 15:04:50 +0200 > Subject: [PATCH] Add documention for serial_autoincrement feature. > > Signed-off-by: Petr Spacek > --- > README | 9 +++++++++ > 1 files changed, 9 insertions(+), 0 deletions(-) > > diff --git a/README b/README > index 08badc53280af708a95525db76095d1fbe077aa4..fb6eb8ed3a30366f08036dca4c55f6aed7b4f70f 100644 > --- a/README > +++ b/README > @@ -221,6 +221,15 @@ ldap_hostname (default "") > is used and named service has Kerberos principal different from > /bin/hostname output. > > +serial_autoincrement (default no) > + Automatically increment SOA serial number in after each change. > + If serial number is lower than current UNIX timestamp, then > + it is set to the timestamp value. If SOA serial is greater or equal > + to current timestamp, then the serial is incremented by one. > + This function requires persistent search and 4 connections at least. > + In multi-master LDAP environments it is recommended to make > + idnsSOAserial attribute non-replicated (locally significant). > + > sync_ptr (default no) > Set this option to "yes" if you would like to keep PTR record > synchronized with coresponding A/AAAA record for all zones. > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From jcholast at redhat.com Fri Jul 13 13:47:53 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Fri, 13 Jul 2012 15:47:53 +0200 Subject: [Freeipa-devel] [PATCH] 281 Enable SOA serial autoincrement In-Reply-To: <4FFEDE4D.5020007@redhat.com> References: <4FEC7314.9030508@redhat.com> <4FEDFB47.40504@redhat.com> <4FF14377.80609@redhat.com> <4FFEDE4D.5020007@redhat.com> Message-ID: <50002709.4060205@redhat.com> Dne 12.7.2012 16:25, Martin Kosek napsal(a): > On 07/02/2012 08:45 AM, Martin Kosek wrote: >> On 06/29/2012 09:00 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> This patch enables currently developed SOA serial autoincrement feature in >>>> bind-dyndb-ldap. The patch may be updated if any assumptions about this feature >>>> are changed (or somebody finds a bug). >>>> >>>> --- >>>> >>>> SOA serial autoincrement is a requirement for major DNS features, >>>> e.g. zone transfers or DNSSEC. Enable it by default in named.conf >>>> both for new and upgraded installations. Name of the bind-dyndb-ldap >>>> option is "serial_autoincrement". >>>> >>>>> From now on, idnsSOAserial attribute also has to be put to >>>> replication agreement exclude list as serial will be incremented >>>> on each DNS server separately and won't be shared. Exclude list >>>> has to be updated both for new replication agreements and the >>>> current ones. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2554 >>> >>> What version of bind/bind-dyndb-ldap is needed for serial_autoincrement? >>> >>> rob >> >> Such version is not ready yet, there is only a semi-working patch from Petr >> Spacek on freeipa-devel list. >> >> When a working version of bind-dyndb-ldap package with working >> serial_autoincrement feature, it should be enough to simply bump package >> version in bind-dyndb-ldap (that's why I tagged this patch as [WIP]). >> >> But otherwise, this patch is reviewable, it should prepare our install tools >> for the new feature, turn it on in named.conf on upgrades and also update >> replication agreements to not replicate SOA serial from now on. >> >> Martin > > Sending a rebased and updated patch with few more fixes: > 1) Minimum number of connections has been rised to 4 to cover the most recent > requirements for bind-dyndb-ldap's serial_automember feature > 2) ipa-upgradeconfig named.conf has been fixed to not crash when the updated > options are not in the file > > I think that we can choose to push this patch earlier before bind-dyndb-ldap > with serial_automember released. We just need to make sure this patch sets > serial_automember option in named.conf correctly + does the right thing with > replication agreement exclude list update. > > Later on, we would just need to bump bind-dyndb-ldap version in our spec file > when that's released. > > Martin > ACK. I have a couple of nitpicks though: 1) There's a stray ">" in the commit message: ">From now on, idnsSOAserial attribute ..." This is probably caused by the mailing list software. Just make sure you don't include it in the actual commit. 2) There's extra comma in ipa-server-install: - persistent_search=options.persistent_search) + persistent_search=options.persistent_search, + serial_autoincrement=options.serial_autoincrement,) <---- 3) In ipa-upgradeconfig: + else: + psearch = psearch.lower() if psearch is not None else None IMO it would be nicer to do: + elif psearch is not None: + psearch = psearch.lower() or: + else: + psearch = psearch and psearch.lower() instead. Honza -- Jan Cholasta From atkac at redhat.com Fri Jul 13 13:47:15 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 13 Jul 2012 15:47:15 +0200 Subject: [Freeipa-devel] [PATCH 0030] Prevent doubled LDAP queries during nonexistent DNS name lookup In-Reply-To: <4FFD857F.6070708@redhat.com> References: <4FFD857F.6070708@redhat.com> Message-ID: <20120713134714.GC3004@redhat.com> On Wed, Jul 11, 2012 at 03:54:07PM +0200, Petr Spacek wrote: > Hello, > > this patch fixes bug introduced by CVE-2012-2134 fix (commit > cd33194c5a61e98cba53212458cce02b849077ba). > > From cd33194c5a61e98cba53212458cce02b849077ba up to now each query > for nonexistent DNS name results to two (exactly same) LDAP queries. Ack, please push it to master. Regards, Adam > From 965a2f9443fcec2b4e32acf726aaa5a6de5b91c3 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 11 Jul 2012 12:10:16 +0200 > Subject: [PATCH] Prevent doubled LDAP queries during nonexistent DNS name > lookups. This problem was introduced in commit > cd33194c5a61e98cba53212458cce02b849077ba (CVE-2012-2134 > fix). > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 7 ++++++- > 1 files changed, 6 insertions(+), 1 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 7f0a6f4b37171a6fa4db79cd32fdd8bc62288e0f..b06036f89fdf088e2d3c3ef964165d23c2d20172 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -1618,6 +1618,7 @@ ldap_query(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn, > isc_result_t result; > int cnt; > int ret; > + int ldap_err_code; > int once = 0; > > REQUIRE(ldap_conn != NULL); > @@ -1661,8 +1662,12 @@ retry: > return ISC_R_SUCCESS; > } > > + ret = ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > + (void *)&ldap_err_code); > + if (ret == LDAP_OPT_SUCCESS && ldap_err_code == LDAP_NO_SUCH_OBJECT) > + return ISC_R_NOTFOUND; > /* some error happened during ldap_search, try to recover */ > - if (!once) { > + else if (!once) { > once++; > result = handle_connection_error(ldap_inst, ldap_conn, > ISC_FALSE); > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Fri Jul 13 13:48:45 2012 From: atkac at redhat.com (Adam Tkac) Date: Fri, 13 Jul 2012 15:48:45 +0200 Subject: [Freeipa-devel] [PATCH 0031] Prevent crashes in ldap_pool_*() function family In-Reply-To: <4FFEEACB.5010603@redhat.com> References: <4FFEEACB.5010603@redhat.com> Message-ID: <20120713134845.GD3004@redhat.com> On Thu, Jul 12, 2012 at 05:18:35PM +0200, Petr Spacek wrote: > Hello, > > this patch fixes occasional crashes caused by incorrect error > handling in ldap_pool_*() functions. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/84 > > It can be caused by memory allocation error OR timeout during > connection establishing phase. > > To trigger this problem first connection has to be established > properly and some other connection has to fail. It is not enough to > timeout at first connection/try, that case was handled properly. Ack, please push it to master. Regards, Adam > From 7ef5c14ffa69cc4d60a76c9db63b8e3ce065d27b Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Thu, 12 Jul 2012 17:10:58 +0200 > Subject: [PATCH] Prevent crashes in ldap_pool_*() function family. > > https://fedorahosted.org/bind-dyndb-ldap/ticket/84 > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 18 +++++++++++------- > 1 files changed, 11 insertions(+), 7 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index aa7f97664d3fd6de43b0ee7b7e6caa0fc0e25dde..dc18d8d51c4980448fa78ae1604c78782d601113 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -538,6 +538,7 @@ destroy_ldap_instance(ldap_instance_t **ldap_instp) > > dns_rbtnodechain_invalidate(&chain); > > + /* TODO: Terminate psearch watcher sooner? */ > if (ldap_inst->psearch && ldap_inst->watcher != 0) { > ldap_inst->exiting = ISC_TRUE; > /* > @@ -628,9 +629,12 @@ destroy_ldap_connection(ldap_pool_t *pool, ldap_connection_t **ldap_connp) > { > ldap_connection_t *ldap_conn; > > - REQUIRE(ldap_connp != NULL && *ldap_connp != NULL); > + REQUIRE(ldap_connp != NULL); > > ldap_conn = *ldap_connp; > + if (ldap_conn == NULL) > + return; > + > DESTROYLOCK(&ldap_conn->lock); > if (ldap_conn->handle != NULL) > ldap_unbind_ext_s(ldap_conn->handle, NULL, NULL); > @@ -2603,20 +2607,21 @@ ldap_pool_create(isc_mem_t *mctx, unsigned int connections, ldap_pool_t **poolp) > return ISC_R_SUCCESS; > > cleanup: > - if (pool != NULL) > - ldap_pool_destroy(&pool); > + ldap_pool_destroy(&pool); > return result; > } > static void > ldap_pool_destroy(ldap_pool_t **poolp) > { > ldap_pool_t *pool; > ldap_connection_t *ldap_conn; > unsigned int i; > > - REQUIRE(poolp != NULL && *poolp != NULL); > + REQUIRE(poolp != NULL); > > pool = *poolp; > + if (pool == NULL) > + return; > > for (i = 0; i < pool->connections; i++) { > ldap_conn = pool->conns[i]; > @@ -2630,6 +2635,7 @@ ldap_pool_destroy(ldap_pool_t **poolp) > semaphore_destroy(&pool->conn_semaphore); > > MEM_PUT_AND_DETACH(pool); > + *poolp = NULL; > } > > static isc_result_t > @@ -2701,9 +2707,7 @@ ldap_pool_connect(ldap_pool_t *pool, ldap_instance_t *ldap_inst) > > cleanup: > for (i = 0; i < pool->connections; i++) { > - ldap_conn = pool->conns[i]; > - if (ldap_conn != NULL) > - destroy_ldap_connection(pool, &ldap_conn); > + destroy_ldap_connection(pool, &pool->conns[i]); > } > return result; > } > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From rcritten at redhat.com Fri Jul 13 14:00:16 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 13 Jul 2012 10:00:16 -0400 Subject: [Freeipa-devel] [PATCH] 286-288 Warn when ID range with incorrect size was created In-Reply-To: <4FFEA08C.3000404@redhat.com> References: <4FFD99B6.8030800@redhat.com> <4FFDD3AB.6080903@redhat.com> <4FFE649C.3040908@redhat.com> <4FFEA08C.3000404@redhat.com> Message-ID: <500029F0.9060400@redhat.com> Martin Kosek wrote: > On 07/12/2012 07:46 AM, Martin Kosek wrote: >> On 07/11/2012 09:27 PM, Rob Crittenden wrote: >>> Martin Kosek wrote: >>>> IPA 3.0 introduced range ID objects in replicated space which specify >>>> a range of IDs assigned via DNA plugin. ipa-ldap-updater generates the >>>> default ID range which should correspond with IDs assigned to IPA >>>> users. >>>> >>>> However, since correct range size is not known, we should at least >>>> warn that a range with invalid size was created so that user can >>>> amend it. >>>> >>>> >>>> I created 2 new tickets to add further improve this area: >>>> >>>> 1) #2918: [doc] Upgrade procedure section should mention ipa-ldap-updater >>>> 2) #2919: Improve safety checks in range command >>>> >>>> >>>> To test this patch, you can: >>>> 1) Install unpatched IPA server (and you may install replicas too) with custom >>>> --idstart and --idmax options where difference is greater then 200000 >>>> 2) Remove default range with range-del command (will be restored during upgrade) >>>> 3) Run RPM upgrade with RPMs built from patched sources - ERROR should now be >>>> printed during update stating that a new range was created but its size is not >>>> right >>> >>> I don't understand step 2, why would someone remove their range before upgrading? >>> >>> I installed with a 50k range, didn't remove it, then upgraded with no warning. >>> I deleted the range and re-installed the packages again, still no warning but a >>> new 200k range was created for me. >>> >>> rob >> >> The step 2 is artificial and is only done to force the default_range update >> plugin to create/restore the default IPA range. The plugin would just be >> skipped otherwise. >> >> We can only detect ranges larger than 200k - judging just from the number of >> free IDs. Thus, 50k range will pass without any warning or error. If you create >> a bigger range (this can be detected unless you deplete all IDs below 200k >> mark), you will receive the warning. All this procedure will not handle all >> situations ATM, its just heuristics to cover most cases... >> >> Martin > > Sending an updated patch with 2 small changes: > 1) Console error formatting was changed similar to ipa-client-install > 2) ipa-ldap-updater does not print information message when IPA is not > configured to stderr so that rpm update output stays clean when updating rpms > in machine without IPA installed > > This is the output of RPM with the new patch set: > > # ipa range-del IDM.LAB.BOS.REDHAT.COM_id_range > -------------------------------------------------- > Deleted ID range "IDM.LAB.BOS.REDHAT.COM_id_range" > -------------------------------------------------- > # rpm -Uvh --force freeipa-* > Preparing... ########################################### [100%] > 1:freeipa-python ########################################### [ 14%] > 2:freeipa-client ########################################### [ 29%] > 3:freeipa-admintools ########################################### [ 43%] > 4:freeipa-server ########################################### [ 57%] > 5:freeipa-server-selinux ########################################### [ 71%] > 6:freeipa-server-trust-ad########################################### [ 86%] > 7:freeipa-debuginfo ########################################### [100%] > ERROR: default_range: could not verify default ID range size > Please use the following command to set correct ID range size > $ ipa range-mod IDM.LAB.BOS.REDHAT.COM_id_range --range-size=RANGE_SIZE > RANGE_SIZE may be computed from --idstart and --idmax options used during IPA > server installation: > RANGE_SIZE = (--idmax) - (--idstart) + 1 > > Martin > Your sys.exit() changes to ipa-ldap-updater cause the return val to be 0 when IPA is not configured. It should return 1. Fix that and ACK. rob From mkosek at redhat.com Fri Jul 13 14:14:08 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 13 Jul 2012 16:14:08 +0200 Subject: [Freeipa-devel] [PATCH] 281 Enable SOA serial autoincrement In-Reply-To: <50002709.4060205@redhat.com> References: <4FEC7314.9030508@redhat.com> <4FEDFB47.40504@redhat.com> <4FF14377.80609@redhat.com> <4FFEDE4D.5020007@redhat.com> <50002709.4060205@redhat.com> Message-ID: <50002D30.1020602@redhat.com> On 07/13/2012 03:47 PM, Jan Cholasta wrote: > Dne 12.7.2012 16:25, Martin Kosek napsal(a): >> On 07/02/2012 08:45 AM, Martin Kosek wrote: >>> On 06/29/2012 09:00 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> This patch enables currently developed SOA serial autoincrement feature in >>>>> bind-dyndb-ldap. The patch may be updated if any assumptions about this >>>>> feature >>>>> are changed (or somebody finds a bug). >>>>> >>>>> --- >>>>> >>>>> SOA serial autoincrement is a requirement for major DNS features, >>>>> e.g. zone transfers or DNSSEC. Enable it by default in named.conf >>>>> both for new and upgraded installations. Name of the bind-dyndb-ldap >>>>> option is "serial_autoincrement". >>>>> >>>>>> From now on, idnsSOAserial attribute also has to be put to >>>>> replication agreement exclude list as serial will be incremented >>>>> on each DNS server separately and won't be shared. Exclude list >>>>> has to be updated both for new replication agreements and the >>>>> current ones. >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2554 >>>> >>>> What version of bind/bind-dyndb-ldap is needed for serial_autoincrement? >>>> >>>> rob >>> >>> Such version is not ready yet, there is only a semi-working patch from Petr >>> Spacek on freeipa-devel list. >>> >>> When a working version of bind-dyndb-ldap package with working >>> serial_autoincrement feature, it should be enough to simply bump package >>> version in bind-dyndb-ldap (that's why I tagged this patch as [WIP]). >>> >>> But otherwise, this patch is reviewable, it should prepare our install tools >>> for the new feature, turn it on in named.conf on upgrades and also update >>> replication agreements to not replicate SOA serial from now on. >>> >>> Martin >> >> Sending a rebased and updated patch with few more fixes: >> 1) Minimum number of connections has been rised to 4 to cover the most recent >> requirements for bind-dyndb-ldap's serial_automember feature >> 2) ipa-upgradeconfig named.conf has been fixed to not crash when the updated >> options are not in the file >> >> I think that we can choose to push this patch earlier before bind-dyndb-ldap >> with serial_automember released. We just need to make sure this patch sets >> serial_automember option in named.conf correctly + does the right thing with >> replication agreement exclude list update. >> >> Later on, we would just need to bump bind-dyndb-ldap version in our spec file >> when that's released. >> >> Martin >> > > ACK. > > I have a couple of nitpicks though: > > 1) There's a stray ">" in the commit message: > > ">From now on, idnsSOAserial attribute ..." > > This is probably caused by the mailing list software. Just make sure you don't > include it in the actual commit. Yeah, made sure it did not get to the commit/push. > > 2) There's extra comma in ipa-server-install: > > - persistent_search=options.persistent_search) > + persistent_search=options.persistent_search, > + serial_autoincrement=options.serial_autoincrement,) <---- Fixed. > > 3) In ipa-upgradeconfig: > > + else: > + psearch = psearch.lower() if psearch is not None else None > > IMO it would be nicer to do: > > + elif psearch is not None: > + psearch = psearch.lower() > > or: > > + else: > + psearch = psearch and psearch.lower() > > instead. I did not like the last option very much, it is not so clear from the line what we are actually doing. So I at least reordered the if-then-else clause so that it can be be read better. Pushed to master. I did not close the ticket though, we can close it when the bind-dyndb-ldap version in the spec file is bumped. Martin From mkosek at redhat.com Fri Jul 13 14:21:30 2012 From: mkosek at redhat.com (Martin Kosek) Date: Fri, 13 Jul 2012 16:21:30 +0200 Subject: [Freeipa-devel] [PATCH] 286-288 Warn when ID range with incorrect size was created In-Reply-To: <500029F0.9060400@redhat.com> References: <4FFD99B6.8030800@redhat.com> <4FFDD3AB.6080903@redhat.com> <4FFE649C.3040908@redhat.com> <4FFEA08C.3000404@redhat.com> <500029F0.9060400@redhat.com> Message-ID: <50002EEA.60003@redhat.com> On 07/13/2012 04:00 PM, Rob Crittenden wrote: > Martin Kosek wrote: >> On 07/12/2012 07:46 AM, Martin Kosek wrote: >>> On 07/11/2012 09:27 PM, Rob Crittenden wrote: >>>> Martin Kosek wrote: >>>>> IPA 3.0 introduced range ID objects in replicated space which specify >>>>> a range of IDs assigned via DNA plugin. ipa-ldap-updater generates the >>>>> default ID range which should correspond with IDs assigned to IPA >>>>> users. >>>>> >>>>> However, since correct range size is not known, we should at least >>>>> warn that a range with invalid size was created so that user can >>>>> amend it. >>>>> >>>>> >>>>> I created 2 new tickets to add further improve this area: >>>>> >>>>> 1) #2918: [doc] Upgrade procedure section should mention ipa-ldap-updater >>>>> 2) #2919: Improve safety checks in range command >>>>> >>>>> >>>>> To test this patch, you can: >>>>> 1) Install unpatched IPA server (and you may install replicas too) with >>>>> custom >>>>> --idstart and --idmax options where difference is greater then 200000 >>>>> 2) Remove default range with range-del command (will be restored during >>>>> upgrade) >>>>> 3) Run RPM upgrade with RPMs built from patched sources - ERROR should now be >>>>> printed during update stating that a new range was created but its size is >>>>> not >>>>> right >>>> >>>> I don't understand step 2, why would someone remove their range before >>>> upgrading? >>>> >>>> I installed with a 50k range, didn't remove it, then upgraded with no warning. >>>> I deleted the range and re-installed the packages again, still no warning >>>> but a >>>> new 200k range was created for me. >>>> >>>> rob >>> >>> The step 2 is artificial and is only done to force the default_range update >>> plugin to create/restore the default IPA range. The plugin would just be >>> skipped otherwise. >>> >>> We can only detect ranges larger than 200k - judging just from the number of >>> free IDs. Thus, 50k range will pass without any warning or error. If you create >>> a bigger range (this can be detected unless you deplete all IDs below 200k >>> mark), you will receive the warning. All this procedure will not handle all >>> situations ATM, its just heuristics to cover most cases... >>> >>> Martin >> >> Sending an updated patch with 2 small changes: >> 1) Console error formatting was changed similar to ipa-client-install >> 2) ipa-ldap-updater does not print information message when IPA is not >> configured to stderr so that rpm update output stays clean when updating rpms >> in machine without IPA installed >> >> This is the output of RPM with the new patch set: >> >> # ipa range-del IDM.LAB.BOS.REDHAT.COM_id_range >> -------------------------------------------------- >> Deleted ID range "IDM.LAB.BOS.REDHAT.COM_id_range" >> -------------------------------------------------- >> # rpm -Uvh --force freeipa-* >> Preparing... ########################################### [100%] >> 1:freeipa-python ########################################### [ 14%] >> 2:freeipa-client ########################################### [ 29%] >> 3:freeipa-admintools ########################################### [ 43%] >> 4:freeipa-server ########################################### [ 57%] >> 5:freeipa-server-selinux ########################################### [ 71%] >> 6:freeipa-server-trust-ad########################################### [ 86%] >> 7:freeipa-debuginfo ########################################### [100%] >> ERROR: default_range: could not verify default ID range size >> Please use the following command to set correct ID range size >> $ ipa range-mod IDM.LAB.BOS.REDHAT.COM_id_range --range-size=RANGE_SIZE >> RANGE_SIZE may be computed from --idstart and --idmax options used during IPA >> server installation: >> RANGE_SIZE = (--idmax) - (--idstart) + 1 >> >> Martin >> > > Your sys.exit() changes to ipa-ldap-updater cause the return val to be 0 when > IPA is not configured. It should return 1. > > Fix that and ACK. > > rob Fixed and pushed all 3 to master. Martin From rcritten at redhat.com Fri Jul 13 15:01:13 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 13 Jul 2012 11:01:13 -0400 Subject: [Freeipa-devel] [PATCH] 290 Enforce CNAME constrains for DNS commands In-Reply-To: <4FFFFB3A.2010806@redhat.com> References: <4FFFFB3A.2010806@redhat.com> Message-ID: <50003839.1030501@redhat.com> Martin Kosek wrote: > RFC 1912 states that no record (besides PTR) is allowed to coexist > with any other record type. When BIND detects this situation, it > refuses to load such records. > > Enforce the constrain for dnsrecord-mod and dnsrecord-add commands. > > https://fedorahosted.org/freeipa/ticket/2601 ACK, pushed to master rob From dpal at redhat.com Fri Jul 13 15:08:46 2012 From: dpal at redhat.com (Dmitri Pal) Date: Fri, 13 Jul 2012 11:08:46 -0400 Subject: [Freeipa-devel] Multitenancy in FreeIPA 3.0x In-Reply-To: References: Message-ID: <500039FE.1@redhat.com> On 07/13/2012 06:45 AM, Klaus Eckel wrote: > Hey All , > > where can to try an Multitenancy IPA 3.0 system or change , config > > my hope .. one ldap-system which sub system > > my hope .. for only Tenant one KDC over other port .. ( same linux > system ) > > Can i try that ! We did not go further than thew following thread: https://www.redhat.com/archives/freeipa-devel/2011-December/msg00282.html And wiki page: http://freeipa.org/page/Multitenancy It is a very profound set of changes but if you are interested in implementing them we will be definitely interested in helping you. Here is also an extensibility guide that one would need to be able to extend IPA. https://fedorahosted.org/freeipa/browser/doc/guide?rev=d09389ab6fe203ce93d1a68986ff93c8ad75a480 A compiled version also attached. > > Klaus > > Best Regards, > Klaus Eckel > , > UNIX > Consultant HPC (AIX,Linux) GPFS, BIA, SAP > ITS/STG (SSIS) > Server, Storage & Data Infrastructure Services IBM Deutschland GmbH > > Laatzener str, 1 > 30539 Hannover > Germany Email: keckel at de.ibm.com > Phone: +49-(0)52319489906 > Handy: +49 (0)170 6323416 > > > Visit the IBM Deutschland ITS Pages. > > > IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Erich Clementi > Gesch?ftsf?hrung: Martin Jetter (Vorsitzender), Reinhard Reschke, > Dieter Scholz, Klaus Lintelmann, Michael Diemer, Martina Koederitz > Sitz der Gesellschaft: > Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 > WEEE-Reg.-Nr. DE 99369940 > > > _______________________________________________ > Freeipa-devel mailing list > Freeipa-devel at redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel -- Thank you, Dmitri Pal Sr. Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-extensibility.pdf Type: application/pdf Size: 355019 bytes Desc: not available URL: From abokovoy at redhat.com Fri Jul 13 15:23:01 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 13 Jul 2012 18:23:01 +0300 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user Message-ID: <20120713152301.GJ9427@redhat.com> Hi, when adding AD trusts support, we need to ensure we have valid kerberos ticket of the user from 'admins' group or otherwise appropriate ACIs will not be granted. This patch introduces a check for that. We already check if ipa-adtrust-install is run by root so this complements existing checks. https://fedorahosted.org/freeipa/ticket/2815 -- / Alexander Bokovoy -------------- next part -------------- >From 4a439e86c26f7a640063d4b20beaf35e6a2967c9 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/2] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration - kinit-ed IPA admin user, to ensure proper ACIs are granted https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..d03657118995022cbf1c34149bc5528a628a71ea 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -86,6 +86,29 @@ def read_netbios_name(netbios_default): return netbios_name +def ensure_kerberos_admin_rights(api): + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + api.Backend.ldap2.connect(ccache.name) + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + api.Backend.ldap2.disconnect() + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except Exception, e: + error_messages = dict( + Krb5Error = "Must have Kerberos credentials to setup AD trusts on server", + RequirementError = "Must have administrative privileges to setup AD trusts on server" + ) + name = type(e).__name__ + if name in error_messages: + sys.exit(error_messages[name]) + else: + sys.exit("Unrecognized error during check of admin rights: %s" % (str(e))) + def main(): safe_options, options = parse_options() @@ -128,6 +151,8 @@ def main(): api.bootstrap(**cfg) api.finalize() + ensure_kerberos_admin_rights(api) + if adtrustinstance.ipa_smb_conf_exists(): if not options.unattended: while True: -- 1.7.10.4 From pspacek at redhat.com Fri Jul 13 15:35:46 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 13 Jul 2012 17:35:46 +0200 Subject: [Freeipa-devel] [PATCH] 0025-0028 Implement SOA serial number increments for external changes In-Reply-To: <20120713134245.GA3004@redhat.com> References: <4FFC34C4.6050902@redhat.com> <20120713134245.GA3004@redhat.com> Message-ID: <50004052.6030007@redhat.com> On 07/13/2012 03:42 PM, Adam Tkac wrote: > On Tue, Jul 10, 2012 at 03:57:24PM +0200, Petr Spacek wrote: >> Hello, >> >> these patches provides SOA serial auto-increment feature for external changes. >> Related ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/67 >> >> It is necessary to set "psearch" AND "serial_autoincrement" to "yes" >> in /etc/named.conf to enable this feature. >> >> In replicated environment idnsSOAserial attribute has to be declared >> as non-replicated. It is done by mkosek's patch 281 for 389 DS & >> FreeIPA. >> >> For testing purposes it is enough to add "idnsSOAserial" to end of >> exclude list in nsDS5ReplicatedAttributeList attribute for each >> replication agreement located in cn=mapping tree,cn=config subtree. >> >> >> My patch 28 contains "trick" necessary for replicated environments >> with 389 DS. 389 sends entry change notification (ECN) in cases when >> non-replicated attribute idnsSOAserial was changed on *other side*. >> In that case no change is visible in DNS attributes, but ECN is sent >> by 389. (Attribute modifyTimestamp is changed also.) >> >> Patch 28 computes digest/hash from all resource records in idnsZone >> object and compares old and new digest after each received ECN. This >> approach eliminates "false changes". >> >> Each patch depends on all preceding patches, but each patch >> implements visible (and testable) part of functionality. > > Hello Peter, > > please check my comments below. > > Regards, Adam Hello, I did all changes except this one: >> +unsigned int >> +rdatalist_length(const dns_rdatalist_t *rdlist) > > There is no reason to have this function exported, please mark it as static (and > probably also as inline). rdatalist_length() is used from rdlist.c and ldap_driver.c. Rebased patches were pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/99663b6d65cf5dc166b3cb6f83be1878b8de3163 https://fedorahosted.org/bind-dyndb-ldap/changeset/cd37fba03c5c86a766d1a9f893036ac3540e8b7c https://fedorahosted.org/bind-dyndb-ldap/changeset/9a3f29c12db99597222ffa2bf0713d0b00cb4699 https://fedorahosted.org/bind-dyndb-ldap/changeset/c379d81508fbfa00ecb5da727ff7b097ebb29a3d https://fedorahosted.org/bind-dyndb-ldap/changeset/93ae7491a80ba8c4789f8770e14c053b67176de4 Petr^2 Spacek From pspacek at redhat.com Fri Jul 13 15:38:48 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 13 Jul 2012 17:38:48 +0200 Subject: [Freeipa-devel] [PATCH 0030] Prevent doubled LDAP queries during nonexistent DNS name lookup In-Reply-To: <20120713134714.GC3004@redhat.com> References: <4FFD857F.6070708@redhat.com> <20120713134714.GC3004@redhat.com> Message-ID: <50004108.8050307@redhat.com> On 07/13/2012 03:47 PM, Adam Tkac wrote: > On Wed, Jul 11, 2012 at 03:54:07PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch fixes bug introduced by CVE-2012-2134 fix (commit >> cd33194c5a61e98cba53212458cce02b849077ba). >> >> From cd33194c5a61e98cba53212458cce02b849077ba up to now each query >> for nonexistent DNS name results to two (exactly same) LDAP queries. > > Ack, please push it to master. > > Regards, Adam > >> From 965a2f9443fcec2b4e32acf726aaa5a6de5b91c3 Mon Sep 17 00:00:00 2001 >> From: Petr Spacek >> Date: Wed, 11 Jul 2012 12:10:16 +0200 >> Subject: [PATCH] Prevent doubled LDAP queries during nonexistent DNS name >> lookups. This problem was introduced in commit >> cd33194c5a61e98cba53212458cce02b849077ba (CVE-2012-2134 >> fix). >> Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/d673f5b54132a14798ec8a355be6cf4911fe10d1 Petr^2 Spacek From pspacek at redhat.com Fri Jul 13 15:42:00 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 13 Jul 2012 17:42:00 +0200 Subject: [Freeipa-devel] [PATCH 0031] Prevent crashes in ldap_pool_*() function family In-Reply-To: <20120713134845.GD3004@redhat.com> References: <4FFEEACB.5010603@redhat.com> <20120713134845.GD3004@redhat.com> Message-ID: <500041C8.9080501@redhat.com> On 07/13/2012 03:48 PM, Adam Tkac wrote: > On Thu, Jul 12, 2012 at 05:18:35PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch fixes occasional crashes caused by incorrect error >> handling in ldap_pool_*() functions. >> >> https://fedorahosted.org/bind-dyndb-ldap/ticket/84 >> >> It can be caused by memory allocation error OR timeout during >> connection establishing phase. >> >> To trigger this problem first connection has to be established >> properly and some other connection has to fail. It is not enough to >> timeout at first connection/try, that case was handled properly. > > Ack, please push it to master. > > Regards, Adam Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/e44ce4d9c42ad9b1226cea5b62e9040f2d7e4df2 Thanks for your time! Petr^2 Spacek From abokovoy at redhat.com Fri Jul 13 17:33:54 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 13 Jul 2012 20:33:54 +0300 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <20120713152301.GJ9427@redhat.com> References: <20120713152301.GJ9427@redhat.com> Message-ID: <20120713173354.GK9427@redhat.com> On Fri, 13 Jul 2012, Alexander Bokovoy wrote: > Hi, > > when adding AD trusts support, we need to ensure we have valid kerberos > ticket of the user from 'admins' group or otherwise appropriate ACIs > will not be granted. > > This patch introduces a check for that. We already check if > ipa-adtrust-install is run by root so this complements existing checks. > > https://fedorahosted.org/freeipa/ticket/2815 After discussing on IRC with Simo and Rob, we came to conclusion that it is possible to switch to LDAPI and autobind feature of dirsrv for authentication and remove requirement for Directory Manager credentials altogether. Updated patch makes use of LDAPI + autobind under root privileges to map automatically to Directory Manager privileges in dirsrv. Additionally it ensures we have Kerberos credentials to fetch keytab with CIFS service key. Service._ldap_mod() is extended to switch to autobind when self.ldapi is set to True and we are running as root. For those interested in why ACIError is mapped to 'outdated Kerberos credentials' error message, this is because we'll get ACIError for 'ipa user-show ' command when authenticated by the Kerberos credentials for in a default ccache only when Kerberos credentials are stale -- either belong to a user that was removed or to a previous IPA install that was wiped before reinstalling. The latter is how I discovered this case. :) -- / Alexander Bokovoy -------------- next part -------------- >From 7e4216272fffe55a8f87c51c418895fe6b4dbd73 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/2] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 42 +++++++++++++++++++++++-------- install/tools/man/ipa-adtrust-install.1 | 3 --- ipaserver/install/adtrustinstance.py | 21 ++++++++-------- ipaserver/install/service.py | 15 +++++++++-- 4 files changed, 55 insertions(+), 26 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..f367d5b2b516bd411bce9275ff299eb3ffdf6bf9 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -86,6 +84,30 @@ def read_netbios_name(netbios_default): return netbios_name +def ensure_kerberos_admin_rights(api): + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + api.Backend.ldap2.connect(ccache.name) + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + api.Backend.ldap2.disconnect() + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except Exception, e: + error_messages = dict( + ACIError = "Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket", + Krb5Error = "Must have Kerberos credentials to setup AD trusts on server", + RequirementError = "Must have administrative privileges to setup AD trusts on server" + ) + name = type(e).__name__ + if name in error_messages: + sys.exit(error_messages[name]) + else: + sys.exit("Unrecognized error during check of admin rights: %s\n%s" % (name, str(e))) + def main(): safe_options, options = parse_options() @@ -128,6 +150,8 @@ def main(): api.bootstrap(**cfg) api.finalize() + ensure_kerberos_admin_rights(api) + if adtrustinstance.ipa_smb_conf_exists(): if not options.unattended: while True: @@ -194,9 +218,8 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm # try the connection try: @@ -205,12 +228,9 @@ def main(): except ldap.INVALID_CREDENTIALS, e: sys.exit("Password is not valid!") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + # See if our LDAP server is up and we can talk to it over GSSAPI + ccache = krbV.default_context().default_ccache().name + api.Backend.ldap2.connect(ccache) smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..f96c21005ee69675ced4513b1b37272715482336 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -107,15 +107,26 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # Support autobind when running as root and using ldapi + if self.ldapi: + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + else: + args += ["-h", hostname] + if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + auth_params = [] + if not (os.getegid() == 0 and self.ldapi): + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: -- 1.7.10.4 From pvoborni at redhat.com Mon Jul 16 08:56:55 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 16 Jul 2012 10:56:55 +0200 Subject: [Freeipa-devel] [PATCH] 171 Fixed display of attributes_widget in IE9 Message-ID: <5003D757.9090409@redhat.com> Attributes widget is using overflow css rule in tbody element. IE9 doesn't handle it well. To fix the issue, attributes widget was slightly modified and conditional css stylesheet was added just for fixing IE problems. https://fedorahosted.org/freeipa/ticket/2822 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0171-Fixed-display-of-attributes_widget-in-IE9.patch Type: text/x-patch Size: 4391 bytes Desc: not available URL: From pvoborni at redhat.com Mon Jul 16 11:53:52 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 16 Jul 2012 13:53:52 +0200 Subject: [Freeipa-devel] [PATCH] 172 Bigger textarea for permission type=subtree Message-ID: <500400D0.4090604@redhat.com> Patch description: Adder dialog and details facet for permission type=subtree have small textarea for defining subtree filter. It was unconfortable to define the filter. This difference was removed. https://fedorahosted.org/freeipa/ticket/2832 Note regarding related ticket: Resizable textareas are browser-specific. We can do it in code too but I don't think it is worth the effort. Textarea in IE still seems smaller than in Firefox, but it has the same number of rows and cols. I think it has enough space for defining the filter so it should be fixing the problem. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0172-Bigger-textarea-for-permission-type-subtree.patch Type: text/x-patch Size: 987 bytes Desc: not available URL: From abokovoy at redhat.com Mon Jul 16 13:12:59 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 16 Jul 2012 16:12:59 +0300 Subject: [Freeipa-devel] [PATCH] 0061 ValidationError takes 'error' named argument, not 'reason' Message-ID: <20120716131259.GA18370@redhat.com> Hi, https://fedorahosted.org/freeipa/ticket/2865 -- / Alexander Bokovoy -------------- next part -------------- >From afb200bbd4054b85f72a9cda8105ca07ee4deb2a Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 16 Jul 2012 12:43:47 +0300 Subject: [PATCH 3/4] ipalib/plugins/trust.py: ValidationError takes 'error' named argument, not 'reason' https://fedorahosted.org/freeipa/ticket/2865 --- ipalib/plugins/trust.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ipalib/plugins/trust.py b/ipalib/plugins/trust.py index 40bd93e654c0365ad202abfd82e84345583459dd..2932835e038d99d9c48f1822e76fbc2e1570f92f 100644 --- a/ipalib/plugins/trust.py +++ b/ipalib/plugins/trust.py @@ -182,13 +182,13 @@ class trust_add(LDAPCreate): realm_admin = options['realm_admin'] if 'realm_passwd' not in options: - raise errors.ValidationError(name=_('AD Trust setup'), reason=_('Realm administrator password should be specified')) + raise errors.ValidationError(name=_('AD Trust setup'), error=_('Realm administrator password should be specified')) realm_passwd = options['realm_passwd'] result = trustinstance.join_ad_full_credentials(keys[-1], realm_server, realm_admin, realm_passwd) if result is None: - raise errors.ValidationError(name=_('AD Trust setup'), reason=_('Unable to verify write permissions to the AD')) + raise errors.ValidationError(name=_('AD Trust setup'), error=_('Unable to verify write permissions to the AD')) return dict(result=dict(), value=trustinstance.remote_domain.info['dns_domain']) @@ -198,7 +198,7 @@ class trust_add(LDAPCreate): if 'trust_secret' in options: result = trustinstance.join_ad_ipa_half(keys[-1], realm_server, options['trust_secret']) return dict(result=dict(), value=trustinstance.remote_domain.info['dns_domain']) - raise errors.ValidationError(name=_('AD Trust setup'), reason=_('Not enough arguments specified to perform trust setup')) + raise errors.ValidationError(name=_('AD Trust setup'), error=_('Not enough arguments specified to perform trust setup')) class trust_del(LDAPDelete): __doc__ = _('Delete a trust.') -- 1.7.10.4 From abokovoy at redhat.com Mon Jul 16 13:14:10 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 16 Jul 2012 16:14:10 +0300 Subject: [Freeipa-devel] [PATCH] 0062 support various forms of user account when establishing trusts Message-ID: <20120716131410.GB18370@redhat.com> Hi, Realm administrator account may be specified using different form: Administrator, DOM\Administrator, Administrator at DOMAIN This patch introduces handling of the second two forms: - In DOM\Administrator only user name is used, short domain name is then taken from a discovered record from the AD DC - In Administrator at DOMAIN first DOMAIN is verified to be the same as the domain we are establishing trust to, and then user name is taken, together with short domain name taken from a discovered record from the AD DC Note that we do not support using to-be-trusted domain's trusted domains' accounts to establish trust as there is basically zero chance to verify that things will work with them. In addition, in order to establish trust one needs to belong to Enterprise Admins group in AD or have specially delegated permissions. These permissions are unlikely delegated to the ones in already trusted domain. https://fedorahosted.org/freeipa/ticket/2864 -- / Alexander Bokovoy -------------- next part -------------- >From 3365e3501a1cdd13d3741fc791c7843839a5a058 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Mon, 16 Jul 2012 13:12:42 +0300 Subject: [PATCH 4/4] Handle various forms of admin accounts when establishing trusts Realm administrator account may be specified using different form: Administrator, DOM\Administrator, Administrator at DOMAIN This patch introduces handling of the second two forms: - In DOM\Administrator only user name is used, short domain name is then taken from a discovered record from the AD DC - In Administrator at DOMAIN first DOMAIN is verified to be the same as the domain we are establishing trust to, and then user name is taken, together with short domain name taken from a discovered record from the AD DC Note that we do not support using to-be-trusted domain's trusted domains' accounts to establish trust as there is basically zero chance to verify that things will work with them. In addition, in order to establish trust one needs to belong to Enterprise Admins group in AD or have specially delegated permissions. These permissions are unlikely delegated to the ones in already trusted domain. https://fedorahosted.org/freeipa/ticket/2864 --- ipalib/plugins/trust.py | 8 ++++++++ ipaserver/dcerpc.py | 5 +++++ 2 files changed, 13 insertions(+) diff --git a/ipalib/plugins/trust.py b/ipalib/plugins/trust.py index 2932835e038d99d9c48f1822e76fbc2e1570f92f..792e6cac2a2f9ebb61f84cc74d01be325995863e 100644 --- a/ipalib/plugins/trust.py +++ b/ipalib/plugins/trust.py @@ -180,6 +180,14 @@ class trust_add(LDAPCreate): # generate random trustdom password to do work on both sides if 'realm_admin' in options: realm_admin = options['realm_admin'] + names = realm_admin.split('@') + if len(names) > 1: + # realm admin name is in UPN format, user at realm, check that + # realm is the same as the one that we are attempting to trust + if keys[-1].lower() != names[-1].lower(): + raise errors.ValidationError(name=_('AD Trust setup'), + error=_('Trusted domain and administrator account use different realms')) + realm_admin = names[0] if 'realm_passwd' not in options: raise errors.ValidationError(name=_('AD Trust setup'), error=_('Realm administrator password should be specified')) diff --git a/ipaserver/dcerpc.py b/ipaserver/dcerpc.py index 07e40c2d35b41a2665232f3e6d853b47aef707bb..6b830f65b854b74fcf080b071212e7658f334adf 100644 --- a/ipaserver/dcerpc.py +++ b/ipaserver/dcerpc.py @@ -363,6 +363,11 @@ class TrustDomainJoins(object): rd.read_only = True if realm_admin and realm_passwd: if 'name' in rd.info: + names = realm_admin.split('\\') + if len(names) > 1: + # realm admin is in DOMAIN\user format + # strip DOMAIN part as we'll enforce the one discovered + realm_admin = names[-1] auth_string = u"%s\%s%%%s" % (rd.info['name'], realm_admin, realm_passwd) td = get_instance(self) td.creds.parse_string(auth_string) -- 1.7.10.4 From rcritten at redhat.com Mon Jul 16 13:23:24 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 16 Jul 2012 09:23:24 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates Message-ID: <500415CC.9080706@redhat.com> Use the new certmonger capability to be able to renew the dogtag subsystem certificates (audit, OCSP, etc). See the ticket for full details. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1033-renewal.patch Type: text/x-diff Size: 42405 bytes Desc: not available URL: From rcritten at redhat.com Mon Jul 16 14:49:59 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 16 Jul 2012 10:49:59 -0400 Subject: [Freeipa-devel] [PATCH] 1034 more robust cli sessions Message-ID: <50042A17.9050802@redhat.com> Make command-line sessions a bit more robust. This patch does two things. Firstly, it wraps all keyring activity in a try/except so if a keyring operation fails it isn't fatal. The user just won't benefit from sessions. The second part adds per-principal sessions. The principal is included in the session key so we can pull the right one depending on the principal initiating the request. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1034-session.patch Type: text/x-diff Size: 5093 bytes Desc: not available URL: From rcritten at redhat.com Mon Jul 16 17:54:51 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 16 Jul 2012 13:54:51 -0400 Subject: [Freeipa-devel] [PATCH] 1034 more robust cli sessions In-Reply-To: <50042A17.9050802@redhat.com> References: <50042A17.9050802@redhat.com> Message-ID: <5004556B.8030008@redhat.com> Rob Crittenden wrote: > Make command-line sessions a bit more robust. > > This patch does two things. Firstly, it wraps all keyring activity in a > try/except so if a keyring operation fails it isn't fatal. The user just > won't benefit from sessions. > > The second part adds per-principal sessions. The principal is included > in the session key so we can pull the right one depending on the > principal initiating the request. Left a debug statement in, this one should build and work. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1034-2-session.patch Type: text/x-diff Size: 5039 bytes Desc: not available URL: From nalin at redhat.com Mon Jul 16 20:26:47 2012 From: nalin at redhat.com (Nalin Dahyabhai) Date: Mon, 16 Jul 2012 16:26:47 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <500415CC.9080706@redhat.com> References: <500415CC.9080706@redhat.com> Message-ID: <20120716202647.GA14554@redhat.com> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: > Use the new certmonger capability to be able to renew the dogtag > subsystem certificates (audit, OCSP, etc). Are the copies of the certificates in the pki-ca CS.cfg file being updated elsewhere? Or is it not turning out to be a problem if they aren't? Nalin From rcritten at redhat.com Mon Jul 16 20:35:36 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 16 Jul 2012 16:35:36 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <20120716202647.GA14554@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> Message-ID: <50047B18.8070209@redhat.com> Nalin Dahyabhai wrote: > On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >> Use the new certmonger capability to be able to renew the dogtag >> subsystem certificates (audit, OCSP, etc). > > Are the copies of the certificates in the pki-ca CS.cfg file being > updated elsewhere? Or is it not turning out to be a problem if they > aren't? I didn't test validating OCSP signatures but the audit subsystem seemed fine (it complained wildly when I had the wrong trust in the NSS db). Andrew, do I need to update CS.cfg as well? rob From awnuk at redhat.com Mon Jul 16 20:51:06 2012 From: awnuk at redhat.com (Andrew Wnuk) Date: Mon, 16 Jul 2012 13:51:06 -0700 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <50047B18.8070209@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> Message-ID: <50047EBA.9090400@redhat.com> On 07/16/2012 01:35 PM, Rob Crittenden wrote: > Nalin Dahyabhai wrote: >> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>> Use the new certmonger capability to be able to renew the dogtag >>> subsystem certificates (audit, OCSP, etc). >> >> Are the copies of the certificates in the pki-ca CS.cfg file being >> updated elsewhere? Or is it not turning out to be a problem if they >> aren't? > > I didn't test validating OCSP signatures but the audit subsystem > seemed fine (it complained wildly when I had the wrong trust in the > NSS db). > > Andrew, do I need to update CS.cfg as well? > Yes, you may need update CS.cfg too. Andrew From simo at redhat.com Mon Jul 16 22:54:26 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 16 Jul 2012 18:54:26 -0400 Subject: [Freeipa-devel] [PATCHSET] 496 add some PAC verification Message-ID: <1342479266.3219.32.camel@willson.li.ssimo.org> This patchset is about Ticket #2849 The point is to verify that the PAC information we are getting from a trusted realm is actually consistent with the information we know about that trust relationship. The patchset adds a way to load trust information in the kdb driver (first 2 patches), reorganizes a bit the code around PAC verification and adds a filtering function to match realm with AD and SID data. Tested on my trust environment and seem to work fine. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-496-0001-1-Move-mspac-structure-to-be-a-private-pointer.patch Type: text/x-patch Size: 6108 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-496-0002-1-Load-list-of-trusted-domain-on-connecting-to-ldap.patch Type: text/x-patch Size: 4498 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-496-0003-1-Properly-name-function-to-add-ipa-external-groups.patch Type: text/x-patch Size: 7151 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-496-0004-1-Split-out-manipulation-of-logon_info-blob.patch Type: text/x-patch Size: 5840 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-496-0005-1-Add-PAC-filtering.patch Type: text/x-patch Size: 5354 bytes Desc: not available URL: From edewata at redhat.com Mon Jul 16 23:09:54 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 16 Jul 2012 18:09:54 -0500 Subject: [Freeipa-devel] [PATCH] 170 Differentiation of widget type and text_widget input type In-Reply-To: <4FFECCC2.7090201@redhat.com> References: <4FFECCC2.7090201@redhat.com> Message-ID: <50049F42.9080702@redhat.com> On 7/12/2012 8:10 AM, Petr Vobornik wrote: > There was a clash of 'type' attribute in widget's spec. Usually 'type' > is used for telling a builder which field and widget to build. Text > widget used this attribute also for definion of html input type. It was > problematic for some special widgets, which defined own field and used > text_widget, like service_type or dnszone_name. In those and possibly > other cases it used widget type for specifying input type which lead to > execution error in Internet Explorer. Firefox and Chrome took it. > > This patch is changing text_widget's 'type' to 'input_type' which > removes the collision and hence fixes the problem. > > https://fedorahosted.org/freeipa/ticket/2806 > and half of: https://fedorahosted.org/freeipa/ticket/2834 ACK. -- Endi S. Dewata From edewata at redhat.com Mon Jul 16 23:10:34 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 16 Jul 2012 18:10:34 -0500 Subject: [Freeipa-devel] [PATCH] 171 Fixed display of attributes_widget in IE9 In-Reply-To: <5003D757.9090409@redhat.com> References: <5003D757.9090409@redhat.com> Message-ID: <50049F6A.8090100@redhat.com> On 7/16/2012 3:56 AM, Petr Vobornik wrote: > Attributes widget is using overflow css rule in tbody element. IE9 > doesn't handle it well. > > To fix the issue, attributes widget was slightly modified and > conditional css stylesheet was added just for fixing IE problems. > > https://fedorahosted.org/freeipa/ticket/2822 ACK. Separate issue, on IE the main navigation tabs aren't rendered nicely. There is a dark line underneath the label (Identity, Policy, IPA Server) when the tab is inactive. If you put the mouse over the tab the dark line will disappear. -- Endi S. Dewata From edewata at redhat.com Mon Jul 16 23:10:53 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 16 Jul 2012 18:10:53 -0500 Subject: [Freeipa-devel] [PATCH] 172 Bigger textarea for permission type=subtree In-Reply-To: <500400D0.4090604@redhat.com> References: <500400D0.4090604@redhat.com> Message-ID: <50049F7D.4030209@redhat.com> On 7/16/2012 6:53 AM, Petr Vobornik wrote: > Patch description: > Adder dialog and details facet for permission type=subtree have small > textarea for defining subtree filter. It was unconfortable to define the > filter. This difference was removed. > > https://fedorahosted.org/freeipa/ticket/2832 > > Note regarding related ticket: > Resizable textareas are browser-specific. We can do it in code too but I > don't think it is worth the effort. > > Textarea in IE still seems smaller than in Firefox, but it has the same > number of rows and cols. I think it has enough space for defining the > filter so it should be fixing the problem. ACK. Possible improvement, instead of using a fixed column size the text area also could be made to occupy 100% of available width. Ideally it should have the same width as the text field or drop down list in this dialog. -- Endi S. Dewata From william at firstyear.id.au Tue Jul 17 00:00:08 2012 From: william at firstyear.id.au (William Brown) Date: Tue, 17 Jul 2012 09:30:08 +0930 Subject: [Freeipa-devel] DHCP support - Request for review In-Reply-To: <500001CC.1030307@redhat.com> References: <4FEB0B64.7020604@firstyear.id.au> <500001CC.1030307@redhat.com> Message-ID: <5004AB08.7000406@firstyear.id.au> On 07/13/2012 08:39 PM, Petr Vobornik wrote: > On 06/27/2012 03:32 PM, William Brown wrote: >> Hi, >> >> I have been working on adding support for FreeIPA to support >> configuration storage for ISC-DHCP 4.X servers. I have added the schema >> which is included at installation, added the template / empty files that >> will be filled in and used for the installation and created the ipalib >> plugin for this. At this stage, this feature is still far from done. I >> would appreciate a review of the work I have done to ensure I am on the >> right track. >> >> I would like to know if there is a better way to add ACLs than by >> manually updating ldap by hand (IE, using the ACL libraries) (See >> /install/share/dhcpd.ldif). >> >> > > I just looked on the plugin part from Web UI (API) perspective. Proper > plugin review should do somebody else. > > What seems bad: > 1) all entity params are required. When I'm looking at ldif, only couple > of attributes are MUST. > > Param is made optional by adding '?' after its name. Example from user.py: > > Str('displayname?', > label=_('Display name'), > default_from=lambda givenname, sn: '%s %s' % (givenname, sn), > autofill=True, > ), > > However you can probably enforce some params to be required if it's > required by DHCP server and not to blindly copy the LDAP schema. > > 2) You don't have to specify 'primary_key=False', it's default. > > 3) Don't use prefix for cli_name like 'dhcp_server_implementation' > cli_name is used by IPA Client as an option name. Same for labels. For > example for adding dhcp server proper command would probably be: > > ipa dhcpserver-add $NAME --service-dn=$SERVICE_DN etc > > not: > > ipa dhcpserver-add $NAME --dhcp-server-service-dn=$SERVICE_DN etc > > so the param def should be: > > Str('dhcpservicedn?', > cli_name='service_dn', > label=_('Service DN'), > ), > > > 5) All params are STR. I think some might be INT or something else. > > 6) You can fill 'default_attributes' list with more param names. What is > mentioned in default attributes is displayed by `XXX-show $CN` command. > What is not have to be displayed by `XXX-show $CN --all` command. > > 7) In labels you use 'Dhcp'. IMO it should be all uppercase. > > Regards I have never written an IPA plugin before, so most of this was new to me. You did answer some of my questions about how you do things like optional attributes etc. I have made most of these changes and will complete the rest later (time allowing). Thanks for your advice. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 940 bytes Desc: OpenPGP digital signature URL: From william at firstyear.id.au Tue Jul 17 04:48:21 2012 From: william at firstyear.id.au (William Brown) Date: Tue, 17 Jul 2012 14:18:21 +0930 Subject: [Freeipa-devel] [Freeipa-users] stopping su - In-Reply-To: <833D8E48405E064EBC54C84EC6B36E404CD36B24@STAWINCOX10MBX1.staff.vuw.ac.nz> References: <833D8E48405E064EBC54C84EC6B36E404CD33890@STAWINCOX10MBX1.staff.vuw.ac.nz>, <500489D1.90703@gmail.com> <833D8E48405E064EBC54C84EC6B36E404CD348C6@STAWINCOX10MBX1.staff.vuw.ac.nz>, <5004EAB2.70908@gmail.com> <833D8E48405E064EBC54C84EC6B36E404CD36B24@STAWINCOX10MBX1.staff.vuw.ac.nz> Message-ID: <5004EE95.6090207@firstyear.id.au> > > sudo -i su - oracle No, you would run "sudo -i oracle". -i = simulate initial login. Alternately, you can use sudo -s oracle for "run shell as oracle" Or you can run sudo -u oracle "command" -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 940 bytes Desc: OpenPGP digital signature URL: From pviktori at redhat.com Tue Jul 17 10:32:25 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 17 Jul 2012 12:32:25 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <20120711152454.GD9427@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFD91B2.5030007@redhat.com> <20120711152454.GD9427@redhat.com> Message-ID: <50053F39.5040902@redhat.com> On 07/11/2012 05:24 PM, Alexander Bokovoy wrote: > On Wed, 11 Jul 2012, Petr Viktorin wrote: >> On 07/07/2012 08:45 PM, John Dennis wrote: >>> The DN work I was doing on master is ready for review and testing. It's >>> been a long haul and I've been working relentlessly to get this work >>> completed. I am on PTO for a week starting today (I know bad timing) but >>> I spent yesterday and my first day of PTO today writing up extensive >>> documentation for the work so others can start taking a look at it while >>> I'm gone. The documentation as well as where to find the code can be >>> found here: >>> >>> http://jdennis.fedorapeople.org/dn_summary.html >>> >>> The document is long but I felt it was better to provide explanations >>> for as much as possible. >>> >>> I may check in during the week but I'm going to try and discipline >>> myself not to and take an actual much needed break. >>> >>> John >>> >> >> Two more code review points: >> ipa-adtrust-install uses DN without importing it, that'll fail > Where? > $ git grep DN ipaserver/install/adtrustinstance.py|grep import > ipaserver/install/adtrustinstance.py:from ipalib.dn import DN > > This is in master. > In John's dn branch, install/tools/ipa-adtrust-install. $ git grep -w DN install/tools/ipa-adtrust-install install/tools/ipa-adtrust-install:209: api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), bind_pw=smb.dm_password) -- Petr? From abokovoy at redhat.com Tue Jul 17 10:48:44 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 17 Jul 2012 13:48:44 +0300 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <50053F39.5040902@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFD91B2.5030007@redhat.com> <20120711152454.GD9427@redhat.com> <50053F39.5040902@redhat.com> Message-ID: <20120717104844.GA12814@redhat.com> On Tue, 17 Jul 2012, Petr Viktorin wrote: >On 07/11/2012 05:24 PM, Alexander Bokovoy wrote: >>On Wed, 11 Jul 2012, Petr Viktorin wrote: >>>On 07/07/2012 08:45 PM, John Dennis wrote: >>>>The DN work I was doing on master is ready for review and testing. It's >>>>been a long haul and I've been working relentlessly to get this work >>>>completed. I am on PTO for a week starting today (I know bad timing) but >>>>I spent yesterday and my first day of PTO today writing up extensive >>>>documentation for the work so others can start taking a look at it while >>>>I'm gone. The documentation as well as where to find the code can be >>>>found here: >>>> >>>>http://jdennis.fedorapeople.org/dn_summary.html >>>> >>>>The document is long but I felt it was better to provide explanations >>>>for as much as possible. >>>> >>>>I may check in during the week but I'm going to try and discipline >>>>myself not to and take an actual much needed break. >>>> >>>>John >>>> >>> >>>Two more code review points: >>>ipa-adtrust-install uses DN without importing it, that'll fail >>Where? >>$ git grep DN ipaserver/install/adtrustinstance.py|grep import >>ipaserver/install/adtrustinstance.py:from ipalib.dn import DN >> >>This is in master. >> > >In John's dn branch, install/tools/ipa-adtrust-install. > >$ git grep -w DN install/tools/ipa-adtrust-install >install/tools/ipa-adtrust-install:209: >api.Backend.ldap2.connect(bind_dn=DN(('cn', 'Directory Manager')), >bind_pw=smb.dm_password) It needs rebasing then. Also after my patch 0060 this particular line is not there anymore. -- / Alexander Bokovoy From pvoborni at redhat.com Tue Jul 17 14:10:06 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 17 Jul 2012 16:10:06 +0200 Subject: [Freeipa-devel] [PATCH] 170 Differentiation of widget type and text_widget input type In-Reply-To: <50049F42.9080702@redhat.com> References: <4FFECCC2.7090201@redhat.com> <50049F42.9080702@redhat.com> Message-ID: <5005723E.5020607@redhat.com> On 07/17/2012 01:09 AM, Endi Sukma Dewata wrote: > On 7/12/2012 8:10 AM, Petr Vobornik wrote: >> There was a clash of 'type' attribute in widget's spec. Usually 'type' >> is used for telling a builder which field and widget to build. Text >> widget used this attribute also for definion of html input type. It was >> problematic for some special widgets, which defined own field and used >> text_widget, like service_type or dnszone_name. In those and possibly >> other cases it used widget type for specifying input type which lead to >> execution error in Internet Explorer. Firefox and Chrome took it. >> >> This patch is changing text_widget's 'type' to 'input_type' which >> removes the collision and hence fixes the problem. >> >> https://fedorahosted.org/freeipa/ticket/2806 >> and half of: https://fedorahosted.org/freeipa/ticket/2834 > > ACK. > Pushed to master. -- Petr Vobornik From pvoborni at redhat.com Tue Jul 17 14:15:12 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 17 Jul 2012 16:15:12 +0200 Subject: [Freeipa-devel] [PATCH] 171 Fixed display of attributes_widget in IE9 In-Reply-To: <50049F6A.8090100@redhat.com> References: <5003D757.9090409@redhat.com> <50049F6A.8090100@redhat.com> Message-ID: <50057370.4040205@redhat.com> On 07/17/2012 01:10 AM, Endi Sukma Dewata wrote: > On 7/16/2012 3:56 AM, Petr Vobornik wrote: >> Attributes widget is using overflow css rule in tbody element. IE9 >> doesn't handle it well. >> >> To fix the issue, attributes widget was slightly modified and >> conditional css stylesheet was added just for fixing IE problems. >> >> https://fedorahosted.org/freeipa/ticket/2822 > > ACK. Pushed to master. > > Separate issue, on IE the main navigation tabs aren't rendered nicely. > There is a dark line underneath the label (Identity, Policy, IPA Server) > when the tab is inactive. If you put the mouse over the tab the dark > line will disappear. > Ticket https://fedorahosted.org/freeipa/ticket/2817 needs update of jquery-ui to be fixed. We have some overrides of jquery-ui styles which are not displayed in IE nicely. I'll regenerate jquery-ui (lib and css) with our greenish style. Then I'll remove our overrides and fix the 'dark line problem' with it. -- Petr Vobornik From rcritten at redhat.com Tue Jul 17 14:21:43 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Jul 2012 10:21:43 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <50047EBA.9090400@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> Message-ID: <500574F7.6060300@redhat.com> Andrew Wnuk wrote: > On 07/16/2012 01:35 PM, Rob Crittenden wrote: >> Nalin Dahyabhai wrote: >>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>> Use the new certmonger capability to be able to renew the dogtag >>>> subsystem certificates (audit, OCSP, etc). >>> >>> Are the copies of the certificates in the pki-ca CS.cfg file being >>> updated elsewhere? Or is it not turning out to be a problem if they >>> aren't? >> >> I didn't test validating OCSP signatures but the audit subsystem >> seemed fine (it complained wildly when I had the wrong trust in the >> NSS db). >> >> Andrew, do I need to update CS.cfg as well? >> > Yes, you may need update CS.cfg too. Ok, added a bit to update CS.cfg with the new certificate. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1033-2-renewal.patch Type: text/x-diff Size: 43539 bytes Desc: not available URL: From pvoborni at redhat.com Tue Jul 17 14:25:44 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Tue, 17 Jul 2012 16:25:44 +0200 Subject: [Freeipa-devel] [PATCH] 172 Bigger textarea for permission type=subtree In-Reply-To: <50049F7D.4030209@redhat.com> References: <500400D0.4090604@redhat.com> <50049F7D.4030209@redhat.com> Message-ID: <500575E8.7020609@redhat.com> On 07/17/2012 01:10 AM, Endi Sukma Dewata wrote: > On 7/16/2012 6:53 AM, Petr Vobornik wrote: >> Patch description: >> Adder dialog and details facet for permission type=subtree have small >> textarea for defining subtree filter. It was unconfortable to define the >> filter. This difference was removed. >> >> https://fedorahosted.org/freeipa/ticket/2832 >> >> Note regarding related ticket: >> Resizable textareas are browser-specific. We can do it in code too but I >> don't think it is worth the effort. >> >> Textarea in IE still seems smaller than in Firefox, but it has the same >> number of rows and cols. I think it has enough space for defining the >> filter so it should be fixing the problem. > > ACK. > > Possible improvement, instead of using a fixed column size the text area > also could be made to occupy 100% of available width. Ideally it should > have the same width as the text field or drop down list in this dialog. > Updated patch attached. I added two styles: .textarea-widget textarea { width: 250px; } .facet-content .textarea-widget textarea { width: 400px; } First one makes textarea width the same as text_widget - good for dialogs. Second one makes textareas wider in facets. IMO it is better - more space for the text. Also previous implementation was about 330px in FF so it isn't big difference. 400px is about the same width as ssh-key widget and in also leaves space for possible action panel. -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0172-1-Bigger-textarea-for-permission-type-subtree.patch Type: text/x-patch Size: 1513 bytes Desc: not available URL: From edewata at redhat.com Tue Jul 17 14:57:36 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 17 Jul 2012 09:57:36 -0500 Subject: [Freeipa-devel] [PATCH] 172 Bigger textarea for permission type=subtree In-Reply-To: <500575E8.7020609@redhat.com> References: <500400D0.4090604@redhat.com> <50049F7D.4030209@redhat.com> <500575E8.7020609@redhat.com> Message-ID: <50057D60.5010301@redhat.com> On 7/17/2012 9:25 AM, Petr Vobornik wrote: >> Possible improvement, instead of using a fixed column size the text area >> also could be made to occupy 100% of available width. Ideally it should >> have the same width as the text field or drop down list in this dialog. > > Updated patch attached. > > I added two styles: > > .textarea-widget textarea { > width: 250px; > } > > .facet-content .textarea-widget textarea { > width: 400px; > } > > First one makes textarea width the same as text_widget - good for dialogs. > > Second one makes textareas wider in facets. IMO it is better - more > space for the text. Also previous implementation was about 330px in FF > so it isn't big difference. 400px is about the same width as ssh-key > widget and in also leaves space for possible action panel. Looks much better. ACK. -- Endi S. Dewata From rcritten at redhat.com Tue Jul 17 15:46:11 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Jul 2012 11:46:11 -0400 Subject: [Freeipa-devel] [PATCH] 0061 ValidationError takes 'error' named argument, not 'reason' In-Reply-To: <20120716131259.GA18370@redhat.com> References: <20120716131259.GA18370@redhat.com> Message-ID: <500588C3.6030004@redhat.com> Alexander Bokovoy wrote: > Hi, > > https://fedorahosted.org/freeipa/ticket/2865 ACK From rcritten at redhat.com Tue Jul 17 15:48:06 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Jul 2012 11:48:06 -0400 Subject: [Freeipa-devel] [PATCH] 0062 support various forms of user account when establishing trusts In-Reply-To: <20120716131410.GB18370@redhat.com> References: <20120716131410.GB18370@redhat.com> Message-ID: <50058936.1080100@redhat.com> Alexander Bokovoy wrote: > Hi, > > Realm administrator account may be specified using different form: > Administrator, DOM\Administrator, Administrator at DOMAIN > > This patch introduces handling of the second two forms: > - In DOM\Administrator only user name is used, short domain name > is then taken from a discovered record from the AD DC > - In Administrator at DOMAIN first DOMAIN is verified to be the same > as the domain we are establishing trust to, and then user name > is taken, together with short domain name taken from a discovered > record from the AD DC > > Note that we do not support using to-be-trusted domain's trusted > domains' accounts to establish trust as there is basically zero chance > to verify that things will work with them. In addition, in order to > establish trust one needs to belong to Enterprise Admins group in AD or > have specially delegated permissions. These permissions are unlikely > delegated to the ones in already trusted domain. > > https://fedorahosted.org/freeipa/ticket/2864 > ACK From rcritten at redhat.com Tue Jul 17 17:38:21 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Jul 2012 13:38:21 -0400 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <20120713173354.GK9427@redhat.com> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> Message-ID: <5005A30D.4040702@redhat.com> Alexander Bokovoy wrote: > On Fri, 13 Jul 2012, Alexander Bokovoy wrote: >> Hi, >> >> when adding AD trusts support, we need to ensure we have valid kerberos >> ticket of the user from 'admins' group or otherwise appropriate ACIs >> will not be granted. >> >> This patch introduces a check for that. We already check if >> ipa-adtrust-install is run by root so this complements existing checks. >> >> https://fedorahosted.org/freeipa/ticket/2815 > After discussing on IRC with Simo and Rob, we came to conclusion that it > is possible to switch to LDAPI and autobind feature of dirsrv for > authentication and remove requirement for Directory Manager credentials > altogether. > > Updated patch makes use of LDAPI + autobind under root privileges to map > automatically to Directory Manager privileges in dirsrv. Additionally it > ensures we have Kerberos credentials to fetch keytab with CIFS service > key. > > Service._ldap_mod() is extended to switch to autobind when self.ldapi is > set to True and we are running as root. > > For those interested in why ACIError is mapped to 'outdated Kerberos > credentials' error message, this is because we'll get ACIError for 'ipa > user-show ' command when authenticated by the Kerberos credentials > for in a default ccache only when Kerberos credentials are stale -- > either belong to a user that was removed or to a previous IPA install > that was wiped before reinstalling. The latter is how I discovered > this case. :) I think that this should raise an exception if one tries to use ldapi, doesn't provide the DM password and is not root. Otherwise it won't authenticate at all. In reality, I think all this service code always runs as root, so it may be a moot point, but this code is kinda convoluted. rob From rcritten at redhat.com Tue Jul 17 18:52:00 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Jul 2012 14:52:00 -0400 Subject: [Freeipa-devel] [PATCH] 1035 case sensitivity when calculating indirect members Message-ID: <5005B450.8000207@redhat.com> When determining whether a member is direct or indirect we were not doing a case-insensitive comparison which led to marking a member as both direct and indirect (in a test case no less). This patch fixes the comparison and the test. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1035-member.patch Type: text/x-diff Size: 2371 bytes Desc: not available URL: From rcritten at redhat.com Tue Jul 17 20:41:27 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 17 Jul 2012 16:41:27 -0400 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <4FF2AE6C.6010800@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> Message-ID: <5005CDF7.2060205@redhat.com> Petr Viktorin wrote: > On 06/29/2012 11:28 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 06/25/2012 03:00 PM, Petr Viktorin wrote: >>>> On 06/20/2012 06:15 PM, Rob Crittenden wrote: >>>>> Petr Viktorin wrote: >>>>>> On 06/04/2012 04:56 PM, Petr Viktorin wrote: >>>>>>> Currently, FreeIPA's install/admin scripts are long pieces of code >>>>>>> that aren't very reusable, importable, or testable. >>>>>>> They have been extended over time with features such as logging and >>>>>>> error handling, but since each tool was extended individually, there >>>>>>> is much inconsistency and code duplication. >>>>>>> This patch starts a framework which the admin tools can use, and >>>>>>> converts ipa-ldap-updater to use the framework. >>>>>>> >>>>>>> In an earlier patch I found that improving a particular >>>>>>> functionality in >>>>>>> all the commands is not workable, so I want to tackle this one tool >>>>>>> at a >>>>>>> time. >>>>>>> I'm starting with ipa-ldap-updater, because it's pretty small, >>>>>>> doesn't >>>>>>> use DNs (I don't want conflicts with John's work), and has the >>>>>>> interesting --upgrade option. >>>>>>> >>>>>>> >>>>>>> The framework does these tasks: >>>>>>> - Parse options >>>>>>> - Select tool to run (see below) >>>>>>> - Validate options >>>>>>> - Set up logging >>>>>>> - Run the tool code >>>>>>> - Handle any errors >>>>>>> - Log success/failure >>>>>>> >>>>>>> The base class has some defaults for these that the tools can >>>>>>> extend/override. >>>>>>> >>>>>>> >>>>>>> To handle the case where one script does two different things >>>>>>> (ipa-ldap-updater with/without --upgrade, or ipa-server-install >>>>>>> with/without --uninstall), I want to split the tool in two classes >>>>>>> rather than have repeated ifs in the code. >>>>>>> This meant that option parsing (and initializing the parser) has >>>>>>> to be >>>>>>> done before creating an instance of the tool. I use a factory >>>>>>> classmethod. >>>>>>> >>>>>>> >>>>>>> I put the admintool base class in ipapython/ as it should be useful >>>>>>> for >>>>>>> ipa-client-install as well. >>>>>>> >>>>>>> >>>>>>> >>>>>>> First part of the work for: >>>>>>> https://fedorahosted.org/freeipa/ticket/2652 >>>>>>> >>>>>>> >>>>>> >>>>>> Attaching rebased patch. >>>>> >>>>> I gather you want people to be calling run_cli() in their admin tools. >>>>> Should main() be made private then? I could see someone getting >>>>> confused >>>>> and using main instead, which would work, but then the return value >>>>> might not do the right thing. >>>>> >>>>> Or maybe just drop run_cli and have main exit with sys.exit()? >>>> >>>> I don't see why running a command as a Python function should be >>>> discouraged. In fact it could even help -- for example logging could >>>> only be set up once, so if we call, say, ipa-ldap-updater from >>>> ipa-server-install, all related logs would go to a single file. >>>> A C-style main (taking a list of arguments and returning the exit >>>> status) is a good thing for modularity and testability. >>>> The `run_cli` method is just a convenient shortcut for the usual usage, >>>> so the calling modules can be as small as possible. >>>> >>>> If people get confused and call main instead of run_cli, they need to >>>> manually pass in sys.argv. I think this is enough of a warning that >>>> their assumptions aren't right. >>>> To make it even clearer I've removed the possibility to pass None as >>>> argv to main() and have it auto-filled. >>>> >>>> Some relevant reading: >>>> http://www.artima.com/weblogs/viewpost.jsp?thread=4829 (old but still >>>> valid) >>>> http://en.wikipedia.org/wiki/Main_function#Python >>>> >>>>> It isn't correctly handling the case of an update not found: >>>>> >>>>> ipa : INFO Parsing file ad >>>>> [Errno 2] No such file or directory: 'ad' >>>>> ipa : INFO File >>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>> 151, in >>>>> execute >>>>> self.run() >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ldap_updater.py", >>>>> >>>>> >>>>> >>>>> line >>>>> 180, in run >>>>> modified = ld.update(self.files) >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", >>>>> line 828, in update >>>>> sys.exit(1) >>>>> >>>>> ipa : INFO The ipa-ldap-updater command failed, exception: >>>>> SystemExit: 1 >>>> >>>> I've added validation for missing files, and improved the error message >>>> ldapupdate raises (for cases the validation doesn't catch, like passing >>>> directories or unreadable files). >>>> Ideally ldapupdate would not try to handle the error itself, but that >>>> code is used in more places that I don't want to break, so I'm leaving >>>> the extraneous print in. >>>> >>>>> Running in test mode with the attached update doesn't seem to work >>>>> either. There is nothing special about this file, just something I had >>>>> lying around: >>>>> >>>>> ipa : INFO File >>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>> 151, in >>>>> execute >>>>> self.run() >>>>> File >>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ldap_updater.py", >>>>> >>>>> >>>>> >>>>> line >>>>> 184, in run >>>>> 'Update complete, changes to be made, test mode', 2) >>>>> >>>>> ipa : INFO The ipa-ldap-updater command failed, exception: >>>>> ScriptError: >>>>> Update complete, changes to be made, test mode >>>>> ipa : ERROR Update complete, changes to be made, test mode >>>>> >>>>> ipa : ERROR None >>>> >>>> Fixed. >>>> >>>>> The unit tests still pass which is good. >>>>> >>>>> With ipa-ldap-updater the return value is a bit strange. All the >>>>> updates >>>>> themselves can fail for one reason or another and the command can >>>>> still >>>>> consider this a success (it may fail because a feature is not enabled, >>>>> for example). Still, the success message displayed at the end is a bit >>>>> jarring when the updates themselves aren't applied. Here is a snippet >>>>> when running ad.update live: >>>>> >>>>> ipa : INFO New entry: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>> ipa : DEBUG --------------------------------------------- >>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>> ipa : DEBUG add: 'account' to objectClass, current value [] >>>>> ipa : DEBUG add: updated value [u'account'] >>>>> ipa : DEBUG --------------------------------------------- >>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>> ipa : DEBUG objectClass: >>>>> ipa : DEBUG account >>>>> ipa : DEBUG add: 'adtrust' to uid, current value [] >>>>> ipa : DEBUG add: updated value [u'adtrust'] >>>>> ipa : DEBUG --------------------------------------------- >>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>> ipa : DEBUG objectClass: >>>>> ipa : DEBUG account >>>>> ipa : DEBUG uid: >>>>> ipa : DEBUG adtrust >>>>> ipa : DEBUG --------------------------------------------- >>>>> ipa : DEBUG Final value >>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>> ipa : DEBUG objectClass: >>>>> ipa : DEBUG account >>>>> ipa : DEBUG uid: >>>>> ipa : DEBUG adtrust >>>>> ipa : INFO Parent DN of >>>>> uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>> may not exist, cannot create the entry >>>>> ipa : INFO The ipa-ldap-updater command was successful >>>>> [root at pinto freeipa]# echo $? >>>>> 0 >>>>> >>>>> This may be contrasting just because it is a contrived case. The >>>>> command >>>>> rval is separate from whether the updates all applied, so maybe this >>>>> is ok. >>>> >>>> The current ipa-ldap-updater also works this way, so this should go >>>> in a >>>> separate ticket. >>>> I worry that changing the return value could make installations fail, >>>> for example. >>>> >>>>> rob >>>> >>>> >>>> Thanks for the review! >>>> >>> >>> Once again, this time with the patch. >> >> Almost there. When running in test mode and an update that would be >> applied should return 2. >> >> rob >> >> > > Fixed > > Ok, things seem to work. One last question. This puts all the guts of the work into a file in ipaserver/install/. Is there any benefit to that over putting the class into the tool itself? I'm trying to think ahead as more commands get migrated, we're just going to be adding more files to ipaserver/install and making shells out of the tools in install/tools. rob From jdennis at redhat.com Tue Jul 17 22:47:57 2012 From: jdennis at redhat.com (John Dennis) Date: Tue, 17 Jul 2012 18:47:57 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FFBE66E.9040802@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFBE66E.9040802@redhat.com> Message-ID: <5005EB9D.1010003@redhat.com> On 07/10/2012 04:23 AM, Petr Viktorin wrote: > I've read your summary (which you should summarize into a commit message > before this is pushed), and gone through the patch. > Here is what I found doing that; I didn't get to actual testing yet. > I also didn't do a detailed review of ldap2.py changes yet. Thank you for your careful review Petr, you had many good suggestions and comments. It was an enormous amount of material to wade through, thank you for your diligence. I'm in agreement with most of your suggestions. > I agree with Simo that it would have been better to put at least your > automated changes in a separate patch. Without knowing what was > automated and what wasn't, I have to eyeball each change at least to > figure that out. Not complaining, it's a reviewer's work after all, and > with an intra-line diff tool it's not that hard, but I can't really > honor your suggestion for a faster review :) I agree fully it would be nice if things were broken up into smaller well defined patches. When I started the work I tried to keep things separate but as I worked I discovered there were so many inter-dependent relationships I was spending as much time juggling patch sets that could be applied independently as I was making forward progress to the real goal. So I abandoned the idea of independent patch sets in favor of getting the work completed as expeditiously as possible, it was a pragmatic trade-off. Now that the work is (nearly) completed I can see where some things can be broken out cleanly that was not obvious in the midst of the work. Hindsight is 20/20. > ==== Blockers ==== > > Mutable objects that support __hash__ are a *very* bad thing to do. It > violates a fundamental assumption in Python, as you outlined. Mutable > objects *must not* be allowed to be dictionary keys. > I understand the patch is not perfect, but this is a big issue that > invites very hard-to-debug errors. > You say you're already working on a patch to fix this. Please post that > ASAP; I can't ack the DN work without it. Understood, I agree with you and I will start work on those changes immediately. > In the long run, it would be good to use immutable DNs only: I think > modeling them after Python strings in this way would be beneficial. We edit DN's in a variety of places thus I feel mutable DN's have a distinct value. If we only had immutable DN's then forming a new DN would require more code that would extract the components from the right locations in the original DN and use those extracted components to build a new DN. That would not be too far from some of the earlier code which used a series of calls over multiple lines to slice and dice the strings. I believe using just one simple obvious Python operator beats multiple lines of convoluted code every time. The expression of the concept is inherently obvious in the line of code using Python operator(s), plus it's vastly more succinct (easier to read, easier to code, more robust). Because the logic to build the new DN is inherently centralized in the object operator it's a single implementation (much easier to maintain and prove correctness) rather than the cut-n-paste coping of code to decompose and reassemble an DN over many locations in the code base. I believe mutable DN's have a valuable role to play and make the best use of Python's object oriented facilities. They go a long way to making IPA more robust and easier to comprehend. > You seem to forged locked=True for some global DNs, e.g. > ipalib/plugins/pwpolicy.py:172: global_policy_dn > ipalib/plugins/migration.py:117: _compat_dn > Locked by default would prevent these omissions. > > > > Please fix a lint failure; ipaserver.ipautil moved to ipapython.ipautil. > > > > ==== Design decision criticism ==== > > > Originally I was against automatic type promotion on the belief if > you were comparing a string to a DN it was a failure of program logic, > but there were some circumstances where it was evil necessity. > > Would you care to elaborate? I also think that's a logic failure; such > circumstances should at least be marked by a TODO. I should have taken better notes because I don't remember the exact place this came up, it was late in the testing and I believe the problem showed up in the ldap updater, possibly some other places. But here is the bottom line: I encountered a handful of places in the code where a string was being compared to a DN object for equality (I seem to recall the equality comparison was indirect by virtue of the "in" operator). What I do recall is it was logically impossible to know a prori the string being tested was supposed to be a DN. If it were possible to know the string represented a DN it would have been easy to promote it to a DN object. But there were a handful of situations where there simply was not enough information available to ascertain if the string was logically a DN or not. This was never an issue when dn's were simple strings. If it's getting compared to a DN that's a pretty strong hint it's supposed to be a DN (fwiw the coercion must occur in the equality operator). So I pondered the problem for a while and decided there was legitimate value in allowing an equality comparison between a string and DN. I could see no downside to it other than it *might* mask an error in program logic. I didn't take the decision lightly. I looked through the Python internals to see where type coercion occurs. While not often it's certainly done where "it makes sense" (e.g. between string types, between numerical types, between buffer types, etc.). I thought one could make the argument that a DN object is akin to a string type and hence falls into the same category of coercion as occurs between string types. I'm not saying I'm in love with it, rather it's a pragmatic necessity and can be justified if you look at it a certain way. We could add a debug message with a backtrace whenever the coercion occurs if we wanted to completely eliminate all places in the code where a string-to-DN comparison occurs. But as I said I'm not sure it's possible to absolutely eliminate this comparison. We've eliminated 99.9% of those cases already, I'm not sure the other 0.1% can be eliminated or that it's worth the work involved. I'm happy with 99.9% :-) > > > The find() and rfind() methods (modeled after their string cousins) > to locate a RDN or DN in an existing DN. > > I would much rather see the index() and rindex() methods, which do the > Pythonic thing of raising an exception rather than silently returning -1 > when the thing isn't found. > Compare: > something[something.find(thing)] => something[-1] if not found > something[something.index(thing)] => raises exception if not found You're right exceptions are more Pythoninc than return codes. I can add index and rindex and convert the code to catch an exception. However, we use find and rfind extensively in the IPA code base (DN.find() and DN.rfind() simply replaced their string equivalents during the conversion). Perhaps you should open a ticket to purge all use of find() and rfind() from our code base and add a prohibition against their use in our Python coding standard document. > > > ipapython/dn.py:448: > Each argument must be scalar convertable to unicode or a callable > object whose result is convertable to unicode. > > Is it really necessary to allow callables here? If the user has a > callable, he/she should call it before making an AVA out of it. Same > with converting to Unicode. > As you point out in dn_summary, automatic coercion is evil. There are two issues here, type coercion and string encoding. Not all coercion is evil, there is value in making it easy to do the right thing and minimizing inline coercion that otherwise obfuscates the logical intent of the code. We shouldn't require programmers to be perfect coders who never forget to perform some operation in the hundreds of places it might be required. Rather the tools should quietly and consistently enforce the rules while leaving the code readable, concise and with clear intent. There is a well established precedent in Python for coercion to string in a string context. Coercion to string is so fundamental to Python it has it's own low-level operator embedded in every class type definition utilized in both Python and the CPython implementations. 1) We know all DN components must be strings. Let's say someone passes an integer object; is there really any harm to converting this to a string representation of an integer? That makes the DN API much friendlier and it's use more succinct. In fact Python does automatic coercion to string in many places where it knows something must be a string, thus it's a well accepted and established practice. We do automatic coercion to string in many places in IPA (a good example is the ldap encode/decode operations). I wish other Python API's also coerced to string as a service to the caller. I wish python-ldap would have coerced to string those elements of it's API which it knows must be strings. If it had we could have just passed DN objects into the python-ldap API instead of having to wrap the class to perform the coercion. This wouldn't be violating a guideline, rather it would be utilizing a inherent feature of the Python language. There are an astonishing number of CPython modules that barf if you pass the wrong string type. Instead they should be friendly and provide the right string coercion. (There is a long and sordid history of this problem and I have a extensive write-up on it). 2) All strings in IPA are unicode. In python 3.x all strings will be unicode (the way it should have been in python 2.x alas). When string comparisons are performed they should be done as unicode objects, not encoded in some undefined encoding. Likewise when returning a string belonging to a component in a DN it should be unicode. Traditionally dn's have been str objects (a python-ldap foible), so the decoding of str to unicode promotes correctness in Python 2.x. Accepting a callable is just a further refinement of the discussion in #1. If you allow for automatic coercion to sting (in a context that demands a string type) then why not allow anything which yields a string? It just makes the API friendlier and easier to use without introducing ugly compromises. Some coercion is evil, but some is darn nice and useful. The trick is recognizing which is which. > > > > ==== Major issues ==== > > ipalib/plugins/aci.py:340: > + groupdn = DN(groupdn) > + try: > + if groupdn[0].attr == 'cn': > + dn = DN() > + entry_attrs = {} > + try: > + (dn, entry_attrs) = ldap.get_entry(groupdn, ['cn']) > + except errors.NotFound, e: > + # FIXME, use real name here > + if test: > + dn = DN(('cn', 'test'), > api.env.container_permission) > + entry_attrs = {'cn': [u'test']} > + if api.env.container_permission in dn: > + kw['permission'] = entry_attrs['cn'][0] > + else: > + if 'cn' in entry_attrs: > + kw['group'] = entry_attrs['cn'][0] > + except IndexError: > + pass > > Why is any IndexError ignored here? Because the original test that entered the code block whose size you're concerned about was: if groupdn.startswith('cn='): The very first executable statement in the try block is if groupdn[0].attr == 'cn' If groupdn is actually empty it immediately exits the block because there will be an IndexError due to trying to access groupdn[0]. The combination of the try and the first if test mimics the original logic for deciding if one should enter the block. However, you make a valid point that there are other indexing statements in the block that could raise an IndexError that should not be silently caught and ignored. I will change that. I chose to use IndexError because error checking on DN "existence checks" in the rest of the code most always uses IndexError and KeyError, I was trying to be consistent. The test for entering the block can be changed to: if len(groupdn) and groupdn[0].attr == 'cn': and remove the try/except. > That try block is much too long, it > could catch unintended errors. Please only `try` the smallest piece of > code that could raise the exception you want to catch. see above > There are enough `DN(xyz.replace('ldap:///',''))` calls in aci.py to > warrant a strip_ldap_prefix helper. (It could also only remove the > prefix, not all the occurrences -- unlikely to matter but it's better to > do things correctly.) No argument, I agree but I didn't write the code and I was trying not to change code I didn't need to. Please open a ticket to get this fixed. > ipalib/plugins/baseldap.py:1028: > + try: > + if dn[0].attr != self.obj.primary_key.name: > + self.obj.handle_duplicate_entry(*keys) > + except (IndexError, KeyError): > + pass > > Again, there's too much in the try block. You don't want to catch errors > from handle_duplicate_entry. Do this instead: > try: > dn_attr = dn[0].attr > except (IndexError, KeyError): > pass > else: > if dn_attr != self.obj.primary_key.name: > self.obj.handle_duplicate_entry(*keys) Good suggestion, I'll fix it. > > ==== Minor issues ==== > > In ipalib/plugins/aci.py: > @@ -343,10 +341,12 @@ def _aci_to_kw(ldap, a, test=False, pkey_only=False): > else: > # See if the target is a group. If so we set the > # targetgroup attr, otherwise we consider it a subtree > - if api.env.container_group in target: > - targetdn = unicode(target.replace('ldap:///','')) > - target = DN(targetdn) > - kw['targetgroup'] = target['cn'] > + targetdn = DN(target.replace('ldap:///','')) > + if api.env.container_group in targetdn: > + try: > + kw['targetgroup'] = targetdn[0]['cn'] > + except (IndexError, KeyError): > + pass > else: > kw['subtree'] = unicode(target) > > Why is (IndexError, KeyError) ignored? Because the original code did not seem to handle the error condition either and I wasn't sure what to do with the error. But you're right, silently ignoring those two exceptions is wrong. I'll fix it, good catch on your part. > Wouldn't be more correct to use endswith(DN(container_group, suffix)) > instead of the `in` operator here, and in other places like this. Good idea, the test you suggest is actually the correct test (shame on me for trying to translate the code too literally). I'll fix it. > ipapython/dn.py:26 > LOCKED_ERROR_MSG = 'Object is locked, it cannot be modified: %s' > and then: > raise TypeError(LOCKED_ERROR_MSG % (repr(self))) > > You should define a TypeError subclass for this. > Using a string instead of a specialized object is exactly what this > patch tries so hard to get rid of... :) Moot point since locking will be replaced in favor of mutable and immutable DN variants. Base Python exceptions would be raised instead for these types of coding errors, that's consistent with how we handle those problems now. > ipapython/entity.py:49: > elif isinstance(entrydata, DN): > self.dn = entrydata > self.data = ipautil.CIDict() > elif isinstance(entrydata, basestring): > self.dn = DN(entrydata) > self.data = ipautil.CIDict() > > Should we not use DNs exclusively here? At least a TODO is warranted. It seemed reasonable to allow the constructor to coerce to DN rather than forcing the caller of the constructor to coerce a parameter. Someday soon I hope the Entity and Entry classes go away as we converge on a single API. In the interim this compromise seems reasonable and meant there were fewer places to find and fix (everywhere where Entity and Entry objects were instantiated). > > ipaserver/ipaldap.py:109: > elif isinstance(entrydata,DN): > self.dn = entrydata > self.data = ipautil.CIDict() > elif isinstance(entrydata,str) or isinstance(entrydata,unicode): > self.dn = DN(entrydata) > self.data = ipautil.CIDict() > > Again, why not use DNs exclusively? At least put in a TODO. > For now at least do isinstance(entrydata, basestring) instead of the two > isinstances. Agreed, there should be one isinstance(), I didn't code it, but I should have fixed it when I saw it. > > > In several places: > + # Use a property for to assure it's always a DN type > + def _set_(self, value): > + self._ = DN(value) > + > + def _get_(self): > + assert isinstance(getattr(self, '_'), DN) > + return self._ > + > + ldap_suffix = property(_get_, _set_) > > Shouldn't the assert be in the setter? Do we not want people to only put > DNs in these properties, instead of silently converting? I'm afraid I don't follow you here. When setting the property it should be coerced to a DN, we want to force the property to always be a DN. However it's possible for someone to set the _ directly even though they should never do that, the assert in the getter is just bullet-proofing against that. > This appears enough times to justify using a helper to construct these. > But if you don't want to generalize, the getattr call is unnecessary. Once again I don't follow you. How would a helper work? I don't know why there is a getattr in the getter, it would seem to be unnecessary unless I was trying to protect against an uninitialized attribute. The getattr should probably be replaced with normal class attribute access. I'll fix that. > tests/util.py:307: > if isinstance(expected, DN): > if isinstance(got, basestring): > got = DN(got) > > This may be overly broad, the tests should be able to check if they got > a DN or a string. What about something like: > > if isinstance(expected, DNString): > assert isinstance(got, basestring) > assert DN(got) == expected.dn > with a suitable DNString class? Sorry I don't understand what you're trying to protect against nor what a DNString class would be or what it's purpose is. > ipapython/dn.py:541: def _set_locked(self, value): > There should be only one way to do things, either the lock/unlock > methods, or setting an attribute. More ways to do one same thing lead to > subtle errors in one of the ways. > > The locking code is duplicated in AVA, RDN and DN; it would be good to > make a Lockable mixin to avoid that. A similar case is with __repr__. > > Anyway unlocking is bad (see above) so these are mostly moot points. A mixin would eliminate code duplication but in this case the duplication is trivial, not sure it warrants introducing a new class in this circumstance (judgment call), but the topic is moot because locking is going to be eliminated. > > ipaserver/plugins/ldap2.py:134: > class ServerSchema(object): > > Please don't use nested classes without reason. There is a valid reason, the ServerSchema class is private to the SchemaCache class. > ==== Nitpicks/opinions ==== > > In ipa-replica-manage:369 > > sys.exit("winsync agreement already exists on subtree %s" % > please remove the trailing space > > > > ipapython/dn.py: > in docstring: DN(arg0, ..., locked=False, first_key_match=True) > followed by: def __init__(self, *args, **kwds): > and: kwds.get('first_key_match', True) > > I don't see the reason for this. Just write `def __init__(self, *args, > locked=False, first_key_match=True)` and put a proper summary in the > first line of the docstring. Same in AVA & RDN. A valid point. I think I was trying to be too flexible/extensible. > ipapython/ipautil.py:201: > - """Convert a kerberos realm into the IPA suffix.""" > > Why are you removing a docstring? Must have been a mistake. I'll fix. > ipaserver/plugins/ldap2.py:79: > _debug_log_ldap = True > if _debug_log_ldap: > import pprint > > Just import it unconditionally if you need it. Not to worry, I'm going to nuke the pprint formatting anyway, it never worked correctly and when it did it didn't add much value. > ipalib/plugins/baseldap.py:1874: > - sortfn=lambda x,y: cmp(x[1][self.obj.primary_key.name][0].lower(), > y[1][self.obj.primary_key.name][0].lower()) > + sortfn=lambda x,y: cmp(x[1][self.obj.primary_key.name][0], > + y[1][self.obj.primary_key.name][0]) > entries.sort(sortfn) > > Since you're touching this: it's better (easier, faster, more readable) > to use a key function. (Also, defining a named function is better with > `def foo` than `foo=lambda...`): > def sortkey(x): > return x[1][self.obj.primary_key.name][0] > entries.sort(key=sortkey) Agreed, it would have been better to rewrite this using a key function rather than preserving the old code, I'll fix. > > > ipapython/entity.py:95: > # Python "internal" class attributes are prefixed and suffixed > with double underscores. > # Make sure we continue to return things like __str__ from > this class. > > I think a better solution would be making Entity a new-style class. > Or even better, killing Entity off altogether. Which is the plan, so > this is a moot point. > Same with Entry in ipaserver/ipaldap.py. Yeah, that would have worked too, probably cleaner, but I really want these classes to go away, not much point in tweaking them now. > ipaserver/install/ldapupdate.py:132: > + assert isinstance(suffix, (None.__class__, DN)) > > type(x) is preferable to x.__class__ (the same way e.g. len(x) is > preferable to x.__len__). However, I think it would be more readable to use: > if suffix is not None: > assert isinstance(suffix, DN) I don't think you can use type() to cover the case when something might be a derived subclass, hence the way it was coded (I anticipated DN might be subclassed). I was trying to allow for both None or any DN or subclass of DN in one statement. Perhaps not the best idea. Your suggestion is more readable, I'll change it to match your suggestion. > ipaserver/plugins/ldap2.py:109: > def value_to_utf8(val): > ''' > If val is not a string we need to convert it to a string > (specifically a unicode string). Naively we might think we need to > call str(val) to convert to a string. This is incorrect because if > val is already a unicode object then str() will call > encode(default_encoding) returning a str object encoded with > default_encoding. But we don't want to apply the default_encoding! > Rather we want to guarantee the val object has been converted to a > unicode string because from a unicode string we want to explicitly > encode to a str using our desired encoding (utf-8 in this case). > > Note: calling unicode on a unicode object simply returns the exact > same object (with it's ref count incremented). This means calling > unicode on a unicode object is effectively a no-op, thus it's not > inefficient. > ''' > That is not an explanation of *what* the function does, but *how* it > does it. Thus it should be in a comment, not in a docstring. > > Also, if something needs such an explanation, it also need unit tests to > ensure it actually works correctly. > > IPASimpleLDAPObject has the same issue -- lengthy descriptions of design > decisions should go in a comment (if not in a commit message or on the > mailing list). Valid point, I'll make them comments. > > Require that every place a dn is used it must be a DN object. This > > translates into lot of `assert isinstance(dn, DN)` sprinkled through > > out the code. > > That seems like a crutch to get your DN work done. We should remove > these once the dust settles. I disagree with removing the asserts. You're correct they were an invaluable aid in performing the conversion but they have a value beyond that. I fully expect programmers to make mistakes with DN usage in the future. Such mistakes are really easy to make especially when for years we just used strings. I've already seen examples of dn string formatting getting committed to master when folks should have known better. As such I believe the asserts will continue to play a valuable role in the (foreseeable?) future. We do use asserts in a fair number of other locations. I don't presume you mean you want all assert statements removed. Instead I think we should make sure the Python interpreter is started with the -O option in production mode, then assertions will be stripped out and not evaluated eliminating the overhead. see http://wiki.python.org/moin/UsingAssertionsEffectively > > Python provides the __all__ module export list, we really should be > > making better use of that. > > I recommend importing names explicitly, or importing a module and using > qualified names, over __all__ and import *. Star imports make it > unnecessarily hard to see where names come from. > (You do use explicit imports in the code, my gripe is just about > recommending * and __all__). We should come to a project consensus on this topic and add it to our Python style guide. This really isn't germane to the dn work. > > I equate Python decorators with C preprocessor hell when CPP macros > > are abused. > > I do not share that view. It is possible to abuse decorators, but not > all of their use is necessarily evil. > It's true that encode.py wasn't a good use of them. Agreed, decorators are not always bad, they have a role to play. I was perhaps being too dramatic in my wording and not selective enough. Only some decorators are like C preprocessor hell, some are perfectly fine. It took me so long to unravel and fully comprehend what the encode/decode decorators were doing in detail that I was probably venting my frustration with that experience. > > Removed all usage of get_schema() which was being called prior to > > accessing the .schema attribute of an object. If a class is using > > internal lazy loading as an optimization it's not right to require > > users of the interface to be aware of internal optimization's. > > Instead the class should handle it's lazy loading completely > > internally by loading the data on first access. > > This is very easy to do with class properties. schema is now a > > property of the class instead of an attribute. When the schema > > property is accessed it actually calls a private internal method to > > perform the lazy loading. > > In other words, you're reinventing the reify decorator: > https://github.com/Pylons/pyramid/blob/master/pyramid/decorator.py#L1 Hmm... I wasn't aware of reify. A quick look at it and it smacks of self-modifying code, more hidden weirdness that's not obvious in static code reading. Personally I vastly prefer things to be obvious and not tricky, clever, obscure, and hidden. > > Being a grammar nazi, I must point out the it's/its confusion and > plurals formed with apostrophes. Yup, can't disagree, but heck I pumped out 10 pages of documentation in a little over a day, there was some grammar road-kill along the way no doubt. BTW, as a non-native English speaker it's admirable to pick these things up. I'm impressed! > ==== > > Thanks for the work! This will really make IPA better when it's polished > a bit. Hopefully all the criticism doesn't sound too negative :) Thank you for the review. It was a lot of material to slog though and your careful attention is appreciated, it will definitely make it better. You made good points and I was in agreement with the majority of them. Thank you. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From abokovoy at redhat.com Wed Jul 18 09:01:24 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 12:01:24 +0300 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <5005A30D.4040702@redhat.com> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> <5005A30D.4040702@redhat.com> Message-ID: <20120718090124.GM12345@redhat.com> On Tue, 17 Jul 2012, Rob Crittenden wrote: >Alexander Bokovoy wrote: >>On Fri, 13 Jul 2012, Alexander Bokovoy wrote: >>>Hi, >>> >>>when adding AD trusts support, we need to ensure we have valid kerberos >>>ticket of the user from 'admins' group or otherwise appropriate ACIs >>>will not be granted. >>> >>>This patch introduces a check for that. We already check if >>>ipa-adtrust-install is run by root so this complements existing checks. >>> >>>https://fedorahosted.org/freeipa/ticket/2815 >>After discussing on IRC with Simo and Rob, we came to conclusion that it >>is possible to switch to LDAPI and autobind feature of dirsrv for >>authentication and remove requirement for Directory Manager credentials >>altogether. >> >>Updated patch makes use of LDAPI + autobind under root privileges to map >>automatically to Directory Manager privileges in dirsrv. Additionally it >>ensures we have Kerberos credentials to fetch keytab with CIFS service >>key. >> >>Service._ldap_mod() is extended to switch to autobind when self.ldapi is >>set to True and we are running as root. >> >>For those interested in why ACIError is mapped to 'outdated Kerberos >>credentials' error message, this is because we'll get ACIError for 'ipa >>user-show ' command when authenticated by the Kerberos credentials >>for in a default ccache only when Kerberos credentials are stale -- >>either belong to a user that was removed or to a previous IPA install >>that was wiped before reinstalling. The latter is how I discovered >>this case. :) > >I think that this should raise an exception if one tries to use >ldapi, doesn't provide the DM password and is not root. Otherwise it >won't authenticate at all. This is not exactly true. $ id uid=757000001(abokovoy) gid=757000001(abokovoy) groups=757000001(abokovoy) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 $ ldapsearch -vvv -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) SASL/EXTERNAL authentication started ldap_sasl_interactive_bind_s: Inappropriate authentication (48) additional info: SASL EXTERNAL bind requires an SSL connection $ ldapsearch -vvv -Y GSSAPI -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) SASL/GSSAPI authentication started SASL username: abokovoy at IPA.LOCAL SASL SSF: 56 SASL data security layer installed. filter: (objectclass=*) requesting: * So GSSAPI auth works with LDAPI access. I can simply enforce -Y GSSAPI when non-root and no dm_password regardless of self.ldapi, this would extend previously available logic to following: - ldapi: use -H ldapi://url instead of -h hostname - dm_password: add Directory Manager auth - root without dm_password: use autobind - non-root without dm_password: use GSSAPI >In reality, I think all this service code always runs as root, so it >may be a moot point, but this code is kinda convoluted. Yep. -- / Alexander Bokovoy From pviktori at redhat.com Wed Jul 18 11:21:13 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 18 Jul 2012 13:21:13 +0200 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig Message-ID: <50069C29.1000304@redhat.com> minimum_connections was sometimes not updated properly on install because the script set psearch on but assumed it was still off. Also, the number of connections was not set if the directive was missing. Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0070-Fix-updating-minimum_connections-in-ipa-upgradeconfi.patch Type: text/x-patch Size: 3043 bytes Desc: not available URL: From pspacek at redhat.com Wed Jul 18 11:32:10 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 18 Jul 2012 13:32:10 +0200 Subject: [Freeipa-devel] [PATCH 0032-0035] Message-ID: <50069EBA.9080400@redhat.com> Hello, attached patch 0032 adds support for MODDN operation to persistent search implementation. Related ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/72 Patches 0033-0035 does minor cleanup in old persistent search code. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0032-Add-support-for-modify-DN-operation-to-persistent-se.patch Type: text/x-patch Size: 8329 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0033-Rename-persistent-search-update_action-to-update_zon.patch Type: text/x-patch Size: 1209 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0034-Minor-code-cleanup-in-persistent-search-error-handli.patch Type: text/x-patch Size: 1520 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0035-Minor-persistent-search-logging-cleanup.patch Type: text/x-patch Size: 2952 bytes Desc: not available URL: From pviktori at redhat.com Wed Jul 18 11:39:21 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 18 Jul 2012 13:39:21 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5005EB9D.1010003@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFBE66E.9040802@redhat.com> <5005EB9D.1010003@redhat.com> Message-ID: <5006A069.5020100@redhat.com> On 07/18/2012 12:47 AM, John Dennis wrote: > On 07/10/2012 04:23 AM, Petr Viktorin wrote: >> I've read your summary (which you should summarize into a commit message >> before this is pushed), and gone through the patch. >> Here is what I found doing that; I didn't get to actual testing yet. >> I also didn't do a detailed review of ldap2.py changes yet. > > Thank you for your careful review Petr, you had many good suggestions > and comments. It was an enormous amount of material to wade through, > thank you for your diligence. I'm in agreement with most of your > suggestions. Great :) I don't have much to reply, I mostly agree too. [...] >> >> > Originally I was against automatic type promotion on the belief if >> you were comparing a string to a DN it was a failure of program logic, >> but there were some circumstances where it was evil necessity. >> >> Would you care to elaborate? I also think that's a logic failure; such >> circumstances should at least be marked by a TODO. > > I should have taken better notes because I don't remember the exact > place this came up, it was late in the testing and I believe the problem > showed up in the ldap updater, possibly some other places. > > But here is the bottom line: I encountered a handful of places in the > code where a string was being compared to a DN object for equality (I > seem to recall the equality comparison was indirect by virtue of the > "in" operator). What I do recall is it was logically impossible to know > a prori the string being tested was supposed to be a DN. If it were > possible to know the string represented a DN it would have been easy to > promote it to a DN object. But there were a handful of situations where > there simply was not enough information available to ascertain if the > string was logically a DN or not. This was never an issue when dn's were > simple strings. If it's getting compared to a DN that's a pretty strong > hint it's supposed to be a DN (fwiw the coercion must occur in the > equality operator). > > So I pondered the problem for a while and decided there was legitimate > value in allowing an equality comparison between a string and DN. I > could see no downside to it other than it *might* mask an error in > program logic. I didn't take the decision lightly. I looked through the > Python internals to see where type coercion occurs. While not often it's > certainly done where "it makes sense" (e.g. between string types, > between numerical types, between buffer types, etc.). I thought one > could make the argument that a DN object is akin to a string type and > hence falls into the same category of coercion as occurs between string > types. I'm not saying I'm in love with it, rather it's a pragmatic > necessity and can be justified if you look at it a certain way. > > We could add a debug message with a backtrace whenever the coercion > occurs if we wanted to completely eliminate all places in the code where > a string-to-DN comparison occurs. But as I said I'm not sure it's > possible to absolutely eliminate this comparison. We've eliminated 99.9% > of those cases already, I'm not sure the other 0.1% can be eliminated or > that it's worth the work involved. I'm happy with 99.9% :-) Fair enough. >> >> > The find() and rfind() methods (modeled after their string cousins) >> to locate a RDN or DN in an existing DN. >> >> I would much rather see the index() and rindex() methods, which do the >> Pythonic thing of raising an exception rather than silently returning -1 >> when the thing isn't found. >> Compare: >> something[something.find(thing)] => something[-1] if not found >> something[something.index(thing)] => raises exception if not found > > You're right exceptions are more Pythoninc than return codes. I can add > index and rindex and convert the code to catch an exception. > > However, we use find and rfind extensively in the IPA code base > (DN.find() and DN.rfind() simply replaced their string equivalents > during the conversion). Perhaps you should open a ticket to purge all > use of find() and rfind() from our code base and add a prohibition > against their use in our Python coding standard document. I don't think it's that urgent. >> >> >> ipapython/dn.py:448: >> Each argument must be scalar convertable to unicode or a callable >> object whose result is convertable to unicode. >> >> Is it really necessary to allow callables here? If the user has a >> callable, he/she should call it before making an AVA out of it. Same >> with converting to Unicode. >> As you point out in dn_summary, automatic coercion is evil. > > There are two issues here, type coercion and string encoding. > > Not all coercion is evil, there is value in making it easy to do the > right thing and minimizing inline coercion that otherwise obfuscates the > logical intent of the code. We shouldn't require programmers to be > perfect coders who never forget to perform some operation in the > hundreds of places it might be required. Rather the tools should quietly > and consistently enforce the rules while leaving the code readable, > concise and with clear intent. > > There is a well established precedent in Python for coercion to string > in a string context. Coercion to string is so fundamental to Python it > has it's own low-level operator embedded in every class type definition > utilized in both Python and the CPython implementations. > > > 1) We know all DN components must be strings. Let's say someone passes > an integer object; is there really any harm to converting this to a > string representation of an integer? That makes the DN API much > friendlier and it's use more succinct. In fact Python does automatic > coercion to string in many places where it knows something must be a > string, thus it's a well accepted and established practice. We do > automatic coercion to string in many places in IPA (a good example is > the ldap encode/decode operations). > > I wish other Python API's also coerced to string as a service to the > caller. I wish python-ldap would have coerced to string those elements > of it's API which it knows must be strings. If it had we could have just > passed DN objects into the python-ldap API instead of having to wrap the > class to perform the coercion. This wouldn't be violating a guideline, > rather it would be utilizing a inherent feature of the Python language. > > There are an astonishing number of CPython modules that barf if you pass > the wrong string type. Instead they should be friendly and provide the > right string coercion. (There is a long and sordid history of this > problem and I have a extensive write-up on it). > > 2) All strings in IPA are unicode. In python 3.x all strings will be > unicode (the way it should have been in python 2.x alas). When string > comparisons are performed they should be done as unicode objects, not > encoded in some undefined encoding. Likewise when returning a string > belonging to a component in a DN it should be unicode. Traditionally > dn's have been str objects (a python-ldap foible), so the decoding of > str to unicode promotes correctness in Python 2.x. > > Accepting a callable is just a further refinement of the discussion in > #1. If you allow for automatic coercion to sting (in a context that > demands a string type) then why not allow anything which yields a > string? It just makes the API friendlier and easier to use without > introducing ugly compromises. Some coercion is evil, but some is darn > nice and useful. The trick is recognizing which is which. You have a point for the conversion to strings. However, I'm still not convinced automatic function calling is the good kind of coercion. It's different from the string conversions Python does automatically -- str(function) gives you '' -- so it's not something one would expect. Silently doing unexpected things will lead to errors. Plus, it saves all of two parentheses of typing, so it's not even that much more friendly. [...] >> >> In several places: >> + # Use a property for to assure it's always a DN type >> + def _set_(self, value): >> + self._ = DN(value) >> + >> + def _get_(self): >> + assert isinstance(getattr(self, '_'), DN) >> + return self._ >> + >> + ldap_suffix = property(_get_, _set_) >> >> Shouldn't the assert be in the setter? Do we not want people to only put >> DNs in these properties, instead of silently converting? > > I'm afraid I don't follow you here. When setting the property it should > be coerced to a DN, we want to force the property to always be a DN. > > However it's possible for someone to set the _ directly even > though they should never do that, the assert in the getter is just > bullet-proofing against that. > >> This appears enough times to justify using a helper to construct these. >> But if you don't want to generalize, the getattr call is unnecessary. > > Once again I don't follow you. How would a helper work? Something like: def checked_dn_property(private_name): def setter(self, value): setattr(self, private_name, DN(value)) def getter(self): value = getattr(self, private_name) assert isinstance(value, DN) return value return property(getter, setter) ... ldap_suffix = checked_dn_property('_ldap_suffix') > I don't know why there is a getattr in the getter, it would seem to be > unnecessary unless I was trying to protect against an uninitialized > attribute. The getattr should probably be replaced with normal class > attribute access. I'll fix that. > >> tests/util.py:307: >> if isinstance(expected, DN): >> if isinstance(got, basestring): >> got = DN(got) >> >> This may be overly broad, the tests should be able to check if they got >> a DN or a string. What about something like: >> >> if isinstance(expected, DNString): >> assert isinstance(got, basestring) >> assert DN(got) == expected.dn >> with a suitable DNString class? > > Sorry I don't understand what you're trying to protect against nor what > a DNString class would be or what it's purpose is. > Our tests check types quite rigorously. The changes in the patch make them ignore the difference between DNs and strings -- the test will pass regardless of whether it gets a DN or a string. It seems that to keep the tests in this spirit, it would be better to have a way to explicitly say we want ?a string in the form of a DN?. For this there could be a DNString class for use in the tests, similar to the Fuzzy class we already use. Something like class DNString(object): def __init__(self, *args): self.dn = DN(args) def __eq__(self, other): assert isinstance(other, basestring) return self.dn == DN(other) def __ne__(self, other): return not self == other But I guess it's not that necessary. >> ipapython/dn.py:541: def _set_locked(self, value): >> There should be only one way to do things, either the lock/unlock >> methods, or setting an attribute. More ways to do one same thing lead to >> subtle errors in one of the ways. >> >> The locking code is duplicated in AVA, RDN and DN; it would be good to >> make a Lockable mixin to avoid that. A similar case is with __repr__. >> >> Anyway unlocking is bad (see above) so these are mostly moot points. > > A mixin would eliminate code duplication but in this case the > duplication is trivial, not sure it warrants introducing a new class in > this circumstance (judgment call), but the topic is moot because locking > is going to be eliminated. > >> >> ipaserver/plugins/ldap2.py:134: >> class ServerSchema(object): >> >> Please don't use nested classes without reason. > > There is a valid reason, the ServerSchema class is private to the > SchemaCache class. If it's private, I'd just name it with an underscore. Nested classes are more trouble than it's worth, for example the string representation will show the unreachable 'ipaserver.plugins.ldap2.ServerSchema', and tools like pickle will choke on them. Flat is better than nested. [...] > >> > Python provides the __all__ module export list, we really should be >> > making better use of that. >> >> I recommend importing names explicitly, or importing a module and using >> qualified names, over __all__ and import *. Star imports make it >> unnecessarily hard to see where names come from. >> (You do use explicit imports in the code, my gripe is just about >> recommending * and __all__). > > We should come to a project consensus on this topic and add it to our > Python style guide. This really isn't germane to the dn work. There's an issue filed in the deferred bucket (https://fedorahosted.org/freeipa/ticket/2653) [...] -- Petr? From pspacek at redhat.com Wed Jul 18 11:41:30 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 18 Jul 2012 13:41:30 +0200 Subject: [Freeipa-devel] [PATCH 0032-0035] Add support for MODDN operation to persistent search implementation In-Reply-To: <50069EBA.9080400@redhat.com> References: <50069EBA.9080400@redhat.com> Message-ID: <5006A0EA.1010504@redhat.com> Sorry for the missing subject! Petr^2 Spacek On 07/18/2012 01:32 PM, Petr Spacek wrote: > adds support for MODDN operation to persistent search implementation From pviktori at redhat.com Wed Jul 18 11:43:24 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 18 Jul 2012 13:43:24 +0200 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <5005CDF7.2060205@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> Message-ID: <5006A15C.3040406@redhat.com> On 07/17/2012 10:41 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 06/29/2012 11:28 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 06/25/2012 03:00 PM, Petr Viktorin wrote: >>>>> On 06/20/2012 06:15 PM, Rob Crittenden wrote: >>>>>> Petr Viktorin wrote: >>>>>>> On 06/04/2012 04:56 PM, Petr Viktorin wrote: >>>>>>>> Currently, FreeIPA's install/admin scripts are long pieces of code >>>>>>>> that aren't very reusable, importable, or testable. >>>>>>>> They have been extended over time with features such as logging and >>>>>>>> error handling, but since each tool was extended individually, >>>>>>>> there >>>>>>>> is much inconsistency and code duplication. >>>>>>>> This patch starts a framework which the admin tools can use, and >>>>>>>> converts ipa-ldap-updater to use the framework. >>>>>>>> >>>>>>>> In an earlier patch I found that improving a particular >>>>>>>> functionality in >>>>>>>> all the commands is not workable, so I want to tackle this one tool >>>>>>>> at a >>>>>>>> time. >>>>>>>> I'm starting with ipa-ldap-updater, because it's pretty small, >>>>>>>> doesn't >>>>>>>> use DNs (I don't want conflicts with John's work), and has the >>>>>>>> interesting --upgrade option. >>>>>>>> >>>>>>>> >>>>>>>> The framework does these tasks: >>>>>>>> - Parse options >>>>>>>> - Select tool to run (see below) >>>>>>>> - Validate options >>>>>>>> - Set up logging >>>>>>>> - Run the tool code >>>>>>>> - Handle any errors >>>>>>>> - Log success/failure >>>>>>>> >>>>>>>> The base class has some defaults for these that the tools can >>>>>>>> extend/override. >>>>>>>> >>>>>>>> >>>>>>>> To handle the case where one script does two different things >>>>>>>> (ipa-ldap-updater with/without --upgrade, or ipa-server-install >>>>>>>> with/without --uninstall), I want to split the tool in two classes >>>>>>>> rather than have repeated ifs in the code. >>>>>>>> This meant that option parsing (and initializing the parser) has >>>>>>>> to be >>>>>>>> done before creating an instance of the tool. I use a factory >>>>>>>> classmethod. >>>>>>>> >>>>>>>> >>>>>>>> I put the admintool base class in ipapython/ as it should be useful >>>>>>>> for >>>>>>>> ipa-client-install as well. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> First part of the work for: >>>>>>>> https://fedorahosted.org/freeipa/ticket/2652 >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Attaching rebased patch. >>>>>> >>>>>> I gather you want people to be calling run_cli() in their admin >>>>>> tools. >>>>>> Should main() be made private then? I could see someone getting >>>>>> confused >>>>>> and using main instead, which would work, but then the return value >>>>>> might not do the right thing. >>>>>> >>>>>> Or maybe just drop run_cli and have main exit with sys.exit()? >>>>> >>>>> I don't see why running a command as a Python function should be >>>>> discouraged. In fact it could even help -- for example logging could >>>>> only be set up once, so if we call, say, ipa-ldap-updater from >>>>> ipa-server-install, all related logs would go to a single file. >>>>> A C-style main (taking a list of arguments and returning the exit >>>>> status) is a good thing for modularity and testability. >>>>> The `run_cli` method is just a convenient shortcut for the usual >>>>> usage, >>>>> so the calling modules can be as small as possible. >>>>> >>>>> If people get confused and call main instead of run_cli, they need to >>>>> manually pass in sys.argv. I think this is enough of a warning that >>>>> their assumptions aren't right. >>>>> To make it even clearer I've removed the possibility to pass None as >>>>> argv to main() and have it auto-filled. >>>>> >>>>> Some relevant reading: >>>>> http://www.artima.com/weblogs/viewpost.jsp?thread=4829 (old but still >>>>> valid) >>>>> http://en.wikipedia.org/wiki/Main_function#Python >>>>> >>>>>> It isn't correctly handling the case of an update not found: >>>>>> >>>>>> ipa : INFO Parsing file ad >>>>>> [Errno 2] No such file or directory: 'ad' >>>>>> ipa : INFO File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>>> 151, in >>>>>> execute >>>>>> self.run() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ldap_updater.py", >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> line >>>>>> 180, in run >>>>>> modified = ld.update(self.files) >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", >>>>>> line 828, in update >>>>>> sys.exit(1) >>>>>> >>>>>> ipa : INFO The ipa-ldap-updater command failed, exception: >>>>>> SystemExit: 1 >>>>> >>>>> I've added validation for missing files, and improved the error >>>>> message >>>>> ldapupdate raises (for cases the validation doesn't catch, like >>>>> passing >>>>> directories or unreadable files). >>>>> Ideally ldapupdate would not try to handle the error itself, but that >>>>> code is used in more places that I don't want to break, so I'm leaving >>>>> the extraneous print in. >>>>> >>>>>> Running in test mode with the attached update doesn't seem to work >>>>>> either. There is nothing special about this file, just something I >>>>>> had >>>>>> lying around: >>>>>> >>>>>> ipa : INFO File >>>>>> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line >>>>>> 151, in >>>>>> execute >>>>>> self.run() >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_ldap_updater.py", >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> line >>>>>> 184, in run >>>>>> 'Update complete, changes to be made, test mode', 2) >>>>>> >>>>>> ipa : INFO The ipa-ldap-updater command failed, exception: >>>>>> ScriptError: >>>>>> Update complete, changes to be made, test mode >>>>>> ipa : ERROR Update complete, changes to be made, test mode >>>>>> >>>>>> ipa : ERROR None >>>>> >>>>> Fixed. >>>>> >>>>>> The unit tests still pass which is good. >>>>>> >>>>>> With ipa-ldap-updater the return value is a bit strange. All the >>>>>> updates >>>>>> themselves can fail for one reason or another and the command can >>>>>> still >>>>>> consider this a success (it may fail because a feature is not >>>>>> enabled, >>>>>> for example). Still, the success message displayed at the end is a >>>>>> bit >>>>>> jarring when the updates themselves aren't applied. Here is a snippet >>>>>> when running ad.update live: >>>>>> >>>>>> ipa : INFO New entry: >>>>>> uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>>> ipa : DEBUG --------------------------------------------- >>>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>>> ipa : DEBUG add: 'account' to objectClass, current value [] >>>>>> ipa : DEBUG add: updated value [u'account'] >>>>>> ipa : DEBUG --------------------------------------------- >>>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>>> ipa : DEBUG objectClass: >>>>>> ipa : DEBUG account >>>>>> ipa : DEBUG add: 'adtrust' to uid, current value [] >>>>>> ipa : DEBUG add: updated value [u'adtrust'] >>>>>> ipa : DEBUG --------------------------------------------- >>>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>>> ipa : DEBUG objectClass: >>>>>> ipa : DEBUG account >>>>>> ipa : DEBUG uid: >>>>>> ipa : DEBUG adtrust >>>>>> ipa : DEBUG --------------------------------------------- >>>>>> ipa : DEBUG Final value >>>>>> ipa : DEBUG dn: uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>>> ipa : DEBUG objectClass: >>>>>> ipa : DEBUG account >>>>>> ipa : DEBUG uid: >>>>>> ipa : DEBUG adtrust >>>>>> ipa : INFO Parent DN of >>>>>> uid=adtrust,cn=notfound,cn=etc,dc=greyoak,dc=com >>>>>> may not exist, cannot create the entry >>>>>> ipa : INFO The ipa-ldap-updater command was successful >>>>>> [root at pinto freeipa]# echo $? >>>>>> 0 >>>>>> >>>>>> This may be contrasting just because it is a contrived case. The >>>>>> command >>>>>> rval is separate from whether the updates all applied, so maybe this >>>>>> is ok. >>>>> >>>>> The current ipa-ldap-updater also works this way, so this should go >>>>> in a >>>>> separate ticket. >>>>> I worry that changing the return value could make installations fail, >>>>> for example. >>>>> >>>>>> rob >>>>> >>>>> >>>>> Thanks for the review! >>>>> >>>> >>>> Once again, this time with the patch. >>> >>> Almost there. When running in test mode and an update that would be >>> applied should return 2. >>> >>> rob >>> >>> >> >> Fixed >> >> > > Ok, things seem to work. One last question. > > This puts all the guts of the work into a file in ipaserver/install/. > > Is there any benefit to that over putting the class into the tool > itself? I'm trying to think ahead as more commands get migrated, we're > just going to be adding more files to ipaserver/install and making > shells out of the tools in install/tools. > > rob Code in a package is importable and thus testable. It could go somewhere like ipaserver/install/tools, but I figured the ipa_ prefix makes it stand out enough so I left it in ipaserver/install. Attaching rebased patch. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0056-05-Framework-for-admin-install-tools-with-ipa-ldap-upda.patch Type: text/x-patch Size: 31163 bytes Desc: not available URL: From pspacek at redhat.com Wed Jul 18 11:46:18 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 18 Jul 2012 13:46:18 +0200 Subject: [Freeipa-devel] [PATCH 0036] Raise connection count automatically if serial_autoincrement is enabled Message-ID: <5006A20A.4010005@redhat.com> Hello, this patch reflects new demand from serial_autoincrement feature. Generally, change in configuration file should by IPA install/upgrade scripts. This patch prevents deadlock in situation where scripts failed in their job (as you can see right now). Will be obsoleted by https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0036-Raise-connection-count-automatically-if-serial_autoi.patch Type: text/x-patch Size: 1078 bytes Desc: not available URL: From pspacek at redhat.com Wed Jul 18 12:35:20 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 18 Jul 2012 14:35:20 +0200 Subject: [Freeipa-devel] [PATCH 0037] Add missing return value check to new_ldap_instance() Message-ID: <5006AD88.5030307@redhat.com> Hello, this patch adds missing return value check to new_ldap_instance(). https://fedorahosted.org/bind-dyndb-ldap/ticket/85 Bug was reported by Coverity. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0037-Add-missing-return-value-check-to-new_ldap_instance.patch Type: text/x-patch Size: 1010 bytes Desc: not available URL: From abokovoy at redhat.com Wed Jul 18 12:55:34 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 15:55:34 +0300 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <20120718090124.GM12345@redhat.com> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> <5005A30D.4040702@redhat.com> <20120718090124.GM12345@redhat.com> Message-ID: <20120718125534.GN12345@redhat.com> On Wed, 18 Jul 2012, Alexander Bokovoy wrote: >On Tue, 17 Jul 2012, Rob Crittenden wrote: >>Alexander Bokovoy wrote: >>>On Fri, 13 Jul 2012, Alexander Bokovoy wrote: >>>>Hi, >>>> >>>>when adding AD trusts support, we need to ensure we have valid kerberos >>>>ticket of the user from 'admins' group or otherwise appropriate ACIs >>>>will not be granted. >>>> >>>>This patch introduces a check for that. We already check if >>>>ipa-adtrust-install is run by root so this complements existing checks. >>>> >>>>https://fedorahosted.org/freeipa/ticket/2815 >>>After discussing on IRC with Simo and Rob, we came to conclusion that it >>>is possible to switch to LDAPI and autobind feature of dirsrv for >>>authentication and remove requirement for Directory Manager credentials >>>altogether. >>> >>>Updated patch makes use of LDAPI + autobind under root privileges to map >>>automatically to Directory Manager privileges in dirsrv. Additionally it >>>ensures we have Kerberos credentials to fetch keytab with CIFS service >>>key. >>> >>>Service._ldap_mod() is extended to switch to autobind when self.ldapi is >>>set to True and we are running as root. >>> >>>For those interested in why ACIError is mapped to 'outdated Kerberos >>>credentials' error message, this is because we'll get ACIError for 'ipa >>>user-show ' command when authenticated by the Kerberos credentials >>>for in a default ccache only when Kerberos credentials are stale -- >>>either belong to a user that was removed or to a previous IPA install >>>that was wiped before reinstalling. The latter is how I discovered >>>this case. :) >> >>I think that this should raise an exception if one tries to use >>ldapi, doesn't provide the DM password and is not root. Otherwise >>it won't authenticate at all. >This is not exactly true. > >$ id >uid=757000001(abokovoy) gid=757000001(abokovoy) groups=757000001(abokovoy) >context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 > >$ ldapsearch -vvv -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null >ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) >SASL/EXTERNAL authentication started >ldap_sasl_interactive_bind_s: Inappropriate authentication (48) > additional info: SASL EXTERNAL bind requires an SSL connection > >$ ldapsearch -vvv -Y GSSAPI -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null >ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) >SASL/GSSAPI authentication started >SASL username: abokovoy at IPA.LOCAL >SASL SSF: 56 >SASL data security layer installed. >filter: (objectclass=*) >requesting: * > >So GSSAPI auth works with LDAPI access. I can simply enforce -Y GSSAPI >when non-root and no dm_password regardless of self.ldapi, this would >extend previously available logic to following: > >- ldapi: use -H ldapi://url instead of -h hostname >- dm_password: add Directory Manager auth >- root without dm_password: use autobind >- non-root without dm_password: use GSSAPI > >>In reality, I think all this service code always runs as root, so >>it may be a moot point, but this code is kinda convoluted. >Yep. Here is updated patch. -- / Alexander Bokovoy -------------- next part -------------- >From 81b1e0305ac8ad3516bb7bcaeb13999480e0f14f Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/5] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 42 +++++++++++++++++++++++-------- install/tools/man/ipa-adtrust-install.1 | 3 --- ipaserver/install/adtrustinstance.py | 21 ++++++++-------- ipaserver/install/service.py | 15 +++++++++-- 4 files changed, 55 insertions(+), 26 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..f367d5b2b516bd411bce9275ff299eb3ffdf6bf9 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -86,6 +84,30 @@ def read_netbios_name(netbios_default): return netbios_name +def ensure_kerberos_admin_rights(api): + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + api.Backend.ldap2.connect(ccache.name) + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + api.Backend.ldap2.disconnect() + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except Exception, e: + error_messages = dict( + ACIError = "Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket", + Krb5Error = "Must have Kerberos credentials to setup AD trusts on server", + RequirementError = "Must have administrative privileges to setup AD trusts on server" + ) + name = type(e).__name__ + if name in error_messages: + sys.exit(error_messages[name]) + else: + sys.exit("Unrecognized error during check of admin rights: %s\n%s" % (name, str(e))) + def main(): safe_options, options = parse_options() @@ -128,6 +150,8 @@ def main(): api.bootstrap(**cfg) api.finalize() + ensure_kerberos_admin_rights(api) + if adtrustinstance.ipa_smb_conf_exists(): if not options.unattended: while True: @@ -194,9 +218,8 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm # try the connection try: @@ -205,12 +228,9 @@ def main(): except ldap.INVALID_CREDENTIALS, e: sys.exit("Password is not valid!") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + # See if our LDAP server is up and we can talk to it over GSSAPI + ccache = krbV.default_context().default_ccache().name + api.Backend.ldap2.connect(ccache) smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..65c8e1b87412fc6aba1187615e39c2ec523e00e5 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -107,15 +107,26 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # Support autobind when running as root and using ldapi + if self.ldapi: + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + else: + args += ["-h", hostname] + if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + auth_params = [] + if os.getegid() != 0: + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: -- 1.7.10.4 From abokovoy at redhat.com Wed Jul 18 12:58:09 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 15:58:09 +0300 Subject: [Freeipa-devel] [PATCH] 0063 change sid_check_is_domain() to sid_check_is_our_sam() Message-ID: <20120718125809.GO12345@redhat.com> Hi, due to API change in Samba4 between beta3 and beta4, following small patch is needed for ipasam. I've also added forward declaration for the function. https://fedorahosted.org/freeipa/ticket/2929 -- / Alexander Bokovoy -------------- next part -------------- >From d58b997587551744515c50e019148adf005f5c3f Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Wed, 18 Jul 2012 15:52:33 +0300 Subject: [PATCH 5/5] Follow change in samba4 beta4 for sid_check_is_domain to sid_check_is_our_sam With c43505b621725c9a754f0ee98318d451b093f2ed in samba git master the function sid_check_is_domain() was renamed to sid_check_is_our_sam(). https://fedorahosted.org/freeipa/ticket/2929 --- daemons/ipa-sam/ipa_sam.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index 86ed3fbd3e6d1894fd398c3c1c94d34c2b7ec273..ab4b116c5f2b3b8dae6e8309403afba5fdf86708 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -83,6 +83,8 @@ enum ndr_err_code ndr_pull_trustAuthInOutBlob(struct ndr_pull *ndr, int ndr_flag bool fetch_ldap_pw(char **dn, char** pw); /* available in libpdb.so */ void nt_lm_owf_gen(const char *pwd, uint8_t nt_p16[16], uint8_t p16[16]); /* available in libcliauth.so */ bool sid_check_is_builtin(const struct dom_sid *sid); /* available in libpdb.so */ +/* available in libpdb.so, renamed from sid_check_is_domain() in c43505b621725c9a754f0ee98318d451b093f2ed */ +bool sid_check_is_our_sam(const struct dom_sid *sid); void strlower_m(char *s); /* available in libutil_str.so */ char *talloc_asprintf_strupper_m(TALLOC_CTX *t, const char *fmt, ...); /* available in libutil_str.so */ void sid_copy(struct dom_sid *dst, const struct dom_sid *src); /* available in libsecurity.so */ @@ -300,7 +302,7 @@ static NTSTATUS ldapsam_lookup_rids(struct pdb_methods *methods, } if (!sid_check_is_builtin(domain_sid) && - !sid_check_is_domain(domain_sid)) { + !sid_check_is_our_sam(domain_sid)) { result = NT_STATUS_INVALID_PARAMETER; goto done; } -- 1.7.10.4 From rcritten at redhat.com Wed Jul 18 13:03:09 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Jul 2012 09:03:09 -0400 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <20120718090124.GM12345@redhat.com> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> <5005A30D.4040702@redhat.com> <20120718090124.GM12345@redhat.com> Message-ID: <5006B40D.2030505@redhat.com> Alexander Bokovoy wrote: > On Tue, 17 Jul 2012, Rob Crittenden wrote: >> Alexander Bokovoy wrote: >>> On Fri, 13 Jul 2012, Alexander Bokovoy wrote: >>>> Hi, >>>> >>>> when adding AD trusts support, we need to ensure we have valid kerberos >>>> ticket of the user from 'admins' group or otherwise appropriate ACIs >>>> will not be granted. >>>> >>>> This patch introduces a check for that. We already check if >>>> ipa-adtrust-install is run by root so this complements existing checks. >>>> >>>> https://fedorahosted.org/freeipa/ticket/2815 >>> After discussing on IRC with Simo and Rob, we came to conclusion that it >>> is possible to switch to LDAPI and autobind feature of dirsrv for >>> authentication and remove requirement for Directory Manager credentials >>> altogether. >>> >>> Updated patch makes use of LDAPI + autobind under root privileges to map >>> automatically to Directory Manager privileges in dirsrv. Additionally it >>> ensures we have Kerberos credentials to fetch keytab with CIFS service >>> key. >>> >>> Service._ldap_mod() is extended to switch to autobind when self.ldapi is >>> set to True and we are running as root. >>> >>> For those interested in why ACIError is mapped to 'outdated Kerberos >>> credentials' error message, this is because we'll get ACIError for 'ipa >>> user-show ' command when authenticated by the Kerberos credentials >>> for in a default ccache only when Kerberos credentials are >>> stale -- >>> either belong to a user that was removed or to a previous IPA install >>> that was wiped before reinstalling. The latter is how I discovered >>> this case. :) >> >> I think that this should raise an exception if one tries to use ldapi, >> doesn't provide the DM password and is not root. Otherwise it won't >> authenticate at all. > This is not exactly true. > > $ id > uid=757000001(abokovoy) gid=757000001(abokovoy) groups=757000001(abokovoy) > context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 > > $ ldapsearch -vvv -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' > >/dev/null > ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) > SASL/EXTERNAL authentication started > ldap_sasl_interactive_bind_s: Inappropriate authentication (48) > additional info: SASL EXTERNAL bind requires an SSL connection > > $ ldapsearch -vvv -Y GSSAPI -H > ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null > ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) > SASL/GSSAPI authentication started > SASL username: abokovoy at IPA.LOCAL > SASL SSF: 56 > SASL data security layer installed. > filter: (objectclass=*) > requesting: * > So GSSAPI auth works with LDAPI access. I can simply enforce -Y GSSAPI > when non-root and no dm_password regardless of self.ldapi, this would > extend previously available logic to following: > > - ldapi: use -H ldapi://url instead of -h hostname > - dm_password: add Directory Manager auth > - root without dm_password: use autobind > - non-root without dm_password: use GSSAPI > >> In reality, I think all this service code always runs as root, so it >> may be a moot point, but this code is kinda convoluted. > Yep. > Sorry, I didn't mean this in general, but this code is going to result in no auth being used which is going to fail. So rather than failing with some no auth error we either enforce that Service is executed as root or raise an exception in this auth code. In other words, if we know something isn't going to work before we try it, we should fail with a more friendly error. rob From simo at redhat.com Wed Jul 18 13:13:23 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 18 Jul 2012 09:13:23 -0400 Subject: [Freeipa-devel] [PATCH] 0063 change sid_check_is_domain() to sid_check_is_our_sam() In-Reply-To: <20120718125809.GO12345@redhat.com> References: <20120718125809.GO12345@redhat.com> Message-ID: <1342617203.3219.73.camel@willson.li.ssimo.org> On Wed, 2012-07-18 at 15:58 +0300, Alexander Bokovoy wrote: > Hi, > > due to API change in Samba4 between beta3 and beta4, following small > patch is needed for ipasam. > > I've also added forward declaration for the function. > > https://fedorahosted.org/freeipa/ticket/2929 Nack, please add build requires in spec file to require beta4. Please consider it acked automatically as soon as you do that and then please push asap. Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Wed Jul 18 13:19:34 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 16:19:34 +0300 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <5006B40D.2030505@redhat.com> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> <5005A30D.4040702@redhat.com> <20120718090124.GM12345@redhat.com> <5006B40D.2030505@redhat.com> Message-ID: <20120718131934.GP12345@redhat.com> On Wed, 18 Jul 2012, Rob Crittenden wrote: >Alexander Bokovoy wrote: >>On Tue, 17 Jul 2012, Rob Crittenden wrote: >>>Alexander Bokovoy wrote: >>>>On Fri, 13 Jul 2012, Alexander Bokovoy wrote: >>>>>Hi, >>>>> >>>>>when adding AD trusts support, we need to ensure we have valid kerberos >>>>>ticket of the user from 'admins' group or otherwise appropriate ACIs >>>>>will not be granted. >>>>> >>>>>This patch introduces a check for that. We already check if >>>>>ipa-adtrust-install is run by root so this complements existing checks. >>>>> >>>>>https://fedorahosted.org/freeipa/ticket/2815 >>>>After discussing on IRC with Simo and Rob, we came to conclusion that it >>>>is possible to switch to LDAPI and autobind feature of dirsrv for >>>>authentication and remove requirement for Directory Manager credentials >>>>altogether. >>>> >>>>Updated patch makes use of LDAPI + autobind under root privileges to map >>>>automatically to Directory Manager privileges in dirsrv. Additionally it >>>>ensures we have Kerberos credentials to fetch keytab with CIFS service >>>>key. >>>> >>>>Service._ldap_mod() is extended to switch to autobind when self.ldapi is >>>>set to True and we are running as root. >>>> >>>>For those interested in why ACIError is mapped to 'outdated Kerberos >>>>credentials' error message, this is because we'll get ACIError for 'ipa >>>>user-show ' command when authenticated by the Kerberos credentials >>>>for in a default ccache only when Kerberos credentials are >>>>stale -- >>>>either belong to a user that was removed or to a previous IPA install >>>>that was wiped before reinstalling. The latter is how I discovered >>>>this case. :) >>> >>>I think that this should raise an exception if one tries to use ldapi, >>>doesn't provide the DM password and is not root. Otherwise it won't >>>authenticate at all. >>This is not exactly true. >> >>$ id >>uid=757000001(abokovoy) gid=757000001(abokovoy) groups=757000001(abokovoy) >>context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 >> >>$ ldapsearch -vvv -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >> >/dev/null >>ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) >>SASL/EXTERNAL authentication started >>ldap_sasl_interactive_bind_s: Inappropriate authentication (48) >> additional info: SASL EXTERNAL bind requires an SSL connection >> >>$ ldapsearch -vvv -Y GSSAPI -H >>ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null >>ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) >>SASL/GSSAPI authentication started >>SASL username: abokovoy at IPA.LOCAL >>SASL SSF: 56 >>SASL data security layer installed. >>filter: (objectclass=*) >>requesting: * >>So GSSAPI auth works with LDAPI access. I can simply enforce -Y GSSAPI >>when non-root and no dm_password regardless of self.ldapi, this would >>extend previously available logic to following: >> >>- ldapi: use -H ldapi://url instead of -h hostname >>- dm_password: add Directory Manager auth >>- root without dm_password: use autobind >>- non-root without dm_password: use GSSAPI >> >>>In reality, I think all this service code always runs as root, so it >>>may be a moot point, but this code is kinda convoluted. >>Yep. >> > >Sorry, I didn't mean this in general, but this code is going to >result in no auth being used which is going to fail. So rather than >failing with some no auth error we either enforce that Service is >executed as root or raise an exception in this auth code. In other >words, if we know something isn't going to work before we try it, we >should fail with a more friendly error. What the code did before in such case (no DM, non-root, non-ldapi) is to use -Y GSSAPI. So this forces to try GSSAPI auth already. My latest patch (-2) expands this also to (no DM, non-root, ldapi) case. However, one case is not covered yet: no DM, root, non-ldapi. We either can try -Y GSSAPI here as well or raise exception. I'd prefer the former. -- / Alexander Bokovoy From simo at redhat.com Wed Jul 18 13:39:18 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 18 Jul 2012 09:39:18 -0400 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <20120718131934.GP12345@redhat.com> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> <5005A30D.4040702@redhat.com> <20120718090124.GM12345@redhat.com> <5006B40D.2030505@redhat.com> <20120718131934.GP12345@redhat.com> Message-ID: <1342618758.3219.74.camel@willson.li.ssimo.org> On Wed, 2012-07-18 at 16:19 +0300, Alexander Bokovoy wrote: > On Wed, 18 Jul 2012, Rob Crittenden wrote: > >Alexander Bokovoy wrote: > >>On Tue, 17 Jul 2012, Rob Crittenden wrote: > >>>Alexander Bokovoy wrote: > >>>>On Fri, 13 Jul 2012, Alexander Bokovoy wrote: > >>>>>Hi, > >>>>> > >>>>>when adding AD trusts support, we need to ensure we have valid kerberos > >>>>>ticket of the user from 'admins' group or otherwise appropriate ACIs > >>>>>will not be granted. > >>>>> > >>>>>This patch introduces a check for that. We already check if > >>>>>ipa-adtrust-install is run by root so this complements existing checks. > >>>>> > >>>>>https://fedorahosted.org/freeipa/ticket/2815 > >>>>After discussing on IRC with Simo and Rob, we came to conclusion that it > >>>>is possible to switch to LDAPI and autobind feature of dirsrv for > >>>>authentication and remove requirement for Directory Manager credentials > >>>>altogether. > >>>> > >>>>Updated patch makes use of LDAPI + autobind under root privileges to map > >>>>automatically to Directory Manager privileges in dirsrv. Additionally it > >>>>ensures we have Kerberos credentials to fetch keytab with CIFS service > >>>>key. > >>>> > >>>>Service._ldap_mod() is extended to switch to autobind when self.ldapi is > >>>>set to True and we are running as root. > >>>> > >>>>For those interested in why ACIError is mapped to 'outdated Kerberos > >>>>credentials' error message, this is because we'll get ACIError for 'ipa > >>>>user-show ' command when authenticated by the Kerberos credentials > >>>>for in a default ccache only when Kerberos credentials are > >>>>stale -- > >>>>either belong to a user that was removed or to a previous IPA install > >>>>that was wiped before reinstalling. The latter is how I discovered > >>>>this case. :) > >>> > >>>I think that this should raise an exception if one tries to use ldapi, > >>>doesn't provide the DM password and is not root. Otherwise it won't > >>>authenticate at all. > >>This is not exactly true. > >> > >>$ id > >>uid=757000001(abokovoy) gid=757000001(abokovoy) groups=757000001(abokovoy) > >>context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 > >> > >>$ ldapsearch -vvv -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' > >> >/dev/null > >>ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) > >>SASL/EXTERNAL authentication started > >>ldap_sasl_interactive_bind_s: Inappropriate authentication (48) > >> additional info: SASL EXTERNAL bind requires an SSL connection > >> > >>$ ldapsearch -vvv -Y GSSAPI -H > >>ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null > >>ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) > >>SASL/GSSAPI authentication started > >>SASL username: abokovoy at IPA.LOCAL > >>SASL SSF: 56 > >>SASL data security layer installed. > >>filter: (objectclass=*) > >>requesting: * > >>So GSSAPI auth works with LDAPI access. I can simply enforce -Y GSSAPI > >>when non-root and no dm_password regardless of self.ldapi, this would > >>extend previously available logic to following: > >> > >>- ldapi: use -H ldapi://url instead of -h hostname > >>- dm_password: add Directory Manager auth > >>- root without dm_password: use autobind > >>- non-root without dm_password: use GSSAPI > >> > >>>In reality, I think all this service code always runs as root, so it > >>>may be a moot point, but this code is kinda convoluted. > >>Yep. > >> > > > >Sorry, I didn't mean this in general, but this code is going to > >result in no auth being used which is going to fail. So rather than > >failing with some no auth error we either enforce that Service is > >executed as root or raise an exception in this auth code. In other > >words, if we know something isn't going to work before we try it, we > >should fail with a more friendly error. > What the code did before in such case (no DM, non-root, non-ldapi) is to > use -Y GSSAPI. So this forces to try GSSAPI auth already. My latest > patch (-2) expands this also to (no DM, non-root, ldapi) case. > > However, one case is not covered yet: no DM, root, non-ldapi. We either > can try -Y GSSAPI here as well or raise exception. I'd prefer the > former. When in doubt always try GSSAPI please. Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Wed Jul 18 13:50:12 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 16:50:12 +0300 Subject: [Freeipa-devel] [PATCH] 0060 Ensure ipa-adtrust-install is run as admin user In-Reply-To: <1342618758.3219.74.camel@willson.li.ssimo.org> References: <20120713152301.GJ9427@redhat.com> <20120713173354.GK9427@redhat.com> <5005A30D.4040702@redhat.com> <20120718090124.GM12345@redhat.com> <5006B40D.2030505@redhat.com> <20120718131934.GP12345@redhat.com> <1342618758.3219.74.camel@willson.li.ssimo.org> Message-ID: <20120718135012.GQ12345@redhat.com> On Wed, 18 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-18 at 16:19 +0300, Alexander Bokovoy wrote: >> On Wed, 18 Jul 2012, Rob Crittenden wrote: >> >Alexander Bokovoy wrote: >> >>On Tue, 17 Jul 2012, Rob Crittenden wrote: >> >>>Alexander Bokovoy wrote: >> >>>>On Fri, 13 Jul 2012, Alexander Bokovoy wrote: >> >>>>>Hi, >> >>>>> >> >>>>>when adding AD trusts support, we need to ensure we have valid kerberos >> >>>>>ticket of the user from 'admins' group or otherwise appropriate ACIs >> >>>>>will not be granted. >> >>>>> >> >>>>>This patch introduces a check for that. We already check if >> >>>>>ipa-adtrust-install is run by root so this complements existing checks. >> >>>>> >> >>>>>https://fedorahosted.org/freeipa/ticket/2815 >> >>>>After discussing on IRC with Simo and Rob, we came to conclusion that it >> >>>>is possible to switch to LDAPI and autobind feature of dirsrv for >> >>>>authentication and remove requirement for Directory Manager credentials >> >>>>altogether. >> >>>> >> >>>>Updated patch makes use of LDAPI + autobind under root privileges to map >> >>>>automatically to Directory Manager privileges in dirsrv. Additionally it >> >>>>ensures we have Kerberos credentials to fetch keytab with CIFS service >> >>>>key. >> >>>> >> >>>>Service._ldap_mod() is extended to switch to autobind when self.ldapi is >> >>>>set to True and we are running as root. >> >>>> >> >>>>For those interested in why ACIError is mapped to 'outdated Kerberos >> >>>>credentials' error message, this is because we'll get ACIError for 'ipa >> >>>>user-show ' command when authenticated by the Kerberos credentials >> >>>>for in a default ccache only when Kerberos credentials are >> >>>>stale -- >> >>>>either belong to a user that was removed or to a previous IPA install >> >>>>that was wiped before reinstalling. The latter is how I discovered >> >>>>this case. :) >> >>> >> >>>I think that this should raise an exception if one tries to use ldapi, >> >>>doesn't provide the DM password and is not root. Otherwise it won't >> >>>authenticate at all. >> >>This is not exactly true. >> >> >> >>$ id >> >>uid=757000001(abokovoy) gid=757000001(abokovoy) groups=757000001(abokovoy) >> >>context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 >> >> >> >>$ ldapsearch -vvv -H ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >> >> >/dev/null >> >>ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) >> >>SASL/EXTERNAL authentication started >> >>ldap_sasl_interactive_bind_s: Inappropriate authentication (48) >> >> additional info: SASL EXTERNAL bind requires an SSL connection >> >> >> >>$ ldapsearch -vvv -Y GSSAPI -H >> >>ldapi://%2fvar%2frun%2fslapd-IPA-LOCAL.socket '*' >/dev/null >> >>ldap_initialize( ldapi://%2Fvar%2Frun%2Fslapd-IPA-LOCAL.socket/??base ) >> >>SASL/GSSAPI authentication started >> >>SASL username: abokovoy at IPA.LOCAL >> >>SASL SSF: 56 >> >>SASL data security layer installed. >> >>filter: (objectclass=*) >> >>requesting: * >> >>So GSSAPI auth works with LDAPI access. I can simply enforce -Y GSSAPI >> >>when non-root and no dm_password regardless of self.ldapi, this would >> >>extend previously available logic to following: >> >> >> >>- ldapi: use -H ldapi://url instead of -h hostname >> >>- dm_password: add Directory Manager auth >> >>- root without dm_password: use autobind >> >>- non-root without dm_password: use GSSAPI >> >> >> >>>In reality, I think all this service code always runs as root, so it >> >>>may be a moot point, but this code is kinda convoluted. >> >>Yep. >> >> >> > >> >Sorry, I didn't mean this in general, but this code is going to >> >result in no auth being used which is going to fail. So rather than >> >failing with some no auth error we either enforce that Service is >> >executed as root or raise an exception in this auth code. In other >> >words, if we know something isn't going to work before we try it, we >> >should fail with a more friendly error. >> What the code did before in such case (no DM, non-root, non-ldapi) is to >> use -Y GSSAPI. So this forces to try GSSAPI auth already. My latest >> patch (-2) expands this also to (no DM, non-root, ldapi) case. >> >> However, one case is not covered yet: no DM, root, non-ldapi. We either >> can try -Y GSSAPI here as well or raise exception. I'd prefer the >> former. > >When in doubt always try GSSAPI please. This gets us back to my patch (-1) because - auth_parms = ["-Y", "GSSAPI"] + auth_params = [] + if not (os.getegid() == 0 and self.ldapi): + auth_parms = ["-Y", "GSSAPI"] covers both (no DM, root, non-ldapi) and (no DM, non-root, non-ldapi) -- / Alexander Bokovoy From pvoborni at redhat.com Wed Jul 18 13:54:30 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 18 Jul 2012 15:54:30 +0200 Subject: [Freeipa-devel] [PATCH] 172 Bigger textarea for permission type=subtree In-Reply-To: <50057D60.5010301@redhat.com> References: <500400D0.4090604@redhat.com> <50049F7D.4030209@redhat.com> <500575E8.7020609@redhat.com> <50057D60.5010301@redhat.com> Message-ID: <5006C016.2010004@redhat.com> On 07/17/2012 04:57 PM, Endi Sukma Dewata wrote: > On 7/17/2012 9:25 AM, Petr Vobornik wrote: >>> Possible improvement, instead of using a fixed column size the text area >>> also could be made to occupy 100% of available width. Ideally it should >>> have the same width as the text field or drop down list in this dialog. >> >> Updated patch attached. >> >> I added two styles: >> >> .textarea-widget textarea { >> width: 250px; >> } >> >> .facet-content .textarea-widget textarea { >> width: 400px; >> } >> >> First one makes textarea width the same as text_widget - good for >> dialogs. >> >> Second one makes textareas wider in facets. IMO it is better - more >> space for the text. Also previous implementation was about 330px in FF >> so it isn't big difference. 400px is about the same width as ssh-key >> widget and in also leaves space for possible action panel. > > Looks much better. ACK. > Pushed to master. -- Petr Vobornik From abokovoy at redhat.com Wed Jul 18 13:56:50 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 16:56:50 +0300 Subject: [Freeipa-devel] [PATCH] 0061 ValidationError takes 'error' named argument, not 'reason' In-Reply-To: <500588C3.6030004@redhat.com> References: <20120716131259.GA18370@redhat.com> <500588C3.6030004@redhat.com> Message-ID: <20120718135650.GR12345@redhat.com> On Tue, 17 Jul 2012, Rob Crittenden wrote: >Alexander Bokovoy wrote: >>Hi, >> >>https://fedorahosted.org/freeipa/ticket/2865 > >ACK Pushed to master -- / Alexander Bokovoy From abokovoy at redhat.com Wed Jul 18 13:57:04 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 16:57:04 +0300 Subject: [Freeipa-devel] [PATCH] 0062 support various forms of user account when establishing trusts In-Reply-To: <50058936.1080100@redhat.com> References: <20120716131410.GB18370@redhat.com> <50058936.1080100@redhat.com> Message-ID: <20120718135704.GS12345@redhat.com> On Tue, 17 Jul 2012, Rob Crittenden wrote: >Alexander Bokovoy wrote: >>Hi, >> >>Realm administrator account may be specified using different form: >>Administrator, DOM\Administrator, Administrator at DOMAIN >> >>This patch introduces handling of the second two forms: >>- In DOM\Administrator only user name is used, short domain name >> is then taken from a discovered record from the AD DC >>- In Administrator at DOMAIN first DOMAIN is verified to be the same >> as the domain we are establishing trust to, and then user name >> is taken, together with short domain name taken from a discovered >> record from the AD DC >> >>Note that we do not support using to-be-trusted domain's trusted >>domains' accounts to establish trust as there is basically zero chance >>to verify that things will work with them. In addition, in order to >>establish trust one needs to belong to Enterprise Admins group in AD or >>have specially delegated permissions. These permissions are unlikely >>delegated to the ones in already trusted domain. >> >>https://fedorahosted.org/freeipa/ticket/2864 >> > >ACK Pushed to master. -- / Alexander Bokovoy From abokovoy at redhat.com Wed Jul 18 13:58:07 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 18 Jul 2012 16:58:07 +0300 Subject: [Freeipa-devel] [PATCH] 0063 change sid_check_is_domain() to sid_check_is_our_sam() In-Reply-To: <1342617203.3219.73.camel@willson.li.ssimo.org> References: <20120718125809.GO12345@redhat.com> <1342617203.3219.73.camel@willson.li.ssimo.org> Message-ID: <20120718135807.GT12345@redhat.com> On Wed, 18 Jul 2012, Simo Sorce wrote: >On Wed, 2012-07-18 at 15:58 +0300, Alexander Bokovoy wrote: >> Hi, >> >> due to API change in Samba4 between beta3 and beta4, following small >> patch is needed for ipasam. >> >> I've also added forward declaration for the function. >> >> https://fedorahosted.org/freeipa/ticket/2929 > >Nack, >please add build requires in spec file to require beta4. > >Please consider it acked automatically as soon as you do that and then >please push asap. Pushed to master. Currently it only will build against ipa-devel repo as somehow Rawhide hasn't yet picked up http://kojipkgs.fedoraproject.org/packages/samba4/4.0.0/128.fc18.beta4/ -- / Alexander Bokovoy From rcritten at redhat.com Wed Jul 18 14:36:34 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Jul 2012 10:36:34 -0400 Subject: [Freeipa-devel] [PATCH] one-liner, don't hardcode serial_autoincrement Message-ID: <5006C9F2.5050800@redhat.com> The bind option serial_autoincrement was being hardcoded to True so passing in an install option to not enable it was not working. Pushed as a one-liner. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1036-autoincrement.patch Type: text/x-diff Size: 1009 bytes Desc: not available URL: From rcritten at redhat.com Wed Jul 18 18:21:56 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Jul 2012 14:21:56 -0400 Subject: [Freeipa-devel] [PATCHES] 495 Fix ipa-replica-manage issues In-Reply-To: <1342125009.2718.26.camel@willson.li.ssimo.org> References: <1342125009.2718.26.camel@willson.li.ssimo.org> Message-ID: <5006FEC4.6070700@redhat.com> Simo Sorce wrote: > These 2 patches fix issues found with ipa-replica-manage and > connect/disconnect commands. > > Fixes ticket #2925 > > Simo. ACK, pushed both to master. I slightly reformatted the commit messages. rob From rcritten at redhat.com Wed Jul 18 19:45:28 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 18 Jul 2012 15:45:28 -0400 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig In-Reply-To: <50069C29.1000304@redhat.com> References: <50069C29.1000304@redhat.com> Message-ID: <50071258.40307@redhat.com> Petr Viktorin wrote: > minimum_connections was sometimes not updated properly on install > because the script set psearch on but assumed it was still off. > Also, the number of connections was not set if the directive was missing. > > Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 Your changes look good but I found perhaps another bug. I think Petr Spacek may need to chime in on this part. If you start out with serial_autoincrement yes and psearch undefined and connections undefined the result is: connections 2 There is a comment in the code that connections needs to be 4 for autoincrement. I believe psearch needs to be enabled for autoincrement. I'm not sure how much sanity checking we want to do, mostly I'm curious if things will blow up with connections 2 and no psearch and autoincrement. rob From pspacek at redhat.com Thu Jul 19 09:55:32 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 19 Jul 2012 11:55:32 +0200 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig In-Reply-To: <50071258.40307@redhat.com> References: <50069C29.1000304@redhat.com> <50071258.40307@redhat.com> Message-ID: <5007D994.6010909@redhat.com> On 07/18/2012 09:45 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> minimum_connections was sometimes not updated properly on install >> because the script set psearch on but assumed it was still off. >> Also, the number of connections was not set if the directive was missing. >> >> Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 > > Your changes look good but I found perhaps another bug. I think Petr Spacek > may need to chime in on this part. > > If you start out with serial_autoincrement yes and psearch undefined and > connections undefined the result is: > > connections 2 > > There is a comment in the code that connections needs to be 4 for > autoincrement. I believe psearch needs to be enabled for autoincrement. I'm > not sure how much sanity checking we want to do, mostly I'm curious if things > will blow up with connections 2 and no psearch and autoincrement. In that case named refuses to start with an error message. I want to change configuration semantics to fail-with-error (rather than partial auto-correcting). Some configuration errors can't be corrected automatically and failure should wake-up admins and force them to correct configuration. Petr^2 Spacek > > rob From pviktori at redhat.com Thu Jul 19 09:59:27 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 19 Jul 2012 11:59:27 +0200 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig In-Reply-To: <50071258.40307@redhat.com> References: <50069C29.1000304@redhat.com> <50071258.40307@redhat.com> Message-ID: <5007DA7F.7080603@redhat.com> On 07/18/2012 09:45 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> minimum_connections was sometimes not updated properly on install >> because the script set psearch on but assumed it was still off. >> Also, the number of connections was not set if the directive was missing. >> >> Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 > > Your changes look good but I found perhaps another bug. I think Petr > Spacek may need to chime in on this part. > > If you start out with serial_autoincrement yes and psearch undefined and > connections undefined the result is: > > connections 2 > > There is a comment in the code that connections needs to be 4 for > autoincrement. I believe psearch needs to be enabled for autoincrement. > I'm not sure how much sanity checking we want to do, mostly I'm curious > if things will blow up with connections 2 and no psearch and autoincrement. > > rob Petr ?pa?ek tells me you'll get an error if serial_autoincrement is enabled without psearch. Do we want to fix things if users start out with broken configuration? -- Petr? From pviktori at redhat.com Thu Jul 19 10:53:59 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 19 Jul 2012 12:53:59 +0200 Subject: [Freeipa-devel] [PATCHES] 495 Fix ipa-replica-manage issues In-Reply-To: <5006FEC4.6070700@redhat.com> References: <1342125009.2718.26.camel@willson.li.ssimo.org> <5006FEC4.6070700@redhat.com> Message-ID: <5007E747.5070200@redhat.com> On 07/18/2012 08:21 PM, Rob Crittenden wrote: > Simo Sorce wrote: >> These 2 patches fix issues found with ipa-replica-manage and >> connect/disconnect commands. >> >> Fixes ticket #2925 >> >> Simo. > > ACK, pushed both to master. > > I slightly reformatted the commit messages. > > rob > This fixed https://fedorahosted.org/freeipa/ticket/2759, so I closed that ticket. -- Petr? From pviktori at redhat.com Thu Jul 19 11:17:25 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 19 Jul 2012 13:17:25 +0200 Subject: [Freeipa-devel] [PATCH] 1034 more robust cli sessions In-Reply-To: <5004556B.8030008@redhat.com> References: <50042A17.9050802@redhat.com> <5004556B.8030008@redhat.com> Message-ID: <5007ECC5.8000303@redhat.com> On 07/16/2012 07:54 PM, Rob Crittenden wrote: > Rob Crittenden wrote: >> Make command-line sessions a bit more robust. >> >> This patch does two things. Firstly, it wraps all keyring activity in a >> try/except so if a keyring operation fails it isn't fatal. The user just >> won't benefit from sessions. >> >> The second part adds per-principal sessions. The principal is included >> in the session key so we can pull the right one depending on the >> principal initiating the request. > > Left a debug statement in, this one should build and work. You've left a mention of the logging in the commit message. Otherwise ACK. I'm now able to use IPA even with a revoked key; kinit to a different user works correctly. > rob > > -- Petr? From atkac at redhat.com Thu Jul 19 11:43:48 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 19 Jul 2012 13:43:48 +0200 Subject: [Freeipa-devel] [PATCH 0032-0035] In-Reply-To: <50069EBA.9080400@redhat.com> References: <50069EBA.9080400@redhat.com> Message-ID: <20120719114347.GA27803@redhat.com> On Wed, Jul 18, 2012 at 01:32:10PM +0200, Petr Spacek wrote: > Hello, > > attached patch 0032 adds support for MODDN operation to persistent > search implementation. > Related ticket: https://fedorahosted.org/bind-dyndb-ldap/ticket/72 > > Patches 0033-0035 does minor cleanup in old persistent search code. Hello Peter, please check one comment below. You don't have to resend modified patchset, just go ahead and push it. Regards, Adam > From 79769c5ad71a10540cdd9b571eed9407e31da9e6 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 18 Jul 2012 13:01:28 +0200 > Subject: [PATCH] Add support for modify DN operation to persistent search. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 108 +++++++++++++++++++++++++++++++++++++++++++--------- > 1 files changed, 89 insertions(+), 19 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 8015db7018ddc9956471a99d6397ab95d83fbc3d..d49d72bc6ad23e8cec1c1f8a45135e79290b1422 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -239,6 +239,7 @@ struct ldap_psearchevent { > isc_mem_t *mctx; > char *dbname; > char *dn; > + char *prevdn; > int chgtype; > }; > > @@ -316,6 +317,9 @@ static isc_result_t ldap_pscontrol_create(isc_mem_t *mctx, LDAPControl **ctrlp); > static void ldap_pscontrol_destroy(isc_mem_t *mctx, LDAPControl **ctrlp); > > static isc_threadresult_t ldap_psearch_watcher(isc_threadarg_t arg); > +static void psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, > + LDAPControl **ctrls); > + > > /* Persistent updates watcher */ > static isc_threadresult_t > @@ -789,6 +793,7 @@ ldap_delete_zone2(ldap_instance_t *inst, dns_name_t *name, isc_boolean_t lock) > if (result == ISC_R_SUCCESS) > unlock = ISC_TRUE; > > + /* TODO: flush cache records belonging to deleted zone */ > CHECK(discard_from_cache(inst->cache, name)); > } > > @@ -2957,37 +2962,69 @@ update_action(isc_task_t *task, isc_event_t *event) > isc_result_t result ; > ldap_instance_t *inst = NULL; > ldap_connection_t *conn = NULL; > - ldap_qresult_t *ldap_qresult = NULL; > - ldap_entry_t *entry; > + ldap_qresult_t *ldap_qresult_zone = NULL; > + ldap_qresult_t *ldap_qresult_record = NULL; > + ldap_entry_t *entry_zone = NULL; > + ldap_entry_t *entry_record = NULL; > isc_boolean_t delete = ISC_TRUE; > isc_mem_t *mctx; > - char *attrs[] = { > + dns_name_t prevname; > + char *attrs_zone[] = { > "idnsName", "idnsUpdatePolicy", "idnsAllowQuery", > "idnsAllowTransfer", "idnsForwardPolicy", "idnsForwarders", NULL > }; > + char *attrs_record[] = { > + "objectClass", "dn", NULL > + }; > > UNUSED(task); > > mctx = pevent->mctx; > + dns_name_init(&prevname, NULL); > > result = manager_get_ldap_instance(pevent->dbname, &inst); > /* TODO: Can it happen? */ > if (result != ISC_R_SUCCESS) > goto cleanup; > > CHECK(ldap_pool_getconnection(inst->pool, &conn)); > - > - CHECK(ldap_query(inst, conn, &ldap_qresult, pevent->dn, > - LDAP_SCOPE_BASE, attrs, 0, > + CHECK(ldap_query(inst, conn, &ldap_qresult_zone, pevent->dn, > + LDAP_SCOPE_BASE, attrs_zone, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > - for (entry = HEAD(ldap_qresult->ldap_entries); > - entry != NULL; > - entry = NEXT(entry, link)) { > + for (entry_zone = HEAD(ldap_qresult_zone->ldap_entries); > + entry_zone != NULL; > + entry_zone = NEXT(entry_zone, link)) { > delete = ISC_FALSE; > result = ldap_parse_zoneentry(entry, inst); > if (result != ISC_R_SUCCESS) > goto cleanup; > + > + if (PSEARCH_MODDN(pevent->chgtype)) { > + if (dn_to_dnsname(inst->mctx, pevent->prevdn, &prevname, NULL) > + == ISC_R_SUCCESS) { > + CHECK(ldap_delete_zone(inst, pevent->prevdn, ISC_TRUE)); > + } else { > + log_debug(5, "update_action: old zone wasn't managed " > + "by plugin, dn '%s'", pevent->prevdn); > + } > + > + /* fill the cache with records from renamed zone */ > + CHECK(ldap_query(inst, conn, &ldap_qresult_record, pevent->dn, > + LDAP_SCOPE_ONELEVEL, attrs_record, 0, > + "(objectClass=idnsRecord)")); > + > + /* LDAP schema requires SOA record (at least) */ > + INSIST(HEAD(ldap_qresult_record->ldap_entries) != NULL); > + for (entry_record = HEAD(ldap_qresult_record->ldap_entries); > + entry_record != NULL; > + entry_record = NEXT(entry_record, link)) { > + > + psearch_update(inst, entry_record, NULL); > + } > + } > + > + INSIST(NEXT(entry_zone, link) == NULL); /* no multiple zones with same DN */ This INSIST seems like too strong for me. Imagine situation when administrator wrongly modifies LDAP database and creates duplicated zone. In this case we will crash. I would prefer to log error here, that multiple zones exist and only the first occurence is used. > } > > if (delete) > @@ -2999,9 +3036,14 @@ cleanup: > "Zones can be outdated, run `rndc reload`", > pevent->dn, isc_result_totext(result)); > > - ldap_query_free(ISC_FALSE, &ldap_qresult); > + ldap_query_free(ISC_FALSE, &ldap_qresult_zone); > + ldap_query_free(ISC_FALSE, &ldap_qresult_record); > ldap_pool_putconnection(inst->pool, &conn); > + if (dns_name_dynamic(&prevname)) > + dns_name_free(&prevname, inst->mctx); > isc_mem_free(mctx, pevent->dbname); > + if (pevent->prevdn != NULL) > + isc_mem_free(mctx, pevent->prevdn); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > isc_event_free(&event); > @@ -3090,8 +3132,12 @@ update_record(isc_task_t *task, isc_event_t *event) > /* Convert domain name from text to struct dns_name_t. */ > dns_name_t name; > dns_name_t origin; > + dns_name_t prevname; > + dns_name_t prevorigin; > dns_name_init(&name, NULL); > dns_name_init(&origin, NULL); > + dns_name_init(&prevname, NULL); > + dns_name_init(&prevorigin, NULL); > CHECK(dn_to_dnsname(mctx, pevent->dn, &name, &origin)); > > if (PSEARCH_DEL(pevent->chgtype)) { > @@ -3103,8 +3149,21 @@ update_record(isc_task_t *task, isc_event_t *event) > cache = ldap_instance_getcache(inst); > CHECK(discard_from_cache(cache, &name)); > > + if (PSEARCH_MODDN(pevent->chgtype)) { > + /* remove previous name only if it was inside DNS subtree */ > + if(dn_to_dnsname(mctx, pevent->prevdn, &prevname, &prevorigin) > + == ISC_R_SUCCESS) { > + log_debug(5, "psearch_update: removing name from cache, dn: '%s'", > + pevent->prevdn); > + CHECK(discard_from_cache(cache, &prevname)); > + } else { > + log_debug(5, "psearch_update: old name wasn't managed " > + "by plugin, dn '%s'", pevent->prevdn); > + } > + } > + > if (PSEARCH_ADD(pevent->chgtype) || PSEARCH_MOD(pevent->chgtype) || > - !PSEARCH_ANY(pevent->chgtype)) { > + PSEARCH_MODDN(pevent->chgtype) || !PSEARCH_ANY(pevent->chgtype)) { > /* > * Find new data in LDAP. !PSEARCH_ANY indicates unchanged entry > * found during initial lookup (i.e. database dump). > @@ -3141,9 +3200,15 @@ cleanup: > > if (dns_name_dynamic(&name)) > dns_name_free(&name, inst->mctx); > + if (dns_name_dynamic(&prevname)) > + dns_name_free(&prevname, inst->mctx); > if (dns_name_dynamic(&origin)) > dns_name_free(&origin, inst->mctx); > + if (dns_name_dynamic(&prevorigin)) > + dns_name_free(&prevorigin, inst->mctx); > isc_mem_free(mctx, pevent->dbname); > + if (pevent->prevdn != NULL) > + isc_mem_free(mctx, pevent->prevdn); > isc_mem_free(mctx, pevent->dn); > isc_mem_detach(&mctx); > isc_event_free(&event); > @@ -3214,8 +3279,9 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > isc_result_t result = ISC_R_SUCCESS; > ldap_psearchevent_t *pevent = NULL; > int chgtype = LDAP_ENTRYCHANGE_NONE; > - char *moddn = NULL; > char *dn = NULL; > + char *prevdn_ldap = NULL; > + char *prevdn = NULL; > char *dbname = NULL; > isc_mem_t *mctx = NULL; > isc_taskaction_t action = NULL; > @@ -3228,7 +3294,7 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > } > > if (ctrls != NULL) > - CHECK(ldap_parse_entrychangectrl(ctrls, &chgtype, &moddn)); > + CHECK(ldap_parse_entrychangectrl(ctrls, &chgtype, &prevdn_ldap)); > > isc_mem_attach(inst->mctx, &mctx); > > @@ -3243,11 +3309,12 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > goto cleanup; > } > > - /* TODO: Handle moddn case. */ > if (PSEARCH_MODDN(chgtype)) { > - log_error("psearch moddn change is not implemented"); > - result = ISC_R_FAILURE; > - goto cleanup; > + prevdn = isc_mem_strdup(mctx, prevdn_ldap); > + if (prevdn == NULL) { > + result = ISC_R_NOMEMORY; > + goto cleanup; > + } > } > > /* > @@ -3281,19 +3348,22 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > pevent->mctx = mctx; > pevent->dbname = dbname; > pevent->dn = dn; > + pevent->prevdn = prevdn; > pevent->chgtype = chgtype; > isc_task_send(inst->task, (isc_event_t **)&pevent); > > cleanup: > if (result != ISC_R_SUCCESS) { > if (dbname != NULL) > isc_mem_free(mctx, dbname); > if (dn != NULL) > isc_mem_free(mctx, dn); > + if (prevdn != NULL) > + isc_mem_free(mctx, prevdn); > if (mctx != NULL) > isc_mem_detach(&mctx); > - if (moddn != NULL) > - ldap_memfree(moddn); > + if (prevdn_ldap != NULL) > + ldap_memfree(prevdn); > > log_error("psearch_update failed for %s zone. " > "Zone can be outdated, run `rndc reload`", > -- > 1.7.7.6 > > From 40c54af3b1b6872a6eeac28624524adb09c4c9b7 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 18 Jul 2012 13:04:10 +0200 > Subject: [PATCH] Rename persistent search update_action() to update_zone(). > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index d49d72bc6ad23e8cec1c1f8a45135e79290b1422..59429e21db7982edf795e0f8e12e22b9f5fa1b02 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -2956,7 +2956,7 @@ cleanup: > * operation but zones don't change often. > */ > static void > -update_action(isc_task_t *task, isc_event_t *event) > +update_zone(isc_task_t *task, isc_event_t *event) > { > ldap_psearchevent_t *pevent = (ldap_psearchevent_t *)event; > isc_result_t result ; > @@ -3327,7 +3327,7 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > if ((class & LDAP_ENTRYCLASS_CONFIG) != 0) > action = update_config; > else if ((class & LDAP_ENTRYCLASS_ZONE) != 0) > - action = update_action; > + action = update_zone; > else if ((class & LDAP_ENTRYCLASS_RR) != 0) > action = update_record; > else { > -- > 1.7.7.6 > > From b4bf985854391a68ad1c8394294edf64c80613d4 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 18 Jul 2012 13:05:59 +0200 > Subject: [PATCH] Minor code cleanup in persistent search error handling. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 10 ++-------- > 1 files changed, 2 insertions(+), 8 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 59429e21db7982edf795e0f8e12e22b9f5fa1b02..3a8ced88758d79719966ce80f97116d6cb2b84e2 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -2982,23 +2982,17 @@ update_zone(isc_task_t *task, isc_event_t *event) > mctx = pevent->mctx; > dns_name_init(&prevname, NULL); > > - result = manager_get_ldap_instance(pevent->dbname, &inst); > - /* TODO: Can it happen? */ > - if (result != ISC_R_SUCCESS) > - goto cleanup; > - > + CHECK(manager_get_ldap_instance(pevent->dbname, &inst)); > CHECK(ldap_pool_getconnection(inst->pool, &conn)); > CHECK(ldap_query(inst, conn, &ldap_qresult_zone, pevent->dn, > LDAP_SCOPE_BASE, attrs_zone, 0, > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > > for (entry_zone = HEAD(ldap_qresult_zone->ldap_entries); > entry_zone != NULL; > entry_zone = NEXT(entry_zone, link)) { > delete = ISC_FALSE; > - result = ldap_parse_zoneentry(entry, inst); > - if (result != ISC_R_SUCCESS) > - goto cleanup; > + CHECK(ldap_parse_zoneentry(entry_zone, inst)); > > if (PSEARCH_MODDN(pevent->chgtype)) { > if (dn_to_dnsname(inst->mctx, pevent->prevdn, &prevname, NULL) > -- > 1.7.7.6 > > From b538e9ca9784cc23105c8f90b6a12104407b27b9 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 18 Jul 2012 13:27:16 +0200 > Subject: [PATCH] Minor persistent search logging cleanup. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 21 +++++++++++---------- > 1 files changed, 11 insertions(+), 10 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 3a8ced88758d79719966ce80f97116d6cb2b84e2..c3499b8878af7ad8a4fc1f71406026b4d3d40aaa 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -3134,8 +3134,8 @@ update_record(isc_task_t *task, isc_event_t *event) > dns_name_init(&prevorigin, NULL); > CHECK(dn_to_dnsname(mctx, pevent->dn, &name, &origin)); > > - if (PSEARCH_DEL(pevent->chgtype)) { > - log_debug(5, "psearch_update: Removing item from cache (%s)", > + if (PSEARCH_DEL(pevent->chgtype) || PSEARCH_MODDN(pevent->chgtype)) { > + log_debug(5, "psearch_update: removing name from cache, dn: '%s'", > pevent->dn); > } > > @@ -3164,7 +3164,7 @@ update_record(isc_task_t *task, isc_event_t *event) > * > * @todo Change this to convert ldap_entry_t to ldapdb_rdatalist_t. > */ > - log_debug(5, "psearch_update: Updating item in cache (%s)", > + log_debug(5, "psearch_update: updating name in cache, dn: '%s'", > pevent->dn); > CHECK(ldapdb_rdatalist_get(mctx, inst, &name, &origin, &rdatalist)); > > @@ -3177,18 +3177,13 @@ update_record(isc_task_t *task, isc_event_t *event) > ldapdb_rdatalist_destroy(mctx, &rdatalist); > } > > - log_debug(20,"psearch change type: none%d, add%d, del%d, mod%d, moddn%d", > - !PSEARCH_ANY(pevent->chgtype), PSEARCH_ADD(pevent->chgtype), > - PSEARCH_DEL(pevent->chgtype), PSEARCH_MOD(pevent->chgtype), > - PSEARCH_MODDN(pevent->chgtype)); > - > /* Do not bump serial during initial database dump. */ > if (inst->serial_autoincrement && PSEARCH_ANY(pevent->chgtype)) { > CHECK(soa_serial_increment(mctx, inst, &origin)); > } > cleanup: > if (result != ISC_R_SUCCESS) > - log_error("update_record (psearch) failed for %s. " > + log_error("update_record (psearch) failed, dn '%s'. " > "Records can be outdated, run `rndc reload`", > pevent->dn); > > @@ -3282,14 +3277,20 @@ psearch_update(ldap_instance_t *inst, ldap_entry_t *entry, LDAPControl **ctrls) > > class = ldap_entry_getclass(entry); > if (class == LDAP_ENTRYCLASS_NONE) { > - log_error("psearch_update: ignoring unknown entry [dn %s]", > + log_error("psearch_update: ignoring entry with unknown class, dn '%s'", > entry->dn); > return; /* ignore it, it's OK */ > } > > if (ctrls != NULL) > CHECK(ldap_parse_entrychangectrl(ctrls, &chgtype, &prevdn_ldap)); > > + > + log_debug(20,"psearch change type: none%d, add%d, del%d, mod%d, moddn%d", > + !PSEARCH_ANY(chgtype), PSEARCH_ADD(chgtype), > + PSEARCH_DEL(chgtype), PSEARCH_MOD(chgtype), > + PSEARCH_MODDN(chgtype)); > + > isc_mem_attach(inst->mctx, &mctx); > > dn = isc_mem_strdup(mctx, entry->dn); > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From atkac at redhat.com Thu Jul 19 11:45:00 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 19 Jul 2012 13:45:00 +0200 Subject: [Freeipa-devel] [PATCH 0036] Raise connection count automatically if serial_autoincrement is enabled In-Reply-To: <5006A20A.4010005@redhat.com> References: <5006A20A.4010005@redhat.com> Message-ID: <20120719114500.GB27803@redhat.com> On Wed, Jul 18, 2012 at 01:46:18PM +0200, Petr Spacek wrote: > Hello, > > this patch reflects new demand from serial_autoincrement feature. > > Generally, change in configuration file should by IPA > install/upgrade scripts. This patch prevents deadlock in situation > where scripts failed in their job (as you can see right now). > > Will be obsoleted by https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . Ack. > From b41d06248a199e618fd963f32cf16d2c8384276f Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 18 Jul 2012 13:39:12 +0200 > Subject: [PATCH] Raise connection count automatically if serial_autoincrement > is enabled. > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 6 ++++++ > 1 files changed, 6 insertions(+), 0 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 8015db7018ddc9956471a99d6397ab95d83fbc3d..a51a9fe384b7133b06b50a8ff498755d37dafffe 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -480,6 +480,12 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > result = ISC_R_FAILURE; > goto cleanup; > } > + if (ldap_inst->serial_autoincrement == ISC_TRUE > + && ldap_inst->connections < 4) { > + log_error("serial_autoincrement needs at least 4 connections, " > + "increasing limit"); > + ldap_inst->connections = 4; > + } > > CHECK(new_ldap_cache(mctx, argv, &ldap_inst->cache, ldap_inst->psearch)); > CHECK(ldap_pool_create(mctx, ldap_inst->connections, &ldap_inst->pool)); > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From pviktori at redhat.com Thu Jul 19 11:45:53 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 19 Jul 2012 13:45:53 +0200 Subject: [Freeipa-devel] [PATCH] 1035 case sensitivity when calculating indirect members In-Reply-To: <5005B450.8000207@redhat.com> References: <5005B450.8000207@redhat.com> Message-ID: <5007F371.7000705@redhat.com> On 07/17/2012 08:52 PM, Rob Crittenden wrote: > When determining whether a member is direct or indirect we were not > doing a case-insensitive comparison which led to marking a member as > both direct and indirect (in a test case no less). > > This patch fixes the comparison and the test. > > rob > When comparing DNs, you should use the DN class, not string manipulations: DN(x) instead of x.lower(). How urgent is this? John's DN patch solves this in a much more thorough way, maybe it'd be better to just wait for that. -- Petr? From atkac at redhat.com Thu Jul 19 11:46:07 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 19 Jul 2012 13:46:07 +0200 Subject: [Freeipa-devel] [PATCH 0037] Add missing return value check to new_ldap_instance() In-Reply-To: <5006AD88.5030307@redhat.com> References: <5006AD88.5030307@redhat.com> Message-ID: <20120719114606.GC27803@redhat.com> On Wed, Jul 18, 2012 at 02:35:20PM +0200, Petr Spacek wrote: > Hello, > > this patch adds missing return value check to new_ldap_instance(). > > https://fedorahosted.org/bind-dyndb-ldap/ticket/85 > > Bug was reported by Coverity. Ack. > From 85574b9ffe4757b93b6eb9b99ceb1172a5c37002 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 18 Jul 2012 14:32:48 +0200 > Subject: [PATCH] Add missing return value check to new_ldap_instance(). > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index a51a9fe384b7133b06b50a8ff498755d37dafffe..f21c449bbfe7e0cb7718ae5479989db16c103958 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -454,7 +454,8 @@ new_ldap_instance(isc_mem_t *mctx, const char *db_name, > result = ISC_R_FAILURE; > goto cleanup; > } else { > - str_sprintf(ldap_inst->krb5_principal, "DNS/%s", hostname); > + CHECK(str_sprintf(ldap_inst->krb5_principal, > + "DNS/%s", hostname)); > log_debug(2, "SASL mech GSSAPI defined but krb5_principal" > "and sasl_user are empty, using default %s", > str_buf(ldap_inst->krb5_principal)); > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Thu Jul 19 11:59:01 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 19 Jul 2012 13:59:01 +0200 Subject: [Freeipa-devel] [PATCH 0032-0035] Add support for MODDN operation to persistent search implementation In-Reply-To: <20120719114347.GA27803@redhat.com> References: <50069EBA.9080400@redhat.com> <20120719114347.GA27803@redhat.com> Message-ID: <5007F685.20104@redhat.com> Hello, I have to explain my motivation behind INSIST a bit. Please see comments below. On 07/19/2012 01:43 PM, Adam Tkac wrote: >> On Wed, Jul 18, 2012 at 01:32:10PM +0200, Petr Spacek wrote: >> + CHECK(ldap_query(inst, conn, &ldap_qresult_zone, pevent->dn, >> >+ LDAP_SCOPE_BASE, attrs_zone, 0, >> > "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); This LDAP query has search scope LDAP_SCOPE_BASE. If I understood LDAP correctly, it can return 0 or 1 result, but no more. (Two objects can't have exactly same DN.) If specified LDAP query returned (or is believed to returned) more than single result, then something went terribly wrong (memory corruption/incorrect pointer arithmetic?). Theoretically this situation should never happen and I can remove this check completely, if you want. >> > >> >- for (entry = HEAD(ldap_qresult->ldap_entries); >> >- entry != NULL; >> >- entry = NEXT(entry, link)) { >> >+ for (entry_zone = HEAD(ldap_qresult_zone->ldap_entries); >> >+ entry_zone != NULL; >> >+ entry_zone = NEXT(entry_zone, link)) { >> > delete = ISC_FALSE; >> > result = ldap_parse_zoneentry(entry, inst); >> > if (result != ISC_R_SUCCESS) >> > goto cleanup; >> >+ >> >+ if (PSEARCH_MODDN(pevent->chgtype)) { >> >+ if (dn_to_dnsname(inst->mctx, pevent->prevdn, &prevname, NULL) >> >+ == ISC_R_SUCCESS) { >> >+ CHECK(ldap_delete_zone(inst, pevent->prevdn, ISC_TRUE)); >> >+ } else { >> >+ log_debug(5, "update_action: old zone wasn't managed " >> >+ "by plugin, dn '%s'", pevent->prevdn); >> >+ } >> >+ >> >+ /* fill the cache with records from renamed zone */ >> >+ CHECK(ldap_query(inst, conn, &ldap_qresult_record, pevent->dn, >> >+ LDAP_SCOPE_ONELEVEL, attrs_record, 0, >> >+ "(objectClass=idnsRecord)")); >> >+ >> >+ /* LDAP schema requires SOA record (at least) */ >> >+ INSIST(HEAD(ldap_qresult_record->ldap_entries) != NULL); >> >+ for (entry_record = HEAD(ldap_qresult_record->ldap_entries); >> >+ entry_record != NULL; >> >+ entry_record = NEXT(entry_record, link)) { >> >+ >> >+ psearch_update(inst, entry_record, NULL); >> >+ } >> >+ } >> >+ >> >+ INSIST(NEXT(entry_zone, link) == NULL); /* no multiple zones with same DN */ > This INSIST seems like too strong for me. Imagine situation when administrator > wrongly modifies LDAP database and creates duplicated zone. In this case we will AFAIK it is not possible, because idnsName attribute is part of DN and duplicate DN is not allowed. > crash. I would prefer to log error here, that multiple zones exist and only the > first occurence is used. > >> > } Petr^2 Spacek From atkac at redhat.com Thu Jul 19 12:09:54 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 19 Jul 2012 14:09:54 +0200 Subject: [Freeipa-devel] [PATCH 0032-0035] Add support for MODDN operation to persistent search implementation In-Reply-To: <5007F685.20104@redhat.com> References: <50069EBA.9080400@redhat.com> <20120719114347.GA27803@redhat.com> <5007F685.20104@redhat.com> Message-ID: <20120719120954.GA28826@redhat.com> On Thu, Jul 19, 2012 at 01:59:01PM +0200, Petr Spacek wrote: > Hello, > > I have to explain my motivation behind INSIST a bit. Please see comments below. > > On 07/19/2012 01:43 PM, Adam Tkac wrote: > >>On Wed, Jul 18, 2012 at 01:32:10PM +0200, Petr Spacek wrote: > >>+ CHECK(ldap_query(inst, conn, &ldap_qresult_zone, pevent->dn, > >>>+ LDAP_SCOPE_BASE, attrs_zone, 0, > >>> "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); > This LDAP query has search scope LDAP_SCOPE_BASE. If I understood > LDAP correctly, it can return 0 or 1 result, but no more. (Two > objects can't have exactly same DN.) > > If specified LDAP query returned (or is believed to returned) more > than single result, then something went terribly wrong (memory > corruption/incorrect pointer arithmetic?). > > Theoretically this situation should never happen and I can remove > this check completely, if you want. > > >>> > >>>- for (entry = HEAD(ldap_qresult->ldap_entries); > >>>- entry != NULL; > >>>- entry = NEXT(entry, link)) { > >>>+ for (entry_zone = HEAD(ldap_qresult_zone->ldap_entries); > >>>+ entry_zone != NULL; > >>>+ entry_zone = NEXT(entry_zone, link)) { > >>> delete = ISC_FALSE; > >>> result = ldap_parse_zoneentry(entry, inst); > >>> if (result != ISC_R_SUCCESS) > >>> goto cleanup; > >>>+ > >>>+ if (PSEARCH_MODDN(pevent->chgtype)) { > >>>+ if (dn_to_dnsname(inst->mctx, pevent->prevdn, &prevname, NULL) > >>>+ == ISC_R_SUCCESS) { > >>>+ CHECK(ldap_delete_zone(inst, pevent->prevdn, ISC_TRUE)); > >>>+ } else { > >>>+ log_debug(5, "update_action: old zone wasn't managed " > >>>+ "by plugin, dn '%s'", pevent->prevdn); > >>>+ } > >>>+ > >>>+ /* fill the cache with records from renamed zone */ > >>>+ CHECK(ldap_query(inst, conn, &ldap_qresult_record, pevent->dn, > >>>+ LDAP_SCOPE_ONELEVEL, attrs_record, 0, > >>>+ "(objectClass=idnsRecord)")); > >>>+ > >>>+ /* LDAP schema requires SOA record (at least) */ > >>>+ INSIST(HEAD(ldap_qresult_record->ldap_entries) != NULL); > >>>+ for (entry_record = HEAD(ldap_qresult_record->ldap_entries); > >>>+ entry_record != NULL; > >>>+ entry_record = NEXT(entry_record, link)) { > >>>+ > >>>+ psearch_update(inst, entry_record, NULL); > >>>+ } > >>>+ } > >>>+ > >>>+ INSIST(NEXT(entry_zone, link) == NULL); /* no multiple zones with same DN */ > >This INSIST seems like too strong for me. Imagine situation when administrator > >wrongly modifies LDAP database and creates duplicated zone. In this case we will > AFAIK it is not possible, because idnsName attribute is part of DN > and duplicate DN is not allowed. Ok, this makes sence. So push the patchset as is, thank you. Regards, Adam > >crash. I would prefer to log error here, that multiple zones exist and only the > >first occurence is used. > > > >>> } > > Petr^2 Spacek > -- Adam Tkac, Red Hat, Inc. From william at firstyear.id.au Thu Jul 19 12:50:06 2012 From: william at firstyear.id.au (William Brown) Date: Thu, 19 Jul 2012 22:20:06 +0930 Subject: [Freeipa-devel] [DHCP] tree layout options Message-ID: <5008027E.4070401@firstyear.id.au> Find attached two different ldifs showing how the tree for DHCP services could be layed out. I personally prefer 2 due to the way that sharedNetwork segments can be named uniquely in a location without clashing with another location. The way that ISC-DHCP generates the config is through essentially a depth-first subtree search of the objects below the dhcpService object (In this case, cn=pultney). Due to this, I think the best way to split ipv4 and ipv6 due to the conflicting DHCP options, would be to make cn=locations,cn=v4,cn=isc,cn=dhcp and cn=locations,cn=v6,isc,cn=dhcp OR cn=locations4,cn=isc,cn=dhcp and cn=locations6,cn=isc,cn=dhcp Additionally, the option1 config does not at this time work with the ISC-DHCP server. It seems there is a bug in that it can parse the dhcpSharedNetworkDN attributes, and push them to a stack to follow them, but never parses the contents of them. Option 2 works, and generates a configuration for the networks and subnets correctly, but does not add any dhcpHost objects not the dhcpFailOverPeer information. I am investigating both. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- version: 1 dn: cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: dhcp dn: cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: isc dn: cn=servers,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: servers dn: cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: networks dn: cn=freeipa.dev.firstyear.id.au,cn=servers,cn=isc,cn=dhcp,dc=dev,dc=first year,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: freeipa.dev.firstyear.id.au dhcpservicedn: cn=pultney,cn=configs,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=i d,dc=au dn: cn=pultney,cn=configs,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpService objectclass: top objectclass: dhcpOptions cn: pultney dhcpfailoverpeerdn: cn=pultney,cn=failover,cn=isc,cn=dhcp,dc=dev,dc=firstyea r,dc=id,dc=au dhcphostdn: cn=hosts,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au dhcpoption: domain-name-servers 10.0.0.1 dhcpprimarydn: cn=freeipa.dev.firstyear.id.au,cn=servers,cn=isc,cn=dhcp,dc=d ev,dc=firstyear,dc=id,dc=au dhcpsharednetworkdn: cn=vlan10,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyea r,dc=id,dc=au dhcpsharednetworkdn: cn=vlan11,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyea r,dc=id,dc=au dhcpsharednetworkdn: cn=vlan30,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyea r,dc=id,dc=au dhcpstatements: ddns-update-style none dhcpstatements: max-lease-time 192000 dn: cn=vlan10,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan10 dn: cn=vlan11,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan11 dn: cn=10.0.10.0,cn=vlan10,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc =id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.10.0 dhcpnetmask: 24 dhcpoption: routers 10.0.10.1 dhcprange: 10.0.10.10 10.0.10.15 dn: cn=10.0.11.0,cn=vlan11,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc =id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.11.0 dhcpnetmask: 24 dhcpoption: routers 10.0.11.1 dhcprange: 10.0.11.10 10.0.11.15 dn: cn=caroline.dev.firstyear.id.au,cn=hosts,cn=isc,cn=dhcp,dc=dev,dc=firsty ear,dc=id,dc=au objectclass: dhcpHost objectclass: top cn: caroline.dev.firstyear.id.au dhcphwaddress: ethernet 00:23:54:12:12:98 dhcpstatements: fixed-address 10.0.10.100 dn: cn=vlan30,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan30 dn: cn=10.0.30.0,cn=vlan30,cn=networks,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc =id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.30.0 dhcpnetmask: 24 dhcpoption: routers 10.0.30.1 dhcprange: 10.0.30.10 10.0.30.15 dn: cn=configs,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: configs dn: cn=hosts,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: hosts dn: cn=dhcpserver.dev.firstyear.id.au,cn=servers,cn=isc,cn=dhcp,dc=dev,dc=fi rstyear,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: dhcpserver.dev.firstyear.id.au dhcpservicedn: cn=pultney,cn=configs,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=i d,dc=au dn: cn=failover,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: failover dn: cn=pultney,cn=failover,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpFailOverPeer objectclass: top cn: pultney dhcpfailoverprimaryport: 2000 dhcpfailoverprimaryserver: freeipa.dev.firstyear.id.au dhcpfailoversecondaryport: 2000 dhcpfailoversecondaryserver: dhcpserver.dev.firstyear.id.au -------------- next part -------------- version: 1 dn: cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: dhcp dn: cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: isc dn: cn=servers,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: servers dn: cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,d c=id,dc=au objectclass: nsContainer objectclass: top cn: networks dn: cn=freeipa.dev.firstyear.id.au,cn=servers,cn=isc,cn=dhcp,dc=dev,dc=first year,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: freeipa.dev.firstyear.id.au dhcpservicedn: cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc =id,dc=au dn: cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpService objectclass: top objectclass: dhcpOptions cn: pultney dhcpoption: domain-name-servers 10.0.0.1 dhcpprimarydn: cn=freeipa.dev.firstyear.id.au,cn=servers,cn=isc,cn=dhcp,dc=d ev,dc=firstyear,dc=id,dc=au dhcpstatements: ddns-update-style none dhcpstatements: max-lease-time 192000 dn: cn=vlan10,cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=f irstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan10 dn: cn=vlan11,cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=f irstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan11 dn: cn=10.0.10.0,cn=vlan10,cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhc p,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.10.0 dhcpnetmask: 24 dhcpoption: routers 10.0.10.1 dhcprange: 10.0.10.10 10.0.10.15 dn: cn=10.0.11.0,cn=vlan11,cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhc p,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.11.0 dhcpnetmask: 24 dhcpoption: routers 10.0.11.1 dhcprange: 10.0.11.10 10.0.11.15 dn: cn=caroline.dev.firstyear.id.au,cn=hosts,cn=pultney,cn=locations,cn=isc, cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpHost objectclass: top cn: caroline.dev.firstyear.id.au dhcphwaddress: ethernet 00:23:54:12:12:98 dhcpstatements: fixed-address 10.0.10.100 dn: cn=vlan30,cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=f irstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan30 dn: cn=10.0.30.0,cn=vlan30,cn=networks,cn=pultney,cn=locations,cn=isc,cn=dhc p,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.30.0 dhcpnetmask: 24 dhcpoption: routers 10.0.30.1 dhcprange: 10.0.30.10 10.0.30.15 dn: cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: locations dn: cn=hosts,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=i d,dc=au objectclass: nsContainer objectclass: top cn: hosts dn: cn=dhcpserver.dev.firstyear.id.au,cn=servers,cn=isc,cn=dhcp,dc=dev,dc=fi rstyear,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: dhcpserver.dev.firstyear.id.au dhcpservicedn: cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc =id,dc=au dn: cn=failover,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,d c=id,dc=au objectclass: dhcpFailOverPeer objectclass: top cn: failover dhcpfailoverprimaryport: 2000 dhcpfailoverprimaryserver: freeipa.dev.firstyear.id.au dhcpfailoversecondaryport: 2000 dhcpfailoversecondaryserver: dhcpserver.dev.firstyear.id.au -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From rcritten at redhat.com Thu Jul 19 13:04:09 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Jul 2012 09:04:09 -0400 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig In-Reply-To: <5007DA7F.7080603@redhat.com> References: <50069C29.1000304@redhat.com> <50071258.40307@redhat.com> <5007DA7F.7080603@redhat.com> Message-ID: <500805C9.7030702@redhat.com> Petr Viktorin wrote: > On 07/18/2012 09:45 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> minimum_connections was sometimes not updated properly on install >>> because the script set psearch on but assumed it was still off. >>> Also, the number of connections was not set if the directive was >>> missing. >>> >>> Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 >> >> Your changes look good but I found perhaps another bug. I think Petr >> Spacek may need to chime in on this part. >> >> If you start out with serial_autoincrement yes and psearch undefined and >> connections undefined the result is: >> >> connections 2 >> >> There is a comment in the code that connections needs to be 4 for >> autoincrement. I believe psearch needs to be enabled for autoincrement. >> I'm not sure how much sanity checking we want to do, mostly I'm curious >> if things will blow up with connections 2 and no psearch and >> autoincrement. >> >> rob > > Petr ?pa?ek tells me you'll get an error if serial_autoincrement is > enabled without psearch. > Do we want to fix things if users start out with broken configuration? > I think that if we can detect that a configuration is broken we should complain loudly, if not fix it (and still complain). rob From rcritten at redhat.com Thu Jul 19 13:07:02 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Jul 2012 09:07:02 -0400 Subject: [Freeipa-devel] [PATCH] 1035 case sensitivity when calculating indirect members In-Reply-To: <5007F371.7000705@redhat.com> References: <5005B450.8000207@redhat.com> <5007F371.7000705@redhat.com> Message-ID: <50080676.3070605@redhat.com> Petr Viktorin wrote: > On 07/17/2012 08:52 PM, Rob Crittenden wrote: >> When determining whether a member is direct or indirect we were not >> doing a case-insensitive comparison which led to marking a member as >> both direct and indirect (in a test case no less). >> >> This patch fixes the comparison and the test. >> >> rob >> > > When comparing DNs, you should use the DN class, not string > manipulations: DN(x) instead of x.lower(). > > How urgent is this? John's DN patch solves this in a much more thorough > way, maybe it'd be better to just wait for that. > The problem is we're currently reporting incorrect membership. I figured this would be a short-term fix unless you think the DN patch commit is imminent. rob From pvoborni at redhat.com Thu Jul 19 13:07:41 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Thu, 19 Jul 2012 15:07:41 +0200 Subject: [Freeipa-devel] [PATCH] 173 IDs and names for dialogs Message-ID: <5008069D.7070202@redhat.com> It's hard to detect if or which type of dialog is displayed because not all dialogs have IDs. On dialog open, it's id or name (if id is not set) is used for containing element id. Many of dialog types were missing id or name so name was added to each dialog type. In HTML, element's id should be unique. Our framework allows opening two dialogs with the same id. It may lead to state where getElementById method may have unpredictable behavior. Therefore attribute 'data-name' with dialog's name was added to dialog's containing element. Automation framework can search more reliable by using this attribute instead of id. https://fedorahosted.org/freeipa/ticket/2853 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0173-IDs-and-names-for-dialogs.patch Type: text/x-patch Size: 7489 bytes Desc: not available URL: From william at firstyear.id.au Thu Jul 19 13:14:11 2012 From: william at firstyear.id.au (William Brown) Date: Thu, 19 Jul 2012 22:44:11 +0930 Subject: [Freeipa-devel] [DHCP] tree layout options In-Reply-To: <5008027E.4070401@firstyear.id.au> References: <5008027E.4070401@firstyear.id.au> Message-ID: <50080823.5040602@firstyear.id.au> > does not add any dhcpHost objects not the dhcpFailOverPeer information. I have found why this is. I was setting ldap-method to dynamic, meaning that the contents of this object were only read at lease request time. setting this to static has allowed these objects to be read at dhcpd initilization time. It would also seem that in ldap_generate_config_string in ldap.c, the dhcpFailOverPeer object class is not "recognized" as an object that should be inspected. I'll add this later. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From rcritten at redhat.com Thu Jul 19 13:21:59 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Jul 2012 09:21:59 -0400 Subject: [Freeipa-devel] [PATCH] 1034 more robust cli sessions In-Reply-To: <5007ECC5.8000303@redhat.com> References: <50042A17.9050802@redhat.com> <5004556B.8030008@redhat.com> <5007ECC5.8000303@redhat.com> Message-ID: <500809F7.5030005@redhat.com> Petr Viktorin wrote: > On 07/16/2012 07:54 PM, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> Make command-line sessions a bit more robust. >>> >>> This patch does two things. Firstly, it wraps all keyring activity in a >>> try/except so if a keyring operation fails it isn't fatal. The user just >>> won't benefit from sessions. >>> >>> The second part adds per-principal sessions. The principal is included >>> in the session key so we can pull the right one depending on the >>> principal initiating the request. >> >> Left a debug statement in, this one should build and work. > > You've left a mention of the logging in the commit message. > > Otherwise ACK. I'm now able to use IPA even with a revoked key; kinit to > a different user works correctly. comment removed, pushed to master rob From simo at redhat.com Thu Jul 19 13:29:04 2012 From: simo at redhat.com (Simo Sorce) Date: Thu, 19 Jul 2012 09:29:04 -0400 Subject: [Freeipa-devel] [DHCP] tree layout options In-Reply-To: <50080823.5040602@firstyear.id.au> References: <5008027E.4070401@firstyear.id.au> <50080823.5040602@firstyear.id.au> Message-ID: <1342704544.3219.217.camel@willson.li.ssimo.org> On Thu, 2012-07-19 at 22:44 +0930, William Brown wrote: > > does not add any dhcpHost objects not the dhcpFailOverPeer information. > > I have found why this is. I was setting ldap-method to dynamic, meaning > that the contents of this object were only read at lease request time. > setting this to static has allowed these objects to be read at dhcpd > initilization time. > > It would also seem that in ldap_generate_config_string in ldap.c, the > dhcpFailOverPeer object class is not "recognized" as an object that > should be inspected. I'll add this later. Hi William, thanks for the ldifs, I will take some time this week to ponder them. I do not like having to separate ipv4 and ipv6 in principle, as I suspect in most cases admin would have to go an create the same locations under both, however I guess we can handle that in the UI relatively easily and always create both. I will try to consider the other factors as soon as I have some time to get my head around the other details. Simo. -- Simo Sorce * Red Hat, Inc * New York From pviktori at redhat.com Thu Jul 19 13:35:33 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 19 Jul 2012 15:35:33 +0200 Subject: [Freeipa-devel] [PATCH] 1035 case sensitivity when calculating indirect members In-Reply-To: <50080676.3070605@redhat.com> References: <5005B450.8000207@redhat.com> <5007F371.7000705@redhat.com> <50080676.3070605@redhat.com> Message-ID: <50080D25.7000009@redhat.com> On 07/19/2012 03:07 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 07/17/2012 08:52 PM, Rob Crittenden wrote: >>> When determining whether a member is direct or indirect we were not >>> doing a case-insensitive comparison which led to marking a member as >>> both direct and indirect (in a test case no less). >>> >>> This patch fixes the comparison and the test. >>> >>> rob >>> >> >> When comparing DNs, you should use the DN class, not string >> manipulations: DN(x) instead of x.lower(). >> >> How urgent is this? John's DN patch solves this in a much more thorough >> way, maybe it'd be better to just wait for that. >> > > The problem is we're currently reporting incorrect membership. I figured > this would be a short-term fix unless you think the DN patch commit is > imminent. > > rob Okay. Still it makes sense to do the right thing, DN(x) instead of x.lower(). -- Petr? From william at firstyear.id.au Thu Jul 19 13:46:52 2012 From: william at firstyear.id.au (William Brown) Date: Thu, 19 Jul 2012 23:16:52 +0930 Subject: [Freeipa-devel] [DHCP] tree layout options In-Reply-To: <1342704544.3219.217.camel@willson.li.ssimo.org> References: <5008027E.4070401@firstyear.id.au> <50080823.5040602@firstyear.id.au> <1342704544.3219.217.camel@willson.li.ssimo.org> Message-ID: <50080FCC.5090803@firstyear.id.au> On 19/07/12 22:59, Simo Sorce wrote: > On Thu, 2012-07-19 at 22:44 +0930, William Brown wrote: >>> does not add any dhcpHost objects not the dhcpFailOverPeer information. >> >> I have found why this is. I was setting ldap-method to dynamic, meaning >> that the contents of this object were only read at lease request time. >> setting this to static has allowed these objects to be read at dhcpd >> initilization time. >> >> It would also seem that in ldap_generate_config_string in ldap.c, the >> dhcpFailOverPeer object class is not "recognized" as an object that >> should be inspected. I'll add this later. > > Hi William, thanks for the ldifs, I will take some time this week to > ponder them. > I do not like having to separate ipv4 and ipv6 in principle, as I > suspect in most cases admin would have to go an create the same > locations under both, however I guess we can handle that in the UI > relatively easily and always create both. > > I will try to consider the other factors as soon as I have some time to > get my head around the other details. > > Simo. > I think that this third ldif may be a bit more along the lines of what you are looking for. I have shown the v4 / v6 split, combined with the tree setup from option 2. I have tried to keep things like locations unique, and only to split v4 / v6 when necesarry. Again, this is tested to generate a working config with pultney/v4 in ISC-dhcp excluding aforementioned known issues. -- Sincerely, William Brown pgp.mit.edu http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x3C0AC6DAB2F928A2 -------------- next part -------------- version: 1 dn: cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: isc dn: cn=servers,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: servers dn: cn=v4,cn=servers,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: v4 dn: cn=v6,cn=servers,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: v6 dn: cn=dhcpserver.dev.firstyear.id.au,cn=v4,cn=servers,cn=isc,cn=dhcp,dc=dev ,dc=firstyear,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: dhcpserver.dev.firstyear.id.au dhcpservicedn: cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firsty ear,dc=id,dc=au dn: cn=freeipa.dev.firstyear.id.au,cn=v4,cn=servers,cn=isc,cn=dhcp,dc=dev,dc =firstyear,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: freeipa.dev.firstyear.id.au dhcpservicedn: cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firsty ear,dc=id,dc=au dn: cn=dhcpserver.dev.firstyear.id.au,cn=v6,cn=servers,cn=isc,cn=dhcp,dc=dev ,dc=firstyear,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: dhcpserver.dev.firstyear.id.au dhcpservicedn: cn=v6,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firsty ear,dc=id,dc=au dn: cn=freeipa.dev.firstyear.id.au,cn=v6,cn=servers,cn=isc,cn=dhcp,dc=dev,dc =firstyear,dc=id,dc=au objectclass: dhcpServer objectclass: top cn: freeipa.dev.firstyear.id.au dhcpservicedn: cn=v6,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firsty ear,dc=id,dc=au dn: cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: locations dn: cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: pultney dn: cn=plaza,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: nsContainer objectclass: top cn: plaza dn: cn=v4,cn=plaza,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc= au objectclass: top objectclass: dhcpService cn: v4 dn: cn=v6,cn=plaza,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,dc= au objectclass: top objectclass: dhcpService cn: v6 dn: cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,d c=au objectclass: top objectclass: dhcpService cn: v4 dhcpoption: domain-name-servers 10.0.0.1 dhcpprimarydn: cn=freeipa.dev.firstyear.id.au,cn=v4,cn=servers,cn=isc,cn=dhc p,dc=dev,dc=firstyear,dc=id,dc=au dhcpsecondarydn: cn=dhcpserver.dev.firstyear.id.au,cn=v4,cn=servers,cn=isc,c n=dhcp,dc=dev,dc=firstyear,dc=id,dc=au dhcpstatements: ddns-update-style none dhcpstatements: max-lease-time 192000 dn: cn=v6,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyear,dc=id,d c=au objectclass: top objectclass: dhcpService cn: v6 dn: cn=hosts,cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyea r,dc=id,dc=au objectclass: nsContainer objectclass: top cn: hosts dn: cn=failover,cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=first year,dc=id,dc=au objectclass: dhcpFailOverPeer objectclass: top cn: failover dhcpfailoverprimaryport: 2000 dhcpfailoverprimaryserver: freeipa.dev.firstyear.id.au dhcpfailoversecondaryport: 2000 dhcpfailoversecondaryserver: dhcpserver.dev.firstyear.id.au dn: cn=failover,cn=v6,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=first year,dc=id,dc=au objectclass: dhcpFailOverPeer objectclass: top cn: failover dhcpfailoverprimaryport: 2000 dhcpfailoverprimaryserver: freeipa.dev.firstyear.id.au dhcpfailoversecondaryport: 2000 dhcpfailoversecondaryserver: dhcpserver.dev.firstyear.id.au dn: cn=hosts,cn=v6,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=firstyea r,dc=id,dc=au objectclass: nsContainer objectclass: top cn: hosts dn: cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=dev,dc=first year,dc=id,dc=au objectclass: nsContainer objectclass: top cn: networks dn: cn=vlan10,cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=de v,dc=firstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan10 dn: cn=10.0.10.0,cn=vlan10,cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc, cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.10.0 dhcpnetmask: 24 dhcpoption: routers 10.0.10.1 dhcprange: 10.0.10.10 10.0.10.15 dn: cn=vlan11,cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=de v,dc=firstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan11 dn: cn=10.0.11.0,cn=vlan11,cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc, cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.11.0 dhcpnetmask: 24 dhcpoption: routers 10.0.11.1 dhcprange: 10.0.11.10 10.0.11.15 dn: cn=vlan30,cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc,cn=dhcp,dc=de v,dc=firstyear,dc=id,dc=au objectclass: dhcpSharedNetwork objectclass: top cn: vlan30 dn: cn=10.0.30.0,cn=vlan30,cn=networks,cn=v4,cn=pultney,cn=locations,cn=isc, cn=dhcp,dc=dev,dc=firstyear,dc=id,dc=au objectclass: dhcpSubnet objectclass: top objectclass: dhcpOptions cn: 10.0.30.0 dhcpnetmask: 24 dhcpoption: routers 10.0.30.1 dhcprange: 10.0.30.10 10.0.30.15 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 945 bytes Desc: OpenPGP digital signature URL: From pspacek at redhat.com Thu Jul 19 14:00:31 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 19 Jul 2012 16:00:31 +0200 Subject: [Freeipa-devel] [PATCH 0032-0035] Add support for MODDN operation to persistent search implementation In-Reply-To: <20120719120954.GA28826@redhat.com> References: <50069EBA.9080400@redhat.com> <20120719114347.GA27803@redhat.com> <5007F685.20104@redhat.com> <20120719120954.GA28826@redhat.com> Message-ID: <500812FF.3080302@redhat.com> On 07/19/2012 02:09 PM, Adam Tkac wrote: > On Thu, Jul 19, 2012 at 01:59:01PM +0200, Petr Spacek wrote: >> Hello, >> >> I have to explain my motivation behind INSIST a bit. Please see comments below. >> >> On 07/19/2012 01:43 PM, Adam Tkac wrote: >>>> On Wed, Jul 18, 2012 at 01:32:10PM +0200, Petr Spacek wrote: >>>> + CHECK(ldap_query(inst, conn, &ldap_qresult_zone, pevent->dn, >>>>> + LDAP_SCOPE_BASE, attrs_zone, 0, >>>>> "(&(objectClass=idnsZone)(idnsZoneActive=TRUE))")); >> This LDAP query has search scope LDAP_SCOPE_BASE. If I understood >> LDAP correctly, it can return 0 or 1 result, but no more. (Two >> objects can't have exactly same DN.) >> >> If specified LDAP query returned (or is believed to returned) more >> than single result, then something went terribly wrong (memory >> corruption/incorrect pointer arithmetic?). >> >> Theoretically this situation should never happen and I can remove >> this check completely, if you want. >> >>>>> >>>>> - for (entry = HEAD(ldap_qresult->ldap_entries); >>>>> - entry != NULL; >>>>> - entry = NEXT(entry, link)) { >>>>> + for (entry_zone = HEAD(ldap_qresult_zone->ldap_entries); >>>>> + entry_zone != NULL; >>>>> + entry_zone = NEXT(entry_zone, link)) { >>>>> delete = ISC_FALSE; >>>>> result = ldap_parse_zoneentry(entry, inst); >>>>> if (result != ISC_R_SUCCESS) >>>>> goto cleanup; >>>>> + >>>>> + if (PSEARCH_MODDN(pevent->chgtype)) { >>>>> + if (dn_to_dnsname(inst->mctx, pevent->prevdn, &prevname, NULL) >>>>> + == ISC_R_SUCCESS) { >>>>> + CHECK(ldap_delete_zone(inst, pevent->prevdn, ISC_TRUE)); >>>>> + } else { >>>>> + log_debug(5, "update_action: old zone wasn't managed " >>>>> + "by plugin, dn '%s'", pevent->prevdn); >>>>> + } >>>>> + >>>>> + /* fill the cache with records from renamed zone */ >>>>> + CHECK(ldap_query(inst, conn, &ldap_qresult_record, pevent->dn, >>>>> + LDAP_SCOPE_ONELEVEL, attrs_record, 0, >>>>> + "(objectClass=idnsRecord)")); >>>>> + >>>>> + /* LDAP schema requires SOA record (at least) */ >>>>> + INSIST(HEAD(ldap_qresult_record->ldap_entries) != NULL); >>>>> + for (entry_record = HEAD(ldap_qresult_record->ldap_entries); >>>>> + entry_record != NULL; >>>>> + entry_record = NEXT(entry_record, link)) { >>>>> + >>>>> + psearch_update(inst, entry_record, NULL); >>>>> + } >>>>> + } >>>>> + >>>>> + INSIST(NEXT(entry_zone, link) == NULL); /* no multiple zones with same DN */ >>> This INSIST seems like too strong for me. Imagine situation when administrator >>> wrongly modifies LDAP database and creates duplicated zone. In this case we will >> AFAIK it is not possible, because idnsName attribute is part of DN >> and duplicate DN is not allowed. > > Ok, this makes sence. So push the patchset as is, thank you. > > Regards, Adam > >>> crash. I would prefer to log error here, that multiple zones exist and only the >>> first occurence is used. >>> >>>>> } >> >> Petr^2 Spacek >> > Pushed to master: ? http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=2d9e71d47997cd75635412cd81349692a8cac1c2 ? http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=16c402e39e467731422b27a6247e0e222e36586c ? http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=4083460acbdce1760aa347ec68abd27d25e1059a ? http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=6f7fd9c9ed9b9c78c1034972f903e8d41de902a8 Petr^2 Spacek From pspacek at redhat.com Thu Jul 19 14:02:09 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 19 Jul 2012 16:02:09 +0200 Subject: [Freeipa-devel] [PATCH 0037] Add missing return value check to new_ldap_instance() In-Reply-To: <20120719114606.GC27803@redhat.com> References: <5006AD88.5030307@redhat.com> <20120719114606.GC27803@redhat.com> Message-ID: <50081361.8070200@redhat.com> On 07/19/2012 01:46 PM, Adam Tkac wrote: > On Wed, Jul 18, 2012 at 02:35:20PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch adds missing return value check to new_ldap_instance(). >> >> https://fedorahosted.org/bind-dyndb-ldap/ticket/85 >> >> Bug was reported by Coverity. > > Ack. Pushed to master: ? http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=640511903fb2cde66dfd759a14f2fab69554f48e Petr^2 Spacek From pspacek at redhat.com Thu Jul 19 14:03:44 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 19 Jul 2012 16:03:44 +0200 Subject: [Freeipa-devel] [PATCH 0036] Raise connection count automatically if serial_autoincrement is enabled In-Reply-To: <20120719114500.GB27803@redhat.com> References: <5006A20A.4010005@redhat.com> <20120719114500.GB27803@redhat.com> Message-ID: <500813C0.2030809@redhat.com> On 07/19/2012 01:45 PM, Adam Tkac wrote: > On Wed, Jul 18, 2012 at 01:46:18PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch reflects new demand from serial_autoincrement feature. >> >> Generally, change in configuration file should by IPA >> install/upgrade scripts. This patch prevents deadlock in situation >> where scripts failed in their job (as you can see right now). >> >> Will be obsoleted by https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . > > Ack. > Pushed to master: http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=0f27c0743ca0dcb6f1f4e8d2bd3e0b6157296e59 Petr^2 Spacek From pviktori at redhat.com Thu Jul 19 15:55:19 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 19 Jul 2012 17:55:19 +0200 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig In-Reply-To: <500805C9.7030702@redhat.com> References: <50069C29.1000304@redhat.com> <50071258.40307@redhat.com> <5007DA7F.7080603@redhat.com> <500805C9.7030702@redhat.com> Message-ID: <50082DE7.2000102@redhat.com> On 07/19/2012 03:04 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 07/18/2012 09:45 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> minimum_connections was sometimes not updated properly on install >>>> because the script set psearch on but assumed it was still off. >>>> Also, the number of connections was not set if the directive was >>>> missing. >>>> >>>> Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 >>> >>> Your changes look good but I found perhaps another bug. I think Petr >>> Spacek may need to chime in on this part. >>> >>> If you start out with serial_autoincrement yes and psearch undefined and >>> connections undefined the result is: >>> >>> connections 2 >>> >>> There is a comment in the code that connections needs to be 4 for >>> autoincrement. I believe psearch needs to be enabled for autoincrement. >>> I'm not sure how much sanity checking we want to do, mostly I'm curious >>> if things will blow up with connections 2 and no psearch and >>> autoincrement. >>> >>> rob >> >> Petr ?pa?ek tells me you'll get an error if serial_autoincrement is >> enabled without psearch. >> Do we want to fix things if users start out with broken configuration? >> > > I think that if we can detect that a configuration is broken we should > complain loudly, if not fix it (and still complain). > > rob > I made the warning into an error message, hopefully that counts as complaining loudly. Anyway with Petr?'s patch 36 it doesn't really matter any more, since the plugin will fix the error (and still complain) for us :) -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0070-02-Fix-updating-minimum_connections-in-ipa-upgradeconfi.patch Type: text/x-patch Size: 3536 bytes Desc: not available URL: From rcritten at redhat.com Thu Jul 19 17:35:38 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Jul 2012 13:35:38 -0400 Subject: [Freeipa-devel] [PATCH] 1035 case sensitivity when calculating indirect members In-Reply-To: <50080D25.7000009@redhat.com> References: <5005B450.8000207@redhat.com> <5007F371.7000705@redhat.com> <50080676.3070605@redhat.com> <50080D25.7000009@redhat.com> Message-ID: <5008456A.9040409@redhat.com> Petr Viktorin wrote: > On 07/19/2012 03:07 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 07/17/2012 08:52 PM, Rob Crittenden wrote: >>>> When determining whether a member is direct or indirect we were not >>>> doing a case-insensitive comparison which led to marking a member as >>>> both direct and indirect (in a test case no less). >>>> >>>> This patch fixes the comparison and the test. >>>> >>>> rob >>>> >>> >>> When comparing DNs, you should use the DN class, not string >>> manipulations: DN(x) instead of x.lower(). >>> >>> How urgent is this? John's DN patch solves this in a much more thorough >>> way, maybe it'd be better to just wait for that. >>> >> >> The problem is we're currently reporting incorrect membership. I figured >> this would be a short-term fix unless you think the DN patch commit is >> imminent. >> >> rob > > Okay. Still it makes sense to do the right thing, DN(x) instead of > x.lower(). Ok. We can hold onto this patch until we get a better feeling on when the DN work will be committed. If it looks to be a while we can commit this as a short-term solution, otherwise it'll just get dropped. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1035-2-member.patch Type: text/x-diff Size: 2371 bytes Desc: not available URL: From jdennis at redhat.com Thu Jul 19 18:01:36 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Jul 2012 14:01:36 -0400 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <5006A15C.3040406@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> <5006A15C.3040406@redhat.com> Message-ID: <50084B80.6000708@redhat.com> Overall I really like the approach, good work. I have not applied and run the patch, so my comments are based only on code reading. I have just a small things which might need changing. One of the ideas in the log manager is to easily support per class logging (borrowed from Java). This allows you to enable/disable or otherwise configure logging per logical code unit as opposed to a big global hammer (i.e. root logger). It also allows for the class name (logger name) to show up in the log output which can be really handy so you know which class is emitting a message (we don't currently turn this on but it's a trival one-line change to the log format spec.). Each significant class should initialize it's own logging, which is very easy to do with this line at the top of a class's __init__() log_mgr.get_logger(self, True) so instead of: self.logger.info() you would say: self.info() A fair amount of the code in the framework is doing this now, but the install code was never cleaned up. That was left for another day, I guess that day is here. Also each log method (i.e. info(), warn(), error(), etc) should pass it's format substitutions as parameters rather than preformatting the message before submitting it to the logger. This is a performance optimization because the logger delays building the message until it knows the message will actually be emitted and not discarded (common case with debug messages). So instead of: self.debug("the contents of this big dict is: %s" % my_dict) You would write: self.debug("the contents of this big dict is: %s", my_dict) A small different in coding, but a lot less work at run time. I'm not sure the following message handling is optimal + message = '\n'.join( + "The hostname resolves to the localhost address (127.0.0.1/::1)", + "Please change your /etc/hosts file so that the hostname", + "resolves to the ip address of your network interface.", + "", + "Please fix your /etc/hosts file and restart the setup program", I'm not a big fan of individual lines of text in the source that are glued together or are emitted individually via a series of print statements. I'd rather see multi-line text as multi-line text using Python's multi-line string operator (e.g. ''' or """). That way you can see the text as it's meant to appear, but there is a bigger reason. Shouldn't this message be internationalized? (e.g. wrapped in _()). When a message is internationalized it's even more important for multi-line text to appear in it entirety to give translators the greatest context and latitude for their translation rather than arbitrarily splitting it up into fragments according to English norms (fragments that might not even translate in another language). Also, a quick glance suggests a number of the messages need i18n treatment. Overall though, I really love the approach, great work! -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu Jul 19 20:33:10 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Jul 2012 16:33:10 -0400 Subject: [Freeipa-devel] [PATCH] 0070 Fix updating minimum_connections in ipa-upgradeconfig In-Reply-To: <50082DE7.2000102@redhat.com> References: <50069C29.1000304@redhat.com> <50071258.40307@redhat.com> <5007DA7F.7080603@redhat.com> <500805C9.7030702@redhat.com> <50082DE7.2000102@redhat.com> Message-ID: <50086F06.30500@redhat.com> Petr Viktorin wrote: > On 07/19/2012 03:04 PM, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 07/18/2012 09:45 PM, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> minimum_connections was sometimes not updated properly on install >>>>> because the script set psearch on but assumed it was still off. >>>>> Also, the number of connections was not set if the directive was >>>>> missing. >>>>> >>>>> Fix of the patch for https://fedorahosted.org/freeipa/ticket/2554 >>>> >>>> Your changes look good but I found perhaps another bug. I think Petr >>>> Spacek may need to chime in on this part. >>>> >>>> If you start out with serial_autoincrement yes and psearch undefined >>>> and >>>> connections undefined the result is: >>>> >>>> connections 2 >>>> >>>> There is a comment in the code that connections needs to be 4 for >>>> autoincrement. I believe psearch needs to be enabled for autoincrement. >>>> I'm not sure how much sanity checking we want to do, mostly I'm curious >>>> if things will blow up with connections 2 and no psearch and >>>> autoincrement. >>>> >>>> rob >>> >>> Petr ?pa?ek tells me you'll get an error if serial_autoincrement is >>> enabled without psearch. >>> Do we want to fix things if users start out with broken configuration? >>> >> >> I think that if we can detect that a configuration is broken we should >> complain loudly, if not fix it (and still complain). >> >> rob >> > > I made the warning into an error message, hopefully that counts as > complaining loudly. > > Anyway with Petr?'s patch 36 it doesn't really matter any more, since > the plugin will fix the error (and still complain) for us :) > ACK, pushed to master rob From jdennis at redhat.com Thu Jul 19 20:52:18 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Jul 2012 16:52:18 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <4FE848D7.1010004@redhat.com> References: <4FE848D7.1010004@redhat.com> Message-ID: <50087382.2000505@redhat.com> On 06/25/2012 07:17 AM, Petr Viktorin wrote: > The translation files we currently store in Git are full of redundant > information: source strings for untranslated messages, and file locations. > The first causes unnecessarily huge files. The second makes diffs > unreadable: when code is edited and line numbers change, metadata for > all messages shows up as changed. This makes reviewing translation > patches, and merging possible conflicts, hard -- it requires specialized > tools. > > This patch changes the Makefile to strip the unneeded data from .po files. > > Translators using Git must now run msgmerge (or, `make merge-po`) to get > .po files they can work with. Transifex users are unaffected, as the > source .pot file is not changed. > > The i18n tests use file locations for producing nice error reports?. > To make this work as before, the .pot is merged in before validation to > restore comments. > Currently this takes a noticeable amount of time, because polib uses a > particularly na?ve algorithm for merging. I've sent a patch to polib to > resolve this; once that makes it downstream merging will be fast again. > > Updating the translations with the new Makefile will cause a >5MB patch. > I don't want to pollute the mailing list with it, at least until the > Makefile patch is reviewed. It's available > https://github.com/encukou/freeipa/commit/65e2e4.patch > > > https://fedorahosted.org/freeipa/ticket/2435 > > > -- > ? And for divining the programming language messages come from, but that > is only done on the .pot file, unaffected by this patch. Good work and it's very close to getting an ACK. There is now a discrepancy between what the Makefile thinks is the list of po files and the actual list of po files after running strip-po. This causes confusing errors. I think the source of this problem is the Makefile has a list of po files in the variable $(po_files) For starters why is: strip-po: @for po_file in $$(ls *.po); do \ instead of: strip-po: @for po_file in $(po_files); do \ If you run "make validate-po" before running "make strip-po" you get: 5 errors in 21 files After stripping the po files "make validate-po" gives you: 14 errors in 21 files The extra 9 errors are due to the fact validate-po is being asked to validate a non-existent po file which it considers an error (which I believe is a correct check). "make msg-stats" gets confused for the same reason, it's asked to examine files that no longer exist. "make mo-files" now fails catastrophically for the same reason, it's being asked to operate on files that don't exist. In general large parts of the Makefile will now be confused or generate errors because the file list is incorrect. Somehow we have to align the list of po files. That presents all sorts of interesting questions: * does the list come from the LINQUAS file? (current method) * does the list come from git? Doesn't work if you're not in a git development tree. This problem is easily seen when the RPM's are built. No file list can be generated because there is no git repo so you end up with 0 files being passed to the validation commands. Since validation is not critical when building RPM's this hasn't been a show stopper but it really needs to be fixed in some way at some point. * does the list come from the current directory contents? What you did with strip-po, but that also has a potential for errors. What if someone deletes or adds a file in their tree by mistake? * should "make strip-po" edit the LINGUAS file? (maybe the best solution). Maybe when it detects an empty file and removes it it should run a sed command to delete the line in LINGUAS? It may not be evident from Makefile.in but over the years there has been competing strategies for how to get our list of files. Simo added the "git ls-files" strategy because he didn't want to have an explict list which had to be maintained (a valid concern) that still left us with the PY_EXPLICIT_FILES list, so how much did that really accomplish? Maybe PY_EXPLICIT_FILES can be removed in favor of a utility that tests the file type (or the hashbang interpreter line). But that still ties things to a git tree (ugh). If you have any great ideas on how to address the file list issue it would be good to hear. However in the interim we have to somehow adjust the po file list after strip-po runs, once that's done I'm happy to ACK it. I wouldn't be surprised if you responded "Well, the file list discrepancy only occurs when a maintainer is explicitly stripping po files and they should know they have to adjust the LINGUAS file therefore these confusing errors won't be seen by someone who would be confused by them". Maybe yes, maybe no. I can think of plenty of times I debugged some build/configure/make failure and groaned because it was in some area that was totally cryptic and unknown to me, took a long time to unravel and had a trivial adjustment fix that would have only been known to an expert in that part of the code. But here are some other issues I saw when exercising the patch, not a problem with the patch but something that needs fixing. "make validate-pot" produces 11 errors, it's because the pot file is old, we have to a "make update-pot" and commit it to git. "make validate-po" shows 5 errors in es.po. I could have sworn I fixed these months ago. Maybe it only got committed to the 2.2. branch? We either have to fix the po file and push it to Transfix (but downloading a current TX version first!) or we have to go into TX and edit the offending msgstr's. Some of these errors are exactly the kind of thing which will cause us to throw an exception at run time if the locale is ES. Anyway, in summary the patch is great, the idea is good, we just need to fix the file list somehow. And at some point the other two issues above will need some attention. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Thu Jul 19 21:04:23 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 19 Jul 2012 17:04:23 -0400 Subject: [Freeipa-devel] [PATCH] 1035 case sensitivity when calculating indirect members In-Reply-To: <5008456A.9040409@redhat.com> References: <5005B450.8000207@redhat.com> <5007F371.7000705@redhat.com> <50080676.3070605@redhat.com> <50080D25.7000009@redhat.com> <5008456A.9040409@redhat.com> Message-ID: <50087657.30008@redhat.com> On 07/19/2012 01:35 PM, Rob Crittenden wrote: > Petr Viktorin wrote: >> On 07/19/2012 03:07 PM, Rob Crittenden wrote: >>> Petr Viktorin wrote: >>>> On 07/17/2012 08:52 PM, Rob Crittenden wrote: >>>>> When determining whether a member is direct or indirect we were not >>>>> doing a case-insensitive comparison which led to marking a member as >>>>> both direct and indirect (in a test case no less). >>>>> >>>>> This patch fixes the comparison and the test. >>>>> >>>>> rob >>>>> >>>> >>>> When comparing DNs, you should use the DN class, not string >>>> manipulations: DN(x) instead of x.lower(). >>>> >>>> How urgent is this? John's DN patch solves this in a much more thorough >>>> way, maybe it'd be better to just wait for that. >>>> >>> >>> The problem is we're currently reporting incorrect membership. I figured >>> this would be a short-term fix unless you think the DN patch commit is >>> imminent. >>> >>> rob >> >> Okay. Still it makes sense to do the right thing, DN(x) instead of >> x.lower(). > > Ok. We can hold onto this patch until we get a better feeling on when > the DN work will be committed. If it looks to be a while we can commit > this as a short-term solution, otherwise it'll just get dropped. Funny, when doing the DN testing I uncovered exactly this problem because I was getting different results than what was happening with an unmodified master. I worked this out with Rob on the phone I believe. Rob was being good by getting a fix in immediately, thank you Rob. But my suggestion would be to just pick up the fix with the dn patch which will hopefully be soon. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Thu Jul 19 22:00:17 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Thu, 19 Jul 2012 18:00:17 -0400 Subject: [Freeipa-devel] [PATCH] change default answer when installing replica on wrong host Message-ID: <50088371.7050701@redhat.com> Pushed this under the one-liner rule. From 78810f906873c261277189b279d4da33dbd05eb2 Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Thu, 19 Jul 2012 00:41:01 -0400 Subject: [PATCH] Default to no when trying trying to install a replica on wrong server. When installing a replica file on the wrong server we warn that this will likely fail and prompt to Continue. This prompt should default to False, not True. https://fedorahosted.org/freeipa/ticket/2325 --- install/tools/ipa-replica-install | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/install/tools/ipa-replica-install b/install/tools/ipa-replica-install index d162b01..063eea0 100755 --- a/install/tools/ipa-replica-install +++ b/install/tools/ipa-replica-install @@ -336,7 +336,7 @@ def main(): if config.host_name != host: try: print "This replica was created for '%s' but this machine is named '%s'" % (config.host_name, host) - if not ipautil.user_input("This may cause problems. Continue?", True): + if not ipautil.user_input("This may cause problems. Continue?", False): sys.exit(0) config.host_name = host print "" -- 1.6.2.5 From pviktori at redhat.com Fri Jul 20 09:09:50 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 20 Jul 2012 11:09:50 +0200 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <50087382.2000505@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> Message-ID: <5009205E.9030902@redhat.com> On 07/19/2012 10:52 PM, John Dennis wrote: > On 06/25/2012 07:17 AM, Petr Viktorin wrote: >> The translation files we currently store in Git are full of redundant >> information: source strings for untranslated messages, and file >> locations. >> The first causes unnecessarily huge files. The second makes diffs >> unreadable: when code is edited and line numbers change, metadata for >> all messages shows up as changed. This makes reviewing translation >> patches, and merging possible conflicts, hard -- it requires specialized >> tools. >> >> This patch changes the Makefile to strip the unneeded data from .po >> files. >> >> Translators using Git must now run msgmerge (or, `make merge-po`) to get >> .po files they can work with. Transifex users are unaffected, as the >> source .pot file is not changed. >> >> The i18n tests use file locations for producing nice error reports?. >> To make this work as before, the .pot is merged in before validation to >> restore comments. >> Currently this takes a noticeable amount of time, because polib uses a >> particularly na?ve algorithm for merging. I've sent a patch to polib to >> resolve this; once that makes it downstream merging will be fast again. >> >> Updating the translations with the new Makefile will cause a >5MB patch. >> I don't want to pollute the mailing list with it, at least until the >> Makefile patch is reviewed. It's available >> https://github.com/encukou/freeipa/commit/65e2e4.patch >> >> >> https://fedorahosted.org/freeipa/ticket/2435 >> >> >> -- >> ? And for divining the programming language messages come from, but that >> is only done on the .pot file, unaffected by this patch. > > Good work and it's very close to getting an ACK. > > There is now a discrepancy between what the Makefile thinks is the list > of po files and the actual list of po files after running strip-po. This > causes confusing errors. > > I think the source of this problem is the Makefile has a list of po > files in the variable $(po_files) > > For starters why is: > > strip-po: > @for po_file in $$(ls *.po); do \ > > instead of: > > strip-po: > @for po_file in $(po_files); do \ Good catch, I'll update it to be consistent with the status quo. But see below. > If you run "make validate-po" before running "make strip-po" you get: > > 5 errors in 21 files > > After stripping the po files "make validate-po" gives you: > > 14 errors in 21 files I left updating the files to a subsequent patch (https://github.com/encukou/freeipa/commit/65e2e4.patch); the LINGUAS update is part of that. > The extra 9 errors are due to the fact validate-po is being asked to > validate a non-existent po file which it considers an error (which I > believe is a correct check). > > "make msg-stats" gets confused for the same reason, it's asked to > examine files that no longer exist. > > "make mo-files" now fails catastrophically for the same reason, it's > being asked to operate on files that don't exist. > > In general large parts of the Makefile will now be confused or generate > errors because the file list is incorrect. > > > Somehow we have to align the list of po files. That presents all sorts > of interesting questions: > > * does the list come from the LINQUAS file? (current method) > > * does the list come from git? Doesn't work if you're not in a git > development tree. This problem is easily seen when the RPM's are built. > No file list can be generated because there is no git repo so you end up > with 0 files being passed to the validation commands. Since validation > is not critical when building RPM's this hasn't been a show stopper but > it really needs to be fixed in some way at some point. I agree that tying ourselves to Git isn't a nice thing to do. I know I am never happy when I can't compile some project in Mercurial after importing it to Git :) If we use the ls-files strategy then that should at least write the list to a version-controlled file, which we fall back to in case we're not in a git tree. > * does the list come from the current directory contents? What you did > with strip-po, but that also has a potential for errors. What if someone > deletes or adds a file in their tree by mistake? I personally would do this -- the most straightforward way to do it. If someone adds or deletes a file by mistake, a `git status` will reveal it. We could have a sanity check that refuses to build if there is a discrepancy between Git and the working tree (of course outside of a Git repo it would just warn). There's one more reason for going with directory contents: when you're pulling from Transifex or otherwise adding/removing the translation files, you have to carefully keep LINGUAS in sync with the tree, otherwise the tools can either blow up or do too little. Debugging that could be frustrating. Having the tools look in the directory itself, and only doing sanity checking at a point where everything should be in order, should make everything easier. > * should "make strip-po" edit the LINGUAS file? (maybe the best > solution). Maybe when it detects an empty file and removes it it should > run a sed command to delete the line in LINGUAS? That strikes me as too much automation, the case where all translations for a language disappear should be pretty rare. I did add a message saying you should update LINGUAS. > It may not be evident from Makefile.in but over the years there has been > competing strategies for how to get our list of files. Simo added the > "git ls-files" strategy because he didn't want to have an explict list > which had to be maintained (a valid concern) that still left us with the > PY_EXPLICIT_FILES list, so how much did that really accomplish? Maybe > PY_EXPLICIT_FILES can be removed in favor of a utility that tests the > file type (or the hashbang interpreter line). But that still ties things > to a git tree (ugh). As for PY_EXPLICIT_FILES, I hope it dies. Actual code should be in importable modules, and the scripts should be one-liners that call the code. I hope IPA goes that way and I'll do what I can to make it happen. > If you have any great ideas on how to address the file list issue it > would be good to hear. However in the interim we have to somehow adjust > the po file list after strip-po runs, once that's done I'm happy to ACK it. > I wouldn't be surprised if you responded "Well, the file list > discrepancy only occurs when a maintainer is explicitly stripping po > files and they should know they have to adjust the LINGUAS file > therefore these confusing errors won't be seen by someone who would be > confused by them". Maybe yes, maybe no. I can think of plenty of times I > debugged some build/configure/make failure and groaned because it was in > some area that was totally cryptic and unknown to me, took a long time > to unravel and had a trivial adjustment fix that would have only been > known to an expert in that part of the code. > > But here are some other issues I saw when exercising the patch, not a > problem with the patch but something that needs fixing. > > "make validate-pot" produces 11 errors, it's because the pot file is > old, we have to a "make update-pot" and commit it to git. As I said the actual language updates are left for another patch. I don't like mixing logic and data changes :) > "make validate-po" shows 5 errors in es.po. I could have sworn I fixed > these months ago. Maybe it only got committed to the 2.2. branch? We > either have to fix the po file and push it to Transfix (but downloading > a current TX version first!) or we have to go into TX and edit the > offending msgstr's. Some of these errors are exactly the kind of thing > which will cause us to throw an exception at run time if the locale is ES. You'll also notice that there are a few more errors in the new messages on Transifex. > Anyway, in summary the patch is great, the idea is good, we just need to > fix the file list somehow. And at some point the other two issues above > will need some attention. > -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0066-02-Arrange-stripping-.po-files.patch Type: text/x-patch Size: 8081 bytes Desc: not available URL: From pviktori at redhat.com Fri Jul 20 11:00:06 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 20 Jul 2012 13:00:06 +0200 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <50084B80.6000708@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> <5006A15C.3040406@redhat.com> <50084B80.6000708@redhat.com> Message-ID: <50093A36.2010409@redhat.com> On 07/19/2012 08:01 PM, John Dennis wrote: > Overall I really like the approach, good work. > > I have not applied and run the patch, so my comments are based only on > code reading. > > I have just a small things which might need changing. > > One of the ideas in the log manager is to easily support per class > logging (borrowed from Java). This allows you to enable/disable or > otherwise configure logging per logical code unit as opposed to a big > global hammer (i.e. root logger). It also allows for the class name > (logger name) to show up in the log output which can be really handy so > you know which class is emitting a message (we don't currently turn this > on but it's a trival one-line change to the log format spec.). > > Each significant class should initialize it's own logging, which is very > easy to do with this line at the top of a class's __init__() > > log_mgr.get_logger(self, True) > > so instead of: > > self.logger.info() > > you would say: > > self.info() > > A fair amount of the code in the framework is doing this now, but the > install code was never cleaned up. That was left for another day, I > guess that day is here. Updated. I also added the necessary lint exception. I'm curious as to why it works that way, though. Normally, to put methods in a class, you would use a base class/mixin. Why this dynamic magic? > Also each log method (i.e. info(), warn(), error(), etc) should pass > it's format substitutions as parameters rather than preformatting the > message before submitting it to the logger. This is a performance > optimization because the logger delays building the message until it > knows the message will actually be emitted and not discarded (common > case with debug messages). > > So instead of: > > self.debug("the contents of this big dict is: %s" % my_dict) > > You would write: > > self.debug("the contents of this big dict is: %s", my_dict) > > A small different in coding, but a lot less work at run time. Fixed, thanks for noticing. > I'm not sure the following message handling is optimal > > + message = '\n'.join( > + "The hostname resolves to the localhost address > (127.0.0.1/::1)", > + "Please change your /etc/hosts file so that the hostname", > + "resolves to the ip address of your network interface.", > + "", > + "Please fix your /etc/hosts file and restart the setup > program", > > > I'm not a big fan of individual lines of text in the source that are > glued together or are emitted individually via a series of print > statements. I'd rather see multi-line text as multi-line text using > Python's multi-line string operator (e.g. ''' or """). That way you can > see the text as it's meant to appear, but there is a bigger reason. You're right, fixed. To keep sane indentation I used textwrap.dedent. > Shouldn't this message be internationalized? (e.g. wrapped in _()). When > a message is internationalized it's even more important for multi-line > text to appear in it entirety to give translators the greatest context > and latitude for their translation rather than arbitrarily splitting it > up into fragments according to English norms (fragments that might not > even translate in another language). > > Also, a quick glance suggests a number of the messages need i18n treatment. None of the tools are internationalized. Perhaps they need to be, but it should be a separate ticket. > Overall though, I really love the approach, great work! Thanks for the review :) -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0056-06-Framework-for-admin-install-tools-with-ipa-ldap-upda.patch Type: text/x-patch Size: 31928 bytes Desc: not available URL: From pspacek at redhat.com Fri Jul 20 12:28:09 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 20 Jul 2012 14:28:09 +0200 Subject: [Freeipa-devel] [PATCH 0038] Fix two memory leaks in ldap_query() Message-ID: <50094ED9.2020709@redhat.com> Hello, this patch fixes two memory leaks in ldap_query(). Both memory leaks occurs after "non-success" queries. It effectively re-implements fix for "ldap_query can incorrectly return ISC_R_SUCCESS even when failed": http://git.fedorahosted.org/git/?p=bind-dyndb-ldap.git;a=commitdiff;h=a7cd8ae747b3a81a02ab9e5dbefe1c595aa24ff6 Please double-check this approach. Thanks. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0038-Fix-two-memory-leaks-in-ldap_query.patch Type: text/x-patch Size: 1577 bytes Desc: not available URL: From jdennis at redhat.com Fri Jul 20 15:39:43 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Jul 2012 11:39:43 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <5009205E.9030902@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> Message-ID: <50097BBF.6010206@redhat.com> Great I agree with everything you said. I'm happy to have the file list be derived from the directory contents. Are you planning on doing that in another patch? FWIW the LINGUAS file was a holdover from when we first set this up based exclusively on GNU gettext suggested examples. As things have evolved it no longer makes sense. Also the contributing translators file is now out of date and was from an earlier era when translators emailed .po files to us, so it was easy to maintain. Now that everything is TX based we should probably nuke that file or figure out someway to extract the contributors from either TX or the po files. I'm not sure we're even giving credit to the translators anymore, but we should. But... I do have one final issue/question. I missed this in the first review. po_files is now dependent on update-pot instead of the pot file. We had decided that we were only going to regenerate the pot file on demand at specific times. Won't this dependency change cause the pot file to be updated frequently? (I realize only in the local tree). Note that when we run the validations we generate a temporary pot file from the current contents of the tree specifically to avoid overwriting the pot file. I suppose having a conversation about when the pot file gets updated is a good one to have, we don't do it often enough IMHO. But I'm not sure it's correct to modify a file under SCM control if it wasn't intentional. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Fri Jul 20 15:59:05 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Jul 2012 11:59:05 -0400 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <50093A36.2010409@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> <5006A15C.3040406@redhat.com> <50084B80.6000708@redhat.com> <50093A36.2010409@redhat.com> Message-ID: <50098049.6070809@redhat.com> >> A fair amount of the code in the framework is doing this now, but the >> install code was never cleaned up. That was left for another day, I >> guess that day is here. > > Updated. I also added the necessary lint exception. > I'm curious as to why it works that way, though. Normally, to put > methods in a class, you would use a base class/mixin. Why this dynamic > magic? Good question. I don't like dynamic magic either, it makes static code reading difficult and diminishes the value of static code analysis/checking. Jason who originally wrote the framework used dynamic magic a fair amount, including the initialization of the logging methods in the plugins. When I wrote the log manager I kept Jason's existing strategy (how many things do change at once?). In hindsight a mixin would have been a better solution, something we should probably fix down the road. Everything looks good, although there might be one minor thing that needs fixing. Shouldn't the logging setup be done first? That way any code that executes has the ability to emit a message. For example what if validate_options() wants to emit a message? -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pviktori at redhat.com Fri Jul 20 16:28:01 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 20 Jul 2012 18:28:01 +0200 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <50097BBF.6010206@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> Message-ID: <50098711.2020603@redhat.com> On 07/20/2012 05:39 PM, John Dennis wrote: > Great I agree with everything you said. > > I'm happy to have the file list be derived from the directory contents. > Are you planning on doing that in another patch? Yes, I want to do it in a new patch. It's a bit more complicated than it looks: creating a new translation will work differently than just adding it to LINGUAS and running create-po. The ticket is for beta 2 so I'd rather not start a new round of reviews. > FWIW the LINGUAS file was a holdover from when we first set this up > based exclusively on GNU gettext suggested examples. As things have > evolved it no longer makes sense. Also the contributing translators file > is now out of date and was from an earlier era when translators emailed > .po files to us, so it was easy to maintain. Now that everything is TX > based we should probably nuke that file or figure out someway to extract > the contributors from either TX or the po files. I'm not sure we're even > giving credit to the translators anymore, but we should. Noted; when the discussion's done I'll file a ticket. > But... I do have one final issue/question. I missed this in the first > review. po_files is now dependent on update-pot instead of the pot file. > We had decided that we were only going to regenerate the pot file on > demand at specific times. Won't this dependency change cause the pot > file to be updated frequently? (I realize only in the local tree). Note > that when we run the validations we generate a temporary pot file from > the current contents of the tree specifically to avoid overwriting the > pot file. Are the po files updated more often? I don't really see a reason to merge the po files with an old pot. > I suppose having a conversation about when the pot file gets updated is > a good one to have, we don't do it often enough IMHO. But I'm not sure > it's correct to modify a file under SCM control if it wasn't intentional. How is Transifex set up here? If it automatically picks up changes when the pot file is modified, then we should back up the translations before changing the pot, so we can't do it automatically. Another wart is the line number cruft in the pot file -- any time it's updated we'll get a huge diff, so it makes sense to update sparingly. If Transifex is not wired to the pot, we could even go as far as removing it from SCM entirely -- it's entirely generated, and rebuilding it takes less than a second. We'd just have to update Transifex manually. Oh, one thing I'll ask about: the Makefile is not automatically rebuilt when I change Makefile.in, so every time I run an autogen invocation I found in the main Makefile. Is there an easier way to do this? -- Petr? From pviktori at redhat.com Fri Jul 20 16:34:28 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 20 Jul 2012 18:34:28 +0200 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <50098049.6070809@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> <5006A15C.3040406@redhat.com> <50084B80.6000708@redhat.com> <50093A36.2010409@redhat.com> <50098049.6070809@redhat.com> Message-ID: <50098894.6030704@redhat.com> On 07/20/2012 05:59 PM, John Dennis wrote: >>> A fair amount of the code in the framework is doing this now, but the >>> install code was never cleaned up. That was left for another day, I >>> guess that day is here. >> >> Updated. I also added the necessary lint exception. >> I'm curious as to why it works that way, though. Normally, to put >> methods in a class, you would use a base class/mixin. Why this dynamic >> magic? > > Good question. I don't like dynamic magic either, it makes static code > reading difficult and diminishes the value of static code > analysis/checking. Jason who originally wrote the framework used dynamic > magic a fair amount, including the initialization of the logging methods > in the plugins. When I wrote the log manager I kept Jason's existing > strategy (how many things do change at once?). In hindsight a mixin > would have been a better solution, something we should probably fix down > the road. Thinking more about this, composition would probably be cleaner than inheritance here (i.e. using self.logger.warn(...) instead of mixing the functionality into the class itself). Well, one day the time to fix it will come. > Everything looks good, although there might be one minor thing that > needs fixing. Shouldn't the logging setup be done first? That way any > code that executes has the ability to emit a message. For example what > if validate_options() wants to emit a message? > That's a good question. If we set up logging before validating the options, we'll log all invocations, including those with misspelled option names and forgotten "sudo"s. Do we want that? -- Petr? From jdennis at redhat.com Fri Jul 20 17:14:20 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Jul 2012 13:14:20 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <50098711.2020603@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> Message-ID: <500991EC.9000703@redhat.com> On 07/20/2012 12:28 PM, Petr Viktorin wrote: > On 07/20/2012 05:39 PM, John Dennis wrote: >> Great I agree with everything you said. >> >> I'm happy to have the file list be derived from the directory contents. >> Are you planning on doing that in another patch? > > Yes, I want to do it in a new patch. > It's a bit more complicated than it looks: creating a new translation > will work differently than just adding it to LINGUAS and running > create-po. The ticket is for beta 2 so I'd rather not start a new round > of reviews. Fine with me to do that in another patch. As for create-po, I think that's also holdover from pre-Transifex days. With Transifex I'd don't ever see a need to create an empty po file. Do you? Maybe we should just nuke the po creation in the Makefile. > >> FWIW the LINGUAS file was a holdover from when we first set this up >> based exclusively on GNU gettext suggested examples. As things have >> evolved it no longer makes sense. Also the contributing translators file >> is now out of date and was from an earlier era when translators emailed >> .po files to us, so it was easy to maintain. Now that everything is TX >> based we should probably nuke that file or figure out someway to extract >> the contributors from either TX or the po files. I'm not sure we're even >> giving credit to the translators anymore, but we should. > > Noted; when the discussion's done I'll file a ticket. > >> But... I do have one final issue/question. I missed this in the first >> review. po_files is now dependent on update-pot instead of the pot file. >> We had decided that we were only going to regenerate the pot file on >> demand at specific times. Won't this dependency change cause the pot >> file to be updated frequently? (I realize only in the local tree). Note >> that when we run the validations we generate a temporary pot file from >> the current contents of the tree specifically to avoid overwriting the >> pot file. > > Are the po files updated more often? I don't really see a reason to > merge the po files with an old pot. What merge are you referring to? The only merge I'm aware of at the moment is during validation, but that merge is done from a temporary updated pot file that is current with the tree. Any other merging is done by Transifex at the time we pull a po file. The frequency of po update doesn't seem relevant, what is your concern in this regard? > >> I suppose having a conversation about when the pot file gets updated is >> a good one to have, we don't do it often enough IMHO. But I'm not sure >> it's correct to modify a file under SCM control if it wasn't intentional. > > How is Transifex set up here? If it automatically picks up changes when > the pot file is modified, then we should back up the translations before > changing the pot, so we can't do it automatically. > Another wart is the line number cruft in the pot file -- any time it's > updated we'll get a huge diff, so it makes sense to update sparingly. Transifex gives you two options for your pot file, either you tell TX the location of your pot file in a public SCM and it watches for updates and automatically pulls it when it changes in SCM -or- you manually push the pot file to TX. We've been using the "watch the pot file in git" option. Thus whenever we commit a new version of the pot file all developers and TX get it simultaneously (well sort of). If we do the manual push method the maintainer has to *both* commit to git *and* push to TX, so the former seems less error prone and more automated. The idea was we would have a string freeze prior to release and/or periodic intervals during branch development to update the pot. But we haven't been good about hitting these. However, note a manual push suffers from the same "somebody has to do it at the right moment" problem. > > If Transifex is not wired to the pot, we could even go as far as > removing it from SCM entirely -- it's entirely generated, and rebuilding > it takes less than a second. > We'd just have to update Transifex manually. It currently is wired to the pot. You make a valid point about currently not needing to maintain the pot in SCM. When we first set up translations we weren't using TX so having the pot file in SCM was a necessity. Personally I don't trust TX's data storage and I think there is value in having each pot we push to TX be recoverable from our SCM. When things blow up (and they do) it's really nice to be able to reassemble the pieces or at lease follow the trail of how things changed. In the past I've had to answer questions like "How the heck did this string get into this po file?" Such questions can only be answered if we have the pot file we gave the translator. TX doesn't maintain it so we have to (or at least I think there is value in it). Perhaps you can read between the lines and detect I don't view TX as the epitome of stability and robustness. It's still young and they are still adding features and changing how it works (kinda like IPA :-) > Oh, one thing I'll ask about: the Makefile is not automatically rebuilt > when I change Makefile.in, so every time I run an autogen invocation I > found in the main Makefile. Is there an easier way to do this? > Ugh ... autotools :-( The Makefile is generated via configure and I thought autoregen had the smarts to detect the dependencies. I too noticed I had to manually re-run configure. Are we running autoregen? If so and it's not working I don't have an answer, sorry. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Fri Jul 20 17:32:51 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 20 Jul 2012 13:32:51 -0400 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <50098894.6030704@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> <5006A15C.3040406@redhat.com> <50084B80.6000708@redhat.com> <50093A36.2010409@redhat.com> <50098049.6070809@redhat.com> <50098894.6030704@redhat.com> Message-ID: <50099643.7060908@redhat.com> On 07/20/2012 12:34 PM, Petr Viktorin wrote: > On 07/20/2012 05:59 PM, John Dennis wrote: >>>> A fair amount of the code in the framework is doing this now, but the >>>> install code was never cleaned up. That was left for another day, I >>>> guess that day is here. >>> >>> Updated. I also added the necessary lint exception. >>> I'm curious as to why it works that way, though. Normally, to put >>> methods in a class, you would use a base class/mixin. Why this dynamic >>> magic? >> >> Good question. I don't like dynamic magic either, it makes static code >> reading difficult and diminishes the value of static code >> analysis/checking. Jason who originally wrote the framework used dynamic >> magic a fair amount, including the initialization of the logging methods >> in the plugins. When I wrote the log manager I kept Jason's existing >> strategy (how many things do change at once?). In hindsight a mixin >> would have been a better solution, something we should probably fix down >> the road. > > Thinking more about this, composition would probably be cleaner than > inheritance here (i.e. using self.logger.warn(...) instead of mixing the > functionality into the class itself). > Well, one day the time to fix it will come. > >> Everything looks good, although there might be one minor thing that >> needs fixing. Shouldn't the logging setup be done first? That way any >> code that executes has the ability to emit a message. For example what >> if validate_options() wants to emit a message? >> > > That's a good question. > If we set up logging before validating the options, we'll log all > invocations, including those with misspelled option names and forgotten > "sudo"s. Do we want that? > Sure, hard to say. Why don't we leave it the way it is now and down the road if it shows up as an issue we'll tweak it then. ACK. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From atkac at redhat.com Mon Jul 23 10:03:30 2012 From: atkac at redhat.com (Adam Tkac) Date: Mon, 23 Jul 2012 12:03:30 +0200 Subject: [Freeipa-devel] [PATCH 0038] Fix two memory leaks in ldap_query() In-Reply-To: <50094ED9.2020709@redhat.com> References: <50094ED9.2020709@redhat.com> Message-ID: <20120723100330.GB4393@redhat.com> On Fri, Jul 20, 2012 at 02:28:09PM +0200, Petr Spacek wrote: > Hello, > > this patch fixes two memory leaks in ldap_query(). Both memory leaks > occurs after "non-success" queries. > > It effectively re-implements fix for "ldap_query can incorrectly > return ISC_R_SUCCESS even when failed": > http://git.fedorahosted.org/git/?p=bind-dyndb-ldap.git;a=commitdiff;h=a7cd8ae747b3a81a02ab9e5dbefe1c595aa24ff6 > > Please double-check this approach. Ack, please push it to master. A > From c8718b98641e7537b2350a625b03b0b7fec6f206 Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Fri, 20 Jul 2012 14:18:41 +0200 > Subject: [PATCH] Fix two memory leaks in ldap_query(). > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 14 ++++++++------ > 1 files changed, 8 insertions(+), 6 deletions(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index 6ac76faecbc250deb28203256fa13d83ae6c80f4..daffac7c7825a99a07c333217638d3beaddfaad2 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -1753,28 +1753,30 @@ retry: > &ldap_qresult->ldap_entries); > if (result != ISC_R_SUCCESS) { > log_error("failed to save LDAP query results"); > - return result; > + goto cleanup; > } > > *ldap_qresultp = ldap_qresult; > return ISC_R_SUCCESS; > + } else { > + result = ISC_R_FAILURE; > } > > ret = ldap_get_option(ldap_conn->handle, LDAP_OPT_RESULT_CODE, > (void *)&ldap_err_code); > - if (ret == LDAP_OPT_SUCCESS && ldap_err_code == LDAP_NO_SUCH_OBJECT) > - return ISC_R_NOTFOUND; > - /* some error happened during ldap_search, try to recover */ > - else if (!once) { > + if (ret == LDAP_OPT_SUCCESS && ldap_err_code == LDAP_NO_SUCH_OBJECT) { > + result = ISC_R_NOTFOUND; > + } else if (!once) { > + /* some error happened during ldap_search, try to recover */ > once++; > result = handle_connection_error(ldap_inst, ldap_conn, > ISC_FALSE); > if (result == ISC_R_SUCCESS) > goto retry; > } > cleanup: > ldap_query_free(ISC_FALSE, &ldap_qresult); > - return ISC_R_FAILURE; > + return result; > } > > /** > -- > 1.7.7.6 > -- Adam Tkac, Red Hat, Inc. From pviktori at redhat.com Mon Jul 23 10:27:58 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Mon, 23 Jul 2012 12:27:58 +0200 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500991EC.9000703@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> Message-ID: <500D272E.6040309@redhat.com> On 07/20/2012 07:14 PM, John Dennis wrote: > On 07/20/2012 12:28 PM, Petr Viktorin wrote: >> On 07/20/2012 05:39 PM, John Dennis wrote: >>> Great I agree with everything you said. >>> >>> I'm happy to have the file list be derived from the directory contents. >>> Are you planning on doing that in another patch? >> >> Yes, I want to do it in a new patch. >> It's a bit more complicated than it looks: creating a new translation >> will work differently than just adding it to LINGUAS and running >> create-po. The ticket is for beta 2 so I'd rather not start a new round >> of reviews. > > Fine with me to do that in another patch. > > As for create-po, I think that's also holdover from pre-Transifex days. > With Transifex I'd don't ever see a need to create an empty po file. Do > you? Maybe we should just nuke the po creation in the Makefile. As a translator (for another project), I don't like Transifex and prefer to send good old Git pull requests. I understand a "traditional" workflow is hard to coordinate with others that use Transifex, but still I'd hate it if we became dependent on Tx. [...] >>> But... I do have one final issue/question. I missed this in the first >>> review. po_files is now dependent on update-pot instead of the pot file. >>> We had decided that we were only going to regenerate the pot file on >>> demand at specific times. Won't this dependency change cause the pot >>> file to be updated frequently? (I realize only in the local tree). Note >>> that when we run the validations we generate a temporary pot file from >>> the current contents of the tree specifically to avoid overwriting the >>> pot file. >> >> Are the po files updated more often? I don't really see a reason to >> merge the po files with an old pot. > > What merge are you referring to? The only merge I'm aware of at the > moment is during validation, but that merge is done from a temporary > updated pot file that is current with the tree. I'm referring to a manual merge-po. po_files are only rebuilt from merge-po, which merges the po files with the pot and adds all the missing translations and line numbers. This is not needed with Transifex workflow, as Tx should do this internally when a pot is pushed to it. > Any other merging is done by Transifex at the time we pull a po file. > > The frequency of po update doesn't seem relevant, what is your concern > in this regard? Is there another cause for the po_files to get rebuilt? >>> I suppose having a conversation about when the pot file gets updated is >>> a good one to have, we don't do it often enough IMHO. But I'm not sure >>> it's correct to modify a file under SCM control if it wasn't >>> intentional. >> >> How is Transifex set up here? If it automatically picks up changes when >> the pot file is modified, then we should back up the translations before >> changing the pot, so we can't do it automatically. >> Another wart is the line number cruft in the pot file -- any time it's >> updated we'll get a huge diff, so it makes sense to update sparingly. > > Transifex gives you two options for your pot file, either you tell TX > the location of your pot file in a public SCM and it watches for updates > and automatically pulls it when it changes in SCM -or- you manually push > the pot file to TX. > > We've been using the "watch the pot file in git" option. Thus whenever > we commit a new version of the pot file all developers and TX get it > simultaneously (well sort of). > > If we do the manual push method the maintainer has to *both* commit to > git *and* push to TX, so the former seems less error prone and more > automated. Well, if the pot file is not in the repo, the maintainer only has to push it to Tx (after building it of course, but that needs to be done anyway). > The idea was we would have a string freeze prior to release and/or > periodic intervals during branch development to update the pot. But we > haven't been good about hitting these. However, note a manual push > suffers from the same "somebody has to do it at the right moment" problem. Is this idea documented anywhere? It's hard to do a string freeze if it's not enforced automatically, let alone if people don't know there should be one. >> If Transifex is not wired to the pot, we could even go as far as >> removing it from SCM entirely -- it's entirely generated, and rebuilding >> it takes less than a second. >> We'd just have to update Transifex manually. > > It currently is wired to the pot. You make a valid point about currently > not needing to maintain the pot in SCM. When we first set up > translations we weren't using TX so having the pot file in SCM was a > necessity. > > Personally I don't trust TX's data storage and I think there is value in > having each pot we push to TX be recoverable from our SCM. When things > blow up (and they do) it's really nice to be able to reassemble the > pieces or at lease follow the trail of how things changed. In the past > I've had to answer questions like "How the heck did this string get into > this po file?" Such questions can only be answered if we have the pot > file we gave the translator. TX doesn't maintain it so we have to (or at > least I think there is value in it). Yes, Transifex only keeps the last pushed pot file. However, I don't think the git repo is a good place to store information that's missing from Transifex. To really keep a good history, we'd also need to pull the po files from Transifex much more often. I guess we could do that in the main repo, but I don't think the other devs would be too happy with daily/weekly automated commits reflecting the current state of the translations. Another option is to keep the history either in a separate branch, or in a separate repository (hosted for example on fedorapeople). In that case we can include pot snapshots, full po files as they're pulled from Tx, or any other info we want, and update it as frequently as needed. > Perhaps you can read between the lines and detect I don't view TX as the > epitome of stability and robustness. It's still young and they are still > adding features and changing how it works (kinda like IPA :-) > >> Oh, one thing I'll ask about: the Makefile is not automatically rebuilt >> when I change Makefile.in, so every time I run an autogen invocation I >> found in the main Makefile. Is there an easier way to do this? >> > > Ugh ... autotools :-( > > The Makefile is generated via configure and I thought autoregen had the > smarts to detect the dependencies. I too noticed I had to manually > re-run configure. Are we running autoregen? If so and it's not working I > don't have an answer, sorry. install/po/Makefile is generated by this line in the main Makefile: cd install; if [ ! -e Makefile ]; then ../autogen.sh --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=$(LIBDIR); fi I'm not very familiar with autotools but I don't think we're using them correctly :) -- Petr? From pspacek at redhat.com Mon Jul 23 10:50:58 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 23 Jul 2012 12:50:58 +0200 Subject: [Freeipa-devel] [PATCH 0038] Fix two memory leaks in ldap_query() In-Reply-To: <20120723100330.GB4393@redhat.com> References: <50094ED9.2020709@redhat.com> <20120723100330.GB4393@redhat.com> Message-ID: <500D2C92.3090609@redhat.com> On 07/23/2012 12:03 PM, Adam Tkac wrote: > On Fri, Jul 20, 2012 at 02:28:09PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch fixes two memory leaks in ldap_query(). Both memory leaks >> occurs after "non-success" queries. >> >> It effectively re-implements fix for "ldap_query can incorrectly >> return ISC_R_SUCCESS even when failed": >> http://git.fedorahosted.org/git/?p=bind-dyndb-ldap.git;a=commitdiff;h=a7cd8ae747b3a81a02ab9e5dbefe1c595aa24ff6 >> >> Please double-check this approach. > > Ack, please push it to master. > > A Pushed to master: http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=77c06ea1910a9737bf7e2d9f5c53eeb83827c332 Petr^2 Spacek From pvoborni at redhat.com Mon Jul 23 14:05:19 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Mon, 23 Jul 2012 16:05:19 +0200 Subject: [Freeipa-devel] [PATCH] 174 Fix autoscroll to top in tables in IE Message-ID: <500D5A1F.7060408@redhat.com> In IE when a window is small (horizontal scrollbar is displayed) click or keyboard input on various parts of UI makes search tables scroll to top. It prevents from selecting items in a table. This issue happens when using absolute positioned element with overflow style. It's a bug in IE. Two workarounds were added to make UI usable in IE. Adding position:relative; style to body element fixes the problem in search pages. It doesn't help in association dialogs though. The bug doesn't occur when some child element has focus. It's possible to set focus to first visible checkbox while scrolling down but user experience is very bad. Better solution seems to scroll back when IE scrolls to top on mousedown. That way mouse click event happens on the target element and it can gain focus and therefore be selected. Some glitches still remains but is usable. https://fedorahosted.org/freeipa/ticket/2835 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0174-Fix-autoscroll-to-top-in-tables-in-IE.patch Type: text/x-patch Size: 2481 bytes Desc: not available URL: From rcritten at redhat.com Mon Jul 23 20:03:27 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Jul 2012 16:03:27 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <500574F7.6060300@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> Message-ID: <500DAE0F.8080906@redhat.com> Rob Crittenden wrote: > Andrew Wnuk wrote: >> On 07/16/2012 01:35 PM, Rob Crittenden wrote: >>> Nalin Dahyabhai wrote: >>>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>>> Use the new certmonger capability to be able to renew the dogtag >>>>> subsystem certificates (audit, OCSP, etc). >>>> >>>> Are the copies of the certificates in the pki-ca CS.cfg file being >>>> updated elsewhere? Or is it not turning out to be a problem if they >>>> aren't? >>> >>> I didn't test validating OCSP signatures but the audit subsystem >>> seemed fine (it complained wildly when I had the wrong trust in the >>> NSS db). >>> >>> Andrew, do I need to update CS.cfg as well? >>> >> Yes, you may need update CS.cfg too. > > Ok, added a bit to update CS.cfg with the new certificate. This should fix some SELinux issues preventing certmonger from monitoring the dogtag certificate database in /var/lib/pki-ca/alias. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1033-3-renewal.patch Type: text/x-diff Size: 44634 bytes Desc: not available URL: From rcritten at redhat.com Mon Jul 23 20:34:47 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Mon, 23 Jul 2012 16:34:47 -0400 Subject: [Freeipa-devel] [PATCH] 0056 Framework for admin/install tools, with ipa-ldap-updater In-Reply-To: <50099643.7060908@redhat.com> References: <4FCCCC8B.1030903@redhat.com> <4FDF38AC.1000508@redhat.com> <4FE1F73E.6000003@redhat.com> <4FE86109.8030900@redhat.com> <4FE8619B.5080608@redhat.com> <4FEE1E0B.8030804@redhat.com> <4FF2AE6C.6010800@redhat.com> <5005CDF7.2060205@redhat.com> <5006A15C.3040406@redhat.com> <50084B80.6000708@redhat.com> <50093A36.2010409@redhat.com> <50098049.6070809@redhat.com> <50098894.6030704@redhat.com> <50099643.7060908@redhat.com> Message-ID: <500DB567.3040005@redhat.com> John Dennis wrote: > On 07/20/2012 12:34 PM, Petr Viktorin wrote: >> On 07/20/2012 05:59 PM, John Dennis wrote: >>>>> A fair amount of the code in the framework is doing this now, but the >>>>> install code was never cleaned up. That was left for another day, I >>>>> guess that day is here. >>>> >>>> Updated. I also added the necessary lint exception. >>>> I'm curious as to why it works that way, though. Normally, to put >>>> methods in a class, you would use a base class/mixin. Why this dynamic >>>> magic? >>> >>> Good question. I don't like dynamic magic either, it makes static code >>> reading difficult and diminishes the value of static code >>> analysis/checking. Jason who originally wrote the framework used dynamic >>> magic a fair amount, including the initialization of the logging methods >>> in the plugins. When I wrote the log manager I kept Jason's existing >>> strategy (how many things do change at once?). In hindsight a mixin >>> would have been a better solution, something we should probably fix down >>> the road. >> >> Thinking more about this, composition would probably be cleaner than >> inheritance here (i.e. using self.logger.warn(...) instead of mixing the >> functionality into the class itself). >> Well, one day the time to fix it will come. >> >>> Everything looks good, although there might be one minor thing that >>> needs fixing. Shouldn't the logging setup be done first? That way any >>> code that executes has the ability to emit a message. For example what >>> if validate_options() wants to emit a message? >>> >> >> That's a good question. >> If we set up logging before validating the options, we'll log all >> invocations, including those with misspelled option names and forgotten >> "sudo"s. Do we want that? >> > > Sure, hard to say. Why don't we leave it the way it is now and down the > road if it shows up as an issue we'll tweak it then. > > ACK. > pushed to master From jdennis at redhat.com Mon Jul 23 20:46:30 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 23 Jul 2012 16:46:30 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500D272E.6040309@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> Message-ID: <500DB826.8090908@redhat.com> On 07/23/2012 06:27 AM, Petr Viktorin wrote: > On 07/20/2012 07:14 PM, John Dennis wrote: >> On 07/20/2012 12:28 PM, Petr Viktorin wrote: >>> On 07/20/2012 05:39 PM, John Dennis wrote: >>>> Great I agree with everything you said. >>>> >>>> I'm happy to have the file list be derived from the directory contents. >>>> Are you planning on doing that in another patch? >>> >>> Yes, I want to do it in a new patch. >>> It's a bit more complicated than it looks: creating a new translation >>> will work differently than just adding it to LINGUAS and running >>> create-po. The ticket is for beta 2 so I'd rather not start a new round >>> of reviews. >> >> Fine with me to do that in another patch. >> >> As for create-po, I think that's also holdover from pre-Transifex days. >> With Transifex I'd don't ever see a need to create an empty po file. Do >> you? Maybe we should just nuke the po creation in the Makefile. > > As a translator (for another project), I don't like Transifex and prefer > to send good old Git pull requests. I understand a "traditional" > workflow is hard to coordinate with others that use Transifex, but still > I'd hate it if we became dependent on Tx. > > [...] >>>> But... I do have one final issue/question. I missed this in the first >>>> review. po_files is now dependent on update-pot instead of the pot file. >>>> We had decided that we were only going to regenerate the pot file on >>>> demand at specific times. Won't this dependency change cause the pot >>>> file to be updated frequently? (I realize only in the local tree). Note >>>> that when we run the validations we generate a temporary pot file from >>>> the current contents of the tree specifically to avoid overwriting the >>>> pot file. >>> >>> Are the po files updated more often? I don't really see a reason to >>> merge the po files with an old pot. >> >> What merge are you referring to? The only merge I'm aware of at the >> moment is during validation, but that merge is done from a temporary >> updated pot file that is current with the tree. > > I'm referring to a manual merge-po. > po_files are only rebuilt from merge-po, which merges the po files with > the pot and adds all the missing translations and line numbers. This is > not needed with Transifex workflow, as Tx should do this internally when > a pot is pushed to it. > >> Any other merging is done by Transifex at the time we pull a po file. >> >> The frequency of po update doesn't seem relevant, what is your concern >> in this regard? > > Is there another cause for the po_files to get rebuilt? Using the TX model there is never a reason to build po files. We just pull them from TX. > >>>> I suppose having a conversation about when the pot file gets updated is >>>> a good one to have, we don't do it often enough IMHO. But I'm not sure >>>> it's correct to modify a file under SCM control if it wasn't >>>> intentional. >>> >>> How is Transifex set up here? If it automatically picks up changes when >>> the pot file is modified, then we should back up the translations before >>> changing the pot, so we can't do it automatically. >>> Another wart is the line number cruft in the pot file -- any time it's >>> updated we'll get a huge diff, so it makes sense to update sparingly. >> >> Transifex gives you two options for your pot file, either you tell TX >> the location of your pot file in a public SCM and it watches for updates >> and automatically pulls it when it changes in SCM -or- you manually push >> the pot file to TX. >> >> We've been using the "watch the pot file in git" option. Thus whenever >> we commit a new version of the pot file all developers and TX get it >> simultaneously (well sort of). >> >> If we do the manual push method the maintainer has to *both* commit to >> git *and* push to TX, so the former seems less error prone and more >> automated. > > Well, if the pot file is not in the repo, the maintainer only has to > push it to Tx (after building it of course, but that needs to be done > anyway). > >> The idea was we would have a string freeze prior to release and/or >> periodic intervals during branch development to update the pot. But we >> haven't been good about hitting these. However, note a manual push >> suffers from the same "somebody has to do it at the right moment" problem. > > Is this idea documented anywhere? It's hard to do a string freeze if > it's not enforced automatically, let alone if people don't know there > should be one. It was discussed in the developer conference calls. > >>> If Transifex is not wired to the pot, we could even go as far as >>> removing it from SCM entirely -- it's entirely generated, and rebuilding >>> it takes less than a second. >>> We'd just have to update Transifex manually. >> >> It currently is wired to the pot. You make a valid point about currently >> not needing to maintain the pot in SCM. When we first set up >> translations we weren't using TX so having the pot file in SCM was a >> necessity. >> >> Personally I don't trust TX's data storage and I think there is value in >> having each pot we push to TX be recoverable from our SCM. When things >> blow up (and they do) it's really nice to be able to reassemble the >> pieces or at lease follow the trail of how things changed. In the past >> I've had to answer questions like "How the heck did this string get into >> this po file?" Such questions can only be answered if we have the pot >> file we gave the translator. TX doesn't maintain it so we have to (or at >> least I think there is value in it). > > Yes, Transifex only keeps the last pushed pot file. > However, I don't think the git repo is a good place to store information > that's missing from Transifex. > > To really keep a good history, we'd also need to pull the po files from > Transifex much more often. I guess we could do that in the main repo, > but I don't think the other devs would be too happy with daily/weekly > automated commits reflecting the current state of the translations. > > Another option is to keep the history either in a separate branch, or in > a separate repository (hosted for example on fedorapeople). In that case > we can include pot snapshots, full po files as they're pulled from Tx, > or any other info we want, and update it as frequently as needed. > >> Perhaps you can read between the lines and detect I don't view TX as the >> epitome of stability and robustness. It's still young and they are still >> adding features and changing how it works (kinda like IPA :-) > > >>> Oh, one thing I'll ask about: the Makefile is not automatically rebuilt >>> when I change Makefile.in, so every time I run an autogen invocation I >>> found in the main Makefile. Is there an easier way to do this? >>> >> >> Ugh ... autotools :-( >> >> The Makefile is generated via configure and I thought autoregen had the >> smarts to detect the dependencies. I too noticed I had to manually >> re-run configure. Are we running autoregen? If so and it's not working I >> don't have an answer, sorry. > > install/po/Makefile is generated by this line in the main Makefile: > > cd install; if [ ! -e Makefile ]; then ../autogen.sh --prefix=/usr > --sysconfdir=/etc --localstatedir=/var --libdir=$(LIBDIR); fi > > I'm not very familiar with autotools but I don't think we're using them > correctly :) I wouldn't be surprised if we're not using autotools correctly. I'm not invested enough in this particular problem to track it down and fix it, I've got bigger fish to fry :-) The only thing holding up the ACK is the question of why po-files now has update_pot as a dependency. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Mon Jul 23 23:12:40 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 23 Jul 2012 19:12:40 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500D272E.6040309@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> Message-ID: <500DDA68.4090003@redhat.com> On 07/23/2012 06:27 AM, Petr Viktorin wrote: > As a translator (for another project), I don't like Transifex and prefer > to send good old Git pull requests. I understand a "traditional" > workflow is hard to coordinate with others that use Transifex, but still > I'd hate it if we became dependent on Tx. For better or worse we are dependent on TX (Transifex). Fedora has adopted TX as it's translation tool, RHEL's translation tools integrate with TX (as well as other translation portals). And SSSD and IPA have made a a commitment to TX based on the direction of Fedora and RHEL. Given that we've adopted TX I don't see the value in maintaining tools that support both TX and non-TX workflows. I'd rather see us delete the non-TX elements. If we have just one workflow it's easier to understand and maintain the code. If we ever decide we need to go back to a non-TX workflow we can always retrieve the deleted code from git. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From edewata at redhat.com Tue Jul 24 03:24:10 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 23 Jul 2012 22:24:10 -0500 Subject: [Freeipa-devel] [PATCH] 173 IDs and names for dialogs In-Reply-To: <5008069D.7070202@redhat.com> References: <5008069D.7070202@redhat.com> Message-ID: <500E155A.2020304@redhat.com> On 7/19/2012 8:07 AM, Petr Vobornik wrote: > It's hard to detect if or which type of dialog is displayed because not > all dialogs have IDs. > > On dialog open, it's id or name (if id is not set) is used for > containing element id. Many of dialog types were missing id or name so > name was added to each dialog type. > > In HTML, element's id should be unique. Our framework allows opening two > dialogs with the same id. It may lead to state where getElementById > method may have unpredictable behavior. Therefore attribute 'data-name' > with dialog's name was added to dialog's containing element. Automation > framework can search more reliable by using this attribute instead of id. > > https://fedorahosted.org/freeipa/ticket/2853 ACK. If possible we should replace all element ID's with paths. So the element name itself is not unique, but the element can still be uniquely identified by following a path from the top to the element itself. The path could be a combination of element type, class, or name, for example: div[name=container] .entity[name=user] .facet[name=search] The automation tool should use the path instead of ID. -- Endi S. Dewata From edewata at redhat.com Tue Jul 24 03:25:50 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 23 Jul 2012 22:25:50 -0500 Subject: [Freeipa-devel] [PATCH] 174 Fix autoscroll to top in tables in IE In-Reply-To: <500D5A1F.7060408@redhat.com> References: <500D5A1F.7060408@redhat.com> Message-ID: <500E15BE.1010702@redhat.com> On 7/23/2012 9:05 AM, Petr Vobornik wrote: > In IE when a window is small (horizontal scrollbar is displayed) click > or keyboard input on various parts of UI makes search tables scroll to > top. It prevents from selecting items in a table. This issue happens > when using absolute positioned element with overflow style. It's a bug > in IE. > > Two workarounds were added to make UI usable in IE. > Adding position:relative; style to body element fixes the problem in > search pages. It doesn't help in association dialogs though. > The bug doesn't occur when some child element has focus. It's possible > to set focus to first visible checkbox while scrolling down but user > experience is very bad. Better solution seems to scroll back when IE > scrolls to top on mousedown. That way mouse click event happens on the > target element and it can gain focus and therefore be selected. Some > glitches still remains but is usable. > > https://fedorahosted.org/freeipa/ticket/2835 ACK. Separate issue, I noticed that on IE the background of the dialog box title is black. On Firefox it's green. Same thing for the Available/Prospective header in the association dialog. -- Endi S. Dewata From pviktori at redhat.com Tue Jul 24 08:17:19 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 10:17:19 +0200 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500DB826.8090908@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> <500DB826.8090908@redhat.com> Message-ID: <500E5A0F.5000000@redhat.com> On 07/23/2012 10:46 PM, John Dennis wrote: [...] > > The only thing holding up the ACK is the question of why po-files now > has update_pot as a dependency. > If files simply depend on $(DOMAIN).pot, then they are considered up-to-date even after they're changed (e.g. with strip-po). They need to depend on a rule that always runs so that they get merged. There's another alternative to achieve this: adding them to .PHONY. The attached version does that, perhaps it's cleaner. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0066-03-Arrange-stripping-.po-files.patch Type: text/x-patch Size: 7718 bytes Desc: not available URL: From abokovoy at redhat.com Tue Jul 24 10:01:47 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 24 Jul 2012 13:01:47 +0300 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts Message-ID: <20120724100147.GP1052@redhat.com> Hi, There are two problems in task naming in LDAP updates: 1. Randomness may be scarce in virtual machines 2. Random number is added to the time value rounded to a second The second issue leads to values that may repeat themselves as time only grows and random number is non-negative as well, so t2+r2 can be equal to t1+t2 generated earlier. Since task name is a DN, there is no strict requirement to use an integer value. Instead, we can take time and attribute name. To get reasonable 'randomness' these values are then hashed with sha1 and use the resulting string as task name. SHA1 may technically be an overkill here as we could simply use indextask_$date_$attribute where $date is a value of time.time() but SHA1 gives a resonable 'randomness' into the string. I was hit by this issue today, see https://fedorahosted.org/freeipa/ticket/2942 -- / Alexander Bokovoy -------------- next part -------------- >From 60ec577e92ac9d2eabb78b27a877fa2dbd96741d Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Tue, 24 Jul 2012 12:07:23 +0300 Subject: [PATCH 3/3] Rework task naming in LDAP updates to avoid conflicting names in certain cases There are two problems in task naming in LDAP updates: 1. Randomness may be scarce in virtual machines 2. Random number is added to the time value rounded to a second The second issue leads to values that may repeat themselves as time only grows and random number is non-negative as well, so t2+r2 can be equal to t1+t2 generated earlier. Since task name is a DN, there is no strict requirement to use an integer value. Instead, we can take time and attribute name. To get reasonable 'randomness' these values are then hashed with sha1 and use the resulting string as task name. https://fedorahosted.org/freeipa/ticket/2942 --- ipaserver/install/ldapupdate.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/ipaserver/install/ldapupdate.py b/ipaserver/install/ldapupdate.py index c64139889d9f84866ac0cd358ed3a3a7d95af7dc..a28ffb8c8fa86d4989db282ea60721db6d0a7c9a 100644 --- a/ipaserver/install/ldapupdate.py +++ b/ipaserver/install/ldapupdate.py @@ -29,9 +29,11 @@ from ipaserver.install import installutils from ipaserver.install import service from ipaserver import ipaldap from ipapython import entity, ipautil +from ipapython.compat import sha1 from ipalib import util from ipalib import errors from ipalib import api +from ipalib.dn import DN import ldap from ldap.dn import escape_dn_chars from ipapython.ipa_log_manager import * @@ -328,16 +330,14 @@ class LDAPUpdate: if self.live_run: time.sleep(5) - r = random.SystemRandom() + tasktime = str(time.time()) + self.sub_dict['IDXHASH'] = sha1('_'.join( + [tasktime, attribute, tasktime, attribute ])).hexdigest() - # Refresh the time to make uniqueness more probable. Add on some - # randomness for good measure. - self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) + cn = self._template_str("indextask_$IDXHASH") + dn = DN(('cn', cn), ('cn', 'index'), ('cn', 'tasks'), ('cn', 'config')) - cn = self._template_str("indextask_$TIME") - dn = "cn=%s, cn=index, cn=tasks, cn=config" % cn - - e = ipaldap.Entry(dn) + e = ipaldap.Entry(str(dn)) e.setValues('objectClass', ['top', 'extensibleObject']) e.setValue('cn', cn) @@ -345,7 +345,7 @@ class LDAPUpdate: e.setValues('nsIndexAttribute', attribute) root_logger.info("Creating task to index attribute: %s", attribute) - root_logger.debug("Task id: %s", dn) + root_logger.debug("Task id: %s", str(dn)) if self.live_run: self.conn.addEntry(e) -- 1.7.10.4 From pviktori at redhat.com Tue Jul 24 10:39:10 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 12:39:10 +0200 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500DDA68.4090003@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> <500DDA68.4090003@redhat.com> Message-ID: <500E7B4E.60300@redhat.com> On 07/24/2012 01:12 AM, John Dennis wrote: > On 07/23/2012 06:27 AM, Petr Viktorin wrote: >> As a translator (for another project), I don't like Transifex and prefer >> to send good old Git pull requests. I understand a "traditional" >> workflow is hard to coordinate with others that use Transifex, but still >> I'd hate it if we became dependent on Tx. > > For better or worse we are dependent on TX (Transifex). Fedora has > adopted TX as it's translation tool, RHEL's translation tools integrate > with TX (as well as other translation portals). And SSSD and IPA have > made a a commitment to TX based on the direction of Fedora and RHEL. > > Given that we've adopted TX I don't see the value in maintaining tools > that support both TX and non-TX workflows. I'd rather see us delete the > non-TX elements. If we have just one workflow it's easier to understand > and maintain the code. If we ever decide we need to go back to a non-TX > workflow we can always retrieve the deleted code from git. > This means you have to be a member of a Fedora translation team to translate. It makes it harder for people to fork the project. A workflow with a mandatory central repository makes it impossible to experiment locally. I'm all for having a standard way to receive contributions, but limiting how people can create those contributions isn't good. I'm all for deleting unused code, but here I think it would be a bad move. -- Petr? From pviktori at redhat.com Tue Jul 24 11:03:50 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 13:03:50 +0200 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <500DAE0F.8080906@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> Message-ID: <500E8116.1020303@redhat.com> On 07/23/2012 10:03 PM, Rob Crittenden wrote: > Rob Crittenden wrote: >> Andrew Wnuk wrote: >>> On 07/16/2012 01:35 PM, Rob Crittenden wrote: >>>> Nalin Dahyabhai wrote: >>>>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>>>> Use the new certmonger capability to be able to renew the dogtag >>>>>> subsystem certificates (audit, OCSP, etc). >>>>> >>>>> Are the copies of the certificates in the pki-ca CS.cfg file being >>>>> updated elsewhere? Or is it not turning out to be a problem if they >>>>> aren't? >>>> >>>> I didn't test validating OCSP signatures but the audit subsystem >>>> seemed fine (it complained wildly when I had the wrong trust in the >>>> NSS db). >>>> >>>> Andrew, do I need to update CS.cfg as well? >>>> >>> Yes, you may need update CS.cfg too. >> >> Ok, added a bit to update CS.cfg with the new certificate. > > This should fix some SELinux issues preventing certmonger from > monitoring the dogtag certificate database in /var/lib/pki-ca/alias. > > rob I don't know enough about dogtag/certmonger to comment on the functionality, but there are minor issues I can find. Attaching a patch to fix them. `make rpms` fails: rpmbuild --define "_topdir /rpmbuild" -ba freeipa.spec error: %changelog not in descending chronological order make: *** [rpms] Error 1 `git am` complains: Applying: Use certmonger to renew CA subsystem certificates /home/pviktori/freeipa/.git/rebase-apply/patch:576: new blank line at EOF. + /home/pviktori/freeipa/.git/rebase-apply/patch:645: new blank line at EOF. + warning: 2 lines add whitespace errors. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: fixes-for-rcrit-1033-03.patch Type: text/x-patch Size: 1961 bytes Desc: not available URL: From pviktori at redhat.com Tue Jul 24 11:35:33 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 13:35:33 +0200 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <20120724100147.GP1052@redhat.com> References: <20120724100147.GP1052@redhat.com> Message-ID: <500E8885.4020703@redhat.com> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: > Hi, > > There are two problems in task naming in LDAP updates: > > 1. Randomness may be scarce in virtual machines > 2. Random number is added to the time value rounded to a second > > The second issue leads to values that may repeat themselves as time > only grows and random number is non-negative as well, so > t2+r2 can be equal to t1+t2 generated earlier. > > Since task name is a DN, there is no strict requirement to use an > integer value. Instead, we can take time and attribute name. To get > reasonable 'randomness' these values are then hashed with sha1 and use > the resulting string as task name. > > SHA1 may technically be an overkill here as we could simply use > > indextask_$date_$attribute > > where $date is a value of time.time() but SHA1 gives a resonable > 'randomness' into the string. What kind of randomness do you mean? SHA1 is deterministic, it doesn't add any randomness at all. It just obscures what's really happening. Same with repeating [tasktime, attribute] two times. > - root_logger.debug("Task id: %s", dn) > + root_logger.debug("Task id: %s", str(dn)) This change is unnecessary; the "%s" means "convert to str". > I was hit by this issue today, see > https://fedorahosted.org/freeipa/ticket/2942 > -- Petr? From abokovoy at redhat.com Tue Jul 24 12:06:54 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 24 Jul 2012 15:06:54 +0300 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <500E8885.4020703@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> Message-ID: <20120724120654.GQ1052@redhat.com> On Tue, 24 Jul 2012, Petr Viktorin wrote: >On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>Hi, >> >>There are two problems in task naming in LDAP updates: >> >>1. Randomness may be scarce in virtual machines >>2. Random number is added to the time value rounded to a second >> >>The second issue leads to values that may repeat themselves as time >>only grows and random number is non-negative as well, so >>t2+r2 can be equal to t1+t2 generated earlier. >> >>Since task name is a DN, there is no strict requirement to use an >>integer value. Instead, we can take time and attribute name. To get >>reasonable 'randomness' these values are then hashed with sha1 and use >>the resulting string as task name. >> >>SHA1 may technically be an overkill here as we could simply use >> >> indextask_$date_$attribute >> >>where $date is a value of time.time() but SHA1 gives a resonable >>'randomness' into the string. > >What kind of randomness do you mean? SHA1 is deterministic, it >doesn't add any randomness at all. It just obscures what's really >happening. Hence using quotes to describe it. We don't need randomness in the task names, we need something that avoids collisions. An issue here is in time.time() -- it may give us sub-second resolution if underlying OS supports it, it may not. Having a second-level resolution is not enough, especially on fast machines, so we can't simply use int(times.time()) as it was in the original version. indextask_$date_$attribute has this issue that we don't have enough guarantee for $date (time.time()) to be unique in sufficiently tight conditions, thus use of SHA-1 to generate something that has better chances to avoid collisions than $data_$attribute. >Same with repeating [tasktime, attribute] two times. This can be reduced as SHA-1 output does not depend on size of the input message. -- / Alexander Bokovoy From pviktori at redhat.com Tue Jul 24 12:23:54 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 14:23:54 +0200 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <20120724120654.GQ1052@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> Message-ID: <500E93DA.5020208@redhat.com> On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: > On Tue, 24 Jul 2012, Petr Viktorin wrote: >> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>> Hi, >>> >>> There are two problems in task naming in LDAP updates: >>> >>> 1. Randomness may be scarce in virtual machines >>> 2. Random number is added to the time value rounded to a second >>> >>> The second issue leads to values that may repeat themselves as time >>> only grows and random number is non-negative as well, so >>> t2+r2 can be equal to t1+t2 generated earlier. >>> >>> Since task name is a DN, there is no strict requirement to use an >>> integer value. Instead, we can take time and attribute name. To get >>> reasonable 'randomness' these values are then hashed with sha1 and use >>> the resulting string as task name. >>> >>> SHA1 may technically be an overkill here as we could simply use >>> >>> indextask_$date_$attribute >>> >>> where $date is a value of time.time() but SHA1 gives a resonable >>> 'randomness' into the string. >> >> What kind of randomness do you mean? SHA1 is deterministic, it doesn't >> add any randomness at all. It just obscures what's really happening. > Hence using quotes to describe it. We don't need randomness in the task > names, we need something that avoids collisions. > > An issue here is in time.time() -- it may give us sub-second resolution > if underlying OS supports it, it may not. Having a second-level > resolution is not enough, especially on fast machines, so we can't > simply use int(times.time()) as it was in the original version. > > indextask_$date_$attribute has this issue that we don't have enough > guarantee for $date (time.time()) to be unique in sufficiently tight > conditions, thus use of SHA-1 to generate something that has better > chances to avoid collisions than $data_$attribute. My point is that if "indextask_$date_$attribute" is not unique, neither is SHA1("indextask_$date_$attribute"). Hashing has no effect on the chance of collisions. You could use Python's pseudorandom number generator (random.randint) instead of random.SystemRandom. It's not cryptographically secure but it's enough to avoid collisions, and it doesn't use up system entropy (except for initial seeding, through `import random`). "indextask_$date_$attribute_$pseudorandomvalue" should be unique enough. >> Same with repeating [tasktime, attribute] two times. > This can be reduced as SHA-1 output does not depend on size of the > input message. -- Petr? From abokovoy at redhat.com Tue Jul 24 12:49:19 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 24 Jul 2012 15:49:19 +0300 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <500E93DA.5020208@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> Message-ID: <20120724124919.GR1052@redhat.com> On Tue, 24 Jul 2012, Petr Viktorin wrote: >On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>Hi, >>>> >>>>There are two problems in task naming in LDAP updates: >>>> >>>>1. Randomness may be scarce in virtual machines >>>>2. Random number is added to the time value rounded to a second >>>> >>>>The second issue leads to values that may repeat themselves as time >>>>only grows and random number is non-negative as well, so >>>>t2+r2 can be equal to t1+t2 generated earlier. >>>> >>>>Since task name is a DN, there is no strict requirement to use an >>>>integer value. Instead, we can take time and attribute name. To get >>>>reasonable 'randomness' these values are then hashed with sha1 and use >>>>the resulting string as task name. >>>> >>>>SHA1 may technically be an overkill here as we could simply use >>>> >>>> indextask_$date_$attribute >>>> >>>>where $date is a value of time.time() but SHA1 gives a resonable >>>>'randomness' into the string. >>> >>>What kind of randomness do you mean? SHA1 is deterministic, it doesn't >>>add any randomness at all. It just obscures what's really happening. >>Hence using quotes to describe it. We don't need randomness in the task >>names, we need something that avoids collisions. >> >>An issue here is in time.time() -- it may give us sub-second resolution >>if underlying OS supports it, it may not. Having a second-level >>resolution is not enough, especially on fast machines, so we can't >>simply use int(times.time()) as it was in the original version. >> >>indextask_$date_$attribute has this issue that we don't have enough >>guarantee for $date (time.time()) to be unique in sufficiently tight >>conditions, thus use of SHA-1 to generate something that has better >>chances to avoid collisions than $data_$attribute. > >My point is that if "indextask_$date_$attribute" is not unique, >neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >on the chance of collisions. > >You could use Python's pseudorandom number generator (random.randint) >instead of random.SystemRandom. It's not cryptographically secure but >it's enough to avoid collisions, and it doesn't use up system entropy >(except for initial seeding, through `import random`). >"indextask_$date_$attribute_$pseudorandomvalue" should be unique enough. Using random here is really bad. What we ideally need is a method to increment sequential calls for the same attribute. We use seconds to differentiate between all tasks but that is not really required, tasks that were processed will be removed. -- / Alexander Bokovoy From pviktori at redhat.com Tue Jul 24 13:47:59 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 15:47:59 +0200 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <20120724124919.GR1052@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> Message-ID: <500EA78F.1010607@redhat.com> On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: > On Tue, 24 Jul 2012, Petr Viktorin wrote: >> On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>> Hi, >>>>> >>>>> There are two problems in task naming in LDAP updates: >>>>> >>>>> 1. Randomness may be scarce in virtual machines >>>>> 2. Random number is added to the time value rounded to a second >>>>> >>>>> The second issue leads to values that may repeat themselves as time >>>>> only grows and random number is non-negative as well, so >>>>> t2+r2 can be equal to t1+t2 generated earlier. >>>>> >>>>> Since task name is a DN, there is no strict requirement to use an >>>>> integer value. Instead, we can take time and attribute name. To get >>>>> reasonable 'randomness' these values are then hashed with sha1 and use >>>>> the resulting string as task name. >>>>> >>>>> SHA1 may technically be an overkill here as we could simply use >>>>> >>>>> indextask_$date_$attribute >>>>> >>>>> where $date is a value of time.time() but SHA1 gives a resonable >>>>> 'randomness' into the string. >>>> >>>> What kind of randomness do you mean? SHA1 is deterministic, it doesn't >>>> add any randomness at all. It just obscures what's really happening. >>> Hence using quotes to describe it. We don't need randomness in the task >>> names, we need something that avoids collisions. >>> >>> An issue here is in time.time() -- it may give us sub-second resolution >>> if underlying OS supports it, it may not. Having a second-level >>> resolution is not enough, especially on fast machines, so we can't >>> simply use int(times.time()) as it was in the original version. >>> >>> indextask_$date_$attribute has this issue that we don't have enough >>> guarantee for $date (time.time()) to be unique in sufficiently tight >>> conditions, thus use of SHA-1 to generate something that has better >>> chances to avoid collisions than $data_$attribute. >> >> My point is that if "indextask_$date_$attribute" is not unique, >> neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >> on the chance of collisions. >> >> You could use Python's pseudorandom number generator (random.randint) >> instead of random.SystemRandom. It's not cryptographically secure but >> it's enough to avoid collisions, and it doesn't use up system entropy >> (except for initial seeding, through `import random`). >> "indextask_$date_$attribute_$pseudorandomvalue" should be unique enough. > Using random here is really bad. > What we ideally need is a method to increment sequential calls for > the same attribute.We use seconds to differentiate > between all tasks but that is not really required, tasks that were > processed will be removed. > Or maybe use $date_$attribute and just warn (ignore the error) if there's a duplicate -- if a reindex task for the same attribute already exists from the same second, do we really need to start a new one? -- Petr? From rcritten at redhat.com Tue Jul 24 13:55:33 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 24 Jul 2012 09:55:33 -0400 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <500EA78F.1010607@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> <500EA78F.1010607@redhat.com> Message-ID: <500EA955.7090804@redhat.com> Petr Viktorin wrote: > On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: >> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>> On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>>> Hi, >>>>>> >>>>>> There are two problems in task naming in LDAP updates: >>>>>> >>>>>> 1. Randomness may be scarce in virtual machines >>>>>> 2. Random number is added to the time value rounded to a second >>>>>> >>>>>> The second issue leads to values that may repeat themselves as time >>>>>> only grows and random number is non-negative as well, so >>>>>> t2+r2 can be equal to t1+t2 generated earlier. >>>>>> >>>>>> Since task name is a DN, there is no strict requirement to use an >>>>>> integer value. Instead, we can take time and attribute name. To get >>>>>> reasonable 'randomness' these values are then hashed with sha1 and >>>>>> use >>>>>> the resulting string as task name. >>>>>> >>>>>> SHA1 may technically be an overkill here as we could simply use >>>>>> >>>>>> indextask_$date_$attribute >>>>>> >>>>>> where $date is a value of time.time() but SHA1 gives a resonable >>>>>> 'randomness' into the string. >>>>> >>>>> What kind of randomness do you mean? SHA1 is deterministic, it doesn't >>>>> add any randomness at all. It just obscures what's really happening. >>>> Hence using quotes to describe it. We don't need randomness in the task >>>> names, we need something that avoids collisions. >>>> >>>> An issue here is in time.time() -- it may give us sub-second resolution >>>> if underlying OS supports it, it may not. Having a second-level >>>> resolution is not enough, especially on fast machines, so we can't >>>> simply use int(times.time()) as it was in the original version. >>>> >>>> indextask_$date_$attribute has this issue that we don't have enough >>>> guarantee for $date (time.time()) to be unique in sufficiently tight >>>> conditions, thus use of SHA-1 to generate something that has better >>>> chances to avoid collisions than $data_$attribute. >>> >>> My point is that if "indextask_$date_$attribute" is not unique, >>> neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >>> on the chance of collisions. >>> >>> You could use Python's pseudorandom number generator (random.randint) >>> instead of random.SystemRandom. It's not cryptographically secure but >>> it's enough to avoid collisions, and it doesn't use up system entropy >>> (except for initial seeding, through `import random`). >>> "indextask_$date_$attribute_$pseudorandomvalue" should be unique enough. >> Using random here is really bad. >> What we ideally need is a method to increment sequential calls for >> the same attribute.We use seconds to differentiate >> between all tasks but that is not really required, tasks that were >> processed will be removed. >> > > Or maybe use $date_$attribute and just warn (ignore the error) if > there's a duplicate -- if a reindex task for the same attribute already > exists from the same second, do we really need to start a new one? > That is true. We can generate a UUID, I think that is probably a better/safer thing to use overall. rob From pviktori at redhat.com Tue Jul 24 14:18:48 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 16:18:48 +0200 Subject: [Freeipa-devel] [PATCH] 0071 Create /etc/sysconfig/network if it doesn't exist Message-ID: <500EAEC8.4010006@redhat.com> When --hostname is given, client installation tries to write to /etc/sysconfig/network, which is not guaranteed to exist. Create it if it doesn't exist and we need to write to it. https://fedorahosted.org/freeipa/ticket/2840 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0071-Create-etc-sysconfig-network-if-it-doesn-t-exist.patch Type: text/x-patch Size: 3130 bytes Desc: not available URL: From abokovoy at redhat.com Tue Jul 24 14:50:37 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 24 Jul 2012 17:50:37 +0300 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <500EA955.7090804@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> <500EA78F.1010607@redhat.com> <500EA955.7090804@redhat.com> Message-ID: <20120724145037.GS1052@redhat.com> On Tue, 24 Jul 2012, Rob Crittenden wrote: >Petr Viktorin wrote: >>On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: >>>On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>>>>On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>>>>Hi, >>>>>>> >>>>>>>There are two problems in task naming in LDAP updates: >>>>>>> >>>>>>>1. Randomness may be scarce in virtual machines >>>>>>>2. Random number is added to the time value rounded to a second >>>>>>> >>>>>>>The second issue leads to values that may repeat themselves as time >>>>>>>only grows and random number is non-negative as well, so >>>>>>>t2+r2 can be equal to t1+t2 generated earlier. >>>>>>> >>>>>>>Since task name is a DN, there is no strict requirement to use an >>>>>>>integer value. Instead, we can take time and attribute name. To get >>>>>>>reasonable 'randomness' these values are then hashed with sha1 and >>>>>>>use >>>>>>>the resulting string as task name. >>>>>>> >>>>>>>SHA1 may technically be an overkill here as we could simply use >>>>>>> >>>>>>> indextask_$date_$attribute >>>>>>> >>>>>>>where $date is a value of time.time() but SHA1 gives a resonable >>>>>>>'randomness' into the string. >>>>>> >>>>>>What kind of randomness do you mean? SHA1 is deterministic, it doesn't >>>>>>add any randomness at all. It just obscures what's really happening. >>>>>Hence using quotes to describe it. We don't need randomness in the task >>>>>names, we need something that avoids collisions. >>>>> >>>>>An issue here is in time.time() -- it may give us sub-second resolution >>>>>if underlying OS supports it, it may not. Having a second-level >>>>>resolution is not enough, especially on fast machines, so we can't >>>>>simply use int(times.time()) as it was in the original version. >>>>> >>>>>indextask_$date_$attribute has this issue that we don't have enough >>>>>guarantee for $date (time.time()) to be unique in sufficiently tight >>>>>conditions, thus use of SHA-1 to generate something that has better >>>>>chances to avoid collisions than $data_$attribute. >>>> >>>>My point is that if "indextask_$date_$attribute" is not unique, >>>>neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >>>>on the chance of collisions. >>>> >>>>You could use Python's pseudorandom number generator (random.randint) >>>>instead of random.SystemRandom. It's not cryptographically secure but >>>>it's enough to avoid collisions, and it doesn't use up system entropy >>>>(except for initial seeding, through `import random`). >>>>"indextask_$date_$attribute_$pseudorandomvalue" should be unique enough. >>>Using random here is really bad. >>>What we ideally need is a method to increment sequential calls for >>>the same attribute.We use seconds to differentiate >>>between all tasks but that is not really required, tasks that were >>>processed will be removed. >>> >> >>Or maybe use $date_$attribute and just warn (ignore the error) if >>there's a duplicate -- if a reindex task for the same attribute already >>exists from the same second, do we really need to start a new one? >> > >That is true. > >We can generate a UUID, I think that is probably a better/safer thing >to use overall. Updated patch attached. 2012-07-24T14:36:31Z INFO Creating task to index attribute: memberOf 2012-07-24T14:36:31Z DEBUG Task id: cn=indextask_memberOf_135624333918176170_14302,cn=index,cn=tasks,cn=config 2012-07-24T14:36:32Z INFO Indexing finished ... 2012-07-24T14:36:43Z INFO Creating task to index attribute: memberUser 2012-07-24T14:36:43Z DEBUG Task id: cn=indextask_memberUser_135624334038532670_14302,cn=index,cn=tasks,cn=config 2012-07-24T14:36:44Z INFO Indexing finished You can note that clock_seq does not change, it is because uuid.uuid1() uses nanosecond resolution and uuid_generate_time() if the latter is available. If we happen to ask for uuids within sub-nanosecond interval, clock_seq will be different then. I'm extracting only time and clock_seq instead of pasting full uuid value because uuid_generate_time() will leak out ethernet MAC address for 48-bit node. We don't need more bits here so I drop these 48 bits and avoid publishing the ethernet MAC address, even in logs. -- / Alexander Bokovoy -------------- next part -------------- >From e5262b5625e8e3b2deaf9228ca8a53dcbea90593 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Tue, 24 Jul 2012 12:07:23 +0300 Subject: [PATCH 3/3] Rework task naming in LDAP updates to avoid conflicting names in certain cases There are two problems in task naming in LDAP updates: 1. Randomness may be scarce in virtual machines 2. Random number is added to the time value rounded to a second The second issue leads to values that may repeat themselves as time only grows and random number is non-negative as well, so t2+r2 can be equal to t1+t2 generated earlier. Since task name is a DN, there is no strict requirement to use an integer value. Instead, we generate an UUID and use its 60-bit time, 14-bit sequential number, and attribute name. https://fedorahosted.org/freeipa/ticket/2942 --- ipaserver/install/ldapupdate.py | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/ipaserver/install/ldapupdate.py b/ipaserver/install/ldapupdate.py index c64139889d9f84866ac0cd358ed3a3a7d95af7dc..d021dc89ca90ea34050ac6f5246a4b895f4a7316 100644 --- a/ipaserver/install/ldapupdate.py +++ b/ipaserver/install/ldapupdate.py @@ -29,9 +29,11 @@ from ipaserver.install import installutils from ipaserver.install import service from ipaserver import ipaldap from ipapython import entity, ipautil +import uuid from ipalib import util from ipalib import errors from ipalib import api +from ipalib.dn import DN import ldap from ldap.dn import escape_dn_chars from ipapython.ipa_log_manager import * @@ -328,16 +330,15 @@ class LDAPUpdate: if self.live_run: time.sleep(5) - r = random.SystemRandom() + cn_uuid = uuid.uuid1() + self.sub_dict['TIME'] = cn_uuid.get_time() + self.sub_dict['SEQ'] = cn_uuid.get_clock_seq() + self.sub_dict['ATTR'] = attribute - # Refresh the time to make uniqueness more probable. Add on some - # randomness for good measure. - self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) + cn = self._template_str("indextask_${ATTR}_${TIME}_${SEQ}") + dn = DN(('cn', cn), ('cn', 'index'), ('cn', 'tasks'), ('cn', 'config')) - cn = self._template_str("indextask_$TIME") - dn = "cn=%s, cn=index, cn=tasks, cn=config" % cn - - e = ipaldap.Entry(dn) + e = ipaldap.Entry(str(dn)) e.setValues('objectClass', ['top', 'extensibleObject']) e.setValue('cn', cn) -- 1.7.10.4 From pviktori at redhat.com Tue Jul 24 16:28:21 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Tue, 24 Jul 2012 18:28:21 +0200 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <20120724145037.GS1052@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> <500EA78F.1010607@redhat.com> <500EA955.7090804@redhat.com> <20120724145037.GS1052@redhat.com> Message-ID: <500ECD25.9030804@redhat.com> On 07/24/2012 04:50 PM, Alexander Bokovoy wrote: > On Tue, 24 Jul 2012, Rob Crittenden wrote: >> Petr Viktorin wrote: >>> On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: >>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>> On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>>>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> There are two problems in task naming in LDAP updates: >>>>>>>> >>>>>>>> 1. Randomness may be scarce in virtual machines >>>>>>>> 2. Random number is added to the time value rounded to a second >>>>>>>> >>>>>>>> The second issue leads to values that may repeat themselves as time >>>>>>>> only grows and random number is non-negative as well, so >>>>>>>> t2+r2 can be equal to t1+t2 generated earlier. >>>>>>>> >>>>>>>> Since task name is a DN, there is no strict requirement to use an >>>>>>>> integer value. Instead, we can take time and attribute name. To >>>>>>>> get >>>>>>>> reasonable 'randomness' these values are then hashed with sha1 and >>>>>>>> use >>>>>>>> the resulting string as task name. >>>>>>>> >>>>>>>> SHA1 may technically be an overkill here as we could simply use >>>>>>>> >>>>>>>> indextask_$date_$attribute >>>>>>>> >>>>>>>> where $date is a value of time.time() but SHA1 gives a resonable >>>>>>>> 'randomness' into the string. >>>>>>> >>>>>>> What kind of randomness do you mean? SHA1 is deterministic, it >>>>>>> doesn't >>>>>>> add any randomness at all. It just obscures what's really happening. >>>>>> Hence using quotes to describe it. We don't need randomness in the >>>>>> task >>>>>> names, we need something that avoids collisions. >>>>>> >>>>>> An issue here is in time.time() -- it may give us sub-second >>>>>> resolution >>>>>> if underlying OS supports it, it may not. Having a second-level >>>>>> resolution is not enough, especially on fast machines, so we can't >>>>>> simply use int(times.time()) as it was in the original version. >>>>>> >>>>>> indextask_$date_$attribute has this issue that we don't have enough >>>>>> guarantee for $date (time.time()) to be unique in sufficiently tight >>>>>> conditions, thus use of SHA-1 to generate something that has better >>>>>> chances to avoid collisions than $data_$attribute. >>>>> >>>>> My point is that if "indextask_$date_$attribute" is not unique, >>>>> neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >>>>> on the chance of collisions. >>>>> >>>>> You could use Python's pseudorandom number generator (random.randint) >>>>> instead of random.SystemRandom. It's not cryptographically secure but >>>>> it's enough to avoid collisions, and it doesn't use up system entropy >>>>> (except for initial seeding, through `import random`). >>>>> "indextask_$date_$attribute_$pseudorandomvalue" should be unique >>>>> enough. >>>> Using random here is really bad. >>>> What we ideally need is a method to increment sequential calls for >>>> the same attribute.We use seconds to differentiate >>>> between all tasks but that is not really required, tasks that were >>>> processed will be removed. >>>> >>> >>> Or maybe use $date_$attribute and just warn (ignore the error) if >>> there's a duplicate -- if a reindex task for the same attribute already >>> exists from the same second, do we really need to start a new one? >>> >> >> That is true. >> >> We can generate a UUID, I think that is probably a better/safer thing >> to use overall. > Updated patch attached. > > 2012-07-24T14:36:31Z INFO Creating task to index attribute: memberOf > 2012-07-24T14:36:31Z DEBUG Task id: > cn=indextask_memberOf_135624333918176170_14302,cn=index,cn=tasks,cn=config > 2012-07-24T14:36:32Z INFO Indexing finished > ... > 2012-07-24T14:36:43Z INFO Creating task to index attribute: memberUser > 2012-07-24T14:36:43Z DEBUG Task id: > cn=indextask_memberUser_135624334038532670_14302,cn=index,cn=tasks,cn=config > > 2012-07-24T14:36:44Z INFO Indexing finished > > You can note that clock_seq does not change, it is because uuid.uuid1() > uses nanosecond resolution and uuid_generate_time() if the latter is > available. If we happen to ask for uuids within sub-nanosecond interval, > clock_seq will be different then. > > I'm extracting only time and clock_seq instead of pasting full uuid > value because uuid_generate_time() will leak out ethernet MAC address > for 48-bit node. We don't need more bits here so I drop these 48 bits > and avoid publishing the ethernet MAC address, even in logs. > > -- > / Alexander Bokovoy Yes, this approach will work. Just some technical comments: > + self.sub_dict['TIME'] = cn_uuid.get_time() > + self.sub_dict['SEQ'] = cn_uuid.get_clock_seq() Use the attributes, `cn_uuid.time` and `cn_uuid.clock_seq`. The accessor methods are undocumented. > + self.sub_dict['ATTR'] = attribute > > - # Refresh the time to make uniqueness more probable. Add on some > - # randomness for good measure. > - self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) > + cn = self._template_str("indextask_${ATTR}_${TIME}_${SEQ}") You're overwriting sub_dict['TIME'], which is used elsewhere in the class. That could cause trouble. There's no reason to use sub_dict here; you can simply do: cn = 'indextask_%s_%s_%s' % (attribute, cn_uuid.time, cn_uuid.clock_seq) -- Petr? From jdennis at redhat.com Tue Jul 24 18:09:34 2012 From: jdennis at redhat.com (John Dennis) Date: Tue, 24 Jul 2012 14:09:34 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> Message-ID: <500EE4DE.6000001@redhat.com> On 07/24/2012 12:03 PM, Stephen Ingram wrote: > On Mon, Jul 23, 2012 at 11:25 AM, John Dennis wrote: >> On 07/23/2012 12:37 PM, John Dennis wrote: >>> >>> Note the timestamps only have 1 second resolution, something >>> which would be easy to fix, so bear that in mind when looking at the >>> output of the script. >> >> >> This is partially as a note to myself, but also to others who want to test. >> The following patch should produce timestamps in the debug output with >> microsecond resolution. > > No one seems to be jumping in on this. Should I run the test again > with this patch? I'm really interested in getting this resolved as I > can't help but think others are experiencing the same problem. I do > have a user in the system that is scheduled for deletion. Would it > help for you to grab a ticket with those credentials so you could > actually see the slowness in action? It would be good to re-run your test with better timing information. Attached is a patch that adds more timing information to the debug log output. Please apply the patch and re-run your test the same way you did before and email me the contents of /var/log/httpd/error_log. In the error_log you should see lines with "elapsed_time=xxxx", e.g. INFO: admin at XXX.XXX.XXX.XXX.XX: batch(({'params': ((), {}), 'method': 'ping'},)): SUCCESS elapsed_time=0.001663 To apply the patch you'll need to be root to modify the files, this should work: % su % cd /usr/lib/python2.7/site-packages % patch -p1 -b < path_to_saved_patch_file If you don't have patch installed: % sudo yum install patch Then restart httpd and re-run the test. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa_timing.patch Type: text/x-patch Size: 3597 bytes Desc: not available URL: From simo at redhat.com Tue Jul 24 18:31:07 2012 From: simo at redhat.com (Simo Sorce) Date: Tue, 24 Jul 2012 14:31:07 -0400 Subject: [Freeipa-devel] Pushed one liner for spec file Message-ID: <1343154667.3219.337.camel@willson.li.ssimo.org> FYI, I pushed the following as a one-liner so that automatica builders gets dependencies right: http://git.fedorahosted.org/git/?p=freeipa.git;a=commitdiff;h=dedb180ddc773baf42bb31efc07a1dda698631bb Simo. -- Simo Sorce * Red Hat, Inc * New York From abokovoy at redhat.com Tue Jul 24 18:36:08 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 24 Jul 2012 21:36:08 +0300 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <500ECD25.9030804@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> <500EA78F.1010607@redhat.com> <500EA955.7090804@redhat.com> <20120724145037.GS1052@redhat.com> <500ECD25.9030804@redhat.com> Message-ID: <20120724183608.GT1052@redhat.com> On Tue, 24 Jul 2012, Petr Viktorin wrote: >On 07/24/2012 04:50 PM, Alexander Bokovoy wrote: >>On Tue, 24 Jul 2012, Rob Crittenden wrote: >>>Petr Viktorin wrote: >>>>On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: >>>>>On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>>>>>>On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>>>On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>>>>>>Hi, >>>>>>>>> >>>>>>>>>There are two problems in task naming in LDAP updates: >>>>>>>>> >>>>>>>>>1. Randomness may be scarce in virtual machines >>>>>>>>>2. Random number is added to the time value rounded to a second >>>>>>>>> >>>>>>>>>The second issue leads to values that may repeat themselves as time >>>>>>>>>only grows and random number is non-negative as well, so >>>>>>>>>t2+r2 can be equal to t1+t2 generated earlier. >>>>>>>>> >>>>>>>>>Since task name is a DN, there is no strict requirement to use an >>>>>>>>>integer value. Instead, we can take time and attribute name. To >>>>>>>>>get >>>>>>>>>reasonable 'randomness' these values are then hashed with sha1 and >>>>>>>>>use >>>>>>>>>the resulting string as task name. >>>>>>>>> >>>>>>>>>SHA1 may technically be an overkill here as we could simply use >>>>>>>>> >>>>>>>>>indextask_$date_$attribute >>>>>>>>> >>>>>>>>>where $date is a value of time.time() but SHA1 gives a resonable >>>>>>>>>'randomness' into the string. >>>>>>>> >>>>>>>>What kind of randomness do you mean? SHA1 is deterministic, it >>>>>>>>doesn't >>>>>>>>add any randomness at all. It just obscures what's really happening. >>>>>>>Hence using quotes to describe it. We don't need randomness in the >>>>>>>task >>>>>>>names, we need something that avoids collisions. >>>>>>> >>>>>>>An issue here is in time.time() -- it may give us sub-second >>>>>>>resolution >>>>>>>if underlying OS supports it, it may not. Having a second-level >>>>>>>resolution is not enough, especially on fast machines, so we can't >>>>>>>simply use int(times.time()) as it was in the original version. >>>>>>> >>>>>>>indextask_$date_$attribute has this issue that we don't have enough >>>>>>>guarantee for $date (time.time()) to be unique in sufficiently tight >>>>>>>conditions, thus use of SHA-1 to generate something that has better >>>>>>>chances to avoid collisions than $data_$attribute. >>>>>> >>>>>>My point is that if "indextask_$date_$attribute" is not unique, >>>>>>neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >>>>>>on the chance of collisions. >>>>>> >>>>>>You could use Python's pseudorandom number generator (random.randint) >>>>>>instead of random.SystemRandom. It's not cryptographically secure but >>>>>>it's enough to avoid collisions, and it doesn't use up system entropy >>>>>>(except for initial seeding, through `import random`). >>>>>>"indextask_$date_$attribute_$pseudorandomvalue" should be unique >>>>>>enough. >>>>>Using random here is really bad. >>>>>What we ideally need is a method to increment sequential calls for >>>>>the same attribute.We use seconds to differentiate >>>>>between all tasks but that is not really required, tasks that were >>>>>processed will be removed. >>>>> >>>> >>>>Or maybe use $date_$attribute and just warn (ignore the error) if >>>>there's a duplicate -- if a reindex task for the same attribute already >>>>exists from the same second, do we really need to start a new one? >>>> >>> >>>That is true. >>> >>>We can generate a UUID, I think that is probably a better/safer thing >>>to use overall. >>Updated patch attached. >> >>2012-07-24T14:36:31Z INFO Creating task to index attribute: memberOf >>2012-07-24T14:36:31Z DEBUG Task id: >>cn=indextask_memberOf_135624333918176170_14302,cn=index,cn=tasks,cn=config >>2012-07-24T14:36:32Z INFO Indexing finished >>... >>2012-07-24T14:36:43Z INFO Creating task to index attribute: memberUser >>2012-07-24T14:36:43Z DEBUG Task id: >>cn=indextask_memberUser_135624334038532670_14302,cn=index,cn=tasks,cn=config >> >>2012-07-24T14:36:44Z INFO Indexing finished >> >>You can note that clock_seq does not change, it is because uuid.uuid1() >>uses nanosecond resolution and uuid_generate_time() if the latter is >>available. If we happen to ask for uuids within sub-nanosecond interval, >>clock_seq will be different then. >> >>I'm extracting only time and clock_seq instead of pasting full uuid >>value because uuid_generate_time() will leak out ethernet MAC address >>for 48-bit node. We don't need more bits here so I drop these 48 bits >>and avoid publishing the ethernet MAC address, even in logs. >> >>-- >>/ Alexander Bokovoy > >Yes, this approach will work. Just some technical comments: > > >>+ self.sub_dict['TIME'] = cn_uuid.get_time() >>+ self.sub_dict['SEQ'] = cn_uuid.get_clock_seq() > >Use the attributes, `cn_uuid.time` and `cn_uuid.clock_seq`. The >accessor methods are undocumented. Fixed. > >>+ self.sub_dict['ATTR'] = attribute >> >>- # Refresh the time to make uniqueness more probable. Add on some >>- # randomness for good measure. >>- self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) >>+ cn = self._template_str("indextask_${ATTR}_${TIME}_${SEQ}") > >You're overwriting sub_dict['TIME'], which is used elsewhere in the >class. That could cause trouble. Since it was set here and others are using it, I kept it updated in a new version as well, based on cn_uuid.time. But since cn_uuid.time is in nanoseconds resolution, I have to divide the value by 1e9. >There's no reason to use sub_dict here; you can simply do: > >cn = 'indextask_%s_%s_%s' % (attribute, cn_uuid.time, cn_uuid.clock_seq) Fixed. This version works fine for me. -- / Alexander Bokovoy -------------- next part -------------- >From 3e404c5ab461a89aa492fa2f98c81e73fc4cdb1c Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Tue, 24 Jul 2012 12:07:23 +0300 Subject: [PATCH 3/3] Rework task naming in LDAP updates to avoid conflicting names in certain cases There are two problems in task naming in LDAP updates: 1. Randomness may be scarce in virtual machines 2. Random number is added to the time value rounded to a second The second issue leads to values that may repeat themselves as time only grows and random number is non-negative as well, so t2+r2 can be equal to t1+t2 generated earlier. Since task name is a DN, there is no strict requirement to use an integer value. Instead, we generate an UUID and use its 60-bit time, 14-bit sequential number, and attribute name. https://fedorahosted.org/freeipa/ticket/2942 --- ipaserver/install/ldapupdate.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/ipaserver/install/ldapupdate.py b/ipaserver/install/ldapupdate.py index c64139889d9f84866ac0cd358ed3a3a7d95af7dc..949b86ad99bce97fe3d09baef7fa42dbbc55ac26 100644 --- a/ipaserver/install/ldapupdate.py +++ b/ipaserver/install/ldapupdate.py @@ -29,9 +29,11 @@ from ipaserver.install import installutils from ipaserver.install import service from ipaserver import ipaldap from ipapython import entity, ipautil +import uuid from ipalib import util from ipalib import errors from ipalib import api +from ipalib.dn import DN import ldap from ldap.dn import escape_dn_chars from ipapython.ipa_log_manager import * @@ -328,16 +330,14 @@ class LDAPUpdate: if self.live_run: time.sleep(5) - r = random.SystemRandom() + cn_uuid = uuid.uuid1() + # cn_uuid.time is in nanoseconds, but other users of LDAPUpdate expect + # seconds in 'TIME' so scale the value down + self.sub_dict['TIME'] = int(cn_uuid.time/1e9) + cn = "indextask_%s_%s_%s" % (attribute, cn_uuid.time, cn_uuid.clock_seq) + dn = DN(('cn', cn), ('cn', 'index'), ('cn', 'tasks'), ('cn', 'config')) - # Refresh the time to make uniqueness more probable. Add on some - # randomness for good measure. - self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) - - cn = self._template_str("indextask_$TIME") - dn = "cn=%s, cn=index, cn=tasks, cn=config" % cn - - e = ipaldap.Entry(dn) + e = ipaldap.Entry(str(dn)) e.setValues('objectClass', ['top', 'extensibleObject']) e.setValue('cn', cn) -- 1.7.10.4 From rcritten at redhat.com Wed Jul 25 03:03:26 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 24 Jul 2012 23:03:26 -0400 Subject: [Freeipa-devel] [PATCH] 1039 fix selinux usermap config options Message-ID: <500F61FE.6010300@redhat.com> The configuration options for the default user and map order were a bit broken in several ways. I wasn't handling the case where one of the values was coming from LDAP so was a list vs as an option which was a string, so all sorts of bad interesting things were happening. There is also the setattr problem. We would normally handle that in a validator so it is not a problem but in this case we may need to compare two options passed in and we can't do that in a validator. So potentially changes may come in as a option, in entry_attrs or from config. I added a few tests to help keep this robust. When testing this remember that the user map order list needs to be quoted otherwise the shell is going to interpret the $. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1039-config.patch Type: text/x-diff Size: 6022 bytes Desc: not available URL: From sbingram at gmail.com Wed Jul 25 06:59:45 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Tue, 24 Jul 2012 23:59:45 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <500EE4DE.6000001@redhat.com> References: <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> Message-ID: On Tue, Jul 24, 2012 at 11:09 AM, John Dennis wrote: > On 07/24/2012 12:03 PM, Stephen Ingram wrote: >> >> On Mon, Jul 23, 2012 at 11:25 AM, John Dennis wrote: >>> >>> On 07/23/2012 12:37 PM, John Dennis wrote: >>>> >>>> >>>> Note the timestamps only have 1 second resolution, something >>>> which would be easy to fix, so bear that in mind when looking at the >>>> output of the script. >>> >>> >>> >>> This is partially as a note to myself, but also to others who want to >>> test. >>> The following patch should produce timestamps in the debug output with >>> microsecond resolution. >> >> >> No one seems to be jumping in on this. Should I run the test again >> with this patch? I'm really interested in getting this resolved as I >> can't help but think others are experiencing the same problem. I do >> have a user in the system that is scheduled for deletion. Would it >> help for you to grab a ticket with those credentials so you could >> actually see the slowness in action? > > > It would be good to re-run your test with better timing information. > Attached is a patch that adds more timing information to the debug log > output. Please apply the patch and re-run your test the same way you did > before and email me the contents of /var/log/httpd/error_log. In the > error_log you should see lines with "elapsed_time=xxxx", e.g. > > INFO: admin at XXX.XXX.XXX.XXX.XX: batch(({'params': ((), {}), 'method': > 'ping'},)): SUCCESS elapsed_time=0.001663 > > To apply the patch you'll need to be root to modify the files, this should > work: > > % su > % cd /usr/lib/python2.7/site-packages > % patch -p1 -b < path_to_saved_patch_file > > If you don't have patch installed: > > % sudo yum install patch > > Then restart httpd and re-run the test. Seeing python2.7, I'm guessing these patches were against Fedora. Since I couldn't get them to apply against RH 6.3 I applied them by hand. I couldn't get the WebUI to load after applying the patches. I'm not sure of the code that caused the problem, but I did see mention of global name datetime not being defined. You wouldn't happen to have patches for RH 6.3? Steve From pspacek at redhat.com Wed Jul 25 08:18:01 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 25 Jul 2012 10:18:01 +0200 Subject: [Freeipa-devel] [PATCH 0040] Handle incomplete/invalid zone unload in same way as BIND's ns_server_del_zone() Message-ID: <500FABB9.3000208@redhat.com> Hello, this patch prevents potential failure during invalid zone unload. Error handling was changed to the same way as in bind/bin/named/server.c ns_server_del_zone(). Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0040-Handle-incomplete-invalid-zone-unload-in-same-way-as.patch Type: text/x-patch Size: 992 bytes Desc: not available URL: From pviktori at redhat.com Wed Jul 25 08:22:51 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 25 Jul 2012 10:22:51 +0200 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <20120724183608.GT1052@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> <500EA78F.1010607@redhat.com> <500EA955.7090804@redhat.com> <20120724145037.GS1052@redhat.com> <500ECD25.9030804@redhat.com> <20120724183608.GT1052@redhat.com> Message-ID: <500FACDB.3000604@redhat.com> On 07/24/2012 08:36 PM, Alexander Bokovoy wrote: > On Tue, 24 Jul 2012, Petr Viktorin wrote: >> On 07/24/2012 04:50 PM, Alexander Bokovoy wrote: >>> On Tue, 24 Jul 2012, Rob Crittenden wrote: >>>> Petr Viktorin wrote: >>>>> On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: >>>>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>> On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>>>>>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>>>> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> There are two problems in task naming in LDAP updates: >>>>>>>>>> >>>>>>>>>> 1. Randomness may be scarce in virtual machines >>>>>>>>>> 2. Random number is added to the time value rounded to a second >>>>>>>>>> >>>>>>>>>> The second issue leads to values that may repeat themselves as >>>>>>>>>> time >>>>>>>>>> only grows and random number is non-negative as well, so >>>>>>>>>> t2+r2 can be equal to t1+t2 generated earlier. >>>>>>>>>> >>>>>>>>>> Since task name is a DN, there is no strict requirement to use an >>>>>>>>>> integer value. Instead, we can take time and attribute name. To >>>>>>>>>> get >>>>>>>>>> reasonable 'randomness' these values are then hashed with sha1 >>>>>>>>>> and >>>>>>>>>> use >>>>>>>>>> the resulting string as task name. >>>>>>>>>> >>>>>>>>>> SHA1 may technically be an overkill here as we could simply use >>>>>>>>>> >>>>>>>>>> indextask_$date_$attribute >>>>>>>>>> >>>>>>>>>> where $date is a value of time.time() but SHA1 gives a resonable >>>>>>>>>> 'randomness' into the string. >>>>>>>>> >>>>>>>>> What kind of randomness do you mean? SHA1 is deterministic, it >>>>>>>>> doesn't >>>>>>>>> add any randomness at all. It just obscures what's really >>>>>>>>> happening. >>>>>>>> Hence using quotes to describe it. We don't need randomness in the >>>>>>>> task >>>>>>>> names, we need something that avoids collisions. >>>>>>>> >>>>>>>> An issue here is in time.time() -- it may give us sub-second >>>>>>>> resolution >>>>>>>> if underlying OS supports it, it may not. Having a second-level >>>>>>>> resolution is not enough, especially on fast machines, so we can't >>>>>>>> simply use int(times.time()) as it was in the original version. >>>>>>>> >>>>>>>> indextask_$date_$attribute has this issue that we don't have enough >>>>>>>> guarantee for $date (time.time()) to be unique in sufficiently >>>>>>>> tight >>>>>>>> conditions, thus use of SHA-1 to generate something that has better >>>>>>>> chances to avoid collisions than $data_$attribute. >>>>>>> >>>>>>> My point is that if "indextask_$date_$attribute" is not unique, >>>>>>> neither is SHA1("indextask_$date_$attribute"). Hashing has no effect >>>>>>> on the chance of collisions. >>>>>>> >>>>>>> You could use Python's pseudorandom number generator >>>>>>> (random.randint) >>>>>>> instead of random.SystemRandom. It's not cryptographically secure >>>>>>> but >>>>>>> it's enough to avoid collisions, and it doesn't use up system >>>>>>> entropy >>>>>>> (except for initial seeding, through `import random`). >>>>>>> "indextask_$date_$attribute_$pseudorandomvalue" should be unique >>>>>>> enough. >>>>>> Using random here is really bad. >>>>>> What we ideally need is a method to increment sequential calls for >>>>>> the same attribute.We use seconds to differentiate >>>>>> between all tasks but that is not really required, tasks that were >>>>>> processed will be removed. >>>>>> >>>>> >>>>> Or maybe use $date_$attribute and just warn (ignore the error) if >>>>> there's a duplicate -- if a reindex task for the same attribute >>>>> already >>>>> exists from the same second, do we really need to start a new one? >>>>> >>>> >>>> That is true. >>>> >>>> We can generate a UUID, I think that is probably a better/safer thing >>>> to use overall. >>> Updated patch attached. >>> >>> 2012-07-24T14:36:31Z INFO Creating task to index attribute: memberOf >>> 2012-07-24T14:36:31Z DEBUG Task id: >>> cn=indextask_memberOf_135624333918176170_14302,cn=index,cn=tasks,cn=config >>> >>> 2012-07-24T14:36:32Z INFO Indexing finished >>> ... >>> 2012-07-24T14:36:43Z INFO Creating task to index attribute: memberUser >>> 2012-07-24T14:36:43Z DEBUG Task id: >>> cn=indextask_memberUser_135624334038532670_14302,cn=index,cn=tasks,cn=config >>> >>> >>> 2012-07-24T14:36:44Z INFO Indexing finished >>> >>> You can note that clock_seq does not change, it is because uuid.uuid1() >>> uses nanosecond resolution and uuid_generate_time() if the latter is >>> available. If we happen to ask for uuids within sub-nanosecond interval, >>> clock_seq will be different then. >>> >>> I'm extracting only time and clock_seq instead of pasting full uuid >>> value because uuid_generate_time() will leak out ethernet MAC address >>> for 48-bit node. We don't need more bits here so I drop these 48 bits >>> and avoid publishing the ethernet MAC address, even in logs. >>> >>> -- >>> / Alexander Bokovoy >> >> Yes, this approach will work. Just some technical comments: >> >> >>> + self.sub_dict['TIME'] = cn_uuid.get_time() >>> + self.sub_dict['SEQ'] = cn_uuid.get_clock_seq() >> >> Use the attributes, `cn_uuid.time` and `cn_uuid.clock_seq`. The >> accessor methods are undocumented. > Fixed. > >> >>> + self.sub_dict['ATTR'] = attribute >>> >>> - # Refresh the time to make uniqueness more probable. Add on >>> some >>> - # randomness for good measure. >>> - self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) >>> + cn = self._template_str("indextask_${ATTR}_${TIME}_${SEQ}") >> >> You're overwriting sub_dict['TIME'], which is used elsewhere in the >> class. That could cause trouble. > Since it was set here and others are using it, I kept it updated in a > new version as well, based on cn_uuid.time. But since cn_uuid.time is in > nanoseconds resolution, I have to divide the value by 1e9. > >> There's no reason to use sub_dict here; you can simply do: >> >> cn = 'indextask_%s_%s_%s' % (attribute, cn_uuid.time, cn_uuid.clock_seq) > Fixed. > > This version works fine for me. > ACK -- Petr? From pviktori at redhat.com Wed Jul 25 08:54:15 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 25 Jul 2012 10:54:15 +0200 Subject: [Freeipa-devel] [PATCH] 1039 fix selinux usermap config options In-Reply-To: <500F61FE.6010300@redhat.com> References: <500F61FE.6010300@redhat.com> Message-ID: <500FB437.7030109@redhat.com> On 07/25/2012 05:03 AM, Rob Crittenden wrote: > The configuration options for the default user and map order were a bit > broken in several ways. > > I wasn't handling the case where one of the values was coming from LDAP > so was a list vs as an option which was a string, so all sorts of bad > interesting things were happening. > > There is also the setattr problem. We would normally handle that in a > validator so it is not a problem but in this case we may need to compare > two options passed in and we can't do that in a validator. So > potentially changes may come in as a option, in entry_attrs or from config. > > I added a few tests to help keep this robust. > > When testing this remember that the user map order list needs to be > quoted otherwise the shell is going to interpret the $. > > rob > ACK, with nitpicks :) I think using `copy.deepcopy(options)` instead of simply `dict(options)` is unnecessary; do you have a special reason for it? And note the style guide discourages line continuation backslashes: + if 'ipaselinuxusermapdefault' in validate or \ + 'ipaselinuxusermaporder' in validate: in favor of parentheses: + if ('ipaselinuxusermapdefault' in validate or + 'ipaselinuxusermaporder' in validate): -- Petr? From pvoborni at redhat.com Wed Jul 25 09:59:25 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 25 Jul 2012 11:59:25 +0200 Subject: [Freeipa-devel] [PATCH] 174 Fix autoscroll to top in tables in IE In-Reply-To: <500E15BE.1010702@redhat.com> References: <500D5A1F.7060408@redhat.com> <500E15BE.1010702@redhat.com> Message-ID: <500FC37D.4080007@redhat.com> On 07/24/2012 05:25 AM, Endi Sukma Dewata wrote: > On 7/23/2012 9:05 AM, Petr Vobornik wrote: >> In IE when a window is small (horizontal scrollbar is displayed) click >> or keyboard input on various parts of UI makes search tables scroll to >> top. It prevents from selecting items in a table. This issue happens >> when using absolute positioned element with overflow style. It's a bug >> in IE. >> >> Two workarounds were added to make UI usable in IE. >> Adding position:relative; style to body element fixes the problem in >> search pages. It doesn't help in association dialogs though. >> The bug doesn't occur when some child element has focus. It's possible >> to set focus to first visible checkbox while scrolling down but user >> experience is very bad. Better solution seems to scroll back when IE >> scrolls to top on mousedown. That way mouse click event happens on the >> target element and it can gain focus and therefore be selected. Some >> glitches still remains but is usable. >> >> https://fedorahosted.org/freeipa/ticket/2835 > > ACK. Pushed to master. > > Separate issue, I noticed that on IE the background of the dialog box > title is black. On Firefox it's green. Same thing for the > Available/Prospective header in the association dialog. > Will be fixed soon along with update of jquery-ui. -- Petr Vobornik From pvoborni at redhat.com Wed Jul 25 10:03:37 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Wed, 25 Jul 2012 12:03:37 +0200 Subject: [Freeipa-devel] [PATCH] 173 IDs and names for dialogs In-Reply-To: <500E155A.2020304@redhat.com> References: <5008069D.7070202@redhat.com> <500E155A.2020304@redhat.com> Message-ID: <500FC479.7070704@redhat.com> On 07/24/2012 05:24 AM, Endi Sukma Dewata wrote: > On 7/19/2012 8:07 AM, Petr Vobornik wrote: >> It's hard to detect if or which type of dialog is displayed because not >> all dialogs have IDs. >> >> On dialog open, it's id or name (if id is not set) is used for >> containing element id. Many of dialog types were missing id or name so >> name was added to each dialog type. >> >> In HTML, element's id should be unique. Our framework allows opening two >> dialogs with the same id. It may lead to state where getElementById >> method may have unpredictable behavior. Therefore attribute 'data-name' >> with dialog's name was added to dialog's containing element. Automation >> framework can search more reliable by using this attribute instead of id. >> >> https://fedorahosted.org/freeipa/ticket/2853 > > ACK. Pushed to master. > > If possible we should replace all element ID's with paths. So the > element name itself is not unique, but the element can still be uniquely > identified by following a path from the top to the element itself. The > path could be a combination of element type, class, or name, for example: > > div[name=container] .entity[name=user] .facet[name=search] > > The automation tool should use the path instead of ID. > Yes, for most things it can be done already. There is a little problem with dialogs though. Some are not placed in facet's div so the path doesn't apply. -- Petr Vobornik From jdennis at redhat.com Wed Jul 25 11:31:32 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 25 Jul 2012 07:31:32 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> Message-ID: <500FD914.7040908@redhat.com> On 07/25/2012 02:59 AM, Stephen Ingram wrote: > Seeing python2.7, I'm guessing these patches were against Fedora. > Since I couldn't get them to apply against RH 6.3 I applied them by > hand. I couldn't get the WebUI to load after applying the patches. I'm > not sure of the code that caused the problem, but I did see mention of > global name datetime not being defined. You wouldn't happen to have > patches for RH 6.3? Sorry, I thought you had a F17 install. It should be easy to fix the datetime issue, just add this add the top of the file: import time, datetime If that simple fix doesn't work then let me know the version you've got and I'll make up a new patch against that version. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Wed Jul 25 11:57:33 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 25 Jul 2012 07:57:33 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500E7B4E.60300@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> <500DDA68.4090003@redhat.com> <500E7B4E.60300@redhat.com> Message-ID: <500FDF2D.9020301@redhat.com> On 07/24/2012 06:39 AM, Petr Viktorin wrote: > On 07/24/2012 01:12 AM, John Dennis wrote: >> On 07/23/2012 06:27 AM, Petr Viktorin wrote: >>> As a translator (for another project), I don't like Transifex and prefer >>> to send good old Git pull requests. I understand a "traditional" >>> workflow is hard to coordinate with others that use Transifex, but still >>> I'd hate it if we became dependent on Tx. >> >> For better or worse we are dependent on TX (Transifex). Fedora has >> adopted TX as it's translation tool, RHEL's translation tools integrate >> with TX (as well as other translation portals). And SSSD and IPA have >> made a a commitment to TX based on the direction of Fedora and RHEL. >> >> Given that we've adopted TX I don't see the value in maintaining tools >> that support both TX and non-TX workflows. I'd rather see us delete the >> non-TX elements. If we have just one workflow it's easier to understand >> and maintain the code. If we ever decide we need to go back to a non-TX >> workflow we can always retrieve the deleted code from git. >> > > This means you have to be a member of a Fedora translation team to > translate. Actually we're not using the Fedora TX instance, rather the transifex.net instance so I don't think we're limited to translators who are members of a Fedora translation team. > It makes it harder for people to fork the project. A workflow > with a mandatory central repository makes it impossible to experiment > locally. > I'm all for having a standard way to receive contributions, but limiting > how people can create those contributions isn't good. > > I'm all for deleting unused code, but here I think it would be a bad move. Actually I don't have strong feelings about this one way or the other. My primary concern with two different workflows was that we have to test and maintain both and one of them is currently unused. My other concern is the added complexity, most developers and release engineers don't understand this stuff so keeping is simple to accommodate those less familiar with the process seemed like a win. But you have a valid point about being flexible, so it's fine with me to keep the old code. We probably need better documentation. John -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Wed Jul 25 11:59:50 2012 From: jdennis at redhat.com (John Dennis) Date: Wed, 25 Jul 2012 07:59:50 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500E5A0F.5000000@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> <500DB826.8090908@redhat.com> <500E5A0F.5000000@redhat.com> Message-ID: <500FDFB6.2090804@redhat.com> On 07/24/2012 04:17 AM, Petr Viktorin wrote: > On 07/23/2012 10:46 PM, John Dennis wrote: > [...] >> >> The only thing holding up the ACK is the question of why po-files now >> has update_pot as a dependency. >> > > If files simply depend on $(DOMAIN).pot, then they are considered > up-to-date even after they're changed (e.g. with strip-po). They need to > depend on a rule that always runs so that they get merged. > > There's another alternative to achieve this: adding them to .PHONY. The > attached version does that, perhaps it's cleaner. > > ACK, thanks for the good work. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From atkac at redhat.com Wed Jul 25 13:10:17 2012 From: atkac at redhat.com (Adam Tkac) Date: Wed, 25 Jul 2012 15:10:17 +0200 Subject: [Freeipa-devel] [PATCH 0040] Handle incomplete/invalid zone unload in same way as BIND's ns_server_del_zone() In-Reply-To: <500FABB9.3000208@redhat.com> References: <500FABB9.3000208@redhat.com> Message-ID: <20120725131014.GA19507@redhat.com> On Wed, Jul 25, 2012 at 10:18:01AM +0200, Petr Spacek wrote: > Hello, > > this patch prevents potential failure during invalid zone unload. > Error handling was changed to the same way as in > bind/bin/named/server.c ns_server_del_zone(). Ack. > From 02e232632a8a04fcd17f1089553961c18c0b175a Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Wed, 25 Jul 2012 10:07:20 +0200 > Subject: [PATCH] Handle incomplete/invalid zone unload in same way as > ns_server_del_zone(). > > Signed-off-by: Petr Spacek > --- > src/ldap_helper.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/ldap_helper.c b/src/ldap_helper.c > index daffac7c7825a99a07c333217638d3beaddfaad2..cc7003a6cdcd2d27404fec936623ed8a3e8fa7f8 100644 > --- a/src/ldap_helper.c > +++ b/src/ldap_helper.c > @@ -823,7 +823,7 @@ ldap_delete_zone2(ldap_instance_t *inst, dns_name_t *name, isc_boolean_t lock) > > /* Do not unload partially loaded zones, they have incomplete structures. */ > dns_db_t *dbp = NULL; > - if (dns_zone_getdb(zone, &dbp) != DNS_R_NOTLOADED) { > + if (dns_zone_getdb(zone, &dbp) == ISC_R_SUCCESS) { > dns_db_detach(&dbp); /* dns_zone_getdb() attaches DB implicitly */ > dns_zone_unload(zone); > } > -- > 1.7.10.4 > -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Wed Jul 25 13:31:34 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 25 Jul 2012 15:31:34 +0200 Subject: [Freeipa-devel] [PATCH 0041] Cleanup in logging code Message-ID: <500FF536.8040308@redhat.com> Hello, this patch clears logging code a bit. Adding functions like log_info() and similar will be trivial from now. It will be necessary for ticket #71: Log successful reconnect https://fedorahosted.org/bind-dyndb-ldap/ticket/71 Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0041-Cleanup-in-logging-code.patch Type: text/x-patch Size: 2499 bytes Desc: not available URL: From rcritten at redhat.com Wed Jul 25 13:33:48 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Jul 2012 09:33:48 -0400 Subject: [Freeipa-devel] [PATCH] 0064 Rework task naming in LDAP updates to avoid conflicts In-Reply-To: <500FACDB.3000604@redhat.com> References: <20120724100147.GP1052@redhat.com> <500E8885.4020703@redhat.com> <20120724120654.GQ1052@redhat.com> <500E93DA.5020208@redhat.com> <20120724124919.GR1052@redhat.com> <500EA78F.1010607@redhat.com> <500EA955.7090804@redhat.com> <20120724145037.GS1052@redhat.com> <500ECD25.9030804@redhat.com> <20120724183608.GT1052@redhat.com> <500FACDB.3000604@redhat.com> Message-ID: <500FF5BC.6060106@redhat.com> Petr Viktorin wrote: > On 07/24/2012 08:36 PM, Alexander Bokovoy wrote: >> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>> On 07/24/2012 04:50 PM, Alexander Bokovoy wrote: >>>> On Tue, 24 Jul 2012, Rob Crittenden wrote: >>>>> Petr Viktorin wrote: >>>>>> On 07/24/2012 02:49 PM, Alexander Bokovoy wrote: >>>>>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>>> On 07/24/2012 02:06 PM, Alexander Bokovoy wrote: >>>>>>>>> On Tue, 24 Jul 2012, Petr Viktorin wrote: >>>>>>>>>> On 07/24/2012 12:01 PM, Alexander Bokovoy wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> There are two problems in task naming in LDAP updates: >>>>>>>>>>> >>>>>>>>>>> 1. Randomness may be scarce in virtual machines >>>>>>>>>>> 2. Random number is added to the time value rounded to a second >>>>>>>>>>> >>>>>>>>>>> The second issue leads to values that may repeat themselves as >>>>>>>>>>> time >>>>>>>>>>> only grows and random number is non-negative as well, so >>>>>>>>>>> t2+r2 can be equal to t1+t2 generated earlier. >>>>>>>>>>> >>>>>>>>>>> Since task name is a DN, there is no strict requirement to >>>>>>>>>>> use an >>>>>>>>>>> integer value. Instead, we can take time and attribute name. To >>>>>>>>>>> get >>>>>>>>>>> reasonable 'randomness' these values are then hashed with sha1 >>>>>>>>>>> and >>>>>>>>>>> use >>>>>>>>>>> the resulting string as task name. >>>>>>>>>>> >>>>>>>>>>> SHA1 may technically be an overkill here as we could simply use >>>>>>>>>>> >>>>>>>>>>> indextask_$date_$attribute >>>>>>>>>>> >>>>>>>>>>> where $date is a value of time.time() but SHA1 gives a resonable >>>>>>>>>>> 'randomness' into the string. >>>>>>>>>> >>>>>>>>>> What kind of randomness do you mean? SHA1 is deterministic, it >>>>>>>>>> doesn't >>>>>>>>>> add any randomness at all. It just obscures what's really >>>>>>>>>> happening. >>>>>>>>> Hence using quotes to describe it. We don't need randomness in the >>>>>>>>> task >>>>>>>>> names, we need something that avoids collisions. >>>>>>>>> >>>>>>>>> An issue here is in time.time() -- it may give us sub-second >>>>>>>>> resolution >>>>>>>>> if underlying OS supports it, it may not. Having a second-level >>>>>>>>> resolution is not enough, especially on fast machines, so we can't >>>>>>>>> simply use int(times.time()) as it was in the original version. >>>>>>>>> >>>>>>>>> indextask_$date_$attribute has this issue that we don't have >>>>>>>>> enough >>>>>>>>> guarantee for $date (time.time()) to be unique in sufficiently >>>>>>>>> tight >>>>>>>>> conditions, thus use of SHA-1 to generate something that has >>>>>>>>> better >>>>>>>>> chances to avoid collisions than $data_$attribute. >>>>>>>> >>>>>>>> My point is that if "indextask_$date_$attribute" is not unique, >>>>>>>> neither is SHA1("indextask_$date_$attribute"). Hashing has no >>>>>>>> effect >>>>>>>> on the chance of collisions. >>>>>>>> >>>>>>>> You could use Python's pseudorandom number generator >>>>>>>> (random.randint) >>>>>>>> instead of random.SystemRandom. It's not cryptographically secure >>>>>>>> but >>>>>>>> it's enough to avoid collisions, and it doesn't use up system >>>>>>>> entropy >>>>>>>> (except for initial seeding, through `import random`). >>>>>>>> "indextask_$date_$attribute_$pseudorandomvalue" should be unique >>>>>>>> enough. >>>>>>> Using random here is really bad. >>>>>>> What we ideally need is a method to increment sequential calls for >>>>>>> the same attribute.We use seconds to differentiate >>>>>>> between all tasks but that is not really required, tasks that were >>>>>>> processed will be removed. >>>>>>> >>>>>> >>>>>> Or maybe use $date_$attribute and just warn (ignore the error) if >>>>>> there's a duplicate -- if a reindex task for the same attribute >>>>>> already >>>>>> exists from the same second, do we really need to start a new one? >>>>>> >>>>> >>>>> That is true. >>>>> >>>>> We can generate a UUID, I think that is probably a better/safer thing >>>>> to use overall. >>>> Updated patch attached. >>>> >>>> 2012-07-24T14:36:31Z INFO Creating task to index attribute: memberOf >>>> 2012-07-24T14:36:31Z DEBUG Task id: >>>> cn=indextask_memberOf_135624333918176170_14302,cn=index,cn=tasks,cn=config >>>> >>>> >>>> 2012-07-24T14:36:32Z INFO Indexing finished >>>> ... >>>> 2012-07-24T14:36:43Z INFO Creating task to index attribute: memberUser >>>> 2012-07-24T14:36:43Z DEBUG Task id: >>>> cn=indextask_memberUser_135624334038532670_14302,cn=index,cn=tasks,cn=config >>>> >>>> >>>> >>>> 2012-07-24T14:36:44Z INFO Indexing finished >>>> >>>> You can note that clock_seq does not change, it is because uuid.uuid1() >>>> uses nanosecond resolution and uuid_generate_time() if the latter is >>>> available. If we happen to ask for uuids within sub-nanosecond >>>> interval, >>>> clock_seq will be different then. >>>> >>>> I'm extracting only time and clock_seq instead of pasting full uuid >>>> value because uuid_generate_time() will leak out ethernet MAC address >>>> for 48-bit node. We don't need more bits here so I drop these 48 bits >>>> and avoid publishing the ethernet MAC address, even in logs. >>>> >>>> -- >>>> / Alexander Bokovoy >>> >>> Yes, this approach will work. Just some technical comments: >>> >>> >>>> + self.sub_dict['TIME'] = cn_uuid.get_time() >>>> + self.sub_dict['SEQ'] = cn_uuid.get_clock_seq() >>> >>> Use the attributes, `cn_uuid.time` and `cn_uuid.clock_seq`. The >>> accessor methods are undocumented. >> Fixed. >> >>> >>>> + self.sub_dict['ATTR'] = attribute >>>> >>>> - # Refresh the time to make uniqueness more probable. Add on >>>> some >>>> - # randomness for good measure. >>>> - self.sub_dict['TIME'] = int(time.time()) + r.randint(0,10000) >>>> + cn = self._template_str("indextask_${ATTR}_${TIME}_${SEQ}") >>> >>> You're overwriting sub_dict['TIME'], which is used elsewhere in the >>> class. That could cause trouble. >> Since it was set here and others are using it, I kept it updated in a >> new version as well, based on cn_uuid.time. But since cn_uuid.time is in >> nanoseconds resolution, I have to divide the value by 1e9. >> >>> There's no reason to use sub_dict here; you can simply do: >>> >>> cn = 'indextask_%s_%s_%s' % (attribute, cn_uuid.time, cn_uuid.clock_seq) >> Fixed. >> >> This version works fine for me. >> > > ACK > pushed to master From pspacek at redhat.com Wed Jul 25 13:40:48 2012 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 25 Jul 2012 15:40:48 +0200 Subject: [Freeipa-devel] [PATCH 0040] Handle incomplete/invalid zone unload in same way as BIND's ns_server_del_zone() In-Reply-To: <20120725131014.GA19507@redhat.com> References: <500FABB9.3000208@redhat.com> <20120725131014.GA19507@redhat.com> Message-ID: <500FF760.7060009@redhat.com> On 07/25/2012 03:10 PM, Adam Tkac wrote: > On Wed, Jul 25, 2012 at 10:18:01AM +0200, Petr Spacek wrote: >> Hello, >> >> this patch prevents potential failure during invalid zone unload. >> Error handling was changed to the same way as in >> bind/bin/named/server.c ns_server_del_zone(). > > Ack. Pushed to the master: https://fedorahosted.org/bind-dyndb-ldap/changeset/85763ded13a2c2a641da4a9bbf0950170a6aecf8 Petr^2 Spacek From pviktori at redhat.com Wed Jul 25 13:58:32 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Wed, 25 Jul 2012 15:58:32 +0200 Subject: [Freeipa-devel] [PATCH] 0072 Internationalization for public errors Message-ID: <500FFB88.6080902@redhat.com> I spent some time while IPA was installing tracking down error mesages that weren't marked for translation :) I also changed the error docstrings, as people are likely to look there for inspiration. https://fedorahosted.org/freeipa/ticket/1953 -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pviktori-0072-Internationalization-for-public-errors.patch Type: text/x-patch Size: 53274 bytes Desc: not available URL: From rcritten at redhat.com Wed Jul 25 14:01:51 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Jul 2012 10:01:51 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <500E8116.1020303@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> Message-ID: <500FFC4F.8090601@redhat.com> Petr Viktorin wrote: > On 07/23/2012 10:03 PM, Rob Crittenden wrote: >> Rob Crittenden wrote: >>> Andrew Wnuk wrote: >>>> On 07/16/2012 01:35 PM, Rob Crittenden wrote: >>>>> Nalin Dahyabhai wrote: >>>>>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>>>>> Use the new certmonger capability to be able to renew the dogtag >>>>>>> subsystem certificates (audit, OCSP, etc). >>>>>> >>>>>> Are the copies of the certificates in the pki-ca CS.cfg file being >>>>>> updated elsewhere? Or is it not turning out to be a problem if they >>>>>> aren't? >>>>> >>>>> I didn't test validating OCSP signatures but the audit subsystem >>>>> seemed fine (it complained wildly when I had the wrong trust in the >>>>> NSS db). >>>>> >>>>> Andrew, do I need to update CS.cfg as well? >>>>> >>>> Yes, you may need update CS.cfg too. >>> >>> Ok, added a bit to update CS.cfg with the new certificate. >> >> This should fix some SELinux issues preventing certmonger from >> monitoring the dogtag certificate database in /var/lib/pki-ca/alias. >> >> rob > > I don't know enough about dogtag/certmonger to comment on the > functionality, but there are minor issues I can find. Attaching a patch > to fix them. > > > `make rpms` fails: > > rpmbuild --define "_topdir /rpmbuild" -ba freeipa.spec > error: %changelog not in descending chronological order > make: *** [rpms] Error 1 > > > > `git am` complains: > > Applying: Use certmonger to renew CA subsystem certificates > /home/pviktori/freeipa/.git/rebase-apply/patch:576: new blank line at EOF. > + > /home/pviktori/freeipa/.git/rebase-apply/patch:645: new blank line at EOF. > + > warning: 2 lines add whitespace errors. Thanks, integrated this patch and added a missing script, renew_ipacert. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1033-4-renewal.patch Type: text/x-diff Size: 47248 bytes Desc: not available URL: From rcritten at redhat.com Wed Jul 25 14:14:31 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Jul 2012 10:14:31 -0400 Subject: [Freeipa-devel] [PATCH] 0066 Arrange stripping .po files In-Reply-To: <500FDFB6.2090804@redhat.com> References: <4FE848D7.1010004@redhat.com> <50087382.2000505@redhat.com> <5009205E.9030902@redhat.com> <50097BBF.6010206@redhat.com> <50098711.2020603@redhat.com> <500991EC.9000703@redhat.com> <500D272E.6040309@redhat.com> <500DB826.8090908@redhat.com> <500E5A0F.5000000@redhat.com> <500FDFB6.2090804@redhat.com> Message-ID: <500FFF47.3030004@redhat.com> John Dennis wrote: > On 07/24/2012 04:17 AM, Petr Viktorin wrote: >> On 07/23/2012 10:46 PM, John Dennis wrote: >> [...] >>> >>> The only thing holding up the ACK is the question of why po-files now >>> has update_pot as a dependency. >>> >> >> If files simply depend on $(DOMAIN).pot, then they are considered >> up-to-date even after they're changed (e.g. with strip-po). They need to >> depend on a rule that always runs so that they get merged. >> >> There's another alternative to achieve this: adding them to .PHONY. The >> attached version does that, perhaps it's cleaner. >> >> > > ACK, thanks for the good work. > pushed to master From jcholast at redhat.com Wed Jul 25 14:31:54 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Wed, 25 Jul 2012 16:31:54 +0200 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <500FFC4F.8090601@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> <500FFC4F.8090601@redhat.com> Message-ID: <5010035A.2060108@redhat.com> Dne 25.7.2012 16:01, Rob Crittenden napsal(a): > Petr Viktorin wrote: >> On 07/23/2012 10:03 PM, Rob Crittenden wrote: >>> Rob Crittenden wrote: >>>> Andrew Wnuk wrote: >>>>> On 07/16/2012 01:35 PM, Rob Crittenden wrote: >>>>>> Nalin Dahyabhai wrote: >>>>>>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>>>>>> Use the new certmonger capability to be able to renew the dogtag >>>>>>>> subsystem certificates (audit, OCSP, etc). >>>>>>> >>>>>>> Are the copies of the certificates in the pki-ca CS.cfg file being >>>>>>> updated elsewhere? Or is it not turning out to be a problem if they >>>>>>> aren't? >>>>>> >>>>>> I didn't test validating OCSP signatures but the audit subsystem >>>>>> seemed fine (it complained wildly when I had the wrong trust in the >>>>>> NSS db). >>>>>> >>>>>> Andrew, do I need to update CS.cfg as well? >>>>>> >>>>> Yes, you may need update CS.cfg too. >>>> >>>> Ok, added a bit to update CS.cfg with the new certificate. >>> >>> This should fix some SELinux issues preventing certmonger from >>> monitoring the dogtag certificate database in /var/lib/pki-ca/alias. >>> >>> rob >> >> I don't know enough about dogtag/certmonger to comment on the >> functionality, but there are minor issues I can find. Attaching a patch >> to fix them. >> >> >> `make rpms` fails: >> >> rpmbuild --define "_topdir /rpmbuild" -ba freeipa.spec >> error: %changelog not in descending chronological order >> make: *** [rpms] Error 1 >> >> >> >> `git am` complains: >> >> Applying: Use certmonger to renew CA subsystem certificates >> /home/pviktori/freeipa/.git/rebase-apply/patch:576: new blank line at >> EOF. >> + >> /home/pviktori/freeipa/.git/rebase-apply/patch:645: new blank line at >> EOF. >> + >> warning: 2 lines add whitespace errors. > > Thanks, integrated this patch and added a missing script, renew_ipacert. > > rob > NACK First, a question: I haven't tested this (yet), but what happens when someone uses the --{dirsrv,http,pkinit}_pkcs12 options of ipa-server-install/ipa-replica-prepare? (There are also other options which I suspect may cause trouble, namely --subject and --selfsign.) install/restart_scripts/renew_ra_cert doesn't seem to be used anywhere. ipa-replica-install --setup-ca fails with: ... [13/15]: configure clone certificate renewals Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. Nickname "ipaCert" doesn't exist in NSS database "/etc/httpd/alias" ipareplica-install.log: ... 2012-07-25T11:49:17Z DEBUG args=/usr/bin/certutil -L -d /etc/httpd/alias -n ipaCert 2012-07-25T11:49:17Z DEBUG stdout= 2012-07-25T11:49:17Z DEBUG stderr=certutil: Could not find cert: ipaCert : File not found 2012-07-25T11:49:17Z INFO File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 604, in run_script return_value = main_function() File "/sbin/ipa-replica-install", line 446, in main (CA, cs) = cainstance.install_replica_ca(config) File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1265, in install_replica_ca subject_base=config.subject_base) File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 554, in configure_instance self.start_creation("Configuring certificate server", 210) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 261, in start_creation method() File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1158, in configure_clone_renewal certmonger.dogtag_start_tracking('dogtag-ipa-retrieve-agent-submit', 'ipaCert', None, '/etc/httpd/alias/pwdfile.txt', '/etc/httpd/alias', 'restart_httpd') File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line 364, in dogtag_start_tracking raise RuntimeError('Nickname "%s" doesn\'t exist in NSS database "%s"' % (nickname, secdir)) 2012-07-25T11:49:17Z INFO The ipa-replica-install command failed, exception: RuntimeError: Nickname "ipaCert" doesn't exist in NSS database "/etc/httpd/alias" (ipa-ca-install doesn't seem to suffer from the above issue.) On clones, the CN=IPA RA,O=REALM certificate is tracked with post-save command '/usr/lib64/ipa/certmonger/restart_httpd "ipaCert"', but restart_httpd does not take any arguments (it does not break anything, it's just weird). Comments on individual files follow: install/certmonger/Makefile.am: Missing closing parenthesis: +EXTRA_DIST = \ + $(app_SCRIPTS \ install/certmonger/dogtag-ipa-retrieve-agent-submit: Typo ("nicknamd"): +# We cheat and pass in the nicknamd as the CA profile to execute against. Are these guaranteed to be upper-case? I'd put operation.upper() here, just to be on the safe side: +if operation not in ['SUBMIT', 'POLL']: + sys.exit(6) # unsupported operation This except block is not necessary, unhandled exceptions are caught in the except block lower in the code: + sys.exit(5) + except Exception, e: + # Unhandled error + sys.exit(3) + finally: install/restart_scripts/restart_dirsrv: You import and initialize api, but then don't use it. install/restart_scripts/*: All these scripts could use more exception handling, but I guess potential bugs can be sorted out later. install/share/default-aci.ldif: The ACIs are wrong (Kerberos principal instead of ldap URI in userdn, in 40-delegation.update it is done right). ipapython/certmonger.py: This is ugly: + if sys.maxsize > 2**32: + libpath = 'lib64' + else: + libpath = 'lib' Is it safe to show the PIN in "getcert -P " in logs? If not, please add an appropriate nolog argument to ipautil.run. ipapython/platform/fedora16.py Can't we pick one name for pki-cad/pki_cad and use only that? selinux/ipa_dogtag/ipa_dogtag.te: Please use tabs here instead of spaces: + class file read; + class file getattr; + class file open; (to be continued) Honza -- Jan Cholasta From simo at redhat.com Wed Jul 25 15:28:28 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 25 Jul 2012 11:28:28 -0400 Subject: [Freeipa-devel] [PATCH] fix buildrequires in spec file Message-ID: <1343230108.2666.9.camel@willson.li.ssimo.org> This has grown to more than a one-liner so I seek an active ack. Simo. -- Simo Sorce * Red Hat, Inc * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-497-1-Add-all-external-samba-libraries-to-BuildRequires.patch Type: text/x-patch Size: 1200 bytes Desc: not available URL: From abokovoy at redhat.com Wed Jul 25 16:06:33 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 25 Jul 2012 19:06:33 +0300 Subject: [Freeipa-devel] [PATCH] fix buildrequires in spec file In-Reply-To: <1343230108.2666.9.camel@willson.li.ssimo.org> References: <1343230108.2666.9.camel@willson.li.ssimo.org> Message-ID: <20120725160633.GC3283@redhat.com> On Wed, 25 Jul 2012, Simo Sorce wrote: >This has grown to more than a one-liner so I seek an active ack. > >Simo. > >-- >Simo Sorce * Red Hat, Inc * New York >>From b107bc7c41c3b6dcb714e558890b37f1e3b4bacd Mon Sep 17 00:00:00 2001 >From: Simo Sorce >Date: Wed, 25 Jul 2012 11:23:11 -0400 >Subject: [PATCH] Add all external samba libraries to BuildRequires > >Also move them in the right spot (if ! only client) so that they are required >only when building the server. >--- > freeipa.spec.in | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > >diff --git a/freeipa.spec.in b/freeipa.spec.in >index 45149c65f2333512fe21e40be903a3e6144d7084..ba5707ab0ce067a81e0849133820401b439ca13a 100644 >--- a/freeipa.spec.in >+++ b/freeipa.spec.in >@@ -33,6 +33,10 @@ BuildRequires: systemd-units > %endif > BuildRequires: samba4-devel >= 4.0.0-128 > BuildRequires: samba4-python >+BuildRequires: libtalloc-devel >+BuildRequires: libtevent-devel >+BuildRequires: libtdb-devel >+BuildRequires: libldb-devel I don't see libtdb and libldb in generated deps anywhere in freeipa rpm packages so they are probably not needed. $ readelf -d /usr/lib64/dirsrv/plugins/libipa*.so|grep Shared|cut -d: -f2|sort -u [libcom_err.so.2] [libcrypto.so.10] [libc.so.6] [libk5crypto.so.3] [libkrb5.so.3] [liblber-2.4.so.2] [libldap_r-2.4.so.2] [libndr-nbt.so.0] [libndr.so.0] [libsamba-util.so.0] [libsss_idmap.so.0] [libtalloc.so.2] [libtevent.so.0] [libuuid.so.1] [libwbclient.so.0] $ readelf -d /usr/lib64/samba/pdb/ipasam.so|grep Shared|cut -d: -f2 [liblber-2.4.so.2] [libldap_r-2.4.so.2] [libkrb5.so.3] [libk5crypto.so.3] [libcom_err.so.2] [libndr.so.0] [libsamba-util.so.0] [libtevent.so.0] [libtalloc.so.2] [libsmbldap.so] [libcliauth.so] [libpdb.so.0] [libsecurity.so] [libsmbconf.so.0] [libc.so.6] -- / Alexander Bokovoy From abokovoy at redhat.com Wed Jul 25 17:32:02 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Wed, 25 Jul 2012 20:32:02 +0300 Subject: [Freeipa-devel] [PATCH] fix buildrequires in spec file In-Reply-To: <1343230108.2666.9.camel@willson.li.ssimo.org> References: <1343230108.2666.9.camel@willson.li.ssimo.org> Message-ID: <20120725173202.GF3283@redhat.com> On Wed, 25 Jul 2012, Simo Sorce wrote: >This has grown to more than a one-liner so I seek an active ack. > >Simo. > >-- >Simo Sorce * Red Hat, Inc * New York >>From b107bc7c41c3b6dcb714e558890b37f1e3b4bacd Mon Sep 17 00:00:00 2001 >From: Simo Sorce >Date: Wed, 25 Jul 2012 11:23:11 -0400 >Subject: [PATCH] Add all external samba libraries to BuildRequires > >Also move them in the right spot (if ! only client) so that they are required >only when building the server. >--- > freeipa.spec.in | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > >diff --git a/freeipa.spec.in b/freeipa.spec.in >index 45149c65f2333512fe21e40be903a3e6144d7084..ba5707ab0ce067a81e0849133820401b439ca13a 100644 >--- a/freeipa.spec.in >+++ b/freeipa.spec.in >@@ -33,6 +33,10 @@ BuildRequires: systemd-units > %endif > BuildRequires: samba4-devel >= 4.0.0-128 > BuildRequires: samba4-python >+BuildRequires: libtalloc-devel >+BuildRequires: libtevent-devel >+BuildRequires: libtdb-devel >+BuildRequires: libldb-devel ACK if you remove libtdb-devel/libldb-devel as explained in a separate mail. -- / Alexander Bokovoy From simo at redhat.com Wed Jul 25 17:35:11 2012 From: simo at redhat.com (Simo Sorce) Date: Wed, 25 Jul 2012 13:35:11 -0400 Subject: [Freeipa-devel] [PATCH] fix buildrequires in spec file In-Reply-To: <20120725173202.GF3283@redhat.com> References: <1343230108.2666.9.camel@willson.li.ssimo.org> <20120725173202.GF3283@redhat.com> Message-ID: <1343237711.2666.10.camel@willson.li.ssimo.org> On Wed, 2012-07-25 at 20:32 +0300, Alexander Bokovoy wrote: > On Wed, 25 Jul 2012, Simo Sorce wrote: > >This has grown to more than a one-liner so I seek an active ack. > > > >Simo. > > > >-- > >Simo Sorce * Red Hat, Inc * New York > > >>From b107bc7c41c3b6dcb714e558890b37f1e3b4bacd Mon Sep 17 00:00:00 2001 > >From: Simo Sorce > >Date: Wed, 25 Jul 2012 11:23:11 -0400 > >Subject: [PATCH] Add all external samba libraries to BuildRequires > > > >Also move them in the right spot (if ! only client) so that they are required > >only when building the server. > >--- > > freeipa.spec.in | 5 ++++- > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > >diff --git a/freeipa.spec.in b/freeipa.spec.in > >index 45149c65f2333512fe21e40be903a3e6144d7084..ba5707ab0ce067a81e0849133820401b439ca13a 100644 > >--- a/freeipa.spec.in > >+++ b/freeipa.spec.in > >@@ -33,6 +33,10 @@ BuildRequires: systemd-units > > %endif > > BuildRequires: samba4-devel >= 4.0.0-128 > > BuildRequires: samba4-python > >+BuildRequires: libtalloc-devel > >+BuildRequires: libtevent-devel > >+BuildRequires: libtdb-devel > >+BuildRequires: libldb-devel > ACK if you remove libtdb-devel/libldb-devel as explained in a separate > mail. > Pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From rcritten at redhat.com Wed Jul 25 20:58:25 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Wed, 25 Jul 2012 16:58:25 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <5010035A.2060108@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> <500FFC4F.8090601@redhat.com> <5010035A.2060108@redhat.com> Message-ID: <50105DF1.5030507@redhat.com> Jan Cholasta wrote: > Dne 25.7.2012 16:01, Rob Crittenden napsal(a): >> Petr Viktorin wrote: >>> On 07/23/2012 10:03 PM, Rob Crittenden wrote: >>>> Rob Crittenden wrote: >>>>> Andrew Wnuk wrote: >>>>>> On 07/16/2012 01:35 PM, Rob Crittenden wrote: >>>>>>> Nalin Dahyabhai wrote: >>>>>>>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>>>>>>> Use the new certmonger capability to be able to renew the dogtag >>>>>>>>> subsystem certificates (audit, OCSP, etc). >>>>>>>> >>>>>>>> Are the copies of the certificates in the pki-ca CS.cfg file being >>>>>>>> updated elsewhere? Or is it not turning out to be a problem if >>>>>>>> they >>>>>>>> aren't? >>>>>>> >>>>>>> I didn't test validating OCSP signatures but the audit subsystem >>>>>>> seemed fine (it complained wildly when I had the wrong trust in the >>>>>>> NSS db). >>>>>>> >>>>>>> Andrew, do I need to update CS.cfg as well? >>>>>>> >>>>>> Yes, you may need update CS.cfg too. >>>>> >>>>> Ok, added a bit to update CS.cfg with the new certificate. >>>> >>>> This should fix some SELinux issues preventing certmonger from >>>> monitoring the dogtag certificate database in /var/lib/pki-ca/alias. >>>> >>>> rob >>> >>> I don't know enough about dogtag/certmonger to comment on the >>> functionality, but there are minor issues I can find. Attaching a patch >>> to fix them. >>> >>> >>> `make rpms` fails: >>> >>> rpmbuild --define "_topdir /rpmbuild" -ba freeipa.spec >>> error: %changelog not in descending chronological order >>> make: *** [rpms] Error 1 >>> >>> >>> >>> `git am` complains: >>> >>> Applying: Use certmonger to renew CA subsystem certificates >>> /home/pviktori/freeipa/.git/rebase-apply/patch:576: new blank line at >>> EOF. >>> + >>> /home/pviktori/freeipa/.git/rebase-apply/patch:645: new blank line at >>> EOF. >>> + >>> warning: 2 lines add whitespace errors. >> >> Thanks, integrated this patch and added a missing script, renew_ipacert. >> >> rob >> > > NACK > > > First, a question: I haven't tested this (yet), but what happens when > someone uses the --{dirsrv,http,pkinit}_pkcs12 options of > ipa-server-install/ipa-replica-prepare? (There are also other options > which I suspect may cause trouble, namely --subject and --selfsign.) CA certs aren't tracked if --selfsign is used. subject doesn't matter, it is unrelated to renewal. The provided PKCS#12 files are unrelated to this patch, but in general we will still attempt to renew the dirsrv and http certs. We automatically track all certs using the IPA CA. If they were not issued by the IPA CA then they will fail to be renewed. I'm thinking we need to deprecate ipa-server-certs and document that using the PKCS#12 options is unsupported. > > install/restart_scripts/renew_ra_cert doesn't seem to be used anywhere. This replaces the ipaCert script. I fixed up the invocation. > ipa-replica-install --setup-ca fails with: > > ... > [13/15]: configure clone certificate renewals > > Your system may be partly configured. > Run /usr/sbin/ipa-server-install --uninstall to clean up. > > Nickname "ipaCert" doesn't exist in NSS database "/etc/httpd/alias" Fixed. > On clones, the CN=IPA RA,O=REALM certificate is tracked with post-save > command '/usr/lib64/ipa/certmonger/restart_httpd "ipaCert"', but > restart_httpd does not take any arguments (it does not break anything, > it's just weird). That's true, suppressed. > > Comments on individual files follow: > > > install/certmonger/Makefile.am: > > Missing closing parenthesis: > > +EXTRA_DIST = \ > + $(app_SCRIPTS \ Fixed, and replaced some spaces with tabs. > > install/certmonger/dogtag-ipa-retrieve-agent-submit: > > Typo ("nicknamd"): Fixed. > Are these guaranteed to be upper-case? I'd put operation.upper() here, > just to be on the safe side: Yes, guaranteed to be upper-case from certmonger. > This except block is not necessary, unhandled exceptions are caught in > the except block lower in the code: > > + sys.exit(5) > + except Exception, e: > + # Unhandled error > + sys.exit(3) > + finally: Sure, removed. I had it there for readability but it is such little code it can still be followed. > > > install/restart_scripts/restart_dirsrv: > > You import and initialize api, but then don't use it. Ah, I did at one point, then ripped it all out (or almost all, apparently). Gone. > > install/restart_scripts/*: > > All these scripts could use more exception handling, but I guess > potential bugs can be sorted out later. Well, they all run in the background so even if they caught errors nothing would see them unless we decide to syslog errors. > > install/share/default-aci.ldif: > > The ACIs are wrong (Kerberos principal instead of ldap URI in userdn, in > 40-delegation.update it is done right). Nice catch, not sure how I missed that. Fixed. > > ipapython/certmonger.py: > > This is ugly: > > + if sys.maxsize > 2**32: > + libpath = 'lib64' > + else: > + libpath = 'lib' I'm open to suggestions, it's the best thing I could find. > Is it safe to show the PIN in "getcert -P " in logs? If not, please > add an appropriate nolog argument to ipautil.run. Probably best to be safe, suppressed. > > ipapython/platform/fedora16.py > > Can't we pick one name for pki-cad/pki_cad and use only that? I added it so we could do calls like: ipaservices.knownservices.pki_cad.restart(instance). No dashes allowed in Python names. The actual service name is pki-cad, so my intention was to make it an alias. > > selinux/ipa_dogtag/ipa_dogtag.te: > > Please use tabs here instead of spaces: > > + class file read; > + class file getattr; > + class file open; Sure, fixed. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1033-5-renewal.patch Type: text/x-diff Size: 45805 bytes Desc: not available URL: From jaramire at redhat.com Thu Jul 26 06:01:23 2012 From: jaramire at redhat.com (Javier Ramirez) Date: Thu, 26 Jul 2012 02:01:23 -0400 (EDT) Subject: [Freeipa-devel] Freeipa wiki editing In-Reply-To: <1972012914.645763.1343282362829.JavaMail.root@redhat.com> Message-ID: <1144308720.645889.1343282483657.JavaMail.root@redhat.com> Hi, As per the instructions found at http://freeipa.com/page/Contribute , I send this email to request for a freeipa wiki account . I have some amends to make to http://freeipa.com/page/ConfiguringAixClients . Thanks. From sbingram at gmail.com Thu Jul 26 06:18:54 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Wed, 25 Jul 2012 23:18:54 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <500FD914.7040908@redhat.com> References: <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> <500FD914.7040908@redhat.com> Message-ID: On Wed, Jul 25, 2012 at 4:31 AM, John Dennis wrote: > On 07/25/2012 02:59 AM, Stephen Ingram wrote: >> >> Seeing python2.7, I'm guessing these patches were against Fedora. >> Since I couldn't get them to apply against RH 6.3 I applied them by >> hand. I couldn't get the WebUI to load after applying the patches. I'm >> not sure of the code that caused the problem, but I did see mention of >> global name datetime not being defined. You wouldn't happen to have >> patches for RH 6.3? > > > Sorry, I thought you had a F17 install. It should be easy to fix the > datetime issue, just add this add the top of the file: > > import time, datetime > > If that simple fix doesn't work then let me know the version you've got and > I'll make up a new patch against that version. Yes, please send a patch for 2.20 with RH 6.3. The import time, datetime did not address all of the errors. Steve From atkac at redhat.com Thu Jul 26 08:06:47 2012 From: atkac at redhat.com (Adam Tkac) Date: Thu, 26 Jul 2012 10:06:47 +0200 Subject: [Freeipa-devel] [PATCH 0041] Cleanup in logging code In-Reply-To: <500FF536.8040308@redhat.com> References: <500FF536.8040308@redhat.com> Message-ID: <20120726080643.GA1308@redhat.com> On Wed, Jul 25, 2012 at 03:31:34PM +0200, Petr Spacek wrote: > Hello, > > this patch clears logging code a bit. Adding functions like > log_info() and similar will be trivial from now. > > It will be necessary for ticket #71: Log successful reconnect > https://fedorahosted.org/bind-dyndb-ldap/ticket/71 Ack. > From 26136d6fe5fce5ac4f3138063bcf4774f268bd3c Mon Sep 17 00:00:00 2001 > From: Petr Spacek > Date: Thu, 19 Jul 2012 14:13:12 +0200 > Subject: [PATCH] Cleanup in logging code. > > Signed-off-by: Petr Spacek > --- > src/log.c | 22 ++-------------------- > src/log.h | 19 ++++++++++++++++--- > 2 files changed, 18 insertions(+), 23 deletions(-) > > diff --git a/src/log.c b/src/log.c > index b23e4720a8dd484a65d8a7e6c58baf257fc9ce50..f731df706b58e1f894659811dae32d4148a8620c 100644 > --- a/src/log.c > +++ b/src/log.c > @@ -28,31 +28,13 @@ > #include "log.h" > > void > -log_debug(int level, const char *format, ...) > +log_write(int level, const char *format, ...) > { > va_list args; > > va_start(args, format); > -#ifdef LOG_AS_ERROR > - UNUSED(level); > isc_log_vwrite(dns_lctx, DNS_LOGCATEGORY_DATABASE, DNS_LOGMODULE_DYNDB, > - ISC_LOG_ERROR, format, args); > -#else > - isc_log_vwrite(dns_lctx, DNS_LOGCATEGORY_DATABASE, DNS_LOGMODULE_DYNDB, > - ISC_LOG_DEBUG(level), format, args); > -#endif > - > - va_end(args); > -} > - > -void > -log_error(const char *format, ...) > -{ > - va_list args; > - > - va_start(args, format); > - isc_log_vwrite(dns_lctx, DNS_LOGCATEGORY_DATABASE, DNS_LOGMODULE_DYNDB, > - ISC_LOG_ERROR, format, args); > + level, format, args); > va_end(args); > > } > diff --git a/src/log.h b/src/log.h > index 0df4e25618fab932bdec97c276580d1b9d31bf08..898639be144dbf6049a1440493c3358e01a5c2dd 100644 > --- a/src/log.h > +++ b/src/log.h > @@ -22,18 +22,31 @@ > #define _LD_LOG_H_ > > #include > +#include > + > +#ifdef LOG_AS_ERROR > +#define GET_LOG_LEVEL(level) ISC_LOG_ERROR > +#else > +#define GET_LOG_LEVEL(level) (level) > +#endif > > #define fatal_error(...) \ > isc_error_fatal(__FILE__, __LINE__, __VA_ARGS__) > > #define log_bug(fmt, ...) \ > log_error("bug in %s(): " fmt, __func__,##__VA_ARGS__) > > #define log_error_r(fmt, ...) \ > - log_error(fmt ": %s", ##__VA_ARGS__, isc_result_totext(result)) > + log_error(fmt ": %s", ##__VA_ARGS__, dns_result_totext(result)) > > /* Basic logging functions */ > -void log_debug(int level, const char *format, ...) ISC_FORMAT_PRINTF(2, 3); > -void log_error(const char *format, ...) ISC_FORMAT_PRINTF(1, 2); > +#define log_error(format, ...) \ > + log_write(GET_LOG_LEVEL(ISC_LOG_ERROR), format, ##__VA_ARGS__) > + > +#define log_debug(level, format, ...) \ > + log_write(GET_LOG_LEVEL(level), format, ##__VA_ARGS__) > + > +void > +log_write(int level, const char *format, ...) ISC_FORMAT_PRINTF(2, 3); > > #endif /* !_LD_LOG_H_ */ > -- > 1.7.10.4 > -- Adam Tkac, Red Hat, Inc. From pspacek at redhat.com Thu Jul 26 08:12:34 2012 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 26 Jul 2012 10:12:34 +0200 Subject: [Freeipa-devel] [PATCH 0041] Cleanup in logging code In-Reply-To: <20120726080643.GA1308@redhat.com> References: <500FF536.8040308@redhat.com> <20120726080643.GA1308@redhat.com> Message-ID: <5010FBF2.3070509@redhat.com> On 07/26/2012 10:06 AM, Adam Tkac wrote: > On Wed, Jul 25, 2012 at 03:31:34PM +0200, Petr Spacek wrote: >> Hello, >> >> this patch clears logging code a bit. Adding functions like >> log_info() and similar will be trivial from now. >> >> It will be necessary for ticket #71: Log successful reconnect >> https://fedorahosted.org/bind-dyndb-ldap/ticket/71 > > Ack. > Pushed to the master: http://git.fedorahosted.org/git?p=bind-dyndb-ldap.git;a=commitdiff;h=b04dfcbe328a8e713597921f7a43c9c8dd801e63 Petr^2 Spacek From pviktori at redhat.com Thu Jul 26 08:17:14 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 26 Jul 2012 10:17:14 +0200 Subject: [Freeipa-devel] [PATCH] 0073 Update translations Message-ID: <5010FD0A.3090206@redhat.com> Update the pot to match current source, and pull translations from Transifex. Warning: when this patch is pushed, the source strings on Transifex will update. The old versions will be lost from the site. The patch is quite large (>5MB), so I haven't attached it here (should I?). Please download it from https://github.com/encukou/freeipa/commit/fd638306ada102204494ec2e7f1b8d2bb7f6f8b1.patch -- Petr? From abokovoy at redhat.com Thu Jul 26 13:16:33 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Thu, 26 Jul 2012 16:16:33 +0300 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket Message-ID: <20120726131633.GA3324@redhat.com> Hi, When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 This ticket also simplifies a bit the way we handle admin connection in Service class and particulary in Service._ldap_mod() by defaulting to LDAPI/autobind in case of running as root and to GSSAPI otherwise. Except few cases in remote replica management (not applicable in _ldap_mod() case) we always run installation tools as root and can benefit from using autobind feature. Unfortunately, it is not yet possible to get away from using DM credentials for all cases as the same class is used to perform initial directory server instance configuration. One side effect is explicit disconnect and reconnect in Service.add_cert_to_service() due to way how SimpleLDAPObject class handles stale connections (no handling at all). I've put some comments in place so that others would not try to err out optimizing it in future. Finally, with next patch series which will introduce syncing ipaNTHash attribute with RC4 key in existing kerberos credentials, we can remove requirements to change passwords or re-kinit for majority of trust cases. This should then conclude our trusts content for beta2 release. -- / Alexander Bokovoy -------------- next part -------------- >From 93a84dff75560e297e775b838fc12330ebd218e5 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/2] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 42 +++++++++++++++++++++-------- install/tools/man/ipa-adtrust-install.1 | 3 --- ipaserver/install/adtrustinstance.py | 21 ++++++++------- ipaserver/install/service.py | 44 ++++++++++++++++++++++--------- 4 files changed, 74 insertions(+), 36 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..f367d5b2b516bd411bce9275ff299eb3ffdf6bf9 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -86,6 +84,30 @@ def read_netbios_name(netbios_default): return netbios_name +def ensure_kerberos_admin_rights(api): + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + api.Backend.ldap2.connect(ccache.name) + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + api.Backend.ldap2.disconnect() + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except Exception, e: + error_messages = dict( + ACIError = "Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket", + Krb5Error = "Must have Kerberos credentials to setup AD trusts on server", + RequirementError = "Must have administrative privileges to setup AD trusts on server" + ) + name = type(e).__name__ + if name in error_messages: + sys.exit(error_messages[name]) + else: + sys.exit("Unrecognized error during check of admin rights: %s\n%s" % (name, str(e))) + def main(): safe_options, options = parse_options() @@ -128,6 +150,8 @@ def main(): api.bootstrap(**cfg) api.finalize() + ensure_kerberos_admin_rights(api) + if adtrustinstance.ipa_smb_conf_exists(): if not options.unattended: while True: @@ -194,9 +218,8 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm # try the connection try: @@ -205,12 +228,9 @@ def main(): except ldap.INVALID_CREDENTIALS, e: sys.exit("Password is not valid!") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + # See if our LDAP server is up and we can talk to it over GSSAPI + ccache = krbV.default_context().default_ccache().name + api.Backend.ldap2.connect(ccache) smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..4fd7f841a6cbb04e49eba8a3ae755a688924c010 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -55,7 +55,7 @@ def print_msg(message, output_fd=sys.stdout): class Service(object): - def __init__(self, service_name, sstore=None, dm_password=None, ldapi=False): + def __init__(self, service_name, sstore=None, dm_password=None, ldapi=True): self.service_name = service_name self.service = ipaservices.service(service_name) self.steps = [] @@ -77,12 +77,13 @@ class Service(object): self.dercert = None def ldap_connect(self): - if self.ldapi: - if not self.realm: - raise RuntimeError('realm must be set to use ldapi connection') + # Always connect with LDAPI when realm is known + # the code in Service class always connects to the LDAP on the same + # machine so we can use LDAPI safely if realm is configured + if self.realm: self.admin_conn = self.__get_conn(None, None, ldapi=True, realm=self.realm) else: - self.admin_conn = self.__get_conn(self.fqdn, self.dm_password) + self.admin_conn = self.__get_conn(self.fqdn, self.dm_password, ldapi=False) def ldap_disconnect(self): self.admin_conn.unbind() @@ -93,7 +94,6 @@ class Service(object): pw_name = None fd = None path = ipautil.SHARE_DIR + ldif - hostname = installutils.get_fqdn() nologlist=[] if sub_dict is not None: @@ -107,15 +107,25 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # As we always connect to the local host, + # use URI of admin connection + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + + auth_params = [] if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + # always try GSSAPI auth when not using DM password or not being root + if os.getegid() != 0: + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: @@ -181,8 +191,19 @@ class Service(object): This server cert should be in DER format. """ - if not self.admin_conn: - self.ldap_connect() + # add_cert_to_service() is relatively rare operation + # we actually call it twice during ipa-server-install, for different + # instances: ds and cs. Unfortunately, it may happen that admin + # connection was created well before add_cert_to_service() is called + # If there are other operations in between, it will become stale and + # since we are using SimpleLDAPObject, not ReconnectLDAPObject, the + # action will fail. Thus, explicitly disconnect and connect again. + # Using ReconnectLDAPObject instead of SimpleLDAPObject was considered + # but consequences for other parts of the framework are largerly + # unknown. + if self.admin_conn: + self.ldap_disconnect() + self.ldap_connect() dn = "krbprincipalname=%s,cn=services,cn=accounts,%s" % (self.principal, self.suffix) mod = [(ldap.MOD_ADD, 'userCertificate', self.dercert)] @@ -272,14 +293,13 @@ class Service(object): # If we are passed a password we'll use it as the DM password # otherwise we'll do a GSSAPI bind. try: -# conn = ipaldap.IPAdmin(fqdn, port=636, cacert=CACERT) if ldapi: conn = ipaldap.IPAdmin(ldapi=ldapi, realm=realm) else: conn = ipaldap.IPAdmin(fqdn, port=389) if dm_password: conn.do_simple_bind(bindpw=dm_password) - elif os.getegid() == 0 and self.ldapi: + elif os.getegid() == 0 and ldapi: try: # autobind pw_name = pwd.getpwuid(os.geteuid()).pw_name -- 1.7.10.4 From pviktori at redhat.com Thu Jul 26 13:49:21 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 26 Jul 2012 15:49:21 +0200 Subject: [Freeipa-devel] [PATCH] 0073 Update translations In-Reply-To: <5010FD0A.3090206@redhat.com> References: <5010FD0A.3090206@redhat.com> Message-ID: <50114AE1.4090209@redhat.com> On 07/26/2012 10:17 AM, Petr Viktorin wrote: > Update the pot to match current source, and pull translations from > Transifex. > > > Warning: when this patch is pushed, the source strings on Transifex will > update. The old versions will be lost from the site. > > > The patch is quite large (>5MB), so I haven't attached it here (should > I?). Please download it from > https://github.com/encukou/freeipa/commit/fd638306ada102204494ec2e7f1b8d2bb7f6f8b1.patch > > NACK. I forgot to validate the messages, so some messages might cause exceptions when used. -- Petr? From pviktori at redhat.com Thu Jul 26 14:57:43 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Thu, 26 Jul 2012 16:57:43 +0200 Subject: [Freeipa-devel] [PATCH] 0073 Update translations In-Reply-To: <50114AE1.4090209@redhat.com> References: <5010FD0A.3090206@redhat.com> <50114AE1.4090209@redhat.com> Message-ID: <50115AE7.8050401@redhat.com> On 07/26/2012 03:49 PM, Petr Viktorin wrote: > On 07/26/2012 10:17 AM, Petr Viktorin wrote: >> Update the pot to match current source, and pull translations from >> Transifex. >> >> >> Warning: when this patch is pushed, the source strings on Transifex will >> update. The old versions will be lost from the site. >> >> >> The patch is quite large (>5MB), so I haven't attached it here (should >> I?). Please download it from >> https://github.com/encukou/freeipa/commit/fd638306ada102204494ec2e7f1b8d2bb7f6f8b1.patch >> >> >> > > NACK. I forgot to validate the messages, so some messages might cause > exceptions when used. > > Validation found a few bad messages in Spanish, and one in Tajik. I've corrected obvious errors; in two cases I wasn't sure how to fix so I've removed the translation. Attaching my fixes for reference. The new patch is here: https://github.com/encukou/freeipa/commit/4eb5b21fe3bcfec33e07c45050d5a56f4328206c.patch We should also change these on Transifex. I assume maintainers can edit all translations. John, could you give me maintainer rights? My username there is EnCuKou. -- Petr? -------------- next part -------------- A non-text attachment was scrubbed... Name: Fix-or-remove-bad-translations.patch Type: text/x-patch Size: 2378 bytes Desc: not available URL: From jcholast at redhat.com Thu Jul 26 16:48:26 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Thu, 26 Jul 2012 18:48:26 +0200 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <50105DF1.5030507@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> <500FFC4F.8090601@redhat.com> <5010035A.2060108@redhat.com> <50105DF1.5030507@redhat.com> Message-ID: <501174DA.9010909@redhat.com> Dne 25.7.2012 22:58, Rob Crittenden napsal(a): > Jan Cholasta wrote: >> Dne 25.7.2012 16:01, Rob Crittenden napsal(a): >>> Petr Viktorin wrote: >>>> On 07/23/2012 10:03 PM, Rob Crittenden wrote: >>>>> Rob Crittenden wrote: >>>>>> Andrew Wnuk wrote: >>>>>>> On 07/16/2012 01:35 PM, Rob Crittenden wrote: >>>>>>>> Nalin Dahyabhai wrote: >>>>>>>>> On Mon, Jul 16, 2012 at 09:23:24AM -0400, Rob Crittenden wrote: >>>>>>>>>> Use the new certmonger capability to be able to renew the dogtag >>>>>>>>>> subsystem certificates (audit, OCSP, etc). >>>>>>>>> >>>>>>>>> Are the copies of the certificates in the pki-ca CS.cfg file being >>>>>>>>> updated elsewhere? Or is it not turning out to be a problem if >>>>>>>>> they >>>>>>>>> aren't? >>>>>>>> >>>>>>>> I didn't test validating OCSP signatures but the audit subsystem >>>>>>>> seemed fine (it complained wildly when I had the wrong trust in the >>>>>>>> NSS db). >>>>>>>> >>>>>>>> Andrew, do I need to update CS.cfg as well? >>>>>>>> >>>>>>> Yes, you may need update CS.cfg too. >>>>>> >>>>>> Ok, added a bit to update CS.cfg with the new certificate. >>>>> >>>>> This should fix some SELinux issues preventing certmonger from >>>>> monitoring the dogtag certificate database in /var/lib/pki-ca/alias. >>>>> >>>>> rob >>>> >>>> I don't know enough about dogtag/certmonger to comment on the >>>> functionality, but there are minor issues I can find. Attaching a patch >>>> to fix them. >>>> >>>> >>>> `make rpms` fails: >>>> >>>> rpmbuild --define "_topdir /rpmbuild" -ba freeipa.spec >>>> error: %changelog not in descending chronological order >>>> make: *** [rpms] Error 1 >>>> >>>> >>>> >>>> `git am` complains: >>>> >>>> Applying: Use certmonger to renew CA subsystem certificates >>>> /home/pviktori/freeipa/.git/rebase-apply/patch:576: new blank line at >>>> EOF. >>>> + >>>> /home/pviktori/freeipa/.git/rebase-apply/patch:645: new blank line at >>>> EOF. >>>> + >>>> warning: 2 lines add whitespace errors. >>> >>> Thanks, integrated this patch and added a missing script, renew_ipacert. >>> >>> rob >>> >> >> NACK >> >> >> First, a question: I haven't tested this (yet), but what happens when >> someone uses the --{dirsrv,http,pkinit}_pkcs12 options of >> ipa-server-install/ipa-replica-prepare? (There are also other options >> which I suspect may cause trouble, namely --subject and --selfsign.) > > CA certs aren't tracked if --selfsign is used. > > subject doesn't matter, it is unrelated to renewal. > > The provided PKCS#12 files are unrelated to this patch, but in general > we will still attempt to renew the dirsrv and http certs. We > automatically track all certs using the IPA CA. If they were not issued > by the IPA CA then they will fail to be renewed. OK, thanks. > > I'm thinking we need to deprecate ipa-server-certs and document that > using the PKCS#12 options is unsupported. > >> >> install/restart_scripts/renew_ra_cert doesn't seem to be used anywhere. > > This replaces the ipaCert script. I fixed up the invocation. > >> ipa-replica-install --setup-ca fails with: >> >> ... >> [13/15]: configure clone certificate renewals >> >> Your system may be partly configured. >> Run /usr/sbin/ipa-server-install --uninstall to clean up. >> >> Nickname "ipaCert" doesn't exist in NSS database "/etc/httpd/alias" > > Fixed. > >> On clones, the CN=IPA RA,O=REALM certificate is tracked with post-save >> command '/usr/lib64/ipa/certmonger/restart_httpd "ipaCert"', but >> restart_httpd does not take any arguments (it does not break anything, >> it's just weird). > > That's true, suppressed. > >> >> Comments on individual files follow: >> >> >> install/certmonger/Makefile.am: >> >> Missing closing parenthesis: >> >> +EXTRA_DIST = \ >> + $(app_SCRIPTS \ > > Fixed, and replaced some spaces with tabs. > >> >> install/certmonger/dogtag-ipa-retrieve-agent-submit: >> >> Typo ("nicknamd"): > > Fixed. > >> Are these guaranteed to be upper-case? I'd put operation.upper() here, >> just to be on the safe side: > > Yes, guaranteed to be upper-case from certmonger. > >> This except block is not necessary, unhandled exceptions are caught in >> the except block lower in the code: >> >> + sys.exit(5) >> + except Exception, e: >> + # Unhandled error >> + sys.exit(3) >> + finally: > > Sure, removed. I had it there for readability but it is such little code > it can still be followed. > >> >> >> install/restart_scripts/restart_dirsrv: >> >> You import and initialize api, but then don't use it. > > Ah, I did at one point, then ripped it all out (or almost all, > apparently). Gone. > >> >> install/restart_scripts/*: >> >> All these scripts could use more exception handling, but I guess >> potential bugs can be sorted out later. > > Well, they all run in the background so even if they caught errors > nothing would see them unless we decide to syslog errors. > >> >> install/share/default-aci.ldif: >> >> The ACIs are wrong (Kerberos principal instead of ldap URI in userdn, in >> 40-delegation.update it is done right). > > Nice catch, not sure how I missed that. Fixed. You forgot to fix the allow(add) one, it still has userdn = "host/$FQDN@$REALM". > >> >> ipapython/certmonger.py: >> >> This is ugly: >> >> + if sys.maxsize > 2**32: >> + libpath = 'lib64' >> + else: >> + libpath = 'lib' > > I'm open to suggestions, it's the best thing I could find. OK, it is mentioned in , so it is probably safe. However, I think that in future it might be nice to have a config.py file (perhaps autoconf generated?) with system paths as constants. > >> Is it safe to show the PIN in "getcert -P " in logs? If not, please >> add an appropriate nolog argument to ipautil.run. > > Probably best to be safe, suppressed. > >> >> ipapython/platform/fedora16.py >> >> Can't we pick one name for pki-cad/pki_cad and use only that? > > I added it so we could do calls like: > > ipaservices.knownservices.pki_cad.restart(instance). No dashes allowed > in Python names. > > The actual service name is pki-cad, so my intention was to make it an > alias. OK, makes sense. > >> >> selinux/ipa_dogtag/ipa_dogtag.te: >> >> Please use tabs here instead of spaces: >> >> + class file read; >> + class file getattr; >> + class file open; > > Sure, fixed. > > rob I did: 1. ipa-server-install on host1, using IPA from master 2. ipa-replica-install on host2, using IPA from master 3. update host1 to IPA with your patch applied 4. update host2 to IPA with your patch applied 5. ipa-ca-install on host2 After that, ipaCert is not tracked on host2 at all (I had to add it manually using "getcert start-tracking -d /etc/httpd/alias -n ipaCert -c dogtag-ipa-retrieve-agent-submit -C /usr/lib64/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt -T ipaCert"). (to be continued) Honza -- Jan Cholasta From ssorce at redhat.com Thu Jul 26 18:35:34 2012 From: ssorce at redhat.com (Simo Sorce) Date: Thu, 26 Jul 2012 14:35:34 -0400 (EDT) Subject: [Freeipa-devel] [PATCH] Minor fix to sidgen plugin In-Reply-To: <492182885.3890857.1343327699742.JavaMail.root@redhat.com> Message-ID: <1964262976.3891605.1343327734632.JavaMail.root@redhat.com> Remove an unnecessary check that may give spurious failures on modified server where 999 is a valid ID. -- Simo Sorce * Red Hat, Inc. * New York -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-simo-498-0001-1-Do-not-check-for-DNA-magic-values.patch Type: text/x-patch Size: 2124 bytes Desc: not available URL: From jdennis at redhat.com Thu Jul 26 19:59:28 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 26 Jul 2012 15:59:28 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FFD3277.9070607@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFD3277.9070607@redhat.com> Message-ID: <5011A1A0.20504@redhat.com> On 07/11/2012 03:59 AM, Petr Viktorin wrote: > I went over the changes to ipaserver/plugins/ldap2.py; now I can start > the testing. > > I must ask about the wrapped _ext methods. I assume python-ldap exposes > them only to mimic the C interface, which has them because C doesn't > have default arguments. Would we lose anything if the class only defined > the _ext methods, and aliased the non-ext variants to them? e.g. > > def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): > assert isinstance(dn, DN) > dn = str(dn) > modlist = self.encode(modlist) > return self.conn.add_ext(dn, modlist, serverctrls, clientctrls) > add = add_ext > > The non-_ext variants are deprecated in the underlying C library, anyway. With the current implementation you are correct. There would be no functional difference if we only utilized the _ext variants. However that means employing knowledge from the wrong side of an API boundary. From our perspective python-ldap exposes certain methods. In particular we use the simpler methods because that is sufficient to meet our needs. I think we should code to the defined API. If python-ldap ever changes or we use an alternate Python ldap library the porting effort would be much cleaner if we don't try to outsmart ourselves. > Why are some of the wrapper methods left out? (compare, delete_ext, > modify, etc.) The set of wrapped methods was chosen based on what we actually use in our code. Originally I set out out wrap every method, but that was a fair amount of work most of which was completely unnecessary because it would never get invoked. Why make it more complicated than it needs to be? If in the future we decide we need to use another method(s) we can add the wrapper on an as-needed basis. > > > Line 1519: > checkmembers = copy.deepcopy(members) > for member_dn in checkmembers: > ... > if m not in checkmembers: > checkmembers.append(m) # FIXME jrd; can't modify list > during traversal > NACK. You really can't modify lists during traversal, a FIXME won't cut it. Absolutely, I didn't write the code but when I saw it I realized it had to get fixed, the FIXME was simply a reminder to go back and clean up previously incorrect code I had spotted but had existed for a long time. Your solution below is pretty close to what I was planning to add. Fixed. > A smaller issue is that the list `in` operator does alinear scan, so > calling it in a loop can very slow as the list grows. Sets are much > better for membership tests. > So I believe you want something like: > > checkmembers = set(DN(m, locked=True) for m in members) > checked = set() > while checkmembers: > member_dn = checkmembers.pop() > checked.add(member_dn) > ... > if m not in checked: > checkmembers.add(m) > > > > Lines 203, 825: > os.environ['KRB5CCNAME'] = ccache_file > Should KRB5CCNAME be unset/restored once the code's done with it. Agreed, once again I didn't write this but saw the problem when I moved the code block and made a note to fix it in the future. This particular place is fixed now. However I think we've got other places that do similar things. This is another example of the tendency to cut-n-paste blocks of duplicate code which drives me nuts. But how much unrelated code does one change in any patch? I did however add a FIXME comment to get that code cleaned up. In the past I would change any problem I saw when I touched an area of the code but I got a lot of complaints about changing things not directly related to the intent of the patch :-( FWIW, the setting of KRB5CCNAME on line 825 is somewhat correct. It's the identity for the duration of the request. However it's redundant with what the session code sets up . ldap2 really shouldn't be doing this, ldap2 needs some clean up. FWIW the KRBCCNAME is removed by the session release_ipa_ccache() method. > > > Line 228: > except IndexError: > The try clause is too long for such a general exception, this can hide > real errors. Agreed, I didn't write the code, it needs clean up, how many non-dn related changes do I make? > > > > ==== > > And as usual I have a lot of nitpicks. Sometimes I just want to share my > ideas about how good code should look like, don't take them too > seriously if you think otherwise :) > > > Line 107 & 127: > return utf8_codec.decode(val)[0] > return utf8_codec.encode(unicode(val))[0] > I believe the following would be more readable: > return val.decode('utf-8') > return unicode(val).encode('utf-8') Yes, your suggestion is easier to read, but it's more efficient to lookup the codec once rather than everytime you decode. > > > Line 134: > class ServerSchema(object): > > Don't use nested classes without reason addressed previously > > Line 148: > def get_schema(self, url, conn=None, force_update=False): > > Is the force_update & flush() useful? None of the code seems to use it. We don't currently use it but we probably should in the future. After we update schema we should refresh our view of it. Just anticipating what will probably be a bug somewhere down the road. At least this is better than what we had and is tending in the right direction. > > > Line 354: > _SCHEMA_OVERRIDE = { > We have a case-insensitive dict class, why not use it here? Excellent suggestion, it's been changed. > > > Lines 388, 750: > def is_dn_syntax(self, attr): > """ > Check the schema to see if the attribute uses DN syntax. > uses_dn_syntax or has_dn_syntax would be a better name. Good suggestion. I made the change. Actually, I wish LDAP attributes were a class, that way the correct comparisons could be automatically applied by Python based on the LDAP syntax of the particular attribute and things like "is_dn" and "is_multivalued" could be a properties of the attribute instance, etc. We are using an object oriented language aren't we? > > > Line 395: > if syntax is None: > return False > return syntax == DN_SYNTAX_OID > The if is redundant. Good catch, absolutely right, fixed. > > > Line 406: > if isinstance(val, bool): > if val: > val = 'TRUE' > else: > val = 'FALSE' > return self.encode(val) > Encoding an ASCII string should be a no-op, why bother doing it? Not sure what you mean by no-op. Do you mean str(val) would return either 'True' or 'False'? In any event we are dependent on the string representation of a boolean being all uppper-case. Not something I invented. > > Line 421: > dct = dict() > for (k, v) in val.iteritems(): > dct[self.encode(k)] = self.encode(v) > return dct > Consider: > dct = dict((self.encode(k), self.encode(v)) for k, v in > val.iteritems()) Why do the work of building a list of tuples when it's not necessary? > Line 437: > if type(target_type) is type and isinstance(original_value, > target_type): > Don't use `type(x) is expected_type` for type checking! Use > isinstance(target_type, type) so you also catch type's subclasses. Good point, fixed. Or > better yet, handle the TypeError that isinstance will raise if you don't > give it a class. > To be honest, I don't think it's worth it to do all this convoluted > checking just to make sure you're not copying unnecessarily. Do you have > a reason I can't see? Because some constructors are expensive and it's good to pay attention to efficiency. > > > Line 594, 604, 613: > ipa_result = self.convert_result(ldap_result) > return ipa_result > Just return the result directly, no need for the extra variable. It was done that way for two reasons. During development it gave me a value to print for debugging purposes and I thought it made the code clearer by indicating what is an IPA value. I don't think the extra variable introduces any significant inefficiencies (in C it would be optimized out, not sure Python's byte compiler does the same). > > Line 876, 894, 1041: > parent_dn -- DN of the parent entry (default '') > The default is None. In any case it's redundant to specify default > values in docstrings, as you can just look at the function itself. > > > Line 1422: > self.handle_errors(e, **{}) > Wouldn't the following work better? > self.handle_errors(e, *[{}]*0) I never liked the way handle_errors was written. The use of the kwds args is odd, convoluted and hard to understand. To be honest I can't parse *[{}]*0 > > > Line 1518: > checkmembers = copy.deepcopy(members) > deepcopy() is rarely what you want; it either copies too much or too > little. The Python developers made a mistake by assigning a short name > to such an ill-specified operation. It's better to do this explicitly. > checkmembers = [DN(m) for m in members] > Or to combine with my previous suggestion, > checkmembers = set(DN(m, locked=True) for m in members) > The use of deepcopy was never necessary, in fact this method had a number of programming errors (i.e. modification during list traversal), all that's been fixed now. > > Line 1609: > indirect = memberof[:] # make a copy of the list > I believe the list() is more readable ?? doesn't need a comment: > indirect = list(memberof) > Yup, more readable but you're bucking the Benevolent Dictator Guido who recommends the use of [:]. Anyway, fixed, list() is absolutely more readable. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Thu Jul 26 20:11:02 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 26 Jul 2012 16:11:02 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5005EB9D.1010003@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFBE66E.9040802@redhat.com> <5005EB9D.1010003@redhat.com> Message-ID: <5011A456.7070403@redhat.com> On 07/17/2012 06:47 PM, John Dennis wrote: >> ipapython/dn.py: >> in docstring: DN(arg0, ..., locked=False, first_key_match=True) >> followed by: def __init__(self, *args, **kwds): >> and: kwds.get('first_key_match', True) >> >> I don't see the reason for this. Just write `def __init__(self, *args, >> locked=False, first_key_match=True)` and put a proper summary in the >> first line of the docstring. Same in AVA & RDN. > > A valid point. I think I was trying to be too flexible/extensible. Sorry, I was unable to make the suggested change. I tried and it refreshed my memory as to why it was coded the way it was. Python considers it a syntax error if keyword arguments follow an arg list reference. e.g. >>> def foo(*args, bar=None): File "", line 1 def foo(*args, bar=None): ^ SyntaxError: invalid syntax I'm not sure why, but that's why the methods are coded as def foo(*args, **kwds): If you have a suggestion as to how to use named parameters in this context let me know or maybe explain why it's an illegal construct (I'm sure it's in the language reference documentation somewhere, I just didn't take the time to research it). -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Thu Jul 26 21:15:40 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 26 Jul 2012 17:15:40 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <5011B37C.8030605@redhat.com> The DN patch has been reworked. The single largest change was to the DN implementation and it's unit test to refactor the AVA, RDN and DN classes into a base class that is immutable and a subclass that supports editing. This makes these classes fully consistent with the Python language definition (an issue fully described in http://jdennis.fedorapeople.org/dn_summary.html). In the interest of breaking things out from a large change set for easier review I'm attaching the patch for dn.py and test_dn.py as well as their current implementations after the refactoring. The comprehensive modified DN patch which takes into account the review comments will follow shortly. Modifying the implementation was relatively straight forward because the class implementations were clean and modular making it easy to move things around. Redoing the unit test was much more challenging because of the greater number of combinations. In essence this is what was done: * The AVA, RDN and DN classes are now immutable. Each has been subclassed as EditableAVA, EditableRDN and EditableDN variants which supports modifying the object. * Each class definition has a is_mutable boolean flag. * The logic used to modify an object in-place was moved from the parent class to the Editable subclass * The code to support locking used to prevent object modification was deleted (just use the immutable variants). * Each class assures when it needs to introduce a component object into itself that component is instantiated using a mutable/immutable variant matching it's own mutability. For example DN's are composed of RDN's, when a DN needs to insert a new RDN object into itself (during construction or as an assignment) it uses an RDN if the DN object is immutable or a EditableRDN if it's mutable. Likewise for AVA's. This preserves the mutability of all constituent objects from which the object is composed. * The unit test was modified thusly: - Most tests now iterate over both the immutable and mutable classes performing the same test. - If the test depends on mutability it checks the is_mutable flag and performs a different check - The object and all it's sub-components are now validated to assure they are of the expected mutable/immutable variant. - Tests were added to validate hashing and behavior when used as a key in a dict or as a set member. - Tests were added to assure you can cast a mutable variant into an immutable variant and back again without loss. Bottom line for developers. In 99.99% of the cases just continue to use DN's as you always have. If you need to modify a DN cast it into a EditableDN (e.g. dn = EditableDN(dn)), perform your edit, and if you end up passing the dn outside your scope you should cast it back to an immutable DN (e.g. dn = DN(dn) -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dn.patch Type: text/x-patch Size: 165284 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dn.py Type: text/x-python Size: 64206 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test_dn.py Type: text/x-python Size: 76314 bytes Desc: not available URL: From jdennis at redhat.com Thu Jul 26 21:30:02 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 26 Jul 2012 17:30:02 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <5011B6DA.7020602@redhat.com> The DN patch has been reworked. The single largest change was to the DN implementation and it's unit test to refactor the AVA, RDN and DN classes into a base class that is immutable and a subclass that supports editing. This makes these classes fully consistent with the Python language definition (an issue fully described in http://jdennis.fedorapeople.org/dn_summary.html). In the interest of breaking things out from a large change set for easier review I'm attaching the patch for dn.py and test_dn.py as well as their current implementations after the refactoring. The comprehensive modified DN patch which takes into account the review comments will follow shortly. Modifying the implementation was relatively straight forward because the class implementations were clean and modular making it easy to move things around. Redoing the unit test was much more challenging because of the greater number of combinations. In essence this is what was done: * The AVA, RDN and DN classes are now immutable. Each has been subclassed as EditableAVA, EditableRDN and EditableDN variants which supports modifying the object. * Each class definition has a is_mutable boolean flag. * The logic used to modify an object in-place was moved from the parent class to the Editable subclass * The code to support locking used to prevent object modification was deleted (just use the immutable variants). * Each class assures when it needs to introduce a component object into itself that component is instantiated using a mutable/immutable variant matching it's own mutability. For example DN's are composed of RDN's, when a DN needs to insert a new RDN object into itself (during construction or as an assignment) it uses an RDN if the DN object is immutable or a EditableRDN if it's mutable. Likewise for AVA's. This preserves the mutability of all constituent objects from which the object is composed. * The unit test was modified thusly: - Most tests now iterate over both the immutable and mutable classes performing the same test. - If the test depends on mutability it checks the is_mutable flag and performs a different check - The object and all it's sub-components are now validated to assure they are of the expected mutable/immutable variant. - Tests were added to validate hashing and behavior when used as a key in a dict or as a set member. - Tests were added to assure you can cast a mutable variant into an immutable variant and back again without loss. Bottom line for developers. In 99.99% of the cases just continue to use DN's as you always have. If you need to modify a DN cast it into a EditableDN (e.g. dn = EditableDN(dn)), perform your edit, and if you end up passing the dn outside your scope you should cast it back to an immutable DN (e.g. dn = DN(dn) -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dn.patch Type: text/x-patch Size: 165284 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dn.py Type: text/x-python Size: 64206 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test_dn.py Type: text/x-python Size: 76314 bytes Desc: not available URL: From jdennis at redhat.com Thu Jul 26 21:48:40 2012 From: jdennis at redhat.com (John Dennis) Date: Thu, 26 Jul 2012 17:48:40 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <4FF883DF.4020502@redhat.com> References: <4FF883DF.4020502@redhat.com> Message-ID: <5011BB38.5050701@redhat.com> I have applied the suggested fixes, rebased against master, run all the unit tests successfully, built RPM's, did a full install without errors, and brought up the web UI successfully. The current code can be found here: git clone git://fedorapeople.org/~jdennis/freeipa.dn.git git checkout dn I did not squash the individual commits (but they should be before we apply to master). Please test (again). I continue to believe the greatest lurking liability is the installer code and the individual command line utilities (e.g. replica-manage, etc.) Aside from the server install I have not exercised those components. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From abokovoy at redhat.com Fri Jul 27 04:04:39 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 27 Jul 2012 07:04:39 +0300 Subject: [Freeipa-devel] [PATCH] Minor fix to sidgen plugin In-Reply-To: <1964262976.3891605.1343327734632.JavaMail.root@redhat.com> References: <492182885.3890857.1343327699742.JavaMail.root@redhat.com> <1964262976.3891605.1343327734632.JavaMail.root@redhat.com> Message-ID: <20120727040439.GA13180@redhat.com> On Thu, 26 Jul 2012, Simo Sorce wrote: >Remove an unnecessary check that may give spurious failures on modified >server where 999 is a valid ID. ACK. -- / Alexander Bokovoy From abokovoy at redhat.com Fri Jul 27 04:07:32 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 27 Jul 2012 07:07:32 +0300 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <20120726131633.GA3324@redhat.com> References: <20120726131633.GA3324@redhat.com> Message-ID: <20120727040732.GB13180@redhat.com> On Thu, 26 Jul 2012, Alexander Bokovoy wrote: > Hi, > > When setting up AD trusts support, ipa-adtrust-install utility > needs to be run as: > - root, for performing Samba configuration and using LDAPI/autobind > - kinit-ed IPA admin user, to ensure proper ACIs are granted to > fetch keytab > > As result, we can get rid of Directory Manager credentials in > ipa-adtrust-install > > https://fedorahosted.org/freeipa/ticket/2815 > > This ticket also simplifies a bit the way we handle admin connection in > Service class and particulary in Service._ldap_mod() by defaulting to > LDAPI/autobind in case of running as root and to GSSAPI otherwise. > Except few cases in remote replica management (not applicable in > _ldap_mod() case) we always run installation tools as root and can > benefit from using autobind feature. Unfortunately, it is not yet > possible to get away from using DM credentials for all cases as the same > class is used to perform initial directory server instance > configuration. > > One side effect is explicit disconnect and reconnect in > Service.add_cert_to_service() due to way how SimpleLDAPObject class > handles stale connections (no handling at all). I've put some comments > in place so that others would not try to err out optimizing it in > future. > > Finally, with next patch series which will introduce syncing ipaNTHash > attribute with RC4 key in existing kerberos credentials, we can remove > requirements to change passwords or re-kinit for majority of trust > cases. This should then conclude our trusts content for beta2 release. Patch updated, fixed small typo (auth_parms was initialized as auth_params which led to non-existing auth_parms in ipa-adtrust-install case). -- / Alexander Bokovoy -------------- next part -------------- >From 46391aa5953426d763c0c1b627c8abbf80d6fec2 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/6] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 42 +++++++++++++++++++++-------- install/tools/man/ipa-adtrust-install.1 | 3 --- ipaserver/install/adtrustinstance.py | 21 ++++++++------- ipaserver/install/service.py | 44 ++++++++++++++++++++++--------- 4 files changed, 74 insertions(+), 36 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..f367d5b2b516bd411bce9275ff299eb3ffdf6bf9 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -86,6 +84,30 @@ def read_netbios_name(netbios_default): return netbios_name +def ensure_kerberos_admin_rights(api): + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + api.Backend.ldap2.connect(ccache.name) + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + api.Backend.ldap2.disconnect() + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except Exception, e: + error_messages = dict( + ACIError = "Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket", + Krb5Error = "Must have Kerberos credentials to setup AD trusts on server", + RequirementError = "Must have administrative privileges to setup AD trusts on server" + ) + name = type(e).__name__ + if name in error_messages: + sys.exit(error_messages[name]) + else: + sys.exit("Unrecognized error during check of admin rights: %s\n%s" % (name, str(e))) + def main(): safe_options, options = parse_options() @@ -128,6 +150,8 @@ def main(): api.bootstrap(**cfg) api.finalize() + ensure_kerberos_admin_rights(api) + if adtrustinstance.ipa_smb_conf_exists(): if not options.unattended: while True: @@ -194,9 +218,8 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm # try the connection try: @@ -205,12 +228,9 @@ def main(): except ldap.INVALID_CREDENTIALS, e: sys.exit("Password is not valid!") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + # See if our LDAP server is up and we can talk to it over GSSAPI + ccache = krbV.default_context().default_ccache().name + api.Backend.ldap2.connect(ccache) smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..56bd4810d873353378c8aa5ca282df13709d0e07 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -55,7 +55,7 @@ def print_msg(message, output_fd=sys.stdout): class Service(object): - def __init__(self, service_name, sstore=None, dm_password=None, ldapi=False): + def __init__(self, service_name, sstore=None, dm_password=None, ldapi=True): self.service_name = service_name self.service = ipaservices.service(service_name) self.steps = [] @@ -77,12 +77,13 @@ class Service(object): self.dercert = None def ldap_connect(self): - if self.ldapi: - if not self.realm: - raise RuntimeError('realm must be set to use ldapi connection') + # Always connect with LDAPI when realm is known + # the code in Service class always connects to the LDAP on the same + # machine so we can use LDAPI safely if realm is configured + if self.realm: self.admin_conn = self.__get_conn(None, None, ldapi=True, realm=self.realm) else: - self.admin_conn = self.__get_conn(self.fqdn, self.dm_password) + self.admin_conn = self.__get_conn(self.fqdn, self.dm_password, ldapi=False) def ldap_disconnect(self): self.admin_conn.unbind() @@ -93,7 +94,6 @@ class Service(object): pw_name = None fd = None path = ipautil.SHARE_DIR + ldif - hostname = installutils.get_fqdn() nologlist=[] if sub_dict is not None: @@ -107,15 +107,25 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # As we always connect to the local host, + # use URI of admin connection + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + + auth_parms = [] if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + # always try GSSAPI auth when not using DM password or not being root + if os.getegid() != 0: + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: @@ -181,8 +191,19 @@ class Service(object): This server cert should be in DER format. """ - if not self.admin_conn: - self.ldap_connect() + # add_cert_to_service() is relatively rare operation + # we actually call it twice during ipa-server-install, for different + # instances: ds and cs. Unfortunately, it may happen that admin + # connection was created well before add_cert_to_service() is called + # If there are other operations in between, it will become stale and + # since we are using SimpleLDAPObject, not ReconnectLDAPObject, the + # action will fail. Thus, explicitly disconnect and connect again. + # Using ReconnectLDAPObject instead of SimpleLDAPObject was considered + # but consequences for other parts of the framework are largerly + # unknown. + if self.admin_conn: + self.ldap_disconnect() + self.ldap_connect() dn = "krbprincipalname=%s,cn=services,cn=accounts,%s" % (self.principal, self.suffix) mod = [(ldap.MOD_ADD, 'userCertificate', self.dercert)] @@ -272,14 +293,13 @@ class Service(object): # If we are passed a password we'll use it as the DM password # otherwise we'll do a GSSAPI bind. try: -# conn = ipaldap.IPAdmin(fqdn, port=636, cacert=CACERT) if ldapi: conn = ipaldap.IPAdmin(ldapi=ldapi, realm=realm) else: conn = ipaldap.IPAdmin(fqdn, port=389) if dm_password: conn.do_simple_bind(bindpw=dm_password) - elif os.getegid() == 0 and self.ldapi: + elif os.getegid() == 0 and ldapi: try: # autobind pw_name = pwd.getpwuid(os.geteuid()).pw_name -- 1.7.10.4 From abokovoy at redhat.com Fri Jul 27 04:15:46 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Fri, 27 Jul 2012 07:15:46 +0300 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <20120712124223.GH9427@redhat.com> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> <20120712074840.GF9427@redhat.com> <1342094882.2599.973.camel@willson.li.ssimo.org> <20120712124223.GH9427@redhat.com> Message-ID: <20120727041546.GC13180@redhat.com> On Thu, 12 Jul 2012, Alexander Bokovoy wrote: >On Thu, 12 Jul 2012, Simo Sorce wrote: >>On Thu, 2012-07-12 at 10:48 +0300, Alexander Bokovoy wrote: >>>On Wed, 11 Jul 2012, Simo Sorce wrote: >>>>From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001 >>>>From: Simo Sorce >>>>Date: Fri, 6 Jul 2012 16:18:29 -0400 >>>>Subject: [PATCH] Add special modify op to regen ipaNTHash >>>> >>>>The NT Hash is the same thing as the RC4-HMAC key, so we add a function to >>>>extract it from krb5 keys if they are available to avoid forcing a password >>>>change when configuring trust relationships. >>>>--- >>>> .../ipa-pwd-extop/ipapwd_prepost.c | 147 +++++++++++++++++++- >>>> 1 file changed, 144 insertions(+), 3 deletions(-) >>>> >>>>diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >>>>index deae6477772f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d 100644 >>>>--- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >>>>+++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c >>>>@@ -41,7 +41,12 @@ >>>> # include >>>> #endif >>>> >>>>-#define _XOPEN_SOURCE /* strptime needs this */ >>>>+/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD >>>>+ * _GNU_SOURCE imply both, and we use it elsewhere, so use this */ >>>>+#ifndef _GNU_SOURCE >>>>+#define _GNU_SOURCE 1 >>>>+#endif >>>>+ >>>> #include >>>> #include >>>> #include >>>>@@ -53,6 +58,7 @@ >>>> #include >>>> #include >>>> #include >>>>+#include >>>> >>>> #include "ipapwd.h" >>>> #include "util.h" >>>>@@ -379,6 +385,12 @@ done: >>>> return 0; >>>> } >>>> >>>>+#define NTHASH_REGEN_VAL "MagicRegen" >>>>+#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL) >>>>+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, >>>>+ char *dn, struct slapi_entry *entry, >>>>+ struct ipapwd_krbcfg *krbcfg); >>>>+ >>>> /* PRE MOD Operation: >>>> * Gets the clean text password (fail the operation if the password came >>>> * pre-hashed, unless this is a replicated operation). >>>>@@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) >>>> int has_krb_keys = 0; >>>> int has_history = 0; >>>> int gen_krb_keys = 0; >>>>+ int is_magic_regen = 0; >>>> int ret, rc; >>>> >>>> LOG_TRACE( "=>\n"); >>>>@@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) >>>> default: >>>> break; >>>> } >>>>+ } else if (slapi_attr_types_equivalent(lmod->mod_type, "ipaNTHash")) { >>>>+ /* check op filtering out LDAP_MOD_BVALUES */ >>>>+ switch (lmod->mod_op & 0x0f) { >>>>+ case LDAP_MOD_REPLACE: >>>This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD. >> >>This is because I resent the old patch :( >> >>Hopefully the correct patch is now attached. >Yes, now it is updated, thanks. > >I'm going to experiment a bit with these patches, adding ipasam >responder to test them. Here is ipasam part. -- / Alexander Bokovoy -------------- next part -------------- >From 0cd261dd74154efc9ef6b09ef283cc1fe3448d5e Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Thu, 26 Jul 2012 22:05:25 +0300 Subject: [PATCH 6/6] When ipaNTHash is missing, ask IPA to generate it from kerberos keys --- daemons/ipa-sam/ipa_sam.c | 96 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 93 insertions(+), 3 deletions(-) diff --git a/daemons/ipa-sam/ipa_sam.c b/daemons/ipa-sam/ipa_sam.c index ab4b116c5f2b3b8dae6e8309403afba5fdf86708..aa54429b5bec4b26906b2a34e59ff95299a67f80 100644 --- a/daemons/ipa-sam/ipa_sam.c +++ b/daemons/ipa-sam/ipa_sam.c @@ -2400,6 +2400,74 @@ static bool init_sam_from_td(struct samu *user, struct pdb_trusted_domain *td, return true; } +static bool ipasam_nthash_retrieve(struct ldapsam_privates *ldap_state, + TALLOC_CTX *mem_ctx, + char *entry_dn, + DATA_BLOB *nthash) +{ + int ret; + bool retval; + LDAPMessage *result; + LDAPMessage *entry = NULL; + int count; + struct smbldap_state *smbldap_state = ldap_state->smbldap_state; + const char *attr_list[] = { + LDAP_ATTRIBUTE_NTHASH, + NULL + }; + + ret = smbldap_search(smbldap_state, entry_dn, + LDAP_SCOPE_BASE, "", attr_list, 0, + &result); + if (ret != LDAP_SUCCESS) { + DEBUG(1, ("Failed to get NT hash: %s\n", + ldap_err2string (ret))); + return false; + } + + count = ldap_count_entries(smbldap_state->ldap_struct, result); + + if (count != 1) { + DEBUG(1, ("Unexpected number of results [%d] for NT hash " + "of the single entry search.\n", count)); + ldap_msgfree(result); + return false; + } + + entry = ldap_first_entry(smbldap_state->ldap_struct, result); + if (entry == NULL) { + DEBUG(0, ("Could not get entry\n")); + ldap_msgfree(result); + return false; + } + + retval = smbldap_talloc_single_blob(mem_ctx, + smbldap_state->ldap_struct, + entry, LDAP_ATTRIBUTE_NTHASH, + nthash); + ldap_msgfree(result); + return retval; +} + +static bool ipasam_nthash_regen(struct ldapsam_privates *ldap_state, + TALLOC_CTX *mem_ctx, + char * entry_dn) +{ + LDAPMod **mods; + int ret; + + mods = NULL; + smbldap_make_mod(ldap_state->smbldap_state->ldap_struct, + NULL, &mods, LDAP_ATTRIBUTE_NTHASH, "MagicRegen"); + + talloc_autofree_ldapmod(mem_ctx, mods); + ret = smbldap_add(ldap_state->smbldap_state, entry_dn, mods); + if (ret != LDAP_SUCCESS) { + DEBUG(5, ("ipasam: attempt to regen ipaNTHash failed\n")); + } + return (ret == LDAP_SUCCESS); +} + static bool init_sam_from_ldap(struct ldapsam_privates *ldap_state, struct samu * sampass, LDAPMessage * entry) @@ -2414,6 +2482,7 @@ static bool init_sam_from_ldap(struct ldapsam_privates *ldap_state, char *profile_path = NULL; char *temp = NULL; bool ret = false; + bool retval = false; DATA_BLOB nthash; TALLOC_CTX *tmp_ctx = talloc_init("init_sam_from_ldap"); @@ -2504,14 +2573,35 @@ static bool init_sam_from_ldap(struct ldapsam_privates *ldap_state, pdb_set_acct_ctrl(sampass, ACB_NORMAL, PDB_SET); - if (!smbldap_talloc_single_blob(tmp_ctx, + retval = smbldap_talloc_single_blob(tmp_ctx, ldap_state->smbldap_state->ldap_struct, entry, LDAP_ATTRIBUTE_NTHASH, - &nthash)) { + &nthash); + if (!retval) { + /* NT Hash is not in place. Attempt to retrieve it from + * the RC4-HMAC key if that exists in Kerberos credentials. + * IPA 389-ds plugin allows to ask for it by setting + * ipaNTHash to MagicRegen value. + * */ + temp = smbldap_talloc_dn(tmp_ctx, ldap_state->smbldap_state->ldap_struct, entry); + if (temp) { + retval = ipasam_nthash_regen(tmp_ctx, + ldap_state->smbldap_state->ldap_struct, + temp); + if (retval) { + retval = ipasam_nthash_retrieve(tmp_ctx, + ldap_state->smbldap_state->ldap_struct, + temp, &nthash); + } + } + } + + if (!retval) { DEBUG(5, ("Failed to read NT hash form LDAP response.\n")); } + if (nthash.length != NT_HASH_LEN && nthash.length != 0) { - DEBUG(5, ("NT hash from LDAP has the wrong size.\n")); + DEBUG(5, ("NT hash from LDAP has the wrong size. Perhaps password was not re-set?\n")); } else { if (!pdb_set_nt_passwd(sampass, nthash.data, PDB_SET)) { DEBUG(5, ("Failed to set NT hash.\n")); -- 1.7.10.4 From pviktori at redhat.com Fri Jul 27 09:06:18 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 27 Jul 2012 11:06:18 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5011A456.7070403@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFBE66E.9040802@redhat.com> <5005EB9D.1010003@redhat.com> <5011A456.7070403@redhat.com> Message-ID: <50125A0A.1030905@redhat.com> On 07/26/2012 10:11 PM, John Dennis wrote: > On 07/17/2012 06:47 PM, John Dennis wrote: >>> ipapython/dn.py: >>> in docstring: DN(arg0, ..., locked=False, first_key_match=True) >>> followed by: def __init__(self, *args, **kwds): >>> and: kwds.get('first_key_match', True) >>> >>> I don't see the reason for this. Just write `def __init__(self, *args, >>> locked=False, first_key_match=True)` and put a proper summary in the >>> first line of the docstring. Same in AVA & RDN. >> >> A valid point. I think I was trying to be too flexible/extensible. > > Sorry, I was unable to make the suggested change. I tried and it > refreshed my memory as to why it was coded the way it was. > > Python considers it a syntax error if keyword arguments follow an arg > list reference. e.g. > > >>> def foo(*args, bar=None): > File "", line 1 > def foo(*args, bar=None): > ^ > SyntaxError: invalid syntax > > I'm not sure why, but that's why the methods are coded as > > def foo(*args, **kwds): > > If you have a suggestion as to how to use named parameters in this > context let me know or maybe explain why it's an illegal construct (I'm > sure it's in the language reference documentation somewhere, I just > didn't take the time to research it). > You're right, I forgot this particular Python wart. It's only fixed in Python 3. Looking at the code further, I recommend just getting rid of first_key_match. First of all, first_key_match isn't used anywhere. More importantly, the functionality is poorly designed. You can't safely mix first_key_match DNs and non-first_key_match DNs, because then you'll never know what the object will do. Whether you get a single value or a list of all shouldn't depend on the object, but should be decided by the call -- I recommend `dn['cn']` versus `dn.get_all('cn')`. Furthermore, first_key_match complicates the DN code unnecessarily. This can be done a subsequent patch; don't let it hold the DN work back. -- Petr? From pspacek at redhat.com Fri Jul 27 10:15:57 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 27 Jul 2012 12:15:57 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection Message-ID: <50126A5D.4060805@redhat.com> Hello, this patch implements "Flush zones and RRs cache when handling persistent search reconnection" behaviour as requested in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0042-Flush-zones-and-RRs-cache-when-handling-persistent-s.patch Type: text/x-patch Size: 5756 bytes Desc: not available URL: From pspacek at redhat.com Fri Jul 27 10:16:07 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 27 Jul 2012 12:16:07 +0200 Subject: [Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection Message-ID: <50126A67.1090306@redhat.com> Hello, this patch implements "Flush zones and RRs cache when handling persistent search reconnection" behaviour as requested in ticket https://fedorahosted.org/bind-dyndb-ldap/ticket/44 . Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0042-Flush-zones-and-RRs-cache-when-handling-persistent-s.patch Type: text/x-patch Size: 5756 bytes Desc: not available URL: From tbabej at redhat.com Fri Jul 27 11:13:42 2012 From: tbabej at redhat.com (Tomas Babej) Date: Fri, 27 Jul 2012 07:13:42 -0400 (EDT) Subject: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py In-Reply-To: <1943038613.836844.1343386832719.JavaMail.root@redhat.com> Message-ID: <105862894.837474.1343387622877.JavaMail.root@redhat.com> Hi, this patch simply checks if ipa-join exists in ipa-client folder, if not, skips tests relying on it. Uses nose.plugin.skip. https://fedorahosted.org/freeipa/ticket/2905 Tomas Babej -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-tbabej-0001-Adds-check-for-existence-of-ipa-join-exec.patch Type: text/x-patch Size: 2851 bytes Desc: not available URL: From pviktori at redhat.com Fri Jul 27 11:23:54 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 27 Jul 2012 13:23:54 +0200 Subject: [Freeipa-devel] [Patch] 0001 Add check for existence of ipa-join in the tree in test_host_plugin.py In-Reply-To: <105862894.837474.1343387622877.JavaMail.root@redhat.com> References: <105862894.837474.1343387622877.JavaMail.root@redhat.com> Message-ID: <50127A4A.3020200@redhat.com> On 07/27/2012 01:13 PM, Tomas Babej wrote: > Hi, > > this patch simply checks if ipa-join exists in ipa-client folder, if not, skips tests relying on it. > Uses nose.plugin.skip. > > https://fedorahosted.org/freeipa/ticket/2905 > > Tomas Babej > Hello, This would be better done in class setup, so you don't have to repeat it in every method. Look at XMLRPC_test.setUpClass() in xmlrpc_test.py for isnpiration. While you're at it, it would be good to move the mkstemp() call into setUpClass as well, so that importing the module doesn't have unnecessary side effects. -- Petr? From pspacek at redhat.com Fri Jul 27 12:23:49 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 27 Jul 2012 14:23:49 +0200 Subject: [Freeipa-devel] [PATCH 0043] Extend API to be compatible with libdns interface >= 90 Message-ID: <50128855.8050002@redhat.com> Hello, this patch prevents compiler warning on systems with libdns interface version >= 90. This libdns version comes with BIND 9.0.0. Both new methods are not obligatory, see in bind/lib/dns/db.c, functions dns_db_findext() and dns_db_findnodeext(). Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0043-Extend-API-to-be-compatible-with-libdns-interface-90.patch Type: text/x-patch Size: 976 bytes Desc: not available URL: From pviktori at redhat.com Fri Jul 27 12:24:42 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 27 Jul 2012 14:24:42 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5011BB38.5050701@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011BB38.5050701@redhat.com> Message-ID: <5012888A.3090207@redhat.com> On 07/26/2012 11:48 PM, John Dennis wrote: > I have applied the suggested fixes, rebased against master, run all the > unit tests successfully, built RPM's, did a full install without errors, > and brought up the web UI successfully. > > The current code can be found here: > > git clone git://fedorapeople.org/~jdennis/freeipa.dn.git > git checkout dn > > I did not squash the individual commits (but they should be before we > apply to master). Thank you! > Please test (again). > > I continue to believe the greatest lurking liability is the installer > code and the individual command line utilities (e.g. replica-manage, > etc.) Aside from the server install I have not exercised those components. Please test them, most of them just don't work. They're practically the only ones that use the old Entity & Entry, so related bugs won't show up unless you run the utilities. ipa-ldap-updater still fails: 2012-07-27T10:21:05Z DEBUG Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", line 112, in __upgrade self.modified = ld.update(self.files) File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 879, in update updates = api.Backend.updateclient.update(POST_UPDATE, self.dm_password, self.ldapi, self.live_run) File "/usr/lib/python2.7/site-packages/ipaserver/install/plugins/updateclient.py", line 134, in update if dn not in rdn_count_list[rdn_count]: IndexError: list index out of range The offending code is: rdn_count = len(DN(dn)) rdn_count_list = dn_by_rdn_count.setdefault(rdn_count, []) if dn not in rdn_count_list[rdn_count]: rdn_count_list[rdn_count].append(dn) rdn_count_list is dn_by_rdn_count[rdn_count]; indexing with rdn_count again is an error. I find the variable names are a bit confusing here. ipa-replica-prepare is also unusable: $ sudo ipa-replica-prepare vm-125.$DOMAIN --ip-address $IP Directory Manager (existing master) password: Preparing replica for vm-125.idm.lab.bos.redhat.com from vm-134.idm.lab.bos.redhat.com preparation of replica failed: '__getitem__' '__getitem__' File "/sbin/ipa-replica-prepare", line 461, in main() File "/sbin/ipa-replica-prepare", line 309, in main dirman_password) File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 99, in enable_replication_version_checking conn.modify_s(entry[0].dn, [(ldap.MOD_REPLACE, 'nsslapd-pluginenabled', 'on')]) File "/usr/lib/python2.7/site-packages/ipaserver/ipaldap.py", line 143, in __getattr__ return self.__dict__[name] i.e. entry[0] tries to call entry.__getitem__. I haven't tested any replica-related tools since I couldn't prepare a replica. ipa-compliance still has the same error as before ipa-managed-entries still fails: File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 607, in run_script return_value = main_function() File "install/tools/ipa-managed-entries", line 133, in main managed_entries = [entry.cn for entry in entries] You need entry.data['cn'] instead. I also get several errors in the DNS plugin test suite: Traceback (most recent call last): File "/home/pviktori/freeipa/ipaserver/rpcserver.py", line 332, in wsgi_execute result = self.Command[name](*args, **options) File "/home/pviktori/freeipa/ipalib/frontend.py", line 435, in __call__ ret = self.run(*args, **options) File "/home/pviktori/freeipa/ipalib/frontend.py", line 747, in run return self.execute(*args, **options) File "/home/pviktori/freeipa/ipalib/plugins/dns.py", line 2458, in execute result = super(dnsrecord_mod, self).execute(*keys, **options) File "/home/pviktori/freeipa/ipalib/plugins/baseldap.py", line 1351, in execute assert isinstance(dn, DN) AssertionError ipa: INFO: admin at IDM.LAB.BOS.REDHAT.COM: dnsrecord_mod(u'dnszone.test', u'testcnamerec', arecord=(u'10.0.0.1',), cnamerecord=None, rights=False, structured=False, all=False, raw=False, version=u'2.41'): AssertionError This is a good catch; the dnsrecord_mod post_callback should return the DN, not None. -- Petr? From pspacek at redhat.com Fri Jul 27 13:06:05 2012 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 27 Jul 2012 15:06:05 +0200 Subject: [Freeipa-devel] [PATCH 0044] Fix and comment ispersistent() call in LDAP driver interface Message-ID: <5012923D.10605@redhat.com> Hello, this patch fixes ispersistent() call in LDAP driver interface. We were lucky, because ISC_R_NOTIMPLEMENTED is evaluated as ISC_TRUE every time, but I want to be sure. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0044-Fix-and-comment-ispersistent-call-in-LDAP-driver-int.patch Type: text/x-patch Size: 1567 bytes Desc: not available URL: From pviktori at redhat.com Fri Jul 27 13:14:43 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 27 Jul 2012 15:14:43 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5011B6DA.7020602@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011B6DA.7020602@redhat.com> Message-ID: <50129443.3050000@redhat.com> On 07/26/2012 11:30 PM, John Dennis wrote: > The DN patch has been reworked. The single largest change was to the DN > implementation and it's unit test to refactor the AVA, RDN and DN > classes into a base class that is immutable and a subclass that supports > editing. This makes these classes fully consistent with the Python > language definition [...]. Unfortunately, that's not quite true yet. If an object implements both __hash__ and __cmp__/__eq__, it must ensure that two equal objects will have the same hash value -- see http://docs.python.org/reference/datamodel.html#object.__hash__ This is not the case: DN(dn) == EditableDN(dn), but hash(DN(dn)) != hash(EditableDN(dn)). Please flag the mutable classes as unhashable by setting __hash__ = None. > Bottom line for developers. In 99.99% of the cases just continue to use > DN's as you always have. If you need to modify a DN cast it into a > EditableDN (e.g. dn = EditableDN(dn)), perform your edit, and if you end > up passing the dn outside your scope or using it as a dict key > you should cast it back to an immutable DN (e.g. dn = DN(dn) -- Petr? From jdennis at redhat.com Fri Jul 27 13:16:13 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 27 Jul 2012 09:16:13 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <50125A0A.1030905@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFBE66E.9040802@redhat.com> <5005EB9D.1010003@redhat.com> <5011A456.7070403@redhat.com> <50125A0A.1030905@redhat.com> Message-ID: <5012949D.3060709@redhat.com> On 07/27/2012 05:06 AM, Petr Viktorin wrote: > On 07/26/2012 10:11 PM, John Dennis wrote: >> On 07/17/2012 06:47 PM, John Dennis wrote: >>>> ipapython/dn.py: >>>> in docstring: DN(arg0, ..., locked=False, first_key_match=True) >>>> followed by: def __init__(self, *args, **kwds): >>>> and: kwds.get('first_key_match', True) >>>> >>>> I don't see the reason for this. Just write `def __init__(self, *args, >>>> locked=False, first_key_match=True)` and put a proper summary in the >>>> first line of the docstring. Same in AVA & RDN. >>> >>> A valid point. I think I was trying to be too flexible/extensible. >> >> Sorry, I was unable to make the suggested change. I tried and it >> refreshed my memory as to why it was coded the way it was. >> >> Python considers it a syntax error if keyword arguments follow an arg >> list reference. e.g. >> >> >>> def foo(*args, bar=None): >> File "", line 1 >> def foo(*args, bar=None): >> ^ >> SyntaxError: invalid syntax >> >> I'm not sure why, but that's why the methods are coded as >> >> def foo(*args, **kwds): >> >> If you have a suggestion as to how to use named parameters in this >> context let me know or maybe explain why it's an illegal construct (I'm >> sure it's in the language reference documentation somewhere, I just >> didn't take the time to research it). >> > > You're right, I forgot this particular Python wart. It's only fixed in > Python 3. > > > Looking at the code further, I recommend just getting rid of > first_key_match. I agree, that feature is not well designed. You are also correct, it's currently not utilized. I will remove it (which should be easy to do). > > First of all, first_key_match isn't used anywhere. > > More importantly, the functionality is poorly designed. You can't safely > mix first_key_match DNs and non-first_key_match DNs, because then you'll > never know what the object will do. > Whether you get a single value or a list of all shouldn't depend on the > object, but should be decided by the call -- I recommend `dn['cn']` > versus `dn.get_all('cn')`. > Furthermore, first_key_match complicates the DN code unnecessarily. > > This can be done a subsequent patch; don't let it hold the DN work back. > -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pviktori at redhat.com Fri Jul 27 15:18:31 2012 From: pviktori at redhat.com (Petr Viktorin) Date: Fri, 27 Jul 2012 17:18:31 +0200 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5011A1A0.20504@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFD3277.9070607@redhat.com> <5011A1A0.20504@redhat.com> Message-ID: <5012B147.50602@redhat.com> The nitpicks continue. None of the things in this mail are holding up the patch, consider it more of a discussion about good practices. (I'm assuming you're interested.) ipaserver/rpcserver.py: elif isinstance(val, (list, tuple)): - new_list = [] - n = len(val) - i = 0 - while i < n: - v = val[i] - if isinstance(v, str): - new_list.append({'__base64__' : base64.b64encode(v)}) - else: - new_list.append(json_encode_binary(v)) - i += 1 + new_list = [json_encode_binary(v) for v in val] del val return new_list You might as well remove the `del val`s. ipapython/dn.py: -Concatenation and In-Place Addition: +Concatenation, In-Place Addition, Insetion: s/Insetion/Insertion/ In the giant comment in ipapython/dn.py: +by reference" (mutable objects can be modifed inside the unbalanced parenthesis +and B are the same object Python says no [2]_. For those familiar with Lisp missing reference Your example here only applies to interactive sessions of current versions of CPython, i.e. the Python implementation we use; other Pythons are free to use same or different objects for equal strs/ints as they choose. Even in CPython literals happen to become the same object if they appear in the same compilation unit: >>> 'abc' == 'abc' True But in general I think lengthy explanations of Python's behavior shuld go in a Python tutorial. The docstring should say what the code does and how to use it, and comments can summarize design/implementation decisions with simple justification (possibly with links to tutorials/documentation -- your prose could make good blog posts). By the way, are you aware that the text in dn.py is not a docstring, because it's not the first statement in the file? +verses using a hash value for dict keys or sets is critical to s/verses/versus/ Good job on the DN tests! On 07/26/2012 09:59 PM, John Dennis wrote: > On 07/11/2012 03:59 AM, Petr Viktorin wrote: >> I went over the changes to ipaserver/plugins/ldap2.py; now I can start >> the testing. >> >> I must ask about the wrapped _ext methods. I assume python-ldap exposes >> them only to mimic the C interface, which has them because C doesn't >> have default arguments. Would we lose anything if the class only defined >> the _ext methods, and aliased the non-ext variants to them? e.g. >> >> def add_ext(self, dn, modlist, serverctrls=None, clientctrls=None): >> assert isinstance(dn, DN) >> dn = str(dn) >> modlist = self.encode(modlist) >> return self.conn.add_ext(dn, modlist, serverctrls, clientctrls) >> add = add_ext >> >> The non-_ext variants are deprecated in the underlying C library, anyway. > > > With the current implementation you are correct. There would be no > functional difference if we only utilized the _ext variants. However > that means employing knowledge from the wrong side of an API boundary. > From our perspective python-ldap exposes certain methods. In particular > we use the simpler methods because that is sufficient to meet our needs. > I think we should code to the defined API. If python-ldap ever changes > or we use an alternate Python ldap library the porting effort would be > much cleaner if we don't try to outsmart ourselves. Fair enough. >> Lines 203, 825: >> os.environ['KRB5CCNAME'] = ccache_file >> Should KRB5CCNAME be unset/restored once the code's done with it. > > Agreed, once again I didn't write this but saw the problem when I moved > the code block and made a note to fix it in the future. This particular > place is fixed now. However I think we've got other places that do > similar things. This is another example of the tendency to cut-n-paste > blocks of duplicate code which drives me nuts. But how much unrelated > code does one change in any patch? I did however add a FIXME comment to > get that code cleaned up. In the past I would change any problem I saw > when I touched an area of the code but I got a lot of complaints about > changing things not directly related to the intent of the patch :-( Also fair. Sorry for different standards, I know it's frustrating to have the line drawn in a different place each time. >> Line 107 & 127: >> return utf8_codec.decode(val)[0] >> return utf8_codec.encode(unicode(val))[0] >> I believe the following would be more readable: >> return val.decode('utf-8') >> return unicode(val).encode('utf-8') > > Yes, your suggestion is easier to read, but it's more efficient to > lookup the codec once rather than everytime you decode. I wouldn't worry about that; lookups are pretty fast -- I'd say comparable to creating an unnecessary tuple. Of course it's hard to know without profiling. >> Line 354: >> _SCHEMA_OVERRIDE = { >> We have a case-insensitive dict class, why not use it here? > > Excellent suggestion, it's been changed. Please also change the use: attr_lower = attr.lower() syntax = self._SCHEMA_OVERRIDE.get(attr_lower) to syntax = self._SCHEMA_OVERRIDE[attr] >> Lines 388, 750: >> def is_dn_syntax(self, attr): >> """ >> Check the schema to see if the attribute uses DN syntax. >> uses_dn_syntax or has_dn_syntax would be a better name. > > Good suggestion. I made the change. > > Actually, I wish LDAP attributes were a class, that way the correct > comparisons could be automatically applied by Python based on the LDAP > syntax of the particular attribute and things like "is_dn" and > "is_multivalued" could be a properties of the attribute instance, etc. > We are using an object oriented language aren't we? All problems can be solved by another level of abstraction. (except for the problem of too many layers of abstraction) Sounds like a ticket for the deferred bucket :) >> Line 406: >> if isinstance(val, bool): >> if val: >> val = 'TRUE' >> else: >> val = 'FALSE' >> return self.encode(val) >> Encoding an ASCII string should be a no-op, why bother doing it? > > Not sure what you mean by no-op. Do you mean str(val) would return > either 'True' or 'False'? > > In any event we are dependent on the string representation of a boolean > being all uppper-case. Not something I invented. I meant the `return self.encode(val)`, with val being either 'TRUE' or 'FALSE'. The following should suffice: if isinstance(val, bool): if val: return 'TRUE' else: return 'FALSE' Sorry for not being clear. >> Line 421: >> dct = dict() >> for (k, v) in val.iteritems(): >> dct[self.encode(k)] = self.encode(v) >> return dct >> Consider: >> dct = dict((self.encode(k), self.encode(v)) for k, v in >> val.iteritems()) > > Why do the work of building a list of tuples when it's not necessary? It's a generator expression, it doesn't build a list. >> Line 437: >> if type(target_type) is type and isinstance(original_value, >> target_type): >> Don't use `type(x) is expected_type` for type checking! Use >> isinstance(target_type, type) so you also catch type's subclasses. > > Good point, fixed. > > Or >> better yet, handle the TypeError that isinstance will raise if you don't >> give it a class. >> To be honest, I don't think it's worth it to do all this convoluted >> checking just to make sure you're not copying unnecessarily. Do you have >> a reason I can't see? > > Because some constructors are expensive and it's good to pay attention > to efficiency. It is, but I'm not convinced this is the right place. Complicating the code (especially in a fairly obscure method with no dosctring) has its own drawbacks. If constructors are too expensive it might be better to optimize those -- for example sharing immutable AVAs/RDNs when copying DNs would probably give a similar speedup here, as well as helping other parts of the code. >> Line 594, 604, 613: >> ipa_result = self.convert_result(ldap_result) >> return ipa_result >> Just return the result directly, no need for the extra variable. > > It was done that way for two reasons. During development it gave me a > value to print for debugging purposes and I thought it made the code > clearer by indicating what is an IPA value. I don't think the extra > variable introduces any significant inefficiencies (in C it would be > optimized out, not sure Python's byte compiler does the same). It doesn't, but the slowdown is negligible. Never mind, this is an extremely minor point. >> Line 1422: >> self.handle_errors(e, **{}) >> Wouldn't the following work better? >> self.handle_errors(e, *[{}]*0) > > I never liked the way handle_errors was written. The use of the kwds > args is odd, convoluted and hard to understand. > > To be honest I can't parse > > *[{}]*0 Sorry, lame attempt at humor. The **{} doesn't do anything at all, you can remove it. The keyword arguments in handle_errors are unused, so you can remove that as well if you want. -- Petr? From pvoborni at redhat.com Fri Jul 27 15:55:23 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 27 Jul 2012 17:55:23 +0200 Subject: [Freeipa-devel] [PATCH] 175, 176 Fixed: Unable to select option in combobox in IE and Chrome Message-ID: <5012B9EB.8010304@redhat.com> [PATCH] 175 Fixed: Unable to select option in combobox in IE and Chrome There's probably a bug regarding z-index stacking in Chrome and IE. It appears when combobox is used in dialog. Combobox's select area had z-index=1010. When first jquery dialog is open it has z-index=1000. Further dialogs have higher z-index. When dialog's z-index exceeds 1010 option in select control can't be selected. IMO it is a browser bug because select control lies in dialog content's stacking context so it should be functional even with z-index=1. This patch raises select area's z-index to 9000000 which should prevent the issue for some time. Also it make's combobox's z-index configurable so we can solve combobox stacking (ie in service-add dialog). Second part of: https://fedorahosted.org/freeipa/ticket/2834 [PATCH] 176 Fixed: combobox stacking in service adder dialog First select's content is displayed under second comboxes content when select is opened when second combobox is opened Bonus for: https://fedorahosted.org/freeipa/ticket/2834 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0175-Fixed-Unable-to-select-option-in-combobox-in-IE-and-.patch Type: text/x-patch Size: 3429 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0176-Fixed-combobox-stacking-in-service-adder-dialog.patch Type: text/x-patch Size: 1581 bytes Desc: not available URL: From pvoborni at redhat.com Fri Jul 27 16:59:10 2012 From: pvoborni at redhat.com (Petr Vobornik) Date: Fri, 27 Jul 2012 18:59:10 +0200 Subject: [Freeipa-devel] [PATCHES] 177-184 Rebase of jquery and jquery-ui libs Message-ID: <5012C8DE.9090105@redhat.com> This effort was driven by ticket https://fedorahosted.org/freeipa/ticket/2817 which needed update of jqury-ui library. When I was at it I also updated core jquery lib. All patches are associated with ticket #2817 which might be bad. Should I split it to more tickets? This effort makes Web UI look consistent in Firefox, Chrome, Opera and IE. I didn't try Safari. Also it fixes some visual issues (mostly in IE and Opera) and makes buttons prettier. Patches 177,178 are rebases of jquery and jquery-ui libraries. Upgrade introduced two new functional bugs and 2 test bugs which are fixed by patches 179, 182 and 183. Update of jquery-ui.css made some parts of ipa.css redundant so the redundancy is removed in patch 180. Patch 180 also removes some styles which make buttons in association dialog look bad. Patch 181 utilizes jquery.button and so makes buttons in association dialogs consistent with the rest of the UI. Patch 184 is just tiny refactoring. Patch notes: [PATCH] 177 Update to jquery.1.7.2.min jquery library wasn't updated for a long time. [PATCH] 178 Update to jquery-ui-1.8.21.custom jquery-ui was regenerated to up to date version. Border radius and IPA custom colors were added to theme so we don't have to override them in ipa.css. [PATCH] 179 Fix for incorrect event handler definition Clicks events should be better defined by jquery calls (usually addEventListener) not as elements attributes. Definition as element attribute causes problems after upgrade to jquery 1.7.2. Two occurances were removed. [PATCH] 180 Removal of unnecessary overrides of jquery-ui styles ipa.css had to be updated to work with updated jquery-ui. This patch removes several duplicate styles. Following issues were fixed: * dialogs titles in IE and Opera were black instead of green * no black line in first navigation level in IE and Opera * all browsers (FF, IE, Chrome, Opera) have the same style for buttons and headers * dialogs has borders again (should we remove its shadow?) Known issues: * selected tab-1 in Chrome and Opera doesn't overlaps background line as in IE and FF. Not sure how to fix without breaking (there are border overlaps) the latter ones. I think it looks good enough. * some buttons are missing padding. Will be fixed in next patch. [PATCH] 181 Unified buttons Buttons in association dialog and action list have different style and behavior than buttons in dialogs. This patch unifies it by using jquery.button widget. [PATCH] 182 Web UI tests fix ACI tests were crashing because of misconfigured facet. Entity link test were crashing because of incorrect jquery selector. [PATCH] 183 Fixed incorrect use of jQuery.attr for setting disabled attribute Occurance: select_widget Update to latest version of jQuery uncovered this issue. [PATCH] 184 Replace use of attr with prop for booleans Recommened way of setting boolean HTML attributes is by $.prop(boolean) method not $.attr(boolean) because it sets DOM object property not an attribute. Latter works because of jquery's backward compatibility. This patch makes things clearer. Some info about prop and attr: http://stackoverflow.com/a/5876747 -- Petr Vobornik -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0177-Update-to-jquery.1.7.2.min.patch Type: text/x-patch Size: 323085 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0178-Update-to-jquery-ui-1.8.21.custom.patch Type: text/x-patch Size: 456610 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0179-Fix-for-incorrect-event-handler-definition.patch Type: text/x-patch Size: 1456 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0180-Removal-of-unnecessary-overrides-of-jquery-ui-styles.patch Type: text/x-patch Size: 5228 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0181-Unified-buttons.patch Type: text/x-patch Size: 2647 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0182-Web-UI-tests-fix.patch Type: text/x-patch Size: 2211 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0183-Fixed-incorrect-use-of-jQuery.attr-for-setting-disab.patch Type: text/x-patch Size: 1339 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-pvoborni-0184-Replace-use-of-attr-with-prop-for-booleans.patch Type: text/x-patch Size: 11074 bytes Desc: not available URL: From simo at redhat.com Fri Jul 27 17:55:15 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 27 Jul 2012 13:55:15 -0400 Subject: [Freeipa-devel] [PATCH] Minor fix to sidgen plugin In-Reply-To: <20120727040439.GA13180@redhat.com> References: <492182885.3890857.1343327699742.JavaMail.root@redhat.com> <1964262976.3891605.1343327734632.JavaMail.root@redhat.com> <20120727040439.GA13180@redhat.com> Message-ID: <1343411715.2666.46.camel@willson.li.ssimo.org> On Fri, 2012-07-27 at 07:04 +0300, Alexander Bokovoy wrote: > On Thu, 26 Jul 2012, Simo Sorce wrote: > >Remove an unnecessary check that may give spurious failures on modified > >server where 999 is a valid ID. > ACK. > Pushed to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From jdennis at redhat.com Fri Jul 27 19:43:41 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 27 Jul 2012 15:43:41 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <5012B147.50602@redhat.com> References: <4FF883DF.4020502@redhat.com> <4FFD3277.9070607@redhat.com> <5011A1A0.20504@redhat.com> <5012B147.50602@redhat.com> Message-ID: <5012EF6D.8020707@redhat.com> On 07/27/2012 11:18 AM, Petr Viktorin wrote: > The nitpicks continue. None of the things in this mail are holding up > the patch, consider it more of a discussion about good practices. (I'm > assuming you're interested.) > > > > ipaserver/rpcserver.py: > elif isinstance(val, (list, tuple)): > - new_list = [] > - n = len(val) > - i = 0 > - while i < n: > - v = val[i] > - if isinstance(v, str): > - new_list.append({'__base64__' : base64.b64encode(v)}) > - else: > - new_list.append(json_encode_binary(v)) > - i += 1 > + new_list = [json_encode_binary(v) for v in val] > del val > return new_list > > You might as well remove the `del val`s. Agreed, it baffled me as to why they were there in the first place. There are few other del's that need to be removed as well. I made the change. > > ipapython/dn.py: > -Concatenation and In-Place Addition: > +Concatenation, In-Place Addition, Insetion: > > s/Insetion/Insertion/ Fixed. > In the giant comment in ipapython/dn.py: > ... > But in general I think lengthy explanations of Python's behavior ... Text is removed. > By the way, are you aware that the text in dn.py is not a docstring, > because it's not the first statement in the file? Good catch, no I missed that. >>> Line 107 & 127: >>> return utf8_codec.decode(val)[0] >>> return utf8_codec.encode(unicode(val))[0] >>> I believe the following would be more readable: >>> return val.decode('utf-8') >>> return unicode(val).encode('utf-8') >> >> Yes, your suggestion is easier to read, but it's more efficient to >> lookup the codec once rather than everytime you decode. > > I wouldn't worry about that; lookups are pretty fast -- I'd say > comparable to creating an unnecessary tuple. Of course it's hard to know > without profiling. Fair enough, it's been changed. >>> Line 354: >>> _SCHEMA_OVERRIDE = { >>> We have a case-insensitive dict class, why not use it here? >> >> Excellent suggestion, it's been changed. > > Please also change the use: > attr_lower = attr.lower() > syntax = self._SCHEMA_OVERRIDE.get(attr_lower) > to > syntax = self._SCHEMA_OVERRIDE[attr] Good catch, fixed. >>> Line 406: >>> if isinstance(val, bool): >>> if val: >>> val = 'TRUE' >>> else: >>> val = 'FALSE' >>> return self.encode(val) >>> Encoding an ASCII string should be a no-op, why bother doing it? >> >> Not sure what you mean by no-op. Do you mean str(val) would return >> either 'True' or 'False'? >> >> In any event we are dependent on the string representation of a boolean >> being all uppper-case. Not something I invented. > > I meant the `return self.encode(val)`, with val being either 'TRUE' or > 'FALSE'. The following should suffice: > > if isinstance(val, bool): > if val: > return 'TRUE' > else: > return 'FALSE' > > Sorry for not being clear. Duh, John smacks his forehead :-) Fixed. >>> Line 421: >>> dct = dict() >>> for (k, v) in val.iteritems(): >>> dct[self.encode(k)] = self.encode(v) >>> return dct >>> Consider: >>> dct = dict((self.encode(k), self.encode(v)) for k, v in >>> val.iteritems()) >> >> Why do the work of building a list of tuples when it's not necessary? > > It's a generator expression, it doesn't build a list. Good point, changed. >>> Line 1422: >>> self.handle_errors(e, **{}) >>> Wouldn't the following work better? >>> self.handle_errors(e, *[{}]*0) >> >> I never liked the way handle_errors was written. The use of the kwds >> args is odd, convoluted and hard to understand. >> >> To be honest I can't parse >> >> *[{}]*0 > > Sorry, lame attempt at humor. > The **{} doesn't do anything at all, you can remove it. > The keyword arguments in handle_errors are unused, so you can remove > that as well if you want. Cruft from previous implementation, it's been removed. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Fri Jul 27 20:04:42 2012 From: jdennis at redhat.com (John Dennis) Date: Fri, 27 Jul 2012 16:04:42 -0400 Subject: [Freeipa-devel] DN patch and documentation In-Reply-To: <50129443.3050000@redhat.com> References: <4FF883DF.4020502@redhat.com> <5011B6DA.7020602@redhat.com> <50129443.3050000@redhat.com> Message-ID: <5012F45A.9050303@redhat.com> On 07/27/2012 09:14 AM, Petr Viktorin wrote: > Unfortunately, that's not quite true yet. > > If an object implements both __hash__ and __cmp__/__eq__, it must ensure > that two equal objects will have the same hash value -- see > http://docs.python.org/reference/datamodel.html#object.__hash__ > > This is not the case: DN(dn) == EditableDN(dn), but hash(DN(dn)) != > hash(EditableDN(dn)). > > Please flag the mutable classes as unhashable by setting __hash__ = None. Fixed. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From rcritten at redhat.com Fri Jul 27 20:50:40 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 27 Jul 2012 16:50:40 -0400 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <501174DA.9010909@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> <500FFC4F.8090601@redhat.com> <5010035A.2060108@redhat.com> <50105DF1.5030507@redhat.com> <501174DA.9010909@redhat.com> Message-ID: <5012FF20.5070904@redhat.com> Jan Cholasta wrote: > Dne 25.7.2012 22:58, Rob Crittenden napsal(a): >> Jan Cholasta wrote: >> >>> All these scripts could use more exception handling, but I guess >>> potential bugs can be sorted out later. >> >> Well, they all run in the background so even if they caught errors >> nothing would see them unless we decide to syslog errors. I decided to syslog the errors, there is no other way around this. >>> >>> install/share/default-aci.ldif: >>> >>> The ACIs are wrong (Kerberos principal instead of ldap URI in userdn, in >>> 40-delegation.update it is done right). >> >> Nice catch, not sure how I missed that. Fixed. > > You forgot to fix the allow(add) one, it still has userdn = > "host/$FQDN@$REALM". > Fixed. > I did: > > 1. ipa-server-install on host1, using IPA from master > 2. ipa-replica-install on host2, using IPA from master > 3. update host1 to IPA with your patch applied > 4. update host2 to IPA with your patch applied > 5. ipa-ca-install on host2 > > After that, ipaCert is not tracked on host2 at all (I had to add it > manually using "getcert start-tracking -d /etc/httpd/alias -n ipaCert -c > dogtag-ipa-retrieve-agent-submit -C > /usr/lib64/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt > -T ipaCert"). Fixed, it wasn't being tracked on upgrades. I filed a ticket for the audit cert renewing for only 6 months. It is a dogtag bug. I've seen some oddness when testing by moving the date forward, CS replication has stopped working. I kick it with ipa-csreplica-manage force-sync --from= and that fixes things. This is unrelated to my patch. rob -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-rcrit-1033-6-renewal.patch Type: text/x-diff Size: 51001 bytes Desc: not available URL: From rcritten at redhat.com Fri Jul 27 21:14:12 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 27 Jul 2012 17:14:12 -0400 Subject: [Freeipa-devel] [PATCH] 1039 fix selinux usermap config options In-Reply-To: <500FB437.7030109@redhat.com> References: <500F61FE.6010300@redhat.com> <500FB437.7030109@redhat.com> Message-ID: <501304A4.4040201@redhat.com> Petr Viktorin wrote: > On 07/25/2012 05:03 AM, Rob Crittenden wrote: >> The configuration options for the default user and map order were a bit >> broken in several ways. >> >> I wasn't handling the case where one of the values was coming from LDAP >> so was a list vs as an option which was a string, so all sorts of bad >> interesting things were happening. >> >> There is also the setattr problem. We would normally handle that in a >> validator so it is not a problem but in this case we may need to compare >> two options passed in and we can't do that in a validator. So >> potentially changes may come in as a option, in entry_attrs or from >> config. >> >> I added a few tests to help keep this robust. >> >> When testing this remember that the user map order list needs to be >> quoted otherwise the shell is going to interpret the $. >> >> rob >> > > ACK, with nitpicks :) > > > I think using `copy.deepcopy(options)` instead of simply `dict(options)` > is unnecessary; do you have a special reason for it? > > > And note the style guide discourages line continuation backslashes: > + if 'ipaselinuxusermapdefault' in validate or \ > + 'ipaselinuxusermaporder' in validate: > > in favor of parentheses: > + if ('ipaselinuxusermapdefault' in validate or > + 'ipaselinuxusermaporder' in validate): > Fixed both and pushed to master. rob From rcritten at redhat.com Fri Jul 27 22:12:24 2012 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 27 Jul 2012 18:12:24 -0400 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <20120727040732.GB13180@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> Message-ID: <50131248.20005@redhat.com> Alexander Bokovoy wrote: > On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >> Hi, >> >> When setting up AD trusts support, ipa-adtrust-install utility >> needs to be run as: >> - root, for performing Samba configuration and using LDAPI/autobind >> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >> fetch keytab >> >> As result, we can get rid of Directory Manager credentials in >> ipa-adtrust-install >> >> https://fedorahosted.org/freeipa/ticket/2815 >> >> This ticket also simplifies a bit the way we handle admin connection in >> Service class and particulary in Service._ldap_mod() by defaulting to >> LDAPI/autobind in case of running as root and to GSSAPI otherwise. >> Except few cases in remote replica management (not applicable in >> _ldap_mod() case) we always run installation tools as root and can >> benefit from using autobind feature. Unfortunately, it is not yet >> possible to get away from using DM credentials for all cases as the same >> class is used to perform initial directory server instance >> configuration. >> >> One side effect is explicit disconnect and reconnect in >> Service.add_cert_to_service() due to way how SimpleLDAPObject class >> handles stale connections (no handling at all). I've put some comments >> in place so that others would not try to err out optimizing it in >> future. >> >> Finally, with next patch series which will introduce syncing ipaNTHash >> attribute with RC4 key in existing kerberos credentials, we can remove >> requirements to change passwords or re-kinit for majority of trust >> cases. This should then conclude our trusts content for beta2 release. > > Patch updated, fixed small typo (auth_parms was initialized as > auth_params which led to non-existing auth_parms in ipa-adtrust-install > case). Nack, a couple of minor issues: The exception handling is rather unusual in ensure_kerberos_admin_rights(api). I'm not sure if this is any more efficient than a series of excepts... You don't need to pass in api, it's a global. It may be safe to see if the user is in the group the way you are doing it, I wonder if it would be clearer to cast those into DN objects. In the Service class what is the point of ldapi if it is going to be ignored in the case we know the realm? What if I really, really just want to use a password? And later where it forces ldapi, it seems better to either commit all the way and drop the ldapi argument or convert it to a better name (like autobind). Typo in big comment block, 'largerly' rob From simo at redhat.com Fri Jul 27 22:54:59 2012 From: simo at redhat.com (Simo Sorce) Date: Fri, 27 Jul 2012 18:54:59 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <20120727041546.GC13180@redhat.com> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> <20120712074840.GF9427@redhat.com> <1342094882.2599.973.camel@willson.li.ssimo.org> <20120712124223.GH9427@redhat.com> <20120727041546.GC13180@redhat.com> Message-ID: <1343429699.20530.3.camel@willson.li.ssimo.org> On Fri, 2012-07-27 at 07:15 +0300, Alexander Bokovoy wrote: > On Thu, 12 Jul 2012, Alexander Bokovoy wrote: > >On Thu, 12 Jul 2012, Simo Sorce wrote: > >>On Thu, 2012-07-12 at 10:48 +0300, Alexander Bokovoy wrote: > >>>On Wed, 11 Jul 2012, Simo Sorce wrote: > >>>>From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001 > >>>>From: Simo Sorce > >>>>Date: Fri, 6 Jul 2012 16:18:29 -0400 > >>>>Subject: [PATCH] Add special modify op to regen ipaNTHash > >>>> > >>>>The NT Hash is the same thing as the RC4-HMAC key, so we add a function to > >>>>extract it from krb5 keys if they are available to avoid forcing a password > >>>>change when configuring trust relationships. > >>>>--- > >>>> .../ipa-pwd-extop/ipapwd_prepost.c | 147 +++++++++++++++++++- > >>>> 1 file changed, 144 insertions(+), 3 deletions(-) > >>>> > >>>>diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > >>>>index deae6477772f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d 100644 > >>>>--- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > >>>>+++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > >>>>@@ -41,7 +41,12 @@ > >>>> # include > >>>> #endif > >>>> > >>>>-#define _XOPEN_SOURCE /* strptime needs this */ > >>>>+/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD > >>>>+ * _GNU_SOURCE imply both, and we use it elsewhere, so use this */ > >>>>+#ifndef _GNU_SOURCE > >>>>+#define _GNU_SOURCE 1 > >>>>+#endif > >>>>+ > >>>> #include > >>>> #include > >>>> #include > >>>>@@ -53,6 +58,7 @@ > >>>> #include > >>>> #include > >>>> #include > >>>>+#include > >>>> > >>>> #include "ipapwd.h" > >>>> #include "util.h" > >>>>@@ -379,6 +385,12 @@ done: > >>>> return 0; > >>>> } > >>>> > >>>>+#define NTHASH_REGEN_VAL "MagicRegen" > >>>>+#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL) > >>>>+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, > >>>>+ char *dn, struct slapi_entry *entry, > >>>>+ struct ipapwd_krbcfg *krbcfg); > >>>>+ > >>>> /* PRE MOD Operation: > >>>> * Gets the clean text password (fail the operation if the password came > >>>> * pre-hashed, unless this is a replicated operation). > >>>>@@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > >>>> int has_krb_keys = 0; > >>>> int has_history = 0; > >>>> int gen_krb_keys = 0; > >>>>+ int is_magic_regen = 0; > >>>> int ret, rc; > >>>> > >>>> LOG_TRACE( "=>\n"); > >>>>@@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > >>>> default: > >>>> break; > >>>> } > >>>>+ } else if (slapi_attr_types_equivalent(lmod->mod_type, "ipaNTHash")) { > >>>>+ /* check op filtering out LDAP_MOD_BVALUES */ > >>>>+ switch (lmod->mod_op & 0x0f) { > >>>>+ case LDAP_MOD_REPLACE: > >>>This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD. > >> > >>This is because I resent the old patch :( > >> > >>Hopefully the correct patch is now attached. > >Yes, now it is updated, thanks. > > > >I'm going to experiment a bit with these patches, adding ipasam > >responder to test them. > Here is ipasam part. ACK the ipasam part. Simo. -- Simo Sorce * Red Hat, Inc * New York From jcholast at redhat.com Mon Jul 30 08:54:42 2012 From: jcholast at redhat.com (Jan Cholasta) Date: Mon, 30 Jul 2012 10:54:42 +0200 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <5012FF20.5070904@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> <500FFC4F.8090601@redhat.com> <5010035A.2060108@redhat.com> <50105DF1.5030507@redhat.com> <501174DA.9010909@redhat.com> <5012FF20.5070904@redhat.com> Message-ID: <50164BD2.7000705@redhat.com> Dne 27.7.2012 22:50, Rob Crittenden napsal(a): > Jan Cholasta wrote: >> Dne 25.7.2012 22:58, Rob Crittenden napsal(a): >>> Jan Cholasta wrote: >>> >>>> All these scripts could use more exception handling, but I guess >>>> potential bugs can be sorted out later. >>> >>> Well, they all run in the background so even if they caught errors >>> nothing would see them unless we decide to syslog errors. > > I decided to syslog the errors, there is no other way around this. > >>>> >>>> install/share/default-aci.ldif: >>>> >>>> The ACIs are wrong (Kerberos principal instead of ldap URI in >>>> userdn, in >>>> 40-delegation.update it is done right). >>> >>> Nice catch, not sure how I missed that. Fixed. >> >> You forgot to fix the allow(add) one, it still has userdn = >> "host/$FQDN@$REALM". >> > > Fixed. > >> I did: >> >> 1. ipa-server-install on host1, using IPA from master >> 2. ipa-replica-install on host2, using IPA from master >> 3. update host1 to IPA with your patch applied >> 4. update host2 to IPA with your patch applied >> 5. ipa-ca-install on host2 >> >> After that, ipaCert is not tracked on host2 at all (I had to add it >> manually using "getcert start-tracking -d /etc/httpd/alias -n ipaCert -c >> dogtag-ipa-retrieve-agent-submit -C >> /usr/lib64/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt >> -T ipaCert"). > > Fixed, it wasn't being tracked on upgrades. > > I filed a ticket for the audit cert renewing for only 6 months. It is a > dogtag bug. OK, thanks. > > I've seen some oddness when testing by moving the date forward, CS > replication has stopped working. I kick it with ipa-csreplica-manage > force-sync --from= and that fixes things. This is unrelated to > my patch. > > rob ACK. Honza -- Jan Cholasta From abokovoy at redhat.com Mon Jul 30 11:34:51 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 30 Jul 2012 14:34:51 +0300 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <50131248.20005@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> Message-ID: <20120730113450.GB19710@redhat.com> On Fri, 27 Jul 2012, Rob Crittenden wrote: >Alexander Bokovoy wrote: >>On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >>>Hi, >>> >>>When setting up AD trusts support, ipa-adtrust-install utility >>>needs to be run as: >>> - root, for performing Samba configuration and using LDAPI/autobind >>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >>> fetch keytab >>> >>>As result, we can get rid of Directory Manager credentials in >>>ipa-adtrust-install >>> >>>https://fedorahosted.org/freeipa/ticket/2815 >>> >>>This ticket also simplifies a bit the way we handle admin connection in >>>Service class and particulary in Service._ldap_mod() by defaulting to >>>LDAPI/autobind in case of running as root and to GSSAPI otherwise. >>>Except few cases in remote replica management (not applicable in >>>_ldap_mod() case) we always run installation tools as root and can >>>benefit from using autobind feature. Unfortunately, it is not yet >>>possible to get away from using DM credentials for all cases as the same >>>class is used to perform initial directory server instance >>>configuration. >>> >>>One side effect is explicit disconnect and reconnect in >>>Service.add_cert_to_service() due to way how SimpleLDAPObject class >>>handles stale connections (no handling at all). I've put some comments >>>in place so that others would not try to err out optimizing it in >>>future. >>> >>>Finally, with next patch series which will introduce syncing ipaNTHash >>>attribute with RC4 key in existing kerberos credentials, we can remove >>>requirements to change passwords or re-kinit for majority of trust >>>cases. This should then conclude our trusts content for beta2 release. >> >>Patch updated, fixed small typo (auth_parms was initialized as >>auth_params which led to non-existing auth_parms in ipa-adtrust-install >>case). > >Nack, a couple of minor issues: > >The exception handling is rather unusual in >ensure_kerberos_admin_rights(api). I'm not sure if this is any more >efficient than a series of excepts... I've rewrote this code and put it directly in the main. >You don't need to pass in api, it's a global. Fixed. >It may be safe to see if the user is in the group the way you are >doing it, I wonder if it would be clearer to cast those into DN >objects. Not sure if checking DNs would be sustaining in long run. Ideally we should check ACI here, not just hardcoded group name. I'd like to keep it explicit with memberof for now because it shows what exactly we want to check. >In the Service class what is the point of ldapi if it is going to be >ignored in the case we know the realm? What if I really, really just >want to use a password? LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. No realm -- no LDAPI use because we otherwise cannot construct the socket name. For 'just want to use a password' case you can simply set self.dm_password. However, I've changed the code in Service.ldap_connect() to do following: 1. if DM password is provided, we'll try to use it 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm 3. Otherwise (ldapi was False or realm not provided), we'll try to connect to fqdn:389 with GSSAPI I think this covers all cases. >And later where it forces ldapi, it seems better to either commit all >the way and drop the ldapi argument or convert it to a better name >(like autobind). ldapi requires realm but can be used with either GSSAPI or autobind. Calling it autobind isn't really correct as autobind only available on ldapi under root. -- / Alexander Bokovoy -------------- next part -------------- >From d7fea3f6f50218821560ea739a26030f3a76640f Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/6] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 37 +++++++++++++---------- install/tools/man/ipa-adtrust-install.1 | 3 -- ipaserver/install/adtrustinstance.py | 21 ++++++------- ipaserver/install/service.py | 53 ++++++++++++++++++++++++--------- 4 files changed, 71 insertions(+), 43 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..f1cff30bd439ee07226034b3e49ae17cc59ce6b0 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -194,24 +192,31 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + except krbV.Krb5Error, e: + sys.exit("Must have Kerberos credentials to setup AD trusts on server") - # try the connection try: - smb.ldap_connect() - smb.ldap_disconnect() - except ldap.INVALID_CREDENTIALS, e: - sys.exit("Password is not valid!") + api.Backend.ldap2.connect(ccache.name) + except errors.ACIError, e: + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + try: + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except errors.RequirementError, e: + sys.exit("Must have administrative privileges to setup AD trusts on server") + except Exception, e: + sys.exit("Unrecognized error during check of admin rights: %s" % (str(e))) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, options.no_msdcs) diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..f08fd47306c301d67483959034c163accd6f3b6e 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -55,7 +55,7 @@ def print_msg(message, output_fd=sys.stdout): class Service(object): - def __init__(self, service_name, sstore=None, dm_password=None, ldapi=False): + def __init__(self, service_name, sstore=None, dm_password=None, ldapi=True): self.service_name = service_name self.service = ipaservices.service(service_name) self.steps = [] @@ -77,12 +77,16 @@ class Service(object): self.dercert = None def ldap_connect(self): - if self.ldapi: - if not self.realm: - raise RuntimeError('realm must be set to use ldapi connection') - self.admin_conn = self.__get_conn(None, None, ldapi=True, realm=self.realm) - else: + # If DM password is provided, we use it + # Otherwise, we attempt LDAPI connection if realm is available and + # ldapi was asked for + # Finally, we attempt to connect to self.fqdn with GSSAPI + if self.dm_password: self.admin_conn = self.__get_conn(self.fqdn, self.dm_password) + elif self.ldapi and self.realm: + self.admin_conn = self.__get_conn(None, None, ldapi=True, realm=self.realm) + else: + self.admin_conn = self.__get_conn(self.fqdn, None, ldapi=self.ldapi, realm=self.realm) def ldap_disconnect(self): self.admin_conn.unbind() @@ -93,7 +97,6 @@ class Service(object): pw_name = None fd = None path = ipautil.SHARE_DIR + ldif - hostname = installutils.get_fqdn() nologlist=[] if sub_dict is not None: @@ -107,15 +110,25 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # As we always connect to the local host, + # use URI of admin connection + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + + auth_parms = [] if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + # always try GSSAPI auth when not using DM password or not being root + if os.getegid() != 0: + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: @@ -181,8 +194,19 @@ class Service(object): This server cert should be in DER format. """ - if not self.admin_conn: - self.ldap_connect() + # add_cert_to_service() is relatively rare operation + # we actually call it twice during ipa-server-install, for different + # instances: ds and cs. Unfortunately, it may happen that admin + # connection was created well before add_cert_to_service() is called + # If there are other operations in between, it will become stale and + # since we are using SimpleLDAPObject, not ReconnectLDAPObject, the + # action will fail. Thus, explicitly disconnect and connect again. + # Using ReconnectLDAPObject instead of SimpleLDAPObject was considered + # but consequences for other parts of the framework are largely + # unknown. + if self.admin_conn: + self.ldap_disconnect() + self.ldap_connect() dn = "krbprincipalname=%s,cn=services,cn=accounts,%s" % (self.principal, self.suffix) mod = [(ldap.MOD_ADD, 'userCertificate', self.dercert)] @@ -270,16 +294,17 @@ class Service(object): def __get_conn(self, fqdn, dm_password, ldapi=False, realm=None): # If we are passed a password we'll use it as the DM password - # otherwise we'll do a GSSAPI bind. + # otherwise we'll do a GSSAPI bind unless we are using LDAPI under + # root, in which case we'll try autobind and revert to GSSAPI if + # that attempt has failed try: -# conn = ipaldap.IPAdmin(fqdn, port=636, cacert=CACERT) if ldapi: conn = ipaldap.IPAdmin(ldapi=ldapi, realm=realm) else: conn = ipaldap.IPAdmin(fqdn, port=389) if dm_password: conn.do_simple_bind(bindpw=dm_password) - elif os.getegid() == 0 and self.ldapi: + elif os.getegid() == 0 and ldapi: try: # autobind pw_name = pwd.getpwuid(os.geteuid()).pw_name -- 1.7.11.2 From mkosek at redhat.com Mon Jul 30 11:41:16 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 30 Jul 2012 13:41:16 +0200 Subject: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates In-Reply-To: <50164BD2.7000705@redhat.com> References: <500415CC.9080706@redhat.com> <20120716202647.GA14554@redhat.com> <50047B18.8070209@redhat.com> <50047EBA.9090400@redhat.com> <500574F7.6060300@redhat.com> <500DAE0F.8080906@redhat.com> <500E8116.1020303@redhat.com> <500FFC4F.8090601@redhat.com> <5010035A.2060108@redhat.com> <50105DF1.5030507@redhat.com> <501174DA.9010909@redhat.com> <5012FF20.5070904@redhat.com> <50164BD2.7000705@redhat.com> Message-ID: <501672DC.3020008@redhat.com> On 07/30/2012 10:54 AM, Jan Cholasta wrote: > Dne 27.7.2012 22:50, Rob Crittenden napsal(a): >> Jan Cholasta wrote: >>> Dne 25.7.2012 22:58, Rob Crittenden napsal(a): >>>> Jan Cholasta wrote: >>>> >>>>> All these scripts could use more exception handling, but I guess >>>>> potential bugs can be sorted out later. >>>> >>>> Well, they all run in the background so even if they caught errors >>>> nothing would see them unless we decide to syslog errors. >> >> I decided to syslog the errors, there is no other way around this. >> >>>>> >>>>> install/share/default-aci.ldif: >>>>> >>>>> The ACIs are wrong (Kerberos principal instead of ldap URI in >>>>> userdn, in >>>>> 40-delegation.update it is done right). >>>> >>>> Nice catch, not sure how I missed that. Fixed. >>> >>> You forgot to fix the allow(add) one, it still has userdn = >>> "host/$FQDN@$REALM". >>> >> >> Fixed. >> >>> I did: >>> >>> 1. ipa-server-install on host1, using IPA from master >>> 2. ipa-replica-install on host2, using IPA from master >>> 3. update host1 to IPA with your patch applied >>> 4. update host2 to IPA with your patch applied >>> 5. ipa-ca-install on host2 >>> >>> After that, ipaCert is not tracked on host2 at all (I had to add it >>> manually using "getcert start-tracking -d /etc/httpd/alias -n ipaCert -c >>> dogtag-ipa-retrieve-agent-submit -C >>> /usr/lib64/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt >>> -T ipaCert"). >> >> Fixed, it wasn't being tracked on upgrades. >> >> I filed a ticket for the audit cert renewing for only 6 months. It is a >> dogtag bug. > > OK, thanks. > >> >> I've seen some oddness when testing by moving the date forward, CS >> replication has stopped working. I kick it with ipa-csreplica-manage >> force-sync --from= and that fixes things. This is unrelated to >> my patch. >> >> rob > > ACK. > > Honza > Pushed to master. Martin From simo at redhat.com Mon Jul 30 12:54:00 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 30 Jul 2012 08:54:00 -0400 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <20120730113450.GB19710@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> Message-ID: <1343652840.20530.6.camel@willson.li.ssimo.org> On Mon, 2012-07-30 at 14:34 +0300, Alexander Bokovoy wrote: > On Fri, 27 Jul 2012, Rob Crittenden wrote: > >Alexander Bokovoy wrote: > >>On Thu, 26 Jul 2012, Alexander Bokovoy wrote: > >>>Hi, > >>> > >>>When setting up AD trusts support, ipa-adtrust-install utility > >>>needs to be run as: > >>> - root, for performing Samba configuration and using LDAPI/autobind > >>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to > >>> fetch keytab > >>> > >>>As result, we can get rid of Directory Manager credentials in > >>>ipa-adtrust-install > >>> > >>>https://fedorahosted.org/freeipa/ticket/2815 > >>> > >>>This ticket also simplifies a bit the way we handle admin connection in > >>>Service class and particulary in Service._ldap_mod() by defaulting to > >>>LDAPI/autobind in case of running as root and to GSSAPI otherwise. > >>>Except few cases in remote replica management (not applicable in > >>>_ldap_mod() case) we always run installation tools as root and can > >>>benefit from using autobind feature. Unfortunately, it is not yet > >>>possible to get away from using DM credentials for all cases as the same > >>>class is used to perform initial directory server instance > >>>configuration. > >>> > >>>One side effect is explicit disconnect and reconnect in > >>>Service.add_cert_to_service() due to way how SimpleLDAPObject class > >>>handles stale connections (no handling at all). I've put some comments > >>>in place so that others would not try to err out optimizing it in > >>>future. > >>> > >>>Finally, with next patch series which will introduce syncing ipaNTHash > >>>attribute with RC4 key in existing kerberos credentials, we can remove > >>>requirements to change passwords or re-kinit for majority of trust > >>>cases. This should then conclude our trusts content for beta2 release. > >> > >>Patch updated, fixed small typo (auth_parms was initialized as > >>auth_params which led to non-existing auth_parms in ipa-adtrust-install > >>case). > > > >Nack, a couple of minor issues: > > > >The exception handling is rather unusual in > >ensure_kerberos_admin_rights(api). I'm not sure if this is any more > >efficient than a series of excepts... > I've rewrote this code and put it directly in the main. > > >You don't need to pass in api, it's a global. > Fixed. > > > >It may be safe to see if the user is in the group the way you are > >doing it, I wonder if it would be clearer to cast those into DN > >objects. > Not sure if checking DNs would be sustaining in long run. Ideally we > should check ACI here, not just hardcoded group name. I'd like to keep > it explicit with memberof for now because it shows what exactly we want > to check. > > >In the Service class what is the point of ldapi if it is going to be > >ignored in the case we know the realm? What if I really, really just > >want to use a password? > LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. > No realm -- no LDAPI use because we otherwise cannot construct the > socket name. For 'just want to use a password' case you can simply set > self.dm_password. > > However, I've changed the code in Service.ldap_connect() to do > following: > > 1. if DM password is provided, we'll try to use it > 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm > 3. Otherwise (ldapi was False or realm not provided), we'll try to > connect to fqdn:389 with GSSAPI > > I think this covers all cases. > > >And later where it forces ldapi, it seems better to either commit all > >the way and drop the ldapi argument or convert it to a better name > >(like autobind). > ldapi requires realm but can be used with either GSSAPI or autobind. > Calling it autobind isn't really correct as autobind only available on > ldapi under root. Alexander, the realm is always available ion api.* so it seem that the case where the realm is missing can basically never happen, and it seem like it make no sense to explicitly check that case. This means you'll never fall back to GSSAPI unless you also check what user are you running as. I think we should not try ldapi unless explicitly requested if we are not root (uid 0). Simo. -- Simo Sorce * Red Hat, Inc * New York From mkosek at redhat.com Mon Jul 30 13:05:39 2012 From: mkosek at redhat.com (Martin Kosek) Date: Mon, 30 Jul 2012 15:05:39 +0200 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <20120730113450.GB19710@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> Message-ID: <501686A3.2000405@redhat.com> On 07/30/2012 01:34 PM, Alexander Bokovoy wrote: > On Fri, 27 Jul 2012, Rob Crittenden wrote: >> Alexander Bokovoy wrote: >>> On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >>>> Hi, >>>> >>>> When setting up AD trusts support, ipa-adtrust-install utility >>>> needs to be run as: >>>> - root, for performing Samba configuration and using LDAPI/autobind >>>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >>>> fetch keytab >>>> >>>> As result, we can get rid of Directory Manager credentials in >>>> ipa-adtrust-install >>>> >>>> https://fedorahosted.org/freeipa/ticket/2815 >>>> >>>> This ticket also simplifies a bit the way we handle admin connection in >>>> Service class and particulary in Service._ldap_mod() by defaulting to >>>> LDAPI/autobind in case of running as root and to GSSAPI otherwise. >>>> Except few cases in remote replica management (not applicable in >>>> _ldap_mod() case) we always run installation tools as root and can >>>> benefit from using autobind feature. Unfortunately, it is not yet >>>> possible to get away from using DM credentials for all cases as the same >>>> class is used to perform initial directory server instance >>>> configuration. >>>> >>>> One side effect is explicit disconnect and reconnect in >>>> Service.add_cert_to_service() due to way how SimpleLDAPObject class >>>> handles stale connections (no handling at all). I've put some comments >>>> in place so that others would not try to err out optimizing it in >>>> future. >>>> >>>> Finally, with next patch series which will introduce syncing ipaNTHash >>>> attribute with RC4 key in existing kerberos credentials, we can remove >>>> requirements to change passwords or re-kinit for majority of trust >>>> cases. This should then conclude our trusts content for beta2 release. >>> >>> Patch updated, fixed small typo (auth_parms was initialized as >>> auth_params which led to non-existing auth_parms in ipa-adtrust-install >>> case). >> >> Nack, a couple of minor issues: >> >> The exception handling is rather unusual in >> ensure_kerberos_admin_rights(api). I'm not sure if this is any more efficient >> than a series of excepts... > I've rewrote this code and put it directly in the main. > >> You don't need to pass in api, it's a global. > Fixed. > > >> It may be safe to see if the user is in the group the way you are doing it, I >> wonder if it would be clearer to cast those into DN objects. > Not sure if checking DNs would be sustaining in long run. Ideally we > should check ACI here, not just hardcoded group name. I'd like to keep > it explicit with memberof for now because it shows what exactly we want > to check. > >> In the Service class what is the point of ldapi if it is going to be ignored >> in the case we know the realm? What if I really, really just want to use a >> password? > LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. > No realm -- no LDAPI use because we otherwise cannot construct the > socket name. For 'just want to use a password' case you can simply set > self.dm_password. > > However, I've changed the code in Service.ldap_connect() to do > following: > > 1. if DM password is provided, we'll try to use it > 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm > 3. Otherwise (ldapi was False or realm not provided), we'll try to > connect to fqdn:389 with GSSAPI > > I think this covers all cases. > >> And later where it forces ldapi, it seems better to either commit all the way >> and drop the ldapi argument or convert it to a better name (like autobind). > ldapi requires realm but can be used with either GSSAPI or autobind. > Calling it autobind isn't really correct as autobind only available on > ldapi under root. > Works fine, I also have just few minor-ish issues: 1) Uncatched exception We may want to also catch for DatabaseException in this section: + api.Backend.ldap2.connect(ccache.name) + except errors.ACIError, e: + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket") Otherwise ipa-adtrust-install throws unexpected exception when IPA is down: # ipactl stop # ipa-adtrust-install ... NetBIOS domain name [IDM]: Unexpected error - see /var/log/ipaserver-install.log for details: DatabaseError: Can't contact LDAP server: 2) Wrong indentation: ... + except errors.RequirementError, e: + sys.exit("Must have administrative privileges to setup AD trusts on server") + except Exception, e: + sys.exit("Unrecognized error during check of admin rights: %s" % (str(e))) Martin From abokovoy at redhat.com Mon Jul 30 13:17:17 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Mon, 30 Jul 2012 16:17:17 +0300 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <1343652840.20530.6.camel@willson.li.ssimo.org> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> <1343652840.20530.6.camel@willson.li.ssimo.org> Message-ID: <20120730131717.GD19710@redhat.com> On Mon, 30 Jul 2012, Simo Sorce wrote: >On Mon, 2012-07-30 at 14:34 +0300, Alexander Bokovoy wrote: >> On Fri, 27 Jul 2012, Rob Crittenden wrote: >> >Alexander Bokovoy wrote: >> >>On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >> >>>Hi, >> >>> >> >>>When setting up AD trusts support, ipa-adtrust-install utility >> >>>needs to be run as: >> >>> - root, for performing Samba configuration and using LDAPI/autobind >> >>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >> >>> fetch keytab >> >>> >> >>>As result, we can get rid of Directory Manager credentials in >> >>>ipa-adtrust-install >> >>> >> >>>https://fedorahosted.org/freeipa/ticket/2815 >> >>> >> >>>This ticket also simplifies a bit the way we handle admin connection in >> >>>Service class and particulary in Service._ldap_mod() by defaulting to >> >>>LDAPI/autobind in case of running as root and to GSSAPI otherwise. >> >>>Except few cases in remote replica management (not applicable in >> >>>_ldap_mod() case) we always run installation tools as root and can >> >>>benefit from using autobind feature. Unfortunately, it is not yet >> >>>possible to get away from using DM credentials for all cases as the same >> >>>class is used to perform initial directory server instance >> >>>configuration. >> >>> >> >>>One side effect is explicit disconnect and reconnect in >> >>>Service.add_cert_to_service() due to way how SimpleLDAPObject class >> >>>handles stale connections (no handling at all). I've put some comments >> >>>in place so that others would not try to err out optimizing it in >> >>>future. >> >>> >> >>>Finally, with next patch series which will introduce syncing ipaNTHash >> >>>attribute with RC4 key in existing kerberos credentials, we can remove >> >>>requirements to change passwords or re-kinit for majority of trust >> >>>cases. This should then conclude our trusts content for beta2 release. >> >> >> >>Patch updated, fixed small typo (auth_parms was initialized as >> >>auth_params which led to non-existing auth_parms in ipa-adtrust-install >> >>case). >> > >> >Nack, a couple of minor issues: >> > >> >The exception handling is rather unusual in >> >ensure_kerberos_admin_rights(api). I'm not sure if this is any more >> >efficient than a series of excepts... >> I've rewrote this code and put it directly in the main. >> >> >You don't need to pass in api, it's a global. >> Fixed. >> >> >> >It may be safe to see if the user is in the group the way you are >> >doing it, I wonder if it would be clearer to cast those into DN >> >objects. >> Not sure if checking DNs would be sustaining in long run. Ideally we >> should check ACI here, not just hardcoded group name. I'd like to keep >> it explicit with memberof for now because it shows what exactly we want >> to check. >> >> >In the Service class what is the point of ldapi if it is going to be >> >ignored in the case we know the realm? What if I really, really just >> >want to use a password? >> LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. >> No realm -- no LDAPI use because we otherwise cannot construct the >> socket name. For 'just want to use a password' case you can simply set >> self.dm_password. >> >> However, I've changed the code in Service.ldap_connect() to do >> following: >> >> 1. if DM password is provided, we'll try to use it >> 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm >> 3. Otherwise (ldapi was False or realm not provided), we'll try to >> connect to fqdn:389 with GSSAPI >> >> I think this covers all cases. >> >> >And later where it forces ldapi, it seems better to either commit all >> >the way and drop the ldapi argument or convert it to a better name >> >(like autobind). >> ldapi requires realm but can be used with either GSSAPI or autobind. >> Calling it autobind isn't really correct as autobind only available on >> ldapi under root. > >Alexander, the realm is always available ion api.* so it seem that the >case where the realm is missing can basically never happen, and it seem >like it make no sense to explicitly check that case. This is the code that has no access to api.* because it is run during install when config files are not yet written. >This means you'll never fall back to GSSAPI unless you also check what >user are you running as. I think we should not try ldapi unless >explicitly requested if we are not root (uid 0). There are checks already, look at Service.__get_conn() for real logic. In Service class there are two stages: 1. Service.ldap_connect() which takes existing values from Service's self and decides how to call __get_conn(). 2. Service.__get_conn() which does connection and binds depending on its arguments. This split seems to be historical as nobody else is calling into __get_conn() at all so all possible users of __get_conn() are in Service.ldap_connect(). -- / Alexander Bokovoy From simo at redhat.com Mon Jul 30 14:33:07 2012 From: simo at redhat.com (Simo Sorce) Date: Mon, 30 Jul 2012 10:33:07 -0400 Subject: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user In-Reply-To: <1343429699.20530.3.camel@willson.li.ssimo.org> References: <1341693912.2599.15.camel@willson.li.ssimo.org> <20120711115521.GA9427@redhat.com> <1342008409.2599.62.camel@willson.li.ssimo.org> <20120711124121.GB9427@redhat.com> <1342012841.2599.75.camel@willson.li.ssimo.org> <20120711134037.GC9427@redhat.com> <1342030285.2599.82.camel@willson.li.ssimo.org> <20120712074840.GF9427@redhat.com> <1342094882.2599.973.camel@willson.li.ssimo.org> <20120712124223.GH9427@redhat.com> <20120727041546.GC13180@redhat.com> <1343429699.20530.3.camel@willson.li.ssimo.org> Message-ID: <1343658787.20530.13.camel@willson.li.ssimo.org> On Fri, 2012-07-27 at 18:54 -0400, Simo Sorce wrote: > On Fri, 2012-07-27 at 07:15 +0300, Alexander Bokovoy wrote: > > On Thu, 12 Jul 2012, Alexander Bokovoy wrote: > > >On Thu, 12 Jul 2012, Simo Sorce wrote: > > >>On Thu, 2012-07-12 at 10:48 +0300, Alexander Bokovoy wrote: > > >>>On Wed, 11 Jul 2012, Simo Sorce wrote: > > >>>>From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001 > > >>>>From: Simo Sorce > > >>>>Date: Fri, 6 Jul 2012 16:18:29 -0400 > > >>>>Subject: [PATCH] Add special modify op to regen ipaNTHash > > >>>> > > >>>>The NT Hash is the same thing as the RC4-HMAC key, so we add a function to > > >>>>extract it from krb5 keys if they are available to avoid forcing a password > > >>>>change when configuring trust relationships. > > >>>>--- > > >>>> .../ipa-pwd-extop/ipapwd_prepost.c | 147 +++++++++++++++++++- > > >>>> 1 file changed, 144 insertions(+), 3 deletions(-) > > >>>> > > >>>>diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > > >>>>index deae6477772f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d 100644 > > >>>>--- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > > >>>>+++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c > > >>>>@@ -41,7 +41,12 @@ > > >>>> # include > > >>>> #endif > > >>>> > > >>>>-#define _XOPEN_SOURCE /* strptime needs this */ > > >>>>+/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD > > >>>>+ * _GNU_SOURCE imply both, and we use it elsewhere, so use this */ > > >>>>+#ifndef _GNU_SOURCE > > >>>>+#define _GNU_SOURCE 1 > > >>>>+#endif > > >>>>+ > > >>>> #include > > >>>> #include > > >>>> #include > > >>>>@@ -53,6 +58,7 @@ > > >>>> #include > > >>>> #include > > >>>> #include > > >>>>+#include > > >>>> > > >>>> #include "ipapwd.h" > > >>>> #include "util.h" > > >>>>@@ -379,6 +385,12 @@ done: > > >>>> return 0; > > >>>> } > > >>>> > > >>>>+#define NTHASH_REGEN_VAL "MagicRegen" > > >>>>+#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL) > > >>>>+static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods, > > >>>>+ char *dn, struct slapi_entry *entry, > > >>>>+ struct ipapwd_krbcfg *krbcfg); > > >>>>+ > > >>>> /* PRE MOD Operation: > > >>>> * Gets the clean text password (fail the operation if the password came > > >>>> * pre-hashed, unless this is a replicated operation). > > >>>>@@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > > >>>> int has_krb_keys = 0; > > >>>> int has_history = 0; > > >>>> int gen_krb_keys = 0; > > >>>>+ int is_magic_regen = 0; > > >>>> int ret, rc; > > >>>> > > >>>> LOG_TRACE( "=>\n"); > > >>>>@@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb) > > >>>> default: > > >>>> break; > > >>>> } > > >>>>+ } else if (slapi_attr_types_equivalent(lmod->mod_type, "ipaNTHash")) { > > >>>>+ /* check op filtering out LDAP_MOD_BVALUES */ > > >>>>+ switch (lmod->mod_op & 0x0f) { > > >>>>+ case LDAP_MOD_REPLACE: > > >>>This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD. > > >> > > >>This is because I resent the old patch :( > > >> > > >>Hopefully the correct patch is now attached. > > >Yes, now it is updated, thanks. > > > > > >I'm going to experiment a bit with these patches, adding ipasam > > >responder to test them. > > Here is ipasam part. > > ACK the ipasam part. Pushed all 4 patches to master. Simo. -- Simo Sorce * Red Hat, Inc * New York From sbingram at gmail.com Mon Jul 30 15:30:26 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Mon, 30 Jul 2012 08:30:26 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: References: <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> <500FD914.7040908@redhat.com> Message-ID: On Wed, Jul 25, 2012 at 11:18 PM, Stephen Ingram wrote: > On Wed, Jul 25, 2012 at 4:31 AM, John Dennis wrote: >> On 07/25/2012 02:59 AM, Stephen Ingram wrote: >>> >>> Seeing python2.7, I'm guessing these patches were against Fedora. >>> Since I couldn't get them to apply against RH 6.3 I applied them by >>> hand. I couldn't get the WebUI to load after applying the patches. I'm >>> not sure of the code that caused the problem, but I did see mention of >>> global name datetime not being defined. You wouldn't happen to have >>> patches for RH 6.3? >> >> >> Sorry, I thought you had a F17 install. It should be easy to fix the >> datetime issue, just add this add the top of the file: >> >> import time, datetime >> >> If that simple fix doesn't work then let me know the version you've got and >> I'll make up a new patch against that version. > > Yes, please send a patch for 2.20 with RH 6.3. The import time, > datetime did not address all of the errors. Not sure if you missed this one or not, but, yes, I do need a patch for 2.20 on RH 6.3. I added datetime, but there were still other errors that I couldn't resolve. Steve From jdennis at redhat.com Mon Jul 30 16:15:59 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 30 Jul 2012 12:15:59 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> <500FD914.7040908@redhat.com> Message-ID: <5016B33F.5060105@redhat.com> On 07/30/2012 11:30 AM, Stephen Ingram wrote: > On Wed, Jul 25, 2012 at 11:18 PM, Stephen Ingram wrote: >> On Wed, Jul 25, 2012 at 4:31 AM, John Dennis wrote: >>> On 07/25/2012 02:59 AM, Stephen Ingram wrote: >>>> >>>> Seeing python2.7, I'm guessing these patches were against Fedora. >>>> Since I couldn't get them to apply against RH 6.3 I applied them by >>>> hand. I couldn't get the WebUI to load after applying the patches. I'm >>>> not sure of the code that caused the problem, but I did see mention of >>>> global name datetime not being defined. You wouldn't happen to have >>>> patches for RH 6.3? >>> >>> >>> Sorry, I thought you had a F17 install. It should be easy to fix the >>> datetime issue, just add this add the top of the file: >>> >>> import time, datetime >>> >>> If that simple fix doesn't work then let me know the version you've got and >>> I'll make up a new patch against that version. >> >> Yes, please send a patch for 2.20 with RH 6.3. The import time, >> datetime did not address all of the errors. > > Not sure if you missed this one or not, but, yes, I do need a patch > for 2.20 on RH 6.3. I added datetime, but there were still other > errors that I couldn't resolve. Sorry, I did see it but I was busy with other things. You can try the attached patch. However you need to have a clean set of files to patch against. The patch command I suggested included the -b option and would have created backup files of the original file with a .orig suffix. These are the files the patch touches. ipalib/session.py ipaserver/plugins/ldap2.py ipaserver/rpcserver.py Find their .orig version and cp it back to the original filename. Then you can try applying the patch again. Caveat: I did not test this against 2.20, I just manually made the same set of changes and generated a patch, it's possible in my hurry I made a mistake you may need to tweak, -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa_2_20_timing.patch Type: text/x-patch Size: 3976 bytes Desc: not available URL: From sbingram at gmail.com Mon Jul 30 17:17:32 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Mon, 30 Jul 2012 10:17:32 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <5016B33F.5060105@redhat.com> References: <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> <500FD914.7040908@redhat.com> <5016B33F.5060105@redhat.com> Message-ID: On Mon, Jul 30, 2012 at 9:15 AM, John Dennis wrote: > On 07/30/2012 11:30 AM, Stephen Ingram wrote: >> >> On Wed, Jul 25, 2012 at 11:18 PM, Stephen Ingram >> wrote: >>> >>> On Wed, Jul 25, 2012 at 4:31 AM, John Dennis wrote: >>>> >>>> On 07/25/2012 02:59 AM, Stephen Ingram wrote: >>>>> >>>>> >>>>> Seeing python2.7, I'm guessing these patches were against Fedora. >>>>> Since I couldn't get them to apply against RH 6.3 I applied them by >>>>> hand. I couldn't get the WebUI to load after applying the patches. I'm >>>>> not sure of the code that caused the problem, but I did see mention of >>>>> global name datetime not being defined. You wouldn't happen to have >>>>> patches for RH 6.3? >>>> >>>> >>>> >>>> Sorry, I thought you had a F17 install. It should be easy to fix the >>>> datetime issue, just add this add the top of the file: >>>> >>>> import time, datetime >>>> >>>> If that simple fix doesn't work then let me know the version you've got >>>> and >>>> I'll make up a new patch against that version. >>> >>> >>> Yes, please send a patch for 2.20 with RH 6.3. The import time, >>> datetime did not address all of the errors. >> >> >> Not sure if you missed this one or not, but, yes, I do need a patch >> for 2.20 on RH 6.3. I added datetime, but there were still other >> errors that I couldn't resolve. > > > Sorry, I did see it but I was busy with other things. You can try the > attached patch. However you need to have a clean set of files to patch > against. The patch command I suggested included the -b option and would have > created backup files of the original file with a .orig suffix. These are the > files the patch touches. > > ipalib/session.py > ipaserver/plugins/ldap2.py > ipaserver/rpcserver.py > > > Find their .orig version and cp it back to the original filename. Then you > can try applying the patch again. > > Caveat: I did not test this against 2.20, I just manually made the same set > of changes and generated a patch, it's possible in my hurry I made a mistake > you may need to tweak, This is pretty much the same thing I applied previously by hand. I noticed that you placed import datetime on its own line so I followed suit. Here are the errors I'm receiving: [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.] mod_wsgi (pid=2634): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'. [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] Traceback (most recent call last): [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/share/ipa/wsgi.py", line 49, in application [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return api.Backend.wsgi_dispatch(environ, start_response) [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 234, in __call__ [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return self.route(environ, start_response) [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 246, in route [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return app(environ, start_response) [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 784, in __call__ [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] session_data = session_mgr.load_session_data(environ.get('HTTP_COOKIE')) [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/lib/python2.6/site-packages/ipalib/session.py", line 1004, in load_session_data [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] self.store_session_data(session_data) [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/lib/python2.6/site-packages/ipalib/session.py", line 1047, in store_session_data [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] fmt_time(session_data['session_start_timestamp']), [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] File "/usr/lib/python2.6/site-packages/ipalib/session.py", line 634, in fmt_time [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return datetime.fromtimestamp(timestamp).strftime(ISO8601_DATETIME_FMT) [Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] AttributeError: 'module' object has no attribute 'fromtimestamp' There seems to be a problem with the format of the log information returned? Perhaps the fromtimestamp attribute was added after version 2.20? Steve From pspacek at redhat.com Mon Jul 30 17:47:05 2012 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 30 Jul 2012 19:47:05 +0200 Subject: [Freeipa-devel] [PATCH 0045] Fix zone transfers with non-FQDNs Message-ID: <5016C899.6050303@redhat.com> Hello, this patch enables (finally!) full-fledged AXFR zone-transfers. One-liner was enough, after two days of debugging ... :-) I tried to find all consequences for this change, but please test it thoroughly. Petr^2 Spacek -------------- next part -------------- A non-text attachment was scrubbed... Name: bind-dyndb-ldap-pspacek-0045-Fix-zone-transfers-with-non-FQDNs.patch Type: text/x-patch Size: 989 bytes Desc: not available URL: From jdennis at redhat.com Mon Jul 30 17:52:25 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 30 Jul 2012 13:52:25 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: References: <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <500D9729.9010406@redhat.com> <500EE4DE.6000001@redhat.com> <500FD914.7040908@redhat.com> <5016B33F.5060105@redhat.com> Message-ID: <5016C9D9.1010205@redhat.com> On 07/30/2012 01:17 PM, Stephen Ingram wrote: This is pretty much the same thing I applied previously by hand. I > noticed that you placed import datetime on its own line so I followed > suit. Here are the errors I'm receiving: > > There seems to be a problem with the format of the log information > returned? Perhaps the fromtimestamp attribute was added after version > 2.20? sorry, that should be: return datetime.datetime.fromtimestamp(timestamp).strftime(ISO8601_DATETIME_FMT) In other words datetime has to be repeated twice. That's because fromtimestamp() is an attribute of the datetime class in the datetime module, thus the first datetime references the module and the second datetime references the datetime class in the datetime module. I wish I had a dime for every time I typed just a single datetime ;-) -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From jdennis at redhat.com Mon Jul 30 22:27:36 2012 From: jdennis at redhat.com (John Dennis) Date: Mon, 30 Jul 2012 18:27:36 -0400 Subject: [Freeipa-devel] slow response In-Reply-To: <500D7DE7.8080405@redhat.com> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> Message-ID: <50170A58.9030901@redhat.com> Stephen finally got his server patched with my timing instrumentation. He sent me his log file and I ran my script to parse it. The results are quite interesting. Basically I timed two things cmd_duration is the elapsed time to execute the IPA command. json_duration is the elapsed time to execute the JSON RPC containing the IPA command. The difference (delta) between the two is the time we spend doing "session bookkeeping". In all but the first RPC we were executing from the session cache. Results are below. What is interesting is every command that executes from the session cache seems to have about a 1.5 second constant overhead. The other interesting bits are that most commands in the server execute quickly (from under a tenth of second to .3 second) and on average the server takes 1.5 seconds to respond to RPC (of which most of it appears to be in session code). What is taking so long with session bookkeeping? I don't know yet. I would need more timing instrumentation. I will say when I looked at the python-krb5 code (which we use to populate the ccache from the session and read back to store in the session) seemed to be remarkably inefficient. We also elected to use file based ccache rather than in-memory ccache (that means there is a bit of file-IO occurring). Has anyone used a Python profiler? Would that be better than adding instrumentation? BTW, the actual commands were stripped to protect data anonymity. cmd_duration: 0.107673 json_duration: 3.176758 delta: 3.069085 cmd_duration: 0.020068 json_duration: 1.543135 delta: 1.523067 cmd_duration: 0.387210 json_duration: 1.802954 delta: 1.415744 cmd_duration: 0.024086 json_duration: 1.405276 delta: 1.381190 cmd_duration: 0.342808 json_duration: 1.950365 delta: 1.607557 cmd_duration: 0.019286 json_duration: 1.398786 delta: 1.379500 cmd_duration: 0.200895 json_duration: 1.754358 delta: 1.553463 cmd_duration: 0.020293 json_duration: 1.410701 delta: 1.390408 cmd_duration: 0.383433 json_duration: 1.819478 delta: 1.436045 cmd_duration: 0.020350 json_duration: 1.406038 delta: 1.385688 cmd_duration: 0.346864 json_duration: 1.771281 delta: 1.424417 cmd_duration: 0.015998 json_duration: 1.707743 delta: 1.691745 cmd_duration: 0.314527 json_duration: 1.730894 delta: 1.416367 cmd_duration: 0.025323 json_duration: 1.582828 delta: 1.557505 -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From pspacek at redhat.com Tue Jul 31 08:39:34 2012 From: pspacek at redhat.com (Petr Spacek) Date: Tue, 31 Jul 2012 10:39:34 +0200 Subject: [Freeipa-devel] slow response In-Reply-To: <50170A58.9030901@redhat.com> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> Message-ID: <501799C6.3090002@redhat.com> On 07/31/2012 12:27 AM, John Dennis wrote: > > What is taking so long with session bookkeeping? I don't know yet. I would > need more timing instrumentation. I will say when I looked at the python-krb5 > code (which we use to populate the ccache from the session and read back to > store in the session) seemed to be remarkably inefficient. We also elected to > use file based ccache rather than in-memory ccache (that means there is a bit > of file-IO occurring). A note regarding python-krbV: I used python-krbV extensively in my thesis for KDC stress test. Python-krbV can obtain several thousands of TGTs per second (even with ccache in a file). AFAIK VFS calls are not done synchronously. But others parts of python-krbV were left uncovered, so it can contain some surprises. === Wild speculation follows === 1.5 second is incredibly long time, it sounds like some kind of timeout. Krb5 libs have usual timeout = 1 second per request. Are all KDCs in /etc/krb5.conf alive and reachable? Is SSSD running on problematic server? Is proper KDC selected by SSSD KDC auto-locator plugin? (See /var/lib/sss/pubconf/) === End of wild speculations === Petr^2 Spacek From abokovoy at redhat.com Tue Jul 31 12:00:05 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 31 Jul 2012 15:00:05 +0300 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <501686A3.2000405@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> <501686A3.2000405@redhat.com> Message-ID: <20120731120005.GF19710@redhat.com> On Mon, 30 Jul 2012, Martin Kosek wrote: >On 07/30/2012 01:34 PM, Alexander Bokovoy wrote: >> On Fri, 27 Jul 2012, Rob Crittenden wrote: >>> Alexander Bokovoy wrote: >>>> On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >>>>> Hi, >>>>> >>>>> When setting up AD trusts support, ipa-adtrust-install utility >>>>> needs to be run as: >>>>> - root, for performing Samba configuration and using LDAPI/autobind >>>>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >>>>> fetch keytab >>>>> >>>>> As result, we can get rid of Directory Manager credentials in >>>>> ipa-adtrust-install >>>>> >>>>> https://fedorahosted.org/freeipa/ticket/2815 >>>>> >>>>> This ticket also simplifies a bit the way we handle admin connection in >>>>> Service class and particulary in Service._ldap_mod() by defaulting to >>>>> LDAPI/autobind in case of running as root and to GSSAPI otherwise. >>>>> Except few cases in remote replica management (not applicable in >>>>> _ldap_mod() case) we always run installation tools as root and can >>>>> benefit from using autobind feature. Unfortunately, it is not yet >>>>> possible to get away from using DM credentials for all cases as the same >>>>> class is used to perform initial directory server instance >>>>> configuration. >>>>> >>>>> One side effect is explicit disconnect and reconnect in >>>>> Service.add_cert_to_service() due to way how SimpleLDAPObject class >>>>> handles stale connections (no handling at all). I've put some comments >>>>> in place so that others would not try to err out optimizing it in >>>>> future. >>>>> >>>>> Finally, with next patch series which will introduce syncing ipaNTHash >>>>> attribute with RC4 key in existing kerberos credentials, we can remove >>>>> requirements to change passwords or re-kinit for majority of trust >>>>> cases. This should then conclude our trusts content for beta2 release. >>>> >>>> Patch updated, fixed small typo (auth_parms was initialized as >>>> auth_params which led to non-existing auth_parms in ipa-adtrust-install >>>> case). >>> >>> Nack, a couple of minor issues: >>> >>> The exception handling is rather unusual in >>> ensure_kerberos_admin_rights(api). I'm not sure if this is any more efficient >>> than a series of excepts... >> I've rewrote this code and put it directly in the main. >> >>> You don't need to pass in api, it's a global. >> Fixed. >> >> >>> It may be safe to see if the user is in the group the way you are doing it, I >>> wonder if it would be clearer to cast those into DN objects. >> Not sure if checking DNs would be sustaining in long run. Ideally we >> should check ACI here, not just hardcoded group name. I'd like to keep >> it explicit with memberof for now because it shows what exactly we want >> to check. >> >>> In the Service class what is the point of ldapi if it is going to be ignored >>> in the case we know the realm? What if I really, really just want to use a >>> password? >> LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. >> No realm -- no LDAPI use because we otherwise cannot construct the >> socket name. For 'just want to use a password' case you can simply set >> self.dm_password. >> >> However, I've changed the code in Service.ldap_connect() to do >> following: >> >> 1. if DM password is provided, we'll try to use it >> 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm >> 3. Otherwise (ldapi was False or realm not provided), we'll try to >> connect to fqdn:389 with GSSAPI >> >> I think this covers all cases. >> >>> And later where it forces ldapi, it seems better to either commit all the way >>> and drop the ldapi argument or convert it to a better name (like autobind). >> ldapi requires realm but can be used with either GSSAPI or autobind. >> Calling it autobind isn't really correct as autobind only available on >> ldapi under root. >> > >Works fine, I also have just few minor-ish issues: > >1) Uncatched exception > >We may want to also catch for DatabaseException in this section: > >+ api.Backend.ldap2.connect(ccache.name) >+ except errors.ACIError, e: >+ sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to >update your ticket") > >Otherwise ipa-adtrust-install throws unexpected exception when IPA is down: > ># ipactl stop ># ipa-adtrust-install >... >NetBIOS domain name [IDM]: > >Unexpected error - see /var/log/ipaserver-install.log for details: >DatabaseError: Can't contact LDAP server: > > >2) Wrong indentation: >... >+ except errors.RequirementError, e: >+ sys.exit("Must have administrative privileges to setup AD trusts on >server") >+ except Exception, e: >+ sys.exit("Unrecognized error during check of admin rights: %s" % >(str(e))) Updated patch, includes fixes for issues mentioned above and also implements autobind suggestions by Simo. We have SimpleServiceInstance() that doesn't have realm known by default and this means certain protection is needed for missing realm. Also DS and CA DS instances cannot use LDAPI and autobind when being setup, only DM password. The patch handles these cases too. -- / Alexander Bokovoy -------------- next part -------------- >From 29bfb9491614afba470787df05a150f424a4e26f Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/6] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 46 +++++++------ install/tools/man/ipa-adtrust-install.1 | 3 - ipaserver/install/adtrustinstance.py | 21 +++--- ipaserver/install/cainstance.py | 3 +- ipaserver/install/dsinstance.py | 2 +- ipaserver/install/krbinstance.py | 2 +- ipaserver/install/service.py | 114 +++++++++++++++++++++----------- 7 files changed, 115 insertions(+), 76 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..02a309306fb5743c94b651ab22afa06b5eb2cc8b 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -24,7 +24,7 @@ from ipaserver.plugins.ldap2 import ldap2 from ipaserver.install import adtrustinstance from ipaserver.install.installutils import * -from ipaserver.install import installutils +from ipaserver.install import service from ipapython import version from ipapython import ipautil, sysrestore from ipalib import api, errors, util @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -98,7 +96,7 @@ def main(): root_logger.debug('%s was invoked with options: %s' % (sys.argv[0], safe_options)) root_logger.debug("missing options might be asked for interactively later\n") - installutils.check_server_configuration() + check_server_configuration() global fstore fstore = sysrestore.FileStore('/var/lib/ipa/sysrestore') @@ -194,24 +192,34 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + except krbV.Krb5Error, e: + sys.exit("Must have Kerberos credentials to setup AD trusts on server") - # try the connection try: - smb.ldap_connect() - smb.ldap_disconnect() - except ldap.INVALID_CREDENTIALS, e: - sys.exit("Password is not valid!") + api.Backend.ldap2.connect(ccache.name) + except errors.ACIError, e: + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket") + except errors.DatabaseError, e: + sys.exit("Cannot connect to the LDAP database. Please check if IPA is running") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + try: + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except errors.RequirementError, e: + sys.exit("Must have administrative privileges to setup AD trusts on server") + except Exception, e: + sys.exit("Unrecognized error during check of admin rights: %s" % (str(e))) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm + smb.autobind = service.ENABLED smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, options.no_msdcs) @@ -250,5 +258,5 @@ information""" return 0 if __name__ == '__main__': - installutils.run_script(main, log_file_name=log_file_name, + run_script(main, log_file_name=log_file_name, operation_name='ipa-adtrust-install') diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py index 62c1dc4d082df740f675d8447a7dd13d69f2ec88..e011c9ad593cea3e421deb8a4d914d389e4916a9 100644 --- a/ipaserver/install/cainstance.py +++ b/ipaserver/install/cainstance.py @@ -224,10 +224,9 @@ def get_outputList(data): class CADSInstance(service.Service): def __init__(self, host_name=None, realm_name=None, domain_name=None, dm_password=None): - service.Service.__init__(self, "pkids") + service.Service.__init__(self, "pkids", dm_password=dm_password, ldapi=False, autobind=service.DISABLED) self.serverid = "PKI-IPA" self.realm_name = realm_name - self.dm_password = dm_password self.sub_dict = None self.domain = domain_name self.fqdn = host_name diff --git a/ipaserver/install/dsinstance.py b/ipaserver/install/dsinstance.py index 25c449a6e865de01d789a739b31906cb70c6f212..9f3ae7252e45ab3289a85711a2b1202bbe04e137 100644 --- a/ipaserver/install/dsinstance.py +++ b/ipaserver/install/dsinstance.py @@ -160,7 +160,7 @@ info: IPA V2.0 class DsInstance(service.Service): def __init__(self, realm_name=None, domain_name=None, dm_password=None, fstore=None): - service.Service.__init__(self, "dirsrv", dm_password=dm_password) + service.Service.__init__(self, "dirsrv", dm_password=dm_password, ldapi=False, autobind=service.DISABLED) self.realm_name = realm_name self.sub_dict = None self.domain = domain_name diff --git a/ipaserver/install/krbinstance.py b/ipaserver/install/krbinstance.py index 2faf8e19693f891f28838df967399f0bfe2b51a4..8cc50fba4b0ba8d760cc892a624bd64ef09541a6 100644 --- a/ipaserver/install/krbinstance.py +++ b/ipaserver/install/krbinstance.py @@ -178,7 +178,7 @@ class KrbInstance(service.Service): self.start_creation("Configuring Kerberos KDC", 30) self.kpasswd = KpasswdInstance() - self.kpasswd.create_instance('KPASSWD', self.fqdn, self.admin_password, self.suffix) + self.kpasswd.create_instance('KPASSWD', self.fqdn, self.admin_password, self.suffix, realm=self.realm) def create_replica(self, realm_name, master_fqdn, host_name, diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..403dee907891eccda8c2a8c6309041f4c660bea7 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -35,6 +35,11 @@ from ipapython.ipa_log_manager import * CACERT = "/etc/ipa/ca.crt" +# Autobind modes +AUTO = 1 +ENABLED = 2 +DISABLED = 3 + # The service name as stored in cn=masters,cn=ipa,cn=etc. In the tuple # the first value is the *nix service name, the second the start order. SERVICE_LIST = { @@ -55,13 +60,14 @@ def print_msg(message, output_fd=sys.stdout): class Service(object): - def __init__(self, service_name, sstore=None, dm_password=None, ldapi=False): + def __init__(self, service_name, sstore=None, dm_password=None, ldapi=True, autobind=AUTO): self.service_name = service_name self.service = ipaservices.service(service_name) self.steps = [] self.output_fd = sys.stdout self.dm_password = dm_password self.ldapi = ldapi + self.autobind = autobind self.fqdn = socket.gethostname() self.admin_conn = None @@ -77,12 +83,44 @@ class Service(object): self.dercert = None def ldap_connect(self): - if self.ldapi: - if not self.realm: - raise RuntimeError('realm must be set to use ldapi connection') - self.admin_conn = self.__get_conn(None, None, ldapi=True, realm=self.realm) - else: - self.admin_conn = self.__get_conn(self.fqdn, self.dm_password) + # If DM password is provided, we use it + # If autobind was requested, attempt autobind when root and ldapi + # If autobind was disabled or not succeeded, go with GSSAPI + # LDAPI can be used with either autobind or GSSAPI + # LDAPI requires realm to be set + try: + if self.ldapi: + if not self.realm: + raise errors.NotFound("realm is missing for %s" % (self)) + conn = ipaldap.IPAdmin(ldapi=self.ldapi, realm=self.realm) + else: + conn = ipaldap.IPAdmin(self.fqdn, port=389) + if self.dm_password: + conn.do_simple_bind(bindpw=self.dm_password) + elif self.autobind in [AUTO, ENABLED]: + if os.getegid() == 0 and self.ldapi: + try: + # autobind + pw_name = pwd.getpwuid(os.geteuid()).pw_name + conn.do_external_bind(pw_name) + except errors.NotFound, e: + if self.autobind == AUTO: + # Fall back + conn.do_sasl_gssapi_bind() + else: + # autobind was required and failed, raise + # exception that it failed + raise e + else: + conn.do_sasl_gssapi_bind() + else: + conn.do_sasl_gssapi_bind() + except Exception, e: + root_logger.debug("Could not connect to the Directory Server on %s: %s" % (self.fqdn, str(e))) + raise e + + self.admin_conn = conn + def ldap_disconnect(self): self.admin_conn.unbind() @@ -93,7 +131,6 @@ class Service(object): pw_name = None fd = None path = ipautil.SHARE_DIR + ldif - hostname = installutils.get_fqdn() nologlist=[] if sub_dict is not None: @@ -107,15 +144,25 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # As we always connect to the local host, + # use URI of admin connection + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + + auth_parms = [] if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + # always try GSSAPI auth when not using DM password or not being root + if os.getegid() != 0: + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: @@ -181,8 +228,19 @@ class Service(object): This server cert should be in DER format. """ - if not self.admin_conn: - self.ldap_connect() + # add_cert_to_service() is relatively rare operation + # we actually call it twice during ipa-server-install, for different + # instances: ds and cs. Unfortunately, it may happen that admin + # connection was created well before add_cert_to_service() is called + # If there are other operations in between, it will become stale and + # since we are using SimpleLDAPObject, not ReconnectLDAPObject, the + # action will fail. Thus, explicitly disconnect and connect again. + # Using ReconnectLDAPObject instead of SimpleLDAPObject was considered + # but consequences for other parts of the framework are largely + # unknown. + if self.admin_conn: + self.ldap_disconnect() + self.ldap_connect() dn = "krbprincipalname=%s,cn=services,cn=accounts,%s" % (self.principal, self.suffix) mod = [(ldap.MOD_ADD, 'userCertificate', self.dercert)] @@ -268,33 +326,6 @@ class Service(object): self.steps = [] - def __get_conn(self, fqdn, dm_password, ldapi=False, realm=None): - # If we are passed a password we'll use it as the DM password - # otherwise we'll do a GSSAPI bind. - try: -# conn = ipaldap.IPAdmin(fqdn, port=636, cacert=CACERT) - if ldapi: - conn = ipaldap.IPAdmin(ldapi=ldapi, realm=realm) - else: - conn = ipaldap.IPAdmin(fqdn, port=389) - if dm_password: - conn.do_simple_bind(bindpw=dm_password) - elif os.getegid() == 0 and self.ldapi: - try: - # autobind - pw_name = pwd.getpwuid(os.geteuid()).pw_name - conn.do_external_bind(pw_name) - except errors.NotFound: - # Fall back - conn.do_sasl_gssapi_bind() - else: - conn.do_sasl_gssapi_bind() - except Exception, e: - root_logger.debug("Could not connect to the Directory Server on %s: %s" % (fqdn, str(e))) - raise e - - return conn - def ldap_enable(self, name, fqdn, dm_password, ldap_suffix): self.disable() if not self.admin_conn: @@ -318,11 +349,14 @@ class Service(object): raise e class SimpleServiceInstance(Service): - def create_instance(self, gensvc_name=None, fqdn=None, dm_password=None, ldap_suffix=None): + def create_instance(self, gensvc_name=None, fqdn=None, dm_password=None, ldap_suffix=None, realm=None): self.gensvc_name = gensvc_name self.fqdn = fqdn self.dm_password = dm_password self.suffix = ldap_suffix + self.realm = realm + if not realm: + self.ldapi = False self.step("starting %s " % self.service_name, self.__start) self.step("configuring %s to start on boot" % self.service_name, self.__enable) -- 1.7.11.2 From mkosek at redhat.com Tue Jul 31 12:23:00 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 31 Jul 2012 14:23:00 +0200 Subject: [Freeipa-devel] [PATCH] 291 Avoid redundant info message during RPM update Message-ID: <5017CE24.4090102@redhat.com> This is just s short patch fixing a regression introduced in ticket 2652. I would like to get this one to beta 2. -- A change to ipa-ldap-updater (and thus an RPM update %post scriptlet) avoiding redundat "IPA is not configured" message in stderr introdocued in c20d4c71b87365b3b8d9c53418a79f992e68cd00 was reverted in another patch (b5c1ce88a4a3b35adb3b22bc68fb10b49322641a). Return the change back to avoid this message during every RPM update when IPA is not configured. admintool framework was also fixed to avoid print an empty line when an exception without an error message is raised. https://fedorahosted.org/freeipa/ticket/2892 -- Martin Kosek Red Hat Software Engineer Brno, Czech Republic -------------- next part -------------- A non-text attachment was scrubbed... Name: freeipa-mkosek-291-avoid-redundant-info-message-during-rpm-update.patch Type: text/x-patch Size: 2614 bytes Desc: not available URL: From abokovoy at redhat.com Tue Jul 31 12:28:56 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 31 Jul 2012 15:28:56 +0300 Subject: [Freeipa-devel] [PATCH] 0067 Set Domain Users SID for ipausers group Message-ID: <20120731122856.GG19710@redhat.com> Hi, Set 'Domain Users' SID for ipausers group during ipa-adtrust-install Since all users belong to ipausers group, setting Domain Users SID (-513) will give them status of domain users. This is needed for Kerberos driver to generate MS-PAC. -- / Alexander Bokovoy -------------- next part -------------- >From 08aa97ebf2b7958ac58a59a2b48a6db466be2972 Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Tue, 31 Jul 2012 15:25:47 +0300 Subject: [PATCH 3/3] Set 'Domain Users' SID for ipausers group during ipa-adtrust-install Since all users belong to ipausers group, setting Domain Users SID (-513) will give them status of domain users. This is needed for Kerberos driver to generate MS-PAC. --- ipaserver/install/adtrustinstance.py | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a..3f7d6e49646ba12a6a0b01e4505e2477a9b288db 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -128,12 +128,13 @@ class ADTRUSTInstance(service.Service): sub_ids = struct.unpack(" References: <5010FD0A.3090206@redhat.com> <50114AE1.4090209@redhat.com> <50115AE7.8050401@redhat.com> Message-ID: <5017D912.5080708@redhat.com> On 07/26/2012 10:57 AM, Petr Viktorin wrote: > On 07/26/2012 03:49 PM, Petr Viktorin wrote: >> On 07/26/2012 10:17 AM, Petr Viktorin wrote: >>> Update the pot to match current source, and pull translations from >>> Transifex. >>> >>> >>> Warning: when this patch is pushed, the source strings on Transifex will >>> update. The old versions will be lost from the site. >>> >>> >>> The patch is quite large (>5MB), so I haven't attached it here (should >>> I?). Please download it from >>> https://github.com/encukou/freeipa/commit/fd638306ada102204494ec2e7f1b8d2bb7f6f8b1.patch >>> >>> >>> >> >> NACK. I forgot to validate the messages, so some messages might cause >> exceptions when used. >> >> > > Validation found a few bad messages in Spanish, and one in Tajik. I've > corrected obvious errors; in two cases I wasn't sure how to fix so I've > removed the translation. > Attaching my fixes for reference. > > The new patch is here: > https://github.com/encukou/freeipa/commit/4eb5b21fe3bcfec33e07c45050d5a56f4328206c.patch > > > > We should also change these on Transifex. > I assume maintainers can edit all translations. John, could you give me > maintainer rights? My username there is EnCuKou. > ACK, the patch applied cleanly and all the validations passed. Please make sure any po files you manually edited have the same edits applied on TX otherwise the next time we pull we'll reintroduce the same problems. FYI, the message stats after applying the patch are: ipa.pot has 1872 messages. There are 13 po translation files. bn_IN: 12/1872 0.6% 1860 untranslated, 0 fuzzy de: 32/1872 1.7% 1840 untranslated, 0 fuzzy es: 891/1872 47.6% 981 untranslated, 0 fuzzy fr: 1635/1872 87.3% 237 untranslated, 0 fuzzy id: 86/1872 4.6% 1786 untranslated, 0 fuzzy ja: 34/1872 1.8% 1838 untranslated, 0 fuzzy kn: 236/1872 12.6% 1636 untranslated, 0 fuzzy nl: 1/1872 0.1% 1871 untranslated, 0 fuzzy pl: 452/1872 24.1% 1420 untranslated, 0 fuzzy ru: 368/1872 19.7% 1504 untranslated, 0 fuzzy tg: 105/1872 5.6% 1767 untranslated, 0 fuzzy uk: 1635/1872 87.3% 237 untranslated, 0 fuzzy zh_CN: 130/1872 6.9% 1742 untranslated, 0 fuzzy -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ From abokovoy at redhat.com Tue Jul 31 13:40:01 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 31 Jul 2012 16:40:01 +0300 Subject: [Freeipa-devel] [PATCH] 0067 Set Domain Users SID for ipausers group In-Reply-To: <20120731122856.GG19710@redhat.com> References: <20120731122856.GG19710@redhat.com> Message-ID: <20120731134001.GH19710@redhat.com> On Tue, 31 Jul 2012, Alexander Bokovoy wrote: > Hi, > > Set 'Domain Users' SID for ipausers group during ipa-adtrust-install > > Since all users belong to ipausers group, setting Domain Users SID > (-513) will give them status of domain users. This is needed for > Kerberos driver to generate MS-PAC. I've created a ticket for this and primary group fallback issue: https://fedorahosted.org/freeipa/ticket/2955 -- / Alexander Bokovoy From mkosek at redhat.com Tue Jul 31 13:40:51 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 31 Jul 2012 15:40:51 +0200 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <20120731120005.GF19710@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> <501686A3.2000405@redhat.com> <20120731120005.GF19710@redhat.com> Message-ID: <5017E063.2050003@redhat.com> On 07/31/2012 02:00 PM, Alexander Bokovoy wrote: > On Mon, 30 Jul 2012, Martin Kosek wrote: >> On 07/30/2012 01:34 PM, Alexander Bokovoy wrote: >>> On Fri, 27 Jul 2012, Rob Crittenden wrote: >>>> Alexander Bokovoy wrote: >>>>> On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >>>>>> Hi, >>>>>> >>>>>> When setting up AD trusts support, ipa-adtrust-install utility >>>>>> needs to be run as: >>>>>> - root, for performing Samba configuration and using LDAPI/autobind >>>>>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >>>>>> fetch keytab >>>>>> >>>>>> As result, we can get rid of Directory Manager credentials in >>>>>> ipa-adtrust-install >>>>>> >>>>>> https://fedorahosted.org/freeipa/ticket/2815 >>>>>> >>>>>> This ticket also simplifies a bit the way we handle admin connection in >>>>>> Service class and particulary in Service._ldap_mod() by defaulting to >>>>>> LDAPI/autobind in case of running as root and to GSSAPI otherwise. >>>>>> Except few cases in remote replica management (not applicable in >>>>>> _ldap_mod() case) we always run installation tools as root and can >>>>>> benefit from using autobind feature. Unfortunately, it is not yet >>>>>> possible to get away from using DM credentials for all cases as the same >>>>>> class is used to perform initial directory server instance >>>>>> configuration. >>>>>> >>>>>> One side effect is explicit disconnect and reconnect in >>>>>> Service.add_cert_to_service() due to way how SimpleLDAPObject class >>>>>> handles stale connections (no handling at all). I've put some comments >>>>>> in place so that others would not try to err out optimizing it in >>>>>> future. >>>>>> >>>>>> Finally, with next patch series which will introduce syncing ipaNTHash >>>>>> attribute with RC4 key in existing kerberos credentials, we can remove >>>>>> requirements to change passwords or re-kinit for majority of trust >>>>>> cases. This should then conclude our trusts content for beta2 release. >>>>> >>>>> Patch updated, fixed small typo (auth_parms was initialized as >>>>> auth_params which led to non-existing auth_parms in ipa-adtrust-install >>>>> case). >>>> >>>> Nack, a couple of minor issues: >>>> >>>> The exception handling is rather unusual in >>>> ensure_kerberos_admin_rights(api). I'm not sure if this is any more efficient >>>> than a series of excepts... >>> I've rewrote this code and put it directly in the main. >>> >>>> You don't need to pass in api, it's a global. >>> Fixed. >>> >>> >>>> It may be safe to see if the user is in the group the way you are doing it, I >>>> wonder if it would be clearer to cast those into DN objects. >>> Not sure if checking DNs would be sustaining in long run. Ideally we >>> should check ACI here, not just hardcoded group name. I'd like to keep >>> it explicit with memberof for now because it shows what exactly we want >>> to check. >>> >>>> In the Service class what is the point of ldapi if it is going to be ignored >>>> in the case we know the realm? What if I really, really just want to use a >>>> password? >>> LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. >>> No realm -- no LDAPI use because we otherwise cannot construct the >>> socket name. For 'just want to use a password' case you can simply set >>> self.dm_password. >>> >>> However, I've changed the code in Service.ldap_connect() to do >>> following: >>> >>> 1. if DM password is provided, we'll try to use it >>> 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm >>> 3. Otherwise (ldapi was False or realm not provided), we'll try to >>> connect to fqdn:389 with GSSAPI >>> >>> I think this covers all cases. >>> >>>> And later where it forces ldapi, it seems better to either commit all the way >>>> and drop the ldapi argument or convert it to a better name (like autobind). >>> ldapi requires realm but can be used with either GSSAPI or autobind. >>> Calling it autobind isn't really correct as autobind only available on >>> ldapi under root. >>> >> >> Works fine, I also have just few minor-ish issues: >> >> 1) Uncatched exception >> >> We may want to also catch for DatabaseException in this section: >> >> + api.Backend.ldap2.connect(ccache.name) >> + except errors.ACIError, e: >> + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to >> update your ticket") >> >> Otherwise ipa-adtrust-install throws unexpected exception when IPA is down: >> >> # ipactl stop >> # ipa-adtrust-install >> ... >> NetBIOS domain name [IDM]: >> >> Unexpected error - see /var/log/ipaserver-install.log for details: >> DatabaseError: Can't contact LDAP server: >> >> >> 2) Wrong indentation: >> ... >> + except errors.RequirementError, e: >> + sys.exit("Must have administrative privileges to setup AD trusts on >> server") >> + except Exception, e: >> + sys.exit("Unrecognized error during check of admin rights: %s" % >> (str(e))) > Updated patch, includes fixes for issues mentioned above and also > implements autobind suggestions by Simo. > > We have SimpleServiceInstance() that doesn't have realm known by default > and this means certain protection is needed for missing realm. Also DS > and CA DS instances cannot use LDAPI and autobind when being setup, only > DM password. The patch handles these cases too. > Hm, ipa-dns-install is now broken: # ipa-dns-install The log file for this installation can be found in /var/log/ipaserver-install.log ============================================================================== This program will setup DNS for the FreeIPA Server. This includes: * Configure DNS (bind) To accept the default shown in brackets, press the Enter key. Existing BIND configuration detected, overwrite? [no]: y Directory Manager password: Unexpected error - see /var/log/ipaserver-install.log for details: ValueError: non-generic 'NotFound' needs format=None; got format='realm is missing for ' This means there are 2 issues: 1) NotFound error is not risen correctly. Instead of + raise errors.NotFound("realm is missing for %s" % (self)) it should rather be: + raise errors.NotFound(reason="realm is missing for %s" % (self)) 2) LDAP connection in ipa-dns-install (bindinstance.py) is not right. Otherwise everything seemed to work ok. Martin From mkosek at redhat.com Tue Jul 31 13:44:39 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 31 Jul 2012 15:44:39 +0200 Subject: [Freeipa-devel] [PATCH] 0073 Update translations In-Reply-To: <5017D912.5080708@redhat.com> References: <5010FD0A.3090206@redhat.com> <50114AE1.4090209@redhat.com> <50115AE7.8050401@redhat.com> <5017D912.5080708@redhat.com> Message-ID: <5017E147.4090807@redhat.com> On 07/31/2012 03:09 PM, John Dennis wrote: > On 07/26/2012 10:57 AM, Petr Viktorin wrote: >> On 07/26/2012 03:49 PM, Petr Viktorin wrote: >>> On 07/26/2012 10:17 AM, Petr Viktorin wrote: >>>> Update the pot to match current source, and pull translations from >>>> Transifex. >>>> >>>> >>>> Warning: when this patch is pushed, the source strings on Transifex will >>>> update. The old versions will be lost from the site. >>>> >>>> >>>> The patch is quite large (>5MB), so I haven't attached it here (should >>>> I?). Please download it from >>>> https://github.com/encukou/freeipa/commit/fd638306ada102204494ec2e7f1b8d2bb7f6f8b1.patch >>>> >>>> >>>> >>>> >>> >>> NACK. I forgot to validate the messages, so some messages might cause >>> exceptions when used. >>> >>> >> >> Validation found a few bad messages in Spanish, and one in Tajik. I've >> corrected obvious errors; in two cases I wasn't sure how to fix so I've >> removed the translation. >> Attaching my fixes for reference. >> >> The new patch is here: >> https://github.com/encukou/freeipa/commit/4eb5b21fe3bcfec33e07c45050d5a56f4328206c.patch >> >> >> >> >> We should also change these on Transifex. >> I assume maintainers can edit all translations. John, could you give me >> maintainer rights? My username there is EnCuKou. >> > > ACK, the patch applied cleanly and all the validations passed. Please make sure > any po files you manually edited have the same edits applied on TX otherwise > the next time we pull we'll reintroduce the same problems. > > FYI, the message stats after applying the patch are: > > ipa.pot has 1872 messages. There are 13 po translation files. > bn_IN: 12/1872 0.6% 1860 untranslated, 0 fuzzy > de: 32/1872 1.7% 1840 untranslated, 0 fuzzy > es: 891/1872 47.6% 981 untranslated, 0 fuzzy > fr: 1635/1872 87.3% 237 untranslated, 0 fuzzy > id: 86/1872 4.6% 1786 untranslated, 0 fuzzy > ja: 34/1872 1.8% 1838 untranslated, 0 fuzzy > kn: 236/1872 12.6% 1636 untranslated, 0 fuzzy > nl: 1/1872 0.1% 1871 untranslated, 0 fuzzy > pl: 452/1872 24.1% 1420 untranslated, 0 fuzzy > ru: 368/1872 19.7% 1504 untranslated, 0 fuzzy > tg: 105/1872 5.6% 1767 untranslated, 0 fuzzy > uk: 1635/1872 87.3% 237 untranslated, 0 fuzzy > zh_CN: 130/1872 6.9% 1742 untranslated, 0 fuzzy > Pushed to master. Martin From abokovoy at redhat.com Tue Jul 31 14:20:29 2012 From: abokovoy at redhat.com (Alexander Bokovoy) Date: Tue, 31 Jul 2012 17:20:29 +0300 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <5017E063.2050003@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> <501686A3.2000405@redhat.com> <20120731120005.GF19710@redhat.com> <5017E063.2050003@redhat.com> Message-ID: <20120731142029.GI19710@redhat.com> On Tue, 31 Jul 2012, Martin Kosek wrote: >On 07/31/2012 02:00 PM, Alexander Bokovoy wrote: >> On Mon, 30 Jul 2012, Martin Kosek wrote: >>> On 07/30/2012 01:34 PM, Alexander Bokovoy wrote: >>>> On Fri, 27 Jul 2012, Rob Crittenden wrote: >>>>> Alexander Bokovoy wrote: >>>>>> On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >>>>>>> Hi, >>>>>>> >>>>>>> When setting up AD trusts support, ipa-adtrust-install utility >>>>>>> needs to be run as: >>>>>>> - root, for performing Samba configuration and using LDAPI/autobind >>>>>>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >>>>>>> fetch keytab >>>>>>> >>>>>>> As result, we can get rid of Directory Manager credentials in >>>>>>> ipa-adtrust-install >>>>>>> >>>>>>> https://fedorahosted.org/freeipa/ticket/2815 >>>>>>> >>>>>>> This ticket also simplifies a bit the way we handle admin connection in >>>>>>> Service class and particulary in Service._ldap_mod() by defaulting to >>>>>>> LDAPI/autobind in case of running as root and to GSSAPI otherwise. >>>>>>> Except few cases in remote replica management (not applicable in >>>>>>> _ldap_mod() case) we always run installation tools as root and can >>>>>>> benefit from using autobind feature. Unfortunately, it is not yet >>>>>>> possible to get away from using DM credentials for all cases as the same >>>>>>> class is used to perform initial directory server instance >>>>>>> configuration. >>>>>>> >>>>>>> One side effect is explicit disconnect and reconnect in >>>>>>> Service.add_cert_to_service() due to way how SimpleLDAPObject class >>>>>>> handles stale connections (no handling at all). I've put some comments >>>>>>> in place so that others would not try to err out optimizing it in >>>>>>> future. >>>>>>> >>>>>>> Finally, with next patch series which will introduce syncing ipaNTHash >>>>>>> attribute with RC4 key in existing kerberos credentials, we can remove >>>>>>> requirements to change passwords or re-kinit for majority of trust >>>>>>> cases. This should then conclude our trusts content for beta2 release. >>>>>> >>>>>> Patch updated, fixed small typo (auth_parms was initialized as >>>>>> auth_params which led to non-existing auth_parms in ipa-adtrust-install >>>>>> case). >>>>> >>>>> Nack, a couple of minor issues: >>>>> >>>>> The exception handling is rather unusual in >>>>> ensure_kerberos_admin_rights(api). I'm not sure if this is any more efficient >>>>> than a series of excepts... >>>> I've rewrote this code and put it directly in the main. >>>> >>>>> You don't need to pass in api, it's a global. >>>> Fixed. >>>> >>>> >>>>> It may be safe to see if the user is in the group the way you are doing it, I >>>>> wonder if it would be clearer to cast those into DN objects. >>>> Not sure if checking DNs would be sustaining in long run. Ideally we >>>> should check ACI here, not just hardcoded group name. I'd like to keep >>>> it explicit with memberof for now because it shows what exactly we want >>>> to check. >>>> >>>>> In the Service class what is the point of ldapi if it is going to be ignored >>>>> in the case we know the realm? What if I really, really just want to use a >>>>> password? >>>> LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. >>>> No realm -- no LDAPI use because we otherwise cannot construct the >>>> socket name. For 'just want to use a password' case you can simply set >>>> self.dm_password. >>>> >>>> However, I've changed the code in Service.ldap_connect() to do >>>> following: >>>> >>>> 1. if DM password is provided, we'll try to use it >>>> 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm >>>> 3. Otherwise (ldapi was False or realm not provided), we'll try to >>>> connect to fqdn:389 with GSSAPI >>>> >>>> I think this covers all cases. >>>> >>>>> And later where it forces ldapi, it seems better to either commit all the way >>>>> and drop the ldapi argument or convert it to a better name (like autobind). >>>> ldapi requires realm but can be used with either GSSAPI or autobind. >>>> Calling it autobind isn't really correct as autobind only available on >>>> ldapi under root. >>>> >>> >>> Works fine, I also have just few minor-ish issues: >>> >>> 1) Uncatched exception >>> >>> We may want to also catch for DatabaseException in this section: >>> >>> + api.Backend.ldap2.connect(ccache.name) >>> + except errors.ACIError, e: >>> + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to >>> update your ticket") >>> >>> Otherwise ipa-adtrust-install throws unexpected exception when IPA is down: >>> >>> # ipactl stop >>> # ipa-adtrust-install >>> ... >>> NetBIOS domain name [IDM]: >>> >>> Unexpected error - see /var/log/ipaserver-install.log for details: >>> DatabaseError: Can't contact LDAP server: >>> >>> >>> 2) Wrong indentation: >>> ... >>> + except errors.RequirementError, e: >>> + sys.exit("Must have administrative privileges to setup AD trusts on >>> server") >>> + except Exception, e: >>> + sys.exit("Unrecognized error during check of admin rights: %s" % >>> (str(e))) >> Updated patch, includes fixes for issues mentioned above and also >> implements autobind suggestions by Simo. >> >> We have SimpleServiceInstance() that doesn't have realm known by default >> and this means certain protection is needed for missing realm. Also DS >> and CA DS instances cannot use LDAPI and autobind when being setup, only >> DM password. The patch handles these cases too. >> > >Hm, ipa-dns-install is now broken: > ># ipa-dns-install > >The log file for this installation can be found in /var/log/ipaserver-install.log >============================================================================== >This program will setup DNS for the FreeIPA Server. > >This includes: > * Configure DNS (bind) > >To accept the default shown in brackets, press the Enter key. > >Existing BIND configuration detected, overwrite? [no]: y >Directory Manager password: > >Unexpected error - see /var/log/ipaserver-install.log for details: >ValueError: non-generic 'NotFound' needs format=None; got format='realm is >missing for ' > >This means there are 2 issues: > >1) NotFound error is not risen correctly. Instead of >+ raise errors.NotFound("realm is missing for %s" % (self)) >it should rather be: >+ raise errors.NotFound(reason="realm is missing for %s" % >(self)) Fixed. >2) LDAP connection in ipa-dns-install (bindinstance.py) is not right. Fixed. As it requires DM there it was easier to disable ldapi for the instance. We may want to review all use of DM password and use instead a common code like in ipa-adtrust-install that checks Kerberos credentials and utilizes GSSAPI/LDAPI/autobind instead. After all, ipa-dns-install also needs root access to set up /etc/named.conf so there is no need to have DM password. -- / Alexander Bokovoy -------------- next part -------------- >From 3126071bc58b57395495157c435d07b976577d0b Mon Sep 17 00:00:00 2001 From: Alexander Bokovoy Date: Fri, 13 Jul 2012 18:12:48 +0300 Subject: [PATCH 2/3] Ensure ipa-adtrust-install is run with Kerberos ticket for admin user When setting up AD trusts support, ipa-adtrust-install utility needs to be run as: - root, for performing Samba configuration and using LDAPI/autobind - kinit-ed IPA admin user, to ensure proper ACIs are granted to fetch keytab As result, we can get rid of Directory Manager credentials in ipa-adtrust-install https://fedorahosted.org/freeipa/ticket/2815 --- install/tools/ipa-adtrust-install | 46 +++++++------ install/tools/man/ipa-adtrust-install.1 | 3 - ipaserver/install/adtrustinstance.py | 21 +++--- ipaserver/install/bindinstance.py | 2 +- ipaserver/install/cainstance.py | 3 +- ipaserver/install/dsinstance.py | 2 +- ipaserver/install/krbinstance.py | 2 +- ipaserver/install/service.py | 114 +++++++++++++++++++++----------- 8 files changed, 116 insertions(+), 77 deletions(-) diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install index 6678018e6346d75d5042894cfb833d38079d3f21..02a309306fb5743c94b651ab22afa06b5eb2cc8b 100755 --- a/install/tools/ipa-adtrust-install +++ b/install/tools/ipa-adtrust-install @@ -24,7 +24,7 @@ from ipaserver.plugins.ldap2 import ldap2 from ipaserver.install import adtrustinstance from ipaserver.install.installutils import * -from ipaserver.install import installutils +from ipaserver.install import service from ipapython import version from ipapython import ipautil, sysrestore from ipalib import api, errors, util @@ -37,8 +37,6 @@ log_file_name = "/var/log/ipaserver-install.log" def parse_options(): parser = IPAOptionParser(version=version.VERSION) - parser.add_option("-p", "--ds-password", dest="dm_password", - sensitive=True, help="directory manager password") parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="print debugging information") parser.add_option("--ip-address", dest="ip_address", @@ -98,7 +96,7 @@ def main(): root_logger.debug('%s was invoked with options: %s' % (sys.argv[0], safe_options)) root_logger.debug("missing options might be asked for interactively later\n") - installutils.check_server_configuration() + check_server_configuration() global fstore fstore = sysrestore.FileStore('/var/lib/ipa/sysrestore') @@ -194,24 +192,34 @@ def main(): if not options.unattended and ( not netbios_name or not options.netbios_name): netbios_name = read_netbios_name(netbios_name) - dm_password = options.dm_password or read_password("Directory Manager", - confirm=False, validate=False) - smb = adtrustinstance.ADTRUSTInstance(fstore, dm_password) + try: + ctx = krbV.default_context() + ccache = ctx.default_ccache() + principal = ccache.principal() + except krbV.Krb5Error, e: + sys.exit("Must have Kerberos credentials to setup AD trusts on server") - # try the connection try: - smb.ldap_connect() - smb.ldap_disconnect() - except ldap.INVALID_CREDENTIALS, e: - sys.exit("Password is not valid!") + api.Backend.ldap2.connect(ccache.name) + except errors.ACIError, e: + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket") + except errors.DatabaseError, e: + sys.exit("Cannot connect to the LDAP database. Please check if IPA is running") - if smb.dm_password: - api.Backend.ldap2.connect(bind_dn="cn=Directory Manager", bind_pw=smb.dm_password) - else: - # See if our LDAP server is up and we can talk to it over GSSAPI - ccache = krbV.default_context().default_ccache().name - api.Backend.ldap2.connect(ccache) + try: + user = api.Command.user_show(unicode(principal[0]))['result'] + group = api.Command.group_show(u'admins')['result'] + if not (user['uid'][0] in group['member_user'] and + group['cn'][0] in user['memberof_group']): + raise errors.RequirementError(name='admins group membership') + except errors.RequirementError, e: + sys.exit("Must have administrative privileges to setup AD trusts on server") + except Exception, e: + sys.exit("Unrecognized error during check of admin rights: %s" % (str(e))) + smb = adtrustinstance.ADTRUSTInstance(fstore) + smb.realm = api.env.realm + smb.autobind = service.ENABLED smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain, netbios_name, options.rid_base, options.secondary_rid_base, options.no_msdcs) @@ -250,5 +258,5 @@ information""" return 0 if __name__ == '__main__': - installutils.run_script(main, log_file_name=log_file_name, + run_script(main, log_file_name=log_file_name, operation_name='ipa-adtrust-install') diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1 index b61da19088b40d6a9e53784f9a061913ecda4321..22337c3df8827670657bf405b6c49ba2f8624d6d 100644 --- a/install/tools/man/ipa-adtrust-install.1 +++ b/install/tools/man/ipa-adtrust-install.1 @@ -27,9 +27,6 @@ trust to an Active Directory domain. This requires that the IPA server is already installed and configured. .SH "OPTIONS" .TP -\fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR -The password to be used by the Directory Server for the Directory Manager user -.TP \fB\-d\fR, \fB\-\-debug\fR Enable debug logging when more verbose output is needed .TP diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py index 20feec4df309b5793aa1c29fdf18bc5bfe180943..9dcbec2d61d935f90e74cc65b30a0f1d0c0f9d2a 100644 --- a/ipaserver/install/adtrustinstance.py +++ b/ipaserver/install/adtrustinstance.py @@ -96,10 +96,9 @@ class ADTRUSTInstance(service.Service): OBJC_GROUP = "ipaNTGroupAttrs" OBJC_DOMAIN = "ipaNTDomainAttrs" - def __init__(self, fstore=None, dm_password=None): + def __init__(self, fstore=None): self.fqdn = None self.ip_address = None - self.realm_name = None self.domain_name = None self.netbios_name = None self.no_msdcs = None @@ -118,7 +117,7 @@ class ADTRUSTInstance(service.Service): self.rid_base = None self.secondary_rid_base = None - service.Service.__init__(self, "smb", dm_password=dm_password) + service.Service.__init__(self, "smb", dm_password=None, ldapi=True) if fstore: self.fstore = fstore @@ -436,6 +435,8 @@ class ADTRUSTInstance(service.Service): # We do not let the system start IPA components on its own, # Instead we reply on the IPA init script to start only enabled # components as found in our LDAP configuration tree + # Note that self.dm_password is None for ADTrustInstance because + # we ensure to be called as root and using ldapi to use autobind try: self.ldap_enable('ADTRUST', self.fqdn, self.dm_password, \ self.suffix) @@ -449,7 +450,7 @@ class ADTRUSTInstance(service.Service): root_logger.info("EXTID Service startup entry already exists.") def __setup_sub_dict(self): - self.sub_dict = dict(REALM = self.realm_name, + self.sub_dict = dict(REALM = self.realm, SUFFIX = self.suffix, NETBIOS_NAME = self.netbios_name, SMB_DN = self.smb_dn, @@ -460,16 +461,16 @@ class ADTRUSTInstance(service.Service): rid_base, secondary_rid_base, no_msdcs=False, smbd_user="samba"): self.fqdn = fqdn self.ip_address = ip_address - self.realm_name = realm_name + self.realm = realm_name self.domain_name = domain_name self.netbios_name = netbios_name self.rid_base = rid_base self.secondary_rid_base = secondary_rid_base self.no_msdcs = no_msdcs self.smbd_user = smbd_user - self.suffix = ipautil.realm_to_suffix(self.realm_name) + self.suffix = ipautil.realm_to_suffix(self.realm) self.ldapi_socket = "%%2fvar%%2frun%%2fslapd-%s.socket" % \ - realm_to_serverid(self.realm_name) + realm_to_serverid(self.realm) self.smb_conf = "/etc/samba/smb.conf" @@ -479,7 +480,7 @@ class ADTRUSTInstance(service.Service): self.trust_dn = str(DN(api.env.container_trusts, self.suffix)) self.smb_dom_dn = str(DN(('cn', self.domain_name), api.env.container_cifsdomains, self.suffix)) - self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm_name + self.cifs_principal = "cifs/" + self.fqdn + "@" + self.realm self.cifs_agent = str(DN(('krbprincipalname', self.cifs_principal.lower()), api.env.container_service, self.suffix)) @@ -522,11 +523,11 @@ class ADTRUSTInstance(service.Service): "range.\nAdd local ID range manually and try " \ "again!") - entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm_name)), + entry = ipaldap.Entry(str(DN(('cn', ('%s_id_range' % self.realm)), api.env.container_ranges, self.suffix))) entry.setValue('objectclass', 'ipaDomainIDRange') - entry.setValue('cn', ('%s_id_range' % self.realm_name)) + entry.setValue('cn', ('%s_id_range' % self.realm)) entry.setValue('ipaBaseID', str(base_id)) entry.setValue('ipaIDRangeSize', str(id_range_size)) self.admin_conn.addEntry(entry) diff --git a/ipaserver/install/bindinstance.py b/ipaserver/install/bindinstance.py index c348cdbb278f222dfbc034cbe1220df26262cb9d..f320202eaa20fc5e8cf1e61ad11a769a4436ba47 100644 --- a/ipaserver/install/bindinstance.py +++ b/ipaserver/install/bindinstance.py @@ -448,7 +448,7 @@ class DnsBackup(object): class BindInstance(service.Service): def __init__(self, fstore=None, dm_password=None): - service.Service.__init__(self, "named", dm_password=dm_password) + service.Service.__init__(self, "named", dm_password=dm_password, ldapi=False, autobind=service.DISABLED) self.dns_backup = DnsBackup(self) self.named_user = None self.domain = None diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py index 2644689a0bc18fdb97d1e66d3f929af24cd101ba..dc4374ccef4f7bd64edb14d77efe35b46895bfb5 100644 --- a/ipaserver/install/cainstance.py +++ b/ipaserver/install/cainstance.py @@ -225,10 +225,9 @@ def get_outputList(data): class CADSInstance(service.Service): def __init__(self, host_name=None, realm_name=None, domain_name=None, dm_password=None): - service.Service.__init__(self, "pkids") + service.Service.__init__(self, "pkids", dm_password=dm_password, ldapi=False, autobind=service.DISABLED) self.serverid = "PKI-IPA" self.realm_name = realm_name - self.dm_password = dm_password self.sub_dict = None self.domain = domain_name self.fqdn = host_name diff --git a/ipaserver/install/dsinstance.py b/ipaserver/install/dsinstance.py index 25c449a6e865de01d789a739b31906cb70c6f212..9f3ae7252e45ab3289a85711a2b1202bbe04e137 100644 --- a/ipaserver/install/dsinstance.py +++ b/ipaserver/install/dsinstance.py @@ -160,7 +160,7 @@ info: IPA V2.0 class DsInstance(service.Service): def __init__(self, realm_name=None, domain_name=None, dm_password=None, fstore=None): - service.Service.__init__(self, "dirsrv", dm_password=dm_password) + service.Service.__init__(self, "dirsrv", dm_password=dm_password, ldapi=False, autobind=service.DISABLED) self.realm_name = realm_name self.sub_dict = None self.domain = domain_name diff --git a/ipaserver/install/krbinstance.py b/ipaserver/install/krbinstance.py index 2faf8e19693f891f28838df967399f0bfe2b51a4..8cc50fba4b0ba8d760cc892a624bd64ef09541a6 100644 --- a/ipaserver/install/krbinstance.py +++ b/ipaserver/install/krbinstance.py @@ -178,7 +178,7 @@ class KrbInstance(service.Service): self.start_creation("Configuring Kerberos KDC", 30) self.kpasswd = KpasswdInstance() - self.kpasswd.create_instance('KPASSWD', self.fqdn, self.admin_password, self.suffix) + self.kpasswd.create_instance('KPASSWD', self.fqdn, self.admin_password, self.suffix, realm=self.realm) def create_replica(self, realm_name, master_fqdn, host_name, diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py index 5c2699e3fa4c115c972528d4c2cc6aa170711837..198cb387011be1239eedbff410863232922a21e1 100644 --- a/ipaserver/install/service.py +++ b/ipaserver/install/service.py @@ -35,6 +35,11 @@ from ipapython.ipa_log_manager import * CACERT = "/etc/ipa/ca.crt" +# Autobind modes +AUTO = 1 +ENABLED = 2 +DISABLED = 3 + # The service name as stored in cn=masters,cn=ipa,cn=etc. In the tuple # the first value is the *nix service name, the second the start order. SERVICE_LIST = { @@ -55,13 +60,14 @@ def print_msg(message, output_fd=sys.stdout): class Service(object): - def __init__(self, service_name, sstore=None, dm_password=None, ldapi=False): + def __init__(self, service_name, sstore=None, dm_password=None, ldapi=True, autobind=AUTO): self.service_name = service_name self.service = ipaservices.service(service_name) self.steps = [] self.output_fd = sys.stdout self.dm_password = dm_password self.ldapi = ldapi + self.autobind = autobind self.fqdn = socket.gethostname() self.admin_conn = None @@ -77,12 +83,44 @@ class Service(object): self.dercert = None def ldap_connect(self): - if self.ldapi: - if not self.realm: - raise RuntimeError('realm must be set to use ldapi connection') - self.admin_conn = self.__get_conn(None, None, ldapi=True, realm=self.realm) - else: - self.admin_conn = self.__get_conn(self.fqdn, self.dm_password) + # If DM password is provided, we use it + # If autobind was requested, attempt autobind when root and ldapi + # If autobind was disabled or not succeeded, go with GSSAPI + # LDAPI can be used with either autobind or GSSAPI + # LDAPI requires realm to be set + try: + if self.ldapi: + if not self.realm: + raise errors.NotFound(reason="realm is missing for %s" % (self)) + conn = ipaldap.IPAdmin(ldapi=self.ldapi, realm=self.realm) + else: + conn = ipaldap.IPAdmin(self.fqdn, port=389) + if self.dm_password: + conn.do_simple_bind(bindpw=self.dm_password) + elif self.autobind in [AUTO, ENABLED]: + if os.getegid() == 0 and self.ldapi: + try: + # autobind + pw_name = pwd.getpwuid(os.geteuid()).pw_name + conn.do_external_bind(pw_name) + except errors.NotFound, e: + if self.autobind == AUTO: + # Fall back + conn.do_sasl_gssapi_bind() + else: + # autobind was required and failed, raise + # exception that it failed + raise e + else: + conn.do_sasl_gssapi_bind() + else: + conn.do_sasl_gssapi_bind() + except Exception, e: + root_logger.debug("Could not connect to the Directory Server on %s: %s" % (self.fqdn, str(e))) + raise e + + self.admin_conn = conn + def ldap_disconnect(self): self.admin_conn.unbind() @@ -93,7 +131,6 @@ class Service(object): pw_name = None fd = None path = ipautil.SHARE_DIR + ldif - hostname = installutils.get_fqdn() nologlist=[] if sub_dict is not None: @@ -107,15 +144,25 @@ class Service(object): if sub_dict.has_key('RANDOM_PASSWORD'): nologlist.append(sub_dict['RANDOM_PASSWORD']) + args = ["/usr/bin/ldapmodify", "-v", "-f", path] + + # As we always connect to the local host, + # use URI of admin connection + if not self.admin_conn: + self.ldap_connect() + args += ["-H", self.admin_conn._uri] + + auth_parms = [] if self.dm_password: [pw_fd, pw_name] = tempfile.mkstemp() os.write(pw_fd, self.dm_password) os.close(pw_fd) auth_parms = ["-x", "-D", "cn=Directory Manager", "-y", pw_name] else: - auth_parms = ["-Y", "GSSAPI"] + # always try GSSAPI auth when not using DM password or not being root + if os.getegid() != 0: + auth_parms = ["-Y", "GSSAPI"] - args = ["/usr/bin/ldapmodify", "-h", hostname, "-v", "-f", path] args += auth_parms try: @@ -181,8 +228,19 @@ class Service(object): This server cert should be in DER format. """ - if not self.admin_conn: - self.ldap_connect() + # add_cert_to_service() is relatively rare operation + # we actually call it twice during ipa-server-install, for different + # instances: ds and cs. Unfortunately, it may happen that admin + # connection was created well before add_cert_to_service() is called + # If there are other operations in between, it will become stale and + # since we are using SimpleLDAPObject, not ReconnectLDAPObject, the + # action will fail. Thus, explicitly disconnect and connect again. + # Using ReconnectLDAPObject instead of SimpleLDAPObject was considered + # but consequences for other parts of the framework are largely + # unknown. + if self.admin_conn: + self.ldap_disconnect() + self.ldap_connect() dn = "krbprincipalname=%s,cn=services,cn=accounts,%s" % (self.principal, self.suffix) mod = [(ldap.MOD_ADD, 'userCertificate', self.dercert)] @@ -268,33 +326,6 @@ class Service(object): self.steps = [] - def __get_conn(self, fqdn, dm_password, ldapi=False, realm=None): - # If we are passed a password we'll use it as the DM password - # otherwise we'll do a GSSAPI bind. - try: -# conn = ipaldap.IPAdmin(fqdn, port=636, cacert=CACERT) - if ldapi: - conn = ipaldap.IPAdmin(ldapi=ldapi, realm=realm) - else: - conn = ipaldap.IPAdmin(fqdn, port=389) - if dm_password: - conn.do_simple_bind(bindpw=dm_password) - elif os.getegid() == 0 and self.ldapi: - try: - # autobind - pw_name = pwd.getpwuid(os.geteuid()).pw_name - conn.do_external_bind(pw_name) - except errors.NotFound: - # Fall back - conn.do_sasl_gssapi_bind() - else: - conn.do_sasl_gssapi_bind() - except Exception, e: - root_logger.debug("Could not connect to the Directory Server on %s: %s" % (fqdn, str(e))) - raise e - - return conn - def ldap_enable(self, name, fqdn, dm_password, ldap_suffix): self.disable() if not self.admin_conn: @@ -318,11 +349,14 @@ class Service(object): raise e class SimpleServiceInstance(Service): - def create_instance(self, gensvc_name=None, fqdn=None, dm_password=None, ldap_suffix=None): + def create_instance(self, gensvc_name=None, fqdn=None, dm_password=None, ldap_suffix=None, realm=None): self.gensvc_name = gensvc_name self.fqdn = fqdn self.dm_password = dm_password self.suffix = ldap_suffix + self.realm = realm + if not realm: + self.ldapi = False self.step("starting %s " % self.service_name, self.__start) self.step("configuring %s to start on boot" % self.service_name, self.__enable) -- 1.7.11.2 From sbingram at gmail.com Tue Jul 31 15:45:47 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Tue, 31 Jul 2012 08:45:47 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <50170A58.9030901@redhat.com> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> Message-ID: On Mon, Jul 30, 2012 at 3:27 PM, John Dennis wrote: > Stephen finally got his server patched with my timing instrumentation. He > sent me his log file and I ran my script to parse it. The results are quite > interesting. > > Basically I timed two things > > cmd_duration is the elapsed time to execute the IPA command. > > json_duration is the elapsed time to execute the JSON RPC containing the IPA > command. The difference (delta) between the two is the time we spend doing > "session bookkeeping". In all but the first RPC we were executing from the > session cache. Results are below. > > What is interesting is every command that executes from the session cache > seems to have about a 1.5 second constant overhead. The other interesting > bits are that most commands in the server execute quickly (from under a > tenth of second to .3 second) and on average the server takes 1.5 seconds to > respond to RPC (of which most of it appears to be in session code). > > What is taking so long with session bookkeeping? I don't know yet. I would > need more timing instrumentation. I will say when I looked at the > python-krb5 code (which we use to populate the ccache from the session and > read back to store in the session) seemed to be remarkably inefficient. We > also elected to use file based ccache rather than in-memory ccache (that > means there is a bit of file-IO occurring). Just out of curiosity, were any, or, all of these, changes from the way things were handled in 2.1? > Has anyone used a Python profiler? Would that be better than adding > instrumentation? > > BTW, the actual commands were stripped to protect data anonymity. > > cmd_duration: 0.107673 json_duration: 3.176758 delta: 3.069085 > cmd_duration: 0.020068 json_duration: 1.543135 delta: 1.523067 > cmd_duration: 0.387210 json_duration: 1.802954 delta: 1.415744 > cmd_duration: 0.024086 json_duration: 1.405276 delta: 1.381190 > cmd_duration: 0.342808 json_duration: 1.950365 delta: 1.607557 > cmd_duration: 0.019286 json_duration: 1.398786 delta: 1.379500 > cmd_duration: 0.200895 json_duration: 1.754358 delta: 1.553463 > cmd_duration: 0.020293 json_duration: 1.410701 delta: 1.390408 > cmd_duration: 0.383433 json_duration: 1.819478 delta: 1.436045 > cmd_duration: 0.020350 json_duration: 1.406038 delta: 1.385688 > cmd_duration: 0.346864 json_duration: 1.771281 delta: 1.424417 > cmd_duration: 0.015998 json_duration: 1.707743 delta: 1.691745 > cmd_duration: 0.314527 json_duration: 1.730894 delta: 1.416367 > cmd_duration: 0.025323 json_duration: 1.582828 delta: 1.557505 Steve From sbingram at gmail.com Tue Jul 31 15:49:56 2012 From: sbingram at gmail.com (Stephen Ingram) Date: Tue, 31 Jul 2012 08:49:56 -0700 Subject: [Freeipa-devel] slow response In-Reply-To: <501799C6.3090002@redhat.com> References: <50045EA9.9070709@redhat.com> <50046A1A.4010100@redhat.com> <5005D295.2020708@redhat.com> <5006BE06.7020109@redhat.com> <50070E55.90307@redhat.com> <5007174E.3080404@redhat.com> <500721F5.4040004@redhat.com> <500729EA.4040507@redhat.com> <500D7DE7.8080405@redhat.com> <50170A58.9030901@redhat.com> <501799C6.3090002@redhat.com> Message-ID: On Tue, Jul 31, 2012 at 1:39 AM, Petr Spacek wrote: > On 07/31/2012 12:27 AM, John Dennis wrote: >> >> >> What is taking so long with session bookkeeping? I don't know yet. I would >> need more timing instrumentation. I will say when I looked at the >> python-krb5 >> code (which we use to populate the ccache from the session and read back >> to >> store in the session) seemed to be remarkably inefficient. We also elected >> to >> use file based ccache rather than in-memory ccache (that means there is a >> bit >> of file-IO occurring). > > > A note regarding python-krbV: > I used python-krbV extensively in my thesis for KDC stress test. Python-krbV > can obtain several thousands of TGTs per second (even with ccache in a > file). AFAIK VFS calls are not done synchronously. But others parts of > python-krbV were left uncovered, so it can contain some surprises. > > === Wild speculation follows === > 1.5 second is incredibly long time, it sounds like some kind of timeout. > Krb5 libs have usual timeout = 1 second per request. > > Are all KDCs in /etc/krb5.conf alive and reachable? In this case, as I'm referring to the extreme slowness of the Web UI, the KDC is on the same system (the ipa server) that is making the request, correct? > Is SSSD running on problematic server? Yes. Again, I'm guessing the problematic server is the IPA server itself. > Is proper KDC selected by SSSD KDC auto-locator plugin? > (See /var/lib/sss/pubconf/) Yes, I checked that file and it is the IP address of the IPA server on the same server. Perhaps should this be 127.0.0.1 instead? I also have checked the resolv.conf file, and indeed the IP points to the IPA server itself (same machine) as expected. Both forward and reverse DNS work. I'm not really sure what else could be setup incorrectly to cause any KDC slowness. Due to the extreme UI slowness issue, I have not created any replicas so this system is it. I'm not so sure I would be able to see the 1.5 s delay if it weren't compounded by the overall slowness of the Web UI, however, the KDC seems to perform well for other systems in the realm. I'm certainly not taxing it with a huge load, but tickets seem to be issued without delay. Steve From mkosek at redhat.com Tue Jul 31 15:51:44 2012 From: mkosek at redhat.com (Martin Kosek) Date: Tue, 31 Jul 2012 17:51:44 +0200 Subject: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket In-Reply-To: <20120731142029.GI19710@redhat.com> References: <20120726131633.GA3324@redhat.com> <20120727040732.GB13180@redhat.com> <50131248.20005@redhat.com> <20120730113450.GB19710@redhat.com> <501686A3.2000405@redhat.com> <20120731120005.GF19710@redhat.com> <5017E063.2050003@redhat.com> <20120731142029.GI19710@redhat.com> Message-ID: <5017FF10.1080206@redhat.com> On 07/31/2012 04:20 PM, Alexander Bokovoy wrote: > On Tue, 31 Jul 2012, Martin Kosek wrote: >> On 07/31/2012 02:00 PM, Alexander Bokovoy wrote: >>> On Mon, 30 Jul 2012, Martin Kosek wrote: >>>> On 07/30/2012 01:34 PM, Alexander Bokovoy wrote: >>>>> On Fri, 27 Jul 2012, Rob Crittenden wrote: >>>>>> Alexander Bokovoy wrote: >>>>>>> On Thu, 26 Jul 2012, Alexander Bokovoy wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> When setting up AD trusts support, ipa-adtrust-install utility >>>>>>>> needs to be run as: >>>>>>>> - root, for performing Samba configuration and using LDAPI/autobind >>>>>>>> - kinit-ed IPA admin user, to ensure proper ACIs are granted to >>>>>>>> fetch keytab >>>>>>>> >>>>>>>> As result, we can get rid of Directory Manager credentials in >>>>>>>> ipa-adtrust-install >>>>>>>> >>>>>>>> https://fedorahosted.org/freeipa/ticket/2815 >>>>>>>> >>>>>>>> This ticket also simplifies a bit the way we handle admin connection in >>>>>>>> Service class and particulary in Service._ldap_mod() by defaulting to >>>>>>>> LDAPI/autobind in case of running as root and to GSSAPI otherwise. >>>>>>>> Except few cases in remote replica management (not applicable in >>>>>>>> _ldap_mod() case) we always run installation tools as root and can >>>>>>>> benefit from using autobind feature. Unfortunately, it is not yet >>>>>>>> possible to get away from using DM credentials for all cases as the same >>>>>>>> class is used to perform initial directory server instance >>>>>>>> configuration. >>>>>>>> >>>>>>>> One side effect is explicit disconnect and reconnect in >>>>>>>> Service.add_cert_to_service() due to way how SimpleLDAPObject class >>>>>>>> handles stale connections (no handling at all). I've put some comments >>>>>>>> in place so that others would not try to err out optimizing it in >>>>>>>> future. >>>>>>>> >>>>>>>> Finally, with next patch series which will introduce syncing ipaNTHash >>>>>>>> attribute with RC4 key in existing kerberos credentials, we can remove >>>>>>>> requirements to change passwords or re-kinit for majority of trust >>>>>>>> cases. This should then conclude our trusts content for beta2 release. >>>>>>> >>>>>>> Patch updated, fixed small typo (auth_parms was initialized as >>>>>>> auth_params which led to non-existing auth_parms in ipa-adtrust-install >>>>>>> case). >>>>>> >>>>>> Nack, a couple of minor issues: >>>>>> >>>>>> The exception handling is rather unusual in >>>>>> ensure_kerberos_admin_rights(api). I'm not sure if this is any more >>>>>> efficient >>>>>> than a series of excepts... >>>>> I've rewrote this code and put it directly in the main. >>>>> >>>>>> You don't need to pass in api, it's a global. >>>>> Fixed. >>>>> >>>>> >>>>>> It may be safe to see if the user is in the group the way you are doing >>>>>> it, I >>>>>> wonder if it would be clearer to cast those into DN objects. >>>>> Not sure if checking DNs would be sustaining in long run. Ideally we >>>>> should check ACI here, not just hardcoded group name. I'd like to keep >>>>> it explicit with memberof for now because it shows what exactly we want >>>>> to check. >>>>> >>>>>> In the Service class what is the point of ldapi if it is going to be ignored >>>>>> in the case we know the realm? What if I really, really just want to use a >>>>>> password? >>>>> LDAPI bind in IPAAdmin.__local_init() requires that there is realm known. >>>>> No realm -- no LDAPI use because we otherwise cannot construct the >>>>> socket name. For 'just want to use a password' case you can simply set >>>>> self.dm_password. >>>>> >>>>> However, I've changed the code in Service.ldap_connect() to do >>>>> following: >>>>> >>>>> 1. if DM password is provided, we'll try to use it >>>>> 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and >>>>> realm >>>>> 3. Otherwise (ldapi was False or realm not provided), we'll try to >>>>> connect to fqdn:389 with GSSAPI >>>>> >>>>> I think this covers all cases. >>>>> >>>>>> And later where it forces ldapi, it seems better to either commit all the >>>>>> way >>>>>> and drop the ldapi argument or convert it to a better name (like autobind). >>>>> ldapi requires realm but can be used with either GSSAPI or autobind. >>>>> Calling it autobind isn't really correct as autobind only available on >>>>> ldapi under root. >>>>> >>>> >>>> Works fine, I also have just few minor-ish issues: >>>> >>>> 1) Uncatched exception >>>> >>>> We may want to also catch for DatabaseException in this section: >>>> >>>> + api.Backend.ldap2.connect(ccache.name) >>>> + except errors.ACIError, e: >>>> + sys.exit("Outdated Kerberos credentials. Use kdestroy and kinit to >>>> update your ticket") >>>> >>>> Otherwise ipa-adtrust-install throws unexpected exception when IPA is down: >>>> >>>> # ipactl stop >>>> # ipa-adtrust-install >>>> ... >>>> NetBIOS domain name [IDM]: >>>> >>>> Unexpected error - see /var/log/ipaserver-install.log for details: >>>> DatabaseError: Can't contact LDAP server: >>>> >>>> >>>> 2) Wrong indentation: >>>> ... >>>> + except errors.RequirementError, e: >>>> + sys.exit("Must have administrative privileges to setup AD >>>> trusts on >>>> server") >>>> + except Exception, e: >>>> + sys.exit("Unrecognized error during check of admin rights: %s" % >>>> (str(e))) >>> Updated patch, includes fixes for issues mentioned above and also >>> implements autobind suggestions by Simo. >>> >>> We have SimpleServiceInstance() that doesn't have realm known by default >>> and this means certain protection is needed for missing realm. Also DS >>> and CA DS instances cannot use LDAPI and autobind when being setup, only >>> DM password. The patch handles these cases too. >>> >> >> Hm, ipa-dns-install is now broken: >> >> # ipa-dns-install >> >> The log file for this installation can be found in >> /var/log/ipaserver-install.log >> ============================================================================== >> This program will setup DNS for the FreeIPA Server. >> >> This includes: >> * Configure DNS (bind) >> >> To accept the default shown in brackets, press the Enter key. >> >> Existing BIND configuration detected, overwrite? [no]: y >> Directory Manager password: >> >> Unexpected error - see /var/log/ipaserver-install.log for details: >> ValueError: non-generic 'NotFound' needs format=None; got format='realm is >> missing for ' >> >> This means there are 2 issues: >> >> 1) NotFound error is not risen correctly. Instead of >> + raise errors.NotFound("realm is missing for %s" % (self)) >> it should rather be: >> + raise errors.NotFound(reason="realm is missing for %s" % >> (self)) > Fixed. > > >> 2) LDAP connection in ipa-dns-install (bindinstance.py) is not right. > Fixed. As it requires DM there it was easier to disable ldapi for the > instance. We may want to review all use of DM password and use instead a > common code like in ipa-adtrust-install that checks Kerberos credentials > and utilizes GSSAPI/LDAPI/autobind instead. > > After all, ipa-dns-install also needs root access to set up > /etc/named.conf so there is no need to have DM password. I opened a ticket for that: https://fedorahosted.org/freeipa/ticket/2957 ipa-dns-install now works fine. I did not find any other issue, so ACK. Pushed to master. Martin From edewata at redhat.com Tue Jul 31 19:33:43 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 31 Jul 2012 14:33:43 -0500 Subject: [Freeipa-devel] [PATCH] 175, 176 Fixed: Unable to select option in combobox in IE and Chrome In-Reply-To: <5012B9EB.8010304@redhat.com> References: <5012B9EB.8010304@redhat.com> Message-ID: <50183317.90009@redhat.com> On 7/27/2012 10:55 AM, Petr Vobornik wrote: > [PATCH] 175 Fixed: Unable to select option in combobox in IE and Chrome > > There's probably a bug regarding z-index stacking in Chrome and IE. It > appears when combobox is used in dialog. Combobox's select area had > z-index=1010. When first jquery dialog is open it has z-index=1000. > Further dialogs have higher z-index. When dialog's z-index exceeds 1010 > option in select control can't be selected. IMO it is a browser bug > because select control lies in dialog content's stacking context so it > should be functional even with z-index=1. > > This patch raises select area's z-index to 9000000 which should prevent > the issue for some time. Also it make's combobox's z-index configurable > so we can solve combobox stacking (ie in service-add dialog). > > Second part of: > https://fedorahosted.org/freeipa/ticket/2834 > > [PATCH] 176 Fixed: combobox stacking in service adder dialog > > First select's content is displayed under second comboxes content when > select is opened when second combobox is opened > > Bonus for: > https://fedorahosted.org/freeipa/ticket/2834 ACK. -- Endi S. Dewata From edewata at redhat.com Tue Jul 31 19:34:16 2012 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 31 Jul 2012 14:34:16 -0500 Subject: [Freeipa-devel] [PATCHES] 177-184 Rebase of jquery and jquery-ui libs In-Reply-To: <5012C8DE.9090105@redhat.com> References: <5012C8DE.9090105@redhat.com> Message-ID: <50183338.7060809@redhat.com> On 7/27/2012 11:59 AM, Petr Vobornik wrote: > This effort was driven by ticket > https://fedorahosted.org/freeipa/ticket/2817 which needed update of > jqury-ui library. When I was at it I also updated core jquery lib. All > patches are associated with ticket #2817 which might be bad. Should I > split it to more tickets? > > This effort makes Web UI look consistent in Firefox, Chrome, Opera and > IE. I didn't try Safari. Also it fixes some visual issues (mostly in IE > and Opera) and makes buttons prettier. > > Patches 177,178 are rebases of jquery and jquery-ui libraries. Upgrade > introduced two new functional bugs and 2 test bugs which are fixed by > patches 179, 182 and 183. Update of jquery-ui.css made some parts of > ipa.css redundant so the redundancy is removed in patch 180. Patch 180 > also removes some styles which make buttons in association dialog look > bad. Patch 181 utilizes jquery.button and so makes buttons in > association dialogs consistent with the rest of the UI. Patch 184 is > just tiny refactoring. > > > Patch notes: > [PATCH] 177 Update to jquery.1.7.2.min > > jquery library wasn't updated for a long time. > > > [PATCH] 178 Update to jquery-ui-1.8.21.custom > > jquery-ui was regenerated to up to date version. > > Border radius and IPA custom colors were added to theme so we don't have > to override them in ipa.css. > > [PATCH] 179 Fix for incorrect event handler definition > > Clicks events should be better defined by jquery calls (usually > addEventListener) not as elements attributes. Definition as element > attribute causes problems after upgrade to jquery 1.7.2. Two occurances > were removed. > > [PATCH] 180 Removal of unnecessary overrides of jquery-ui styles > > ipa.css had to be updated to work with updated jquery-ui. This patch > removes several duplicate styles. > > Following issues were fixed: > * dialogs titles in IE and Opera were black instead of green > * no black line in first navigation level in IE and Opera > * all browsers (FF, IE, Chrome, Opera) have the same style for buttons > and headers > * dialogs has borders again (should we remove its shadow?) > > Known issues: > * selected tab-1 in Chrome and Opera doesn't overlaps background line > as in IE and FF. Not sure how to fix without breaking (there are border > overlaps) the latter ones. I think it looks good enough. > * some buttons are missing padding. Will be fixed in next patch. > > [PATCH] 181 Unified buttons > > Buttons in association dialog and action list have different style and > behavior than buttons in dialogs. This patch unifies it by using > jquery.button widget. > > [PATCH] 182 Web UI tests fix > > ACI tests were crashing because of misconfigured facet. > Entity link test were crashing because of incorrect jquery selector. > > [PATCH] 183 Fixed incorrect use of jQuery.attr for setting disabled > attribute > > Occurance: select_widget > > Update to latest version of jQuery uncovered this issue. > > [PATCH] 184 Replace use of attr with prop for booleans > > Recommened way of setting boolean HTML attributes is by $.prop(boolean) > method not $.attr(boolean) because it sets DOM object property not an > attribute. Latter works because of jquery's backward compatibility. This > patch makes things clearer. > > Some info about prop and attr: http://stackoverflow.com/a/5876747 The build fails, it looks like the Makefile in install/ui/images needs to be updated: make[5]: Entering directory `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install/ui/images' make[5]: *** No rule to make target `ui-bg_glass_40_111111_1x400.png', needed by `all-am'. Stop. make[5]: Leaving directory `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install/ui/images' make[4]: *** [all-recursive] Error 1 make[4]: Leaving directory `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install/ui' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff/install' make[1]: *** [all] Error 1 make[1]: Leaving directory `/home/edewata/IPA/freeipa/rpmbuild/BUILD/freeipa-3.0.0GIT97e51ff' error: Bad exit status from /var/tmp/rpm-tmp.thVPHP (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.thVPHP (%build) make: *** [rpms] Error 1 -- Endi S. Dewata From mgregg at redhat.com Tue Jul 31 21:50:06 2012 From: mgregg at redhat.com (Michael Gregg) Date: Tue, 31 Jul 2012 14:50:06 -0700 Subject: [Freeipa-devel] Strange issue I keep hitting with invalid tickets Message-ID: <5018530E.70105@redhat.com> I am not sure why, but when I let my ipa machines sit around for a while(overnight-24hours), and then kinit. When I try to run IPA commands I get output like this: [root at zippyvm12 ~]# ipa host-find ipa: ERROR: Local error: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) This issue seems to be addressed here: https://access.redhat.com/knowledge/solutions/133433 It's strange, because when I kinit again, I seem to have a valid credentials, like here: [root at zippyvm12 ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: admin at TESTRELM.COM Valid starting Expires Service principal 07/31/12 17:31:16 08/01/12 17:31:14 krbtgt/TESTRELM.COM at TESTRELM.COM 07/31/12 17:32:39 08/01/12 17:31:14 HTTP/zippyvm12.testrelm.com at TESTRELM.COM The work around for me seems to be deleting /tmp/krb5* Then, I kinit again, and it all starts to work again. My question is, why is this happening? Any ideas? Michael-